🗊Презентация Assessing and comparing classification algorithms

Категория: Математика
Нажмите для полного просмотра!
Assessing and comparing classification algorithms, слайд №1Assessing and comparing classification algorithms, слайд №2Assessing and comparing classification algorithms, слайд №3Assessing and comparing classification algorithms, слайд №4Assessing and comparing classification algorithms, слайд №5Assessing and comparing classification algorithms, слайд №6Assessing and comparing classification algorithms, слайд №7Assessing and comparing classification algorithms, слайд №8Assessing and comparing classification algorithms, слайд №9Assessing and comparing classification algorithms, слайд №10Assessing and comparing classification algorithms, слайд №11Assessing and comparing classification algorithms, слайд №12Assessing and comparing classification algorithms, слайд №13Assessing and comparing classification algorithms, слайд №14Assessing and comparing classification algorithms, слайд №15Assessing and comparing classification algorithms, слайд №16Assessing and comparing classification algorithms, слайд №17Assessing and comparing classification algorithms, слайд №18Assessing and comparing classification algorithms, слайд №19Assessing and comparing classification algorithms, слайд №20Assessing and comparing classification algorithms, слайд №21Assessing and comparing classification algorithms, слайд №22Assessing and comparing classification algorithms, слайд №23Assessing and comparing classification algorithms, слайд №24
Скачать
Вы можете ознакомиться и скачать презентацию на тему Assessing and comparing classification algorithms. Доклад-сообщение содержит 24 слайдов. Презентации для любого класса можно скачать бесплатно. Если материал и наш сайт презентаций Mypresentation Вам понравились – поделитесь им с друзьями с помощью социальных кнопок и добавьте в закладки в своем браузере.

Слайды и текст этой презентации

Слайд 1


INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, 2004 alpaydin@boun.edu.tr http://www.cmpe.boun.edu.tr/~ethem/i2ml
Описание слайда:
INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, 2004 alpaydin@boun.edu.tr http://www.cmpe.boun.edu.tr/~ethem/i2ml

Слайд 2


Assessing and comparing classification algorithms, слайд №2
Описание слайда:

Слайд 3


Introduction Questions: Assessment of the expected error of a learning algorithm: Is the error rate of 1-NN less than 2%? Comparing the expected errors of two algorithms: Is k-NN more accurate than MLP ? Training/validation/test sets Resampling methods: K-fold cross-validation
Описание слайда:
Introduction Questions: Assessment of the expected error of a learning algorithm: Is the error rate of 1-NN less than 2%? Comparing the expected errors of two algorithms: Is k-NN more accurate than MLP ? Training/validation/test sets Resampling methods: K-fold cross-validation

Слайд 4


Algorithm Preference Criteria (Application-dependent): Misclassification error, or risk (loss functions) Training time/space complexity Testing time/space complexity Interpretability Easy programmability Cost-sensitive learning
Описание слайда:
Algorithm Preference Criteria (Application-dependent): Misclassification error, or risk (loss functions) Training time/space complexity Testing time/space complexity Interpretability Easy programmability Cost-sensitive learning

Слайд 5


Resampling and K-Fold Cross-Validation The need for multiple training/validation sets {Xi,Vi}i: Training/validation sets of fold i K-fold cross-validation: Divide X into k, Xi,i=1,...,K Ti share K-2 parts
Описание слайда:
Resampling and K-Fold Cross-Validation The need for multiple training/validation sets {Xi,Vi}i: Training/validation sets of fold i K-fold cross-validation: Divide X into k, Xi,i=1,...,K Ti share K-2 parts

Слайд 6


5×2 Cross-Validation 5 times 2 fold cross-validation (Dietterich, 1998)
Описание слайда:
5×2 Cross-Validation 5 times 2 fold cross-validation (Dietterich, 1998)

Слайд 7


Bootstrapping Draw instances from a dataset with replacement Prob that we do not pick an instance after N draws that is, only 36.8% is new!
Описание слайда:
Bootstrapping Draw instances from a dataset with replacement Prob that we do not pick an instance after N draws that is, only 36.8% is new!

Слайд 8


Measuring Error Error rate = # of errors / # of instances = (FN+FP) / N Recall = # of found positives / # of positives = TP / (TP+FN) = sensitivity = hit rate Precision = # of found positives / # of found = TP / (TP+FP) Specificity = TN / (TN+FP) False alarm rate = FP / (FP+TN) = 1 - Specificity
Описание слайда:
Measuring Error Error rate = # of errors / # of instances = (FN+FP) / N Recall = # of found positives / # of positives = TP / (TP+FN) = sensitivity = hit rate Precision = # of found positives / # of found = TP / (TP+FP) Specificity = TN / (TN+FP) False alarm rate = FP / (FP+TN) = 1 - Specificity

Слайд 9


ROC Curve
Описание слайда:
ROC Curve

Слайд 10


Interval Estimation X = { xt }t where xt ~ N ( μ, σ2) m ~ N ( μ, σ2/N)
Описание слайда:
Interval Estimation X = { xt }t where xt ~ N ( μ, σ2) m ~ N ( μ, σ2/N)

Слайд 11


Assessing and comparing classification algorithms, слайд №11
Описание слайда:

Слайд 12


Hypothesis Testing Reject a null hypothesis if not supported by the sample with enough confidence X = { xt }t where xt ~ N ( μ, σ2) H0: μ = μ0 vs. H1: μ ≠ μ0 Accept H0 with level of significance α if μ0 is in the 100(1- α) confidence interval Two-sided test
Описание слайда:
Hypothesis Testing Reject a null hypothesis if not supported by the sample with enough confidence X = { xt }t where xt ~ N ( μ, σ2) H0: μ = μ0 vs. H1: μ ≠ μ0 Accept H0 with level of significance α if μ0 is in the 100(1- α) confidence interval Two-sided test

Слайд 13


One-sided test: H0: μ ≤ μ0 vs. H1: μ > μ0 Accept if Variance unknown: Use t, instead of z Accept H0: μ = μ0 if
Описание слайда:
One-sided test: H0: μ ≤ μ0 vs. H1: μ > μ0 Accept if Variance unknown: Use t, instead of z Accept H0: μ = μ0 if

Слайд 14


Assessing Error: H0: p ≤ p0 vs. H1: p > p0 Single training/validation set: Binomial Test If error prob is p0, prob that there are e errors or less in N validation trials is
Описание слайда:
Assessing Error: H0: p ≤ p0 vs. H1: p > p0 Single training/validation set: Binomial Test If error prob is p0, prob that there are e errors or less in N validation trials is

Слайд 15


Normal Approximation to the Binomial Number of errors X is approx N with mean Np0 and var Np0(1-p0)
Описание слайда:
Normal Approximation to the Binomial Number of errors X is approx N with mean Np0 and var Np0(1-p0)

Слайд 16


Paired t Test Multiple training/validation sets xti = 1 if instance t misclassified on fold i Error rate of fold i: With m and s2 average and var of pi we accept p0 or less error if is less than tα,K-1
Описание слайда:
Paired t Test Multiple training/validation sets xti = 1 if instance t misclassified on fold i Error rate of fold i: With m and s2 average and var of pi we accept p0 or less error if is less than tα,K-1

Слайд 17


Comparing Classifiers: H0: μ0 = μ1 vs. H1: μ0 ≠ μ1 Single training/validation set: McNemar’s Test Under H0, we expect e01= e10=(e01+ e10)/2
Описание слайда:
Comparing Classifiers: H0: μ0 = μ1 vs. H1: μ0 ≠ μ1 Single training/validation set: McNemar’s Test Under H0, we expect e01= e10=(e01+ e10)/2

Слайд 18


K-Fold CV Paired t Test Use K-fold cv to get K training/validation folds pi1, pi2: Errors of classifiers 1 and 2 on fold i pi = pi1 – pi2 : Paired difference on fold i The null hypothesis is whether pi has mean 0
Описание слайда:
K-Fold CV Paired t Test Use K-fold cv to get K training/validation folds pi1, pi2: Errors of classifiers 1 and 2 on fold i pi = pi1 – pi2 : Paired difference on fold i The null hypothesis is whether pi has mean 0

Слайд 19


5×2 cv Paired t Test Use 5×2 cv to get 2 folds of 5 tra/val replications (Dietterich, 1998) pi(j) : difference btw errors of 1 and 2 on fold j=1, 2 of replication i=1,...,5
Описание слайда:
5×2 cv Paired t Test Use 5×2 cv to get 2 folds of 5 tra/val replications (Dietterich, 1998) pi(j) : difference btw errors of 1 and 2 on fold j=1, 2 of replication i=1,...,5

Слайд 20


5×2 cv Paired F Test
Описание слайда:
5×2 cv Paired F Test

Слайд 21


Comparing L>2 Algorithms: Analysis of Variance (Anova) Errors of L algorithms on K folds We construct two estimators to σ2 . One is valid if H0 is true, the other is always valid. We reject H0 if the two estimators disagree.
Описание слайда:
Comparing L>2 Algorithms: Analysis of Variance (Anova) Errors of L algorithms on K folds We construct two estimators to σ2 . One is valid if H0 is true, the other is always valid. We reject H0 if the two estimators disagree.

Слайд 22


Assessing and comparing classification algorithms, слайд №22
Описание слайда:

Слайд 23


Assessing and comparing classification algorithms, слайд №23
Описание слайда:

Слайд 24


Other Tests Range test (Newman-Keuls): Nonparametric tests (Sign test, Kruskal-Wallis) Contrasts: Check if 1 and 2 differ from 3,4, and 5 Multiple comparisons require Bonferroni correction If there are m tests, to have an overall significance of α, each test should have a significance of α/m. Regression: CLT states that the sum of iid variables from any distribution is approximately normal and the preceding methods can be used. Other loss functions ?
Описание слайда:
Other Tests Range test (Newman-Keuls): Nonparametric tests (Sign test, Kruskal-Wallis) Contrasts: Check if 1 and 2 differ from 3,4, and 5 Multiple comparisons require Bonferroni correction If there are m tests, to have an overall significance of α, each test should have a significance of α/m. Regression: CLT states that the sum of iid variables from any distribution is approximately normal and the preceding methods can be used. Other loss functions ?

Скачать

Похожие презентации
Mypresentation.ru
Загрузить презентацию