🗊Презентация Lecture 7

Категория: Образование
Нажмите для полного просмотра!
Lecture 7, слайд №1Lecture 7, слайд №2Lecture 7, слайд №3Lecture 7, слайд №4Lecture 7, слайд №5Lecture 7, слайд №6Lecture 7, слайд №7Lecture 7, слайд №8Lecture 7, слайд №9Lecture 7, слайд №10Lecture 7, слайд №11Lecture 7, слайд №12Lecture 7, слайд №13Lecture 7, слайд №14Lecture 7, слайд №15Lecture 7, слайд №16Lecture 7, слайд №17Lecture 7, слайд №18Lecture 7, слайд №19Lecture 7, слайд №20Lecture 7, слайд №21Lecture 7, слайд №22Lecture 7, слайд №23Lecture 7, слайд №24Lecture 7, слайд №25Lecture 7, слайд №26Lecture 7, слайд №27Lecture 7, слайд №28Lecture 7, слайд №29Lecture 7, слайд №30Lecture 7, слайд №31Lecture 7, слайд №32Lecture 7, слайд №33Lecture 7, слайд №34Lecture 7, слайд №35Lecture 7, слайд №36Lecture 7, слайд №37Lecture 7, слайд №38Lecture 7, слайд №39Lecture 7, слайд №40Lecture 7, слайд №41Lecture 7, слайд №42Lecture 7, слайд №43Lecture 7, слайд №44Lecture 7, слайд №45Lecture 7, слайд №46Lecture 7, слайд №47Lecture 7, слайд №48Lecture 7, слайд №49Lecture 7, слайд №50Lecture 7, слайд №51Lecture 7, слайд №52Lecture 7, слайд №53Lecture 7, слайд №54Lecture 7, слайд №55Lecture 7, слайд №56Lecture 7, слайд №57Lecture 7, слайд №58
Скачать

Содержание ▲

Вы можете ознакомиться и скачать презентацию на тему Lecture 7. Доклад-сообщение содержит 58 слайдов. Презентации для любого класса можно скачать бесплатно. Если материал и наш сайт презентаций Mypresentation Вам понравились – поделитесь им с друзьями с помощью социальных кнопок и добавьте в закладки в своем браузере.

Слайды и текст этой презентации

Слайд 1


Intro to Machine Learning Lecture 7 Adil Khan a.khan@innopolis.ru
Описание слайда:
Intro to Machine Learning Lecture 7 Adil Khan a.khan@innopolis.ru

Слайд 2


Recap Decision Trees (in class) for classification Using categorical predictors Using classification error as our metric Decision Trees (in lab) For regression Using continuous predictors Using entropy, gini, and information gain
Описание слайда:
Recap Decision Trees (in class) for classification Using categorical predictors Using classification error as our metric Decision Trees (in lab) For regression Using continuous predictors Using entropy, gini, and information gain

Слайд 3


Impurity Measures: Covered in Lab last Week
Описание слайда:
Impurity Measures: Covered in Lab last Week

Слайд 4


Practice Yourself
Описание слайда:
Practice Yourself

Слайд 5


Today’s Objectives Overfitting in Decision Trees (Tree Pruning) Ensemble Learning ( combine the power of multiple models in a single model while overcoming their weaknesses) Bagging (overcoming variance) Boosting (overcoming bias)
Описание слайда:
Today’s Objectives Overfitting in Decision Trees (Tree Pruning) Ensemble Learning ( combine the power of multiple models in a single model while overcoming their weaknesses) Bagging (overcoming variance) Boosting (overcoming bias)

Слайд 6


Overfitting in Decision Trees
Описание слайда:
Overfitting in Decision Trees

Слайд 7


Decision Boundaries at Different Depths
Описание слайда:
Decision Boundaries at Different Depths

Слайд 8


Generally Speaking
Описание слайда:
Generally Speaking

Слайд 9


Decision Tree Over fitting on Real Data
Описание слайда:
Decision Tree Over fitting on Real Data

Слайд 10


Simple is Better When two trees have the same classification error on validation set, choose the one that is simpler
Описание слайда:
Simple is Better When two trees have the same classification error on validation set, choose the one that is simpler

Слайд 11


Modified Tree Learning Problem
Описание слайда:
Modified Tree Learning Problem

Слайд 12


Finding Simple Trees Early Stopping: Stop learning before the tree becomes too complex Pruning: Simplify tree after learning algorithm terminates
Описание слайда:
Finding Simple Trees Early Stopping: Stop learning before the tree becomes too complex Pruning: Simplify tree after learning algorithm terminates

Слайд 13


Criteria 1 for Early Stopping Limit the depth: stop splitting after max_depth is reached
Описание слайда:
Criteria 1 for Early Stopping Limit the depth: stop splitting after max_depth is reached

Слайд 14


Criteria 2 for Early Stopping Use a threshold for decrease in error with a split Stop if the error does not decrease more than Mostly works, but may cause problems in some cases
Описание слайда:
Criteria 2 for Early Stopping Use a threshold for decrease in error with a split Stop if the error does not decrease more than Mostly works, but may cause problems in some cases

Слайд 15


Criteria 3 for Early Stopping
Описание слайда:
Criteria 3 for Early Stopping

Слайд 16


Early Stopping: Summary
Описание слайда:
Early Stopping: Summary

Слайд 17


Pruning
Описание слайда:
Pruning

Слайд 18


Which Tree is Simpler?
Описание слайда:
Which Tree is Simpler?

Слайд 19


Which Tree is Simpler
Описание слайда:
Which Tree is Simpler

Слайд 20


Thus, Our Measure of Complexity
Описание слайда:
Thus, Our Measure of Complexity

Слайд 21


New Optimization Goal Total Cost = Measure of Fit + Measure of Complexity Measure of Fit = Classification Error (large means bad fit to the data) Measure of complexity = Number of Leaves (large means likely to overfit)
Описание слайда:
New Optimization Goal Total Cost = Measure of Fit + Measure of Complexity Measure of Fit = Classification Error (large means bad fit to the data) Measure of complexity = Number of Leaves (large means likely to overfit)

Слайд 22


Tree Pruning Algorithm Let T be the final tree Start at the bottom of T and traverse up, apply prune_split at each decision node M
Описание слайда:
Tree Pruning Algorithm Let T be the final tree Start at the bottom of T and traverse up, apply prune_split at each decision node M

Слайд 23


prune_split Prune_split (, ) Compute total cost Let be the tree after pruning at Compute If < , prune to
Описание слайда:
prune_split Prune_split (, ) Compute total cost Let be the tree after pruning at Compute If < , prune to

Слайд 24


Ensemble Learning
Описание слайда:
Ensemble Learning

Слайд 25


Bias and Variance A complex model could exhibit high variance A simple model could exhibit high bias
Описание слайда:
Bias and Variance A complex model could exhibit high variance A simple model could exhibit high bias

Слайд 26


Ensemble Classifier in General
Описание слайда:
Ensemble Classifier in General

Слайд 27


Ensemble Classifier in General
Описание слайда:
Ensemble Classifier in General

Слайд 28


Ensemble Classifier in General
Описание слайда:
Ensemble Classifier in General

Слайд 29


Important A necessary and sufficient condition for an ensemble of classifiers to be more accurate than any of its individual members is if the members are accurate and diverse (Hansen & Salamon, 1990)
Описание слайда:
Important A necessary and sufficient condition for an ensemble of classifiers to be more accurate than any of its individual members is if the members are accurate and diverse (Hansen & Salamon, 1990)

Слайд 30


Bagging: Reducing Variance using An Ensemble of Classifiers from Bootstrap Samples
Описание слайда:
Bagging: Reducing Variance using An Ensemble of Classifiers from Bootstrap Samples

Слайд 31


Aside: Bootstrapping
Описание слайда:
Aside: Bootstrapping

Слайд 32


Bagging
Описание слайда:
Bagging

Слайд 33


Why Bagging Works? Averaging reduces variance Let be i.i.d random variables
Описание слайда:
Why Bagging Works? Averaging reduces variance Let be i.i.d random variables

Слайд 34


Bagging Summary Bagging was first proposed by Leo Breiman in a technical report in 1994 He also showed that bagging can improve the accuracy of unstable models and decrease the degree of overfitting. I highly recommend you read about his research in L. Breiman. Bagging Predictors. Machine Learning, 24(2):123–140, 1996,
Описание слайда:
Bagging Summary Bagging was first proposed by Leo Breiman in a technical report in 1994 He also showed that bagging can improve the accuracy of unstable models and decrease the degree of overfitting. I highly recommend you read about his research in L. Breiman. Bagging Predictors. Machine Learning, 24(2):123–140, 1996,

Слайд 35


Random Forests – Example of Bagging Draw a random bootstrap sample Grow a decision tree from the bootstrap sample. At each node: Randomly select d features without replacement ( ). Split the node using the feature that provides the best split according to the objective function, for instance, by maximizing the information gain. Repeat the steps 1 to 2 k times. Aggregate the prediction by each tree to assign the class label by majority voting
Описание слайда:
Random Forests – Example of Bagging Draw a random bootstrap sample Grow a decision tree from the bootstrap sample. At each node: Randomly select d features without replacement ( ). Split the node using the feature that provides the best split according to the objective function, for instance, by maximizing the information gain. Repeat the steps 1 to 2 k times. Aggregate the prediction by each tree to assign the class label by majority voting

Слайд 36


Making a Prediction
Описание слайда:
Making a Prediction

Слайд 37


Boosting: Converting Weak Learners to Strong Learners through Ensemble Learning
Описание слайда:
Boosting: Converting Weak Learners to Strong Learners through Ensemble Learning

Слайд 38


Boosting and Bagging Works in a similar way as bagging. Except: Models are built sequentially: each model is built using information from previously built models. Boosting does not involve bootstrap sampling; instead each tree is fit on a modified version of the original data set
Описание слайда:
Boosting and Bagging Works in a similar way as bagging. Except: Models are built sequentially: each model is built using information from previously built models. Boosting does not involve bootstrap sampling; instead each tree is fit on a modified version of the original data set

Слайд 39


Boosting: (1) Train A Classifier
Описание слайда:
Boosting: (1) Train A Classifier

Слайд 40


Boosting: (2) Train Next Classifier by Focusing More on the Hard Points
Описание слайда:
Boosting: (2) Train Next Classifier by Focusing More on the Hard Points

Слайд 41


What does it mean to focus more?
Описание слайда:
What does it mean to focus more?

Слайд 42


Example (Unweighted): Learning a Simple Decision Stump
Описание слайда:
Example (Unweighted): Learning a Simple Decision Stump

Слайд 43


Example (Weighted): Learning a Decision Stump on Weighted Data
Описание слайда:
Example (Weighted): Learning a Decision Stump on Weighted Data

Слайд 44


Boosting
Описание слайда:
Boosting

Слайд 45


AdaBoost (Example of Boosting) Start with the same weights for all points: For each Learn with data weights Compute coefficient Recompute weights Final model predicts as:
Описание слайда:
AdaBoost (Example of Boosting) Start with the same weights for all points: For each Learn with data weights Compute coefficient Recompute weights Final model predicts as:

Слайд 46


Lecture 7, слайд №46
Описание слайда:

Слайд 47


Weighted Classification Error Total weight of the mistakes: Total weight of all points: Weighted error measures fraction of weight of mistakes: Best possible values is 0.0
Описание слайда:
Weighted Classification Error Total weight of the mistakes: Total weight of all points: Weighted error measures fraction of weight of mistakes: Best possible values is 0.0

Слайд 48


AdaBoost: Computing Classifier’s Weights
Описание слайда:
AdaBoost: Computing Classifier’s Weights

Слайд 49


AdaBoost Start with the same weights for all points: For each Learn with data weights Compute coefficient Recompute weights Final model predicts by:
Описание слайда:
AdaBoost Start with the same weights for all points: For each Learn with data weights Compute coefficient Recompute weights Final model predicts by:

Слайд 50


Lecture 7, слайд №50
Описание слайда:

Слайд 51


AdaBoost: Recomputing A Sample’s Weight
Описание слайда:
AdaBoost: Recomputing A Sample’s Weight

Слайд 52


AdaBoost: Recomputing A Sample’s Weight
Описание слайда:
AdaBoost: Recomputing A Sample’s Weight

Слайд 53


AdaBoost
Описание слайда:
AdaBoost

Слайд 54


AdaBoost: Normalizing Sample Weights
Описание слайда:
AdaBoost: Normalizing Sample Weights

Слайд 55


AdaBoost
Описание слайда:
AdaBoost

Слайд 56


Self Study What is the effect of of: Increasing the number of classifiers in bagging vs. Increasing the number of classifiers in boosting
Описание слайда:
Self Study What is the effect of of: Increasing the number of classifiers in bagging vs. Increasing the number of classifiers in boosting

Слайд 57


Boosting Summary
Описание слайда:
Boosting Summary

Слайд 58


Summary Decision Tree Pruning Ensemble Learning Bagging Boosting
Описание слайда:
Summary Decision Tree Pruning Ensemble Learning Bagging Boosting

Скачать

Теги Lecture 7
Похожие презентации
Mypresentation.ru
Загрузить презентацию