The origin of the boosting method for designing learnign machines is traced back to the work of Valiant and Kearns, who posed the question of whether a weak learning algorithm, meaning one that does slightly better than random guessing, can be boosted into a strong one with a good performance index.
- At the heart of such techniques lies the base leaner, which is a week one.
Boosting consists of an iterative scheme, where
- at each step the base learn is optimally computed using a different training set;
Boosting consists of an iterative scheme, where at each step the base learner is optimally computed using a different training set;
- the set at the current iteration is generated either:
- according to an iteratively obtained data distribution or,
- usually, via a weighting of the training samples, each time using a different set of weights.
The final learner is obtained via a weighted average of all the hierarchically designed base learners.
It turns out that, given a sufficient number of iterations, one can significantly imporve the (poor) performance of a weak learner. (?)