The origin of the boosting method for designing learnign machines is traced back to the work of Valiant and Kearns, who posed the question of whether a weak learning algorithm, meaning one that does slightly better than random guessing, can be boosted into a strong one with a good performance index.

• At the heart of such techniques lies the base leaner, which is a week one.

Boosting consists of an iterative scheme, where

• at each step the base learn is optimally computed using a different training set;

Boosting consists of an iterative scheme, where at each step the base learner is optimally computed using a different training set;

• the set at the current iteration is generated either:
  • according to an iteratively obtained data distribution or,
  • usually, via a weighting of the training samples, each time using a different set of weights.

The final learner is obtained via a weighted average of all the hierarchically designed base learners.

It turns out that, given a sufficient number of iterations, one can significantly imporve the (poor) performance of a weak learner. (?)

---

Read More »

In general, approaches to cope with concept drift can be classified into two categories:

• approaches that adapt a learner at regular intervals without considering whether changes have really occurred.
• approaches that first detect concept changes and afterwards the learner is adapted to these changes.

  In the fist approach, drifting concepts are often handled by time windows or weighted examples according to their age or utility.

• Weighted examples are based on the simple idea that the importance often example should decrease with time.
• When a tie window is used, at each time step the learner is induced only from the examples that are included in the window.
  • The difficult is how to select the appropriate window’s size.
  • On the other end, a large window would produce good and stable learning results in stable phases but can not react quickly to concept changes.

Read More »