Boosting, Gradient Boosting, and Adaptive Boosting

Published:

The note frames boosting as gradient descent on the empirical risk in function space, derives the downhill condition for each weak model from a first-order Taylor expansion, makes it concrete as gradient boosting, and then shows AdaBoost’s sample reweighting is gradient boosting with exponential loss.

Direct Link