Boosting, Gradient Boosting, and Adaptive Boosting
Published:
The note frames boosting as gradient descent on the empirical risk in function space, derives the downhill condition for each weak model from a first-order Taylor expansion, makes it concrete as gradient boosting, and then shows AdaBoost’s sample reweighting is gradient boosting with exponential loss.