Boosting Algorithms in Machine Learning, Part II: Gradient Boosting | by Gurjinder Kaur | Nov, 2024

On this article, we’ll find out about gradient boosting, a machine studying algorithm that lays the inspiration of in style frameworks like XGBoost and LightGBM that are award-winning options for a number of machine studying competitions.

It is a nice article for anybody occupied with utilizing ensembles in machine studying or an ideal refresher for somebody who’s already a professional however simply desires to take a break from dot matches and dot predicts and desires a glance below the hood!

We’ll cowl the fundamentals of ensemble studying and clarify how the Gradient Boosting algorithm makes predictions with a step-by-step instance. We’ll additionally discover the connection between gradient descent and gradient boosting and discover out if there may be any connection. Let’s get began!

Ensemble studying is the method of coaching and mixing a number of fashions, typically weak learners, to create a powerful learner with a better predictive energy. Two methods to do that are bagging and boosting.

1. Bagging