What is LambdaMART?

LambdaMART is one of the Learning to Rank (LTR) algorithms developed by Chris Burges and his colleagues at Microsoft Research.

LTR

Learning to Rank (LTR) is a group of three main techniques that apply supervised Machine Learning (ML) algorithms to solve various ranking problems – it is commonly used in search engine ranking. LTR performs the act of ranking a list’s items and comes up with an optimal order for those items.

LambdaMART

LambdaMART is a technique where ranking is transformed into a pairwise classification or regression problem. The algorithms consider a pair of items at a single time, coming up with a viable ordering of those items before initiating the final order of the entire list.

LambdaMART is a combination of LambdaRank and MARTMultiple Additive Regression Trees (MART), an ensemble model of boosted regression trees known to deliver high prediction accuracy for diverse tasks – it is widely used in practice. (Multiple Additive Regression Trees). MART uses gradient boosted decision trees for prediction tasks. However, LambdaMART improves this by using gradient boosted decision trees with a cost function derived from LambdaRank to order any ranking situation. LambdaMART has shown better results than LambdaRank and the original RankNet on experimental datasets.

Below is one of the algorithms commonly used for LambdaMART:

Updating parameters

LambdaMART computes its decision using all the data that falls to that node. While LambdaMART only updates a few parameters at one time (namely, the split values for the current leaf nodes) it utilizes the entire data. This shows that, as long as the overall utility increases, LambdaMART is able to choose splits and leaf values that may decrease the utility for some queries.