---
title: "Learning to Rank using Gradient Descent — Burges 2005"
description: "The paper addresses learning to rank web search results."
authors: ["Chris Burges", "Tal Shaked", "Erin Renshaw", "Ari Lazier", "Matt Deeds", "Nicole Hamilton", "Greg Hullender"]
year: 2005
venue: "ICML 2005"
source: "https://www.microsoft.com/en-us/research/wp-content/uploads/2005/08/icml_ranking.pdf"
doi: "10.1145/1102351.1102363"
licence: "author's copy (Microsoft Research) — read only"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/burges-2005-learning-rank-using-gradient/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/burges-2005-learning-rank-using-gradient/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2005](https://md.research.elamur.ai/free/en/papers/2005/) / Learning to Rank using Gradient Descent

# Learning to Rank using Gradient Descent

**Authors:** Chris Burges, Tal Shaked, [Erin Renshaw](https://md.research.elamur.ai/free/en/authors/erin-renshaw/), Ari Lazier, Matt Deeds, Nicole Hamilton, Greg Hullender · **Year:** 2005 · **Venue:** ICML 2005 · **Licence:** author's copy (Microsoft Research) — read only

**Links:** [Publisher's page](https://doi.org/10.1145/1102351.1102363) · [PDF](https://www.microsoft.com/en-us/research/wp-content/uploads/2005/08/icml_ranking.pdf) · [DOI 10.1145/1102351.1102363](https://doi.org/10.1145/1102351.1102363) · [OpenAlex](https://openalex.org/W2143331230)

**Citations:** 2757 citations (openalex-snapshot · 2026-10-14) · [139](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper addresses learning to rank web search results. Documents are compared only against others returned for the same query, and only their relative order matters. Earlier methods such as PRank framed this as ordinal regression with learned rank thresholds. The authors argue that this solves a harder problem than necessary. Instead, they learn a real-valued scoring function from ordered pairs of documents. A logistic function maps the difference between two scores to the probability that one document should outrank the other, and the model is trained with cross-entropy against target pair probabilities. This cost works with any differentiable model. Here it is paired with neural networks trained by an extended backpropagation, a combination they call RankNet. They also show how pairwise target probabilities can be combined consistently. On data from a commercial search engine, RankNet beat PRank variants and RankProp on NDCG, while remaining reasonably fast to train. The work matters because it provides a simple, probabilistic, scalable pairwise ranking loss. It became a foundation for later learning-to-rank systems.

## Key points
- The dataset contains 17,004 queries, each with up to 1,000 documents pre-ranked by a simpler system. It was split into 11,336 training, 2,834 validation and 2,834 test queries, and only about 1% of test documents carry relevance labels.
- Test NDCG@15 results:
  - two-layer RankNet (10 hidden units): 0.488
  - linear RankNet: 0.477
  - RankProp: 0.460
  - OAP-BPM (large-margin PRank): 0.454
  - linear PRank: 0.412
  - quadratic PRank: 0.327
- Kernel PRank needed over 12 hours per epoch, so it was approximated with explicit quadratic features (162,734 dimensions). Training that version took about 40 hours, compared with about 6 hours for the two-layer RankNet.
- A theorem shows that target probabilities for adjacent pairs in any ordering are enough to determine consistent probabilities for every pair. Setting a target of 0.5 provides a principled way to train on items of equal rank.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Computing & GPUs](https://md.research.elamur.ai/free/en/shelves/computing/)

## Cited by

Cited by

139 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 128 not yet.

- [InternLM2 Technical Report](https://md.research.elamur.ai/free/en/papers/cai-2024-internlm2-technical-report/)
- [Taking MT Evaluation Metrics to Extremes: Beyond Correlation with Human Judgments](https://md.research.elamur.ai/free/en/papers/fomicheva-2019-taking-mt-evaluation-metrics-extremes/)
- [Aligning AI With Shared Human Values](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-aligning-ai-shared-human-values/)
- [Tuning as Ranking](https://md.research.elamur.ai/free/en/papers/hopkins-2011-tuning-as-ranking/)
- [LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion](https://md.research.elamur.ai/free/en/papers/jiang-2023-llm-blender-ensembling-large-language/)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/)
- [Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval](https://md.research.elamur.ai/free/en/papers/liu-2015-representation-learning-multi-task-deep/)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/liu-2019-multi-task-deep-neural-networks/)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://md.research.elamur.ai/free/en/papers/sun-2023-chatgpt-good-search-investigating-large/)
- [Learning Discriminative Projections for Text Similarity Measures](https://md.research.elamur.ai/free/en/papers/yih-2011-learning-discriminative-projections-text-similarity/)
- [Large Language Models for Information Retrieval: A Survey](https://md.research.elamur.ai/free/en/papers/zhu-2023-large-language-models-information-retrieval/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{burges2005learning,
  title = {{Learning to Rank using Gradient Descent}},
  author = {Burges, Chris and Shaked, Tal and Renshaw, Erin and Lazier, Ari and Deeds, Matt and Hamilton, Nicole and Hullender, Greg},
  booktitle = {ICML 2005},
  year = {2005},
  doi = {10.1145/1102351.1102363},
  url = {https://doi.org/10.1145/1102351.1102363}
}
```

- [← previous Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/carreras-2005-introduction-conll-2005-shared-task/)
- [next → Machine Learning for Coreference Resolution: From Local Classification to Global Ranking](https://md.research.elamur.ai/free/en/papers/ng-2005-machine-learning-coreference-resolution-local/)
- [↑ back to papers of 2005](https://md.research.elamur.ai/free/en/papers/2005/)

## References

18 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Baum, E., & Wilczek, F. (1988). Supervised learning of probability distributions by neural networks. Neural Information Processing Systems (pp. 52–61). [find](https://scholar.google.com/scholar?q=Baum%2C+E%2E%2C+%26+Wilczek%2C+F%2E+%281988%29%2E+Supervised+learning+of+probability+distributions+by+neural+networks%2E+Neural+Information+Processing+Systems+%28pp%2E+52%E2%80%9361%29%2E)
2. Bradley, R., & Terry, M. (1952). The Rank Analysis of Incomplete Block Designs 1: The Method of Paired Comparisons. Biometrika, 39, 324–245. [find](https://scholar.google.com/scholar?q=Bradley%2C+R%2E%2C+%26+Terry%2C+M%2E+%281952%29%2E+The+Rank+Analysis+of+Incomplete+Block+Designs+1%3A+The+Method+of+Paired+Comparisons%2E+Biometrika%2C+39%2C+324%E2%80%93245%2E)
3. Bromley, J., Bentz, J.W., Bottou, L., Guyon, I., Le-Cun, Y., Moore, C., Sackinger, E., & Shah, R. (1993). Signature Verification Using a ”Siamese” Time Delay Neural Network. Advances in Pattern Recognition Systems using Neural Network Technologies, World Scientific (pp. 25–44) [find](https://scholar.google.com/scholar?q=Bromley%2C+J%2E%2C+Bentz%2C+J%2EW%2E%2C+Bottou%2C+L%2E%2C+Guyon%2C+I%2E%2C+Le%2DCun%2C+Y%2E%2C+Moore%2C+C%2E%2C+Sackinger%2C+E%2E%2C+%26+Shah%2C+R%2E+%281993%29%2E+Signature+Verification+Using+a+%E2%80%9DSiamese%E2%80%9D+Time+Delay+Neural+Network%2E+Advances+in+Pattern+Recogn)
4. Burges, C. (1996). Simplified support vector decision rules. Proc. International Conference on Machine Learning (ICML) 13 (pp. 71–77). [find](https://scholar.google.com/scholar?q=Burges%2C+C%2E+%281996%29%2E+Simplified+support+vector+decision+rules%2E+Proc%2E+International+Conference+on+Machine+Learning+%28ICML%29+13+%28pp%2E+71%E2%80%9377%29%2E)
5. Caruana, R., Baluja, S., & Mitchell, T. (1996). Using the future to “sort out” the present: Rankprop and multitask learning for medical risk evaluation. Advances in Neural Information Processing Systems (NIPS) 8 (pp. 959–965). [find](https://scholar.google.com/scholar?q=Caruana%2C+R%2E%2C+Baluja%2C+S%2E%2C+%26+Mitchell%2C+T%2E+%281996%29%2E+Using+the+future+to+%E2%80%9Csort+out%E2%80%9D+the+present%3A+Rankprop+and+multitask+learning+for+medical+risk+evaluation%2E+Advances+in+Neural+Information+Processing+Syste)
6. Crammer, K., & Singer, Y. (2002). Pranking with ranking. NIPS 14. [find](https://scholar.google.com/scholar?q=Crammer%2C+K%2E%2C+%26+Singer%2C+Y%2E+%282002%29%2E+Pranking+with+ranking%2E+NIPS+14%2E)
7. Dekel, O., Manning, C., & Singer, Y. (2004). Log-linear models for label-ranking. NIPS 16. [find](https://scholar.google.com/scholar?q=Dekel%2C+O%2E%2C+Manning%2C+C%2E%2C+%26+Singer%2C+Y%2E+%282004%29%2E+Log%2Dlinear+models+for+label%2Dranking%2E+NIPS+16%2E)
8. Freund, Y., Iyer, R., Schapire, R., & Singer, Y. (2003). An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4, 933–969. [find](https://scholar.google.com/scholar?q=Freund%2C+Y%2E%2C+Iyer%2C+R%2E%2C+Schapire%2C+R%2E%2C+%26+Singer%2C+Y%2E+%282003%29%2E+An+efficient+boosting+algorithm+for+combining+preferences%2E+Journal+of+Machine+Learning+Research%2C+4%2C+933%E2%80%93969%2E)
9. Harrington, E. (2003). Online ranking/collaborative filtering using the Perceptron algorithm. ICML 20. [find](https://scholar.google.com/scholar?q=Harrington%2C+E%2E+%282003%29%2E+Online+ranking%2Fcollaborative+filtering+using+the+Perceptron+algorithm%2E+ICML+20%2E)
10. Hastie, T., & Tibshirani, R. (1998). Classification by pairwise coupling. NIPS 10. [find](https://scholar.google.com/scholar?q=Hastie%2C+T%2E%2C+%26+Tibshirani%2C+R%2E+%281998%29%2E+Classification+by+pairwise+coupling%2E+NIPS+10%2E)
11. Herbrich, R., Graepel, T., & Obermayer, K. (2000). Large margin rank boundaries for ordinal regression. Advances in Large Margin Classifiers, MIT Press (pp. 115–132). [find](https://scholar.google.com/scholar?q=Herbrich%2C+R%2E%2C+Graepel%2C+T%2E%2C+%26+Obermayer%2C+K%2E+%282000%29%2E+Large+margin+rank+boundaries+for+ordinal+regression%2E+Advances+in+Large+Margin+Classifiers%2C+MIT+Press+%28pp%2E+115%E2%80%93132%29%2E)
12. Jarvelin, K., & Kekalainen, J. (2000). IR evaluation methods for retrieving highly relevant documents. Proc. 23rd ACM SIGIR (pp. 41–48). [find](https://scholar.google.com/scholar?q=Jarvelin%2C+K%2E%2C+%26+Kekalainen%2C+J%2E+%282000%29%2E+IR+evaluation+methods+for+retrieving+highly+relevant+documents%2E+Proc%2E+23rd+ACM+SIGIR+%28pp%2E+41%E2%80%9348%29%2E)
13. Kimeldorf, G. S., & Wahba, G. (1971). Some results on Tchebycheffian Spline Functions. J. Mathematical Analysis and Applications, 33, 82–95. [find](https://scholar.google.com/scholar?q=Kimeldorf%2C+G%2E+S%2E%2C+%26+Wahba%2C+G%2E+%281971%29%2E+Some+results+on+Tchebycheffian+Spline+Functions%2E+J%2E+Mathematical+Analysis+and+Applications%2C+33%2C+82%E2%80%9395%2E)
14. LeCun, Y., Bottou, L., Orr, G. B., & Müller, K.-R. (1998). Efficient backprop. Neural Networks: Tricks of the Trade, Springer (pp. 9–50). [find](https://scholar.google.com/scholar?q=LeCun%2C+Y%2E%2C+Bottou%2C+L%2E%2C+Orr%2C+G%2E+B%2E%2C+%26+Mu%CC%88ller%2C+K%2E%2DR%2E+%281998%29%2E+Efficient+backprop%2E+Neural+Networks%3A+Tricks+of+the+Trade%2C+Springer+%28pp%2E+9%E2%80%9350%29%2E)
15. Mason, L., Baxter, J., Bartlett, P., & Frean, M. (2000). Boosting algorithms as gradient descent. NIPS 12 (pp. 512–518). [find](https://scholar.google.com/scholar?q=Mason%2C+L%2E%2C+Baxter%2C+J%2E%2C+Bartlett%2C+P%2E%2C+%26+Frean%2C+M%2E+%282000%29%2E+Boosting+algorithms+as+gradient+descent%2E+NIPS+12+%28pp%2E+512%E2%80%93518%29%2E)
16. Mitchell, T. M. (1997). Machine learning. New York: McGraw-Hill. [find](https://scholar.google.com/scholar?q=Mitchell%2C+T%2E+M%2E+%281997%29%2E+Machine+learning%2E+New+York%3A+McGraw%2DHill%2E)
17. Refregier, P., & Vallet, F. (1991). Probabilistic approaches for multiclass classification with neural networks. International Conference on Artificial Neural Networks (pp. 1003–1006). [find](https://scholar.google.com/scholar?q=Refregier%2C+P%2E%2C+%26+Vallet%2C+F%2E+%281991%29%2E+Probabilistic+approaches+for+multiclass+classification+with+neural+networks%2E+International+Conference+on+Artificial+Neural+Networks+%28pp%2E+1003%E2%80%931006%29%2E)
18. Schölkopf, B., & Smola, A. (2002). Learning with kernels. MIT Press. [find](https://scholar.google.com/scholar?q=Scho%CC%88lkopf%2C+B%2E%2C+%26+Smola%2C+A%2E+%282002%29%2E+Learning+with+kernels%2E+MIT+Press%2E)