---
title: "Boosting Entity Linking Performance by Leveraging Unlabeled"
description: "Le and Titov build a linker from only Wikipedia and raw, unannotated documents. 1.2 points. 18 references, cited by 15 papers here."
authors: ["Phong Le", "Ivan Titov"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1187/"
doi: "10.18653/v1/p19-1187"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/le-2019-boosting-entity-linking-performance-leveraging/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/le-2019-boosting-entity-linking-performance-leveraging/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/) / Boosting Entity Linking Performance by Leveraging Unlabeled Documents

# Boosting Entity Linking Performance by Leveraging Unlabeled Documents

**Authors:** Phong Le and [Ivan Titov](https://md.research.elamur.ai/free/en/authors/ivan-titov/) · **Year:** 2019 · **Venue:** [Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2019](https://md.research.elamur.ai/free/en/venues/acl/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1187/) · [PDF](https://aclanthology.org/P19-1187.pdf) · [DOI 10.18653/v1/p19-1187](https://doi.org/10.18653/v1/p19-1187) · [OpenAlex](https://openalex.org/W2952490462)

**Citations:** 46 citations (OpenAlex, 2026-10-14) · 15 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Entity linkers map textual mentions to knowledge-base entries such as Wikipedia pages. Strong recent linkers are trained on manually annotated corpora like AIDA CoNLL, which are expensive to build, exist mostly for English, and transfer poorly to new domains. Le and Titov build a linker from only Wikipedia and raw, unannotated documents.

The method has two stages. First, Wikipedia anchor statistics and a document-specific subgraph of the Wikipedia link graph produce, through parameter-free message passing, a short high-recall list of plausible candidates for each mention, plus a set of likely wrong entities. Second, these noisy lists act as weak supervision for a document-level neural model. The model treats the true entities as latent and scores each candidate by local context fit plus attention-weighted coherence with candidates of other mentions, using a "star model" approximation. It is trained with a margin-based ranking loss.

Across six standard benchmarks the approach rivals fully-supervised systems. It also clearly beats earlier Wikipedia-based linkers. This makes it attractive for languages and domains without annotated data.

## Key points
- The model ranks first on four of six standard test sets and on average. It stays competitive on the AIDA CoNLL test data even though the AIDA annotations were never used.
- It beats prior Wikipedia-based linkers, including Lazic et al. (2015), and a variant of itself trained only to predict Wikipedia links. This shows the unlabeled documents genuinely help.
- In ablations, removing local context scores drops F1 to 82.4%. Removing global attention costs 1.2 points.
- Accuracy rises with the number of unlabeled documents but levels off after roughly 10,000.
- On AIDA-A, accuracy for PER entities is 97.20% versus 97.73% for a fully-supervised model. Gaps are larger for LOC, ORG and especially MISC (75.71 vs 83.27), where nationalities are often confused with languages.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) — University, Edinburgh, United Kingdom · 359 works · 77,352 citations
- [University of Amsterdam](https://md.research.elamur.ai/free/en/universities/university-of-amsterdam/) — University, Amsterdam, Netherlands · 144 works · 173,906 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · 200 works

## Cite

Cite as: Phong Le and Ivan Titov. 2019. Boosting Entity Linking Performance by Leveraging Unlabeled Documents. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{le2019boosting,
  title = {{Boosting Entity Linking Performance by Leveraging Unlabeled Documents}},
  author = {Phong Le and Ivan Titov},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1187},
  url = {https://doi.org/10.18653/v1/p19-1187}
}
```

- [← previous ArSentD-LEV: A Multi-Topic Corpus for Target-based Sentiment Analysis in Arabic Levantine Tweets](https://md.research.elamur.ai/free/en/papers/baly-2019-arsentd-lev-multi-topic-corpus/)
- [next → How well do NLI models capture verb veridicality?](https://md.research.elamur.ai/free/en/papers/ross-2019-how-well-do-nli-models/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/)

## References

18 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ming-Wei Chang, Lev Ratinov, and Dan Roth. 2007. Guiding semi-supervision with constraint-driven learning. In Proceedings of the 45th annual meeting of the association of computational linguistics, pages 280–287. [find](https://scholar.google.com/scholar?q=Ming%2DWei+Chang%2C+Lev+Ratinov%2C+and+Dan+Roth%2E+2007%2E+Guiding+semi%2Dsupervision+with+constraint%2Ddriven+learning%2E+In+Proceedings+of+the+45th+annual+meeting+of+the+association+of+computational+linguistics%2C+pa)
2. Miao Fan, Qiang Zhou, and Thomas Fang Zheng. 2015. Distant supervision for entity linking. Proceedings of PACLIC. [find](https://scholar.google.com/scholar?q=Miao+Fan%2C+Qiang+Zhou%2C+and+Thomas+Fang+Zheng%2E+2015%2E+Distant+supervision+for+entity+linking%2E+Proceedings+of+PACLIC%2E)
3. Evgeniy Gabrilovich, Michael Ringgaard, and Amarnag Subramanya. 2013. Facc1: Freebase annotation of clueweb corpora. [find](https://scholar.google.com/scholar?q=Evgeniy+Gabrilovich%2C+Michael+Ringgaard%2C+and+Amarnag+Subramanya%2E+2013%2E+Facc1%3A+Freebase+annotation+of+clueweb+corpora%2E)
4. Octavian-Eugen Ganea and Thomas Hofmann. 2017. Deep joint entity disambiguation with local neural attention. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2609–2619. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Octavian%2DEugen+Ganea+and+Thomas+Hofmann%2E+2017%2E+Deep+joint+entity+disambiguation+with+local+neural+attention%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+)
5. Amir Globerson, Nevena Lazic, Soumen Chakrabarti, Amarnag Subramanya, Michael Ringaard, and Fernando Pereira. 2016. Collective entity resolution with multi-focal attention. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 621–6… [find](https://scholar.google.com/scholar?q=Amir+Globerson%2C+Nevena+Lazic%2C+Soumen+Chakrabarti%2C+Amarnag+Subramanya%2C+Michael+Ringaard%2C+and+Fernando+Pereira%2E+2016%2E+Collective+entity+resolution+with+multi%2Dfocal+attention%2E+In+Proceedings+of+the+54th+)
6. Zhaochen Guo and Denilson Barbosa. 2016. Robust named entity disambiguation with random walks. Semantic Web, (Preprint). [find](https://scholar.google.com/scholar?q=Zhaochen+Guo+and+Denilson+Barbosa%2E+2016%2E+Robust+named+entity+disambiguation+with+random+walks%2E+Semantic+Web%2C+%28Preprint%29%2E)
7. Johannes Hoffart, Mohamed Amir Yosef, Ilaria Bordino, Hagen Fürstenau, Manfred Pinkal, Marc Spaniol, Bilyana Taneva, Stefan Thater, and Gerhard Weikum. 2011. Robust disambiguation of named entities in text. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, p… [find](https://scholar.google.com/scholar?q=Johannes+Hoffart%2C+Mohamed+Amir+Yosef%2C+Ilaria+Bordino%2C+Hagen+Fu%CC%88rstenau%2C+Manfred+Pinkal%2C+Marc+Spaniol%2C+Bilyana+Taneva%2C+Stefan+Thater%2C+and+Gerhard+Weikum%2E+2011%2E+Robust+disambiguation+of+named+entities+i)
8. Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S. Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages… [find](https://scholar.google.com/scholar?q=Raphael+Hoffmann%2C+Congle+Zhang%2C+Xiao+Ling%2C+Luke+Zettlemoyer%2C+and+Daniel+S%2E+Weld%2E+2011%2E+Knowledge%2Dbased+weak+supervision+for+information+extraction+of+overlapping+relations%2E+In+Proceedings+of+the+49th+)
9. Harr Chen, SRK Branavan, Regina Barzilay, and David R Karger. 2009. Content modeling using latent permutations. Journal of Artificial Intelligence Research, 36:129–163. [find](https://scholar.google.com/scholar?q=Harr+Chen%2C+SRK+Branavan%2C+Regina+Barzilay%2C+and+David+R+Karger%2E+2009%2E+Content+modeling+using+latent+permutations%2E+Journal+of+Artificial+Intelligence+Research%2C+36%3A129%E2%80%93163%2E)
10. Nevena Lazic, Amarnag Subramanya, Michael Ringgaard, and Fernando Pereira. 2015. [Plato: A selective context model for entity resolution](https://md.research.elamur.ai/free/en/papers/lazic-2015-plato-selective-context-model-entity/). Transactions of the Association for Computational Linguistics, 3:503–515.
11. Xiao Cheng and Dan Roth. 2013. [Relational inference for wikification](https://md.research.elamur.ai/free/en/papers/cheng-2013-relational-inference-wikification/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1787–1796, Seattle, Washington, USA. Association for Computational Linguistics.
12. Phong Le and Ivan Titov. 2018. Improving Entity Linking by Modeling Latent Relations between Mentions. Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Phong+Le+and+Ivan+Titov%2E+2018%2E+Improving+Entity+Linking+by+Modeling+Latent+Relations+between+Mentions%2E+Proceedings+of+ACL%2E)
13. Andrew Chisholm and Ben Hachey. 2015. [Entity disambiguation with web links](https://md.research.elamur.ai/free/en/papers/chisholm-2015-entity-disambiguation-web-links/). Transactions of the Association of Computational Linguistics, 3:145–156.
14. Michael Collins and Terry Koo. 2005. [Discriminative reranking for natural language parsing](https://md.research.elamur.ai/free/en/papers/collins-2000-discriminative-reranking-natural-language-parsing/). Computational Linguistics, 31(1):25–70.
15. V. I. Levenshtein. 1966. Binary Codes Capable of Correcting Deletions, Insertions and Reversals. Soviet Physics Doklady, 10:707. [find](https://scholar.google.com/scholar?q=V%2E+I%2E+Levenshtein%2E+1966%2E+Binary+Codes+Capable+of+Correcting+Deletions%2C+Insertions+and+Reversals%2E+Soviet+Physics+Doklady%2C+10%3A707%2E)
16. David Milne and Ian H Witten. 2008. Learning to link with wikipedia. In Proceedings of the 17th ACM conference on Information and knowledge management, pages 509–518. ACM. [find](https://scholar.google.com/scholar?q=David+Milne+and+Ian+H+Witten%2E+2008%2E+Learning+to+link+with+wikipedia%2E+In+Proceedings+of+the+17th+ACM+conference+on+Information+and+knowledge+management%2C+pages+509%E2%80%93518%2E+ACM%2E)
17. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. [Distant supervision for relation extraction without labeled data](https://md.research.elamur.ai/free/en/papers/mintz-2009-distant-supervision-relation-extraction-without/). In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2…
18. Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, and Yoshiyasu Takefuji. 2016. [Joint learning of the embedding of words and entities for named entity disambiguation](https://md.research.elamur.ai/free/en/papers/yamada-2016-joint-learning-embedding-words-entities/). In Proceedings of CoNLL.

*Updated 30 October 2026*