---
title: "An Improved Baseline for Sentence-level Relation Extraction (2022)"
description: "The authors build a simple classifier on a PLM. 72.7% F1 on TACRED. 24 references, cited by 62 papers here."
authors: ["Wenxuan Zhou", "Muhao Chen"]
year: 2022
venue: "Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)"
source: "https://aclanthology.org/2022.aacl-short.21/"
doi: "10.18653/v1/2022.aacl-short.21"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhou-2022-improved-baseline-sentence-level-relation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhou-2022-improved-baseline-sentence-level-relation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/3/) / An Improved Baseline for Sentence-level Relation Extraction

# An Improved Baseline for Sentence-level Relation Extraction

**Authors:** [Wenxuan Zhou](https://md.research.elamur.ai/free/en/authors/wenxuan-zhou/) and [Muhao Chen](https://md.research.elamur.ai/free/en/authors/muhao-chen/) · **Year:** 2022 · **Venue:** [Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)](https://md.research.elamur.ai/free/en/venues/aacl/) ([AACL 2022](https://md.research.elamur.ai/free/en/venues/aacl/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.aacl-short.21/) · [PDF](https://aclanthology.org/2022.aacl-short.21.pdf) · [DOI 10.18653/v1/2022.aacl-short.21](https://doi.org/10.18653/v1/2022.aacl-short.21) · [OpenAlex](https://openalex.org/W4404784155)

**Citations:** 62 citations (OpenAlex, 2026-10-14) · [62](https://md.research.elamur.ai/free/en/papers/zhou-2022-improved-baseline-sentence-level-relation/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Sentence-level relation extraction (RE) predicts the relation between a subject and an object entity in one sentence. Even strong pretrained models such as LUKE only reached about 72.7% F1 on TACRED. Zhou and Chen point to two causes. First, existing models encode the given entity information (names, spans, NER types) incompletely. Second, labels in benchmarks like TACRED are noisy or poorly defined.

The authors build a simple classifier on a PLM. It feeds the final-layer hidden states of the subject and object into a ReLU projection followed by softmax. They compare five ways of marking entities in the input text, including a new variant, the "typed entity marker (punct)". This variant wraps entities with "@"/"#" and inserts the NER type as plain words, so no new vocabulary tokens are needed.

Tested on TACRED, TACREV and Re-TACRED, RoBERTa-LARGE with this marking sets new state-of-the-art results on all three. The results show that well-designed input formatting matters, and that on cleaner data PLMs already perform very well.

## Key points
- **Best results:** RoBERTa-LARGE with typed entity marker (punct) reaches 74.6% F1 on TACRED, 83.2% on TACREV and 91.1% on Re-TACRED. LUKE scores 72.7 / 80.6 / 90.3 on the same sets.
- **Entity types help:** Typed markers clearly beat untyped markers and entity masking, so keeping both entity names and types in the input improves RE.
- **Marker symbols matter:** On RoBERTa-LARGE, introducing new special tokens hurts performance.
  - The typed marker with new tokens scores 71.0, versus 74.6 for the punctuation version.
  - Entity mask drops to 60.9.
- **Smaller gains on clean data:** On Re-TACRED, typed markers add only 0.2–0.8 F1, compared with 2.0–3.9 on TACRED and TACREV. The authors link this to TACRED label noise that is biased toward entity side information.
- **Setup:** Built on HuggingFace Transformers with Adam (learning rate 5e-5 for BERT-BASE, 3e-5 for large models), batch size 64 and 5 epochs. Scores are the median of 5 seeds. Code is at github.com/wzhouad/RE_improved_baseline.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Southern California](https://md.research.elamur.ai/free/en/universities/university-of-southern-california/) — University, Los Angeles, United States · 104 works · 33,876 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔎 Information extraction](https://md.research.elamur.ai/free/en/topics/information-extraction/) · 109 works

## Cite

Cite as: Wenxuan Zhou and Muhao Chen. 2022. An Improved Baseline for Sentence-level Relation Extraction. Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhou2022improved,
  title = {{An Improved Baseline for Sentence-level Relation Extraction}},
  author = {Wenxuan Zhou and Muhao Chen},
  booktitle = {Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
  year = {2022},
  doi = {10.18653/v1/2022.aacl-short.21},
  url = {https://doi.org/10.18653/v1/2022.aacl-short.21}
}
```

- [← previous XLM-E: Cross-lingual Language Model Pre-training via ELECTRA](https://md.research.elamur.ai/free/en/papers/chi-2022-xlm-e-cross-lingual-language/)
- [next → ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts](https://md.research.elamur.ai/free/en/papers/asai-2022-attempt-parameter-efficient-multi-task/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/3/)

## References

24 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020. [TACRED revisited: A thorough evaluation of the TACRED relation extraction task](https://md.research.elamur.ai/free/en/papers/alt-2020-tacred-revisited-thorough-evaluation-tacred/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1558– 1569, Online. Association for Computational Linguist…
2. Mayank Kulkarni, Debanjan Mahata, Ravneet Arora, and Rajarshi Bhowmik. 2022. Learning rich representation of keyphrases from text. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 891–906, Seattle, United States. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mayank+Kulkarni%2C+Debanjan+Mahata%2C+Ravneet+Arora%2C+and+Rajarshi+Bhowmik%2E+2022%2E+Learning+rich+representation+of+keyphrases+from+text%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+NAACL+2)
3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). CoRR, abs/1409.0473.
4. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
5. Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019. [Matching the blanks: Distributional similarity for relation learning](https://md.research.elamur.ai/free/en/papers/soares-2019-matching-blanks-distributional-similarity-relation/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2895–2905, Florence, Italy. Association for Comp…
6. Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022. Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction. In Proceedings of the ACM Web Conference 2022, pages 2778–2788. [find](https://scholar.google.com/scholar?q=Xiang+Chen%2C+Ningyu+Zhang%2C+Xin+Xie%2C+Shumin+Deng%2C+Yunzhi+Yao%2C+Chuanqi+Tan%2C+Fei+Huang%2C+Luo+Si%2C+and+Huajun+Chen%2E+2022%2E+Knowprompt%3A+Knowledge%2Daware+prompt%2Dtuning+with+synergistic+optimization+for+relation+)
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
8. Priya Goyal, P. Dollár, Ross B. Girshick, P. Noordhuis, L. Wesolowski, Aapo Kyrola, Andrew Tulloch, Y. Jia, and Kaiming He. 2017. Accurate, large mini-batch sgd: Training imagenet in 1 hour. ArXiv, abs/1706.02677. [find](https://scholar.google.com/scholar?q=Priya+Goyal%2C+P%2E+Dolla%CC%81r%2C+Ross+B%2E+Girshick%2C+P%2E+Noordhuis%2C+L%2E+Wesolowski%2C+Aapo+Kyrola%2C+Andrew+Tulloch%2C+Y%2E+Jia%2C+and+Kaiming+He%2E+2017%2E+Accurate%2C+large+mini%2Dbatch+sgd%3A+Training+imagenet+in+1+hour%2E+ArXiv%2C+a)
9. Keming Lu, I Hsu, Wenxuan Zhou, Mingyu Derek Ma, Muhao Chen, et al. 2022. Summarization as indirect supervision for relation extraction. arXiv preprint arXiv:2205.09837. [find](https://scholar.google.com/scholar?q=Keming+Lu%2C+I+Hsu%2C+Wenxuan+Zhou%2C+Mingyu+Derek+Ma%2C+Muhao+Chen%2C+et+al%2E+2022%2E+Summarization+as+indirect+supervision+for+relation+extraction%2E+arXiv+preprint+arXiv%3A2205%2E09837%2E)
10. Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, and Jie Zhou. 2020. [Learning from Context or Names? An Empirical Study on Neural Relation Extraction](https://md.research.elamur.ai/free/en/papers/peng-2020-learning-context-or-names-empirical/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3661–3672, Onlin…
11. Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019. Knowledge enhanced contextual word representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on… [find](https://scholar.google.com/scholar?q=Matthew+E%2E+Peters%2C+Mark+Neumann%2C+Robert+Logan%2C+Roy+Schwartz%2C+Vidur+Joshi%2C+Sameer+Singh%2C+and+Noah+A%2E+Smith%2E+2019%2E+Knowledge+enhanced+contextual+word+representations%2E+In+Proceedings+of+the+2019+Conferen)
12. George Stoica, Emmanouil Antonios Platanios, and Barnabás Póczos. 2021. Re-tacred: Addressing shortcomings of the tacred dataset. In Proceedings of the AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=George+Stoica%2C+Emmanouil+Antonios+Platanios%2C+and+Barnaba%CC%81s+Po%CC%81czos%2E+2021%2E+Re%2Dtacred%3A+Addressing+shortcomings+of+the+tacred+dataset%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2E)
13. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in Neural Information Processing Systems, 30:5998–6008.
14. Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Cuihong Cao, Daxin Jiang, Ming Zhou, et al. 2020. [K-adapter: Infusing knowledge into pre-trained models with adapters](https://md.research.elamur.ai/free/en/papers/wang-2020-k-adapter-infusing-knowledge-pre/). arXiv preprint arXiv:2002.01808.
15. Yiwei Wang, Muhao Chen, Wenxuan Zhou, Yujun Cai, Yuxuan Liang, and Bryan Hooi. 2022a. Graph-Cache: Message passing as caching for sentence-level relation extraction. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1698–1708, Seattle, United States. Association for Com… [find](https://scholar.google.com/scholar?q=Yiwei+Wang%2C+Muhao+Chen%2C+Wenxuan+Zhou%2C+Yujun+Cai%2C+Yuxuan+Liang%2C+and+Bryan+Hooi%2E+2022a%2E+Graph%2DCache%3A+Message+passing+as+caching+for+sentence%2Dlevel+relation+extraction%2E+In+Findings+of+the+Association+for)
16. Yiwei Wang, Muhao Chen, Wenxuan Zhou, Yujun Cai, Yuxuan Liang, Dayiheng Liu, Baosong Yang, Juncheng Liu, and Bryan Hooi. 2022b. Should we rely on entity mentions for relation extraction? debiasing relation extraction with counterfactual analysis. In Proceedings of the 2022 Conference of the North Am… [find](https://scholar.google.com/scholar?q=Yiwei+Wang%2C+Muhao+Chen%2C+Wenxuan+Zhou%2C+Yujun+Cai%2C+Yuxuan+Liang%2C+Dayiheng+Liu%2C+Baosong+Yang%2C+Juncheng+Liu%2C+and+Bryan+Hooi%2E+2022b%2E+Should+we+rely+on+entity+mentions+for+relation+extraction%3F+debiasing+rel)
17. [Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho…](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
18. Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, and Yuji Matsumoto. 2020. LUKE: Deep contextualized entity representations with entity-aware self-attention. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6442–6454, Online. Associat… [find](https://scholar.google.com/scholar?q=Ikuya+Yamada%2C+Akari+Asai%2C+Hiroyuki+Shindo%2C+Hideaki+Takeda%2C+and+Yuji+Matsumoto%2E+2020%2E+LUKE%3A+Deep+contextualized+entity+representations+with+entity%2Daware+self%2Dattention%2E+In+Proceedings+of+the+2020+Confe)
19. Yuhao Zhang, Peng Qi, and Christopher D. Manning. 2018. [Graph convolution over pruned dependency trees improves relation extraction](https://md.research.elamur.ai/free/en/papers/zhang-2018-graph-convolution-over-pruned-dependency/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2205–2215, Brussels, Belgium. Association for Computational Linguistic…
20. Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D. Manning. 2017. Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 35–45, Copenhagen, Denmark. Association for Compu… [find](https://scholar.google.com/scholar?q=Yuhao+Zhang%2C+Victor+Zhong%2C+Danqi+Chen%2C+Gabor+Angeli%2C+and+Christopher+D%2E+Manning%2E+2017%2E+Position%2Daware+attention+and+supervised+data+improve+slot+filling%2E+In+Proceedings+of+the+2017+Conference+on+Empir)
21. Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: Enhanced language representation with informative entities. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1441–1451, Florence, Italy. Association for Computation… [find](https://scholar.google.com/scholar?q=Zhengyan+Zhang%2C+Xu+Han%2C+Zhiyuan+Liu%2C+Xin+Jiang%2C+Maosong+Sun%2C+and+Qun+Liu%2E+2019%2E+ERNIE%3A+Enhanced+language+representation+with+informative+entities%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Asso)
22. Zexuan Zhong and Danqi Chen. 2021. A frustratingly easy approach for joint entity and relation extraction. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. [find](https://scholar.google.com/scholar?q=Zexuan+Zhong+and+Danqi+Chen%2E+2021%2E+A+frustratingly+easy+approach+for+joint+entity+and+relation+extraction%2E+In+Proceedings+of+the+2021+Conference+of+the+North+American+Chapter+of+the+Association+for+Co)
23. Wenxuan Zhou and Muhao Chen. 2021. [An improved baseline for sentence-level relation extraction](https://md.research.elamur.ai/free/en/papers/zhou-2022-improved-baseline-sentence-level-relation/). arXiv preprint arXiv:2102.01373v1. — [arXiv:2102.01373](https://arxiv.org/abs/2102.01373)
24. Wenxuan Zhou, Kevin Huang, Tengyu Ma, and Jing Huang. 2021. Document-level relation extraction with adaptive thresholding and localized context pooling. In Proceedings of the AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Wenxuan+Zhou%2C+Kevin+Huang%2C+Tengyu+Ma%2C+and+Jing+Huang%2E+2021%2E+Document%2Dlevel+relation+extraction+with+adaptive+thresholding+and+localized+context+pooling%2E+In+Proceedings+of+the+AAAI+Conference+on+Artifi)

*Updated 24 October 2026*