---
title: "Constituency Parsing using LLMs — Bai 2023"
description: "The paper asks whether modern large language models can produce phrase-structure trees, a task where supervised… 76 references, cited by 13 papers here."
authors: ["Xuefeng Bai", "Jialong Wu", "Yulong Chen", "Zhongqing Wang", "Kehai Chen", "Min Zhang", "Yue Zhang"]
year: 2023
venue: "arXiv:2310.19462"
source: "https://arxiv.org/abs/2310.19462"
doi: "10.1109/taslpro.2025.3600867"
arxiv: "2310.19462"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bai-2023-constituency-parsing-using-llms/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bai-2023-constituency-parsing-using-llms/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/) / Constituency Parsing using LLMs

# Constituency Parsing using LLMs

**Authors:** [Xuefeng Bai](https://md.research.elamur.ai/free/en/authors/xuefeng-bai/), Jialong Wu, [Yulong Chen](https://md.research.elamur.ai/free/en/authors/yulong-chen/), [Zhongqing Wang](https://md.research.elamur.ai/free/en/authors/zhongqing-wang/), [Kehai Chen](https://md.research.elamur.ai/free/en/authors/kehai-chen/), [Min Zhang](https://md.research.elamur.ai/free/en/authors/min-zhang/) and [Yue Zhang](https://md.research.elamur.ai/free/en/authors/yue-zhang/) · **Year:** 2023 · **Venue:** arXiv:2310.19462 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://doi.org/10.1109/taslpro.2025.3600867) · [PDF](https://arxiv.org/pdf/2310.19462) · [DOI 10.1109/taslpro.2025.3600867](https://doi.org/10.1109/taslpro.2025.3600867) · [arXiv 2310.19462](https://arxiv.org/abs/2310.19462) · [OpenAlex](https://openalex.org/W4413359453)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [13](https://md.research.elamur.ai/free/en/papers/bai-2023-constituency-parsing-using-llms/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
The paper asks whether modern large language models can produce phrase-structure trees, a task where supervised parsers reach about 96% while unsupervised methods remain below 65%. The authors cast constituency parsing as sequence generation. They serialize trees in three ways: bracket strings, transition-action sequences, and natural-language span statements. They then test a range of closed- and open-source LLMs in zero-shot, few-shot and fully fine-tuned modes, against strong graph-based parsers, on one in-domain and five out-of-domain test sets. LLMs clearly beat older sequence-based parsers but still trail graph-based systems after supervised training. Much of the gap comes from outputs that are malformed or do not match the input sentence. To address this, the authors add two fixes: Learning from Erroneous Samples (LES) and Parsing with Multi-agent Collaboration (PMC). Both reduce broken trees and raise F1 in all settings. The work matters because it shows a practical route to annotation-light syntactic parsing with general-purpose LLMs.

## Key points
- Trees are linearized three ways: bracket-based (following Vinyals et al.), transition-based (following Liu and Zhang) and span-based. The span-based format reads most like natural language but yields much longer sequences.
- Evaluation covers zero-shot, few-shot and full fine-tuning on six datasets. The treebanks referenced include the Penn Treebank (PTB) and the Multi-domain Constituent Treebank (MCTB).
- The core weakness of LLM parsers is the generation of invalid (ill-formed) trees and unfaithful trees, meaning trees whose words diverge from the input sentence.
- LES places faulty example parses with error annotations in the prompt. PMC uses two extra agents that check validity and faithfulness and send feedback so the parsing agent can revise its tree.
- In the zero-shot setting the enhanced approach reaches an F1 of 76.87, which the authors report as a new best for end-to-end unsupervised constituency parsing.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) · [AI & machine learning](https://md.research.elamur.ai/free/en/shelves/ai/)

Topics: [📐 Syntax & grammar formalisms](https://md.research.elamur.ai/free/en/topics/syntax-grammar/)

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 119 works · 110,950 citations
- [Harbin Institute of Technology](https://md.research.elamur.ai/free/en/universities/harbin-institute-of-technology/) — University, Harbin, China · 58 works · 25,250 citations
- [Westlake University](https://md.research.elamur.ai/free/en/universities/westlake-university/) — University, Hangzhou, China · 22 works · 6,635 citations
- [Soochow University](https://md.research.elamur.ai/free/en/universities/soochow-university/) — University, Suzhou, China · 34 works · 3,917 citations
- [Southeast University](https://md.research.elamur.ai/free/en/universities/southeast-university-cn/) — University, Nanjing, China · 2 works · 149 citations

## Cite

Cite as: Xuefeng Bai, Jialong Wu, Yulong Chen, Zhongqing Wang, Kehai Chen, Min Zhang and Yue Zhang. 2023. Constituency Parsing using LLMs. arXiv:2310.19462.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{bai2023constituency,
  title = {{Constituency Parsing using LLMs}},
  author = {Xuefeng Bai, Jialong Wu, Yulong Chen, Zhongqing Wang, Kehai Chen, Min Zhang and Yue Zhang},
  year = {2023},
  doi = {10.1109/taslpro.2025.3600867},
  eprint = {2310.19462},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1109/taslpro.2025.3600867}
}
```

- [← previous When Truth Matters - Addressing Pragmatic Categories in Natural Language Inference (NLI) by Large Language Models (LLMs)](https://md.research.elamur.ai/free/en/papers/gubelmann-2023-when-truth-matters-addressing-pragmatic/)
- [next → AmbiCoref: Evaluating Human and Model Sensitivity to Ambiguous Coreference](https://md.research.elamur.ai/free/en/papers/yuan-2023-ambicoref-evaluating-human-model-sensitivity/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/)

## References

76 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. P. Marcus, B. Santorini, and M. A. Marcinkiewicz, “Building a large annotated corpus of English: The Penn Treebank,” Computational Linguistics, vol. 19, no. 2, pp. 313–330, 1993. [our card](https://md.research.elamur.ai/free/en/papers/marcus-1993-building-large-annotated-corpus-english/)
2. M. Collins, “Three generative, lexicalised models for statistical parsing,” in Proc. of ACL. Association for Computational Linguistics, 1997, pp. 16–23. [our card](https://md.research.elamur.ai/free/en/papers/collins-1997-three-generative-lexicalised-models-statistical/)
3. E. Charniak, “A maximum-entropy-inspired parser,” in Proc. of NAACL. Association for Computational Linguistics, 2000, p. 132–139. [our card](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/)
4. S. Petrov and D. Klein, “Improved inference for unlexicalized parsing,” in Proc. of NAACL. Association for Computational Linguistics, 2007, pp. 404–411. [our card](https://md.research.elamur.ai/free/en/papers/petrov-2007-improved-inference-unlexicalized-parsing/)
5. X. Wang, H. Pham, P. Yin, and G. Neubig, “A tree-based decoder for neural machine translation,” in Proc. of EMNLP. Association for Computational Linguistics, 2018, pp. 4772–4777. [find](https://scholar.google.com/scholar?q=X%2E+Wang%2C+H%2E+Pham%2C+P%2E+Yin%2C+and+G%2E+Neubig%2C+%E2%80%9CA+tree%2Dbased+decoder+for+neural+machine+translation%2C%E2%80%9D+in+Proc%2E+of+EMNLP%2E+Association+for+Computational+Linguistics%2C+2018%2C+pp%2E+4772%E2%80%934777%2E)
6. M. Jiang and J. Diesner, “A constituency parsing tree based method for relation extraction from abstracts of scholarly publications,” in Proc. of TextGraphs. Association for Computational Linguistics, 2019, pp. 186–191. [find](https://scholar.google.com/scholar?q=M%2E+Jiang+and+J%2E+Diesner%2C+%E2%80%9CA+constituency+parsing+tree+based+method+for+relation+extraction+from+abstracts+of+scholarly+publications%2C%E2%80%9D+in+Proc%2E+of+TextGraphs%2E+Association+for+Computational+Linguistics%2C)
7. J. Xu and G. Durrett, “Neural extractive text summarization with syntactic compression,” in Proc. of EMNLP. Association for Computational Linguistics, 2019, pp. 3292–3303. [find](https://scholar.google.com/scholar?q=J%2E+Xu+and+G%2E+Durrett%2C+%E2%80%9CNeural+extractive+text+summarization+with+syntactic+compression%2C%E2%80%9D+in+Proc%2E+of+EMNLP%2E+Association+for+Computational+Linguistics%2C+2019%2C+pp%2E+3292%E2%80%933303%2E)
8. N. Kitaev, S. Cao, and D. Klein, “Multilingual constituency parsing with self-attention and pre-training,” in Proc. of ACL. Association for Computational Linguistics, 2019, pp. 3499–3505. [our card](https://md.research.elamur.ai/free/en/papers/kitaev-2019-multilingual-constituency-parsing-self-attention/)
9. X. Gu, Y. Shen, J. Shen, J. Shang, and J. Han, “Phrase-aware unsupervised constituency parsing,” in Proc. of ACL. Association for Computational Linguistics, 2022, pp. 6406–6415. [find](https://scholar.google.com/scholar?q=X%2E+Gu%2C+Y%2E+Shen%2C+J%2E+Shen%2C+J%2E+Shang%2C+and+J%2E+Han%2C+%E2%80%9CPhrase%2Daware+unsupervised+constituency+parsing%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Computational+Linguistics%2C+2022%2C+pp%2E+6406%E2%80%936415%2E)
10. Y. Chen, Y. Liu, L. Dong, S. Wang, C. Zhu, M. Zeng, and Y. Zhang, “AdaPrompt: Adaptive model training for prompt-based NLP,” in Findings of ACL: EMNLP 2022. Association for Computational Linguistics, 2022, pp. 6057–6068. [find](https://scholar.google.com/scholar?q=Y%2E+Chen%2C+Y%2E+Liu%2C+L%2E+Dong%2C+S%2E+Wang%2C+C%2E+Zhu%2C+M%2E+Zeng%2C+and+Y%2E+Zhang%2C+%E2%80%9CAdaPrompt%3A+Adaptive+model+training+for+prompt%2Dbased+NLP%2C%E2%80%9D+in+Findings+of+ACL%3A+EMNLP+2022%2E+Association+for+Computational+Linguistics%2C+)
11. X. Sun, X. Li, J. Li, F. Wu, S. Guo, T. Zhang, and G. Wang, “Text classification via large language models,” in Findings of ACL: EMNLP 2023. Association for Computational Linguistics, 2023, pp. 8990–9005. [our card](https://md.research.elamur.ai/free/en/papers/sun-2023-text-classification-large-language-models/)
12. Z. Zhang, S. Wang, Y. Xu, Y. Fang, W. Yu, Y. Liu, H. Zhao, C. Zhu, and M. Zeng, “Task compass: Scaling multi-task pre-training with task prefix,” in Findings of ACL: EMNLP 2022. Association for Computational Linguistics, 2022, pp. 5671–5685. [find](https://scholar.google.com/scholar?q=Z%2E+Zhang%2C+S%2E+Wang%2C+Y%2E+Xu%2C+Y%2E+Fang%2C+W%2E+Yu%2C+Y%2E+Liu%2C+H%2E+Zhao%2C+C%2E+Zhu%2C+and+M%2E+Zeng%2C+%E2%80%9CTask+compass%3A+Scaling+multi%2Dtask+pre%2Dtraining+with+task+prefix%2C%E2%80%9D+in+Findings+of+ACL%3A+EMNLP+2022%2E+Association+for+Comput)
13. A. Toroghi, W. Guo, A. Pesaranghader, and S. Sanner, “Verifiable, debuggable, and repairable commonsense logical reasoning via LLM-based theory resolution,” in Proc. of EMNLP. Association for Computational Linguistics, 2024, pp. 6634–6652. [find](https://scholar.google.com/scholar?q=A%2E+Toroghi%2C+W%2E+Guo%2C+A%2E+Pesaranghader%2C+and+S%2E+Sanner%2C+%E2%80%9CVerifiable%2C+debuggable%2C+and+repairable+commonsense+logical+reasoning+via+LLM%2Dbased+theory+resolution%2C%E2%80%9D+in+Proc%2E+of+EMNLP%2E+Association+for+Computat)
14. T. Goyal, J. J. Li, and G. Durrett, “News summarization and evaluation in the era of gpt-3,” ArXiv preprint, vol. abs/2209.12356, 2022. [find](https://scholar.google.com/scholar?q=T%2E+Goyal%2C+J%2E+J%2E+Li%2C+and+G%2E+Durrett%2C+%E2%80%9CNews+summarization+and+evaluation+in+the+era+of+gpt%2D3%2C%E2%80%9D+ArXiv+preprint%2C+vol%2E+abs%2F2209%2E12356%2C+2022%2E)
15. Y. Chen, Y. Liu, R. Xu, Z. Yang, C. Zhu, M. Zeng, and Y. Zhang, “UniSumm and SummZoo: Unified model and diverse benchmark for few-shot summarization,” in Proc. of ACL. Association for Computational Linguistics, 2023, pp. 12 833–12 855. [find](https://scholar.google.com/scholar?q=Y%2E+Chen%2C+Y%2E+Liu%2C+R%2E+Xu%2C+Z%2E+Yang%2C+C%2E+Zhu%2C+M%2E+Zeng%2C+and+Y%2E+Zhang%2C+%E2%80%9CUniSumm+and+SummZoo%3A+Unified+model+and+diverse+benchmark+for+few%2Dshot+summarization%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Computational+Li)
16. T. Zhang, F. Ladhak, E. Durmus, P. Liang, K. McKeown, and T. B. Hashimoto, “Benchmarking large language models for news summarization,” Transactions of the Association for Computational Linguistics, vol. 12, pp. 39–57, 2024. [our card](https://md.research.elamur.ai/free/en/papers/zhang-2024-benchmarking-large-language-models-news/)
17. W. Jiao, W. Wang, J.-t. Huang, X. Wang, and Z. Tu, “Is chatgpt a good translator? a preliminary study,” ArXiv preprint, vol. abs/2301.08745, 2023. [our card](https://md.research.elamur.ai/free/en/papers/jiao-2023-chatgpt-good-translator-yes-gpt/)
18. C. Qin, A. Zhang, Z. Zhang, J. Chen, M. Yasunaga, and D. Yang, “Is ChatGPT a general-purpose natural language processing task solver?” in Proc. of EMNLP. Association for Computational Linguistics, 2023, pp. 1339–1384. [our card](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/)
19. M. Collins, J. Hajic, L. Ramshaw, and C. Tillmann, “A statistical parser for Czech,” in Proc. of ACL. Association for Computational Linguistics, 1999, pp. 505–512. [our card](https://md.research.elamur.ai/free/en/papers/collins-1999-statistical-parser-czech/)
20. G. Durrett and D. Klein, “Neural CRF parsing,” in Proc. of ACL. Association for Computational Linguistics, 2015, pp. 302–312. [find](https://scholar.google.com/scholar?q=G%2E+Durrett+and+D%2E+Klein%2C+%E2%80%9CNeural+CRF+parsing%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Computational+Linguistics%2C+2015%2C+pp%2E+302%E2%80%93312%2E)
21. H. Yamada and Y. Matsumoto, “Statistical dependency analysis with support vector machines,” in Proceedings of the Eighth International Conference on Parsing Technologies, 2003, pp. 195–206. [our card](https://md.research.elamur.ai/free/en/papers/yamada-2003-statistical-dependency-analysis-support-vector/)
22. K. Sagae and A. Lavie, “A classifier-based parser with linear run-time complexity,” in Proceedings of the Ninth International Workshop on Parsing Technology. Association for Computational Linguistics, 2005, pp. 125–132. [our card](https://md.research.elamur.ai/free/en/papers/sagae-2005-classifier-based-parser-linear-run/)
23. J. Nivre and R. McDonald, “Integrating graph-based and transition-based dependency parsers,” in Proc. of ACL. Association for Computational Linguistics, 2008, pp. 950–958. [our card](https://md.research.elamur.ai/free/en/papers/nivre-2008-integrating-graph-based-transition-based/)
24. Y. Zhang and S. Clark, “Transition-based parsing of the Chinese treebank using a global discriminative model,” in Proc. of IWPT. Association for Computational Linguistics, 2009, pp. 162–171. [our card](https://md.research.elamur.ai/free/en/papers/zhang-2009-transition-based-parsing-chinese-treebank/)
25. O. Vinyals, L. Kaiser, T. Koo, S. Petrov, I. Sutskever, and G. E. Hinton, “Grammar as a foreign language,” in Proc. of NeurIPS, 2015, pp. 2773– 2781. [find](https://scholar.google.com/scholar?q=O%2E+Vinyals%2C+L%2E+Kaiser%2C+T%2E+Koo%2C+S%2E+Petrov%2C+I%2E+Sutskever%2C+and+G%2E+E%2E+Hinton%2C+%E2%80%9CGrammar+as+a+foreign+language%2C%E2%80%9D+in+Proc%2E+of+NeurIPS%2C+2015%2C+pp%2E+2773%E2%80%93+2781%2E)
26. H. Kamigaito, K. Hayashi, T. Hirao, H. Takamura, M. Okumura, and M. Nagata, “Supervised attention for sequence-to-sequence constituency parsing,” in Proc. of IJCNLP (Volume 2: Short Papers). Asian Federation of Natural Language Processing, 2017, pp. 7–12. [find](https://scholar.google.com/scholar?q=H%2E+Kamigaito%2C+K%2E+Hayashi%2C+T%2E+Hirao%2C+H%2E+Takamura%2C+M%2E+Okumura%2C+and+M%2E+Nagata%2C+%E2%80%9CSupervised+attention+for+sequence%2Dto%2Dsequence+constituency+parsing%2C%E2%80%9D+in+Proc%2E+of+IJCNLP+%28Volume+2%3A+Short+Papers%29%2E+Asian+Fed)
27. L. Liu, M. Zhu, and S. Shi, “Improving sequence-to-sequence constituency parsing,” in Proc. of AAAI. AAAI Press, 2018, pp. 4873–4880. [find](https://scholar.google.com/scholar?q=L%2E+Liu%2C+M%2E+Zhu%2C+and+S%2E+Shi%2C+%E2%80%9CImproving+sequence%2Dto%2Dsequence+constituency+parsing%2C%E2%80%9D+in+Proc%2E+of+AAAI%2E+AAAI+Press%2C+2018%2C+pp%2E+4873%E2%80%934880%2E)
28. N. Kitaev and D. Klein, “Constituency parsing with a self-attentive encoder,” in Proc. of ACL. Association for Computational Linguistics, 2018, pp. 2676–2686. [our card](https://md.research.elamur.ai/free/en/papers/kitaev-2018-constituency-parsing-self-attentive-encoder/)
29. Y. Zhang, H. Zhou, and Z. Li, “Fast and accurate neural CRF constituency parsing,” in Proc. of IJCAI. ijcai.org, 2020, pp. 4046–4053. [find](https://scholar.google.com/scholar?q=Y%2E+Zhang%2C+H%2E+Zhou%2C+and+Z%2E+Li%2C+%E2%80%9CFast+and+accurate+neural+CRF+constituency+parsing%2C%E2%80%9D+in+Proc%2E+of+IJCAI%2E+ijcai%2Eorg%2C+2020%2C+pp%2E+4046%E2%80%934053%2E)
30. C. Dyer, A. Kuncoro, M. Ballesteros, and N. A. Smith, “Recurrent neural network grammars,” in Proc. of NAACL-HLT. Association for Computational Linguistics, 2016, pp. 199–209. [find](https://scholar.google.com/scholar?q=C%2E+Dyer%2C+A%2E+Kuncoro%2C+M%2E+Ballesteros%2C+and+N%2E+A%2E+Smith%2C+%E2%80%9CRecurrent+neural+network+grammars%2C%E2%80%9D+in+Proc%2E+of+NAACL%2DHLT%2E+Association+for+Computational+Linguistics%2C+2016%2C+pp%2E+199%E2%80%93209%2E)
31. J. Cross and L. Huang, “Span-based constituency parsing with a structure-label system and provably optimal dynamic oracles,” in Proc. of EMNLP. Association for Computational Linguistics, 2016, pp. 1–11. [our card](https://md.research.elamur.ai/free/en/papers/cross-2016-span-based-constituency-parsing-structure/)
32. J. Liu and Y. Zhang, “In-order transition-based constituent parsing,” Transactions of the Association for Computational Linguistics, vol. 5, pp. 413–424, 2017. [our card](https://md.research.elamur.ai/free/en/papers/liu-2017-order-transition-based-constituent-parsing/)
33. D. Fernández-González and C. Gómez-Rodríguez, “Enriched in-order linearization for faster sequence-to-sequence constituent parsing,” in Proc. of ACL. Association for Computational Linguistics, 2020, pp. 4092–4099. [find](https://scholar.google.com/scholar?q=D%2E+Fern%C3%A1ndez%2DGonz%C3%A1lez+and+C%2E+G%C3%B3mez%2DRodr%C3%ADguez%2C+%E2%80%9CEnriched+in%2Dorder+linearization+for+faster+sequence%2Dto%2Dsequence+constituent+parsing%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Computational+Linguistics%2C+2020%2C+p)
34. T.-T. Nguyen, X.-P. Nguyen, S. Joty, and X. Li, “A conditional splitting framework for efficient constituency parsing,” in Proc. of ACL. Association for Computational Linguistics, 2021, pp. 5795–5807. [find](https://scholar.google.com/scholar?q=T%2E%2DT%2E+Nguyen%2C+X%2E%2DP%2E+Nguyen%2C+S%2E+Joty%2C+and+X%2E+Li%2C+%E2%80%9CA+conditional+splitting+framework+for+efficient+constituency+parsing%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Computational+Linguistics%2C+2021%2C+pp%2E+5795%E2%80%935807%2E)
35. S. Yang and K. Tu, “Bottom-up constituency parsing and nested named entity recognition with pointer networks,” in Proc. of ACL. Association for Computational Linguistics, 2022, pp. 2403–2416. [find](https://scholar.google.com/scholar?q=S%2E+Yang+and+K%2E+Tu%2C+%E2%80%9CBottom%2Dup+constituency+parsing+and+nested+named+entity+recognition+with+pointer+networks%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Computational+Linguistics%2C+2022%2C+pp%2E+2403%E2%80%932416%2E)
36. Y. Tian, F. Xia, and Y. Song, “Large language models are no longer shallow parsers,” in Proc. of ACL. Association for Computational Linguistics, 2024, pp. 7131–7142. [find](https://scholar.google.com/scholar?q=Y%2E+Tian%2C+F%2E+Xia%2C+and+Y%2E+Song%2C+%E2%80%9CLarge+language+models+are+no+longer+shallow+parsers%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Computational+Linguistics%2C+2024%2C+pp%2E+7131%E2%80%937142%2E)
37. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Ber… [our card](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
38. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., “Palm: Scaling language modeling with pathways,” J. Mach. Learn. Res., vol. 24, pp. 240:1–240:113, 2023. [our card](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
39. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., “Llama: Open and efficient foundation language models,” ArXiv preprint, vol. abs/2302.13971, 2023. [our card](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/)
40. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019. [find](https://scholar.google.com/scholar?q=A%2E+Radford%2C+J%2E+Wu%2C+R%2E+Child%2C+D%2E+Luan%2C+D%2E+Amodei%2C+I%2E+Sutskever+et+al%2E%2C+%E2%80%9CLanguage+models+are+unsupervised+multitask+learners%2C%E2%80%9D+OpenAI+blog%2C+vol%2E+1%2C+no%2E+8%2C+p%2E+9%2C+2019%2E)
41. V. Sanh, A. Webson, C. Raffel, S. H. Bach, L. Sutawika, Z. Alyafeai, A. Chaffin, A. Stiegler, A. Raja, M. Dey, M. S. Bari, C. Xu, U. Thakker, S. S. Sharma, E. Szczechla, T. Kim, G. Chhablani, N. V. Nayak, D. Datta, J. Chang, M. T. Jiang, H. Wang, M. Manica, S. Shen, Z. X. Yong, H. Pandey, R. Bawden,… [our card](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
42. A. Webson and E. Pavlick, “Do prompt-based models really understand the meaning of their prompts?” in Proc. of NAACL. Association for Computational Linguistics, 2022, pp. 2300–2344. [our card](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/)
43. P. Liang, R. Bommasani, T. Lee, D. Tsipras, D. Soylu, M. Yasunaga, Y. Zhang, D. Narayanan, Y. Wu, A. Kumar et al., “Holistic evaluation of language models,” Trans. Mach. Learn. Res., vol. 2023, 2023. [find](https://scholar.google.com/scholar?q=P%2E+Liang%2C+R%2E+Bommasani%2C+T%2E+Lee%2C+D%2E+Tsipras%2C+D%2E+Soylu%2C+M%2E+Yasunaga%2C+Y%2E+Zhang%2C+D%2E+Narayanan%2C+Y%2E+Wu%2C+A%2E+Kumar+et+al%2E%2C+%E2%80%9CHolistic+evaluation+of+language+models%2C%E2%80%9D+Trans%2E+Mach%2E+Learn%2E+Res%2E%2C+vol%2E+2023%2C+2023%2E)
44. S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin et al., “Opt: Open pre-trained transformer language models,” ArXiv preprint, vol. abs/2205.01068, 2022. [our card](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/)
45. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. F. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback… [our card](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
46. N. Lambert, L. Castricato, L. von Werra, and A. Havrilla, “Illustrating reinforcement learning from human feedback (rlhf),” HuggingFace Blog, vol. 2022, 2022. [find](https://scholar.google.com/scholar?q=N%2E+Lambert%2C+L%2E+Castricato%2C+L%2E+von+Werra%2C+and+A%2E+Havrilla%2C+%E2%80%9CIllustrating+reinforcement+learning+from+human+feedback+%28rlhf%29%2C%E2%80%9D+HuggingFace+Blog%2C+vol%2E+2022%2C+2022%2E)
47. Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan et al., “Training a helpful and harmless assistant with reinforcement learning from human feedback,” ArXiv preprint, vol. abs/2204.05862, 2022. [find](https://scholar.google.com/scholar?q=Y%2E+Bai%2C+A%2E+Jones%2C+K%2E+Ndousse%2C+A%2E+Askell%2C+A%2E+Chen%2C+N%2E+DasSarma%2C+D%2E+Drain%2C+S%2E+Fort%2C+D%2E+Ganguli%2C+T%2E+Henighan+et+al%2E%2C+%E2%80%9CTraining+a+helpful+and+harmless+assistant+with+reinforcement+learning+from+human+feed)
48. L. Gao, J. Schulman, and J. Hilton, “Scaling laws for reward model overoptimization,” in Proc. of ICML, vol. 202. PMLR, 2023, pp. 10 835–10 866. [our card](https://md.research.elamur.ai/free/en/papers/gao-2022-scaling-laws-reward-model-overoptimization/)
49. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” in Proc. of ICLR. OpenReview.net, 2021. [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/)
50. D. Xu, W. Chen, W. Peng, C. Zhang, T. Xu, X. Zhao, X. Wu, Y. Zheng, Y. Wang, and E. Chen, “Large language models for generative information extraction: a survey,” Frontiers Comput. Sci., vol. 18, no. 6, p. 186357, 2024. [find](https://scholar.google.com/scholar?q=D%2E+Xu%2C+W%2E+Chen%2C+W%2E+Peng%2C+C%2E+Zhang%2C+T%2E+Xu%2C+X%2E+Zhao%2C+X%2E+Wu%2C+Y%2E+Zheng%2C+Y%2E+Wang%2C+and+E%2E+Chen%2C+%E2%80%9CLarge+language+models+for+generative+information+extraction%3A+a+survey%2C%E2%80%9D+Frontiers+Comput%2E+Sci%2E%2C+vol%2E+18%2C+no%2E+)
51. Z. Li, Y. Zeng, Y. Zuo, W. Ren, W. Liu, M. Su, Y. Guo, Y. Liu, X. Li, Z. Hu, L. Bai, W. Li, Y. Liu, P. Yang, X. Jin, J. Guo, and X. Cheng, “Knowcoder: Coding structured knowledge into llms for universal information extraction,” in Proc. of ACL. Association for Computational Linguistics, 2024, pp. 87… [find](https://scholar.google.com/scholar?q=Z%2E+Li%2C+Y%2E+Zeng%2C+Y%2E+Zuo%2C+W%2E+Ren%2C+W%2E+Liu%2C+M%2E+Su%2C+Y%2E+Guo%2C+Y%2E+Liu%2C+X%2E+Li%2C+Z%2E+Hu%2C+L%2E+Bai%2C+W%2E+Li%2C+Y%2E+Liu%2C+P%2E+Yang%2C+X%2E+Jin%2C+J%2E+Guo%2C+and+X%2E+Cheng%2C+%E2%80%9CKnowcoder%3A+Coding+structured+knowledge+into+llms+for+univers)
52. É. Simon, H. Olsen, H. You, S. Touileb, L. Øvrelid, and E. Velldal, “Generative approaches to event extraction: Survey and outlook,” in Proc. of FuturED. Association for Computational Linguistics, 2024, pp. 73–86. [find](https://scholar.google.com/scholar?q=%C3%89%2E+Simon%2C+H%2E+Olsen%2C+H%2E+You%2C+S%2E+Touileb%2C+L%2E+%C3%98vrelid%2C+and+E%2E+Velldal%2C+%E2%80%9CGenerative+approaches+to+event+extraction%3A+Survey+and+outlook%2C%E2%80%9D+in+Proc%2E+of+FuturED%2E+Association+for+Computational+Linguistics%2C+202)
53. Z. Meng, T. Liu, H. Zhang, K. Feng, and P. Zhao, “CEAN: Contrastive event aggregation network with LLM-based augmentation for event extraction,” in Proc. of EACL. Association for Computational Linguistics, 2024, pp. 321–333. [find](https://scholar.google.com/scholar?q=Z%2E+Meng%2C+T%2E+Liu%2C+H%2E+Zhang%2C+K%2E+Feng%2C+and+P%2E+Zhao%2C+%E2%80%9CCEAN%3A+Contrastive+event+aggregation+network+with+LLM%2Dbased+augmentation+for+event+extraction%2C%E2%80%9D+in+Proc%2E+of+EACL%2E+Association+for+Computational+Linguis)
54. Y. Zhu, X. Wang, J. Chen, S. Qiao, Y. Ou, Y. Yao, S. Deng, H. Chen, and N. Zhang, “Llms for knowledge graph construction and reasoning: recent capabilities and future opportunities,” World Wide Web (WWW), vol. 27, no. 5, p. 58, 2024. [find](https://scholar.google.com/scholar?q=Y%2E+Zhu%2C+X%2E+Wang%2C+J%2E+Chen%2C+S%2E+Qiao%2C+Y%2E+Ou%2C+Y%2E+Yao%2C+S%2E+Deng%2C+H%2E+Chen%2C+and+N%2E+Zhang%2C+%E2%80%9CLlms+for+knowledge+graph+construction+and+reasoning%3A+recent+capabilities+and+future+opportunities%2C%E2%80%9D+World+Wide+Web+%28W)
55. P. Schneider, M. Klettner, K. Jokinen, E. Simperl, and F. Matthes, “Evaluating large language models in semantic parsing for conversational question answering over knowledge graphs,” in Proc. of ICAART. SCITEPRESS, 2024, pp. 807–814. [find](https://scholar.google.com/scholar?q=P%2E+Schneider%2C+M%2E+Klettner%2C+K%2E+Jokinen%2C+E%2E+Simperl%2C+and+F%2E+Matthes%2C+%E2%80%9CEvaluating+large+language+models+in+semantic+parsing+for+conversational+question+answering+over+knowledge+graphs%2C%E2%80%9D+in+Proc%2E+of+ICAAR)
56. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” in Proc. of ICLR. OpenReview.net, 2022. [our card](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/)
57. S. Yang, L. Cui, R. Ning, D. Wu, and Y. Zhang, “Challenges to open-domain constituency parsing,” in Findings of ACL: ACL 2022. Association for Computational Linguistics, 2022, pp. 112–127. [find](https://scholar.google.com/scholar?q=S%2E+Yang%2C+L%2E+Cui%2C+R%2E+Ning%2C+D%2E+Wu%2C+and+Y%2E+Zhang%2C+%E2%80%9CChallenges+to+open%2Ddomain+constituency+parsing%2C%E2%80%9D+in+Findings+of+ACL%3A+ACL+2022%2E+Association+for+Computational+Linguistics%2C+2022%2C+pp%2E+112%E2%80%93127%2E)
58. Y. Tian, Y. Song, F. Xia, and T. Zhang, “Improving constituency parsing with span attention,” in Findings of ACL: EMNLP 2020. Association for Computational Linguistics, 2020, pp. 1691–1703. [find](https://scholar.google.com/scholar?q=Y%2E+Tian%2C+Y%2E+Song%2C+F%2E+Xia%2C+and+T%2E+Zhang%2C+%E2%80%9CImproving+constituency+parsing+with+span+attention%2C%E2%80%9D+in+Findings+of+ACL%3A+EMNLP+2020%2E+Association+for+Computational+Linguistics%2C+2020%2C+pp%2E+1691%E2%80%931703%2E)
59. K. Yang and J. Deng, “Strongly incremental constituency parsing with graph neural networks,” in Proc. of NeurIPS, 2020, pp. 21 687–21 698. [find](https://scholar.google.com/scholar?q=K%2E+Yang+and+J%2E+Deng%2C+%E2%80%9CStrongly+incremental+constituency+parsing+with+graph+neural+networks%2C%E2%80%9D+in+Proc%2E+of+NeurIPS%2C+2020%2C+pp%2E+21+687%E2%80%9321+698%2E)
60. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. of NeurIPS, 2017, pp. 5998–6008. [our card](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
61. S. Cao, N. Kitaev, and D. Klein, “Unsupervised parsing via constituency tests,” in Proc. of EMNLP. Association for Computational Linguistics, 2020, pp. 4798–4808. [find](https://scholar.google.com/scholar?q=S%2E+Cao%2C+N%2E+Kitaev%2C+and+D%2E+Klein%2C+%E2%80%9CUnsupervised+parsing+via+constituency+tests%2C%E2%80%9D+in+Proc%2E+of+EMNLP%2E+Association+for+Computational+Linguistics%2C+2020%2C+pp%2E+4798%E2%80%934808%2E)
62. R. OpenAI, “Gpt-4 technical report,” arXiv, pp. 2303–08 774, 2023. [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
63. W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez, I. Stoica, and E. P. Xing, “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” LMSYS blog, vol. 2023, 2023. [find](https://scholar.google.com/scholar?q=W%2E%2DL%2E+Chiang%2C+Z%2E+Li%2C+Z%2E+Lin%2C+Y%2E+Sheng%2C+Z%2E+Wu%2C+H%2E+Zhang%2C+L%2E+Zheng%2C+S%2E+Zhuang%2C+Y%2E+Zhuang%2C+J%2E+E%2E+Gonzalez%2C+I%2E+Stoica%2C+and+E%2E+P%2E+Xing%2C+%E2%80%9CVicuna%3A+An+open%2Dsource+chatbot+impressing+gpt%2D4+with+90%25%2A+chatgpt+qu)
64. L. Cui, S. Yang, and Y. Zhang, “Investigating non-local features for neural constituency parsing,” in Proc. of ACL. Association for Computational Linguistics, 2022, pp. 2065–2075. [find](https://scholar.google.com/scholar?q=L%2E+Cui%2C+S%2E+Yang%2C+and+Y%2E+Zhang%2C+%E2%80%9CInvestigating+non%2Dlocal+features+for+neural+constituency+parsing%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Computational+Linguistics%2C+2022%2C+pp%2E+2065%E2%80%932075%2E)
65. C. Xu, D. Guo, N. Duan, and J. McAuley, “Baize: An open-source chat model with parameter-efficient tuning on self-chat data,” in Proc. of EMNLP. Association for Computational Linguistics, 2023, pp. 6268– 6278. [our card](https://md.research.elamur.ai/free/en/papers/xu-2023-baize-open-source-chat-model/)
66. J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le, “Finetuned language models are zero-shot learners,” in Proc. of ICLR. OpenReview.net, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Wei%2C+M%2E+Bosma%2C+V%2E+Y%2E+Zhao%2C+K%2E+Guu%2C+A%2E+W%2E+Yu%2C+B%2E+Lester%2C+N%2E+Du%2C+A%2E+M%2E+Dai%2C+and+Q%2E+V%2E+Le%2C+%E2%80%9CFinetuned+language+models+are+zero%2Dshot+learners%2C%E2%80%9D+in+Proc%2E+of+ICLR%2E+OpenReview%2Enet%2C+2022%2E)
67. Y. Ge, W. Hua, K. Mei, J. Ji, J. Tan, S. Xu, Z. Li, and Y. Zhang, “Openagi: When LLM meets domain experts,” in Proc. of NeurIPS, 2023, pp. 5539–5568. [find](https://scholar.google.com/scholar?q=Y%2E+Ge%2C+W%2E+Hua%2C+K%2E+Mei%2C+J%2E+Ji%2C+J%2E+Tan%2C+S%2E+Xu%2C+Z%2E+Li%2C+and+Y%2E+Zhang%2C+%E2%80%9COpenagi%3A+When+LLM+meets+domain+experts%2C%E2%80%9D+in+Proc%2E+of+NeurIPS%2C+2023%2C+pp%2E+5539%E2%80%935568%2E)
68. C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu, S. Zhang, G. Ghosh, M. Lewis, L. Zettlemoyer, and O. Levy, “LIMA: less is more for alignment,” in Proc. of NeurIPS, 2023, pp. 55 006–55 021. [find](https://scholar.google.com/scholar?q=C%2E+Zhou%2C+P%2E+Liu%2C+P%2E+Xu%2C+S%2E+Iyer%2C+J%2E+Sun%2C+Y%2E+Mao%2C+X%2E+Ma%2C+A%2E+Efrat%2C+P%2E+Yu%2C+L%2E+Yu%2C+S%2E+Zhang%2C+G%2E+Ghosh%2C+M%2E+Lewis%2C+L%2E+Zettlemoyer%2C+and+O%2E+Levy%2C+%E2%80%9CLIMA%3A+less+is+more+for+alignment%2C%E2%80%9D+in+Proc%2E+of+NeurIPS%2C+2023)
69. S. Gururangan, A. Marasovic, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, and N. A. Smith, “Don’t stop pretraining: Adapt language models to domains and tasks,” in Proc. of ACL. Association for Computational Linguistics, 2020, pp. 8342–8360. [find](https://scholar.google.com/scholar?q=S%2E+Gururangan%2C+A%2E+Marasovic%2C+S%2E+Swayamdipta%2C+K%2E+Lo%2C+I%2E+Beltagy%2C+D%2E+Downey%2C+and+N%2E+A%2E+Smith%2C+%E2%80%9CDon%E2%80%99t+stop+pretraining%3A+Adapt+language+models+to+domains+and+tasks%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Compu)
70. W. Lu, R. K. Luu, and M. J. Buehler, “Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities,” npj Computational Materials, vol. 11, no. 1, p. 84, 2025. [find](https://scholar.google.com/scholar?q=W%2E+Lu%2C+R%2E+K%2E+Luu%2C+and+M%2E+J%2E+Buehler%2C+%E2%80%9CFine%2Dtuning+large+language+models+for+domain+adaptation%3A+Exploration+of+training+strategies%2C+scaling%2C+model+merging+and+synergistic+capabilities%2C%E2%80%9D+npj+Computation)
71. F. C. N. Pereira and D. H. D. Warren, “Parsing as deduction,” in Proc. of ACL. Association for Computational Linguistics, 1983, pp. 137–144. [find](https://scholar.google.com/scholar?q=F%2E+C%2E+N%2E+Pereira+and+D%2E+H%2E+D%2E+Warren%2C+%E2%80%9CParsing+as+deduction%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Computational+Linguistics%2C+1983%2C+pp%2E+137%E2%80%93144%2E)
72. J. Maynez, S. Narayan, B. Bohnet, and R. McDonald, “On faithfulness and factuality in abstractive summarization,” in Proc. of ACL. Association for Computational Linguistics, 2020, pp. 1906–1919. [find](https://scholar.google.com/scholar?q=J%2E+Maynez%2C+S%2E+Narayan%2C+B%2E+Bohnet%2C+and+R%2E+McDonald%2C+%E2%80%9COn+faithfulness+and+factuality+in+abstractive+summarization%2C%E2%80%9D+in+Proc%2E+of+ACL%2E+Association+for+Computational+Linguistics%2C+2020%2C+pp%2E+1906%E2%80%931919%2E)
73. Y. Bang, S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung, Q. V. Do, Y. Xu, and P. Fung, “A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity,” in Proc. of IJCNLP. Association for Computational Linguistics, 202… [our card](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/)
74. A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi, “The curious case of neural text degeneration,” in Proc. of ICLR. OpenReview.net, 2020. [find](https://scholar.google.com/scholar?q=A%2E+Holtzman%2C+J%2E+Buys%2C+L%2E+Du%2C+M%2E+Forbes%2C+and+Y%2E+Choi%2C+%E2%80%9CThe+curious+case+of+neural+text+degeneration%2C%E2%80%9D+in+Proc%2E+of+ICLR%2E+OpenReview%2Enet%2C+2020%2E)
75. N. Kornell, M. J. Hays, and R. A. Bjork, “Unsuccessful retrieval attempts enhance subsequent learning.” Journal of experimental psychology. Learning, memory, and cognition, vol. 35 (4), pp. 989–98, 2009. [find](https://scholar.google.com/scholar?q=N%2E+Kornell%2C+M%2E+J%2E+Hays%2C+and+R%2E+A%2E+Bjork%2C+%E2%80%9CUnsuccessful+retrieval+attempts+enhance+subsequent+learning%2E%E2%80%9D+Journal+of+experimental+psychology%2E+Learning%2C+memory%2C+and+cognition%2C+vol%2E+35+%284%29%2C+pp%2E+989%E2%80%9398%2C+20)
76. J. Metcalfe, “Learning from errors,” Annual review of psychology, vol. 68, p. 465—489, 2017. [find](https://scholar.google.com/scholar?q=J%2E+Metcalfe%2C+%E2%80%9CLearning+from+errors%2C%E2%80%9D+Annual+review+of+psychology%2C+vol%2E+68%2C+p%2E+465%E2%80%94489%2C+2017%2E)