---
title: "Bidirectional LMs are Better Knowledge Memorizers? A"
description: "The authors build WikiDYK from Wikipedia's \"Did You Know...\" section, where editors review and post… 46 references, cited by 4 papers here."
authors: ["Yuwei Zhang", "Wenhao Yu", "Shangbin Feng", "Yifan Zhu", "Letian Peng", "Jayanth Srinivasa", "Gaowen Liu", "Jingbo Shang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.771/"
doi: "10.18653/v1/2026.acl-long.771"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-bidirectional-lms-are-better-knowledge/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-bidirectional-lms-are-better-knowledge/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) / Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection

# Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection

**Authors:** [Yuwei Zhang](https://md.research.elamur.ai/free/en/authors/yuwei-zhang/), [Wenhao Yu](https://md.research.elamur.ai/free/en/authors/wenhao-yu/), [Shangbin Feng](https://md.research.elamur.ai/free/en/authors/shangbin-feng/), [Yifan Zhu](https://md.research.elamur.ai/free/en/authors/yifan-zhu/), [Letian Peng](https://md.research.elamur.ai/free/en/authors/letian-peng/), [Jayanth Srinivasa](https://md.research.elamur.ai/free/en/authors/jayanth-srinivasa/), [Gaowen Liu](https://md.research.elamur.ai/free/en/authors/gaowen-liu/) and [Jingbo Shang](https://md.research.elamur.ai/free/en/authors/jingbo-shang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.771/) · [PDF](https://aclanthology.org/2026.acl-long.771.pdf) · [DOI 10.18653/v1/2026.acl-long.771](https://doi.org/10.18653/v1/2026.acl-long.771) · [OpenAlex](https://openalex.org/W4417300535)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 4 in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks how well language models can absorb new facts after pre-training. Earlier knowledge-injection studies mostly relied on synthetic data drawn from noisy Wikipedia snapshots, and their questions were often poorly specified.

The authors build WikiDYK from Wikipedia's "Did You Know..." section, where editors review and post about ten new facts daily from new or substantially expanded articles. A lightweight prompting pipeline turns facts from January 2022 to April 2025 into question–answer pairs of five types, from simple recall to multi-hop reasoning. The benchmark can grow automatically as new entries appear.

Under continued pre-training, small bidirectional models (BiLMs) memorize injected facts far better than larger, newer causal models (CLMs), even with matched objectives and training amounts. To offset the limited size of BiLMs, the authors route queries through a scope classifier to an ensemble of BiLMs that act as external knowledge stores for an LLM.

The results challenge the default use of causal architectures for storing knowledge and supply a realistic, self-updating testbed for this research.

## Key points
- **WikiDYK size and design:** 12,290 facts and 77,180 questions. Question types are Reliability, Generality, Paraphrase, Portability and Locality, evaluated with Match and F1.
- **Novelty check:** before injection, off-the-shelf models score close to chance on the new facts but much higher on Locality questions about pre-training knowledge.
- **CLMs vs. BiLMs:** CLMs reach about 23% lower Reliability accuracy than BiLMs (e.g., Flan-T5). The authors attribute this to CLMs seeing only left-side context during training.
- **Ensemble framework:** clustering facts, training one BiLM per cluster, and routing queries with a scope classifier raises Reliability by up to 29.1%.
  - In ablations with Flan-T5-220M, semantic clustering beats temporal clustering.
  - With a perfect (oracle) classifier, the two clustering methods perform about equally, so the classifier is the main bottleneck.
- **Stated limitation:** the BiLM advantage is shown only empirically, using existing pretrained checkpoints rather than models pre-trained under identical data and hyperparameters. Code is available at github.com/zhang-yu-wei/WikiDYK.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 353 works · 200,043 citations
- [University of Notre Dame](https://md.research.elamur.ai/free/en/universities/university-of-notre-dame/) — University, Notre Dame, United States · 51 works · 36,717 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works

## Cite

Cite as: Yuwei Zhang, Wenhao Yu, Shangbin Feng, Yifan Zhu, Letian Peng, Jayanth Srinivasa, Gaowen Liu and Jingbo Shang. 2026. Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026bidirectional,
  title = {{Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection}},
  author = {Yuwei Zhang, Wenhao Yu, Shangbin Feng, Yifan Zhu, Letian Peng, Jayanth Srinivasa, Gaowen Liu and Jingbo Shang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.771},
  url = {https://doi.org/10.18653/v1/2026.acl-long.771}
}
```

- [← previous Beyond Majority Voting: Towards Fine-grained and More Reliable Reward Signal for Test-Time Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/wang-2026-beyond-majority-voting-fine-grained/)
- [next → BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generation](https://md.research.elamur.ai/free/en/papers/pai-2026-billy-steering-large-language-models/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/)

## References

46 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. 2023. The reversal curse: Llms trained on" a is b" fail to learn" b is a". arXiv preprint arXiv:2309.12288. [find](https://scholar.google.com/scholar?q=Lukas+Berglund%2C+Meg+Tong%2C+Max+Kaufmann%2C+Mikita+Balesni%2C+Asa+Cooper+Stickland%2C+Tomasz+Korbak%2C+and+Owain+Evans%2E+2023%2E+The+reversal+curse%3A+Llms+trained+on%22+a+is+b%22+fail+to+learn%22+b+is+a%22%2E+arXiv+preprint+)
2. Hoyeon Chang, Jinho Park, Seonghyeon Ye, Sohee Yang, Youngkyung Seo, Du-Seong Chang, and Minjoon Seo. 2024. How do large language models acquire factual knowledge during pretraining? In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Hoyeon+Chang%2C+Jinho+Park%2C+Seonghyeon+Ye%2C+Sohee+Yang%2C+Youngkyung+Seo%2C+Du%2DSeong+Chang%2C+and+Minjoon+Seo%2E+2024%2E+How+do+large+language+models+acquire+factual+knowledge+during+pretraining%3F+In+The+Thirty%2Deig)
3. Howard Chen, Jiayi Geng, Adithya Bhaskar, Dan Friedman, and Danqi Chen. 2024. Continual memorization of factoids in large language models. arXiv preprint arXiv:2411.07175. [find](https://scholar.google.com/scholar?q=Howard+Chen%2C+Jiayi+Geng%2C+Adithya+Bhaskar%2C+Dan+Friedman%2C+and+Danqi+Chen%2E+2024%2E+Continual+memorization+of+factoids+in+large+language+models%2E+arXiv+preprint+arXiv%3A2411%2E07175%2E)
4. Jeffrey Cheng, Marc Marone, Orion Weller, Dawn Lawrie, Daniel Khashabi, and Benjamin Van Durme. 2024. Dated data: Tracing knowledge cutoffs in large language models. arXiv preprint arXiv:2403.12958. [find](https://scholar.google.com/scholar?q=Jeffrey+Cheng%2C+Marc+Marone%2C+Orion+Weller%2C+Dawn+Lawrie%2C+Daniel+Khashabi%2C+and+Benjamin+Van+Durme%2E+2024%2E+Dated+data%3A+Tracing+knowledge+cutoffs+in+large+language+models%2E+arXiv+preprint+arXiv%3A2403%2E12958%2E)
5. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, and 1 others. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Yunxuan+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+and+1+others%2E+2024%2E+Scaling+instruction%2Dfinetuned+language+mo)
6. Shangbin Feng, Weijia Shi, Yuyang Bai, Vidhisha Balachandran, Tianxing He, and Yulia Tsvetkov. 2023. [Knowledge card: Filling llms’ knowledge gaps with plug-in specialized language models](https://md.research.elamur.ai/free/en/papers/feng-2023-knowledge-card-filling-llms-knowledge/). arXiv preprint arXiv:2305.09955.
7. Peng Fu, Yiming Zhang, Haobo Wang, Weikang Qiu, and Junbo Zhao. 2023. Revisiting the knowledge injection frameworks. arXiv preprint arXiv:2311.01150. [find](https://scholar.google.com/scholar?q=Peng+Fu%2C+Yiming+Zhang%2C+Haobo+Wang%2C+Weikang+Qiu%2C+and+Junbo+Zhao%2E+2023%2E+Revisiting+the+knowledge+injection+frameworks%2E+arXiv+preprint+arXiv%3A2311%2E01150%2E)
8. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
9. Roger B. Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamile Lukosiute, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R. Bowman. 2023. Studying large language model general… [find](https://scholar.google.com/scholar?q=Roger+B%2E+Grosse%2C+Juhan+Bae%2C+Cem+Anil%2C+Nelson+Elhage%2C+Alex+Tamkin%2C+Amirhossein+Tajdini%2C+Benoit+Steiner%2C+Dustin+Li%2C+Esin+Durmus%2C+Ethan+Perez%2C+Evan+Hubinger%2C+Kamile+Lukosiute%2C+Karina+Nguyen%2C+Nicholas+Jos)
10. Anshita Gupta, Debanjan Mondal, Akshay Krishna Sheshadri, Wenlong Zhao, Xiang Lorraine Li, Sarah Wiegreffe, and Niket Tandon. 2023. Editing common sense in transformers. arXiv preprint arXiv:2305.14956. [find](https://scholar.google.com/scholar?q=Anshita+Gupta%2C+Debanjan+Mondal%2C+Akshay+Krishna+Sheshadri%2C+Wenlong+Zhao%2C+Xiang+Lorraine+Li%2C+Sarah+Wiegreffe%2C+and+Niket+Tandon%2E+2023%2E+Editing+common+sense+in+transformers%2E+arXiv+preprint+arXiv%3A2305%2E1495)
11. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. [Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/). arXiv preprint arXiv:2111.09543.
12. Jason Hoelscher-Obermaier, Julia Persson, Esben Kran, Ioannis Konstas, and Fazl Barez. 2023. Detecting edit failures in large language models: An improved specificity benchmark. arXiv preprint arXiv:2305.17553. [find](https://scholar.google.com/scholar?q=Jason+Hoelscher%2DObermaier%2C+Julia+Persson%2C+Esben+Kran%2C+Ioannis+Konstas%2C+and+Fazl+Barez%2E+2023%2E+Detecting+edit+failures+in+large+language+models%3A+An+improved+specificity+benchmark%2E+arXiv+preprint+arXiv%3A2)
13. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). ICLR, 1(2):3.
14. Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, and Minjoon Seo. 2022. Temporalwiki: A lifelong benchmark for training and evaluating ever-evolving language models. arXiv preprint arXiv:2204.14211. [find](https://scholar.google.com/scholar?q=Joel+Jang%2C+Seonghyeon+Ye%2C+Changho+Lee%2C+Sohee+Yang%2C+Joongbo+Shin%2C+Janghoon+Han%2C+Gyeonghun+Kim%2C+and+Minjoon+Seo%2E+2022%2E+Temporalwiki%3A+A+lifelong+benchmark+for+training+and+evaluating+ever%2Devolving+langua)
15. Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Stanley Jungkyu Choi, and Minjoon Seo. 2021. Towards continual knowledge learning of language models. arXiv preprint arXiv:2110.03215. [find](https://scholar.google.com/scholar?q=Joel+Jang%2C+Seonghyeon+Ye%2C+Sohee+Yang%2C+Joongbo+Shin%2C+Janghoon+Han%2C+Gyeonghun+Kim%2C+Stanley+Jungkyu+Choi%2C+and+Minjoon+Seo%2E+2021%2E+Towards+continual+knowledge+learning+of+language+models%2E+arXiv+preprint+ar)
16. Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir Radev, Noah A Smith, Yejin Choi, Kentaro Inui, and 1 others. 2023. Real-time qa: What’s the answer right now? Advances in neural information processing systems, 36:49025– 49043. [find](https://scholar.google.com/scholar?q=Jungo+Kasai%2C+Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Akari+Asai%2C+Xinyan+Yu%2C+Dragomir+Radev%2C+Noah+A+Smith%2C+Yejin+Choi%2C+Kentaro+Inui%2C+and+1+others%2E+2023%2E+Real%2Dtime+qa%3A+What%E2%80%99s+the+answer+right+now%3F+Advances+in)
17. Yujin Kim, Jaehong Yoon, Seonghyeon Ye, Sangmin Bae, Namgyu Ho, Sung Ju Hwang, and Se-Young Yun. 2023. Carpe diem: On the evaluation of world knowledge in lifelong language models. arXiv preprint arXiv:2311.08106. [find](https://scholar.google.com/scholar?q=Yujin+Kim%2C+Jaehong+Yoon%2C+Seonghyeon+Ye%2C+Sangmin+Bae%2C+Namgyu+Ho%2C+Sung+Ju+Hwang%2C+and+Se%2DYoung+Yun%2E+2023%2E+Carpe+diem%3A+On+the+evaluation+of+world+knowledge+in+lifelong+language+models%2E+arXiv+preprint+arXi)
18. Dawid J Kopiczko, Tijmen Blankevoort, and Yuki M Asano. 2024. Bitune: Bidirectional instruction-tuning. arXiv preprint arXiv:2405.14862. — [arXiv:2405.14862](https://arxiv.org/abs/2405.14862) [find](https://scholar.google.com/scholar?q=Dawid+J+Kopiczko%2C+Tijmen+Blankevoort%2C+and+Yuki+M+Asano%2E+2024%2E+Bitune%3A+Bidirectional+instruction%2Dtuning%2E+arXiv+preprint+arXiv%3A2405%2E14862%2E+%E2%80%94+%5BarXiv%3A2405%2E14862%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2405%2E14862%29)
19. Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. 2022b. Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229. [find](https://scholar.google.com/scholar?q=Kevin+Meng%2C+Arnab+Sen+Sharma%2C+Alex+Andonian%2C+Yonatan+Belinkov%2C+and+David+Bau%2E+2022b%2E+Mass%2Dediting+memory+in+a+transformer%2E+arXiv+preprint+arXiv%3A2210%2E07229%2E)
20. Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. 2022. Memory-based model editing at scale. In International Conference on Machine Learning, pages 15817–15831. PMLR. [find](https://scholar.google.com/scholar?q=Eric+Mitchell%2C+Charles+Lin%2C+Antoine+Bosselut%2C+Christopher+D+Manning%2C+and+Chelsea+Finn%2E+2022%2E+Memory%2Dbased+model+editing+at+scale%2E+In+International+Conference+on+Machine+Learning%2C+pages+15817%E2%80%9315831%2E+PM)
21. Yasumasa Onoe, Michael JQ Zhang, Shankar Padmanabhan, Greg Durrett, and Eunsol Choi. 2023. Can lms learn new entities from descriptions? challenges in propagating injected knowledge. arXiv preprint arXiv:2305.01651. [find](https://scholar.google.com/scholar?q=Yasumasa+Onoe%2C+Michael+JQ+Zhang%2C+Shankar+Padmanabhan%2C+Greg+Durrett%2C+and+Eunsol+Choi%2E+2023%2E+Can+lms+learn+new+entities+from+descriptions%3F+challenges+in+propagating+injected+knowledge%2E+arXiv+preprint+ar)
22. Aochong Oliver Li and Tanya Goyal. 2025. Memorization vs. reasoning: Updating llms with new knowledge. arXiv preprint arXiv:2504.12523. [find](https://scholar.google.com/scholar?q=Aochong+Oliver+Li+and+Tanya+Goyal%2E+2025%2E+Memorization+vs%2E+reasoning%3A+Updating+llms+with+new+knowledge%2E+arXiv+preprint+arXiv%3A2504%2E12523%2E)
23. Oded Ovadia, Menachem Brief, Moshik Mishaeli, and Oren Elisha. 2023. [Fine-tuning or retrieval? comparing knowledge injection in llms](https://md.research.elamur.ai/free/en/papers/ovadia-2024-fine-tuning-or-retrieval-comparing/). arXiv preprint arXiv:2312.05934.
24. Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A Smith, and Luke Zettlemoyer. 2022. Branch-train-merge: Embarrassingly parallel training of expert language models. arXiv preprint arXiv:2208.03306. [find](https://scholar.google.com/scholar?q=Margaret+Li%2C+Suchin+Gururangan%2C+Tim+Dettmers%2C+Mike+Lewis%2C+Tim+Althoff%2C+Noah+A+Smith%2C+and+Luke+Zettlemoyer%2E+2022%2E+Branch%2Dtrain%2Dmerge%3A+Embarrassingly+parallel+training+of+expert+language+models%2E+arXiv+p)
25. Letian Peng, Chenyang An, Shibo Hao, Chengyu Dong, and Jingbo Shang. 2025. Linear correlation in lm’s compositional generalization and hallucination. CoRR, abs/2502.04520. — [arXiv:2502.04520](https://arxiv.org/abs/2502.04520) [find](https://scholar.google.com/scholar?q=Letian+Peng%2C+Chenyang+An%2C+Shibo+Hao%2C+Chengyu+Dong%2C+and+Jingbo+Shang%2E+2025%2E+Linear+correlation+in+lm%E2%80%99s+compositional+generalization+and+hallucination%2E+CoRR%2C+abs%2F2502%2E04520%2E+%E2%80%94+%5BarXiv%3A2502%2E04520%5D%28https%3A%2F)
26. Adam Liska, Tomas Kocisky, Elena Gribovskaya, Tayfun Terzi, Eren Sezener, Devang Agrawal, D’Autume Cyprien De Masson, Tim Scholtes, Manzil Zaheer, Susannah Young, and 1 others. 2022. Streamingqa: A benchmark for adaptation to new knowledge over time in question answering models. In International Con… [find](https://scholar.google.com/scholar?q=Adam+Liska%2C+Tomas+Kocisky%2C+Elena+Gribovskaya%2C+Tayfun+Terzi%2C+Eren+Sezener%2C+Devang+Agrawal%2C+D%E2%80%99Autume+Cyprien+De+Masson%2C+Tim+Scholtes%2C+Manzil+Zaheer%2C+Susannah+Young%2C+and+1+others%2E+2022%2E+Streamingqa%3A+A+be)
27. Jiahua Liu, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2019a. Xqa: A cross-lingual open-domain question answering dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2358–2368. [find](https://scholar.google.com/scholar?q=Jiahua+Liu%2C+Yankai+Lin%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2019a%2E+Xqa%3A+A+cross%2Dlingual+open%2Ddomain+question+answering+dataset%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational)
28. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692.
29. Jun-Yu Ma, Jia-Chen Gu, Zhen-Hua Ling, Quan Liu, and Cong Liu. 2023. Untying the reversal curse via bidirectional language model editing. arXiv preprint arXiv:2310.10322. — [arXiv:2310.10322](https://arxiv.org/abs/2310.10322) [find](https://scholar.google.com/scholar?q=Jun%2DYu+Ma%2C+Jia%2DChen+Gu%2C+Zhen%2DHua+Ling%2C+Quan+Liu%2C+and+Cong+Liu%2E+2023%2E+Untying+the+reversal+curse+via+bidirectional+language+model+editing%2E+arXiv+preprint+arXiv%3A2310%2E10322%2E+%E2%80%94+%5BarXiv%3A2310%2E10322%5D%28https%3A%2F%2F)
30. Nick Mecklenburg, Yiyou Lin, Xiaoxiao Li, Daniel Holstein, Leonardo Nunes, Sara Malvar, Bruno Silva, Ranveer Chandra, Vijay Aski, Pavan Kumar Reddy Yannam, and 1 others. 2024. Injecting new knowledge into large language models via supervised fine-tuning. arXiv preprint arXiv:2404.00213. — [arXiv:2404.00213](https://arxiv.org/abs/2404.00213) [find](https://scholar.google.com/scholar?q=Nick+Mecklenburg%2C+Yiyou+Lin%2C+Xiaoxiao+Li%2C+Daniel+Holstein%2C+Leonardo+Nunes%2C+Sara+Malvar%2C+Bruno+Silva%2C+Ranveer+Chandra%2C+Vijay+Aski%2C+Pavan+Kumar+Reddy+Yannam%2C+and+1+others%2E+2024%2E+Injecting+new+knowledge+)
31. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022a. Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems, 36. ArXiv:2202.05262. [find](https://scholar.google.com/scholar?q=Kevin+Meng%2C+David+Bau%2C+Alex+Andonian%2C+and+Yonatan+Belinkov%2E+2022a%2E+Locating+and+editing+factual+associations+in+GPT%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E+ArXiv%3A2202%2E05262%2E)
32. Abhay Prakash, Manoj Kumar Chinnakotla, Dhaval Patel, and Puneet Garg. 2015. Did you know?-mining interesting trivia for entities from wikipedia. In IJ-CAI, pages 3164–3170. [find](https://scholar.google.com/scholar?q=Abhay+Prakash%2C+Manoj+Kumar+Chinnakotla%2C+Dhaval+Patel%2C+and+Puneet+Garg%2E+2015%2E+Did+you+know%3F%2Dmining+interesting+trivia+for+entities+from+wikipedia%2E+In+IJ%2DCAI%2C+pages+3164%E2%80%933170%2E)
33. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67.
34. Yi Ren and Danica J. Sutherland. 2025. Learning dynamics of LLM finetuning. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. [find](https://scholar.google.com/scholar?q=Yi+Ren+and+Danica+J%2E+Sutherland%2E+2025%2E+Learning+dynamics+of+LLM+finetuning%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations%2C+ICLR+2025%2C+Singapore%2C+April+24%2D28%2C+2025%2E+OpenReview%2E)
35. Piotr Rybak, Robert Mroczkowski, Janusz Tracz, and Ireneusz Gawlik. 2020. Klej: Comprehensive benchmark for polish language understanding. arXiv preprint arXiv:2005.00630. [find](https://scholar.google.com/scholar?q=Piotr+Rybak%2C+Robert+Mroczkowski%2C+Janusz+Tracz%2C+and+Ireneusz+Gawlik%2E+2020%2E+Klej%3A+Comprehensive+benchmark+for+polish+language+understanding%2E+arXiv+preprint+arXiv%3A2005%2E00630%2E)
36. Zirui Song, Bin Yan, Yuhan Liu, Miao Fang, Mingzhe Li, Rui Yan, and Xiuying Chen. 2025. Injecting domain-specific knowledge into large language models: A comprehensive survey. arXiv preprint arXiv:2502.10708. [find](https://scholar.google.com/scholar?q=Zirui+Song%2C+Bin+Yan%2C+Yuhan+Liu%2C+Miao+Fang%2C+Mingzhe+Li%2C+Rui+Yan%2C+and+Xiuying+Chen%2E+2025%2E+Injecting+domain%2Dspecific+knowledge+into+large+language+models%3A+A+comprehensive+survey%2E+arXiv+preprint+arXiv%3A250)
37. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+and+1+others%2E+2025%2E+Gemma+3+technical+rep)
38. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
39. Peng Wang, Ningyu Zhang, Bozhong Tian, Zekun Xi, Yunzhi Yao, Ziwen Xu, Mengru Wang, Shengyu Mao, Xiaohan Wang, Siyuan Cheng, and 1 others. 2023. Easyedit: An easy-to-use knowledge editing framework for large language models. arXiv preprint arXiv:2308.07269. [find](https://scholar.google.com/scholar?q=Peng+Wang%2C+Ningyu+Zhang%2C+Bozhong+Tian%2C+Zekun+Xi%2C+Yunzhi+Yao%2C+Ziwen+Xu%2C+Mengru+Wang%2C+Shengyu+Mao%2C+Xiaohan+Wang%2C+Siyuan+Cheng%2C+and+1+others%2E+2023%2E+Easyedit%3A+An+easy%2Dto%2Duse+knowledge+editing+framework+fo)
40. Yu Wang, Yifan Gao, Xiusi Chen, Haoming Jiang, Shiyang Li, Jingfeng Yang, Qingyu Yin, Zheng Li, Xian Li, Bing Yin, Jingbo Shang, and Julian J. McAuley. 2024. [MEMORYLLM: towards self-updatable large language models](https://md.research.elamur.ai/free/en/papers/wang-2024-memoryllm-towards-self-updatable-large/). In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austr…
41. Yu Wang, Dmitry Krotov, Yuanzhe Hu, Yifan Gao, Wangchunshu Zhou, Julian McAuley, Dan Gutfreund, Rogerio Feris, and Zexue He. 2025a. M+: Extending memoryllm with scalable long-term memory. Preprint, arXiv:2502.00592. [find](https://scholar.google.com/scholar?q=Yu+Wang%2C+Dmitry+Krotov%2C+Yuanzhe+Hu%2C+Yifan+Gao%2C+Wangchunshu+Zhou%2C+Julian+McAuley%2C+Dan+Gutfreund%2C+Rogerio+Feris%2C+and+Zexue+He%2E+2025a%2E+M%2B%3A+Extending+memoryllm+with+scalable+long%2Dterm+memory%2E+Preprint%2C+ar)
42. Yu Wang, Xinshuang Liu, Xiusi Chen, Sean O’Brien, Junda Wu, and Julian McAuley. 2025b. Self-updatable large language models by integrating context into model parameters. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yu+Wang%2C+Xinshuang+Liu%2C+Xiusi+Chen%2C+Sean+O%E2%80%99Brien%2C+Junda+Wu%2C+and+Julian+McAuley%2E+2025b%2E+Self%2Dupdatable+large+language+models+by+integrating+context+into+model+parameters%2E+In+The+Thirteenth+Internationa)
43. Ruoxi Xu, Yunjie Ji, Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, Ben He, Yingfei Sun, Xiangang Li, and Le Sun. 2025. Memorizing is not enough: Deep knowledge injection through reasoning. arXiv preprint arXiv:2504.00472. [find](https://scholar.google.com/scholar?q=Ruoxi+Xu%2C+Yunjie+Ji%2C+Boxi+Cao%2C+Yaojie+Lu%2C+Hongyu+Lin%2C+Xianpei+Han%2C+Ben+He%2C+Yingfei+Sun%2C+Xiangang+Li%2C+and+Le+Sun%2E+2025%2E+Memorizing+is+not+enough%3A+Deep+knowledge+injection+through+reasoning%2E+arXiv+prepr)
44. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+and+1+others%2E+2024%2E+Qwen2%2E+5+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E15115)
45. Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, and 1 others. 2023. Plug-and-play knowledge injection for pre-trained language models. arXiv preprint arXiv:2305.17691. [find](https://scholar.google.com/scholar?q=Zhengyan+Zhang%2C+Zhiyuan+Zeng%2C+Yankai+Lin%2C+Huadong+Wang%2C+Deming+Ye%2C+Chaojun+Xiao%2C+Xu+Han%2C+Zhiyuan+Liu%2C+Peng+Li%2C+Maosong+Sun%2C+and+1+others%2E+2023%2E+Plug%2Dand%2Dplay+knowledge+injection+for+pre%2Dtrained+langua)
46. Zexuan Zhong, Zhengxuan Wu, Christopher D Manning, Christopher Potts, and Danqi Chen. 2023. Mquake: Assessing knowledge editing in language models via multi-hop questions. arXiv preprint arXiv:2305.14795. [find](https://scholar.google.com/scholar?q=Zexuan+Zhong%2C+Zhengxuan+Wu%2C+Christopher+D+Manning%2C+Christopher+Potts%2C+and+Danqi+Chen%2E+2023%2E+Mquake%3A+Assessing+knowledge+editing+in+language+models+via+multi%2Dhop+questions%2E+arXiv+preprint+arXiv%3A2305%2E14)

*Updated 1 November 2026*