---
title: "EyeMulator — paper by Zhang et al. (2026) — summary and ideas"
description: "From these they build two small artifacts: Beta-distribution salience priors for each token type, and… 55 references, cited by 1 paper here."
authors: ["Yifan Zhang", "Chen Huang", "Yueke Zhang", "Jiahao Zhang", "Toby Jia-Jun Li", "Collin McMillan", "Kevin Leach", "Yu Huang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1158/"
doi: "10.18653/v1/2026.acl-long.1158"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-eyemulator-improving-code-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-eyemulator-improving-code-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) / EyeMulator: Improving Code Language Models by Mimicking Human Visual Attention

# EyeMulator: Improving Code Language Models by Mimicking Human Visual Attention

**Authors:** [Yifan Zhang](https://md.research.elamur.ai/free/en/authors/yifan-zhang/), [Chen Huang](https://md.research.elamur.ai/free/en/authors/chen-huang/), Yueke Zhang, [Jiahao Zhang](https://md.research.elamur.ai/free/en/authors/jiahao-zhang/), [Toby Jia-Jun Li](https://md.research.elamur.ai/free/en/authors/toby-jia-jun-li/), Collin McMillan, Kevin Leach and Yu Huang · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1158/) · [PDF](https://aclanthology.org/2026.acl-long.1158.pdf) · [DOI 10.18653/v1/2026.acl-long.1158](https://doi.org/10.18653/v1/2026.acl-long.1158) · [OpenAlex](https://openalex.org/W7166844988)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Code language models usually learn where to "look" from token statistics alone. Human programmers instead concentrate on meaningful parts of code, such as conditions, signatures and data flow, and skim boilerplate. EyeMulator transfers this human selectivity into CodeLLMs without changing their architecture. The authors process eye-tracking recordings from the EyeTrans corpus and map fixations onto AST tokens. From these they build two small artifacts: Beta-distribution salience priors for each token type, and n-gram transition probabilities describing how experts move between tokens. These artifacts are used to synthesize pseudo-scan paths for ordinary training code that has no gaze data. Fine-tuning then combines a token-weighted SFT loss with a token-level DPO loss that favors salient tokens over background ones. Experiments covered StarCoder, Llama-3.2 and DeepSeek-Coder on CodeXGlue translation, summarization and completion. The method beat baselines by large margins, and ablations attribute the gains to human-like attention. The approach is cheap, needing under 1MB of extra data, and is portable across models.

## Key points
- **Gaze data:** the EyeTrans corpus contains 120Hz recordings of 27 programmers reading and writing Java code.
  - Fixations of at least 100 ms were detected with I-DT and aligned to AST leaf tokens.
  - About 3% of points could not be mapped and were dropped.
  - The result was 1,565 scan paths.
- **Distilled artifacts:**
  - Bayesian salience estimates per semantic class, using Beta posteriors.
  - Bigram and trigram transition tables, with n-grams seen fewer than 5 times removed.
- **Training objective:** a weighted cross-entropy loss is combined with a token-level DPO term.
  - The "winning" side of the DPO term is the pseudo-scan path, and the "losing" side is its complement.
  - Weights assigned to AST tokens are copied to all of their BPE subwords.
- **Results:** gains reached 34.35 CodeBLEU points in translation and 22.92 BERTScore points in summarization. Tests used StarCoder-1B, Llama-3.2-1B and DeepSeek-Coder-1.3B.
- **Resources and limits:** the artifacts are released on Zenodo (record 17,205,682). The study is limited to 1–1.3B-parameter models and to priors derived from Java only.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 180 works · 34,304 citations
- [University of Notre Dame](https://md.research.elamur.ai/free/en/universities/university-of-notre-dame/) — University, Notre Dame, United States · 51 works · 36,717 citations
- [Vanderbilt University](https://md.research.elamur.ai/free/en/universities/vanderbilt-university/) — University, Nashville, United States · 14 works · 1,413 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Yifan Zhang, Chen Huang, Yueke Zhang, Jiahao Zhang, Toby Jia-Jun Li, Collin McMillan, Kevin Leach and Yu Huang. 2026. EyeMulator: Improving Code Language Models by Mimicking Human Visual Attention. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026eyemulator,
  title = {{EyeMulator: Improving Code Language Models by Mimicking Human Visual Attention}},
  author = {Yifan Zhang, Chen Huang, Yueke Zhang, Jiahao Zhang, Toby Jia-Jun Li, Collin McMillan, Kevin Leach and Yu Huang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1158},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1158}
}
```

- [← previous Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation](https://md.research.elamur.ai/free/en/papers/wang-2026-exploring-graph-learning-tasks-pure/)
- [next → FactUEP at SemEval-2026 Task 4: Structured Narrative Similarity Scoring with Aspect Decomposition and Weak-Signal Gating](https://md.research.elamur.ai/free/en/papers/sawinski-2026-factuep-semeval-2026-task-4/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/)

## References

55 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Silvia Abrahão, John Grundy, Mauro Pezzè, Margaret-Anne Storey, and Damian A. Tamburri. 2025. Software engineering by and for humans in an AI era. ACM Transactions on Software Engineering and Methodology, 34(5):1–46. [find](https://scholar.google.com/scholar?q=Silvia+Abrah%C3%A3o%2C+John+Grundy%2C+Mauro+Pezz%C3%A8%2C+Margaret%2DAnne+Storey%2C+and+Damian+A%2E+Tamburri%2E+2025%2E+Software+engineering+by+and+for+humans+in+an+AI+era%2E+ACM+Transactions+on+Software+Engineering+and+Methodol)
2. Aakash Bansal, Bonita Sharif, and Collin McMillan. 2023a. Towards modeling human attention from eye movements for neural source code summarization. Proceedings of the ACM on Human-Computer Interaction, 7(ETRA):1–19. [find](https://scholar.google.com/scholar?q=Aakash+Bansal%2C+Bonita+Sharif%2C+and+Collin+McMillan%2E+2023a%2E+Towards+modeling+human+attention+from+eye+movements+for+neural+source+code+summarization%2E+Proceedings+of+the+ACM+on+Human%2DComputer+Interaction)
3. Aakash Bansal, Chia-Yi Su, Zachary Karas, Yifan Zhang, Yu Huang, Toby Jia-Jun Li, and Collin McMillan. 2023b. Modeling programmer attention as scanpath prediction. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 1732–1736. IEEE. [find](https://scholar.google.com/scholar?q=Aakash+Bansal%2C+Chia%2DYi+Su%2C+Zachary+Karas%2C+Yifan+Zhang%2C+Yu+Huang%2C+Toby+Jia%2DJun+Li%2C+and+Collin+McMillan%2E+2023b%2E+Modeling+programmer+attention+as+scanpath+prediction%2E+In+2023+38th+IEEE%2FACM+International+)
4. Tristan Coignion, Clément Quinton, and Romain Rou-voy. 2024. A performance study of llm-generated code on leetcode. In Proceedings of the 28th international conference on evaluation and assessment in software engineering, pages 79–89. [find](https://scholar.google.com/scholar?q=Tristan+Coignion%2C+Cl%C3%A9ment+Quinton%2C+and+Romain+Rou%2Dvoy%2E+2024%2E+A+performance+study+of+llm%2Dgenerated+code+on+leetcode%2E+In+Proceedings+of+the+28th+international+conference+on+evaluation+and+assessment+in+)
5. Yangruibo Ding, Jinjun Peng, Marcus J. Min, Gail Kaiser, Junfeng Yang, and Baishakhi Ray. 2024. Semcoder: Training code language models with comprehensive semantics reasoning. Preprint, arXiv:2406.01006. — [arXiv:2406.01006](https://arxiv.org/abs/2406.01006) [find](https://scholar.google.com/scholar?q=Yangruibo+Ding%2C+Jinjun+Peng%2C+Marcus+J%2E+Min%2C+Gail+Kaiser%2C+Junfeng+Yang%2C+and+Baishakhi+Ray%2E+2024%2E+Semcoder%3A+Training+code+language+models+with+comprehensive+semantics+reasoning%2E+Preprint%2C+arXiv%3A2406%2E010)
6. Shihan Dou, Yan Liu, Haoxiang Jia, Limao Xiong, Enyu Zhou, Wei Shen, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Xuanjing Huang, and Tao Gui. 2024. StepCoder: Improve code generation with reinforcement learning from compiler feedback. ar… [find](https://scholar.google.com/scholar?q=Shihan+Dou%2C+Yan+Liu%2C+Haoxiang+Jia%2C+Limao+Xiong%2C+Enyu+Zhou%2C+Wei+Shen%2C+Junjie+Shan%2C+Caishuang+Huang%2C+Xiao+Wang%2C+Xiaoran+Fan%2C+Zhiheng+Xi%2C+Yuhao+Zhou%2C+Tao+Ji%2C+Rui+Zheng%2C+Qi+Zhang%2C+Xuanjing+Huang%2C+and+Tao+)
7. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. KTO: Model alignment as prospect theoretic optimization. arXiv preprint. [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2C+Winnie+Xu%2C+Niklas+Muennighoff%2C+Dan+Jurafsky%2C+and+Douwe+Kiela%2E+2024%2E+KTO%3A+Model+alignment+as+prospect+theoretic+optimization%2E+arXiv+preprint%2E)
8. Zihan Fang, Yifan Zhang, Yueke Zhang, Kevin Leach, and Yu Huang. 2025. Dpo-f+: Aligning code repair feedback with developers’ preferences. arXiv preprint arXiv:2511.01043. — [arXiv:2511.01043](https://arxiv.org/abs/2511.01043) [find](https://scholar.google.com/scholar?q=Zihan+Fang%2C+Yifan+Zhang%2C+Yueke+Zhang%2C+Kevin+Leach%2C+and+Yu+Huang%2E+2025%2E+Dpo%2Df%2B%3A+Aligning+code+repair+feedback+with+developers%E2%80%99+preferences%2E+arXiv+preprint+arXiv%3A2511%2E01043%2E+%E2%80%94+%5BarXiv%3A2511%2E01043%5D%28https%3A%2F)
9. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and 1 others. 2020. [Codebert: A pre-trained model for programming and natural languages](https://md.research.elamur.ai/free/en/papers/feng-2020-codebert-pre-trained-model-programming/). arXiv preprint arXiv:2002.08155.
10. Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. 2023. [Understanding the effects of rlhf on llm generalisation and diversity](https://md.research.elamur.ai/free/en/papers/kirk-2023-understanding-effects-rlhf-llm-generalisation/). arXiv preprint arXiv:2310.06452.
11. Lisa Grabinger, Florian Hauser, Christian Wolff, and Jürgen Mottok. 2024. On eye tracking in software engineering. SN Computer Science, 5(6):729. [find](https://scholar.google.com/scholar?q=Lisa+Grabinger%2C+Florian+Hauser%2C+Christian+Wolff%2C+and+J%C3%BCrgen+Mottok%2E+2024%2E+On+eye+tracking+in+software+engineering%2E+SN+Computer+Science%2C+5%286%29%3A729%2E)
12. Jiliang Li, Yifan Zhang, Zachary Karas, Collin McMillan, Kevin Leach, and Yu Huang. 2024. Do machines and humans focus on similar code? exploring explainability of large language models in code summarization. In Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension, page… [find](https://scholar.google.com/scholar?q=Jiliang+Li%2C+Yifan+Zhang%2C+Zachary+Karas%2C+Collin+McMillan%2C+Kevin+Leach%2C+and+Yu+Huang%2E+2024%2E+Do+machines+and+humans+focus+on+similar+code%3F+exploring+explainability+of+large+language+models+in+code+summar)
13. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Kadian, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint.
14. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024a. DeepSeek-coder: When the large language model meets programming – the rise of code intelligence. arXiv preprint. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Qihao+Zhu%2C+Dejian+Yang%2C+Zhenda+Xie%2C+Kai+Dong%2C+Wentao+Zhang%2C+Guanting+Chen%2C+Xiao+Bi%2C+Y%2E+Wu%2C+Y%2E+K%2E+Li%2C+Fuli+Luo%2C+Yingfei+Xiong%2C+and+Wenfeng+Liang%2E+2024a%2E+DeepSeek%2Dcoder%3A+When+the+large+languag)
15. Yucan Guo, Zixuan Li, Xiaolong Jin, Yantao Liu, Yutao Zeng, Wenxuan Liu, Xiang Li, Pan Yang, Long Bai, Jiafeng Guo, and 1 others. 2024b. Retrieval-augmented code generation for universal information extraction. In CCF International Conference on Natural Language Processing and Chinese Computing, pag… [find](https://scholar.google.com/scholar?q=Yucan+Guo%2C+Zixuan+Li%2C+Xiaolong+Jin%2C+Yantao+Liu%2C+Yutao+Zeng%2C+Wenxuan+Liu%2C+Xiang+Li%2C+Pan+Yang%2C+Long+Bai%2C+Jiafeng+Guo%2C+and+1+others%2E+2024b%2E+Retrieval%2Daugmented+code+generation+for+universal+information+e)
16. E. Harth and P. Dugerdil. 2017. Program understanding models: An historical overview and a classification. In Proceedings of the 12th International Conference on Software Technologies (ICSOFT), volume 1, pages 402–413. SciTePress. [find](https://scholar.google.com/scholar?q=E%2E+Harth+and+P%2E+Dugerdil%2E+2017%2E+Program+understanding+models%3A+An+historical+overview+and+a+classification%2E+In+Proceedings+of+the+12th+International+Conference+on+Software+Technologies+%28ICSOFT%29%2C+volume)
17. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Akiki, and 1 others. 2023. StarCoder: May the source be with you! arXiv preprint. [find](https://scholar.google.com/scholar?q=Raymond+Li%2C+Loubna+Ben+Allal%2C+Yangtian+Zi%2C+Niklas+Muennighoff%2C+Denis+Kocetkov%2C+Chenghao+Mou%2C+Marc+Marone%2C+Akiki%2C+and+1+others%2E+2023%2E+StarCoder%3A+May+the+source+be+with+you%21+arXiv+preprint%2E)
18. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. Is your code generated by Chat-GPT really correct? Rigorous evaluation of large language models for code generation. arXiv preprint. [find](https://scholar.google.com/scholar?q=Jiawei+Liu%2C+Chunqiu+Steven+Xia%2C+Yuyao+Wang%2C+and+Lingming+Zhang%2E+2023%2E+Is+your+code+generated+by+Chat%2DGPT+really+correct%3F+Rigorous+evaluation+of+large+language+models+for+code+generation%2E+arXiv+preprin)
19. Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, and 1 others. 2021. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXiv:2102.04664. [find](https://scholar.google.com/scholar?q=Shuai+Lu%2C+Daya+Guo%2C+Shuo+Ren%2C+Junjie+Huang%2C+Alexey+Svyatkovskiy%2C+Ambrosio+Blanco%2C+Colin+Clement%2C+Dawn+Drain%2C+Daxin+Jiang%2C+Duyu+Tang%2C+and+1+others%2E+2021%2E+Codexglue%3A+A+machine+learning+benchmark+dataset)
20. Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. SimPO: Simple preference optimization with a reference-free reward. arXiv preprint. [find](https://scholar.google.com/scholar?q=Yu+Meng%2C+Mengzhou+Xia%2C+and+Danqi+Chen%2E+2024%2E+SimPO%3A+Simple+preference+optimization+with+a+reference%2Dfree+reward%2E+arXiv+preprint%2E)
21. Pengfei He, Shaowei Wang, Shaiful Chowdhury, and Tse-Hsun Chen. 2024. Exploring demonstration retrievers in rag for coding tasks: Yeas and nays! arXiv preprint arXiv:2410.09662. — [arXiv:2410.09662](https://arxiv.org/abs/2410.09662) [find](https://scholar.google.com/scholar?q=Pengfei+He%2C+Shaowei+Wang%2C+Shaiful+Chowdhury%2C+and+Tse%2DHsun+Chen%2E+2024%2E+Exploring+demonstration+retrievers+in+rag+for+coding+tasks%3A+Yeas+and+nays%21+arXiv+preprint+arXiv%3A2410%2E09662%2E+%E2%80%94+%5BarXiv%3A2410%2E09662%5D%28h)
22. Meta AI. 2024. Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. Meta AI Blog. [https://ai.meta.com/blog/](https://ai.meta.com/blog/) llama-3-2-connect-2024-vision-edge-mobile-devices/. [↗](https://ai.meta.com/blog/)
23. Y. Huang, K. Leach, Z. Sharafi, T. Santander, and W. Weimer. 2020. Biases and differences in code review using medical imaging and eye-tracking: genders, humans, and machines. In Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software E… [find](https://scholar.google.com/scholar?q=Y%2E+Huang%2C+K%2E+Leach%2C+Z%2E+Sharafi%2C+T%2E+Santander%2C+and+W%2E+Weimer%2E+2020%2E+Biases+and+differences+in+code+review+using+medical+imaging+and+eye%2Dtracking%3A+genders%2C+humans%2C+and+machines%2E+In+Proceedings+of+the+28)
24. Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an llm to help with code understanding. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, pages 1–13. [find](https://scholar.google.com/scholar?q=Daye+Nam%2C+Andrew+Macvean%2C+Vincent+Hellendoorn%2C+Bogdan+Vasilescu%2C+and+Brad+Myers%2E+2024%2E+Using+an+llm+to+help+with+code+understanding%2E+In+Proceedings+of+the+IEEE%2FACM+46th+International+Conference+on+Sof)
25. Dominik Huber, Matteo Paltenghi, and Michael Pradel. 2023. Where to look when repairing code? Comparing the attention of neural models and developers. arXiv preprint. [find](https://scholar.google.com/scholar?q=Dominik+Huber%2C+Matteo+Paltenghi%2C+and+Michael+Pradel%2E+2023%2E+Where+to+look+when+repairing+code%3F+Comparing+the+attention+of+neural+models+and+developers%2E+arXiv+preprint%2E)
26. Ruili Jiang, Kehai Chen, Xuefeng Bai, Zhixuan He, Juntao Li, Muyun Yang, Tiejun Zhao, Liqiang Nie, and Min Zhang. 2024. A survey on human preference learning for large language models. arXiv preprint. [find](https://scholar.google.com/scholar?q=Ruili+Jiang%2C+Kehai+Chen%2C+Xuefeng+Bai%2C+Zhixuan+He%2C+Juntao+Li%2C+Muyun+Yang%2C+Tiejun+Zhao%2C+Liqiang+Nie%2C+and+Min+Zhang%2E+2024%2E+A+survey+on+human+preference+learning+for+large+language+models%2E+arXiv+preprint%2E)
27. Zachary Karas, Aakash Bansal, Yifan Zhang, Toby Li, Collin McMillan, and Yu Huang. 2024. A tale of two comprehensions? analyzing student programmer attention during code summarization. ACM Transactions on Software Engineering and Methodology. [find](https://scholar.google.com/scholar?q=Zachary+Karas%2C+Aakash+Bansal%2C+Yifan+Zhang%2C+Toby+Li%2C+Collin+McMillan%2C+and+Yu+Huang%2E+2024%2E+A+tale+of+two+comprehensions%3F+analyzing+student+programmer+attention+during+code+summarization%2E+ACM+Transaction)
28. Thu-Trang Nguyen, Thanh Trong Vu, Hieu Dinh Vo, and Son Nguyen. 2024. An empirical study on capability of large language models in understanding code semantics. arXiv preprint. [find](https://scholar.google.com/scholar?q=Thu%2DTrang+Nguyen%2C+Thanh+Trong+Vu%2C+Hieu+Dinh+Vo%2C+and+Son+Nguyen%2E+2024%2E+An+empirical+study+on+capability+of+large+language+models+in+understanding+code+semantics%2E+arXiv+preprint%2E)
29. M. P. O’Brien. 2003. Software comprehension: A review and research direction. Technical report, Department of Computer Science & Information Systems, University of Limerick. [find](https://scholar.google.com/scholar?q=M%2E+P%2E+O%E2%80%99Brien%2E+2003%2E+Software+comprehension%3A+A+review+and+research+direction%2E+Technical+report%2C+Department+of+Computer+Science+%26+Information+Systems%2C+University+of+Limerick%2E)
30. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. [Train…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
31. Matteo Paltenghi and Michael Pradel. 2021. Thinking like a developer? Comparing the attention of humans with neural models of code. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 867–879, Melbourne, Australia. IEEE. [find](https://scholar.google.com/scholar?q=Matteo+Paltenghi+and+Michael+Pradel%2E+2021%2E+Thinking+like+a+developer%3F+Comparing+the+attention+of+humans+with+neural+models+of+code%2E+In+2021+36th+IEEE%2FACM+International+Conference+on+Automated+Software)
32. Zhichao Wang, Bin Bi, Shiva Kumar Pentyala, Kiran Ramnath, Sougata Chaudhuri, Shubham Mehrotra, Xiang-Bo Mao, Sitaram Asur, and 1 others. 2024. A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more. arXiv preprint arXiv:2407.16216. — [arXiv:2407.16216](https://arxiv.org/abs/2407.16216) [find](https://scholar.google.com/scholar?q=Zhichao+Wang%2C+Bin+Bi%2C+Shiva+Kumar+Pentyala%2C+Kiran+Ramnath%2C+Sougata+Chaudhuri%2C+Shubham+Mehrotra%2C+Xiang%2DBo+Mao%2C+Sitaram+Asur%2C+and+1+others%2E+2024%2E+A+comprehensive+survey+of+llm+alignment+techniques%3A+Rlhf)
33. Md Rizwan Parvez, Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021. [Retrieval augmented code generation and summarization](https://md.research.elamur.ai/free/en/papers/parvez-2021-retrieval-augmented-code-generation-summarization/). arXiv preprint arXiv:2108.11601.
34. Zora Zhiruo Wang, Akari Asai, Xinyan Velocity Yu, Frank F. Xu, Yiqing Xie, Graham Neubig, and Daniel Fried. 2025. Coderag-bench: Can retrieval augment code generation? Preprint, arXiv:2406.14497. [find](https://scholar.google.com/scholar?q=Zora+Zhiruo+Wang%2C+Akari+Asai%2C+Xinyan+Velocity+Yu%2C+Frank+F%2E+Xu%2C+Yiqing+Xie%2C+Graham+Neubig%2C+and+Daniel+Fried%2E+2025%2E+Coderag%2Dbench%3A+Can+retrieval+augment+code+generation%3F+Preprint%2C+arXiv%3A2406%2E14497%2E)
35. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). arXiv preprint.
36. Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang. 2023. Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint. arXiv preprint arXiv:2312.11456. [find](https://scholar.google.com/scholar?q=Wei+Xiong%2C+Hanze+Dong%2C+Chenlu+Ye%2C+Ziqi+Wang%2C+Han+Zhong%2C+Heng+Ji%2C+Nan+Jiang%2C+and+Tong+Zhang%2E+2023%2E+Iterative+preference+learning+from+human+feedback%3A+Bridging+theory+and+practice+for+rlhf+under+kl%2Dcons)
37. Paige Rodeghero, Collin McMillan, Paul W. McBurney, Nigel Bosch, and Sidney D’Mello. 2014. Improving automated source code summarization via an eye-tracking study of programmers. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, page 390–401, New York, NY, USA.… [find](https://scholar.google.com/scholar?q=Paige+Rodeghero%2C+Collin+McMillan%2C+Paul+W%2E+McBurney%2C+Nigel+Bosch%2C+and+Sidney+D%E2%80%99Mello%2E+2014%2E+Improving+automated+source+code+summarization+via+an+eye%2Dtracking+study+of+programmers%2E+In+Proceedings+of+the)
38. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). Preprint, arXiv:2402.03300.
39. Zohreh Sharafi, Timothy Shaffer, Bonita Sharif, and Yann-Gaël Guéhéneuc. 2015a. Eye-tracking metrics in software engineering. In 2015 Asia-Pacific Software Engineering Conference (APSEC), pages 96–103. [find](https://scholar.google.com/scholar?q=Zohreh+Sharafi%2C+Timothy+Shaffer%2C+Bonita+Sharif%2C+and+Yann%2DGa%C3%ABl+Gu%C3%A9h%C3%A9neuc%2E+2015a%2E+Eye%2Dtracking+metrics+in+software+engineering%2E+In+2015+Asia%2DPacific+Software+Engineering+Conference+%28APSEC%29%2C+pages+96%E2%80%93103)
40. Zohreh Sharafi, Bonita Sharif, Yann-Gaël Guéhéneuc, Andrew Begel, Roman Bednarik, and Martha Crosby. 2020. A practical guide on conducting eye tracking studies in software engineering. Empirical Software Engineering, 25(5):3128–3174. [find](https://scholar.google.com/scholar?q=Zohreh+Sharafi%2C+Bonita+Sharif%2C+Yann%2DGa%C3%ABl+Gu%C3%A9h%C3%A9neuc%2C+Andrew+Begel%2C+Roman+Bednarik%2C+and+Martha+Crosby%2E+2020%2E+A+practical+guide+on+conducting+eye+tracking+studies+in+software+engineering%2E+Empirical+Softw)
41. Zohreh Sharafi, Zéphyrin Soh, and Yann-Gaël Guéhéneuc. 2015b. A systematic literature review on the usage of eye-tracking in software engineering. Information and Software Technology, 67:79–107. [find](https://scholar.google.com/scholar?q=Zohreh+Sharafi%2C+Z%C3%A9phyrin+Soh%2C+and+Yann%2DGa%C3%ABl+Gu%C3%A9h%C3%A9neuc%2E+2015b%2E+A+systematic+literature+review+on+the+usage+of+eye%2Dtracking+in+software+engineering%2E+Information+and+Software+Technology%2C+67%3A79%E2%80%93107%2E)
42. Mohammed Latif Siddiq, Lindsay Roney, Jiahao Zhang, and Joanna Cecilia Da Silva Santos. 2024. Quality assessment of ChatGPT generated code and their use by developers. In Proceedings of the 21st International Conference on Mining Software Repositories, pages 152–156, Lisbon Portugal. ACM. [find](https://scholar.google.com/scholar?q=Mohammed+Latif+Siddiq%2C+Lindsay+Roney%2C+Jiahao+Zhang%2C+and+Joanna+Cecilia+Da+Silva+Santos%2E+2024%2E+Quality+assessment+of+ChatGPT+generated+code+and+their+use+by+developers%2E+In+Proceedings+of+the+21st+Inter)
43. Stewart Slocum, Asher Parker-Sartori, and Dylan Hadfield-Menell. 2025. Diverse preference learning for capabilities and alignment. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Stewart+Slocum%2C+Asher+Parker%2DSartori%2C+and+Dylan+Hadfield%2DMenell%2E+2025%2E+Diverse+preference+learning+for+capabilities+and+alignment%2E+In+The+Thirteenth+International+Conference+on+Learning+Representation)
44. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30.
45. Wenda Xu, Daniel Deutsch, Mara Finkelstein, Juraj Juraska, Biao Zhang, Zhongtao Liu, William Yang Wang, Lei Li, and Markus Freitag. 2024. LLMRefine: Pinpointing and refining large language models via fine-grained actionable feedback. arXiv preprint. [find](https://scholar.google.com/scholar?q=Wenda+Xu%2C+Daniel+Deutsch%2C+Mara+Finkelstein%2C+Juraj+Juraska%2C+Biao+Zhang%2C+Zhongtao+Liu%2C+William+Yang+Wang%2C+Lei+Li%2C+and+Markus+Freitag%2E+2024%2E+LLMRefine%3A+Pinpointing+and+refining+large+language+models+via+)
46. Xinwei Yang, Zhaofeng Liu, Chen Huang, Jiashuai Zhang, Tong Zhang, Yifan Zhang, and Wenqiang Lei. 2025a. Elaboration: A comprehensive benchmark on human-llm competitive programming. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa… [find](https://scholar.google.com/scholar?q=Xinwei+Yang%2C+Zhaofeng+Liu%2C+Chen+Huang%2C+Jiashuai+Zhang%2C+Tong+Zhang%2C+Yifan+Zhang%2C+and+Wenqiang+Lei%2E+2025a%2E+Elaboration%3A+A+comprehensive+benchmark+on+human%2Dllm+competitive+programming%2E+In+Proceedings+of+)
47. Zezhou Yang, Sirong Chen, Cuiyun Gao, Zhenhao Li, Xing Hu, Kui Liu, and Xin Xia. 2025b. An empirical study of retrieval-augmented code generation: Challenges and opportunities. ACM Transactions on Software Engineering and Methodology. [find](https://scholar.google.com/scholar?q=Zezhou+Yang%2C+Sirong+Chen%2C+Cuiyun+Gao%2C+Zhenhao+Li%2C+Xing+Hu%2C+Kui+Liu%2C+and+Xin+Xia%2E+2025b%2E+An+empirical+study+of+retrieval%2Daugmented+code+generation%3A+Challenges+and+opportunities%2E+ACM+Transactions+on+Sof)
48. Yifan Zhang, Chen Huang, Zachary Karas, Dung Thuy Nguyen, Kevin Leach, and Yu Huang. 2025a. Enhancing code llm training with programmer attention. arXiv preprint arXiv:2503.14936. — [arXiv:2503.14936](https://arxiv.org/abs/2503.14936) [find](https://scholar.google.com/scholar?q=Yifan+Zhang%2C+Chen+Huang%2C+Zachary+Karas%2C+Dung+Thuy+Nguyen%2C+Kevin+Leach%2C+and+Yu+Huang%2E+2025a%2E+Enhancing+code+llm+training+with+programmer+attention%2E+arXiv+preprint+arXiv%3A2503%2E14936%2E+%E2%80%94+%5BarXiv%3A2503%2E14936%5D)
49. Yifan Zhang and Kevin Leach. 2025. Leveraging human insights for enhanced llm-based code repair. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering, pages 1536–1537. [find](https://scholar.google.com/scholar?q=Yifan+Zhang+and+Kevin+Leach%2E+2025%2E+Leveraging+human+insights+for+enhanced+llm%2Dbased+code+repair%2E+In+Proceedings+of+the+33rd+ACM+International+Conference+on+the+Foundations+of+Software+Engineering%2C+pag)
50. Yifan Zhang, Jiliang Li, Zachary Karas, Aakash Bansal, Toby Jia-Jun Li, Collin McMillan, Kevin Leach, and Yu Huang. 2024. Eyetrans: Merging human and machine attention for neural code summarization. Proceedings of the ACM on Software Engineering, 1(FSE):115–136. [find](https://scholar.google.com/scholar?q=Yifan+Zhang%2C+Jiliang+Li%2C+Zachary+Karas%2C+Aakash+Bansal%2C+Toby+Jia%2DJun+Li%2C+Collin+McMillan%2C+Kevin+Leach%2C+and+Yu+Huang%2E+2024%2E+Eyetrans%3A+Merging+human+and+machine+attention+for+neural+code+summarization%2E+P)
51. Yueke Zhang, Yifan Zhang, Kevin Leach, and Yu Huang. 2025b. Codegrad: Integrating multi-step verification with gradient-based llm refinement. arXiv preprint arXiv:2508.10059. — [arXiv:2508.10059](https://arxiv.org/abs/2508.10059) [find](https://scholar.google.com/scholar?q=Yueke+Zhang%2C+Yifan+Zhang%2C+Kevin+Leach%2C+and+Yu+Huang%2E+2025b%2E+Codegrad%3A+Integrating+multi%2Dstep+verification+with+gradient%2Dbased+llm+refinement%2E+arXiv+preprint+arXiv%3A2508%2E10059%2E+%E2%80%94+%5BarXiv%3A2508%2E10059%5D%28http)
52. Ziyao Zhang, Yanlin Wang, Chong Wang, Jiachi Chen, and Zibin Zheng. 2025c. LLM hallucinations in practical code generation: Phenomena, mechanism, and mitigation. arXiv preprint. [find](https://scholar.google.com/scholar?q=Ziyao+Zhang%2C+Yanlin+Wang%2C+Chong+Wang%2C+Jiachi+Chen%2C+and+Zibin+Zheng%2E+2025c%2E+LLM+hallucinations+in+practical+code+generation%3A+Phenomena%2C+mechanism%2C+and+mitigation%2E+arXiv+preprint%2E)
53. ROUGE-L: Measures the Longest Common Subsequence (LCS) between the candidate summary and the reference, capturing sentence-level structure. [find](https://scholar.google.com/scholar?q=ROUGE%2DL%3A+Measures+the+Longest+Common+Subsequence+%28LCS%29+between+the+candidate+summary+and+the+reference%2C+capturing+sentence%2Dlevel+structure%2E)
54. Li Zhong and Zilong Wang. 2024. Can llm replace stack overflow? a study on robustness and reliability of large language model code generation. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 21841–21849. [find](https://scholar.google.com/scholar?q=Li+Zhong+and+Zilong+Wang%2E+2024%2E+Can+llm+replace+stack+overflow%3F+a+study+on+robustness+and+reliability+of+large+language+model+code+generation%2E+In+Proceedings+of+the+AAAI+conference+on+artificial+intel)
55. A • METEOR: Computes the harmonic mean of precision and recall, incorporating stemming and synonym matching to capture semantic overlap. Evaluation Metrics To ensure reproducibility, we provide formal definitions for all metrics used to evaluate task performance (RQ2–RQ4) and attention alignment (RQ… [find](https://scholar.google.com/scholar?q=A+%E2%80%A2+METEOR%3A+Computes+the+harmonic+mean+of+precision+and+recall%2C+incorporating+stemming+and+synonym+matching+to+capture+semantic+overlap%2E+Evaluation+Metrics+To+ensure+reproducibility%2C+we+provide+formal)

*Updated 31 October 2026*