---
title: "Topology-Enhanced Alignment for Large Language Models:"
description: "Common LLM alignment objectives (SFT, RLHF, DPO) optimise token likelihoods or scalar preference scores. 30 references."
authors: ["Yurui Pan", "Ke Xu", "Bo Peng"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1242/"
doi: "10.18653/v1/2026.findings-acl.1242"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pan-2026-topology-enhanced-alignment-large-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pan-2026-topology-enhanced-alignment-large-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/46/) / Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference Optimization

# Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference Optimization

**Authors:** Yurui Pan, [Ke Xu](https://md.research.elamur.ai/free/en/authors/ke-xu/) and [Bo Peng](https://md.research.elamur.ai/free/en/authors/bo-peng/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1242/) · [PDF](https://aclanthology.org/2026.findings-acl.1242.pdf) · [DOI 10.18653/v1/2026.findings-acl.1242](https://doi.org/10.18653/v1/2026.findings-acl.1242) · [OpenAlex](https://openalex.org/W7166879139)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Common LLM alignment objectives (SFT, RLHF, DPO) optimise token likelihoods or scalar preference scores. They place no constraint on how hidden representations move from a prompt to a response. The authors treat generation as a path through hidden space and regularise that path with 0-dimensional persistent homology.

For SFT, Trajectory Topology Loss (TTL) puts a batch's mean-pooled prompt and gold-answer embeddings into one point cloud. A Union-Find procedure then finds the merge edges that first connect prompt and answer components. A cosine term pushes the model's prompt-to-answer shift toward these "bridges".

For DPO, Topological Preference Optimization (TPO) pulls the rejected-to-chosen hidden-state difference at an intermediate layer toward topic-specific preference vectors. These vectors are built offline through clustering and LLM labelling, and an EMA scheme balances the DPO and TPO terms.

On Qwen2.5-7B-Instruct with UltraChat and Anthropic HH-RLHF, both methods beat plain baselines and non-topological regularisers without worsening toxicity. This suggests that inexpensive topological signals can make alignment more controllable. However, only one model and English data were tested.

## Key points
- Base model is Qwen2.5-7B-Instruct. UltraChat is used for SFT experiments, and Anthropic HH-RLHF is used for DPO experiments.
- On HH-RLHF, TPO improves on plain DPO: RewardBench rises from 84.5 to 87.2, AlpacaEval win rate from 52.1% to 55.4%, and harmlessness from 90.2% to 93.5%.
- Topic-aware preference vectors outperform a single global vector (RM 87.2 vs 85.1; helpfulness 8.81 vs 8.68). Removing EMA dynamic weighting lowers results.
- The full pipeline (SFT+TTL, then DPO+TPO) reaches RewardBench 88.1, AlpacaEval 56.5%, MT-Bench 8.88 and harmlessness 94.5%. The SFT+DPO baseline scores 84.5, 52.1%, 8.65 and 90.2% on the same metrics.
- Ablations show that persistent-homology bridges beat random, per-example and nearest-neighbour pairings. Limitations include English-only evaluation, LLM-based topic labelling, and pairwise-distance cost that grows quadratically with batch size.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/) — University, Shanghai, China · 167 works · 20,987 citations
- [Tongji University](https://md.research.elamur.ai/free/en/universities/tongji-university/) — University, Shanghai, China · 19 works · 2,162 citations

Also on the byline: [Shanghai Ocean University](https://ror.org/04n40zv07).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Yurui Pan, Ke Xu and Bo Peng. 2026. Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference Optimization. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pan2026topology,
  title = {{Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference Optimization}},
  author = {Yurui Pan, Ke Xu and Bo Peng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1242},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1242}
}
```

- [← previous Topology Matters: Measuring Memory Leakage in Multi-Agent LLMs](https://md.research.elamur.ai/free/en/papers/liu-2026-topology-matters-measuring-memory-leakage/)
- [next → Toward Cognitive Alignment in Large Language Models: Integrating Linguistic Theory and Human Data](https://md.research.elamur.ai/free/en/papers/zaghouani-2026-cognitive-alignment-large-language-models/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/46/)

## References

30 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alessandro Achille and Stefano Soatto. 2018. Emergence of invariance and disentanglement in deep representations. Journal of Machine Learning Research, 19(50):1–54. [find](https://scholar.google.com/scholar?q=Alessandro+Achille+and+Stefano+Soatto%2E+2018%2E+Emergence+of+invariance+and+disentanglement+in+deep+representations%2E+Journal+of+Machine+Learning+Research%2C+19%2850%29%3A1%E2%80%9354%2E)
2. Henry Adams, Trey Emerson, Martin Kirby, Christopher J. Neville, Chris Peterson, Patrick Shipman, Svetlana Chepushtanova, Mariah Hanson, Fabio Motta, and Leonard Ziegelmeier. 2015. Persistence images: A stable vector representation of persistent homology. Journal of Machine Learning Research, 18(8):… [find](https://scholar.google.com/scholar?q=Henry+Adams%2C+Trey+Emerson%2C+Martin+Kirby%2C+Christopher+J%2E+Neville%2C+Chris+Peterson%2C+Patrick+Shipman%2C+Svetlana+Chepushtanova%2C+Mariah+Hanson%2C+Fabio+Motta%2C+and+Leonard+Ziegelmeier%2E+2015%2E+Persistence+images%3A)
3. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, and 12 others. 202… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
4. Rubén Ballester, Carles Casacuberta, and Sergio Escalera. 2024. Topological data analysis for neural network analysis: A comprehensive survey. arXiv preprint arXiv:2312.05840. — [arXiv:2312.05840](https://arxiv.org/abs/2312.05840) [find](https://scholar.google.com/scholar?q=Rub%C3%A9n+Ballester%2C+Carles+Casacuberta%2C+and+Sergio+Escalera%2E+2024%2E+Topological+data+analysis+for+neural+network+analysis%3A+A+comprehensive+survey%2E+arXiv+preprint+arXiv%3A2312%2E05840%2E+%E2%80%94+%5BarXiv%3A2312%2E05840%5D%28htt)
5. David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3319–3327. [find](https://scholar.google.com/scholar?q=David+Bau%2C+Bolei+Zhou%2C+Aditya+Khosla%2C+Aude+Oliva%2C+and+Antonio+Torralba%2E+2017%2E+Network+dissection%3A+Quantifying+interpretability+of+deep+visual+representations%2E+In+Proceedings+of+the+IEEE+Conference+on+)
6. Ralph Allan Bradley and Milton E. Terry. 1952. The rank analysis of incomplete block designs: The method of paired comparisons. Biometrika, 39:324– 345. [find](https://scholar.google.com/scholar?q=Ralph+Allan+Bradley+and+Milton+E%2E+Terry%2E+1952%2E+The+rank+analysis+of+incomplete+block+designs%3A+The+method+of+paired+comparisons%2E+Biometrika%2C+39%3A324%E2%80%93+345%2E)
7. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
8. Peter Bubenik. 2015. Statistical topological data analysis using persistence landscapes. Journal of Machine Learning Research, 16(1):77–102. [find](https://scholar.google.com/scholar?q=Peter+Bubenik%2E+2015%2E+Statistical+topological+data+analysis+using+persistence+landscapes%2E+Journal+of+Machine+Learning+Research%2C+16%281%29%3A77%E2%80%93102%2E)
9. Gunnar Carlsson. 2009. Topology and data. Bulletin of the American Mathematical Society, 46(2):255–308. [find](https://scholar.google.com/scholar?q=Gunnar+Carlsson%2E+2009%2E+Topology+and+data%2E+Bulletin+of+the+American+Mathematical+Society%2C+46%282%29%3A255%E2%80%93308%2E)
10. Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. arXiv preprint arXiv:1706.03741. [find](https://scholar.google.com/scholar?q=Paul+Christiano%2C+Jan+Leike%2C+Tom+B%2E+Brown%2C+Miljan+Martic%2C+Shane+Legg%2C+and+Dario+Amodei%2E+2017%2E+Deep+reinforcement+learning+from+human+preferences%2E+arXiv+preprint+arXiv%3A1706%2E03741%2E)
11. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. [Enhancing chat language models by scaling high-quality instructional conversations](https://md.research.elamur.ai/free/en/papers/ding-2023-enhancing-chat-language-models-scaling/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3029–3051.
12. Herbert Edelsbrunner and John Harer. 2010. Computational topology: An introduction. American Mathematical Society. [find](https://scholar.google.com/scholar?q=Herbert+Edelsbrunner+and+John+Harer%2E+2010%2E+Computational+topology%3A+An+introduction%2E+American+Mathematical+Society%2E)
13. Kawin Ethayarajh. 2019. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. In Proceedings of EMNLP-IJCNLP, pages 55–65. [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2E+2019%2E+How+contextual+are+contextualized+word+representations%3F+comparing+the+geometry+of+bert%2C+elmo%2C+and+gpt%2D2+embeddings%2E+In+Proceedings+of+EMNLP%2DIJCNLP%2C+pages+55%E2%80%9365%2E)
14. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. [Realtoxicityprompts: Evaluating neural toxic degeneration in language models](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3356–3369.
15. Robert Ghrist. 2008. Barcodes: the persistent topology of data. Bulletin of the American Mathematical Society, 45(1):61–75. [find](https://scholar.google.com/scholar?q=Robert+Ghrist%2E+2008%2E+Barcodes%3A+the+persistent+topology+of+data%2E+Bulletin+of+the+American+Mathematical+Society%2C+45%281%29%3A61%E2%80%9375%2E)
16. Dan Hendrycks and Kevin Gimpel. 2017. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks+and+Kevin+Gimpel%2E+2017%2E+A+baseline+for+detecting+misclassified+and+out%2Dof%2Ddistribution+examples+in+neural+networks%2E+In+International+Conference+on+Learning+Representations%2E)
17. Christoph Hofer and 1 others. 2019. Deep learning with topological signatures. In NeurIPS. [find](https://scholar.google.com/scholar?q=Christoph+Hofer+and+1+others%2E+2019%2E+Deep+learning+with+topological+signatures%2E+In+NeurIPS%2E)
18. Joseph Kruskal. 1956. On the shortest spanning subtree of a graph. Proc. AMS. [find](https://scholar.google.com/scholar?q=Joseph+Kruskal%2E+1956%2E+On+the+shortest+spanning+subtree+of+a+graph%2E+Proc%2E+AMS%2E)
19. Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems, volume 31, pages 7167–7177. [find](https://scholar.google.com/scholar?q=Kimin+Lee%2C+Kibok+Lee%2C+Honglak+Lee%2C+and+Jinwoo+Shin%2E+2018%2E+A+simple+unified+framework+for+detecting+out%2Dof%2Ddistribution+samples+and+adversarial+attacks%2E+In+Advances+in+Neural+Information+Processing+Sys)
20. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). In Proceedings of the ICLR 2013 Workshop on Representation Learning for NLP.
21. Guillermo Ortiz-Jiménez, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. 2020. Neural anisotropy directions. arXiv preprint arXiv:2006.09717. — [arXiv:2006.09717](https://arxiv.org/abs/2006.09717) [find](https://scholar.google.com/scholar?q=Guillermo+Ortiz%2DJim%C3%A9nez%2C+Apostolos+Modas%2C+Seyed%2DMohsen+Moosavi%2DDezfooli%2C+and+Pascal+Frossard%2E+2020%2E+Neural+anisotropy+directions%2E+arXiv+preprint+arXiv%3A2006%2E09717%2E+%E2%80%94+%5BarXiv%3A2006%2E09717%5D%28https%3A%2F%2Farxiv%2Eor)
22. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. [Train…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
23. Robin L. Plackett. 1975. The analysis of permutations. Journal of the Royal Statistical Society. Series C (Applied Statistics), 24(2):193–202. [find](https://scholar.google.com/scholar?q=Robin+L%2E+Plackett%2E+1975%2E+The+analysis+of+permutations%2E+Journal+of+the+Royal+Statistical+Society%2E+Series+C+%28Applied+Statistics%29%2C+24%282%29%3A193%E2%80%93202%2E)
24. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). arXiv preprint arXiv:2305.18290.
25. Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl-Dickstein. 2017. On the expressive power of neural networks. In Proceedings of the 34th International Conference on Machine Learning, pages 2847–2854. [find](https://scholar.google.com/scholar?q=Maithra+Raghu%2C+Ben+Poole%2C+Jon+Kleinberg%2C+Surya+Ganguli%2C+and+Jascha+Sohl%2DDickstein%2E+2017%2E+On+the+expressive+power+of+neural+networks%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Lear)
26. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of EMNLP-IJCNLP, pages 3982– 3992.
27. K. Rieck, H. Leitte, C. Höfer, and H. Wagner. 2019. Neural persistence: A complexity measure for deep neural networks using algebraic topology. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=K%2E+Rieck%2C+H%2E+Leitte%2C+C%2E+H%C3%B6fer%2C+and+H%2E+Wagner%2E+2019%2E+Neural+persistence%3A+A+complexity+measure+for+deep+neural+networks+using+algebraic+topology%2E+In+International+Conference+on+Learning+Representations%2E)
28. Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2020. Learning to summarize from human feedback. In Advances in Neural Information Processing Systems, volume 33, pages 3008–3021. [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeff+Wu%2C+Daniel+M%2E+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+Christiano%2E+2020%2E+Learning+to+summarize+from+human+feedback%2E+In+Advances+in+Neura)
29. Robert Endre Tarjan. 1975. Efficiency of a good but not linear set union algorithm. Journal of the ACM, 22(2):215–225. [find](https://scholar.google.com/scholar?q=Robert+Endre+Tarjan%2E+1975%2E+Efficiency+of+a+good+but+not+linear+set+union+algorithm%2E+Journal+of+the+ACM%2C+22%282%29%3A215%E2%80%93225%2E)
30. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems, volume 30, pages 5998–6008.

*Updated 1 November 2026*