--- title: "Rdgai: Classifying transcriptional changes — Turnbull 2025" description: "Phylogenetic analyses of manuscript traditions usually count every textual variant as equally informative." authors: ["Robert Turnbull"] year: 2025 venue: "arXiv:2511.13801" source: "https://arxiv.org/abs/2511.13801" doi: "10.48550/arxiv.2511.13801" arxiv: "2511.13801" licence: "CC BY-NC-SA 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by-nc-sa/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/turnbull-2025-rdgai-classifying-transcriptional-changes-using/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/turnbull-2025-rdgai-classifying-transcriptional-changes-using/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / Rdgai: Classifying transcriptional changes using Large Language Models with a test case from an Arabic Gospel tradition # Rdgai: Classifying transcriptional changes using Large Language Models with a test case from an Arabic Gospel tradition **Authors:** Robert Turnbull · **Year:** 2025 · **Venue:** arXiv:2511.13801 · **Licence:** [CC BY-NC-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-nc-sa/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2511.13801) · [PDF](https://arxiv.org/pdf/2511.13801) · [DOI 10.48550/arxiv.2511.13801](https://doi.org/10.48550/arxiv.2511.13801) · [arXiv 2511.13801](https://arxiv.org/abs/2511.13801) · [OpenAlex](https://openalex.org/W4417209547) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary Phylogenetic analyses of manuscript traditions usually count every textual variant as equally informative. Spelling-level noise therefore weighs as much as meaningful rewording. Bayesian phylogenetics can estimate a separate rate for each category of change, but someone has to label every pairwise transition between readings in each variation unit, and doing that by hand is very slow. Turnbull presents Rdgai, a Python tool that reads a TEI XML critical apparatus. The user defines transition categories with descriptions and hand-labels some transitions in a GUI or an Excel export. Rdgai puts these labelled examples into an LLM prompt, and the LLM classifies the remaining transitions and justifies each choice. Results are written back into the TEI XML, which teiphy can turn into Beast 2 input. The test case was the Arabic Gospel translation ar^b (John 8:12–51) with four categories. Claude 3.5 Sonnet classified held-out transitions with over 90% accuracy. This makes rate-aware phylogenetics practical for smaller research projects. ## Key points - An earlier study of ar^b (John 5:36–8:51) used more than 800 variation units, more than 2,000 readings and about 60 change categories. It estimated that insignificant changes happened about 1.8 times as often as significant ones. - The test file has more than 800 hand-labelled transitions in four classes: Orthography, Single Minor Word Change, Single Major Word Change and Multiple Word Changes. Half were available as prompt examples and the other half were used for scoring. - Five models were compared with 10, 20 or 30 examples per category: GPT-4o, GPT-4o-mini, Claude 3.5 Sonnet, Claude 3.5 Haiku and Llama 3.3 70B. - The best result was 91.4% (Sonnet, 10 examples). - Adding more examples barely helped, while the choice of model mattered much more. - Closed models beat Llama. - In the confusion matrix, the main errors were minor changes labelled as major (6 cases) or as multi-word (4 cases). - Rdgai is released under Apache 2.0 on GitHub and PyPI, with full test coverage. It works with teiphy, which produces Beast 2 input. ## Interesting ideas - Categories can be declared as inverse pairs, linked through the corresp attribute, or as symmetric. Labelling one direction of a transition then fixes the label for the reverse direction automatically, which roughly halves annotation work. — For Mova (frames): converse frame pairs such as sell/buy or give/receive could be declared once as inverses, so that one annotated or derived role assignment yields its mirror with no extra rules. - Category definitions are stored inside the data file and serve both human readers and the LLM prompt. When errors occurred, they traced back to a subcase that was described but never illustrated, such as one particular demonstrative pronoun. — For Mova (learn): teacher-tier prompts could be built directly from Mova's label inventory, and the pipeline could flag label subtypes that have a definition but no worked example. - The model must give a written justification with each label. This exposed how it misread the guidelines, for example by counting a merged word as two separate words. — For Mova (learn): requiring rationales from LLM teachers on silver data would let auditors find systematic misreadings and turn them into self-induced repair rules or clearer guidelines. - The experiments showed that a handful of examples per class is already enough and that model choice matters far more. The authors also propose ensembling several models as a fix for the remaining errors. — For Mova (learn): Mova could keep teacher prompts short and spend effort on the choice of teacher, reusing the existing multi-witness agreement idea to combine several teachers' labels. - Rdgai treats differences between two variants as items to classify into a small set of typed categories, ranging from spelling-only to multi-word. — For Mova (tree): disagreements between witness parsers could be sorted the same way, for example into label-only, single-attachment and multi-arc differences, so that rate statistics show where each parser tends to drift. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{turnbull2025rdgai, title = {{Rdgai: Classifying transcriptional changes using Large Language Models with a test case from an Arabic Gospel tradition}}, author = {Turnbull, Robert}, year = {2025}, doi = {10.48550/arxiv.2511.13801}, eprint = {2511.13801}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2511.13801} } ``` - [← previous Proceedings of the Second International Workshop on Construction Grammars and NLP](https://md.research.elamur.ai/free/en/papers/anon-2025-proceedings-second-international-workshop-construction/) - [next → Reverse Probing: Evaluating Knowledge Transfer via Finetuned Task Embeddings for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/anikina-2025-reverse-probing-evaluating-knowledge-transfer/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) ## References 13 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Brown, Tom, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, et al. “Language Models Are Few-Shot Learners.” In Advances in Neural Information Processing Systems , edited by H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, 33:1877–1901. [find](https://scholar.google.com/scholar?q=Brown%2C+Tom%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+et+al%2E+%E2%80%9CLanguage+Models+Are+Few%2DShot+Learners%2E%E2%80%9D+In+Advances+in+Neural+Information+Process) 2. Curran Associates, Inc., 2020. [https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf](https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf). [↗](https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf) 3. Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.” In Proceedings of the 2019 [find](https://scholar.google.com/scholar?q=Devlin%2C+Jacob%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+%E2%80%9CBERT%3A+Pre%2DTraining+of+Deep+Bidirectional+Transformers+for+Language+Understanding%2E%E2%80%9D+In+Proceedings+of+the+2019) 4. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , edited by Jill Burstein, Christy Doran, and Thamar Solorio, 4171–86. Minneapolis, Minnesota: Association for Computational Linguistics, 2019. https://doi.org/10.18653/v1/N19-1423. [↗](https://doi.org/10.18653/v1/n19-1423) 5. Felsenstein, Joseph. Inferring Phylogenies . Sunderland, Massachusetts: Sinauer, 2004. [find](https://scholar.google.com/scholar?q=Felsenstein%2C+Joseph%2E+Inferring+Phylogenies+%2E+Sunderland%2C+Massachusetts%3A+Sinauer%2C+2004%2E) 6. Howard, Jeremy, and Sebastian Ruder. “Universal Language Model Fine-Tuning for Text Classification.” In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , edited by Iryna Gurevych and Yusuke Miyao, 328–39. Melbourne, Australia: Association for Computational Linguistics, 2018. https://doi.org/10.18653/v1/P18-1031. [↗](https://doi.org/10.18653/v1/p18-1031) 7. Ide, Nancy M., and C. M. Sperberg-McQueen. “The TEI: History, Goals and Future.” Computers and the Humanities 29, no. 1 (1995): 5–15. https://doi.org/10.1007/978-94-011-0325-1_2. [↗](https://doi.org/10.1007/978-94-011-0325-1_2) 8. McCollum, Joey, and Robert Turnbull. “teiphy: A Python Package for Converting TEI XML Collations to NEXUS and Other Formats.” Journal of Open Source Software 7, no. 80 (2022): 4879. https://doi.org/10.21105/joss.04879. ———. “Using Bayesian Phylogenetics to Infer Manuscript Transmission History.” Digital Scholarship in the Humanities 39, no. 1 (2024): 258–79. https://doi.org/10.1093/llc/fqad089. [↗](https://doi.org/10.21105/joss.04879) 9. Radford, Alec, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. “Language Models Are Unsupervised Multitask Learners,” 2019. [https://api.semanticscholar.org/CorpusID:160025533](https://api.semanticscholar.org/CorpusID:160025533). [↗](https://api.semanticscholar.org/CorpusID:160025533) 10. Schubert, Erich, and Peter J. Rousseeuw. “Fast and Eager k-Medoids Clustering: O(k) Runtime Improvement of the PAM, CLARA, and CLARANS Algorithms.” Information Systems 101 (2021): 101804. https://doi.org/https://doi.org/10.1016/j.is.2021.101804. [↗](https://doi.org/10.1016/j.is.2021.101804) 11. TEI Consortium. “TEI P5: Guidelines for Electronic Text Encoding and Interchange: Critical Apparatus [v.4.5.0].” [https://www.tei-c.org/release/doc/tei-p5-doc/en/html/TC.html](https://www.tei-c.org/release/doc/tei-p5-doc/en/html/TC.html), 2022. [↗](https://www.tei-c.org/release/doc/tei-p5-doc/en/html/TC.html) 12. Trovato, Paolo. Everything You Always Wanted to Know about Lachmann’s Method: A Non-Standard Handbook of Genealogical Textual Criticism in the Age of Post-Structuralism, Cladistics, and Copy-Text . Padova: libreriauniversitaria.it edizioni, 2014. [find](https://scholar.google.com/scholar?q=Trovato%2C+Paolo%2E+Everything+You+Always+Wanted+to+Know+about+Lachmann%E2%80%99s+Method%3A+A+Non%2DStandard+Handbook+of+Genealogical+Textual+Criticism+in+the+Age+of+Post%2DStructuralism%2C+Cladistics%2C+and+Copy%2DText+%2E+Pa) 13. Turnbull, Robert. Codex Sinaiticus Arabicus and Its Family: A Bayesian Approach . Vol. 66. New Testament Tools, Studies and Documents. Brill, 2025. https://doi.org/10.1163/9789004704619. [↗](https://doi.org/10.1163/9789004704619)