--- title: "Indic-TunedLens — Panchal 2026" description: "Tools for reading the internals of large language models, such as Logit Lens and Tuned Lens, were built mainly for English." authors: ["Mihir Panchal", "Deeksha Varshney", "Mamta", "Asif Ekbal"] year: 2026 venue: "Proceedings of the 13th Workshop on NLP for Similar Languages, Varieties and Dialects" source: "https://aclanthology.org/2026.vardial-1.14/" doi: "10.18653/v1/2026.vardial-1.14" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/panchal-2026-indic-tunedlens-interpreting-multilingual-models/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/panchal-2026-indic-tunedlens-interpreting-multilingual-models/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) / Indic-TunedLens: Interpreting Multilingual Models in Indian Languages # Indic-TunedLens: Interpreting Multilingual Models in Indian Languages **Authors:** Mihir Panchal, Deeksha Varshney, [Mamta](https://md.research.elamur.ai/free/en/authors/mamta/), [Asif Ekbal](https://md.research.elamur.ai/free/en/authors/asif-ekbal/) · **Year:** 2026 · **Venue:** Proceedings of the 13th Workshop on NLP for Similar Languages, Varieties and Dialects · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.vardial-1.14/) · [PDF](https://aclanthology.org/2026.vardial-1.14.pdf) · [DOI 10.18653/v1/2026.vardial-1.14](https://doi.org/10.18653/v1/2026.vardial-1.14) · [OpenAlex](https://openalex.org/W7140133878) **Citations:** 0 citations (openalex-snapshot · 2026-10-14) ## Summary Tools for reading the internals of large language models, such as Logit Lens and Tuned Lens, were built mainly for English. They work poorly on Indian languages, which have rich morphology, flexible word order, and many different scripts. Earlier work also suggests that multilingual LLMs tend to reason in an English-centred latent space. The authors frame this failure as a projection problem and propose Indic-TunedLens. For each layer, the method learns an affine "translator" (a matrix plus a bias). The translator maps intermediate hidden states into the model's output head. It is trained on multilingual Indic text to minimise the KL divergence from the model's own final-layer next-token distribution. Using Sarvam-1 as the base model, the lens is trained on Sangraha data and evaluated on an Indic multilingual MMLU subset. Evaluation covers entropy, layer-wise agreement, rank-based accuracy and token-position accuracy. Indic-TunedLens produces interpretable predictions from the first layer onward, while standard Logit Lens only does so from around layer 20. This gives researchers a practical way to inspect how multilingual models process Indian languages. ## Key points - **Base model:** Sarvam-1, a model with about 1B parameters. It was the only model tested, due to resource limits. - **Training data:** the Sangraha corpus, which spans 11 languages: Hindi, Tamil, Telugu, Marathi, Nepali, Bengali, Gujarati, Panjabi, Kannada, Malayalam and English. - **Evaluation data:** an MMLU subset (Dac Lai et al., 2023) covering 10 of those languages. Panjabi is used for training only, to probe transfer to a related language the lens never saw at test time. - **Comparison with English-centric lenses:** Indic-TunedLens shows smoother entropy decline across layers. Its intermediate predictions are meaningful Hindi tokens, whereas the standard lenses give English-biased, unstable outputs. The paper also reports a roughly 20-layer interpretability gap between Indic-TunedLens and Logit Lens. - **Availability and limitations:** the tool is released on Hugging Face Spaces and the code on GitHub. The authors note three limitations: - only one model was tested; - evaluation is restricted to MMLU; - better interpretability does not translate into downstream task gains. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{panchal2026indic, title = {{Indic-TunedLens: Interpreting Multilingual Models in Indian Languages}}, author = {Panchal, Mihir and Varshney, Deeksha and Mamta and Ekbal, Asif}, booktitle = {Proceedings of the 13th Workshop on NLP for Similar Languages, Varieties and Dialects}, year = {2026}, doi = {10.18653/v1/2026.vardial-1.14}, url = {https://doi.org/10.18653/v1/2026.vardial-1.14} } ``` - [← previous Incorporating Multiword Expressions in Galician Neural Machine Translation: Compositionality, Efficiency, and Performance](https://md.research.elamur.ai/free/en/papers/solla-2026-incorporating-multiword-expressions-galician-neural/) - [next → IndigiEval: Evaluating LLMs in North American Indigenous Languages](https://md.research.elamur.ai/free/en/papers/mainzinger-2026-indigieval-evaluating-llms-north-american/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) ## References 9 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. 2023. Eliciting latent predictions from transformers with the tuned lens. arXiv preprint arXiv:2303.08112. [find](https://scholar.google.com/scholar?q=Nora+Belrose%2C+Zach+Furman%2C+Logan+Smith%2C+Danny+Halawi%2C+Igor+Ostrovsky%2C+Lev+McKinney%2C+Stella+Biderman%2C+and+Jacob+Steinhardt%2E+2023%2E+Eliciting+latent+predictions+from+transformers+with+the+tuned+lens%2E+arX) [our card](https://md.research.elamur.ai/free/en/papers/belrose-2023-eliciting-latent-predictions-transformers-tuned/) 2. Cheng-Ting Chou, George Liu, Jessica Sun, Cole Blondin, Kevin Zhu, Vasu Sharma, and Sean O’Brien. 2025. Causal language control in multilingual transformers via sparse feature steering. arXiv preprint arXiv:2507.13410. [find](https://scholar.google.com/scholar?q=Cheng%2DTing+Chou%2C+George+Liu%2C+Jessica+Sun%2C+Cole+Blondin%2C+Kevin+Zhu%2C+Vasu+Sharma%2C+and+Sean+O%E2%80%99Brien%2E+2025%2E+Causal+language+control+in+multilingual+transformers+via+sparse+feature+steering%2E+arXiv+preprint) 3. Viet Dac Lai, Chien Van Nguyen, Nghia Trung Ngo, Thuat Nguyen, Franck Dernoncourt, Ryan A Rossi, and Thien Huu Nguyen. 2023. Okapi: Instruction-tuned large language models in multiple languages Jawaid Ahmed Siddiqui, Siti Sophiayati Yuhaniz, Ghulam Mujtaba, Safdar Ali Soomro, and Zafar Ali Mahar. 20… [find](https://scholar.google.com/scholar?q=Viet+Dac+Lai%2C+Chien+Van+Nguyen%2C+Nghia+Trung+Ngo%2C+Thuat+Nguyen%2C+Franck+Dernoncourt%2C+Ryan+A+Rossi%2C+and+Thien+Huu+Nguyen%2E+2023%2E+Okapi%3A+Instruction%2Dtuned+large+language+models+in+multiple+languages+Jawaid) 4. Saikrishna Srirampur, Ravi Chandibhamar, and Radhika Mamidi. 2014. Statistical morph analyzer (SMA++) for Indian languages. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects, pages 103–109, Dublin, Ireland. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Saikrishna+Srirampur%2C+Ravi+Chandibhamar%2C+and+Radhika+Mamidi%2E+2014%2E+Statistical+morph+analyzer+%28SMA%2B%2B%29+for+Indian+languages%2E+In+Proceedings+of+the+First+Workshop+on+Applying+NLP+Tools+to+Similar+Langua) 5. Divyansh Subramani and David Bau. 2022. Directional stimulus optimization: Open-ended text generation as activation steering. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Divyansh+Subramani+and+David+Bau%2E+2022%2E+Directional+stimulus+optimization%3A+Open%2Dended+text+generation+as+activation+steering%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natural+Lang) 6. Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Wayne Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. Language-specific neurons: The key to multilingual capabilities in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Tianyi+Tang%2C+Wenyang+Luo%2C+Haoyang+Huang%2C+Dongdong+Zhang%2C+Xiaolei+Wang%2C+Wayne+Xin+Zhao%2C+Furu+Wei%2C+and+Ji%2DRong+Wen%2E+2024%2E+Language%2Dspecific+neurons%3A+The+key+to+multilingual+capabilities+in+large+languag) [our card](https://md.research.elamur.ai/free/en/papers/tang-2024-language-specific-neurons-key-multilingual/) 7. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+Advances+in+neural+information+processing+) 8. (2023). As shown in Table 2, the testing dataset includes samples across ten languages, with relatively balanced distribution ranging from 215 to 277 samples per language. This balanced distribution enables fair comparative analysis while testing the generalization capabilities of our approach acros… [find](https://scholar.google.com/scholar?q=%282023%29%2E+As+shown+in+Table+2%2C+the+testing+dataset+includes+samples+across+ten+languages%2C+with+relatively+balanced+distribution+ranging+from+215+to+277+samples+per+language%2E+This+balanced+distribution+e) 9. Language Bengali English Gujarati Hindi Kannada Malayalam Marathi Nepali Tamil Telugu Andrea W Wen-Yi and David Mimno. 2023. Hyper-polyglot llms: Cross-lingual interpretability in token embeddings. arXiv preprint arXiv:2311.18034. — [arXiv:2311.18034](https://arxiv.org/abs/2311.18034) [find](https://scholar.google.com/scholar?q=Language+Bengali+English+Gujarati+Hindi+Kannada+Malayalam+Marathi+Nepali+Tamil+Telugu+Andrea+W+Wen%2DYi+and+David+Mimno%2E+2023%2E+Hyper%2Dpolyglot+llms%3A+Cross%2Dlingual+interpretability+in+token+embeddings%2E+ar)