---
title: "ILSIC — paper by Paul et al. (2026) — summary and ideas"
description: "Real users, however, describe their problems informally and without legal vocabulary. 24 references."
authors: ["Shounak Paul", "Raghav Dogra", "Pawan Goyal", "Saptarshi Ghosh"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: EACL 2026"
source: "https://aclanthology.org/2026.findings-eacl.354/"
doi: "10.18653/v1/2026.findings-eacl.354"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/paul-2026-ilsic-corpora-identifying-indian-legal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/paul-2026-ilsic-corpora-identifying-indian-legal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/33/) / ILSIC: Corpora for Identifying Indian Legal Statutes from Queries by Laymen

# ILSIC: Corpora for Identifying Indian Legal Statutes from Queries by Laymen

**Authors:** Shounak Paul, Raghav Dogra, [Pawan Goyal](https://md.research.elamur.ai/free/en/authors/pawan-goyal/) and [Saptarshi Ghosh](https://md.research.elamur.ai/free/en/authors/saptarshi-ghosh/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.354/) · [PDF](https://aclanthology.org/2026.findings-eacl.354.pdf) · [DOI 10.18653/v1/2026.findings-eacl.354](https://doi.org/10.18653/v1/2026.findings-eacl.354) · [OpenAlex](https://openalex.org/W7140117496)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Most work on Legal Statute Identification (LSI) trains and evaluates models on case facts taken from court judgments. Real users, however, describe their problems informally and without legal vocabulary. The authors build ILSIC, an English-language resource for Indian law that supports two goals: benchmarking LSI on layperson questions, and directly comparing court-derived and layperson training data over the same set of statutes. They scraped about 30k question threads from the forum kaanoon.com. Gold statutes come from the lawyers' replies, which were mined with GPT-3.5 Turbo and then normalized using regular expressions and fuzzy matching.

The benchmark covers lexical and dense retrievers as well as LLMs, tested under zero-shot, few-shot, RAG and fine-tuning setups. Every system scores below 35% µF1. Models trained only on court facts perform poorly on lay questions. Court facts rewritten into lay style by an LLM give no improvement. Sequential court-then-lay training helps in only some cases. The resource exposes a genuine domain gap that matters for building legal assistants aimed at citizens.

## Key points
- **ILSIC-Lay**: 8,127 lay queries labeled with 569 Indian statutes (567 in the introduction), split 6,465 / 826 / 836 for train, validation and test.
- **ILSIC-Multi**: covers 399 statutes and has two training groups, one of 5,793 lay queries and one of 12,930 court facts. Both are evaluated on a shared test set of 757 lay queries.
- **Models tested**: BM25, SBERT, SAILER, GPT-4.1, Llama-3 and Gemma-3. GPT-4.1 zero-shot achieves the best macro-F1, while fine-tuned Llama-3 and Gemma-3 achieve higher µF1.
- **Court vs. lay training**:
  - Training only on court data falls below zero-shot µF1.
  - Synthetic lay queries, produced from court facts with GPT-4o-mini, do not help.
  - Court-to-lay transfer learning noticeably improves only Llama-3.
  - All models struggle with rare statutes.
- **Availability**: data, code and trained models are released at github.com/Law-AI/ilsic under CC BY-NC-SA 4.0. The data comes from a single forum.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Indian Institute of Technology Kharagpur](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-kharagpur/) — University, Kharagpur, India · 24 works · 714 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works · [🤝 Political Science and International Relations](https://md.research.elamur.ai/free/en/shelves/social-sciences/political-science-and-international-relations/) · 182 works

🧭 Topics (1): [⚖️ Legal AI](https://md.research.elamur.ai/free/en/topics/legal-ai/) · 134 works

💡 Concepts (1): [💡 Latent semantic indexing](https://md.research.elamur.ai/free/en/concepts/latent-semantic-indexing/) · 29 works

## Cite

Cite as: Shounak Paul, Raghav Dogra, Pawan Goyal and Saptarshi Ghosh. 2026. ILSIC: Corpora for Identifying Indian Legal Statutes from Queries by Laymen. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{paul2026ilsic,
  title = {{ILSIC: Corpora for Identifying Indian Legal Statutes from Queries by Laymen}},
  author = {Shounak Paul, Raghav Dogra, Pawan Goyal and Saptarshi Ghosh},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.354},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.354}
}
```

- [← previous Illusions of the Gold Standard: A Large-scale Analysis of Human Evaluation Protocols for Long-form Text Generation](https://md.research.elamur.ai/free/en/papers/mei-2026-illusions-gold-standard-large-scale/)
- [next → ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation](https://md.research.elamur.ai/free/en/papers/abdaljalil-2026-imageeval-2026-culturally-grounded-arabic/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/33/)

## References

24 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Paheli Bhattacharya, Kripabandhu Ghosh, Saptarshi Ghosh, Arindam Pal, Parth Mehta, Arnab Bhattacharya, and Prasenjit Majumder. 2019. Overview of the fire 2019 aila track: Artificial intelligence for legal assistance. In FIRE (working notes), pages 1– 12. [find](https://scholar.google.com/scholar?q=Paheli+Bhattacharya%2C+Kripabandhu+Ghosh%2C+Saptarshi+Ghosh%2C+Arindam+Pal%2C+Parth+Mehta%2C+Arnab+Bhattacharya%2C+and+Prasenjit+Majumder%2E+2019%2E+Overview+of+the+fire+2019+aila+track%3A+Artificial+intelligence+for+l)
2. Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. 2019. [Neural legal judgment prediction in english](https://md.research.elamur.ai/free/en/papers/chalkidis-2019-neural-legal-judgment-prediction-english/). arXiv preprint arXiv:1906.02059. [↗](https://arxiv.org/abs/1906.02059)
3. Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, Nikolaos Aletras, Ion Androutsopoulos, and Prodromos Malakasiotis. 2021. [Paragraph-level rationale extraction through regularization: A case study on European court of human rights cases](https://md.research.elamur.ai/free/en/papers/chalkidis-2021-paragraph-level-rationale-extraction-through/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 226–241, Online. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2103.13084)
4. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. [Qlora: Efficient finetuning of quantized llms](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/). arXiv preprint arXiv:2305.14314. [↗](https://arxiv.org/abs/2305.14314)
5. Dinh-Truong Do, Son T Luu, Trang Pham, Trung Vo, Nguyen-Hoang Chu, Quang-Huy Chu, Cuong Nguyen, Minh Nguyen, An Trieu, Dat Nguyen, et al. 2024. A summary of the alqac 2024 competition. In 2024 16th International Conference on Knowledge and System Engineering (KSE), pages 422–427. IEEE. [find](https://scholar.google.com/scholar?q=Dinh%2DTruong+Do%2C+Son+T+Luu%2C+Trang+Pham%2C+Trung+Vo%2C+Nguyen%2DHoang+Chu%2C+Quang%2DHuy+Chu%2C+Cuong+Nguyen%2C+Minh+Nguyen%2C+An+Trieu%2C+Dat+Nguyen%2C+et+al%2E+2024%2E+A+summary+of+the+alqac+2024+competition%2E+In+2024+16th+In)
6. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv e-prints, pages arXiv–2407.
7. Abhinav Joshi, Akshat Sharma, Sai Kiran Tanikella, and Ashutosh Modi. 2023. U-CREAT: Unsupervised case retrieval using events extrAcTion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13899–13915, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Abhinav+Joshi%2C+Akshat+Sharma%2C+Sai+Kiran+Tanikella%2C+and+Ashutosh+Modi%2E+2023%2E+U%2DCREAT%3A+Unsupervised+case+retrieval+using+events+extrAcTion%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+f)
8. Prathamesh Kalamkar, Astha Agarwal, Aman Tiwari, Smita Gupta, Saurabh Karn, and Vivek Raghavan. 2022. Named entity recognition in indian court judgments. arXiv preprint arXiv:2211.03442. [↗](https://arxiv.org/abs/2211.03442)
9. Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, et al. 2025. Gemma 3 technical report. CoRR. [find](https://scholar.google.com/scholar?q=Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+Louis+Rouillard%2C+et+al%2E+2025%2E+Gemma+3+technical+repor)
10. Haitao Li, Qingyao Ai, Jia Chen, Qian Dong, Yueyue Wu, Yiqun Liu, Chong Chen, and Qi Tian. 2023a. Sailer: Structure-aware pre-trained language model for legal case retrieval. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’23, page 1035–1044, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Haitao+Li%2C+Qingyao+Ai%2C+Jia+Chen%2C+Qian+Dong%2C+Yueyue+Wu%2C+Yiqun+Liu%2C+Chong+Chen%2C+and+Qi+Tian%2E+2023a%2E+Sailer%3A+Structure%2Daware+pre%2Dtrained+language+model+for+legal+case+retrieval%2E+In+Proceedings+of+the+46t)
11. Haitao Li, You Chen, Zhekai Ge, Qingyao Ai, Yiqun Liu, Quan Zhou, and Shuai Huo. 2024. Towards an in-depth comprehension of case relevance for better legal retrieval. In JSAI International Symposium on Artificial Intelligence, pages 212–227. Springer. [find](https://scholar.google.com/scholar?q=Haitao+Li%2C+You+Chen%2C+Zhekai+Ge%2C+Qingyao+Ai%2C+Yiqun+Liu%2C+Quan+Zhou%2C+and+Shuai+Huo%2E+2024%2E+Towards+an+in%2Ddepth+comprehension+of+case+relevance+for+better+legal+retrieval%2E+In+JSAI+International+Symposium+o)
12. Haitao Li, Weihang Su, Changyue Wang, Yueyue Wu, Qingyao Ai, and Yiqun Liu. 2023b. Thuir@coliee 2023: Incorporating structural knowledge into pre-trained language models for legal case retrieval. [↗](https://doi.org/10.48550/arxiv.2305.06812)
13. Ehsan Lotfi, Nikolay Banar, Nerses Yuzbashyan, and Walter Daelemans. 2024. [Bilingual bsard: Extending statutory article retrieval to dutch](https://md.research.elamur.ai/free/en/papers/lotfi-2025-bilingual-bsard-extending-statutory-article/). arXiv preprint arXiv:2412.07462. [↗](https://arxiv.org/abs/2412.07462)
14. Antoine Louis and Gerasimos Spanakis. 2021. A statutory article retrieval dataset in french. arXiv preprint arXiv:2108.11792. [↗](https://arxiv.org/abs/2108.11792)
15. Chau Nguyen, Son T Luu, Thanh Tran, An Trieu, Anh Dang, Dat Nguyen, Hiep Nguyen, Tin Pham, Trang Pham, Thien-Trung Vo, et al. 2023. A summary of the alqac 2023 competition. In 2023 15th International Conference on Knowledge and Systems Engineering (KSE), pages 1–6. IEEE. [find](https://scholar.google.com/scholar?q=Chau+Nguyen%2C+Son+T+Luu%2C+Thanh+Tran%2C+An+Trieu%2C+Anh+Dang%2C+Dat+Nguyen%2C+Hiep+Nguyen%2C+Tin+Pham%2C+Trang+Pham%2C+Thien%2DTrung+Vo%2C+et+al%2E+2023%2E+A+summary+of+the+alqac+2023+competition%2E+In+2023+15th+International+)
16. Shounak Paul, Pawan Goyal, and Saptarshi Ghosh. 2022. Lesicin: A heterogeneous graph-based approach for automatic legal statute identification from indian legal documents. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 11139–11146. [find](https://scholar.google.com/scholar?q=Shounak+Paul%2C+Pawan+Goyal%2C+and+Saptarshi+Ghosh%2E+2022%2E+Lesicin%3A+A+heterogeneous+graph%2Dbased+approach+for+automatic+legal+statute+identification+from+indian+legal+documents%2E+In+Proceedings+of+the+AAAI+c)
17. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
18. Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333–389. [↗](https://doi.org/10.1561/1500000019)
19. Gerard Salton and Christopher Buckley. 1988. Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5):513– 523. [↗](https://doi.org/10.1016/0306-4573(88)90021-0)
20. Weihang Su, Yiran Hu, Anzhe Xie, Qingyao Ai, Quezi Bing, Ning Zheng, Yun Liu, Weixing Shen, and Yiqun Liu. 2024. Stard: A chinese statute retrieval dataset derived from real-life queries by non-professionals. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10658–10671. [find](https://scholar.google.com/scholar?q=Weihang+Su%2C+Yiran+Hu%2C+Anzhe+Xie%2C+Qingyao+Ai%2C+Quezi+Bing%2C+Ning+Zheng%2C+Yun+Liu%2C+Weixing+Shen%2C+and+Yiqun+Liu%2E+2024%2E+Stard%3A+A+chinese+statute+retrieval+dataset+derived+from+real%2Dlife+queries+by+non%2Dprofes)
21. Pengfei Wang, Yu Fan, Shuzi Niu, Ze Yang, Yongfeng Zhang, and Jiafeng Guo. 2019. Hierarchical matching network for crime classification. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’19, page 325–334, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Pengfei+Wang%2C+Yu+Fan%2C+Shuzi+Niu%2C+Ze+Yang%2C+Yongfeng+Zhang%2C+and+Jiafeng+Guo%2E+2019%2E+Hierarchical+matching+network+for+crime+classification%2E+In+Proceedings+of+the+42nd+International+ACM+SIGIR+Conference+o)
22. Chaojun Xiao, Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Yansong Feng, Xianpei Han, Zhen Hu, Heng Wang, et al. 2018. [Cail2018: A large-scale legal dataset for judgment prediction](https://md.research.elamur.ai/free/en/papers/xiao-2018-cail2018-large-scale-legal-dataset/). arXiv preprint arXiv:1807.02478. [↗](https://arxiv.org/abs/1807.02478)
23. Yiming Zeng, Ruili Wang, John Zeleznikow, and Elizabeth Kemp. 2007. A knowledge representation model for the intelligent retrieval of legal cases. International Journal of Law and Information Technology, 15(3):299–319. [↗](https://doi.org/10.1093/ijlit/eal023)
24. Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Chaojun Xiao, Zhiyuan Liu, and Maosong Sun. 2018. [Legal judgment prediction via topological learning](https://md.research.elamur.ai/free/en/papers/zhong-2018-legal-judgment-prediction-topological-learning/). In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 3540– 3549.

*Updated 1 November 2026*