---
title: "SELENE — paper by Verma et al. (2026) — summary and ideas"
description: "The Amazon authors propose SELENE, a training-free extension of CFMAD with two parts. 30 references, cited by 1 paper here."
authors: ["Akshay Verma", "Swapnil Gupta", "Deepak Gupta", "Prateek Sircar", "Siddharth Pillai"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)"
source: "https://aclanthology.org/2026.eacl-industry.7/"
doi: "10.18653/v1/2026.eacl-industry.7"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/verma-2026-selene-selective-evidence-weighted-llm/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/verma-2026-selene-selective-evidence-weighted-llm/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/9/) / SELENE: Selective and Evidence-Weighted LLM Debating for Efficient and Reliable Reasoning

# SELENE: Selective and Evidence-Weighted LLM Debating for Efficient and Reliable Reasoning

**Authors:** [Akshay Verma](https://md.research.elamur.ai/free/en/authors/akshay-verma/), [Swapnil Gupta](https://md.research.elamur.ai/free/en/authors/swapnil-gupta/), [Deepak Gupta](https://md.research.elamur.ai/free/en/authors/deepak-gupta/), [Prateek Sircar](https://md.research.elamur.ai/free/en/authors/prateek-sircar/) and [Siddharth Pillai](https://md.research.elamur.ai/free/en/authors/siddharth-pillai/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-industry.7/) · [PDF](https://aclanthology.org/2026.eacl-industry.7.pdf) · [DOI 10.18653/v1/2026.eacl-industry.7](https://doi.org/10.18653/v1/2026.eacl-industry.7) · [OpenAlex](https://openalex.org/W7140093991)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/verma-2026-selene-selective-evidence-weighted-llm/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Multi-agent debate (MAD) lets several LLMs argue before a judge picks an answer, which can curb hallucinations. However, debating every question is expensive, and a lone judge is easily swayed by wording or verbosity. The Amazon authors propose SELENE, a training-free extension of CFMAD with two parts.

- **Selective Debate Initiation (SDI)** inspects the agents' first answers. It measures semantic disagreement through embedding cosine distances, and the gap between each agent's stated confidence and its token log-probability. Only queries scoring high on either signal enter a bounded debate. That debate stops early once disagreement stops changing.
- **Evidence-Weighted Self-Consistency (EWSC)** replaces the single judge with several judgments over paraphrased or subsampled evidence. Each candidate's score is weighted toward low-variance (stable) judgments.

On BoolQ, CosmosQA and a proprietary e-commerce QnA set, SELENE cuts token use by roughly half. Skipped queries lose only slightly in accuracy, and verdicts become clearly more consistent. This makes debate-style reasoning more realistic under production compute budgets.

## Key points
- **SDI skip rates and token cost:** SDI skips debate for 58% of BoolQ, 43% of CosmosQA and 27% of Internal-QnA queries. Relative token cost drops from 3.7× to 1.4× (BoolQ), 3.7× to 1.8× (CosmosQA) and 3.9× to 2.1× (Internal-QnA).
- **Accuracy on skipped queries:** the loss versus full CFMAD debate is small, at about 0.8–1.5 points (e.g., BoolQ 83.6% → 82.1%).
- **Long debates (more than two rounds):** EWSC outperforms CFMAD by +4.9 on BoolQ (86.1), +5.7 on CosmosQA (80.2) and +7.7 on Internal-QnA.
- **Judge Stability across K=3 paraphrased evidence variants:** this metric is 1 minus the variance of judge scores. It rises from 0.84 to 0.93 (BoolQ), 0.79 to 0.89 (CosmosQA) and 0.72 to 0.88 (Internal-QnA).
- **Internal-QnA dataset:** a 20K-sample proprietary e-commerce catalog dataset.
- **Limitation:** the gating thresholds rely on logit-based signals tested on GPT-4-class models, so they may drift on smaller or instruction-tuned LLMs.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Akshay Verma, Swapnil Gupta, Deepak Gupta, Prateek Sircar and Siddharth Pillai. 2026. SELENE: Selective and Evidence-Weighted LLM Debating for Efficient and Reliable Reasoning. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{verma2026selene,
  title = {{SELENE: Selective and Evidence-Weighted LLM Debating for Efficient and Reliable Reasoning}},
  author = {Akshay Verma, Swapnil Gupta, Deepak Gupta, Prateek Sircar and Siddharth Pillai},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.7},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.7}
}
```

- [← previous Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training](https://md.research.elamur.ai/free/en/papers/xia-2026-search-p1-path-centric-reward/)
- [next → Self-Explaining Hate Speech Detection with Moral Rationales](https://md.research.elamur.ai/free/en/papers/vargas-2026-self-explaining-hate-speech-detection/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/9/)

## References

30 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jie Chen, Bowen Zhao, Dian Yu, and Bill Yuchen Lin. 2023. Faithful chain-of-verification improves reasoning in large language models. arXiv preprint arXiv:2310.04383. — [arXiv:2310.04383](https://arxiv.org/abs/2310.04383) [find](https://scholar.google.com/scholar?q=Jie+Chen%2C+Bowen+Zhao%2C+Dian+Yu%2C+and+Bill+Yuchen+Lin%2E+2023%2E+Faithful+chain%2Dof%2Dverification+improves+reasoning+in+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E04383%2E+%E2%80%94+%5BarXiv%3A2310%2E04383%5D%28https%3A%2F%2Farxi)
2. Zhenyu Cui, Hao Wang, Cheng Zhang, and Jie Zhou. 2025. Free-mad: Bias-reduced multi-agent debate via aggregated trajectory voting. arXiv preprint arXiv:2502.02134. — [arXiv:2502.02134](https://arxiv.org/abs/2502.02134) [find](https://scholar.google.com/scholar?q=Zhenyu+Cui%2C+Hao+Wang%2C+Cheng+Zhang%2C+and+Jie+Zhou%2E+2025%2E+Free%2Dmad%3A+Bias%2Dreduced+multi%2Dagent+debate+via+aggregated+trajectory+voting%2E+arXiv+preprint+arXiv%3A2502%2E02134%2E+%E2%80%94+%5BarXiv%3A2502%2E02134%5D%28https%3A%2F%2Farxiv%2Eo)
3. Yilun Du, Jiayuan Li, and Christopher D. Manning. 2023. [Improving factuality and reasoning in language models through multi-agent debate](https://md.research.elamur.ai/free/en/papers/li-2023-chain-knowledge-grounding-large-language/). arXiv preprint arXiv:2305.13269.
4. Wei Fang, Lin Chen, Rui Zhang, Ming Li, and Xiaodong Liu. 2025. Counterfactual multi-agent debate improves factuality and diversity in llm reasoning. arXiv preprint arXiv:2501.04210. — [arXiv:2501.04210](https://arxiv.org/abs/2501.04210) [find](https://scholar.google.com/scholar?q=Wei+Fang%2C+Lin+Chen%2C+Rui+Zhang%2C+Ming+Li%2C+and+Xiaodong+Liu%2E+2025%2E+Counterfactual+multi%2Dagent+debate+improves+factuality+and+diversity+in+llm+reasoning%2E+arXiv+preprint+arXiv%3A2501%2E04210%2E+%E2%80%94+%5BarXiv%3A2501%2E042)
5. Jack Hills and Sam Anadkat. 2023. Using logprobs. [https://cookbook.openai.com/examples/](https://cookbook.openai.com/examples/) using_logprobs. Accessed: 2025-11-03. [↗](https://cookbook.openai.com/examples/)
6. Zeqiu Ji, Yujin Lee, Jason Fries, Danqi Chen, et al. 2023. [Survey on hallucination in large language models](https://md.research.elamur.ai/free/en/papers/rawte-2023-survey-hallucination-large-foundation-models/). arXiv preprint arXiv:2309.05922.
7. Saurav Kadavath, Andy Lin, Nicholas Schiefer, Jacob Hilton, Owain Evans, Samuel R. Bowman, and Andreas Stuhlmüller. 2022. Language models (mostly) know what they know. In Advances in Neural Information Processing Systems (NeurIPS). [find](https://scholar.google.com/scholar?q=Saurav+Kadavath%2C+Andy+Lin%2C+Nicholas+Schiefer%2C+Jacob+Hilton%2C+Owain+Evans%2C+Samuel+R%2E+Bowman%2C+and+Andreas+Stuhlm%C3%BCller%2E+2022%2E+Language+models+%28mostly%29+know+what+they+know%2E+In+Advances+in+Neural+Informatio)
8. Anant Khandelwal, Happy Mittal, Shreyas Sunil Kulkarni, and Deepak Gupta. 2023. Large scale generative multimodal attribute extraction for e-commerce attributes. In ACL (industry), pages 305–312. [find](https://scholar.google.com/scholar?q=Anant+Khandelwal%2C+Happy+Mittal%2C+Shreyas+Sunil+Kulkarni%2C+and+Deepak+Gupta%2E+2023%2E+Large+scale+generative+multimodal+attribute+extraction+for+e%2Dcommerce+attributes%2E+In+ACL+%28industry%29%2C+pages+305%E2%80%93312%2E)
9. Jie Li, Jian Zhou, Tao Lin, Han Wang, and Fang Liu. 2024. Calibrating large language models with log-probability guidance for factual reliability. arXiv preprint arXiv:2402.08032. — [arXiv:2402.08032](https://arxiv.org/abs/2402.08032) [find](https://scholar.google.com/scholar?q=Jie+Li%2C+Jian+Zhou%2C+Tao+Lin%2C+Han+Wang%2C+and+Fang+Liu%2E+2024%2E+Calibrating+large+language+models+with+log%2Dprobability+guidance+for+factual+reliability%2E+arXiv+preprint+arXiv%3A2402%2E08032%2E+%E2%80%94+%5BarXiv%3A2402%2E08032%5D)
10. Weizhe Liang, Yanzhe Zhang, Yixin Kwon, Tianyi Ye, Yizhong Zheng, Jason Weston, Luke Zettlemoyer, and Mark Yatskar. 2023. [Encouraging divergent thinking in large language models through multi-agent debate](https://md.research.elamur.ai/free/en/papers/du-2023-improving-factuality-reasoning-language-models/). arXiv preprint arXiv:2305.14325.
11. Stephanie Lin, Jacob Hilton, and Owain Evans. 2023. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL).
12. Potsawee Manakul, Alham Liusie, and Mark J. F. Gales. 2023. Selfcheckgpt: Detecting llm hallucinations via token-level sampling. arXiv preprint arXiv:2303.08896. [find](https://scholar.google.com/scholar?q=Potsawee+Manakul%2C+Alham+Liusie%2C+and+Mark+J%2E+F%2E+Gales%2E+2023%2E+Selfcheckgpt%3A+Detecting+llm+hallucinations+via+token%2Dlevel+sampling%2E+arXiv+preprint+arXiv%3A2303%2E08896%2E)
13. Noah Shinn, Francesco Cassano, Bradley Labash, Dinesh Gopinath, Matthew Finlayson, Anca Dragan, Dorsa Sadigh, and Noah Goodman. 2023. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). arXiv preprint arXiv:2303.11366.
14. Chenglei Si, Hongxin Xie, Fangyuan Xu, Yue Zhang, Jing Ma, and Rui Zhang. 2023. Measuring uncertainty in large language models for hallucination detection. arXiv preprint arXiv:2305.13669. [find](https://scholar.google.com/scholar?q=Chenglei+Si%2C+Hongxin+Xie%2C+Fangyuan+Xu%2C+Yue+Zhang%2C+Jing+Ma%2C+and+Rui+Zhang%2E+2023%2E+Measuring+uncertainty+in+large+language+models+for+hallucination+detection%2E+arXiv+preprint+arXiv%3A2305%2E13669%2E)
15. Prateek Sircar, Aniket Chakrabarti, Deepak Gupta, and Anirban Majumdar. 2022. Distantly supervised aspect clustering and naming for e-commerce reviews. In NAACL-HLT (Industry Papers), pages 94–102. [find](https://scholar.google.com/scholar?q=Prateek+Sircar%2C+Aniket+Chakrabarti%2C+Deepak+Gupta%2C+and+Anirban+Majumdar%2E+2022%2E+Distantly+supervised+aspect+clustering+and+naming+for+e%2Dcommerce+reviews%2E+In+NAACL%2DHLT+%28Industry+Papers%29%2C+pages+94%E2%80%93102%2E)
16. Vinay Kumar Verma, Shreyas Sunil Kulkarni, Happy Mittal, and Deepak Gupta. 2025. Moemoe: Question guided dense and scalable sparse mixture-of-expert for multi-source multi-modal answering. arXiv preprint arXiv:2503.06296. [find](https://scholar.google.com/scholar?q=Vinay+Kumar+Verma%2C+Shreyas+Sunil+Kulkarni%2C+Happy+Mittal%2C+and+Deepak+Gupta%2E+2025%2E+Moemoe%3A+Question+guided+dense+and+scalable+sparse+mixture%2Dof%2Dexpert+for+multi%2Dsource+multi%2Dmodal+answering%2E+arXiv+prepr)
17. Hao Wang, Ming Zhou, Yue Zhang, and Zhiyuan Li. 2024. Cross-examination: Improving judge reliability in multi-agent llm debates. arXiv preprint arXiv:2403.04127. — [arXiv:2403.04127](https://arxiv.org/abs/2403.04127) [find](https://scholar.google.com/scholar?q=Hao+Wang%2C+Ming+Zhou%2C+Yue+Zhang%2C+and+Zhiyuan+Li%2E+2024%2E+Cross%2Dexamination%3A+Improving+judge+reliability+in+multi%2Dagent+llm+debates%2E+arXiv+preprint+arXiv%3A2403%2E04127%2E+%E2%80%94+%5BarXiv%3A2403%2E04127%5D%28https%3A%2F%2Farxiv%2Eorg)
18. Xuezhi Wang, Jason Wei, Dale Schuurmans, Ed Chi, Quoc Le, and Denny Zhou. 2022. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). In International Conference on Learning Representations (ICLR).
19. Yizhong Wang and et al. 2024. [Large language models are inconsistent and biased evaluators](https://md.research.elamur.ai/free/en/papers/stureborg-2024-large-language-models-inconsistent-biased/). arXiv preprint arXiv:2405.01724.
20. Model BoolQ CosmosQA Internal-QnA GPT-4o-mini CFMAD SELENE ∆ (vs CFMAD) 84.1 85.4 +1.3 74.0 76.1 +2.1 – – +8.6 [find](https://scholar.google.com/scholar?q=Model+BoolQ+CosmosQA+Internal%2DQnA+GPT%2D4o%2Dmini+CFMAD+SELENE+%E2%88%86+%28vs+CFMAD%29+84%2E1+85%2E4+%2B1%2E3+74%2E0+76%2E1+%2B2%2E1+%E2%80%93+%E2%80%93+%2B8%2E6)
21. Yizhong Wang, Han Zhou, Tianyi Zhang, and Zhiyuan Liu. 2023. Self-contrast: Aligning large language models via contrastive self-refinement. In Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Han+Zhou%2C+Tianyi+Zhang%2C+and+Zhiyuan+Liu%2E+2023%2E+Self%2Dcontrast%3A+Aligning+large+language+models+via+contrastive+self%2Drefinement%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2E)
22. CFMAD Claude 3 Haiku SELENE ∆ (vs CFMAD) 83.2 84.6 +1.4 73.5 75.7 +2.2 – – +8.2 [find](https://scholar.google.com/scholar?q=CFMAD+Claude+3+Haiku+SELENE+%E2%88%86+%28vs+CFMAD%29+83%2E2+84%2E6+%2B1%2E4+73%2E5+75%2E7+%2B2%2E2+%E2%80%93+%E2%80%93+%2B8%2E2)
23. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems (NeurIPS).
24. Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. 2025. [When more is less: Understanding chain-of-thought length in llms](https://md.research.elamur.ai/free/en/papers/wu-2025-more-less-understanding-chain-thought/). ArXiv, abs/2502.07266.
25. Rui Yang, Wei Lin, Hao Chen, and Lei Zhou. 2024. Judge summarization: Enhancing verdict quality in multi-agent llm debates. arXiv preprint arXiv:2402.01987. — [arXiv:2402.01987](https://arxiv.org/abs/2402.01987) [find](https://scholar.google.com/scholar?q=Rui+Yang%2C+Wei+Lin%2C+Hao+Chen%2C+and+Lei+Zhou%2E+2024%2E+Judge+summarization%3A+Enhancing+verdict+quality+in+multi%2Dagent+llm+debates%2E+arXiv+preprint+arXiv%3A2402%2E01987%2E+%E2%80%94+%5BarXiv%3A2402%2E01987%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
26. Wenqi Zhang, Yongliang Shen, Linjuan Wu, Qiuying Peng, Jun Wang, Yue Ting Zhuang, and Weiming Lu. 2024a. [Self-contrast: Better reflection through inconsistent solving perspectives](https://md.research.elamur.ai/free/en/papers/zhang-2024-self-contrast-better-reflection-through/). In Annual Meeting of the Association for Computational Linguistics.
27. Yixin Zhang, Yuchen Chen, Hao Li, Qian Liu, and Hongtao Xu. 2024b. Lm-detect: Likelihood-based hallucination detection in large language models. arXiv preprint arXiv:2404.06112. *Table 6: Cross-LLM comparison of accuracy (%) on BoolQ, CosmosQA, and Internal-QnA. Both GPT-4omini and Claude 3 Haiku ex… — [arXiv:2404.06112](https://arxiv.org/abs/2404.06112) [find](https://scholar.google.com/scholar?q=Yixin+Zhang%2C+Yuchen+Chen%2C+Hao+Li%2C+Qian+Liu%2C+and+Hongtao+Xu%2E+2024b%2E+Lm%2Ddetect%3A+Likelihood%2Dbased+hallucination+detection+in+large+language+models%2E+arXiv+preprint+arXiv%3A2404%2E06112%2E+%2ATable+6%3A+Cross%2DLLM+co)
28. B All prompts are executed using GPT-4-Turbo-2025-04-09 with temperature = 0.3 and top-p = 0.9. Each box below shows the actual prompt used in SE-LENE at various stages. [find](https://scholar.google.com/scholar?q=B+All+prompts+are+executed+using+GPT%2D4%2DTurbo%2D2025%2D04%2D09+with+temperature+%3D+0%2E3+and+top%2Dp+%3D+0%2E9%2E+Each+box+below+shows+the+actual+prompt+used+in+SE%2DLENE+at+various+stages%2E)
29. Ke Zhou, Yuan Zhao, Pengfei Wang, and Chenguang Wang. 2024a. Factuality assessment of large language models via token probability entropy. arXiv preprint arXiv:2401.03455. — [arXiv:2401.03455](https://arxiv.org/abs/2401.03455) [find](https://scholar.google.com/scholar?q=Ke+Zhou%2C+Yuan+Zhao%2C+Pengfei+Wang%2C+and+Chenguang+Wang%2E+2024a%2E+Factuality+assessment+of+large+language+models+via+token+probability+entropy%2E+arXiv+preprint+arXiv%3A2401%2E03455%2E+%E2%80%94+%5BarXiv%3A2401%2E03455%5D%28https%3A%2F)
30. Liang Zhou, Ying Zhu, and et al. 2024b. Larger and more instructable large language models become less robust to prompt variations. Nature, 633:679–687. [find](https://scholar.google.com/scholar?q=Liang+Zhou%2C+Ying+Zhu%2C+and+et+al%2E+2024b%2E+Larger+and+more+instructable+large+language+models+become+less+robust+to+prompt+variations%2E+Nature%2C+633%3A679%E2%80%93687%2E)

*Updated 2 November 2026*