--- title: "CrossOracle: An Agentic Framework for Real-Time — Singh 2026" description: "CrossOracle is a deployed cascaded spoken dialogue system for real-time voice conversation in Hinglish (Hindi-English code-switching)." authors: ["Yash Raj Singh"] year: 2026 venue: "Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue" source: "https://aclanthology.org/2026.sigdial-1.18/" doi: "10.18653/v1/2026.sigdial-1.18" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/singh-2026-crossoracle-agentic-framework-real-time/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/singh-2026-crossoracle-agentic-framework-real-time/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) / CrossOracle: An Agentic Framework for Real-Time Expert Personas with Parallelized Acoustic Synthesis # CrossOracle: An Agentic Framework for Real-Time Expert Personas with Parallelized Acoustic Synthesis **Authors:** Yash Raj Singh · **Year:** 2026 · **Venue:** Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.sigdial-1.18/) · [PDF](https://aclanthology.org/2026.sigdial-1.18.pdf) · [DOI 10.18653/v1/2026.sigdial-1.18](https://doi.org/10.18653/v1/2026.sigdial-1.18) · [OpenAlex](https://openalex.org/W7214005291) **Citations:** 0 citations (openalex-snapshot · 2026-10-14) ## Summary CrossOracle is a deployed cascaded spoken dialogue system for real-time voice conversation in Hinglish (Hindi-English code-switching). It offers eleven expert personas, such as Doctor, Lawyer and Astrologer, plus a user-defined persona option. The author targets the delay that builds up when ASR, an LLM and TTS run in sequence. The paper argues that a cascade is more controllable and cheaper than end-to-end speech-to-speech models like AudioPaLM, which also handle mixed-language input poorly. The system is a Node.js WebSocket layer over Azure Cognitive Services and ElevenLabs Turbo v2.5. It cuts the streaming LLM output into clause-sized pieces using script-aware rules and synthesises them in a background queue while generation continues. An abort mechanism cancels all pending work when the user interrupts. A filter rejects ASR outputs dominated by spurious Latin text. Server logs from 8,000 turns show faster first audio and lower RTF than a fixed-size buffering baseline, with uninterrupted playback. The work offers a practical recipe for low-latency code-switched voice agents, although perceived speech quality (MOS) has not yet been evaluated. ## Key points - The checkFlush() chunking heuristic sends text to TTS on three triggers. In priority order, these are a Devanagari danda, Latin sentence-final punctuation (with an abbreviation blocklist), and newlines, dashes or ellipses. Chunks must be at least 30 characters and are capped at 220. - Across 8,000 turns, CrossOracle reached a mean TTFB of 4.9 s, compared with 6.5 s for a naive fixed-token buffer. Mean TTS RTF was 0.107 versus 0.134. - Barge-in handling succeeded in 100% of cases, with server-side abort taking under 1 ms. Cached persona greetings play in about 866 ms. - ASR is fixed to the hi-IN locale. Transcripts with a Devanagari character ratio below 0.1 and length above 8 characters are discarded. Accepted transcripts averaged a ratio of 0.61, while hallucinated Latin output averaged about 0.02. - One edge case broke the playback guarantee: a lone emoji token produced an RTF of 2.513. The author has since added a regex filter that strips such characters before synthesis. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · [Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{singh2026crossoracle, title = {{CrossOracle: An Agentic Framework for Real-Time Expert Personas with Parallelized Acoustic Synthesis}}, author = {Singh, Yash Raj}, booktitle = {Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue}, year = {2026}, doi = {10.18653/v1/2026.sigdial-1.18}, url = {https://doi.org/10.18653/v1/2026.sigdial-1.18} } ``` - [← previous Cross-lingual Lexical Semantic Change in Romance Languages](https://md.research.elamur.ai/free/en/papers/uban-2026-cross-lingual-lexical-semantic-change/) - [next → Crowdsourcing Piedmontese to Test LLMs on Non-Standard Orthography](https://md.research.elamur.ai/free/en/papers/vico-2026-crowdsourcing-piedmontese-test-llms-non/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) ## References 6 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Jaspreet Kaur, Gurpreet Singh, and Rajneesh Sharma. 2020. A review of Devanagari script and its challenges. Journal of Indic Languages. [find](https://scholar.google.com/scholar?q=Jaspreet+Kaur%2C+Gurpreet+Singh%2C+and+Rajneesh+Sharma%2E+2020%2E+A+review+of+Devanagari+script+and+its+challenges%2E+Journal+of+Indic+Languages%2E) 2. Stephen C Levinson. 2015. Turn-taking in human communication: Origins and implications for language processing. Trends in Cognitive Sciences, 20(1):6– 14. [find](https://scholar.google.com/scholar?q=Stephen+C+Levinson%2E+2015%2E+Turn%2Dtaking+in+human+communication%3A+Origins+and+implications+for+language+processing%2E+Trends+in+Cognitive+Sciences%2C+20%281%29%3A6%E2%80%93+14%2E) 3. Kentaro Mitsui, Soumi Maiti, and Shinji Watanabe. 2024. PSLM: Parallel generation of text and speech with LLMs for low-latency spoken dialogue systems. arXiv preprint arXiv:2406.12428. [find](https://scholar.google.com/scholar?q=Kentaro+Mitsui%2C+Soumi+Maiti%2C+and+Shinji+Watanabe%2E+2024%2E+PSLM%3A+Parallel+generation+of+text+and+speech+with+LLMs+for+low%2Dlatency+spoken+dialogue+systems%2E+arXiv+preprint+arXiv%3A2406%2E12428%2E) 4. OpenAI. 2023. GPT-4 technical report. Technical report, OpenAI. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2023%2E+GPT%2D4+technical+report%2E+Technical+report%2C+OpenAI%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/) 5. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jong+Wook+Kim%2C+Tao+Xu%2C+Greg+Brockman%2C+Christine+McLeavey%2C+and+Ilya+Sutskever%2E+2023%2E+Robust+speech+recognition+via+large%2Dscale+weak+supervision%2E+In+International+Conference+on+Machine+Lea) [our card](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/) 6. Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. 2021. FastSpeech 2: Fast and high-quality end-to-end text to speech. In [find](https://scholar.google.com/scholar?q=Yi+Ren%2C+Chenxu+Hu%2C+Xu+Tan%2C+Tao+Qin%2C+Sheng+Zhao%2C+Zhou+Zhao%2C+and+Tie%2DYan+Liu%2E+2021%2E+FastSpeech+2%3A+Fast+and+high%2Dquality+end%2Dto%2Dend+text+to+speech%2E+In)