--- title: "M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box" description: "This paper presents M4, a benchmark for black-box detection, where a detector sees only the text and… 3 references, cited by 88 papers here." authors: ["Yuxia Wang", "Jonibek Mansurov", "Petar Ivanov", "Jinyan Su", "Artem Shelmanov", "Akim Tsvigun", "Chenxi Whitehouse", "Osama Mohammed Afzal", "Tarek Mahmoud", "Toru Sasaki", "Thomas Arnold", "Alham Fikri Aji", "Nizar Habash", "Iryna Gurevych", "Preslav Nakov"] year: 2023 venue: "arXiv:2305.14902" source: "https://arxiv.org/abs/2305.14902" doi: "10.48550/arxiv.2305.14902" arxiv: "2305.14902" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/wang-2023-m4-multi-generator-multi-domain/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2023-m4-multi-generator-multi-domain/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/) / M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection # M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection **Authors:** [Yuxia Wang](https://md.research.elamur.ai/free/en/authors/yuxia-wang/), [Jonibek Mansurov](https://md.research.elamur.ai/free/en/authors/jonibek-mansurov/), Petar Ivanov, [Jinyan Su](https://md.research.elamur.ai/free/en/authors/jinyan-su/), [Artem Shelmanov](https://md.research.elamur.ai/free/en/authors/artem-shelmanov/), [Akim Tsvigun](https://md.research.elamur.ai/free/en/authors/akim-tsvigun/), [Chenxi Whitehouse](https://md.research.elamur.ai/free/en/authors/chenxi-whitehouse/), [Osama Mohammed Afzal](https://md.research.elamur.ai/free/en/authors/osama-mohammed-afzal/), Tarek Mahmoud, Toru Sasaki, Thomas Arnold, [Alham Fikri Aji](https://md.research.elamur.ai/free/en/authors/alham-fikri-aji/), [Nizar Habash](https://md.research.elamur.ai/free/en/authors/nizar-habash/), [Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/) and [Preslav Nakov](https://md.research.elamur.ai/free/en/authors/preslav-nakov/) · **Year:** 2023 · **Venue:** arXiv:2305.14902 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2305.14902) · [PDF](https://arxiv.org/pdf/2305.14902) · [DOI 10.48550/arxiv.2305.14902](https://doi.org/10.48550/arxiv.2305.14902) · [arXiv 2305.14902](https://arxiv.org/abs/2305.14902) · [OpenAlex](https://openalex.org/W4378465121) **Citations:** 19 citations (OpenAlex, 2026-10-14) · [88](https://md.research.elamur.ai/free/en/papers/wang-2023-m4-multi-generator-multi-domain/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary People can barely tell LLM output from human writing, and earlier detection resources were narrow: mostly English-only, built around one generator such as ChatGPT, or limited to a single domain like news. This paper presents M4, a benchmark for black-box detection, where a detector sees only the text and has no access to the model that may have produced it. The authors combined existing human-written corpora (Wikipedia, WikiHow, Reddit ELI5, arXiv, PeerRead, Baike/Web QA, RuATD, news) with outputs from six LLMs, each prompted in several styles. They applied light cleaning and checked quality at the model level. Seven detectors were then evaluated across domains, generators, languages and time periods. The main finding is that detectors transfer poorly to domains or generators not seen during training, and in those cases they tend to label machine text as human-written. The problem is therefore far from solved. M4 is publicly released and was later extended into SemEval-2024 Task 8, giving the field a shared testbed for building more robust detectors. ## Key points - **Size:** About 147k human–machine parallel items: roughly 102k in English and 45k in other languages (9k each for Chinese, Russian and Bulgarian; 6k each for Urdu, Indonesian and Arabic). Over 10M additional non-parallel human texts are included. - **Generators:** GPT-4, ChatGPT, text-davinci-003, Cohere, Dolly-v2 and BLOOMz 176B. Each was given 2–8 prompts in different styles, such as simple explanations or a formal academic tone. - **Length and cleaning:** Texts have a minimum length of 1,000 characters, or the equivalent for non-English languages. Formatting artifacts such as bullet points, repeated newlines and Wikipedia references were removed so detectors cannot rely on trivial cues. - **Generalization:** Cross-domain and cross-generator settings remain difficult. BLOOMz output was especially hard to separate from human text. - **Availability:** Data and code are at github.com/mbzuai-nlp/M4. The authors plan to keep adding generators, domains and languages. Licensing follows the original corpora and LLM terms. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 256 works · 123,373 citations - [Technische Universität Darmstadt](https://md.research.elamur.ai/free/en/universities/technische-universitat-darmstadt/) — University, Darmstadt, Germany · 114 works · 36,326 citations - [New York University Abu Dhabi](https://md.research.elamur.ai/free/en/universities/new-york-university-abu-dhabi/) — University, Abu Dhabi, United Arab Emirates · 124 works · 4,709 citations - [Mohamed bin Zayed University of Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/mohamed-bin-zayed-university-of-artificial-intelligence/) — University, Abu Dhabi, United Arab Emirates · 138 works · 6,590 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works ## Cite Cite as: Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Fikri Aji, Nizar Habash, Iryna Gurevych and Preslav Nakov. 2023. M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection. arXiv:2305.14902. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{wang2023m4, title = {{M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection}}, author = {Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Fikri Aji, Nizar Habash, Iryna Gurevych and Preslav Nakov}, year = {2023}, doi = {10.48550/arxiv.2305.14902}, eprint = {2305.14902}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2305.14902} } ``` - [← previous Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models](https://md.research.elamur.ai/free/en/papers/arora-2023-have-llms-advanced-enough-challenging/) - [next → Benchmarking Foundation Models with Language-Model-as-an-Examiner](https://md.research.elamur.ai/free/en/papers/bai-2023-benchmarking-foundation-models-language-model/) - [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/) ## References 3 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Bias: Biases may exist in both human-written and machine-generated texts, and it is possible that our M4 dataset may be influenced by biases from human collection, thus affecting the detection outcomes. We leave the analysis of such biases to our future work. [find](https://scholar.google.com/scholar?q=Bias%3A+Biases+may+exist+in+both+human%2Dwritten+and+machine%2Dgenerated+texts%2C+and+it+is+possible+that+our+M4+dataset+may+be+influenced+by+biases+from+human+collection%2C+thus+affecting+the+detection+outcome) 2. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm. [find](https://scholar.google.com/scholar?q=Mike+Conover%2C+Matt+Hayes%2C+Ankit+Mathur%2C+Jianwei+Xie%2C+Jun+Wan%2C+Sam+Shah%2C+Ali+Ghodsi%2C+Patrick+Wendell%2C+Matei+Zaharia%2C+and+Reynold+Xin%2E+2023%2E+Free+dolly%3A+Introducing+the+world%E2%80%99s+first+truly+open+instruct) 3. Feasibility of Black-Box Machine-Generated Text Detection [find](https://scholar.google.com/scholar?q=Feasibility+of+Black%2DBox+Machine%2DGenerated+Text+Detection) *Updated 2 November 2026*