--- title: "DRCD: a Chinese Machine Reading Comprehension Dataset" description: "They argue this format tests comprehension on its own, without retrieval. 89.59% EM. 11 references, cited by 56 papers here." authors: ["Chih Chieh Shao", "Trois Liu", "Yuting Lai", "Yiying Tseng", "Sam Tsai"] year: 2018 venue: "arXiv:1806.00920" source: "https://arxiv.org/abs/1806.00920" doi: "10.48550/arxiv.1806.00920" arxiv: "1806.00920" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/shao-2018-drcd-chinese-machine-reading-comprehension/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shao-2018-drcd-chinese-machine-reading-comprehension/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/) / DRCD: a Chinese Machine Reading Comprehension Dataset # DRCD: a Chinese Machine Reading Comprehension Dataset **Authors:** Chih Chieh Shao, Trois Liu, Yuting Lai, Yiying Tseng and Sam Tsai · **Year:** 2018 · **Venue:** arXiv:1806.00920 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/1806.00920) · [PDF](https://arxiv.org/pdf/1806.00920) · [DOI 10.48550/arxiv.1806.00920](https://doi.org/10.48550/arxiv.1806.00920) · [arXiv 1806.00920](https://arxiv.org/abs/1806.00920) · [OpenAlex](https://openalex.org/W2806081754) **Citations:** 80 citations (OpenAlex, 2026-10-14) · [56](https://md.research.elamur.ai/free/en/papers/shao-2018-drcd-chinese-machine-reading-comprehension/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary DRCD (Delta Reading Comprehension Dataset) fills a gap: no large-scale machine reading comprehension (MRC) benchmark existed for traditional Chinese, since most resources are English or simplified Chinese. The authors, from Delta Research Center, chose the span-based format, where each answer is a contiguous fragment of a given passage. They argue this format tests comprehension on its own, without retrieval. Following the SQuAD collection procedure, they selected high-PageRank Chinese Wikipedia articles. Annotators cleaned these articles, split them into topical paragraphs of 250–1,500 characters, and wrote questions in their own words with answers located in the text. A second annotator answered the dev and test questions to estimate human performance. Baselines range from a TF-IDF sentence-matching method through R-Net, BiDAF and QANet to fine-tuned Chinese BERT. BERT performed best (F1 89.59%, EM 82.34%) but its F1 still trails humans (93.30%). The dataset is publicly released and intended as a source for transfer learning in traditional Chinese question answering and search. ## Key points - **Size:** 10,014 paragraphs from 2,108 Chinese Wikipedia articles, split into train (26,932 questions / 8,014 paragraphs), dev (3,524 / 1,000) and test (3,485 / 1,000). - These splits sum to 33,941 QA pairs. - The conclusion instead cites 40,410 pairs, a discrepancy the paper does not explain. - **Lengths:** paragraphs average 435.8 characters, questions 21.07 and answers 4.86. Paragraphs are much longer than in SQuAD because annotators split text by topic. - **Answer types:** 70.45% entity, 18.03% numeric and 11.50% description (answers over 6 characters). - **Question types:** "which" (30.96%) and "what" (28.42%) are the most common; "why" accounts for only 0.27%. - **Test-set F1 / EM:** - TF-IDF: ~17% / 0.05% - R-Net: 38% / 23.8% - BiDAF: 51.18% / 28.08% - QANet (with jieba tokenization): 78.03% / 65.56% - BERT: 89.59% / 82.34% - Human: 93.30% / 80.43% *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works 🧭 Topics (1): [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 487 works ## Cite Cite as: Chih Chieh Shao, Trois Liu, Yuting Lai, Yiying Tseng and Sam Tsai. 2018. DRCD: a Chinese Machine Reading Comprehension Dataset. arXiv:1806.00920. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{shao2018drcd, title = {{DRCD: a Chinese Machine Reading Comprehension Dataset}}, author = {Chih Chieh Shao, Trois Liu, Yuting Lai, Yiying Tseng and Sam Tsai}, year = {2018}, doi = {10.48550/arxiv.1806.00920}, eprint = {1806.00920}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.1806.00920} } ``` - [← previous Character-Level Language Modeling with Deeper Self-Attention](https://md.research.elamur.ai/free/en/papers/al-rfou-2018-character-level-language-modeling-deeper/) - [next → Learning Simplifications for Specific Target Audiences](https://md.research.elamur.ai/free/en/papers/scarton-2018-learning-simplifications-specific-target-audiences/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/) ## References 11 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. [Teaching machines to read and comprehend](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/). In Advances in Neural Information Processing Systems, pages 1693– 1701. 2. Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. 2015. [The goldilocks principle: Reading children’s books with explicit memory representations](https://md.research.elamur.ai/free/en/papers/hill-2015-goldilocks-principle-reading-children-s/). arXiv preprint arXiv:1511.02301. 3. Yiming Cui, Ting Liu, Zhipeng Chen, Shijin Wang, and Guoping Hu. 2016. Consensus attentionbased neural networks for chinese reading comprehension. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. [find](https://scholar.google.com/scholar?q=Yiming+Cui%2C+Ting+Liu%2C+Zhipeng+Chen%2C+Shijin+Wang%2C+and+Guoping+Hu%2E+2016%2E+Consensus+attentionbased+neural+networks+for+chinese+reading+comprehension%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International) 4. The COLING 2016 Organizing Committee, pages 1777– 1786. [http://aclweb.org/anthology/C16-1167](http://aclweb.org/anthology/C16-1167). [↗](http://aclweb.org/anthology/C16-1167) 5. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. [Race: Large-scale reading comprehension dataset from examinations](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/). arXiv preprint arXiv:1704.04683. 6. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [Squad: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). arXiv preprint arXiv:1606.05250. 7. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. [Triviaqa: A large scale distantly supervised challenge comprehension](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/). ACL. dataset for reading Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms Marco: a Human Generated Mach… 8. Wei He, Kai Liu, Yajuan Lyu, Shiqi Zhao, Xinyan Xiao, Yuan Liu, Yizhong Wang, Hua Wu, Qiaoqiao She, Xuan Liu, et al. 2017. [DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications](https://md.research.elamur.ai/free/en/papers/he-2017-dureader-chinese-machine-reading/). arXiv preprint arXiv:1711.05073 (2017). 9. Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. 2017. [Gated Self-Matching Networks for Reading Comprehension and Question Answering](https://md.research.elamur.ai/free/en/papers/wang-2017-gated-self-matching-networks-reading/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 189–198 10. Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V. Le. 2018. [Qanet: Combining local convolution with global self-attention for reading comprehension](https://md.research.elamur.ai/free/en/papers/yu-2018-qanet-combining-local-convolution-global/). 11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. 2018. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805 (2018). *Updated 2 November 2026*