--- title: "Deep Reinforcement Learning for Chinese Zero — Yin 2018" description: "Chinese often omits pronouns, leaving \"zero pronouns\" whose referents must be recovered from context." authors: ["Qingyu Yin", "Yu Zhang", "Wei-Nan Zhang", "Ting Liu", "William Yang Wang"] year: 2018 venue: "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)" source: "https://aclanthology.org/P18-1053/" doi: "10.18653/v1/p18-1053" arxiv: "1806.03711" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/yin-2018-deep-reinforcement-learning-chinese-zero/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/yin-2018-deep-reinforcement-learning-chinese-zero/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2018](https://md.research.elamur.ai/free/papers/2018/) / Deep Reinforcement Learning for Chinese Zero Pronoun Resolution # Deep Reinforcement Learning for Chinese Zero Pronoun Resolution **Authors:** [Qingyu Yin](https://md.research.elamur.ai/free/authors/qingyu-yin/), [Yu Zhang](https://md.research.elamur.ai/free/authors/yu-zhang/), [Wei-Nan Zhang](https://md.research.elamur.ai/free/authors/wei-nan-zhang/), [Ting Liu](https://md.research.elamur.ai/free/authors/ting-liu/), [William Yang Wang](https://md.research.elamur.ai/free/authors/william-yang-wang/) · **Year:** 2018 · **Venue:** Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/P18-1053/) · [PDF](https://aclanthology.org/P18-1053.pdf) · [DOI 10.18653/v1/p18-1053](https://doi.org/10.18653/v1/p18-1053) · [arXiv 1806.03711](https://arxiv.org/abs/1806.03711) · [OpenAlex](https://openalex.org/W2963888230) **Citations:** 51 citations (openalex-snapshot · 2026-10-14) · [16](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Chinese often omits pronouns, leaving "zero pronouns" whose referents must be recovered from context. Earlier neural resolvers score each pair of a zero pronoun and one candidate antecedent in isolation. They therefore ignore what earlier decisions reveal about later ones. Yin and colleagues recast anaphoric zero pronoun resolution as a Markov decision process. A policy-network agent walks through the candidate noun phrases one by one and decides whether each one is an antecedent. At every step, the agent's state includes a pooled summary of the candidates it has already accepted. This lets earlier choices inform later ones, such as rejecting a nearby mention that clashes with the entity already identified. The agent is pre-trained with supervision and then optimized with a reward equal to the F-score of its selected antecedents. It therefore targets the evaluation metric directly rather than a per-decision loss. On OntoNotes 5.0, the model beats prior state-of-the-art systems. The authors present it as the first reinforcement learning approach to Chinese zero pronoun resolution. ## Key points - **Candidate selection:** candidates are maximal or modifier noun phrases located at most two sentences from the zero pronoun. This follows the heuristic of Chen and Ng (2016) and Yin et al. (2017a,b). - **Encoders:** the zero pronoun is encoded with the ZP-centered neural network of Yin et al. (2017a), which uses two RNNs over the preceding and following context. Each candidate phrase is encoded by a left-to-right RNN over its content words. - **State vector:** each state combines four parts: - the zero pronoun vector, - the current candidate vector, - pooled information from previously predicted antecedents, - pairwise features. - **Learning:** the authors chose a stochastic policy network trained with SGD over a Deep Q-Network. Their reasons were to avoid getting stuck in intermediate states and to obtain a more interpretable policy. Training starts with supervised pre-training. - **Robustness:** across 38 random seeds, F-scores ranged from 56.5% to 57.5%, with a median of 57.1%. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) Topics: [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/) ## Cited by Cited by 16 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 13 not yet. - [Tackling Zero Pronoun Resolution and Non-Zero Coreference Resolution Jointly](https://md.research.elamur.ai/free/papers/chen-2021-tackling-zero-pronoun-resolution-non/) - [End-to-end Deep Reinforcement Learning Based Coreference Resolution](https://md.research.elamur.ai/free/papers/fei-2019-end-end-deep-reinforcement-learning/) - [A Joint Model for Dropped Pronoun Recovery and Conversational Discourse Parsing in Chinese Conversational Speech](https://md.research.elamur.ai/free/papers/yang-2021-joint-model-dropped-pronoun-recovery/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{yin2018deep, title = {{Deep Reinforcement Learning for Chinese Zero Pronoun Resolution}}, author = {Yin, Qingyu and Zhang, Yu and Zhang, Wei-Nan and Liu, Ting and Wang, William Yang}, booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, year = {2018}, doi = {10.18653/v1/p18-1053}, eprint = {1806.03711}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/p18-1053} } ``` - [← previous Deep Neural Networks for Coreference Resolution for Polish](https://md.research.elamur.ai/free/papers/niton-2018-deep-neural-networks-coreference-resolution/) - [next → Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://md.research.elamur.ai/free/papers/li-2018-delete-retrieve-generate-simple-approach/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/papers/2018/) ## References 3 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Chen Chen and Vincent Ng. 2014. Chinese zero pronoun resolution: An unsupervised approach combining ranking and integer linear programming. In Twenty-Eighth AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Chen+Chen+and+Vincent+Ng%2E+2014%2E+Chinese+zero+pronoun+resolution%3A+An+unsupervised+approach+combining+ranking+and+integer+linear+programming%2E+In+Twenty%2DEighth+AAAI+Conference+on+Artificial+Intelligence%2E) 2. SRK Branavan, David Silver, and Regina Barzilay. 2012. Learning to win by reading manuals in a monte-carlo framework. Journal of Artificial Intelligence Research, 43:661–704. [find](https://scholar.google.com/scholar?q=SRK+Branavan%2C+David+Silver%2C+and+Regina+Barzilay%2E+2012%2E+Learning+to+win+by+reading+manuals+in+a+monte%2Dcarlo+framework%2E+Journal+of+Artificial+Intelligence+Research%2C+43%3A661%E2%80%93704%2E) 3. Chen Chen and Vincent Ng. 2013. Chinese zero pronoun resolution: Some recent advances. In EMNLP, pages 1360–1365. \#\# Conclusion [find](https://scholar.google.com/scholar?q=Chen+Chen+and+Vincent+Ng%2E+2013%2E+Chinese+zero+pronoun+resolution%3A+Some+recent+advances%2E+In+EMNLP%2C+pages+1360%E2%80%931365%2E+%5C%23%5C%23+Conclusion) [our card](https://md.research.elamur.ai/free/papers/chen-2013-chinese-zero-pronoun-resolution-some/)