---
title: "One Model to Learn Both — Wang 2019"
description: "Pro-drop languages such as Chinese and Japanese often omit pronouns that English requires."
authors: ["Longyue Wang", "Zhaopeng Tu", "Xing Wang", "Shuming Shi"]
year: 2019
venue: "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)"
source: "https://aclanthology.org/D19-1085/"
doi: "10.18653/v1/d19-1085"
arxiv: "1909.00369"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/wang-2019-one-model-learn-both-zero/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/wang-2019-one-model-learn-both-zero/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2019](https://md.research.elamur.ai/free/papers/2019/page/3/) / One Model to Learn Both: Zero Pronoun Prediction and Translation

# One Model to Learn Both: Zero Pronoun Prediction and Translation

**Authors:** [Longyue Wang](https://md.research.elamur.ai/free/authors/longyue-wang/), [Zhaopeng Tu](https://md.research.elamur.ai/free/authors/zhaopeng-tu/), [Xing Wang](https://md.research.elamur.ai/free/authors/xing-wang/), [Shuming Shi](https://md.research.elamur.ai/free/authors/shuming-shi/) · **Year:** 2019 · **Venue:** Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D19-1085/) · [PDF](https://aclanthology.org/D19-1085.pdf) · [DOI 10.18653/v1/d19-1085](https://doi.org/10.18653/v1/d19-1085) · [arXiv 1909.00369](https://arxiv.org/abs/1909.00369) · [OpenAlex](https://openalex.org/W2971287409)

**Citations:** 25 citations (openalex-snapshot · 2026-10-14) · [16](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Pro-drop languages such as Chinese and Japanese often omit pronouns that English requires. As a result, neural MT systems translating into English can produce incomplete or incorrect sentences. Earlier fixes ran a separate ZP predictor before translation. Its modest accuracy (about 66% F1) passed errors into the translation step and made the pipeline more complex. This paper combines both tasks in one network. It builds on the encoder-decoder-reconstructor framework and treats ZP prediction as sequence labelling over reconstructor states: each source token is tagged either "no ZP" or with the pronoun missing before it. Training labels come from ZPs recovered automatically from parallel data through word alignment, so no ZP annotation is needed at test time. A hierarchical encoder also summarizes preceding sentences and feeds this discourse context into the reconstructor. On a 2.15M-pair Chinese–English subtitle corpus, the full model beats the pipeline approach by 2.29 BLEU and 11% prediction accuracy, with consistent gains on Japanese–English. Handling dropped pronouns end-to-end with document context proves both more accurate and simpler to deploy.

## Key points
- The training objective combines three terms: translation likelihood, reconstruction of the source, and ZP labelling, where each label is either "N" (no pronoun) or a word from a pronoun vocabulary.
- Supervision relies on the alignment-based auto-annotation of Wang et al. (2016), which is about 90–93% accurate. Previous external ZP predictors reached only 66% (full prediction) or 88% (position-only prediction).
- Discourse context helps because, for roughly 23% of ZPs, the antecedent lies at least two sentences earlier. Injecting this context into the reconstructor worked better than injecting it into the decoder.
- The main results are on 2.15M Chinese–English sentence pairs, the widely used ZP translation data from Wang et al. (2018a). The model gains +2.29 BLEU and +11% ZP prediction accuracy over the external-predictor approach, setting a new state of the art on this subtitle corpus.
- In a manual study of 500 test sentences, the discourse-aware model fixed 74% of ZP-related translation errors and introduced only 12% new ones. Subjective ZPs were the main source of the remaining problems.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Discourse analysis](https://md.research.elamur.ai/free/topics/discourse-analysis/)

## Cited by

Cited by

16 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 14 not yet.

- [Document-Level Machine Translation with Large Language Models](https://md.research.elamur.ai/free/papers/wang-2023-document-level-machine-translation-large/)
- [GuoFeng: A Benchmark for Zero Pronoun Recovery and Translation](https://md.research.elamur.ai/free/papers/xu-2022-guofeng-benchmark-zero-pronoun-recovery/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2019one,
  title = {{One Model to Learn Both: Zero Pronoun Prediction and Translation}},
  author = {Wang, Longyue and Tu, Zhaopeng and Wang, Xing and Shi, Shuming},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1085},
  eprint = {1909.00369},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d19-1085}
}
```

- [← previous On NMT Search Errors and Model Errors: Cat Got Your Tongue?](https://md.research.elamur.ai/free/papers/stahlberg-2019-nmt-search-errors-model-errors/)
- [next → Open Sesame: Getting inside BERT's Linguistic Knowledge](https://md.research.elamur.ai/free/papers/lin-2019-open-sesame-getting-inside-bert/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/papers/2019/page/3/)

## References

34 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rachel Bawden, Rico Sennrich, Alexandra Birch, and Barry Haddow. 2018. Evaluating discourse phenomena in neural machine translation. In NAACL. [find](https://scholar.google.com/scholar?q=Rachel+Bawden%2C+Rico+Sennrich%2C+Alexandra+Birch%2C+and+Barry+Haddow%2E+2018%2E+Evaluating+discourse+phenomena+in+neural+machine+translation%2E+In+NAACL%2E) [our card](https://md.research.elamur.ai/free/papers/bawden-2017-evaluating-discourse-phenomena-neural-machine/)
2. Chen Chen and Vincent Ng. 2013. Chinese zero pronoun resolution: Some recent advances. In EMNLP. [find](https://scholar.google.com/scholar?q=Chen+Chen+and+Vincent+Ng%2E+2013%2E+Chinese+zero+pronoun+resolution%3A+Some+recent+advances%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/chen-2013-chinese-zero-pronoun-resolution-some/)
3. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+Van+Merrie%CC%88nboer%2C+Caglar+Gulcehre%2C+Dzmitry+Bahdanau%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014%2E+Learning+phrase+representations+using+rnn+encoder%2Ddecoder+for+statisti)
4. Hirotoshi Taira, Katsuhito Sudoh, and Masaaki Nagata. 2012. Zero pronoun resolution can improve the quality of J-E translation. In Workshop on Syntax, Semantics and Structure in Statistical Translation. [find](https://scholar.google.com/scholar?q=Hirotoshi+Taira%2C+Katsuhito+Sudoh%2C+and+Masaaki+Nagata%2E+2012%2E+Zero+pronoun+resolution+can+improve+the+quality+of+J%2DE+translation%2E+In+Workshop+on+Syntax%2C+Semantics+and+Structure+in+Statistical+Translatio) [our card](https://md.research.elamur.ai/free/papers/taira-2012-zero-pronoun-resolution-can-improve/)
5. Tagyoung Chung and Daniel Gildea. 2010. Effects of empty categories on machine translation. In EMNLP. [find](https://scholar.google.com/scholar?q=Tagyoung+Chung+and+Daniel+Gildea%2E+2010%2E+Effects+of+empty+categories+on+machine+translation%2E+In+EMNLP%2E)
6. Michael Collins, Philipp Koehn, and Ivona Kucerova. 2005. Clause restructuring for statistical machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Michael+Collins%2C+Philipp+Koehn%2C+and+Ivona+Kucerova%2E+2005%2E+Clause+restructuring+for+statistical+machine+translation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/collins-2005-clause-restructuring-statistical-machine-translation/)
7. Shunsuke Takeno, Masaaki Nagata, and Kazuhide Yamamoto. 2017. Controlling target features in neural machine translation via prefix constraints. In The 4th Workshop on Asian Translation. [find](https://scholar.google.com/scholar?q=Shunsuke+Takeno%2C+Masaaki+Nagata%2C+and+Kazuhide+Yamamoto%2E+2017%2E+Controlling+target+features+in+neural+machine+translation+via+prefix+constraints%2E+In+The+4th+Workshop+on+Asian+Translation%2E)
8. Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017. Convolutional sequence to sequence learning. In ICML. [find](https://scholar.google.com/scholar?q=Jonas+Gehring%2C+Michael+Auli%2C+David+Grangier%2C+Denis+Yarats%2C+and+Yann+N%2E+Dauphin%2E+2017%2E+Convolutional+sequence+to+sequence+learning%2E+In+ICML%2E) [our card](https://md.research.elamur.ai/free/papers/gehring-2017-convolutional-sequence-sequence-learning/)
9. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in opus. In LREC. [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+opus%2E+In+LREC%2E)
10. Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing coadaptation of feature detectors. arXiv preprint arXiv:1207.0580. [find](https://scholar.google.com/scholar?q=Geoffrey+E+Hinton%2C+Nitish+Srivastava%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+R+Salakhutdinov%2E+2012%2E+Improving+neural+networks+by+preventing+coadaptation+of+feature+detectors%2E+arXiv+preprint+arXiv)
11. Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. Neural machine translation with reconstruction. In AAAI. [find](https://scholar.google.com/scholar?q=Zhaopeng+Tu%2C+Yang+Liu%2C+Lifeng+Shang%2C+Xiaohua+Liu%2C+and+Hang+Li%2E+2017%2E+Neural+machine+translation+with+reconstruction%2E+In+AAAI%2E)
12. Zhaopeng Tu, Yang Liu, Shuming Shi, and Tong Zhang. 2018. Learning to remember translation history with a continuous cache. TACL. [find](https://scholar.google.com/scholar?q=Zhaopeng+Tu%2C+Yang+Liu%2C+Shuming+Shi%2C+and+Tong+Zhang%2E+2018%2E+Learning+to+remember+translation+history+with+a+continuous+cache%2E+TACL%2E)
13. Sebastien Jean, Stanislas Lauly, Orhan Firat, and Kyunghyun Cho. 2017a. Does neural machine translation benefit from larger context? arXiv preprint arXiv:1704.05135. [find](https://scholar.google.com/scholar?q=Sebastien+Jean%2C+Stanislas+Lauly%2C+Orhan+Firat%2C+and+Kyunghyun+Cho%2E+2017a%2E+Does+neural+machine+translation+benefit+from+larger+context%3F+arXiv+preprint+arXiv%3A1704%2E05135%2E)
14. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+NIPS%2E)
15. Sebastien Jean, Stanislas Lauly, Orhan Firat, and Kyunghyun Cho. 2017b. Neural machine translation for cross-lingual pronoun prediction. In Workshop on Discourse in Machine Translation. [find](https://scholar.google.com/scholar?q=Sebastien+Jean%2C+Stanislas+Lauly%2C+Orhan+Firat%2C+and+Kyunghyun+Cho%2E+2017b%2E+Neural+machine+translation+for+cross%2Dlingual+pronoun+prediction%2E+In+Workshop+on+Discourse+in+Machine+Translation%2E)
16. Elena Voita, Pavel Serdyukov, Rico Sennrich, and Ivan Titov. 2018. Context-aware neural machine translation learns anaphora resolution. In ACL. [find](https://scholar.google.com/scholar?q=Elena+Voita%2C+Pavel+Serdyukov%2C+Rico+Sennrich%2C+and+Ivan+Titov%2E+2018%2E+Context%2Daware+neural+machine+translation+learns+anaphora+resolution%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/voita-2018-context-aware-neural-machine-translation/)
17. Fang Kong and Guodong Zhou. 2010. A tree kernel-based unified framework for chinese zero anaphora resolution. In EMNLP. [find](https://scholar.google.com/scholar?q=Fang+Kong+and+Guodong+Zhou%2E+2010%2E+A+tree+kernel%2Dbased+unified+framework+for+chinese+zero+anaphora+resolution%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/kong-2010-tree-kernel-based-unified-framework/)
18. Longyue Wang, Zhaopeng Tu, Shuming Shi, Tong Zhang, Yvette Graham, and Qun Liu. 2018a. Translating pro-drop languages with reconstruction models. In AAAI. [find](https://scholar.google.com/scholar?q=Longyue+Wang%2C+Zhaopeng+Tu%2C+Shuming+Shi%2C+Tong+Zhang%2C+Yvette+Graham%2C+and+Qun+Liu%2E+2018a%2E+Translating+pro%2Ddrop+languages+with+reconstruction+models%2E+In+AAAI%2E)
19. Ronan Le Nagard and Philipp Koehn. 2010. Aiding pronoun translation with co-reference resolution. In WMT-MetricsMATR. [find](https://scholar.google.com/scholar?q=Ronan+Le+Nagard+and+Philipp+Koehn%2E+2010%2E+Aiding+pronoun+translation+with+co%2Dreference+resolution%2E+In+WMT%2DMetricsMATR%2E) [our card](https://md.research.elamur.ai/free/papers/nagard-2010-aiding-pronoun-translation-co-reference/)
20. Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. 2017. Exploiting cross-sentence context for neural machine translation. In EMNLP. [find](https://scholar.google.com/scholar?q=Longyue+Wang%2C+Zhaopeng+Tu%2C+Andy+Way%2C+and+Qun+Liu%2E+2017%2E+Exploiting+cross%2Dsentence+context+for+neural+machine+translation%2E+In+EMNLP%2E)
21. Charles N Li and SA Thomson. 1979. Third-person pronouns and zero-anaphora in chinese discourse in discourse and syntax. Syntax and Semantics, 12. [find](https://scholar.google.com/scholar?q=Charles+N+Li+and+SA+Thomson%2E+1979%2E+Third%2Dperson+pronouns+and+zero%2Danaphora+in+chinese+discourse+in+discourse+and+syntax%2E+Syntax+and+Semantics%2C+12%2E)
22. Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. 2018b. Learning to jointly translate and predict dropped pronouns with a shared reconstruction mechanism. In EMNLP. [find](https://scholar.google.com/scholar?q=Longyue+Wang%2C+Zhaopeng+Tu%2C+Andy+Way%2C+and+Qun+Liu%2E+2018b%2E+Learning+to+jointly+translate+and+predict+dropped+pronouns+with+a+shared+reconstruction+mechanism%2E+In+EMNLP%2E)
23. Yang Liu and Qun Liu. 2010. Joint parsing and translation. In COLING. [find](https://scholar.google.com/scholar?q=Yang+Liu+and+Qun+Liu%2E+2010%2E+Joint+parsing+and+translation%2E+In+COLING%2E)
24. Longyue Wang, Zhaopeng Tu, Xiaojun Zhang, Hang Li, Andy Way, and Qun Liu. 2016. A novel approach for dropped pronoun translation. In NAACL. [find](https://scholar.google.com/scholar?q=Longyue+Wang%2C+Zhaopeng+Tu%2C+Xiaojun+Zhang%2C+Hang+Li%2C+Andy+Way%2C+and+Qun+Liu%2E+2016%2E+A+novel+approach+for+dropped+pronoun+translation%2E+In+NAACL%2E)
25. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+BLEU%3A+A+method+for+automatic+evaluation+of+machine+translation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
26. Bing Xiang, Xiaoqiang Luo, and Bowen Zhou. 2013. Enlisting the ghost: Modeling empty categories for machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Bing+Xiang%2C+Xiaoqiang+Luo%2C+and+Bowen+Zhou%2E+2013%2E+Enlisting+the+ghost%3A+Modeling+empty+categories+for+machine+translation%2E+In+ACL%2E)
27. Arum Park, Seunghee Lim, and Munpyo Hong. 2015. Zero object resolution in korean. In PACLIC. [find](https://scholar.google.com/scholar?q=Arum+Park%2C+Seunghee+Lim%2C+and+Munpyo+Hong%2E+2015%2E+Zero+object+resolution+in+korean%2E+In+PACLIC%2E)
28. Xinyan Xiao, Yang Liu, Young-Sook Hwang, Qun Liu, and Shouxun Lin. 2010. Joint tokenization and translation. In COLING. [find](https://scholar.google.com/scholar?q=Xinyan+Xiao%2C+Yang+Liu%2C+Young%2DSook+Hwang%2C+Qun+Liu%2C+and+Shouxun+Lin%2E+2010%2E+Joint+tokenization+and+translation%2E+In+COLING%2E)
29. Sudha Rao, Allyson Ettinger, Hal Daumé III, and Philip Resnik. 2015. Dialogue focus tracking for zero pronoun resolution. In NAACL. [find](https://scholar.google.com/scholar?q=Sudha+Rao%2C+Allyson+Ettinger%2C+Hal+Daume%CC%81+III%2C+and+Philip+Resnik%2E+2015%2E+Dialogue+focus+tracking+for+zero+pronoun+resolution%2E+In+NAACL%2E)
30. Qingyu Yin, Yu Zhang, Weinan Zhang, and Ting Liu. 2017. Chinese zero pronoun resolution with deep memory network. In EMNLP. [find](https://scholar.google.com/scholar?q=Qingyu+Yin%2C+Yu+Zhang%2C+Weinan+Zhang%2C+and+Ting+Liu%2E+2017%2E+Chinese+zero+pronoun+resolution+with+deep+memory+network%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/yin-2017-chinese-zero-pronoun-resolution-deep/)
31. Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI. [find](https://scholar.google.com/scholar?q=Iulian+V%2E+Serban%2C+Alessandro+Sordoni%2C+Yoshua+Bengio%2C+Aaron+Courville%2C+and+Joelle+Pineau%2E+2016%2E+Building+end%2Dto%2Dend+dialogue+systems+using+generative+hierarchical+neural+network+models%2E+In+AAAI%2E) [our card](https://md.research.elamur.ai/free/papers/serban-2015-building-end-end-dialogue-systems/)
32. Matthew D Zeiler. 2012. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. [find](https://scholar.google.com/scholar?q=Matthew+D+Zeiler%2E+2012%2E+ADADELTA%3A+an+adaptive+learning+rate+method%2E+arXiv+preprint+arXiv%3A1212%2E5701%2E) [our card](https://md.research.elamur.ai/free/papers/zeiler-2012-adadelta-adaptive-learning-rate-method/)
33. Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simonsen, and Jian-Yun Nie. 2015. A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In CIKM. [find](https://scholar.google.com/scholar?q=Alessandro+Sordoni%2C+Yoshua+Bengio%2C+Hossein+Vahabi%2C+Christina+Lioma%2C+Jakob+Grue+Simonsen%2C+and+Jian%2DYun+Nie%2E+2015%2E+A+hierarchical+recurrent+encoder%2Ddecoder+for+generative+context%2Daware+query+suggestion%2E)
34. Shanheng Zhao and Hwee Tou Ng. 2007. Identification and resolution of chinese zero pronouns: A machine learning approach. In EMNLP. [find](https://scholar.google.com/scholar?q=Shanheng+Zhao+and+Hwee+Tou+Ng%2E+2007%2E+Identification+and+resolution+of+chinese+zero+pronouns%3A+A+machine+learning+approach%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/zhao-2007-identification-resolution-chinese-zero-pronouns/)