--- title: "Nakdan: Professional Hebrew Diacritizer — Shmidman 2020" description: "On a 6,000-word modern test, word-level accuracy is 88.2%, against 80.9% for Morfix and 66.4% for Snopi. 10 references, cited by 11 papers here." authors: ["Avi Shmidman", "Shaltiel Shmidman", "Moshe Koppel", "Yoav Goldberg"] year: 2020 venue: "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations" source: "https://aclanthology.org/2020.acl-demos.23/" doi: "10.18653/v1/2020.acl-demos.23" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/shmidman-2020-nakdan-professional-hebrew-diacritizer/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shmidman-2020-nakdan-professional-hebrew-diacritizer/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/) / Nakdan: Professional Hebrew Diacritizer # Nakdan: Professional Hebrew Diacritizer **Authors:** [Avi Shmidman](https://md.research.elamur.ai/free/en/authors/avi-shmidman/); [Shaltiel Shmidman](https://md.research.elamur.ai/free/en/authors/shaltiel-shmidman/); [Moshe Koppel](https://md.research.elamur.ai/free/en/authors/moshe-koppel/); [Yoav Goldberg](https://md.research.elamur.ai/free/en/authors/yoav-goldberg/) · **Year:** 2020 · **Venue:** Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations **Link:** [aclanthology.org/2020.acl-demos.23](https://aclanthology.org/2020.acl-demos.23/) · library `aclbulk-2020-shmidman-nakdan-professional-hebrew-diacritizer` · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) [**12** citations](https://openalex.org/works?filter=cites:W3022124040) OpenAlex, 2026-10-14 · [**11** citations in our library](https://md.research.elamur.ai/free/en/papers/shmidman-2020-nakdan-professional-hebrew-diacritizer/cited-by/) of the 140,884 papers whose bibliographies we read · [Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations](https://md.research.elamur.ai/free/en/venues/acl/) · published by [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · [licence](https://creativecommons.org/licenses/by/4.0/) - [Publisher's page](https://aclanthology.org/2020.acl-demos.23/) - [PDF](https://aclanthology.org/2020.acl-demos.23.pdf) - [DOI 10.18653/v1/2020.acl-demos.23](https://doi.org/10.18653/v1/2020.acl-demos.23) - [OpenAlex](https://openalex.org/W3022124040) **Authors (4)** 1. [Avi Shmidman](https://openalex.org/A5013212409) [Semantic Scholar](https://www.semanticscholar.org/author/3361477) · h 5 · 114 citations · 53 works 2. [Shaltiel Shmidman](https://openalex.org/A5041361841) [Semantic Scholar](https://www.semanticscholar.org/author/1680059376) · h 3 · 27 citations · 21 works 3. [Moshe Koppel](https://openalex.org/A5019908505) [Semantic Scholar](https://www.semanticscholar.org/author/145200175) · h 37 · 6,949 citations · 150 works 4. [Yoav Goldberg](https://openalex.org/A5028476919) [ORCID](https://orcid.org/0000-0002-6497-829X) · [Semantic Scholar](https://www.semanticscholar.org/author/79775260) · h 59 · 23,282 citations · 401 works Citations and author metrics: OpenAlex, 2026-10-14; authors with the same name are not always told apart. ## What it is about Nakdan adds vowel points to undiacritized Hebrew, for modern, Rabbinic and poetic texts, through a web interface that lets an editor choose among the alternatives. The pipeline has three stages: a POS tagger and a morphological tagger (bi-LSTMs with a character model, trained on about 200K hand-tagged tokens, 92% coarse-POS and 79% full-morphology accuracy); a filter that intersects each word's candidate points with a hand-built lexicon of about 50K lexemes and about 5.5M inflected forms plus proper-name lists; and a bi-LSTM that ranks the surviving candidates in context, with beam search over the points of each letter. On a 6,000-word modern test, word-level accuracy is 88.2%, against 80.9% for Morfix and 66.4% for Snopi. The authors also report that knowing the points cuts the morphological ambiguity from 9.1 to 2.4 analyses per form, and knowing the morphology cuts the point ambiguity from 6.2 to 1.4. Rabbinic and poetic genres get their own models, and the Rabbinic test (Bet Yosef) is harder for all systems. ## For Mova - I1. Filter, then rank, for the unpointed mode of `tools/hbo`: take the segmenter's candidate analyses for a word, keep only those the lightest tagger allows, and rank the survivors. Report the average number of candidates before and after each filter, as the paper does. A random choice from the filtered set gives 87.1% on the modern test, which is the baseline a rule-based ranking must beat. - I2. A table of collocation overrides: the paper keeps about 10K hand-written pairs that fix the state of a word in a fixed phrase (the paper's example: the noun bayit is forced into its construct form inside one fixed two-word collocation, and into the plural construct inside its plural). Mova can keep such a table in [languages/hbo/04-noun.md](https://mova.elamur.ai/free/en/docs/languages/hbo/04-noun/) with the reason for each entry. - I3. Genre split: the paper states that the modern morphology model does not transfer to Rabbinic Hebrew and must be retrained on tagged historical text. Add a note to [languages/hbo/11-mishnaic.md](https://mova.elamur.ai/free/en/docs/languages/hbo/11-mishnaic/) that Mishnaic rules are tested on their own gold set, and do not reuse the Biblical tables unchanged. I1 is design for `tools/hbo` (unpointed mode) and its measure; I2 belongs in [languages/hbo/04-noun.md](https://mova.elamur.ai/free/en/docs/languages/hbo/04-noun/) and [languages/hbo/08-syntax.md](https://mova.elamur.ai/free/en/docs/languages/hbo/08-syntax/); I3 is a rule of testing for [languages/hbo/11-mishnaic.md](https://mova.elamur.ai/free/en/docs/languages/hbo/11-mishnaic/). The 6,000-word modern test corpus is released by the authors and can be used as a measurement of [languages/he/02-script-and-spelling.md](https://mova.elamur.ai/free/en/docs/languages/he/02-script-and-spelling/). ## Institutions [🔬**Allen Institute for Artificial Intelligence** Seattle · United States 169 works · 65,355 citations](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) · [🎓**Bar-Ilan University** Ramat Gan · Israel 102 works · 17,587 citations](https://md.research.elamur.ai/free/en/universities/bar-ilan-university/) 📚 Shelves 5[Computer Science · 7,349 works](https://md.research.elamur.ai/free/en/shelves/computer-science/) · [Artificial Intelligence · 7,183 works](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · [Arts and Humanities · 7,418 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · [Language and Linguistics · 7,357 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · [Literature and Literary Theory · 271 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/literature-and-literary-theory/) 🧭 Topics 4[Semitic languages · 1,336 works](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · [Morphology · 245 works](https://md.research.elamur.ai/free/en/topics/morphology/) · [Hebrew & Aramaic · 474 works](https://md.research.elamur.ai/free/en/topics/hebrew-aramaic/) · [Biblical Hebrew · 254 works](https://md.research.elamur.ai/free/en/topics/biblical-hebrew/) ## Cite Cite as Avi Shmidman, Shaltiel Shmidman, Moshe Koppel and Yoav Goldberg. 2020. Nakdan: Professional Hebrew Diacritizer. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2020.acl-demos.23](https://doi.org/10.18653/v1/2020.acl-demos.23) ```bibtex @inproceedings{shmidman2020nakdan, title = {{Nakdan: Professional Hebrew Diacritizer}}, author = {Avi Shmidman, Shaltiel Shmidman, Moshe Koppel and Yoav Goldberg}, booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations}, year = {2020}, doi = {10.18653/v1/2020.acl-demos.23}, url = {https://doi.org/10.18653/v1/2020.acl-demos.23} } ``` - [← previous Graph-to-Graph Transformer for Transition-based Dependency Parsing](https://md.research.elamur.ai/free/en/papers/mohammadshahi-2020-graph-graph-transformer-transition-based/) - [next → Speaker-change Aware CRF for Dialogue Act Classification](https://md.research.elamur.ai/free/en/papers/shang-2020-speaker-change-aware-crf-dialogue/) - [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/) ## References 10 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Gheith Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic diacritization of arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18:183–197. [find](https://scholar.google.com/scholar?q=Gheith+Abandah%2C+Alex+Graves%2C+Balkees+Al%2DShagoor%2C+Alaa+Arabiyat%2C+Fuad+Jamour%2C+and+Majid+Al%2DTaee%2E+2015%2E+Automatic+diacritization+of+arabic+text+using+recurrent+neural+networks%2E+International+Journal+on+) 2. Meni Adler and Michael Elhadad. 2006. An unsupervised morpheme-based HMM for hebrew morphological disambiguation. In ACL 2006, 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Sydney, Australia, 17-21 July 2006. [our card](https://md.research.elamur.ai/free/en/papers/adler-2006-unsupervised-morpheme-based-hmm-hebrew/) 3. Yonatan Belinkov and James R. Glass. 2015. Arabic diacritization with recurrent neural networks. In EMNLP. [our card](https://md.research.elamur.ai/free/en/papers/belinkov-2015-arabic-diacritization-recurrent-neural-networks/) 4. Yaacov Choueka and Yoni Neeman. 1995. Nakdan-text,(an in-context text-vocalizer for modern hebrew). In BISFAI-95, The Fifth Bar Ilan Symposium for Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Yaacov+Choueka+and+Yoni+Neeman%2E+1995%2E+Nakdan%2Dtext%2C%28an+in%2Dcontext+text%2Dvocalizer+for+modern+hebrew%29%2E+In+BISFAI%2D95%2C+The+Fifth+Bar+Ilan+Symposium+for+Artificial+Intelligence%2E) 5. Michael Elhadad, Yael Netzer, David Gabay, and Meni Adler. 2005. Hebrew morphological tagging guidelines. Technical report, Technical report, Ben-Gurion University, Dept. of Computer Science. [find](https://scholar.google.com/scholar?q=Michael+Elhadad%2C+Yael+Netzer%2C+David+Gabay%2C+and+Meni+Adler%2E+2005%2E+Hebrew+morphological+tagging+guidelines%2E+Technical+report%2C+Technical+report%2C+Ben%2DGurion+University%2C+Dept%2E+of+Computer+Science%2E) 6. Ya’akov Gal. 2002. An hmm approach to vowel restoration in arabic and hebrew. In Proceedings of the ACL-02 workshop on Computational approaches to semitic languages, pages 1–7. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/gal-2002-hmm-approach-vowel-restoration-arabic/) 7. Yoav Goldberg. 2017. Neural network methods for natural language processing. Synthesis Lectures on Human Language Technologies, 10(1):1–309. [find](https://scholar.google.com/scholar?q=Yoav+Goldberg%2E+2017%2E+Neural+network+methods+for+natural+language+processing%2E+Synthesis+Lectures+on+Human+Language+Technologies%2C+10%281%29%3A1%E2%80%93309%2E) 8. Amir More and Reut Tsarfaty. 2016. Data-driven morphological analysis and disambiguation for morphologically rich languages and universal dependencies. In Proceedings of COLING 2016. [find](https://scholar.google.com/scholar?q=Amir+More+and+Reut+Tsarfaty%2E+2016%2E+Data%2Ddriven+morphological+analysis+and+disambiguation+for+morphologically+rich+languages+and+universal+dependencies%2E+In+Proceedings+of+COLING+2016%2E) 9. Hamdy Mubarak, Ahmed Abdelali, Hassan Sajjad, Younes Samih, and Kareem Darwish. 2019. Highly effective Arabic diacritization using sequence to sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2390–2395, Minneapolis, Minnesota. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hamdy+Mubarak%2C+Ahmed+Abdelali%2C+Hassan+Sajjad%2C+Younes+Samih%2C+and+Kareem+Darwish%2E+2019%2E+Highly+effective+Arabic+diacritization+using+sequence+to+sequence+modeling%2E+In+Proceedings+of+the+2019+Conference+) 10. Mohsen Rashwan, Ahmad Sallab, Hazem Raafat, and Ahmed Rafea. 2015. Deep learning framework with confused sub-set resolution architecture for automatic arabic diacritization. IEEE Transactions on Audio Speech and Language Processing, 23. [find](https://scholar.google.com/scholar?q=Mohsen+Rashwan%2C+Ahmad+Sallab%2C+Hazem+Raafat%2C+and+Ahmed+Rafea%2E+2015%2E+Deep+learning+framework+with+confused+sub%2Dset+resolution+architecture+for+automatic+arabic+diacritization%2E+IEEE+Transactions+on+Audio) Updated 23 October 2026