--- title: "Nakdan: Professional Hebrew Diacritizer — Shmidman 2020" description: "Nakdan adds vowel points to undiacritized Hebrew, for modern, Rabbinic and poetic texts, through a web interface that lets an editor choose among the…" authors: ["Avi Shmidman", "Shaltiel Shmidman", "Moshe Koppel", "Yoav Goldberg"] year: 2020 venue: "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations" source: "https://aclanthology.org/2020.acl-demos.23/" doi: "10.18653/v1/2020.acl-demos.23" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/aclbulk-2020-shmidman-nakdan-professional-hebrew-diacritizer/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/aclbulk-2020-shmidman-nakdan-professional-hebrew-diacritizer/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/3/) / Nakdan: Professional Hebrew Diacritizer # Nakdan: Professional Hebrew Diacritizer **Authors:** [Avi Shmidman](https://md.research.elamur.ai/free/en/authors/avi-shmidman/); [Shaltiel Shmidman](https://md.research.elamur.ai/free/en/authors/shaltiel-shmidman/); [Moshe Koppel](https://md.research.elamur.ai/free/en/authors/moshe-koppel/); [Yoav Goldberg](https://md.research.elamur.ai/free/en/authors/yoav-goldberg/) · **Year:** 2020 · **Venue:** Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations **Link:** [aclanthology.org/2020.acl-demos.23](https://aclanthology.org/2020.acl-demos.23/) · library `aclbulk-2020-shmidman-nakdan-professional-hebrew-diacritizer` · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) [**12** citations](https://openalex.org/works?filter=cites:W3022124040) OpenAlex · 2026-10-14 · [**11** citations in our library](https://md.research.elamur.ai/free/en/papers/aclbulk-2020-shmidman-nakdan-professional-hebrew-diacritizer/cited-by/) of the 140673 papers whose bibliographies we read · [Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations](https://aclanthology.org/volumes/2020.acl-demos/) · [licence](https://creativecommons.org/licenses/by/4.0/) - [Publisher's page](https://aclanthology.org/2020.acl-demos.23/) - [PDF](https://aclanthology.org/2020.acl-demos.23.pdf) - [DOI 10.18653/v1/2020.acl-demos.23](https://doi.org/10.18653/v1/2020.acl-demos.23) - [OpenAlex](https://openalex.org/W3022124040) **Authors (4)** 1. [Avi Shmidman](https://openalex.org/A5013212409) [Semantic Scholar](https://www.semanticscholar.org/author/3361477) · h 5 · 114 citations · 53 works 2. [Shaltiel Shmidman](https://openalex.org/A5041361841) [Semantic Scholar](https://www.semanticscholar.org/author/1680059376) · h 3 · 27 citations · 21 works 3. [Moshe Koppel](https://openalex.org/A5019908505) [Semantic Scholar](https://www.semanticscholar.org/author/145200175) · h 37 · 6949 citations · 150 works 4. [Yoav Goldberg](https://openalex.org/A5028476919) [ORCID](https://orcid.org/0000-0002-6497-829X) · [Semantic Scholar](https://www.semanticscholar.org/author/79775260) · h 59 · 23282 citations · 401 works Citations and author metrics: OpenAlex, 2026-10-14; authors with the same name are not always told apart. ## What it is about Nakdan adds vowel points to undiacritized Hebrew, for modern, Rabbinic and poetic texts, through a web interface that lets an editor choose among the alternatives. The pipeline has three stages: a POS tagger and a morphological tagger (bi-LSTMs with a character model, trained on about 200K hand-tagged tokens, 92% coarse-POS and 79% full-morphology accuracy); a filter that intersects each word's candidate points with a hand-built lexicon of about 50K lexemes and about 5.5M inflected forms plus proper-name lists; and a bi-LSTM that ranks the surviving candidates in context, with beam search over the points of each letter. On a 6,000-word modern test, word-level accuracy is 88.2%, against 80.9% for Morfix and 66.4% for Snopi. The authors also report that knowing the points cuts the morphological ambiguity from 9.1 to 2.4 analyses per form, and knowing the morphology cuts the point ambiguity from 6.2 to 1.4. Rabbinic and poetic genres get their own models, and the Rabbinic test (Bet Yosef) is harder for all systems. ## Ideas - I1. Filter, then rank, for the unpointed mode of `tools/hbo`: take the segmenter's candidate analyses for a word, keep only those the lightest tagger allows, and rank the survivors. Report the average number of candidates before and after each filter, as the paper does. A random choice from the filtered set gives 87.1% on the modern test, which is the baseline a rule-based ranking must beat. - I2. A table of collocation overrides: the paper keeps about 10K hand-written pairs that fix the state of a word in a fixed phrase (the paper's example: the noun bayit is forced into its construct form inside one fixed two-word collocation, and into the plural construct inside its plural). Mova can keep such a table in [languages/hbo/04-noun.md](https://mova.elamur.ai/free/en/docs/languages/hbo/04-noun/) with the reason for each entry. - I3. Genre split: the paper states that the modern morphology model does not transfer to Rabbinic Hebrew and must be retrained on tagged historical text. Add a note to [languages/hbo/11-mishnaic.md](https://mova.elamur.ai/free/en/docs/languages/hbo/11-mishnaic/) that Mishnaic rules are tested on their own gold set, and do not reuse the Biblical tables unchanged. ## For Mova I1 is design for `tools/hbo` (unpointed mode) and its measure; I2 belongs in [languages/hbo/04-noun.md](https://mova.elamur.ai/free/en/docs/languages/hbo/04-noun/) and [languages/hbo/08-syntax.md](https://mova.elamur.ai/free/en/docs/languages/hbo/08-syntax/); I3 is a rule of testing for [languages/hbo/11-mishnaic.md](https://mova.elamur.ai/free/en/docs/languages/hbo/11-mishnaic/). The 6,000-word modern test corpus is released by the authors and can be used as a measurement of [languages/he/02-script-and-spelling.md](https://mova.elamur.ai/free/en/docs/languages/he/02-script-and-spelling/). Topics: [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) ## Cite The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2020.acl-demos.23](https://doi.org/10.18653/v1/2020.acl-demos.23) ```bibtex @inproceedings{shmidman2020nakdan, title = {{Nakdan: Professional Hebrew Diacritizer}}, author = {Shmidman, Avi and Shmidman, Shaltiel and Koppel, Moshe and Goldberg, Yoav}, booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations}, year = {2020}, doi = {10.18653/v1/2020.acl-demos.23}, url = {https://doi.org/10.18653/v1/2020.acl-demos.23} } ``` - [← previous MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines](https://md.research.elamur.ai/free/en/papers/zang-2020-multiwoz-2-2-dialogue-dataset/) - [next → Named Entity Recognition as Dependency Parsing](https://md.research.elamur.ai/free/en/papers/yu-2020-named-entity-recognition-as-dependency/) - [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/3/) ## References 10 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Gheith Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic diacritization of arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18:183–197. [↗](https://doi.org/10.1007/s10032-015-0242-2) 2. Meni Adler and Michael Elhadad. 2006. An unsupervised morpheme-based HMM for hebrew morphological disambiguation. In ACL 2006, 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Sydney, Australia, 17-21 July 2006. [find](https://scholar.google.com/scholar?q=Meni+Adler+and+Michael+Elhadad%2E+2006%2E+An+unsupervised+morpheme%2Dbased+HMM+for+hebrew+morphological+disambiguation%2E+In+ACL+2006%2C+21st+International+Conference+on+Computational+Linguistics+and+44th+Annua) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2006-adler-unsupervised-morpheme-based-hmm-hebrew/) 3. Yonatan Belinkov and James R. Glass. 2015. Arabic diacritization with recurrent neural networks. In EMNLP. [find](https://scholar.google.com/scholar?q=Yonatan+Belinkov+and+James+R%2E+Glass%2E+2015%2E+Arabic+diacritization+with+recurrent+neural+networks%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2015-belinkov-arabic-diacritization-recurrent-neural-networks/) 4. Yaacov Choueka and Yoni Neeman. 1995. Nakdan-text,(an in-context text-vocalizer for modern hebrew). In BISFAI-95, The Fifth Bar Ilan Symposium for Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Yaacov+Choueka+and+Yoni+Neeman%2E+1995%2E+Nakdan%2Dtext%2C%28an+in%2Dcontext+text%2Dvocalizer+for+modern+hebrew%29%2E+In+BISFAI%2D95%2C+The+Fifth+Bar+Ilan+Symposium+for+Artificial+Intelligence%2E) 5. Michael Elhadad, Yael Netzer, David Gabay, and Meni Adler. 2005. Hebrew morphological tagging guidelines. Technical report, Technical report, Ben-Gurion University, Dept. of Computer Science. [find](https://scholar.google.com/scholar?q=Michael+Elhadad%2C+Yael+Netzer%2C+David+Gabay%2C+and+Meni+Adler%2E+2005%2E+Hebrew+morphological+tagging+guidelines%2E+Technical+report%2C+Technical+report%2C+Ben%2DGurion+University%2C+Dept%2E+of+Computer+Science%2E) 6. Ya’akov Gal. 2002. An hmm approach to vowel restoration in arabic and hebrew. In Proceedings of the ACL-02 workshop on Computational approaches to semitic languages, pages 1–7. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ya%E2%80%99akov+Gal%2E+2002%2E+An+hmm+approach+to+vowel+restoration+in+arabic+and+hebrew%2E+In+Proceedings+of+the+ACL%2D02+workshop+on+Computational+approaches+to+semitic+languages%2C+pages+1%E2%80%937%2E+Association+for+Computa) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2002-gal-hmm-approach-vowel-restoration-arabic/) 7. Yoav Goldberg. 2017. Neural network methods for natural language processing. Synthesis Lectures on Human Language Technologies, 10(1):1–309. [↗](https://doi.org/10.1007/978-3-031-02165-7) 8. Amir More and Reut Tsarfaty. 2016. Data-driven morphological analysis and disambiguation for morphologically rich languages and universal dependencies. In Proceedings of COLING 2016. [find](https://scholar.google.com/scholar?q=Amir+More+and+Reut+Tsarfaty%2E+2016%2E+Data%2Ddriven+morphological+analysis+and+disambiguation+for+morphologically+rich+languages+and+universal+dependencies%2E+In+Proceedings+of+COLING+2016%2E) 9. Hamdy Mubarak, Ahmed Abdelali, Hassan Sajjad, Younes Samih, and Kareem Darwish. 2019. Highly effective Arabic diacritization using sequence to sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2390–2395, Minneapolis, Minnesota. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hamdy+Mubarak%2C+Ahmed+Abdelali%2C+Hassan+Sajjad%2C+Younes+Samih%2C+and+Kareem+Darwish%2E+2019%2E+Highly+effective+Arabic+diacritization+using+sequence+to+sequence+modeling%2E+In+Proceedings+of+the+2019+Conference+) 10. Mohsen Rashwan, Ahmad Sallab, Hazem Raafat, and Ahmed Rafea. 2015. Deep learning framework with confused sub-set resolution architecture for automatic arabic diacritization. IEEE Transactions on Audio Speech and Language Processing, 23. [↗](https://doi.org/10.1109/taslp.2015.2395255)