--- title: "Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language — Shmidman 2022" description: "The authors argue that Modern Hebrew models (HeBERT, AlephBERT) falter on Rabbinic texts because those texts mix in… 9 references, cited by 8 papers here." authors: ["Avi Shmidman", "Joshua Guedalia", "Shaltiel Shmidman", "Cheyn Shmuel Shmidman", "Eli Handel", "Moshe Koppel"] year: 2022 venue: "arXiv:2208.01875" source: "https://arxiv.org/abs/2208.01875" arxiv: "2208.01875" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/shmidman-2022-introducing-berel-bert-embeddings-rabbinic/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shmidman-2022-introducing-berel-bert-embeddings-rabbinic/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) / Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language # Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language **Authors:** [Avi Shmidman](https://md.research.elamur.ai/free/en/authors/avi-shmidman/), [Joshua Guedalia](https://md.research.elamur.ai/free/en/authors/joshua-guedalia/), [Shaltiel Shmidman](https://md.research.elamur.ai/free/en/authors/shaltiel-shmidman/), [Cheyn Shmuel Shmidman](https://md.research.elamur.ai/free/en/authors/cheyn-shmuel-shmidman/), Eli Handel and [Moshe Koppel](https://md.research.elamur.ai/free/en/authors/moshe-koppel/) · **Year:** 2022 · **Venue:** arXiv 2208.01875 (DICTA) **Link:** [arxiv.org/abs/2208.01875](https://arxiv.org/abs/2208.01875) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) [**5** citations](https://openalex.org/works?filter=cites:W4302329960) OpenAlex, 2026-10-14 · [**8** citations in our library](https://md.research.elamur.ai/free/en/papers/shmidman-2022-introducing-berel-bert-embeddings-rabbinic/cited-by/) of the 140,884 papers whose bibliographies we read · [arXiv:2208.01875](https://openalex.org/S4306400194) · [licence](http://creativecommons.org/licenses/by/4.0/) - [Publisher's page](https://arxiv.org/abs/2208.01875) - [PDF](https://arxiv.org/pdf/2208.01875) - [arXiv 2208.01875](https://arxiv.org/abs/2208.01875) - [OpenAlex](https://openalex.org/W4302329960) **Authors (6)** 1. [Avi Shmidman](https://openalex.org/A5013212409) [Semantic Scholar](https://www.semanticscholar.org/author/3361477) · h 5 · 114 citations · 53 works 2. [Joshua Guedalia](https://openalex.org/A5059119329) [ORCID](https://orcid.org/0000-0002-3458-7896) · [Semantic Scholar](https://www.semanticscholar.org/author/1490908142) · h 14 · 487 citations · 48 works 3. [Shaltiel Shmidman](https://openalex.org/A5041361841) [Semantic Scholar](https://www.semanticscholar.org/author/1680059376) · h 3 · 27 citations · 21 works 4. [Cheyn Shmuel Shmidman](https://openalex.org/A5015862561) [Semantic Scholar](https://www.semanticscholar.org/author/2180164925) · h 3 · 13 citations · 6 works 5. [Eli Handel](https://openalex.org/A5060379507) [Semantic Scholar](https://www.semanticscholar.org/author/2180164617) · h 1 · 5 citations · 1 works 6. [Moshe Koppel](https://openalex.org/A5019908505) [Semantic Scholar](https://www.semanticscholar.org/author/145200175) · h 37 · 6,949 citations · 150 works Citations and author metrics: OpenAlex, 2026-10-14; authors with the same name are not always told apart. ## What it is about A short paper releasing a BERT-base model trained on about 220 million words of Rabbinic Hebrew (the Sefaria and Dicta libraries). The authors argue that Modern Hebrew models (HeBERT, AlephBERT) falter on Rabbinic texts because those texts mix in Talmudic Aramaic (even Aramaic affixes on Hebrew words), spell plene and defective freely, and abbreviate one word in every five to ten (apocopes with an apostrophe, acronyms with a gershayim). Their tokenizer change is to keep an abbreviation whole instead of splitting at the apostrophe or quotation mark. They test on a challenge set of homographs and release the model and the set for unrestricted use; the extracted text holds no detailed numbers. ## Ideas - I1. Tokenisation: treat a word with ׳ or ״ inside as one token (abbreviation or acronym) before any prefix splitting; a rule worth having in the Mishnaic and later-Hebrew readers, and a list of common Rabbinic abbreviations with their expansions. - I2. Spelling normalisation: map plene and defective spellings to one consonantal key (drop medial ו/י as matres) when looking a word up in the Torah dictionary; measure how many Mishnah word types then find a Torah lemma. ## For Mova - I3. The homograph challenge set, if its licence is open as stated, is a ready test for any sense or lemma choice Mova makes on unvocalised Hebrew. I1 belongs to [languages/hbo/11-mishnaic.md](https://mova.elamur.ai/free/en/docs/languages/hbo/11-mishnaic/) (orthography, abbreviations) and to whatever tokeniser reads Mishnaic text; I2 to the dictionary builder lookup and `12-biblical-mishnaic-modern.md` (spelling across stages). I3 is a measure for a future unvocalised-Hebrew lemmatiser; the model itself is a neural teacher at most, never the answerer. ## Institutions [🎓**Bar-Ilan University** Ramat Gan · Israel 102 works · 17,587 citations](https://md.research.elamur.ai/free/en/universities/bar-ilan-university/) Also on the byline: [Jerusalem Institute for Israel Studies](https://ror.org/02zax3c17). 📚 Shelves 5[Computer Science · 7,204 works](https://md.research.elamur.ai/free/en/shelves/computer-science/) · [Artificial Intelligence · 7,042 works](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · [Arts and Humanities · 7,381 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · [Language and Linguistics · 7,325 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · [Literature and Literary Theory · 223 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/literature-and-literary-theory/) 🧭 Topics 3[Semitic languages · 1,336 works](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · [Hebrew & Aramaic · 474 works](https://md.research.elamur.ai/free/en/topics/hebrew-aramaic/) · [Biblical Hebrew · 254 works](https://md.research.elamur.ai/free/en/topics/biblical-hebrew/) ## Cite Cite as Avi Shmidman, Joshua Guedalia, Shaltiel Shmidman, Cheyn Shmuel Shmidman, Eli Handel and Moshe Koppel. 2022. Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language. arXiv:2208.01875. The paper itself, not this card. BibTeX from the library's record of the work. [arXiv 2208.01875](https://arxiv.org/abs/2208.01875) ```bibtex @misc{shmidman2022introducing, title = {{Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language}}, author = {Avi Shmidman, Joshua Guedalia, Shaltiel Shmidman, Cheyn Shmuel Shmidman, Eli Handel and Moshe Koppel}, year = {2022}, eprint = {2208.01875}, archivePrefix = {arXiv}, url = {https://arxiv.org/abs/2208.01875} } ``` - [← previous ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccination](https://md.research.elamur.ai/free/en/papers/mubarak-2022-arcovidvac-analyzing-arabic-tweets-about/) - [next → Sarcasm Detection is Way Too Easy! An Empirical Comparison of Human and Machine Sarcasm Detection](https://md.research.elamur.ai/free/en/papers/farha-2022-sarcasm-detection-is-way-too/) - [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) ## References 9 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Avihay Chriqui and Inbal Yahav. 2021. Hebert & hebemo: a hebrew bert model and a tool for polarity analysis and emotion recognition. [our card](https://md.research.elamur.ai/free/en/papers/chriqui-2021-hebert-hebemo-hebrew-bert-model/) 2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423) [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/) 3. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meet ing of the Association for Computational Linguistics, pages 7871 7880, Online. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/) 4. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E) 5. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. [our card](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/) 6. Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Shaked Greenfeld, and Reut Tsarfaty. 2021. Alephbert: A hebrew large pre-trained language model to start-off your hebrew NLP application with. CoRR, abs/2104.04052. [↗](https://arxiv.org/abs/2104.04052) [our card](https://md.research.elamur.ai/free/en/papers/seker-2021-alephbert-hebrew-large-pre-trained/) 7. Reut Tsarfaty, Amit Seker, Shoval Sadde, and Stav Klein. 2019. What’s wrong with Hebrew NLP? and how to make it right. CoRR, abs/1908.05453. [↗](https://arxiv.org/abs/1908.05453) [our card](https://md.research.elamur.ai/free/en/papers/tsarfaty-2019-what-s-wrong-hebrew-nlp/) 8. Shuly Wintner. 2014. Morphological Processing of Semitic Languages, pages 43–66. Springer Berlin Heidelberg, Berlin, Heidelberg. [find](https://scholar.google.com/scholar?q=Shuly+Wintner%2E+2014%2E+Morphological+Processing+of+Semitic+Languages%2C+pages+43%E2%80%9366%2E+Springer+Berlin+Heidelberg%2C+Berlin%2C+Heidelberg%2E) 9. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. CoRR, abs/1910.03771. [↗](https://arxiv.org/abs/1910.03771) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/) Updated 23 October 2026