--- title: "Problem-oriented Corpus Annotation and the — Joode 2019" description: "A position paper arguing that biblical scholars should add their own research-specific annotations to open datasets such as ETCBC…" authors: ["Johan de Joode"] year: 2019 venue: "HIPHIL Novum 5" source: "https://doi.org/10.7146/hn.v5i2.142730" doi: "10.7146/hn.v5i2.142730" licence: "HIPHIL Novum vols 1–8: free to read, no CC licence (all rights reserved to the authors) — read only" licence_url: "https://tidsskrift.dk/hiphilnovum/about" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/hbo-2019-joode-problem-oriented-corpus-annotation-hebrew/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hbo-2019-joode-problem-oriented-corpus-annotation-hebrew/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / Problem-oriented Corpus Annotation and the Hebrew Bible # Problem-oriented Corpus Annotation and the Hebrew Bible **Authors:** Johan de Joode · **Year:** 2019 · **Venue:** HIPHIL Novum 5/2, 6– **Link:** [tidsskrift.dk/hiphilnovum/article/view/142730](https://tidsskrift.dk/hiphilnovum/article/view/142730) · **Licence:** copyrighted (free to read, no CC licence) — summary only [**0** citations](https://openalex.org/works?filter=cites:W4391938569) OpenAlex · 2026-10-14 · [HIPHIL Novum 5](https://openalex.org/S5406958382) · [licence](https://tidsskrift.dk/hiphilnovum/about) - [Publisher's page](https://doi.org/10.7146/hn.v5i2.142730) - [PDF](https://tidsskrift.dk/hiphilnovum/article/download/142730/186434) - [DOI 10.7146/hn.v5i2.142730](https://doi.org/10.7146/hn.v5i2.142730) - [OpenAlex](https://openalex.org/W4391938569) **Authors (1)** 1. [Johan de Joode](https://openalex.org/A5023540299) [ORCID](https://orcid.org/0000-0001-7209-880X) · [Semantic Scholar](https://www.semanticscholar.org/author/2488267102) · h 5 · 2070 citations · 33 works Citations and author metrics: OpenAlex, 2026-10-14; authors with the same name are not always told apart. ## What it is about A position paper arguing that biblical scholars should add their own research-specific annotations to open datasets such as ETCBC ("problem-oriented annotation", de Haan 1984), instead of free-text notes in closed tools. From the author's experience annotating conceptual metaphors in Job (Biblia Metaphorica) and from Fort et al. (2012) on what makes annotation hard, he recommends: start from a query, not from running text, so units are pre-segmented; annotate in several passes, one word class or feature at a time; use small fixed tagsets, binary where possible, with one label for "needs more reflection"; keep annotations modular, stand-off and keyed to stable identifiers, in a format humans and machines can read; and measure quality by agreement (proportion agreeing, kappa). He also notes that no exhaustive Hebrew Bible dataset was then free for commercial use. ## Ideas - I1. When a teacher model or a person labels Hebrew phenomena for Mova (pragmatic acts, metaphor, animacy), give it pre-segmented units from a tree query and a closed, small label set with an explicit "unsure" label, one feature per pass. - I2. Keep such labels stand-off, keyed by `sent_id` + token id, never edited into the CoNLL-U, so they survive reconversion of the trees. - I3. A two-pass metaphor finder for the Torah: first label noun animacy (binary) in the dictionary, then flag clauses whose inanimate subject governs a verb that usually takes an animate subject. ## For Mova I1 and I2 are conventions for `macula prag`'s teacher comparison and for the parse database ([languages/hbo/13-mova-tools.md](https://mova.elamur.ai/free/en/docs/languages/hbo/13-mova-tools/) §8, §10). I3 needs an animacy column in the dictionary builder (`10-lexicon.md`, semantic fields) and a count over the trees; its output is a candidate list, not a claim. ## Cite The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.7146/hn.v5i2.142730](https://doi.org/10.7146/hn.v5i2.142730) ```bibtex @inproceedings{joode2019problem, title = {{Problem-oriented Corpus Annotation and the Hebrew Bible}}, author = {Joode, Johan de}, booktitle = {HIPHIL Novum 5}, year = {2019}, doi = {10.7146/hn.v5i2.142730}, url = {https://doi.org/10.7146/hn.v5i2.142730} } ``` - [← previous Probing What Different NLP Tasks Teach Machines about Function Word Comprehension](https://md.research.elamur.ai/free/en/papers/kim-2019-probing-what-different-nlp-tasks/) - [next → Proceedings of the Third Workshop on Universal Dependencies (UDW, SyntaxFest 2019)](https://md.research.elamur.ai/free/en/papers/anon-2019-proceedings-third-workshop-universal-dependencies/) - [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) ## References 12 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Bayerl, Petra Saskia, and Karsten Ingmar Paul. (2011) “What Determines Inter-Coder Agreement in Manual Annotations? A Meta-Analytic Investigation” Computational Linguistics 37:4, 699–725. Available online from http://www.mitpressjournals.org/doi/10.1162/COLI_a_00074 [↗](https://doi.org/10.1162/coli_a_00074) 2. Carletta, J. (1996) Assessing Agreement on Classification Tasks: The Kappa Statistic. Computational Linguistics 22:2. Available online from http://homepages.inf.ed.ac.uk/jeanc/squib.pdf [↗](https://doi.org/10.48550/arxiv.cmp-lg/9602004) [our card](https://md.research.elamur.ai/free/en/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/) 3. de Haan P. (1984) “Problem-oriented tagging of English corpus data.” In Corpus linguistics, J. Aarts and W. Meijs (eds.). Amsterdam: Rodopi. [↗](https://doi.org/10.1163/9789004483446_009) 4. Fort, Karën, Adeline Nazarenko, and Sophie Rosset. (2012) “Modeling the Complexity of Manual Annotation Tasks: A Grid of Analysis.” Proc. COLING 2012 Tech. Pap. Pages 895–910, COLING 2012, Mumbai, December 2012. [find](https://scholar.google.com/scholar?q=Fort%2C+Kar%C3%ABn%2C+Adeline+Nazarenko%2C+and+Sophie+Rosset%2E+%282012%29+%E2%80%9CModeling+the+Complexity+of+Manual+Annotation+Tasks%3A+A+Grid+of+Analysis%2E%E2%80%9D+Proc%2E+COLING+2012+Tech%2E+Pap%2E+Pages+895%E2%80%93910%2C+COLING+2012%2C+Mumbai%2C+Dec) 5. Leech, G. (2005) “Adding Linguistic Annotation” in Developing Linguistic Corpora: a Guide to Good Practice. M. Wynne (ed.). Oxford: Oxbow Books: 17-29. Available online from [http://ota.ox.ac.uk/docu-ments/creating/dlc/](http://ota.ox.ac.uk/docu-ments/creating/dlc/) [↗](http://ota.ox.ac.uk/docu-ments/creating/dlc/) 6. Mahlberg M., Stockwell P., de Joode J., Smith C., O’Donnell M. (2016). “CLiC Dickens: Novel Uses of Concordances for the Integration of Corpus Stylistics and Cognitive Poetics.” Corpora, 11 (3), 433-463. [↗](https://doi.org/10.3366/cor.2016.0102) 7. Manning, Christopher D., Prabhakar Ragahvan, and Hinrich Schutze. (2009) “An Introduction to Information Retrieval.” Information Retrieval. Cambridge: Cambridge University Press. Available online from [http://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf](http://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf) 6 Amazon’s Mechanical Turk can make this even more scalable. McEnery, Tony, and Andrew. [↗](http://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf) 8. Hardie. (2012) Corpus Linguistics Method, Theory and Practice. Cambridge: Cambridge University Press. [find](https://scholar.google.com/scholar?q=Hardie%2E+%282012%29+Corpus+Linguistics+Method%2C+Theory+and+Practice%2E+Cambridge%3A+Cambridge+University+Press%2E) 9. McEnery, Tony, and Andrew Wilson. (2011) Corpus Linguistics: An Introduction. 2nd edition. Edinburgh: Edinburgh University Press. [find](https://scholar.google.com/scholar?q=McEnery%2C+Tony%2C+and+Andrew+Wilson%2E+%282011%29+Corpus+Linguistics%3A+An+Introduction%2E+2nd+edition%2E+Edinburgh%3A+Edinburgh+University+Press%2E) 10. McIntyre, Dan. (2007) “Trusting the Text: Corpus Linguistics and Stylistics.” International Journal of Corpus Linguistics 12:4, 563–75. [find](https://scholar.google.com/scholar?q=McIntyre%2C+Dan%2E+%282007%29+%E2%80%9CTrusting+the+Text%3A+Corpus+Linguistics+and+Stylistics%2E%E2%80%9D+International+Journal+of+Corpus+Linguistics+12%3A4%2C+563%E2%80%9375%2E) 11. Pragglejaz Group. (2007). “MIP: A Method for Identifying Metaphorically Used Words in Discourse.” Metaphor and Symbol 22:1, 1-39. [↗](https://doi.org/10.1080/10926480709336752) 12. Sinclair, John. (2004) Trust the Text: Language, Corpus and Discourse. London: Routledge, 2004. [↗](https://doi.org/10.4324/9780203594070)