--- title: "The INCEpTION Platform — Klie 2018" description: "INCEpTION is an open-source, web-based annotation platform for semantic annotation tasks such as entity linking, fact linking, knowledge base population…" authors: ["Jan-Christoph Klie", "Michael Bugert", "Beto Boullosa", "Richard Eckart de Castilho", "Iryna Gurevych"] year: 2018 venue: "Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations" source: "https://aclanthology.org/C18-2002/" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/klie-2018-inception-platform-machine-assisted-knowledge/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/klie-2018-inception-platform-machine-assisted-knowledge/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/) / The INCEpTION Platform: Machine-Assisted and Knowledge-Oriented Interactive Annotation # The INCEpTION Platform: Machine-Assisted and Knowledge-Oriented Interactive Annotation **Authors:** [Jan-Christoph Klie](https://md.research.elamur.ai/free/en/authors/jan-christoph-klie/), [Michael Bugert](https://md.research.elamur.ai/free/en/authors/michael-bugert/), Beto Boullosa, [Richard Eckart de Castilho](https://md.research.elamur.ai/free/en/authors/richard-eckart-de-castilho/), [Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/) · **Year:** 2018 · **Venue:** Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/C18-2002/) · [PDF](https://aclanthology.org/C18-2002.pdf) · [OpenAlex](https://openalex.org/W2889470921) **Citations:** 186 citations (openalex-snapshot · 2026-10-14) · [236](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary INCEpTION is an open-source, web-based annotation platform for semantic annotation tasks such as entity linking, fact linking, knowledge base population and semantic frame annotation. These tasks are slow and require annotators to know the underlying knowledge resource well. Building on WebAnno and UIMA, the authors combine three capabilities in a single tool: machine assistance, built-in knowledge management, and a modular, extensible architecture. Recommenders display label suggestions inline next to existing annotations. They retrain continuously as the user accepts, rejects or adds labels. An optional active learning mode uses uncertainty sampling to direct the user to useful suggestions without restricting free annotation. RDF knowledge bases can be built and edited inside the tool or queried remotely via SPARQL, for example Wikidata or DBPedia. As a system demonstration, the paper gives no quantitative evaluation. It reports only qualitative feedback from use cases such as FAMULUS, where annotators said an external recommender improved both speed and quality. The main contribution is cheaper bootstrapping of annotated corpora for new domains. ## Key points - The annotation schema consists of user-defined layers, each either a span or a relation between spans. Features attached to these layers may take text, numeric or true/false values, point to knowledge base concepts, or link to other annotations. - Recommenders come in two forms: - **Internal:** implemented through a Java interface. - **External:** reached over a simple HTTP protocol that exchanges UIMA CAS XMI. A quality threshold can be set for each recommender so that weak models do not flood users with bad suggestions early on. - Active learning uses uncertainty sampling and works on one layer at a time. The user can accept, reject or skip a suggestion. Skipped items come back later, and all decisions go into a learning history that can be reviewed and undone. - Knowledge management supports classes, instances, properties and qualifiers. A configurable mapping handles Wikidata, DBPedia, OWL, CIDOC-CRM and SKOS, though advanced OWL features are deliberately not covered. A context-aware candidate ranking, adapted from Sorokin and Gurevych (2018), drives entity linking suggestions. - The authors contrast INCEpTION with earlier tools: - WebAnno requires manual retraining, offers a single algorithm and shows suggestions in a split screen. - Prodi.gy forces users to follow the active learning order. Reported use cases are FAMULUS (argument components in medical case reports), EDoHa (hypothesis/evidence linking) and entity re-ranking. The software is released under Apache License 2.0. ## Interesting ideas - **Stage-dependent recommenders.** Several recommenders run at once. Cheap, high-precision ones such as a dynamic dictionary help early on, and context-aware, recall-oriented sequence classifiers take over once more data exists. This is a practical way to bootstrap any new-domain corpus where little training data exists at first. - **Advisory active learning.** The system highlights the item it wants judged, but the annotator can still edit anything else, and every edit feeds back into the models. This keeps annotators productive and avoids the frustration of a rigid labeling queue. - **Models as separate HTTP services.** External recommenders run behind a simple HTTP interface. A crashing model cannot bring down the tool, heavy models can sit on other machines, and any programming language works. This makes it easy to plug existing pretrained models, such as Python deep learning ones, into an annotation workflow. - **Event-driven modules that double as logging.** The same event system that keeps recommenders updated also records user and system actions. These logs let project managers assess annotators and let researchers study whether assistance biases the labels, which is a useful evaluation habit for any assisted-annotation project. - **Growing the knowledge base during annotation.** Fact linking lets users mark subjects, predicates and objects in text to populate a knowledge base. This suits domains where no suitable knowledge base exists yet and one has to be built alongside the corpus. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 236 papers of our library cite it (of the 140884 whose bibliographies we read); 15 of them have a page here, 221 not yet. - [From Linguistics to Practice: a Case Study of Offensive Language Taxonomy in Hebrew](https://md.research.elamur.ai/free/en/papers/aclbulk-2024-liebeskind-linguistics-practice-case-study-offensive/) - [Visual Representations of Temporal Relations between Events and Time Expressions in News Stories](https://md.research.elamur.ai/free/en/papers/amorim-2025-visual-representations-temporal-relations-between/) - [Annotation de la continuité référentielle dans un corpus scolaire – premiers résultats](https://md.research.elamur.ai/free/en/papers/barletta-2024-annotation-de-la-continuite-referentielle/) - [Generalizing Cross-Document Event Coreference Resolution Across Multiple Corpora](https://md.research.elamur.ai/free/en/papers/bugert-2021-generalizing-cross-document-event-coreference/) - [Annotating Mentions Alone Enables Efficient Domain Adaptation for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/gandhi-2023-annotating-mentions-alone-enables-efficient/) - [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://md.research.elamur.ai/free/en/papers/hedderich-2021-survey-recent-approaches-natural-language/) - [AraMIP: Extending MIPVU Towards Metaphor Identification in Arabic](https://md.research.elamur.ai/free/en/papers/marathe-2026-aramip-extending-mipvu-towards-metaphor/) - [A Corpus of Persuasion Techniques in Slavic Languages](https://md.research.elamur.ai/free/en/papers/piskorski-2026-corpus-persuasion-techniques-slavic-languages/) - [ConGA: Guidelines for Contextual Gender Annotation. a Framework for Annotating Gender in Machine Translation](https://md.research.elamur.ai/free/en/papers/rescigno-2026-conga-guidelines-contextual-gender-annotation/) - [CoBra: A Compound Branching Resource for Nominal Triconstituent Compounds in English and German](https://md.research.elamur.ai/free/en/papers/schacht-2026-cobra-compound-branching-resource-nominal/) - [GePaDeSE: A New Resource for Clause-Level Aspect in German Parliamentary Debates](https://md.research.elamur.ai/free/en/papers/schlenker-2026-gepadese-new-resource-clause-level/) - [CorSpell: Introducing a Semiautomatic Tool for Spelling Normalization in Brazilian Portuguese](https://md.research.elamur.ai/free/en/papers/schoffen-2026-corspell-introducing-semiautomatic-tool-spelling/) - [LLMs Out-of-the-Box Do Not Generate Context-Appropriate Word Order in Russian](https://md.research.elamur.ai/free/en/papers/shabaeva-2026-llms-out-box-do-not/) - [Integrating TEI, NER/NEL, Textometry, and Linked Data for a Semantically Enriched Interview Corpus](https://md.research.elamur.ai/free/en/papers/stankovic-2026-integrating-tei-ner-nel-textometry/) - [CATS: An Annotation Scheme of Causality and Temporal Structure](https://md.research.elamur.ai/free/en/papers/yu-2026-cats-annotation-scheme-causality-temporal/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{klie2018inception, title = {{The INCEpTION Platform: Machine-Assisted and Knowledge-Oriented Interactive Annotation}}, author = {Klie, Jan-Christoph and Bugert, Michael and Boullosa, Beto and Castilho, Richard Eckart de and Gurevych, Iryna}, booktitle = {Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations}, year = {2018}, url = {https://aclanthology.org/C18-2002/} } ``` - [← previous The IIT Bombay English-Hindi Parallel Corpus](https://md.research.elamur.ai/free/en/papers/kunchukuttan-2018-iit-bombay-english-hindi-parallel/) - [next → The NarrativeQA Reading Comprehension Challenge](https://md.research.elamur.ai/free/en/papers/kocisky-2018-narrativeqa-reading-comprehension-challenge/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/) ## References 10 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Kalina Bontcheva, Hamish Cunningham, Ian Roberts, Angus Roberts, Valentin Tablan, Niraj Aswani, and Genevieve Gorrell. 2013. GATE Teamware: A Web-based Collaborative Text Annotation Framework. Language Resources and Evaluation, 47(4):1007–1029. [find](https://scholar.google.com/scholar?q=Kalina+Bontcheva%2C+Hamish+Cunningham%2C+Ian+Roberts%2C+Angus+Roberts%2C+Valentin+Tablan%2C+Niraj+Aswani%2C+and+Genevieve+Gorrell%2E+2013%2E+GATE+Teamware%3A+A+Web%2Dbased+Collaborative+Text+Annotation+Framework%2E+Languag) 2. Christian Girardi Emanuele Pianta and Roberto Zanoli. 2008. The TextPro Tool Suite. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), pages 2603–2607. [find](https://scholar.google.com/scholar?q=Christian+Girardi+Emanuele+Pianta+and+Roberto+Zanoli%2E+2008%2E+The+TextPro+Tool+Suite%2E+In+Proceedings+of+the+Sixth+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9908%29%2C+pages+2603%E2%80%932607) 3. David Ferrucci, Adam Lally, Karin Verspoor, and Eric Nyberg. 2009. Unstructured information management architecture (UIMA) version 1.0. OASIS Standard. [find](https://scholar.google.com/scholar?q=David+Ferrucci%2C+Adam+Lally%2C+Karin+Verspoor%2C+and+Eric+Nyberg%2E+2009%2E+Unstructured+information+management+architecture+%28UIMA%29+version+1%2E0%2E+OASIS+Standard%2E) 4. David D. Lewis and William A. Gale. 1994. A Sequential Algorithm for Training Text Classifiers. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3–12. [find](https://scholar.google.com/scholar?q=David+D%2E+Lewis+and+William+A%2E+Gale%2E+1994%2E+A+Sequential+Algorithm+for+Training+Text+Classifiers%2E+In+Proceedings+of+the+17th+Annual+International+ACM+SIGIR+Conference+on+Research+and+Development+in+Info) 5. Philip V. Ogren. 2006. Knowtator: A Protégé plug-in for annotated corpus construction. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Demonstrations, pages 273–275. [find](https://scholar.google.com/scholar?q=Philip+V%2E+Ogren%2E+2006%2E+Knowtator%3A+A+Prote%CC%81ge%CC%81+plug%2Din+for+annotated+corpus+construction%2E+In+Proceedings+of+the+Human+Language+Technology+Conference+of+the+NAACL%2C+Companion+Volume%3A+Demonstrations%2C+page) 6. Frédéric Papazian, Robert Bossy, and Claire Nédellec. 2012. AlvisAE: a collaborative Web text annotation editor for knowledge acquisition. In Proceedings of the Sixth Linguistic Annotation Workshop, pages 149–152. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Fre%CC%81de%CC%81ric+Papazian%2C+Robert+Bossy%2C+and+Claire+Ne%CC%81dellec%2E+2012%2E+AlvisAE%3A+a+collaborative+Web+text+annotation+editor+for+knowledge+acquisition%2E+In+Proceedings+of+the+Sixth+Linguistic+Annotation+Workshop) 7. Claudia Schulz, Michael Sailer, Jan Kiesewetter, Christian M. Meyer, Iryna Gurevych, Frank Fischer, and Martin R. Fischer. 2017. Fallsimulationen und automatisches adaptives Feedback mittels Künstlicher Intelligenz in digitalen Lernumgebungen. e-teaching.org Themenspecial ”Was macht Lernen mit digi… [find](https://scholar.google.com/scholar?q=Claudia+Schulz%2C+Michael+Sailer%2C+Jan+Kiesewetter%2C+Christian+M%2E+Meyer%2C+Iryna+Gurevych%2C+Frank+Fischer%2C+and+Martin+R%2E+Fischer%2E+2017%2E+Fallsimulationen+und+automatisches+adaptives+Feedback+mittels+Ku%CC%88nstlic) 8. Daniil Sorokin and Iryna Gurevych. 2018. Mixing context granularities for improved entity linking on question answering data across entity categories. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics (*SEM), pages 65–75. [find](https://scholar.google.com/scholar?q=Daniil+Sorokin+and+Iryna+Gurevych%2E+2018%2E+Mixing+context+granularities+for+improved+entity+linking+on+question+answering+data+across+entity+categories%2E+In+Proceedings+of+the+Seventh+Joint+Conference+on) 9. Chris Stahlhut, Christian Stab, and Iryna Gurevych. 2018. Pilot experiments of hypothesis validation through evidence detection for historians. In Proceedings of Design of Experimental Search & Information REtrieval Systems (DESIRES). (in press). [find](https://scholar.google.com/scholar?q=Chris+Stahlhut%2C+Christian+Stab%2C+and+Iryna+Gurevych%2E+2018%2E+Pilot+experiments+of+hypothesis+validation+through+evidence+detection+for+historians%2E+In+Proceedings+of+Design+of+Experimental+Search+%26+Inform) 10. Seid Muhie Yimam, Chris Biemann, Richard Eckart de Castilho, and Iryna Gurevych. 2014. Automatic Annotation Suggestions and Custom Annotation Layers in WebAnno. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 91–96. [find](https://scholar.google.com/scholar?q=Seid+Muhie+Yimam%2C+Chris+Biemann%2C+Richard+Eckart+de+Castilho%2C+and+Iryna+Gurevych%2E+2014%2E+Automatic+Annotation+Suggestions+and+Custom+Annotation+Layers+in+WebAnno%2E+In+Proceedings+of+52nd+Annual+Meeting+o)