---
title: "A Web-Based Tool for Analysing Normative Documents in English"
description: "Contracts, privacy policies, licences and service-level agreements are long, and their clauses refer to each other, so a reader… 23 references."
authors: ["John J. Camilleri", "Mohammad Reza Haghshenas", "Gerardo Schneider"]
year: 2017
venue: "arXiv:1707.03997"
source: "https://arxiv.org/abs/1707.03997"
doi: "10.48550/arxiv.1707.03997"
arxiv: "1707.03997"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/camilleri-2017-web-based-tool-analysing-normative/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/camilleri-2017-web-based-tool-analysing-normative/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/4/) / A Web-Based Tool for Analysing Normative Documents in English

# A Web-Based Tool for Analysing Normative Documents in English

**Authors:** John J. Camilleri, Mohammad Reza Haghshenas and Gerardo Schneider · **Year:** 2017 · **Venue:** arXiv:1707.03997 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1707.03997) · [PDF](https://arxiv.org/pdf/1707.03997) · [DOI 10.48550/arxiv.1707.03997](https://doi.org/10.48550/arxiv.1707.03997) · [arXiv 1707.03997](https://arxiv.org/abs/1707.03997) · [OpenAlex](https://openalex.org/W2735370250)

**Citations:** 9 citations (OpenAlex, 2026-10-14)

## Summary
Contracts, privacy policies, licences and service-level agreements are long, and their clauses refer to each other, so a reader cannot easily answer questions such as whether their data may be shared or what they are committing to. Keyword search handles exceptions and cross-references poorly. The authors combine several components from their earlier work into one web application, the Contract Verifier.

The pipeline works as follows:
- The ConPar extractor reads Stanford Parser dependency trees and fills an editable table of clauses.
- The user corrects that table.
- A script turns the table into a formal model based on C-O Diagrams, stored in an XML format.
- The model is shown back to the user in a controlled English and in a compact formal notation.
- The user picks questions from English templates. Some are answered by walking the model's structure. Others are answered by model checking after translation to UPPAAL timed automata.

A university-course rulebook serves as the running example. The tool is a proof of concept and has had no full user study. Its value is showing a complete, explainable path from text to verifiable norms.

## Key points
- ConPar extracts four kinds of information per clause: agent (subject), action (verb plus object), modality (obligation, permission, prohibition) and temporal or other conditions. It also splits a sentence into sub-clauses joined by conjunction, choice or sequence, and outputs TSV.
- Earlier evaluation of extraction on a test set of 4 documents gave F1 between 0.49 and 0.86. Parsing plus extraction takes roughly half a second per sentence.
- Models are stored as COML, an XML format based on C-O Diagrams. They are displayed three ways:
  - as the post-edited text,
  - as a CNL built in Grammatical Framework (GF), using a lexicon of over 64,000 entries,
  - as the CODSH shorthand.
- Every clause has its own identifier and a clock that restarts when the clause becomes active. Users can reference these to state relative deadlines.
- Structural queries take time linear in the model size. Semantic queries run in UPPAAL and suffer from state-space explosion, so they are realistic only for contracts with tens of clauses.

## Interesting ideas
- The tool places a plain editable table between automatic extraction and the formal model, so people fix errors in rows rather than in the formalism. This makes imperfect extraction usable.
- The finished model is verbalised back into a restricted English, letting the user check what the system actually understood. If a word is missing from the lexicon, only this view fails and the rest of the tool keeps working.
- Deontic modality is modelled as a first-class property of a clause. Sentences are refined into sub-clauses linked by and, or, or then, and a clause can carry a penalty for violation.
- When a query gives an unexpected answer, the system treats it as evidence of one of two problems: a mistake in the model, or a genuine flaw in the source text, such as an obligation with no deadline.
- Questions are split into a cheap tier answered by traversing the model and an expensive tier answered by temporal model checking. Per-clause clocks that restart on activation express relative time.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): Dump extracted frames (agent, action, modality, conditions) as one row each, so annotators can correct them quickly and the corrections can be compared against parser output.
- **I2** (back): Generate a controlled-English rendering of the meaning graph as an explainability check, and treat lexicon gaps as a local, reported failure rather than a crash.
- **I3** (frames): Add obligation, permission and prohibition, plus reparation links, to the frame layer so rule-like texts (school rules, fable morals) yield structured norms instead of plain events.
- **I4** (answer): When a reader's answer looks wrong, classify it as either a misreading or an under-specified source text, and report which one it is.
- **I5** (world): Answer most questions by direct graph lookup and reserve costly temporal reasoning for event-order questions, anchoring relative deadlines to when each state or event begins.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Gothenburg](https://md.research.elamur.ai/free/en/universities/university-of-gothenburg/) — University, Gothenburg, Sweden · 20 works · 1,231 citations
- [Chalmers University of Technology](https://md.research.elamur.ai/free/en/universities/chalmers-university-of-technology/) — University, Gothenburg, Sweden · 10 works · 493 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 744 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 116 works · [🤝 Political Science and International Relations](https://md.research.elamur.ai/free/en/shelves/social-sciences/political-science-and-international-relations/) · 72 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works

## Cite

Cite as: John J. Camilleri, Mohammad Reza Haghshenas and Gerardo Schneider. 2017. A Web-Based Tool for Analysing Normative Documents in English. arXiv:1707.03997.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{camilleri2017web,
  title = {{A Web-Based Tool for Analysing Normative Documents in English}},
  author = {John J. Camilleri, Mohammad Reza Haghshenas and Gerardo Schneider},
  year = {2017},
  doi = {10.48550/arxiv.1707.03997},
  eprint = {1707.03997},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1707.03997}
}
```

- [← previous LIMSI@CoNLL'17: UD Shared Task](https://md.research.elamur.ai/free/en/papers/aufrant-2017-limsi-conll-17-ud-shared/)
- [next → Developing Cooperative Learning Based E-Module to Teach Basic English Grammar of the First Semester of English Study Program Students at FKIP – UIR](https://md.research.elamur.ai/free/en/papers/ahmad-2017-developing-cooperative-learning-based-e/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/4/)

## References

23 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alur, R., Dill, D.L.: A Theory of Timed Automata. Theoretical Computer Science 126(2), 183–235 (1994) [find](https://scholar.google.com/scholar?q=Alur%2C+R%2E%2C+Dill%2C+D%2EL%2E%3A+A+Theory+of+Timed+Automata%2E+Theoretical+Computer+Science+126%282%29%2C+183%E2%80%93235+%281994%29)
2. Angelov, K., Camilleri, J.J., Schneider, G.: A Framework for Conflict Analysis of Normative Texts Written in Controlled Natural Language. JLAP 82(5-7), 216–240 (2013) [find](https://scholar.google.com/scholar?q=Angelov%2C+K%2E%2C+Camilleri%2C+J%2EJ%2E%2C+Schneider%2C+G%2E%3A+A+Framework+for+Conflict+Analysis+of+Normative+Texts+Written+in+Controlled+Natural+Language%2E+JLAP+82%285%2D7%29%2C+216%E2%80%93240+%282013%29)
3. Athan, T., Governatori, G., Palmirani, M., Paschke, A., Wyner, A.: LegalRuleML: Design principles and foundations. In: RW 2015. pp. 1–37. Berlin (2015) Tests carried out on a dual-core MacBook Air from 2013. [find](https://scholar.google.com/scholar?q=Athan%2C+T%2E%2C+Governatori%2C+G%2E%2C+Palmirani%2C+M%2E%2C+Paschke%2C+A%2E%2C+Wyner%2C+A%2E%3A+LegalRuleML%3A+Design+principles+and+foundations%2E+In%3A+RW+2015%2E+pp%2E+1%E2%80%9337%2E+Berlin+%282015%29+Tests+carried+out+on+a+dual%2Dcore+MacBook+Air+fro)
4. Azzopardi, S., Gatt, A., Pace, G.J.: Integrating natural language and formal analysis for legal documents. In: Conference on Language Technologies and Digital Humanities 2016 (2016) [find](https://scholar.google.com/scholar?q=Azzopardi%2C+S%2E%2C+Gatt%2C+A%2E%2C+Pace%2C+G%2EJ%2E%3A+Integrating+natural+language+and+formal+analysis+for+legal+documents%2E+In%3A+Conference+on+Language+Technologies+and+Digital+Humanities+2016+%282016%29)
5. Azzopardi, S., Gatt, A., Pace, G.J.: Reasoning about partial contracts. In: JURIX 2016. pp. 23–32 (2016) [find](https://scholar.google.com/scholar?q=Azzopardi%2C+S%2E%2C+Gatt%2C+A%2E%2C+Pace%2C+G%2EJ%2E%3A+Reasoning+about+partial+contracts%2E+In%3A+JURIX+2016%2E+pp%2E+23%E2%80%9332+%282016%29)
6. Behrmann, G., David, A., Larsen, K.G.: A Tutorial on UPPAAL 4.0. Tech. rep., Department of Computer Science, Aalborg University (2006) [find](https://scholar.google.com/scholar?q=Behrmann%2C+G%2E%2C+David%2C+A%2E%2C+Larsen%2C+K%2EG%2E%3A+A+Tutorial+on+UPPAAL+4%2E0%2E+Tech%2E+rep%2E%2C+Department+of+Computer+Science%2C+Aalborg+University+%282006%29)
7. Boer, A., Winkels, R., Vitali, F.: MetaLex XML and the Legal Knowledge Interchange Format. In: Computable Models of the Law, pp. 21–41. Springer (2008) [find](https://scholar.google.com/scholar?q=Boer%2C+A%2E%2C+Winkels%2C+R%2E%2C+Vitali%2C+F%2E%3A+MetaLex+XML+and+the+Legal+Knowledge+Interchange+Format%2E+In%3A+Computable+Models+of+the+Law%2C+pp%2E+21%E2%80%9341%2E+Springer+%282008%29)
8. Camilleri, J.J., Grūzı̄tis, N., Schneider, G.: Extracting Formal Models from Normative Texts. In: NLDB 2016, LNCS, vol. 9612. Springer (2016) [find](https://scholar.google.com/scholar?q=Camilleri%2C+J%2EJ%2E%2C+Gru%CC%84z%C4%B1%CC%84tis%2C+N%2E%2C+Schneider%2C+G%2E%3A+Extracting+Formal+Models+from+Normative+Texts%2E+In%3A+NLDB+2016%2C+LNCS%2C+vol%2E+9612%2E+Springer+%282016%29)
9. Camilleri, J.J., Paganelli, G., Schneider, G.: A CNL for Contract-Oriented Diagrams. In: CNL 2014, LNCS, vol. 8625, pp. 135–146. Springer (2014) [find](https://scholar.google.com/scholar?q=Camilleri%2C+J%2EJ%2E%2C+Paganelli%2C+G%2E%2C+Schneider%2C+G%2E%3A+A+CNL+for+Contract%2DOriented+Diagrams%2E+In%3A+CNL+2014%2C+LNCS%2C+vol%2E+8625%2C+pp%2E+135%E2%80%93146%2E+Springer+%282014%29)
10. Camilleri, J.J., Schneider, G.: Modelling and Analysis of Normative Documents. JLAMP 87 (2017), (To appear). Preprint available at [http://www.cse.chalmers.se/~cajohn/papers/jlamp-nwpt2015.pdf](http://www.cse.chalmers.se/~cajohn/papers/jlamp-nwpt2015.pdf) [↗](http://www.cse.chalmers.se/~cajohn/papers/jlamp-nwpt2015.pdf)
11. Cheng, T.T., Cua, J., Tan, M., Yao, K., Roxas, R.: Information extraction from legal documents. In: SNLP 2009. pp. 157–162 (2009) [find](https://scholar.google.com/scholar?q=Cheng%2C+T%2ET%2E%2C+Cua%2C+J%2E%2C+Tan%2C+M%2E%2C+Yao%2C+K%2E%2C+Roxas%2C+R%2E%3A+Information+extraction+from+legal+documents%2E+In%3A+SNLP+2009%2E+pp%2E+157%E2%80%93162+%282009%29)
12. Dı́az, G., Cambronero, M.E., Martı́nez, E., Schneider, G.: Specification and Verification of Normative Texts using C-O Diagrams. IEEE Transactions on Software Engineering 40(8), 795–817 (2014) [find](https://scholar.google.com/scholar?q=D%C4%B1%CC%81az%2C+G%2E%2C+Cambronero%2C+M%2EE%2E%2C+Mart%C4%B1%CC%81nez%2C+E%2E%2C+Schneider%2C+G%2E%3A+Specification+and+Verification+of+Normative+Texts+using+C%2DO+Diagrams%2E+IEEE+Transactions+on+Software+Engineering+40%288%29%2C+795%E2%80%93817+%282014%29)
13. Fenech, S., Pace, G.J., Schneider, G.: CLAN: A Tool for Contract Analysis and Conflict Discovery. In: ATVA 2009. LNCS, vol. 5799, pp. 90–96. Springer (2009) [find](https://scholar.google.com/scholar?q=Fenech%2C+S%2E%2C+Pace%2C+G%2EJ%2E%2C+Schneider%2C+G%2E%3A+CLAN%3A+A+Tool+for+Contract+Analysis+and+Conflict+Discovery%2E+In%3A+ATVA+2009%2E+LNCS%2C+vol%2E+5799%2C+pp%2E+90%E2%80%9396%2E+Springer+%282009%29)
14. Klein, D., Manning, C.D.: [Accurate Unlexicalized Parsing](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/). In: ACL 2003. pp. 423– 430 (2003)
15. Larsen, K.G., Pettersson, P., Yi, W.: UPPAAL in a Nutshell. STTT 1(1-2), 134– 152 (1997) [find](https://scholar.google.com/scholar?q=Larsen%2C+K%2EG%2E%2C+Pettersson%2C+P%2E%2C+Yi%2C+W%2E%3A+UPPAAL+in+a+Nutshell%2E+STTT+1%281%2D2%29%2C+134%E2%80%93+152+%281997%29)
16. Martı́nez, E., Cambronero, E., Dı́az, G., Schneider, G.: A Model for Visual Specification of e-Contracts. In: SCC 2010. pp. 1–8. IEEE Computer Society (2010) [find](https://scholar.google.com/scholar?q=Mart%C4%B1%CC%81nez%2C+E%2E%2C+Cambronero%2C+E%2E%2C+D%C4%B1%CC%81az%2C+G%2E%2C+Schneider%2C+G%2E%3A+A+Model+for+Visual+Specification+of+e%2DContracts%2E+In%3A+SCC+2010%2E+pp%2E+1%E2%80%938%2E+IEEE+Computer+Society+%282010%29)
17. Mercatali, P., Romano, F., Boschi, L., Spinicci, E.: Automatic Translation from Textual Representations of Laws to Formal Models through UML. In: JURIX 2005. pp. 71–80. IOS Press (2005) [find](https://scholar.google.com/scholar?q=Mercatali%2C+P%2E%2C+Romano%2C+F%2E%2C+Boschi%2C+L%2E%2C+Spinicci%2C+E%2E%3A+Automatic+Translation+from+Textual+Representations+of+Laws+to+Formal+Models+through+UML%2E+In%3A+JURIX+2005%2E+pp%2E+71%E2%80%9380%2E+IOS+Press+%282005%29)
18. Object Management Group (OMG): Semantics of business vocabulary and business rules (SBVR). Tech. Rep. formal/2015-05-07 (2015), [http://www.omg.org/spec/SBVR/1.3/PDF](http://www.omg.org/spec/SBVR/1.3/PDF) [↗](http://www.omg.org/spec/SBVR/1.3/PDF)
19. Prisacariu, C., Schneider, G.: A formal language for electronic contracts. In: FMOODS 2007. LNCS, vol. 4468, pp. 174–189. Springer (2007) [find](https://scholar.google.com/scholar?q=Prisacariu%2C+C%2E%2C+Schneider%2C+G%2E%3A+A+formal+language+for+electronic+contracts%2E+In%3A+FMOODS+2007%2E+LNCS%2C+vol%2E+4468%2C+pp%2E+174%E2%80%93189%2E+Springer+%282007%29)
20. Prisacariu, C., Schneider, G.: A dynamic deontic logic for complex contracts. JLAP 81(4), 458–490 (2012) [find](https://scholar.google.com/scholar?q=Prisacariu%2C+C%2E%2C+Schneider%2C+G%2E%3A+A+dynamic+deontic+logic+for+complex+contracts%2E+JLAP+81%284%29%2C+458%E2%80%93490+%282012%29)
21. Ranta, A.: Grammatical Framework: Programming with Multilingual Grammars. CSLI Publications, Stanford (2011) [find](https://scholar.google.com/scholar?q=Ranta%2C+A%2E%3A+Grammatical+Framework%3A+Programming+with+Multilingual+Grammars%2E+CSLI+Publications%2C+Stanford+%282011%29)
22. Wyner, A., Peters, W.: On rule extraction from regulations. In: Atkinson, K. (ed.) JURIX 2011, pp. 113–122. IOS Press, Amsterdam (2011) [find](https://scholar.google.com/scholar?q=Wyner%2C+A%2E%2C+Peters%2C+W%2E%3A+On+rule+extraction+from+regulations%2E+In%3A+Atkinson%2C+K%2E+%28ed%2E%29+JURIX+2011%2C+pp%2E+113%E2%80%93122%2E+IOS+Press%2C+Amsterdam+%282011%29)
23. Wyner, A.Z., Angelov, K., Barzdins, G., Damljanovic, D., Davis, B., Fuchs, N.E., Höfler, S., Jones, K., Kaljurand, K., Kuhn, T.: On controlled natural languages: Properties and prospects. In: CNL 2009. pp. 281–289 (2009) [find](https://scholar.google.com/scholar?q=Wyner%2C+A%2EZ%2E%2C+Angelov%2C+K%2E%2C+Barzdins%2C+G%2E%2C+Damljanovic%2C+D%2E%2C+Davis%2C+B%2E%2C+Fuchs%2C+N%2EE%2E%2C+Ho%CC%88fler%2C+S%2E%2C+Jones%2C+K%2E%2C+Kaljurand%2C+K%2E%2C+Kuhn%2C+T%2E%3A+On+controlled+natural+languages%3A+Properties+and+prospects%2E+In%3A+CNL+200)

*Updated 24 October 2026*