---
title: "Transforming Large Collections of Scientific Publications to XML"
description: "The paper's main contribution is a distributed build system. 15 references, cited by 7 papers here."
authors: ["Heinrich Stamerjohanns", "Michael Kohlhase", "Deyan Ginev", "Catalin David", "Bruce R. Miller"]
year: 2010
venue: "Mathematics in Computer Science 3(3)"
source: "https://link.springer.com/article/10.1007/s11786-010-0024-7"
licence: "author's copy — read only"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/stamerjohanns-2010-transforming-large-collections-scientific-publications/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/stamerjohanns-2010-transforming-large-collections-scientific-publications/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/page/2/) / Transforming Large Collections of Scientific Publications to XML

# Transforming Large Collections of Scientific Publications to XML

**Authors:** Heinrich Stamerjohanns, [Michael Kohlhase](https://md.research.elamur.ai/free/en/authors/michael-kohlhase/), Deyan Ginev, Catalin David and Bruce R. Miller · **Year:** 2010 · **Venue:** Mathematics in Computer Science 3(3) · **Licence:** author's copy — read only

**Links:** [Publisher's page](https://doi.org/10.1007/s11786-010-0024-7) · [PDF](https://kwarc.info/people/mkohlhase/papers/mcs09.pdf) · [OpenAlex](https://openalex.org/W2090095217)

**Citations:** 47 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/stamerjohanns-2010-transforming-large-collections-scientific-publications/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most mathematical and scientific papers are written in LaTeX. LaTeX mixes content with layout and lets authors define their own macros, so machines struggle to recover the meaning of formulas from it. The arXMLiv project tackles this at scale by converting the arχiv e-print collection into XHTML+MathML with LaTeXML. LaTeXML re-implements much of the TeX engine and keeps document structure and math semantics as intact as it can. The paper's main contribution is a distributed build system. It schedules conversion jobs across several hosts and parses LaTeXML's logs. It stores per-document outcomes in an SQL database, including errors, missing packages and unsupported macros. These statistics show developers which package bindings and tool fixes matter most, so the system works both as a production pipeline and as a test harness. After the whole pre-2007 arχiv was processed, more than 90% of documents could be converted. Around 61% converted cleanly. The result is a large real-world MathML corpus for semantic analysis, math search and accessibility.

## Key points
- The corpus covers the complete arχiv through 2006: more than 400,000 documents, about 150 GB, shared over NFS. One full run takes roughly a processor-year of computation.
- Outcomes:
  - About 61% of documents convert with no or only minor warnings.
  - Another ~30% produce XHTML+MathML of uncertain quality.
  - Together these add over 240,000 usable documents.
- The build system consists of a PHP build manager, a queue manager, an SQL work queue and statistics database, and Makefiles that call latexml and latexmlpost. Each conversion has a 180-second timeout, and results can be browsed at arxmliv.kwarc.info.
- The team wrote more than two thousand LaTeXML binding files for arχiv packages. Abbreviative macros use DefMacro, and semantic macros use DefConstructor. Some packages, such as pstricks, are only partly supported.
- Planned next steps:
  - Better OpenMath/Content-MathML generation and notation disambiguation.
  - Conversion of further corpora; early tests on Zentralblatt Math show nearly perfect conversion rates.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Constructor University](https://md.research.elamur.ai/free/en/universities/constructor-university/) — University, Bremen, Germany · 8 works · 30,344 citations
- [National Institute of Standards and Technology](https://md.research.elamur.ai/free/en/labs/national-institute-of-standards-and-technology/) — Research institute, Gaithersburg, United States · 5 works · 492 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,888 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 496 works · [📚 Literature and Literary Theory](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/literature-and-literary-theory/) · 542 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,948 works

## Cite

Cite as: Heinrich Stamerjohanns, Michael Kohlhase, Deyan Ginev, Catalin David and Bruce R. Miller. 2010. Transforming Large Collections of Scientific Publications to XML. Mathematics in Computer Science 3(3).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{stamerjohanns2010transforming,
  title = {{Transforming Large Collections of Scientific Publications to XML}},
  author = {Heinrich Stamerjohanns, Michael Kohlhase, Deyan Ginev, Catalin David and Bruce R. Miller},
  booktitle = {Mathematics in Computer Science 3(3)},
  year = {2010},
  doi = {10.1007/s11786-010-0024-7},
  url = {https://doi.org/10.1007/s11786-010-0024-7}
}
```

- [← previous Exploiting Scope for Shallow Discourse Parsing](https://md.research.elamur.ai/free/en/papers/prasad-2010-exploiting-scope-shallow-discourse-parsing/)
- [next → A Semi-Supervised Approach to Improve Classification of Infrequent Discourse Relations Using Feature Vector Extension](https://md.research.elamur.ai/free/en/papers/hernault-2010-semi-supervised-approach-improve-classification/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/page/2/)

## References

15 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ron Ausbrooks, Bert Bos, Olga Caprotti, David Carlisle, Giorgi Chavchanidze, Ananth Coorg, Stéphane Dalmas, Stan Devitt, Sam Dooley, Margaret Hinchcliffe, Patrick Ion, Michael Kohlhase, Azzeddine Lazrek, Dennis Leas, Paul Libbrecht, Manolis Mavrikis, Bruce Miller, Robert Miner, Murray Sargent, Kyle… [find](https://scholar.google.com/scholar?q=Ron+Ausbrooks%2C+Bert+Bos%2C+Olga+Caprotti%2C+David+Carlisle%2C+Giorgi+Chavchanidze%2C+Ananth+Coorg%2C+Ste%CC%81phane+Dalmas%2C+Stan+Devitt%2C+Sam+Dooley%2C+Margaret+Hinchcliffe%2C+Patrick+Ion%2C+Michael+Kohlhase%2C+Azzeddine+Laz)
2. Romeo Anghelache. Hermes - a semantic xml+mathml+unicode e-publishing/self-archiving tool for latex authored scientific articles. web page at [http://hermes.roua.org/](http://hermes.roua.org/), 2007. [↗](http://hermes.roua.org/)
3. arXiv.org e-Print archive, seen December2007. web page at [http://www.arxiv.org](http://www.arxiv.org). [↗](http://www.arxiv.org)
4. Stephen Buswell, Olga Caprotti, David P. Carlisle, Michael C. Dewar, Marc Gaetano, and Michael Kohlhase. The Open Math standard, version 2.0. Technical report, The Open Math Society, 2004. [find](https://scholar.google.com/scholar?q=Stephen+Buswell%2C+Olga+Caprotti%2C+David+P%2E+Carlisle%2C+Michael+C%2E+Dewar%2C+Marc+Gaetano%2C+and+Michael+Kohlhase%2E+The+Open+Math+standard%2C+version+2%2E0%2E+Technical+report%2C+The+Open+Math+Society%2C+2004%2E)
5. Deyan Ginev, Constantin Jucovschi, Stefan Anca, Mihai Grigore, Catalin David, and Michael Kohlhase. An architecture for linguistic and semantic analysis on the arXMLiv corpus. In Applications of Semantic Technologies (AST) Workshop at Informatik 2009, 2009. in press. [find](https://scholar.google.com/scholar?q=Deyan+Ginev%2C+Constantin+Jucovschi%2C+Stefan+Anca%2C+Mihai+Grigore%2C+Catalin+David%2C+and+Michael+Kohlhase%2E+An+architecture+for+linguistic+and+semantic+analysis+on+the+arXMLiv+corpus%2E+In+Applications+of+Seman)
6. Michael Kohlhase. OMDoc – An open markup format for mathematical documents [Version 1.2]. Number 4180 in LNAI. Springer Verlag, 2006. [find](https://scholar.google.com/scholar?q=Michael+Kohlhase%2E+OMDoc+%E2%80%93+An+open+markup+format+for+mathematical+documents+%5BVersion+1%2E2%5D%2E+Number+4180+in+LNAI%2E+Springer+Verlag%2C+2006%2E)
7. Michael Kohlhase. Using L A TEX as a semantic markup format. Mathematics in Computer Science, 2008. [find](https://scholar.google.com/scholar?q=Michael+Kohlhase%2E+Using+L+A+TEX+as+a+semantic+markup+format%2E+Mathematics+in+Computer+Science%2C+2008%2E)
8. Michael Kohlhase and Ioan Şucan. A search engine for mathematical formulae. In Tetsuo Ida, Jacques Calmet, and Dongming Wang, editors, Proceedings of Artificial Intelligence and Symbolic Computation, AISC’2006, number 4120 in LNAI, pages 241–253. Springer Verlag, 2006. [find](https://scholar.google.com/scholar?q=Michael+Kohlhase+and+Ioan+S%CC%A7ucan%2E+A+search+engine+for+mathematical+formulae%2E+In+Tetsuo+Ida%2C+Jacques+Calmet%2C+and+Dongming+Wang%2C+editors%2C+Proceedings+of+Artificial+Intelligence+and+Symbolic+Computation%2C)
9. Mathplayer: Speech instructions and examples. web page at [http://www.dessci.com/en/products/mathplayer/tech/accessibility.htm](http://www.dessci.com/en/products/mathplayer/tech/accessibility.htm). [↗](http://www.dessci.com/en/products/mathplayer/tech/accessibility.htm)
10. Math Web Search. [http://kwarc.info/projects/mws/](http://kwarc.info/projects/mws/), seen Dec. 2008. [↗](http://kwarc.info/projects/mws/)
11. Bruce Miller. LaTeXML: A L A TEX to xml converter. Web Manual at [http://dlmf.nist.gov/LaTeXML/](http://dlmf.nist.gov/LaTeXML/), seen September2007. + [↗](http://dlmf.nist.gov/LaTeXML/)
12. Heinrich Stamerjohanns, Deyan Ginev, Catalin David, Dimitar Misev, Vladimir Zamdzhiev, and Michael Kohlhase. Mathml-aware article conversion from L A TEX, a comparison study. In Petr Sojka, editor, Towards Digital Mathematics Library, DML 2009 workshop. Masaryk University, Brno, 2009. [find](https://scholar.google.com/scholar?q=Heinrich+Stamerjohanns%2C+Deyan+Ginev%2C+Catalin+David%2C+Dimitar+Misev%2C+Vladimir+Zamdzhiev%2C+and+Michael+Kohlhase%2E+Mathml%2Daware+article+conversion+from+L+A+TEX%2C+a+comparison+study%2E+In+Petr+Sojka%2C+editor%2C+To)
13. TeX4ht: LaTeX and TeX for hypertext. web page at [http://www.tug.org/applications/tex4ht/mn.html](http://www.tug.org/applications/tex4ht/mn.html). [↗](http://www.tug.org/applications/tex4ht/mn.html)
14. Mark van den Brand and Jürgen Stuber. Extracting mathematical semantics from latex documents. In Proc. Intl. Workshop on Principles and Practice of Semantic Web Reasoning (PPSWR 2003), number 2901 in LNCS, pages 160–173, Mumbai, India, 2003. Springer. [find](https://scholar.google.com/scholar?q=Mark+van+den+Brand+and+Ju%CC%88rgen+Stuber%2E+Extracting+mathematical+semantics+from+latex+documents%2E+In+Proc%2E+Intl%2E+Workshop+on+Principles+and+Practice+of+Semantic+Web+Reasoning+%28PPSWR+2003%29%2C+number+2901+in)
15. Zentralblatt MATH, seen December2007. web page at [http://www.zentralblatt-math.org](http://www.zentralblatt-math.org). H. Stamerjohanns Computer Science, Jacobs University Bremen e-mail: h.stamerjohanns@jacobs-university.de M. Kohlhase Computer Science, Jacobs University Bremen e-mail: m.kohlhase@jacobs-university.de D. Ginev Comput… [↗](http://www.zentralblatt-math.org)

*Updated 3 November 2026*