---
title: "Flipper — paper by Nakhlé et al. (2026) — summary and ideas"
description: "The authors build Flipper, a parallel corpus drawn from publicly available PDF documents of asset managers (annual reports… 17 references."
authors: ["Mariam Nakhlé", "Rachel Atherly", "Gabriela nicole Gonzalez Saez", "Marco Dinarelli", "Raheel Qader", "Hervé Blanchon"]
year: 2026
venue: "The 7th Financial Narrative Processing Workshop"
source: "https://aclanthology.org/2026.fnp-1.7/"
doi: "10.63317/22f2djwbns96"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nakhle-2026-flipper-extended-document-level-financial/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nakhle-2026-flipper-extended-document-level-financial/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/) / Flipper: An Extended Document-Level Financial Dataset for Training and Evaluation with Annotated Discourse Phenomena

# Flipper: An Extended Document-Level Financial Dataset for Training and Evaluation with Annotated Discourse Phenomena

**Authors:** Mariam Nakhlé, Rachel Atherly, Gabriela nicole Gonzalez Saez, [Marco Dinarelli](https://md.research.elamur.ai/free/en/authors/marco-dinarelli/), Raheel Qader and Hervé Blanchon · **Year:** 2026 · **Venue:** [The 7th Financial Narrative Processing Workshop](https://md.research.elamur.ai/free/en/venues/fnp/) ([Financial Narrative Processing Workshop 2026](https://md.research.elamur.ai/free/en/venues/fnp/2026/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.fnp-1.7/) · [PDF](https://aclanthology.org/2026.fnp-1.7.pdf) · [DOI 10.63317/22f2djwbns96](https://doi.org/10.63317/22f2djwbns96) · [OpenAlex](https://openalex.org/W7166570442)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Machine translation still struggles with phenomena that depend on context outside the current sentence, such as pronouns, agreement and consistent terminology. These errors are costly in finance, where translated documentation is legally required. Document-level financial corpora are rare, and evaluation mostly relies on sentence-level metrics or artificial test suites. The authors build Flipper, a parallel corpus drawn from publicly available PDF documents of asset managers (annual reports, Key Information Documents, factsheets, ESG reports) in five language pairs. They extend the pipeline behind the earlier DOLFIN dataset (Nakhlé et al., 2025) in three ways: whole sections are aligned with a windowed LASER scoring scheme, near-duplicates are removed with MinHash after masking numbers, and existing tools plus custom scripts add inline XML tags for five context-sensitive phenomena. A phenomenon-dense test split is held out and the rest is used for training. Fine-tuning gemma-3-1b-it on Flipper improves COMET for three of five pairs and four of five phenomena. The resource supports both training and realistic, targeted document-level evaluation in a high-stakes domain.

## Key points
- **Coverage and size:** Flipper covers En–De, En–Es, En–Fr, En–It and Fr–Es. It contains about 122.9k parallel documents/sections, 3.6M sentences and roughly 118M tokens (source and target combined). It is released on Hugging Face as DragonLLM/Flipper.
- **Alignment and deduplication:**
  - Each source sentence is scored by LASER against a target window, and the section score is the average of these scores.
  - The acceptance threshold of 0.80 was chosen after manually checking 1,200 sections.
  - Deduplication uses MinHash (text-dedup library) at threshold 0.5, with numbers masked so that templated fund documents collapse into one.
- **Annotated phenomena:** five phenomena are tagged (the reordering of sentences, the form of verbs, consistent use of terms, gender and formality), giving 133,885 tags/documents overall.
  - Verb form is by far the most frequent (91,606, via MuDA).
  - Fr–Es received only reordering annotations, because the existing tools are English-centric.
- **Evaluation split:** 2,000 sections per pair (10k in total), selected for the quantity and variety of annotation tags. This places it between a standard test set and a test suite.
- **Fine-tuning experiment:** gemma-3-1b-it was fully fine-tuned with TRL for 1 epoch and scored with wmt22-comet-da.
  - Gains: En–Es 77.90→80.25, En–Fr 75.06→75.91, Fr–Es 78.19→78.87.
  - Drops: En–De 69.77→67.81, En–It 74.30→73.73.
  - By phenomenon, Formality (59.22→61.16) and Verb form (74.07→75.86) improve most, while Gender declines (77.33→75.62).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Centre National de la Recherche Scientifique](https://md.research.elamur.ai/free/en/labs/centre-national-de-la-recherche-scientifique/) — Research institute, Paris, France · 168 works · 62,369 citations
- [Université Grenoble Alpes](https://md.research.elamur.ai/free/en/universities/universite-grenoble-alpes/) — University, Saint-Martin-d'Hères, France · 20 works · 8,059 citations
- [Laboratoire d'Informatique de Grenoble](https://md.research.elamur.ai/free/en/labs/laboratoire-d-informatique-de-grenoble/) — Research institute, Grenoble, France · 21 works · 688 citations
- [Institut polytechnique de Grenoble](https://md.research.elamur.ai/free/en/universities/institut-polytechnique-de-grenoble/) — University, Grenoble, France · 11 works · 207 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧵 Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/) · 466 works

## Cite

Cite as: Mariam Nakhlé, Rachel Atherly, Gabriela nicole Gonzalez Saez, Marco Dinarelli, Raheel Qader and Hervé Blanchon. 2026. Flipper: An Extended Document-Level Financial Dataset for Training and Evaluation with Annotated Discourse Phenomena. The 7th Financial Narrative Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nakhle2026flipper,
  title = {{Flipper: An Extended Document-Level Financial Dataset for Training and Evaluation with Annotated Discourse Phenomena}},
  author = {Mariam Nakhlé, Rachel Atherly, Gabriela nicole Gonzalez Saez, Marco Dinarelli, Raheel Qader and Hervé Blanchon},
  booktitle = {The 7th Financial Narrative Processing Workshop},
  year = {2026},
  doi = {10.63317/22f2djwbns96},
  url = {https://doi.org/10.63317/22f2djwbns96}
}
```

- [← previous FLAICOL: Flip-Point-Led Augmentation for Imbalanced Code-Mixed Offensive Language Detection](https://md.research.elamur.ai/free/en/papers/mohammed-2026-flaicol-flip-point-led-augmentation/)
- [next → Floating or Suggesting Ideas? A Large-Scale Contrastive Analysis of Metaphorical and Literal Verb–Object Constructions](https://md.research.elamur.ai/free/en/papers/piccirilli-2026-floating-or-suggesting-ideas-large/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/)

## References

17 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Duarte M. Alves, José Pombal, Nuno M. Guerreiro, Pedro H. Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and André F. T. Martins. 2024. [Tower: An open multilingual large language model for translation-related tasks](https://md.research.elamur.ai/free/en/papers/alves-2024-tower-open-multilingual-large-language/).
2. Dogu Araci. 2019. Finbert: Financial sentiment analysis with pre-trained language models. [find](https://scholar.google.com/scholar?q=Dogu+Araci%2E+2019%2E+Finbert%3A+Financial+sentiment+analysis+with+pre%2Dtrained+language+models%2E)
3. Rachel Bawden, Rico Sennrich, Alexandra Birch, and Barry Haddow. 2018. [Evaluating discourse phenomena in neural machine translation](https://md.research.elamur.ai/free/en/papers/bawden-2017-evaluating-discourse-phenomena-neural-machine/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Paper…
4. Gaëtan Caillaut, Raheel Qader, Jingshu Liu, Mariam Nakhlé, Arezki Sadoune, Massinissa Ah-mim, and Jean-Gabriel Barthelemy. 2025. The llm pro finance suite: Multilingual large language models for financial applications. 2020. A set of recommendations for assessing human–machine parity in language tra… [find](https://scholar.google.com/scholar?q=Ga%C3%ABtan+Caillaut%2C+Raheel+Qader%2C+Jingshu+Liu%2C+Mariam+Nakhl%C3%A9%2C+Arezki+Sadoune%2C+Massinissa+Ah%2Dmim%2C+and+Jean%2DGabriel+Barthelemy%2E+2025%2E+The+llm+pro+finance+suite%3A+Multilingual+large+language+models+for+finan)
5. Mauro Cettolo, Christian Girardi, and Marcello Federico. 2012. [WIT3: Web inventory of transcribed and translated talks](https://md.research.elamur.ai/free/en/papers/cettolo-2012-wit3-web-inventory-transcribed-translated/). In Proceedings of the 16th Annual Conference of the European Association for Machine Translation, pages 261–268, Trento, Italy. European Association for Machine Translation.
6. Pierre Lison and Jörg Tiedemann. 2016. [OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles](https://md.research.elamur.ai/free/en/papers/lison-2016-opensubtitles2016-extracting-large-parallel-corpora/). In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 923– 929, Portorož, Slovenia. European Language Resources Association (E…
7. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2021. [FinQA: A dataset of numerical reasoning over financial data](https://md.research.elamur.ai/free/en/papers/chen-2021-finqa-dataset-numerical-reasoning-over/). In Proceedings of the 2021 Conference on Empirical Methods in Natural…
8. Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. 2018. Www’18 open challenge: Financial opinion mining and question answering. In Companion Proceedings of the The Web Conference 2018, WWW ’18, page 1941–1942, Republic and Canton of G… [find](https://scholar.google.com/scholar?q=Macedo+Maia%2C+Siegfried+Handschuh%2C+Andr%C3%A9+Freitas%2C+Brian+Davis%2C+Ross+McDermott%2C+Manel+Zarrouk%2C+and+Alexandra+Balahur%2E+2018%2E+Www%E2%80%9918+open+challenge%3A+Financial+opinion+mining+and+question+answering%2E+In+Com)
9. Patrick Fernandes, Kayo Yin, Emmy Liu, André Martins, and Graham Neubig. 2023. When does translation require context? a data-driven, multilingual exploration. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 606–626, Toronto, C… [find](https://scholar.google.com/scholar?q=Patrick+Fernandes%2C+Kayo+Yin%2C+Emmy+Liu%2C+Andr%C3%A9+Martins%2C+and+Graham+Neubig%2E+2023%2E+When+does+translation+require+context%3F+a+data%2Ddriven%2C+multilingual+exploration%2E+In+Proceedings+of+the+61st+Annual+Meeting)
10. Pekka Malo, Ankur Sinha, Pyry Takala, Pekka Korhonen, and Jyrki Wallenius. 2014. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the American Society for Information Science and Technology. [find](https://scholar.google.com/scholar?q=Pekka+Malo%2C+Ankur+Sinha%2C+Pyry+Takala%2C+Pekka+Korhonen%2C+and+Jyrki+Wallenius%2E+2014%2E+Good+debt+or+bad+debt%3A+Detecting+semantic+orientations+in+economic+texts%2E+Journal+of+the+American+Society+for+Informati)
11. Chenghao Mou, Chris Ha, Kenneth Enevoldsen, and Peiyuan Liu. 2023. Chenghaomou/text-dedup: Reference snapshot. [find](https://scholar.google.com/scholar?q=Chenghao+Mou%2C+Chris+Ha%2C+Kenneth+Enevoldsen%2C+and+Peiyuan+Liu%2E+2023%2E+Chenghaomou%2Ftext%2Ddedup%3A+Reference+snapshot%2E)
12. Simon Jehnen, Joaquín Ordieres-Meré, and Javier Villalba-Díez. 2025. Fintextsim: Enhancing financial text analysis with bertopic. [find](https://scholar.google.com/scholar?q=Simon+Jehnen%2C+Joaqu%C3%ADn+Ordieres%2DMer%C3%A9%2C+and+Javier+Villalba%2DD%C3%ADez%2E+2025%2E+Fintextsim%3A+Enhancing+financial+text+analysis+with+bertopic%2E)
13. Mathias Müller, Annette Rios, Elena Voita, and Rico Sennrich. 2018. [A large-scale test set for the evaluation of context-aware pronoun translation in neural machine translation](https://md.research.elamur.ai/free/en/papers/muller-2018-large-scale-test-set-evaluation/). arXiv preprint arXiv:1810.02268.
14. Tom Kocmi, Ekaterina Artemova, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Konstantin Dranch, Anton Dvorkovich, Sergey Dukanov, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Howard Lakougna, Jessica Lundin, Christof Monz, Kento… — [Findings of the WMT25 General Machine Translation Shared Task: Time to Stop Evaluating on Easy Test Sets](https://md.research.elamur.ai/free/en/papers/kocmi-2025-findings-wmt25-general-machine-translation/)
15. Mariam Nakhlé, Marco Dinarelli, Raheel Qader, Emmanuelle Esperança-Rodier, and Hervé Blanchon. 2025. DOLFIN - document-level financial test-set for machine translation. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 5544–5556, Albuquerque, New Mexico. Association for… [find](https://scholar.google.com/scholar?q=Mariam+Nakhl%C3%A9%2C+Marco+Dinarelli%2C+Raheel+Qader%2C+Emmanuelle+Esperan%C3%A7a%2DRodier%2C+and+Herv%C3%A9+Blanchon%2E+2025%2E+DOLFIN+%2D+document%2Dlevel+financial+test%2Dset+for+machine+translation%2E+In+Findings+of+the+Association+)
16. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
17. Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. 2022. [COMET-22: Unbabel-IST 2022 submission for the metrics shared task](https://md.research.elamur.ai/free/en/papers/rei-2022-comet-22-unbabel-ist-2022/). In Proceedings of the Seventh Conference on Machine Translation (WMT), pages…

*Updated 1 November 2026*