---
title: "LabelFusion — paper by Schlee et al. (2026)"
description: "The authors build a ten-topic multi-label benchmark from Reuters-21578, using the ModApte split and 9,034 documents. 22 references."
authors: ["Michael Schlee", "Christoph Weisser", "Timo Kivimäki", "Melchizedek Mashiku", "Benjamin Saefken"]
year: 2026
venue: "The 7th Financial Narrative Processing Workshop"
source: "https://aclanthology.org/2026.fnp-1.9/"
doi: "10.63317/5k4dk6iczjq5"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/schlee-2026-labelfusion-fusing-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/schlee-2026-labelfusion-fusing-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) / LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification

# LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification

**Authors:** Michael Schlee, Christoph Weisser, Timo Kivimäki, Melchizedek Mashiku and Benjamin Saefken · **Year:** 2026 · **Venue:** [The 7th Financial Narrative Processing Workshop](https://md.research.elamur.ai/free/en/venues/fnp/) ([Financial Narrative Processing Workshop 2026](https://md.research.elamur.ai/free/en/venues/fnp/2026/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.fnp-1.9/) · [PDF](https://aclanthology.org/2026.fnp-1.9.pdf) · [DOI 10.63317/5k4dk6iczjq5](https://doi.org/10.63317/5k4dk6iczjq5) · [OpenAlex](https://openalex.org/W4417296849)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Commodity-focused financial NLP pipelines, such as gold or oil price forecasting from news sentiment, first need to filter out articles unrelated to the target asset. Supervised encoders like RoBERTa need many annotated examples for this filtering step, and they perform poorly when labels are scarce. The authors build a ten-topic multi-label benchmark from Reuters-21578, using the ModApte split and 9,034 documents. They compare a prompted GPT-5-nano, a fine-tuned RoBERTa, a TFIDF+LR baseline and their own LabelFusion model. LabelFusion appends the LLM's binary label vector to RoBERTa's [CLS] embedding and passes the result to a small trainable MLP that makes the final decision. Each model is trained on 5% to 100% of the training set. With all data, LabelFusion performs best. With limited data, however, prompting the LLM alone wins by a large margin, and it keeps that lead until roughly 80% of the labels are available. The practical guidance is to rely on LLM prompting when annotations are limited and to switch to fusion once enough labeled data exists. The method is released as a software package.

## Key points
- **Dataset:** Reuters-21578 is limited to the ten topics that occur most often. These cover corporate news (earn, acq), macro and financial themes (money-fx, interest, trade), commodities (crude, grain, wheat, corn) and shipping (ship).
  - Splits are 5,842 train, 647 validation (multilabel-stratified) and 2,545 test documents.
  - Classes are strongly imbalanced, from 3,964 earn documents down to 237 corn.
  - 9.2% of documents carry multiple labels.
- **Full-data results:** LabelFusion reaches macro F1 96.0% and accuracy 92.3%. Standalone RoBERTa reaches F1 94.6% and GPT-5-nano reaches F1 93.9%.
- **Low-data results:** zero-shot GPT-5-nano already achieves F1 75.9%. At 5% of the training data (292 documents), the prompted LLM scores F1 93.0, against 71.7 for Fusion and 37.2 for RoBERTa.
- **Architecture:** the fused input is [h ; z] ∈ R^(768+K), built from the roberta-base embedding and the LLM's label predictions. It feeds an MLP with hidden layers [64, 32], and labels are assigned at a fixed 0.5 threshold.
- **Prompt construction:** the prompt's role description is generated automatically through meta-prompting from labeled examples. GPT-5-nano is run at temperature 0.1.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Göttingen](https://md.research.elamur.ai/free/en/universities/university-of-gottingen/) — University, Göttingen, Germany · 17 works · 158 citations
- [University of Bath](https://md.research.elamur.ai/free/en/universities/university-of-bath/) — University, Bath, United Kingdom · 5 works · 351 citations

Also on the byline: [Hochschule Bielefeld](https://ror.org/00edvg943) · [Centers for Disease Control and Prevention](https://ror.org/042twtr12) · [Centers for Disease Control and Prevention](https://ror.org/00qzjvm58).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 209 works · [🎯 Management Science and Operations Research](https://md.research.elamur.ai/free/en/shelves/decision-sciences/management-science-and-operations-research/) · 108 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Michael Schlee, Christoph Weisser, Timo Kivimäki, Melchizedek Mashiku and Benjamin Saefken. 2026. LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification. The 7th Financial Narrative Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{schlee2026labelfusion,
  title = {{LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification}},
  author = {Michael Schlee, Christoph Weisser, Timo Kivimäki, Melchizedek Mashiku and Benjamin Saefken},
  booktitle = {The 7th Financial Narrative Processing Workshop},
  year = {2026},
  doi = {10.63317/5k4dk6iczjq5},
  url = {https://doi.org/10.63317/5k4dk6iczjq5}
}
```

- [← previous LabelBuddy: An Open Source Music and Audio Language Annotation Tagging Tool Using AI Assistance](https://md.research.elamur.ai/free/en/papers/prokopiou-2026-labelbuddy-open-source-music-audio/)
- [next → LaCo: Layer-wise Compensation for Pruned Large Language Models](https://md.research.elamur.ai/free/en/papers/liu-2026-laco-layer-wise-compensation-pruned/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/)

## References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Harika Abburi et al. 2023. Generative ai text classification using ensemble llm approaches. arXiv preprint arXiv:2309.07755. — [arXiv:2309.07755](https://arxiv.org/abs/2309.07755) [find](https://scholar.google.com/scholar?q=Harika+Abburi+et+al%2E+2023%2E+Generative+ai+text+classification+using+ensemble+llm+approaches%2E+arXiv+preprint+arXiv%3A2309%2E07755%2E+%E2%80%94+%5BarXiv%3A2309%2E07755%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2309%2E07755%29)
2. Anonymous. 2025. Labelfusion: Learning to fuse llms and transformer classifiers for robust text classification. [find](https://scholar.google.com/scholar?q=Anonymous%2E+2025%2E+Labelfusion%3A+Learning+to+fuse+llms+and+transformer+classifiers+for+robust+text+classification%2E)
3. Dogu Araci. 2019. FinBERT: Financial sentiment analysis with pre-trained language models. [find](https://scholar.google.com/scholar?q=Dogu+Araci%2E+2019%2E+FinBERT%3A+Financial+sentiment+analysis+with+pre%2Dtrained+language+models%2E)
4. Tom B. Brown et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901.
5. Seung Chan Chae and Sun-Yong Choi. 2023. Forecasting the S&P 500 index using mathematical-based sentiment analysis and deep learning models. Axioms, 12(9):835. [find](https://scholar.google.com/scholar?q=Seung+Chan+Chae+and+Sun%2DYong+Choi%2E+2023%2E+Forecasting+the+S%26P+500+index+using+mathematical%2Dbased+sentiment+analysis+and+deep+learning+models%2E+Axioms%2C+12%289%29%3A835%2E)
6. Youngjin Chae and Thomas Davidson. 2025. Large language models for text classification. Sociological Methods & Research. [find](https://scholar.google.com/scholar?q=Youngjin+Chae+and+Thomas+Davidson%2E+2025%2E+Large+language+models+for+text+classification%2E+Sociological+Methods+%26+Research%2E)
7. Yilin Chen, Xiaolong Li, and Yonghong Hu. 2024. Dual-attention transformer for financial news sentiment and stock price prediction. Expert Systems with Applications, 238:121134. [find](https://scholar.google.com/scholar?q=Yilin+Chen%2C+Xiaolong+Li%2C+and+Yonghong+Hu%2E+2024%2E+Dual%2Dattention+transformer+for+financial+news+sentiment+and+stock+price+prediction%2E+Expert+Systems+with+Applications%2C+238%3A121134%2E)
8. Franca Debole and Fabrizio Sebastiani. 2005. An analysis of the relative hardness of reuters-21578 subsets. Journal of the American Society for Information Science and Technology, 56(6):584– 596. [find](https://scholar.google.com/scholar?q=Franca+Debole+and+Fabrizio+Sebastiani%2E+2005%2E+An+analysis+of+the+relative+hardness+of+reuters%2D21578+subsets%2E+Journal+of+the+American+Society+for+Information+Science+and+Technology%2C+56%286%29%3A584%E2%80%93+596%2E)
9. Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. [Making pre-trained language models better few-shot learners](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/). In ACL, pages 3816–3830.
10. Jungwoo Gwak and Yoonsang Jung. 2025. Layer-aware embedding fusion for llms. arXiv preprint arXiv:2504.05764. — [arXiv:2504.05764](https://arxiv.org/abs/2504.05764) [find](https://scholar.google.com/scholar?q=Jungwoo+Gwak+and+Yoonsang+Jung%2E+2025%2E+Layer%2Daware+embedding+fusion+for+llms%2E+arXiv+preprint+arXiv%3A2504%2E05764%2E+%E2%80%94+%5BarXiv%3A2504%2E05764%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2504%2E05764%29)
11. Boshko Koloski, Senja Pollak, Roberto Navigli, and Blaž Škrlj. 2024. Automl-guided fusion of entity and llm-based representations. In ICONIP, pages 89–102. [find](https://scholar.google.com/scholar?q=Boshko+Koloski%2C+Senja+Pollak%2C+Roberto+Navigli%2C+and+Bla%C5%BE+%C5%A0krlj%2E+2024%2E+Automl%2Dguided+fusion+of+entity+and+llm%2Dbased+representations%2E+In+ICONIP%2C+pages+89%E2%80%93102%2E)
12. David D. Lewis. 1997. Reuters-21578 text categorization test collection. Technical report, AT&T Labs Research. [find](https://scholar.google.com/scholar?q=David+D%2E+Lewis%2E+1997%2E+Reuters%2D21578+text+categorization+test+collection%2E+Technical+report%2C+AT%26T+Labs+Research%2E)
13. Yilin Ma, Xin Gao, and Bowen Yang. 2024. LIAM: Label-interaction aware model for multi-label text classification. Neurocomputing, 580:127139. [find](https://scholar.google.com/scholar?q=Yilin+Ma%2C+Xin+Gao%2C+and+Bowen+Yang%2E+2024%2E+LIAM%3A+Label%2Dinteraction+aware+model+for+multi%2Dlabel+text+classification%2E+Neurocomputing%2C+580%3A127139%2E)
14. Morteza Nabipour, Pejman Naeem, and Hamed Jabani. 2026. Federated learning for financial sentiment-augmented price prediction. In Proceedings of the 2026 International Conference on Machine Learning and Applications (ICMLA). IEEE. [find](https://scholar.google.com/scholar?q=Morteza+Nabipour%2C+Pejman+Naeem%2C+and+Hamed+Jabani%2E+2026%2E+Federated+learning+for+financial+sentiment%2Daugmented+price+prediction%2E+In+Proceedings+of+the+2026+International+Conference+on+Machine+Learning+a)
15. Timo Schick and Hinrich Schütze. 2021. Exploiting cloze-questions for few-shot text classification. In EACL, pages 255–269. [find](https://scholar.google.com/scholar?q=Timo+Schick+and+Hinrich+Sch%C3%BCtze%2E+2021%2E+Exploiting+cloze%2Dquestions+for+few%2Dshot+text+classification%2E+In+EACL%2C+pages+255%E2%80%93269%2E)
16. Konstantinos Sechidis, Grigorios Tsoumakas, and Ioannis Vlahavas. 2011. On the stratification of multi-label data. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD), pages 145–160, Berlin, Heidelberg. Springer. [find](https://scholar.google.com/scholar?q=Konstantinos+Sechidis%2C+Grigorios+Tsoumakas%2C+and+Ioannis+Vlahavas%2E+2011%2E+On+the+stratification+of+multi%2Dlabel+data%2E+In+Proceedings+of+the+European+Conference+on+Machine+Learning+and+Knowledge+Discovery)
17. Anand Sinha and Ravi Shastri. 2020. Impact of news sentiment on commodity returns. Journal of Behavioral Finance, 21(3):310–325. [find](https://scholar.google.com/scholar?q=Anand+Sinha+and+Ravi+Shastri%2E+2020%2E+Impact+of+news+sentiment+on+commodity+returns%2E+Journal+of+Behavioral+Finance%2C+21%283%29%3A310%E2%80%93325%2E)
18. Piotr Szymański and Tomasz Kajdanowicz. 2017. A network perspective on stratification of multi-label data. In Proceedings of the First International Workshop on Learning with Imbalanced Domains: Theory and Applications (LIDTA), pages 22–35. PMLR. [find](https://scholar.google.com/scholar?q=Piotr+Szyma%C5%84ski+and+Tomasz+Kajdanowicz%2E+2017%2E+A+network+perspective+on+stratification+of+multi%2Dlabel+data%2E+In+Proceedings+of+the+First+International+Workshop+on+Learning+with+Imbalanced+Domains%3A+Theor)
19. Zhiqiang Wang, Yiran Pang, and Yanbin Lin. 2023. Large language models are zero-shot text classifiers. arXiv preprint arXiv:2312.01044. [find](https://scholar.google.com/scholar?q=Zhiqiang+Wang%2C+Yiran+Pang%2C+and+Yanbin+Lin%2E+2023%2E+Large+language+models+are+zero%2Dshot+text+classifiers%2E+arXiv+preprint+arXiv%3A2312%2E01044%2E)
20. Jianfeng Yang, Yufei Wang, and Xiang Li. 2022. Prediction of stock price direction using the LASSO-LSTM model combining technical indicators and financial sentiment analysis. PeerJ Computer Science, 8:e1148. [find](https://scholar.google.com/scholar?q=Jianfeng+Yang%2C+Yufei+Wang%2C+and+Xiang+Li%2E+2022%2E+Prediction+of+stock+price+direction+using+the+LASSO%2DLSTM+model+combining+technical+indicators+and+financial+sentiment+analysis%2E+PeerJ+Computer+Science%2C+8)
21. Hao Yuan, Weiguang Han, and Yucheng Li. 2024. LACN: Label-aware co-training network for multi-label text classification. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), pages 3412–3422. ELRA and ICCL. [find](https://scholar.google.com/scholar?q=Hao+Yuan%2C+Weiguang+Han%2C+and+Yucheng+Li%2E+2024%2E+LACN%3A+Label%2Daware+co%2Dtraining+network+for+multi%2Dlabel+text+classification%2E+In+Proceedings+of+the+2024+Joint+International+Conference+on+Computational+Ling)
22. Dingkun Zeng, Erxue Zha, Jian Kuang, and Yong Shen. 2024. Multi-label text classification based on semantic-sensitive graph convolutional network. Knowledge-Based Systems, 282:111107. [find](https://scholar.google.com/scholar?q=Dingkun+Zeng%2C+Erxue+Zha%2C+Jian+Kuang%2C+and+Yong+Shen%2E+2024%2E+Multi%2Dlabel+text+classification+based+on+semantic%2Dsensitive+graph+convolutional+network%2E+Knowledge%2DBased+Systems%2C+282%3A111107%2E)

*Updated 1 November 2026*