---
title: "Apprentissage par renforcement pour — Aissi 2025"
description: "The authors study how reinforcement learning (RL) affects LLM-based agents that are fine-tuned to make sequential decisions in a text-based interactive…"
authors: ["Mohamed Salim Aissi", "Clement Romac", "Thomas Carta", "Sylvain Lamprier", "Pierre-Yves Oudeyer", "Olivier Sigaud", "Laure Soulier", "Nicolas Thome"]
year: 2025
venue: "Actes des 32ème Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 2 : traductions d'articles publiés"
source: "https://aclanthology.org/2025.jeptalnrecital-trad.5/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/aissi-2025-apprentissage-par-renforcement-pour-l/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/aissi-2025-apprentissage-par-renforcement-pour-l/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2025](https://md.research.elamur.ai/free/papers/2025/) / Apprentissage par renforcement pour l'alignement des agents LLMs avec des environnements interactifs : quantification et réduction du surapprentissage aux prompts

# Apprentissage par renforcement pour l'alignement des agents LLMs avec des environnements interactifs : quantification et réduction du surapprentissage aux prompts

**Authors:** Mohamed Salim Aissi, Clement Romac, Thomas Carta, [Sylvain Lamprier](https://md.research.elamur.ai/free/authors/sylvain-lamprier/), Pierre-Yves Oudeyer, Olivier Sigaud, Laure Soulier, Nicolas Thome · **Year:** 2025 · **Venue:** Actes des 32ème Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 2 : traductions d'articles publiés · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.jeptalnrecital-trad.5/) · [PDF](https://aclanthology.org/2025.jeptalnrecital-trad.5.pdf)

## Summary
The authors study how reinforcement learning (RL) affects LLM-based agents that are fine-tuned to make sequential decisions in a text-based interactive environment. They want to know how much such an agent depends on the exact wording of the prompt it was trained on. To find out, they propose a new analytical approach: after RL fine-tuning, the model is tested on other, paraphrased versions of the instructions. On unfamiliar wordings, the agent's results drop noticeably. In other words, the model overfits to the prompt and does not learn the task itself. To find the cause, the authors examine the model's internal representations and the tokens that most strongly influence its decisions. To reduce this dependence, they add a contrastive loss to training, which makes the agent more robust and improves generalization. The work matters for anyone who fine-tunes language models for interactive tasks: an agent that works with only one instruction template is unreliable in real-world use.

## Key points
- Object of study: LLM agents fine-tuned with RL in a text-based environment for sequential decision-making.
- Main finding: after RL training, the agent is sensitive to the wording of the prompt, and its performance drops on variants that were not seen during training.
- To find the source of this sensitivity, the authors analyze the model's internal representations and its most salient (saliency) tokens.
- As a remedy, the authors propose a contrastive loss that increases robustness and the ability to generalize.
- The original was published in NAACL 2025 Findings (https://aclanthology.org/2025.findings-naacl.390/). A French translation appeared in TALN 2025 (vol. 2, pp. 6–7) under the CC BY 4.0 license. The authors are from Sorbonne Université/ISIR, Inria (Flowers), Hugging Face and Univ Angers.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{aissi2025apprentissage,
  title = {{Apprentissage par renforcement pour l'alignement des agents LLMs avec des environnements interactifs : quantification et réduction du surapprentissage aux prompts}},
  author = {Aissi, Mohamed Salim and Romac, Clement and Carta, Thomas and Lamprier, Sylvain and Oudeyer, Pierre-Yves and Sigaud, Olivier and Soulier, Laure and Thome, Nicolas},
  booktitle = {Actes des 32ème Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 2 : traductions d'articles publiés},
  year = {2025},
  url = {https://aclanthology.org/2025.jeptalnrecital-trad.5/}
}
```

- [← previous Annotating the French Wiktionary with supersenses for large scale lexical analysis: a use case to assess form-meaning relationships within the nominal lexicon](https://md.research.elamur.ai/free/papers/angleraud-2025-french-wiktionary-supersenses/)
- [next → Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models](https://md.research.elamur.ai/free/papers/askari-2025-are-babylms-deaf-gricean-maxims/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/papers/2025/)