---
title: "Learning and Leveraging Verifiers to Improve Planning"
description: "At inference time, the generator builds plans step by step and restarts from the initial state whenever… 13 references, cited by 7 papers here."
authors: ["Daman Arora", "Subbarao Kambhampati"]
year: 2023
venue: "arXiv:2305.17077"
source: "https://arxiv.org/abs/2305.17077"
doi: "10.48550/arxiv.2305.17077"
arxiv: "2305.17077"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/arora-2023-learning-leveraging-verifiers-improve-planning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/arora-2023-learning-leveraging-verifiers-improve-planning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/) / Learning and Leveraging Verifiers to Improve Planning Capabilities of Pre-trained Language Models

# Learning and Leveraging Verifiers to Improve Planning Capabilities of Pre-trained Language Models

**Authors:** [Daman Arora](https://md.research.elamur.ai/free/en/authors/daman-arora/) and [Subbarao Kambhampati](https://md.research.elamur.ai/free/en/authors/subbarao-kambhampati/) · **Year:** 2023 · **Venue:** arXiv:2305.17077 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2305.17077) · [PDF](https://arxiv.org/pdf/2305.17077) · [DOI 10.48550/arxiv.2305.17077](https://doi.org/10.48550/arxiv.2305.17077) · [arXiv 2305.17077](https://arxiv.org/abs/2305.17077) · [OpenAlex](https://openalex.org/W4378718493)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/arora-2023-learning-leveraging-verifiers-improve-planning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This paper examines why fine-tuned language models plan poorly. In the Blocksworld domain, the authors find that the main failure is proposing actions whose preconditions do not hold in the current state. They fine-tune GPT-2 as a generator that reads a textual goal and current state and outputs the next action and resulting state. Training data come from trajectories of a random agent that avoids revisiting states, starting from 10,000 initial configurations. A second GPT-2-based model acts as a verifier that judges whether an action is applicable in a given state. Its negative examples are produced cheaply by pairing each state with an action drawn at random from the same dataset, so no human labels or external simulator are required. At inference time, the generator builds plans step by step and restarts from the initial state whenever the verifier rejects a transition. With 25 attempts, this generate-and-test loop solves far more instances and almost eliminates invalid plans. The result suggests that learned verifiers combined with diverse sampling let LLM-based planning benefit from additional attempts.

## Key points
- On 200 Blocksworld test instances, with k = 25, top-p 0.99, temperature 1 and a 40-step limit:
  - Goal-reaching rate (GRR) rose from 37.5% (generator@25) to 65.5% (generator+verifier@25).
  - Plans with an illegal action (BTR) dropped from 52.5% to 5%.
- When the number of attempts k was varied over {1, 8, 16, 25}:
  - The generator alone plateaued around 16 attempts.
  - The verifier-augmented planner kept improving.
- Building the verifier by fine-tuning the already fine-tuned generator (V_generator) worked better than fine-tuning vanilla GPT-2 (V_base).
  - GRR was 0.655 versus 0.635.
  - V_base had a BTR of 0.105.
- The verifier passes GPT-2's last-token embedding through an MLP to classify an action as valid or invalid. It can produce both false positives and false negatives.
- A temperature sweep over {0.2, 0.5, 0.8, 1.1} produced an inverted-U curve for GRR.
  - Low temperature limits plan diversity.
  - High temperature causes more bad transitions.
  - The authors suggest tuning temperature on a validation set.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Arizona State University](https://md.research.elamur.ai/free/en/universities/arizona-state-university/) — University, Tempe, United States · 54 works · 2,356 citations
- [Indian Institute of Technology Delhi](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-delhi/) — University, New Delhi, India · 24 works · 6,933 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Daman Arora and Subbarao Kambhampati. 2023. Learning and Leveraging Verifiers to Improve Planning Capabilities of Pre-trained Language Models. arXiv:2305.17077.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{arora2023learning,
  title = {{Learning and Leveraging Verifiers to Improve Planning Capabilities of Pre-trained Language Models}},
  author = {Daman Arora and Subbarao Kambhampati},
  year = {2023},
  doi = {10.48550/arxiv.2305.17077},
  eprint = {2305.17077},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.17077}
}
```

- [← previous KSAA-RD Shared Task: Arabic Reverse Dictionary](https://md.research.elamur.ai/free/en/papers/matham-2023-ksaa-rd-shared-task-arabic/)
- [next → Legend at ArAIEval Shared Task: Persuasion Technique Detection using a Language-Agnostic Text Representation Model](https://md.research.elamur.ai/free/en/papers/ojo-2023-legend-araieval-shared-task-persuasion/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/)

## References

13 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahn, M.; Brohan, A.; Brown, N.; Chebotar, Y.; Cortes, O.; David, B.; Finn, C.; Fu, C.; Gopalakrishnan, K.; Hausman, K.; Herzog, A.; Ho, D.; Hsu, J.; Ibarz, J.; Ichter, B.; Irpan, A.; Jang, E.; Ruano, R. J.; Jeffrey, K.; Jesmonth, S.; Joshi, N. J.; Julian, R.; Kalashnikov, D.; Kuang, Y.; Lee, K.-H.;… — [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://md.research.elamur.ai/free/en/papers/ahn-2022-i-i-say-grounding-language/)
2. Chen, L.; Lu, K.; Rajeswaran, A.; Lee, K.; Grover, A.; Laskin, M.; Abbeel, P.; Srinivas, A.; and Mordatch, I. 2021. Decision Transformer: Reinforcement Learning via Sequence Modeling. arXiv:2106.01345. [find](https://scholar.google.com/scholar?q=Chen%2C+L%2E%3B+Lu%2C+K%2E%3B+Rajeswaran%2C+A%2E%3B+Lee%2C+K%2E%3B+Grover%2C+A%2E%3B+Laskin%2C+M%2E%3B+Abbeel%2C+P%2E%3B+Srinivas%2C+A%2E%3B+and+Mordatch%2C+I%2E+2021%2E+Decision+Transformer%3A+Reinforcement+Learning+via+Sequence+Modeling%2E+arXiv%3A2106%2E01345)
3. Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J. 2021. [Training Verifiers to Solve Math Word Problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv:2110.14168.
4. Ghallab, M.; Knoblock, C.; Wilkins, D.; Barrett, A.; Christianson, D.; Friedman, M.; Kwok, C.; Golden, K.; Pen-berthy, S.; Smith, D.; Sun, Y.; and Weld, D. 1998. PDDL - The Planning Domain Definition Language. [find](https://scholar.google.com/scholar?q=Ghallab%2C+M%2E%3B+Knoblock%2C+C%2E%3B+Wilkins%2C+D%2E%3B+Barrett%2C+A%2E%3B+Christianson%2C+D%2E%3B+Friedman%2C+M%2E%3B+Kwok%2C+C%2E%3B+Golden%2C+K%2E%3B+Pen%2Dberthy%2C+S%2E%3B+Smith%2C+D%2E%3B+Sun%2C+Y%2E%3B+and+Weld%2C+D%2E+1998%2E+PDDL+%2D+The+Planning+Domain+Definition+)
5. Holtzman, A.; Buys, J.; Du, L.; Forbes, M.; and Choi, Y. 2020. [The Curious Case of Neural Text Degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). arXiv:1904.09751.
6. Kojima, T.; Gu, S. S.; Reid, M.; Matsuo, Y.; and Iwasawa, Y. 2023. [Large Language Models are Zero-Shot Reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). arXiv:2205.11916.
7. Pallagani, V.; Muppasani, B.; Murugesan, K.; Rossi, F.; Horesh, L.; Srivastava, B.; Fabiano, F.; and Loreggia, A. 2022. Plansformer: Generating Symbolic Plans using Transformers. arXiv:2212.08681. — [arXiv:2212.08681](https://arxiv.org/abs/2212.08681) [find](https://scholar.google.com/scholar?q=Pallagani%2C+V%2E%3B+Muppasani%2C+B%2E%3B+Murugesan%2C+K%2E%3B+Rossi%2C+F%2E%3B+Horesh%2C+L%2E%3B+Srivastava%2C+B%2E%3B+Fabiano%2C+F%2E%3B+and+Loreggia%2C+A%2E+2022%2E+Plansformer%3A+Generating+Symbolic+Plans+using+Transformers%2E+arXiv%3A2212%2E08681%2E+%E2%80%94+%5B)
8. Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; and Sutskever, I. 2019. Language Models are Unsupervised Multitask Learners. Seipp, J.; Torralba, Á.; and Hoffmann, J. 2022. PDDL Generators. https://doi.org/10.5281/zenodo.6382173. [↗](https://doi.org/10.5281/zenodo.6382173)
9. Valmeekam, K.; Olmo, A.; Sreedharan, S.; and Kambhampati, S. 2023. [Large Language Models Still Can’t Plan (A Benchmark for LLMs on Planning and Reasoning about Change)](https://md.research.elamur.ai/free/en/papers/valmeekam-2022-planbench-extensible-benchmark-evaluating-large/). arXiv:2206.10498.
10. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; and Polosukhin, I. 2017. [Attention Is All You Need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). arXiv:1706.03762.
11. Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Ichter, B.; Xia, F.; Chi, E.; Le, Q.; and Zhou, D. 2023. [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). arXiv:2201.11903.
12. Yang, K.; Deng, J.; and Chen, D. 2022. Generating Natural Language Proofs with Verifier-Guided Search. arXiv:2205.12443. [find](https://scholar.google.com/scholar?q=Yang%2C+K%2E%3B+Deng%2C+J%2E%3B+and+Chen%2C+D%2E+2022%2E+Generating+Natural+Language+Proofs+with+Verifier%2DGuided+Search%2E+arXiv%3A2205%2E12443%2E)
13. Zhuo, H. H.; Zha, Y.; Kambhampati, S.; and Tian, X. 2020. Discovering Underlying Plans Based on Shallow Models. ACM Trans. Intell. Syst. Technol., 11(2): 18:1–18:30. [find](https://scholar.google.com/scholar?q=Zhuo%2C+H%2E+H%2E%3B+Zha%2C+Y%2E%3B+Kambhampati%2C+S%2E%3B+and+Tian%2C+X%2E+2020%2E+Discovering+Underlying+Plans+Based+on+Shallow+Models%2E+ACM+Trans%2E+Intell%2E+Syst%2E+Technol%2E%2C+11%282%29%3A+18%3A1%E2%80%9318%3A30%2E)

*Updated 2 November 2026*