--- title: "Optuna: A Next-generation Hyperparameter — Akiba 2019" description: "Hyperparameter tuning is one of the most laborious parts of machine learning work." authors: ["Takuya Akiba", "Shotaro Sano", "Toshihiko Yanase", "Takeru Ohta", "Masanori Koyama"] year: 2019 venue: "arXiv:1907.10902" source: "https://arxiv.org/abs/1907.10902" doi: "10.48550/arxiv.1907.10902" arxiv: "1907.10902" licence: "arXiv — non-exclusive license to distribute" licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/akiba-2019-optuna-next-generation-hyperparameter-optimization/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/akiba-2019-optuna-next-generation-hyperparameter-optimization/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/) / Optuna: A Next-generation Hyperparameter Optimization Framework # Optuna: A Next-generation Hyperparameter Optimization Framework **Authors:** Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, Masanori Koyama · **Year:** 2019 · **Venue:** arXiv:1907.10902 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/) **Links:** [Publisher's page](https://arxiv.org/abs/1907.10902) · [PDF](https://arxiv.org/pdf/1907.10902) · [DOI 10.48550/arxiv.1907.10902](https://doi.org/10.48550/arxiv.1907.10902) · [arXiv 1907.10902](https://arxiv.org/abs/1907.10902) · [OpenAlex](https://openalex.org/W2962897394) **Citations:** 914 citations (openalex-snapshot · 2026-10-14) · [332](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Hyperparameter tuning is one of the most laborious parts of machine learning work. Existing tools such as Hyperopt, Spearmint, SMAC, Autotune, Vizier, Katib and Tune share several weaknesses. Users must declare the whole search space in advance, which becomes unwieldy for large experiments with many conditional parameters. Many tools lack efficient pruning of weak trials, and setups tend to be either heavyweight or limited in scale. The Preferred Networks authors propose three design criteria for next-generation tools: a define-by-run interface, efficient sampling combined with pruning, and an architecture suited both to quick interactive sessions and to distributed runs. They implement these criteria in Optuna. Borrowing the define-by-run idea from dynamic deep learning frameworks such as Chainer and PyTorch, Optuna lets the objective function request hyperparameters at runtime, so ordinary Python loops and conditionals shape the search space. An MNIST classifier example needs far less code than its Hyperopt equivalent. The authors report that Optuna outperforms several major black-box optimizers, making flexible tuning accessible with minimal setup. ## Key points - Optuna is presented as the first hyperparameter optimization framework built on the define-by-run principle. Table 1 classifies older optimizers and deep learning libraries by API style (define-and-run vs define-by-run). - Core vocabulary: a whole optimization process is a "study" and each objective evaluation is a "trial." Calls like `suggest_int`, `suggest_categorical` and `suggest_loguniform` sample values using the history of earlier trials. - Table 2 compares frameworks on pruning, lightweight setup, distributed support, dashboard and open-source status. SMAC, GPyOpt, Spearmint and Hyperopt are marked as lightweight but without pruning. Autotune, Vizier, Katib and Tune offer pruning but are not lightweight. - The design supports modular code and heterogeneous spaces. One example picks between a RandomForestClassifier and an MLPClassifier, each with its own parameters. - Optuna is released under the MIT license (github.com/pfnet/optuna). It had been in production use at Preferred Networks for over a year at the time of writing. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/) ## Cited by Cited by 332 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 324 not yet. - [TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew](https://md.research.elamur.ai/free/en/papers/babylm-2025-gelboim-tafberta-learning-grammatical-rules-small/) - [Generalizing Cross-Document Event Coreference Resolution Across Multiple Corpora](https://md.research.elamur.ai/free/en/papers/bugert-2021-generalizing-cross-document-event-coreference/) - [The Truth, The Whole Truth, and Nothing but the Truth: A New Benchmark Dataset for Hebrew Text Credibility Assessment](https://md.research.elamur.ai/free/en/papers/findings-2023-hagag-truth-whole-truth-nothing-but/) - [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://md.research.elamur.ai/free/en/papers/khattab-2023-dspy-compiling-declarative-language-model/) - [COMET-22: Unbabel-IST 2022 Submission for the Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/rei-2022-comet-22-unbabel-ist-2022/) - [CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://md.research.elamur.ai/free/en/papers/rei-2022-cometkiwi-ist-unbabel-2022-submission/) - [Scaling up CometKiwi: Unbabel-IST 2023 Submission for the Quality Estimation Shared Task](https://md.research.elamur.ai/free/en/papers/rei-2023-scaling-up-cometkiwi-unbabel-ist/) - [Dialogue Act-based Breakdown Detection in Negotiation Dialogues](https://md.research.elamur.ai/free/en/papers/yamaguchi-2021-dialogue-act-based-breakdown-detection/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{akiba2019optuna, title = {{Optuna: A Next-generation Hyperparameter Optimization Framework}}, author = {Akiba, Takuya and Sano, Shotaro and Yanase, Toshihiko and Ohta, Takeru and Koyama, Masanori}, year = {2019}, doi = {10.48550/arxiv.1907.10902}, eprint = {1907.10902}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.1907.10902} } ``` - [← previous OpenKiwi: An Open Source Framework for Quality Estimation](https://md.research.elamur.ai/free/en/papers/kepler-2019-openkiwi-open-source-framework-quality/) - [next → Opuscula Ugaritico-Accadico-Hebraica: Relative particles, paʿam, and Amraphel](https://md.research.elamur.ai/free/en/papers/hbo-2019-wikander-opuscula-ugaritico-accadico-hebraica-relative/) - [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/)