---
title: "Attention-Based Models for Speech — Chorowski 2015"
description: "Attention-based sequence generators had worked well for translation and handwriting synthesis, but speech is harder."
authors: ["Jan Chorowski", "Dzmitry Bahdanau", "Dmitriy Serdyuk", "Kyunghyun Cho", "Yoshua Bengio"]
year: 2015
venue: "arXiv:1506.07503"
source: "https://arxiv.org/abs/1506.07503"
doi: "10.48550/arxiv.1506.07503"
arxiv: "1506.07503"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chorowski-2015-attention-based-models-speech-recognition/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chorowski-2015-attention-based-models-speech-recognition/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/) / Attention-Based Models for Speech Recognition

# Attention-Based Models for Speech Recognition

**Authors:** [Jan Chorowski](https://md.research.elamur.ai/free/en/authors/jan-chorowski/), [Dzmitry Bahdanau](https://md.research.elamur.ai/free/en/authors/dzmitry-bahdanau/), Dmitriy Serdyuk, [Kyunghyun Cho](https://md.research.elamur.ai/free/en/authors/kyunghyun-cho/), [Yoshua Bengio](https://md.research.elamur.ai/free/en/authors/yoshua-bengio/) · **Year:** 2015 · **Venue:** arXiv:1506.07503 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1506.07503) · [PDF](https://arxiv.org/pdf/1506.07503) · [DOI 10.48550/arxiv.1506.07503](https://doi.org/10.48550/arxiv.1506.07503) · [arXiv 1506.07503](https://arxiv.org/abs/1506.07503) · [OpenAlex](https://openalex.org/W854541894)

**Citations:** 1644 citations (openalex-snapshot · 2026-10-14) · [131](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Attention-based sequence generators had worked well for translation and handwriting synthesis, but speech is harder. Inputs span thousands of noisy frames, and similar sounds recur within one recording, so attention driven only by content can lose track of where it is. Chorowski, Bahdanau, Serdyuk, Cho and Bengio study phoneme recognition on TIMIT with an attention-based recurrent sequence generator (ARSG) that reads a deep BiRNN encoding of acoustic features.

A direct adaptation of the Bahdanau et al. translation model does well on the standard test set. It collapses on long inputs, however, apparently because it learns to track absolute position. The authors fix this with hybrid attention: they convolve the previous alignment with learned filters and feed the resulting features into the scoring function alongside content. They also add windowed decoding and a smoother normalization that discourages focusing on a single frame.

The resulting fully neural recognizer matches conventional systems and handles utterances far longer than those seen in training. The location-aware principle is generic and could be applied to other attention models.

## Key points
- Baseline (content-only attention from machine translation): 18.7% PER on TIMIT. On artificially lengthened utterances it aligns roughly the first 40 phones, then jumps to the end of the recording.
- Adding convolutional location features lowers PER to 18.0%. The model stays near 20% PER on inputs built from up to about 10–11 repeated or concatenated test utterances.
- The smoothed score normalization further reduces PER to 17.6%.
- Windowing around the median of the previous alignment cuts attention cost from O(LT) to O(L+T). It was used only at decoding time, and a wide window (±150 frames) beat narrower ones.
- The experiments used Theano, PyLearn2 and Blocks. An earlier version appeared at the NIPS 2014 Deep Learning Workshop.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

131 papers of our library cite it (of the 140673 whose bibliographies we read); 15 of them have a page here, 116 not yet.

- [Morphological Inflection Generation with Hard Monotonic Attention](https://md.research.elamur.ai/free/en/papers/aharoni-2017-morphological-inflection-generation-hard-monotonic/)
- [Unsupervised Data Selection for TTS: Using Arabic Broadcast News as a Case Study](https://md.research.elamur.ai/free/en/papers/baali-2023-unsupervised-data-selection-tts-using/)
- [An Actor-Critic Algorithm for Sequence Prediction](https://md.research.elamur.ai/free/en/papers/bahdanau-2016-actor-critic-algorithm-sequence-prediction/)
- [Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism](https://md.research.elamur.ai/free/en/papers/firat-2016-multi-way-multilingual-neural-machine/)
- [Convolutional Sequence to Sequence Learning](https://md.research.elamur.ai/free/en/papers/gehring-2017-convolutional-sequence-sequence-learning/)
- [Arabic Speech Recognition by End-to-End, Modular Systems and Human](https://md.research.elamur.ai/free/en/papers/hussein-2021-arabic-speech-recognition-end-end/)
- [Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement](https://md.research.elamur.ai/free/en/papers/lee-2018-deterministic-non-autoregressive-neural-sequence/)
- [Does Syntax Need to Grow on Trees? Sources of Hierarchical Inductive Bias in Sequence-to-Sequence Networks](https://md.research.elamur.ai/free/en/papers/mccoy-2020-does-syntax-need-grow-trees/)
- [Neural-based Context Representation Learning for Dialog Act Classification](https://md.research.elamur.ai/free/en/papers/ortega-2017-neural-based-context-representation-learning/)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://md.research.elamur.ai/free/en/papers/pereyra-2017-regularizing-neural-networks-penalizing-confident/)
- [Reasoning about Entailment with Neural Attention](https://md.research.elamur.ai/free/en/papers/rocktaschel-2015-reasoning-about-entailment-neural-attention/)
- [Efficient Content-Based Sparse Attention with Routing Transformers](https://md.research.elamur.ai/free/en/papers/roy-2020-efficient-content-based-sparse-attention/)
- [Pay Less Attention with Lightweight and Dynamic Convolutions](https://md.research.elamur.ai/free/en/papers/wu-2019-pay-less-attention-lightweight-dynamic/)
- [ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs](https://md.research.elamur.ai/free/en/papers/yin-2016-abcnn-attention-based-convolutional-neural/)
- [Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification](https://md.research.elamur.ai/free/en/papers/zhou-2016-attention-based-bidirectional-long-short/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{chorowski2015attention,
  title = {{Attention-Based Models for Speech Recognition}},
  author = {Chorowski, Jan and Bahdanau, Dzmitry and Serdyuk, Dmitriy and Cho, Kyunghyun and Bengio, Yoshua},
  year = {2015},
  doi = {10.48550/arxiv.1506.07503},
  eprint = {1506.07503},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1506.07503}
}
```

- [← previous ASTD: Arabic Sentiment Tweets Dataset](https://md.research.elamur.ai/free/en/papers/nabil-2015-astd-arabic-sentiment-tweets-dataset/)
- [next → Better Document-level Sentiment Analysis from RST Discourse Parsing](https://md.research.elamur.ai/free/en/papers/bhatia-2015-better-document-level-sentiment-analysis/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/)

## References

30 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alex Graves. Generating sequences with recurrent neural networks. arXiv:1308.0850, August 2013. [find](https://scholar.google.com/scholar?q=Alex+Graves%2E+Generating+sequences+with+recurrent+neural+networks%2E+arXiv%3A1308%2E0850%2C+August+2013%2E)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473, September 2014. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+arXiv%3A1409%2E0473%2C+September+2014%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. arXiv:1502.03044, February 2015. [find](https://scholar.google.com/scholar?q=Kelvin+Xu%2C+Jimmy+Ba%2C+Ryan+Kiros%2C+Kyunghyun+Cho%2C+Aaron+Courville%2C+Ruslan+Salakhutdinov%2C+Richard+Zemel%2C+and+Yoshua+Bengio%2E+Show%2C+attend+and+tell%3A+Neural+image+caption+generation+with+visual+attention%2E+a) [our card](https://md.research.elamur.ai/free/en/papers/xu-2015-show-attend-tell-neural-image/)
4. Volodymyr Mnih, Nicolas Heess, Alex Graves, et al. Recurrent models of visual attention. In Advances in Neural Information Processing Systems, pages 2204–2212, 2014. [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Nicolas+Heess%2C+Alex+Graves%2C+et+al%2E+Recurrent+models+of+visual+attention%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+2204%E2%80%932212%2C+2014%2E)
5. Jan Chorowski, Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. End-to-end continuous speech recognition using attention-based recurrent NN: First results. arXiv:1412.1602 [cs, stat], December 2014. — [arXiv:1412.1602](https://arxiv.org/abs/1412.1602) [find](https://scholar.google.com/scholar?q=Jan+Chorowski%2C+Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+End%2Dto%2Dend+continuous+speech+recognition+using+attention%2Dbased+recurrent+NN%3A+First+results%2E+arXiv%3A1412%2E1602+%5Bcs%2C+stat%5D%2C+December+2014)
6. Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. arXiv:1410.5401, 2014. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Greg+Wayne%2C+and+Ivo+Danihelka%2E+Neural+turing+machines%2E+arXiv%3A1410%2E5401%2C+2014%2E)
7. Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. arXiv:1410.3916, 2014. [find](https://scholar.google.com/scholar?q=Jason+Weston%2C+Sumit+Chopra%2C+and+Antoine+Bordes%2E+Memory+networks%2E+arXiv%3A1410%2E3916%2C+2014%2E)
8. Mark Gales and Steve Young. The application of hidden markov models in speech recognition. Found. Trends Signal Process., 1(3):195–304, January 2007. [find](https://scholar.google.com/scholar?q=Mark+Gales+and+Steve+Young%2E+The+application+of+hidden+markov+models+in+speech+recognition%2E+Found%2E+Trends+Signal+Process%2E%2C+1%283%29%3A195%E2%80%93304%2C+January+2007%2E)
9. G. Hinton, Li Deng, Dong Yu, G.E. Dahl, A Mohamed, N. Jaitly, A Senior, V. Vanhoucke, P. Nguyen, T.N. Sainath, and B. Kingsbury. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 29(6):82–97, November 2012. [find](https://scholar.google.com/scholar?q=G%2E+Hinton%2C+Li+Deng%2C+Dong+Yu%2C+G%2EE%2E+Dahl%2C+A+Mohamed%2C+N%2E+Jaitly%2C+A+Senior%2C+V%2E+Vanhoucke%2C+P%2E+Nguyen%2C+T%2EN%2E+Sainath%2C+and+B%2E+Kingsbury%2E+Deep+neural+networks+for+acoustic+modeling+in+speech+recognition%3A+The+s)
10. Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, et al. Deepspeech: Scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567, 2014. [find](https://scholar.google.com/scholar?q=Awni+Hannun%2C+Carl+Case%2C+Jared+Casper%2C+Bryan+Catanzaro%2C+Greg+Diamos%2C+Erich+Elsen%2C+Ryan+Prenger%2C+Sanjeev+Satheesh%2C+Shubho+Sengupta%2C+Adam+Coates%2C+et+al%2E+Deepspeech%3A+Scaling+up+end%2Dto%2Dend+speech+recogniti) [our card](https://md.research.elamur.ai/free/en/papers/hannun-2014-deep-speech-scaling-up-end/)
11. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural. Comput., 9(8):1735–1780, 1997. [find](https://scholar.google.com/scholar?q=S%2E+Hochreiter+and+J%2E+Schmidhuber%2E+Long+short%2Dterm+memory%2E+Neural%2E+Comput%2E%2C+9%288%29%3A1735%E2%80%931780%2C+1997%2E)
12. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP 2014, October 2014. to appear. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+van+Merrienboer%2C+Caglar+Gulcehre%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+Learning+phrase+representations+using+RNN+encoder%2Ddecoder+for+statistical+machine+translation%2E+)
13. Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In ICML-06, 2006. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Santiago+Ferna%CC%81ndez%2C+Faustino+Gomez%2C+and+Ju%CC%88rgen+Schmidhuber%2E+Connectionist+temporal+classification%3A+Labelling+unsegmented+sequence+data+with+recurrent+neural+networks%2E+In+ICML%2D06%2C+2006%2E)
14. Alex Graves. Sequence transduction with recurrent neural networks. In ICML-12, 2012. [find](https://scholar.google.com/scholar?q=Alex+Graves%2E+Sequence+transduction+with+recurrent+neural+networks%2E+In+ICML%2D12%2C+2012%2E) [our card](https://md.research.elamur.ai/free/en/papers/graves-2012-sequence-transduction-recurrent-neural-networks/)
15. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient based learning applied to document recognition. Proc. IEEE, 1998. [find](https://scholar.google.com/scholar?q=Y%2E+LeCun%2C+L%2E+Bottou%2C+Y%2E+Bengio%2C+and+P%2E+Haffner%2E+Gradient+based+learning+applied+to+document+recognition%2E+Proc%2E+IEEE%2C+1998%2E)
16. Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. Speech recognition with deep recurrent neural networks. In ICASSP 2013, pages 6645–6649. IEEE, 2013. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Abdel%2Drahman+Mohamed%2C+and+Geoffrey+Hinton%2E+Speech+recognition+with+deep+recurrent+neural+networks%2E+In+ICASSP+2013%2C+pages+6645%E2%80%936649%2E+IEEE%2C+2013%2E)
17. Alex Graves and Navdeep Jaitly. Towards end-to-end speech recognition with recurrent neural networks. In ICML-14, pages 1764–1772, 2014. [find](https://scholar.google.com/scholar?q=Alex+Graves+and+Navdeep+Jaitly%2E+Towards+end%2Dto%2Dend+speech+recognition+with+recurrent+neural+networks%2E+In+ICML%2D14%2C+pages+1764%E2%80%931772%2C+2014%2E)
18. Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. Weakly supervised memory networks. arXiv preprint arXiv:1503.08895, 2015. [find](https://scholar.google.com/scholar?q=Sainbayar+Sukhbaatar%2C+Arthur+Szlam%2C+Jason+Weston%2C+and+Rob+Fergus%2E+Weakly+supervised+memory+networks%2E+arXiv+preprint+arXiv%3A1503%2E08895%2C+2015%2E)
19. J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, D. S. Pallett, and N. L. Dahlgren. DARPA TIMIT acoustic phonetic continuous speech corpus, 1993. [find](https://scholar.google.com/scholar?q=J%2E+S%2E+Garofolo%2C+L%2E+F%2E+Lamel%2C+W%2E+M%2E+Fisher%2C+J%2E+G%2E+Fiscus%2C+D%2E+S%2E+Pallett%2C+and+N%2E+L%2E+Dahlgren%2E+DARPA+TIMIT+acoustic+phonetic+continuous+speech+corpus%2C+1993%2E)
20. Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, and others. The kaldi speech recognition toolkit. In Proc. ASRU, pages 1–4, 2011. [find](https://scholar.google.com/scholar?q=Daniel+Povey%2C+Arnab+Ghoshal%2C+Gilles+Boulianne%2C+Lukas+Burget%2C+Ondrej+Glembek%2C+Nagendra+Goel%2C+Mirko+Hannemann%2C+Petr+Motlicek%2C+Yanmin+Qian%2C+Petr+Schwarz%2C+and+others%2E+The+kaldi+speech+recognition+toolkit%2E)
21. Matthew D Zeiler. ADADELTA: An adaptive learning rate method. arXiv:1212.5701, 2012. [find](https://scholar.google.com/scholar?q=Matthew+D+Zeiler%2E+ADADELTA%3A+An+adaptive+learning+rate+method%2E+arXiv%3A1212%2E5701%2C+2012%2E) [our card](https://md.research.elamur.ai/free/en/papers/zeiler-2012-adadelta-adaptive-learning-rate-method/)
22. Alex Graves. Practical variational inference for neural networks. In J. Shawe-Taylor, R.S. Zemel, P.L. Bartlett, F. Pereira, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 24, pages 2348–2356. Curran Associates, Inc., 2011. [find](https://scholar.google.com/scholar?q=Alex+Graves%2E+Practical+variational+inference+for+neural+networks%2E+In+J%2E+Shawe%2DTaylor%2C+R%2ES%2E+Zemel%2C+P%2EL%2E+Bartlett%2C+F%2E+Pereira%2C+and+K%2EQ%2E+Weinberger%2C+editors%2C+Advances+in+Neural+Information+Processing+Sys)
23. Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012. [find](https://scholar.google.com/scholar?q=Geoffrey+E+Hinton%2C+Nitish+Srivastava%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+R+Salakhutdinov%2E+Improving+neural+networks+by+preventing+co%2Dadaptation+of+feature+detectors%2E+arXiv+preprint+arXiv%3A1207)
24. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. arXiv preprint arXiv:1409.3215, 2014. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V%2E+Le%2E+Sequence+to+sequence+learning+with+neural+networks%2E+arXiv+preprint+arXiv%3A1409%2E3215%2C+2014%2E)
25. László Tóth. Combining time-and frequency-domain convolution in convolutional neural network-based phone recognition. In ICASSP 2014, pages 190–194, 2014. [find](https://scholar.google.com/scholar?q=La%CC%81szlo%CC%81+To%CC%81th%2E+Combining+time%2Dand+frequency%2Ddomain+convolution+in+convolutional+neural+network%2Dbased+phone+recognition%2E+In+ICASSP+2014%2C+pages+190%E2%80%93194%2C+2014%2E)
26. Caglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loic Barrault, Huei-Chi Lin, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. On using monolingual corpora in neural machine translation. arXiv preprint arXiv:1503.03535, 2015. [find](https://scholar.google.com/scholar?q=Caglar+Gulcehre%2C+Orhan+Firat%2C+Kelvin+Xu%2C+Kyunghyun+Cho%2C+Loic+Barrault%2C+Huei%2DChi+Lin%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+On+using+monolingual+corpora+in+neural+machine+translation%2E+arXi) [our card](https://md.research.elamur.ai/free/en/papers/gulcehre-2015-using-monolingual-corpora-neural-machine/)
27. James Bergstra, Olivier Breuleux, Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010. Oral… [find](https://scholar.google.com/scholar?q=James+Bergstra%2C+Olivier+Breuleux%2C+Fre%CC%81de%CC%81ric+Bastien%2C+Pascal+Lamblin%2C+Razvan+Pascanu%2C+Guillaume+Desjardins%2C+Joseph+Turian%2C+David+Warde%2DFarley%2C+and+Yoshua+Bengio%2E+Theano%3A+a+CPU+and+GPU+math+expression+)
28. Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, James Bergstra, Ian J. Goodfellow, Arnaud Bergeron, Nicolas Bouchard, and Yoshua Bengio. Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop, 2012. [find](https://scholar.google.com/scholar?q=Fre%CC%81de%CC%81ric+Bastien%2C+Pascal+Lamblin%2C+Razvan+Pascanu%2C+James+Bergstra%2C+Ian+J%2E+Goodfellow%2C+Arnaud+Bergeron%2C+Nicolas+Bouchard%2C+and+Yoshua+Bengio%2E+Theano%3A+new+features+and+speed+improvements%2E+Deep+Learning+)
29. Ian J. Goodfellow, David Warde-Farley, Pascal Lamblin, Vincent Dumoulin, Mehdi Mirza, Razvan Pascanu, James Bergstra, Frédéric Bastien, and Yoshua Bengio. Pylearn2: a machine learning research library. arXiv preprint arXiv:1308.4214, 2013. — [arXiv:1308.4214](https://arxiv.org/abs/1308.4214) [find](https://scholar.google.com/scholar?q=Ian+J%2E+Goodfellow%2C+David+Warde%2DFarley%2C+Pascal+Lamblin%2C+Vincent+Dumoulin%2C+Mehdi+Mirza%2C+Razvan+Pascanu%2C+James+Bergstra%2C+Fre%CC%81de%CC%81ric+Bastien%2C+and+Yoshua+Bengio%2E+Pylearn2%3A+a+machine+learning+research+libra)
30. Bart van Merriënboer, Dzmitry Bahdanau, Vincent Dumoulin, Dmitriy Serdyuk, David Warde-Farley, Jan Chorowski, and Yoshua Bengio. Blocks and fuel: Frameworks for deep learning. arXiv:1506.00619 [cs, stat], June 2015. [find](https://scholar.google.com/scholar?q=Bart+van+Merrie%CC%88nboer%2C+Dzmitry+Bahdanau%2C+Vincent+Dumoulin%2C+Dmitriy+Serdyuk%2C+David+Warde%2DFarley%2C+Jan+Chorowski%2C+and+Yoshua+Bengio%2E+Blocks+and+fuel%3A+Frameworks+for+deep+learning%2E+arXiv%3A1506%2E00619+%5Bcs%2C+s)