---
title: "LSTM-Based Text Generation: A Study on Historical Datasets"
description: "The paper presents this as evidence that LSTMs generate historical text efficiently. 21 references."
authors: ["Mustafa Abbas Hussein Hussein", "Serkan Sava\\cs"]
year: 2024
venue: "arXiv:2403.07087 · 16th International Istanbul Scientific Research Congress on Life, Engineering, Architecture, and Mathematical Sciences Proceedings Book, Pages: 42-49, 2024"
source: "https://arxiv.org/abs/2403.07087"
doi: "10.5281/zenodo.10776102"
arxiv: "2403.07087"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hussein-2024-lstm-based-text-generation-study/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hussein-2024-lstm-based-text-generation-study/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/) / LSTM-Based Text Generation: A Study on Historical Datasets

# LSTM-Based Text Generation: A Study on Historical Datasets

**Authors:** Mustafa Abbas Hussein Hussein and Serkan Sava\\cs · **Year:** 2024 · **Venue:** arXiv:2403.07087 · 16th International Istanbul Scientific Research Congress on Life, Engineering, Architecture, and Mathematical Sciences Proceedings Book, Pages: 42-49, 2024 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://doi.org/10.5281/zenodo.10776102) · [PDF](https://arxiv.org/pdf/2403.07087) · [DOI 10.5281/zenodo.10776102](https://doi.org/10.5281/zenodo.10776102) · [arXiv 2403.07087](https://arxiv.org/abs/2403.07087) · [OpenAlex](https://openalex.org/W4392781663)

**Citations:** 4 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks how well a plain recurrent network can learn to write in the style of older literary English. The authors use two public Kaggle collections: Shakespeare's complete plays, with play, line position and speaker per row, and an English Nietzsche corpus. After removing citations and numbering, they split each 70/30 into training and test parts. They then train a three-layer character-level LSTM in Keras. Characters are mapped to integers for input, and a lookup dictionary turns predicted indices back into letters. After 100 training iterations, they report next-character accuracy of 0.9521 (loss 0.2518) on Nietzsche and 0.9125 (loss 0.3876) on Shakespeare. Both figures beat numbers quoted from two earlier works that trained for longer. The paper presents this as evidence that LSTMs generate historical text efficiently. However, it does not report human judgments of output quality or perplexity. It also does not show sample generations or make clear whether the accuracy comes from training or held-out data. The claimed gains should therefore be read cautiously.

## Key points
- Data: a Shakespeare plays dataset (LiamLarsen, 2017), with play name, act/scene/line location, speaker and spoken line, and a Nietzsche English text dataset (Kris, 2018).
- Method: a 3-layer LSTM in Keras that predicts the next character, with a 70% / 30% train/test split after light cleaning.
- Nietzsche: accuracy 0.9521 and loss 0.2518 in 100 iterations, versus 0.758 in 160 iterations reported for Gao & Glowacka (2016).
- Shakespeare: accuracy 0.9125 and loss 0.3876 in 100 iterations, versus 0.746 and loss 1.020 in 500 iterations for Hussain et al. (2021).
- The authors list three intended evaluation goals: no repeated phrases, grammatical new sentences, and output that stays relevant to the input context. However, only accuracy and loss curves are actually reported.

## Interesting ideas
- The paper names three separate criteria for generated text (no repetition, grammaticality, staying on the input's context), which is more useful than a single accuracy score.
- The Shakespeare corpus labels every line with its play, its position in the play and its speaker, so dialogue comes with built-in speaker attribution.
- The study measures how quickly the model learns by reporting the number of training iterations needed next to accuracy.
- The paper's comparison takes headline numbers from earlier papers without matching splits or setups, which shows why apparent gains can be hard to trust.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (back): Mova's graph → text generator could be scored on these three axes separately, with context fidelity checked by re-parsing the output and comparing its frames to the source graph.
- **I2** (frames): such speaker-tagged drama could serve as cheap gold data for tracking who says what, testing speech-act and coreference handling for "I/you" across dialogue turns.
- **I3** (learn): when retraining the perceptron parser or gparse on new silver data, logging epochs-to-plateau next to LAS would make cost/benefit comparisons between data tiers explicit.
- **I4** (learn): this argues for always rerunning baselines on the same split and adding negative controls (e.g., shuffled-input models) before claiming an improvement.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [✍️ Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/) · 214 works

## Cite

Cite as: Mustafa Abbas Hussein Hussein and Serkan Sava\\cs. 2024. LSTM-Based Text Generation: A Study on Historical Datasets. arXiv:2403.07087 · 16th International Istanbul Scientific Research Congress on Life, Engineering, Architecture, and Mathematical Sciences Proceedings Book, Pages: 42-49, 2024.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{hussein2024lstm,
  title = {{LSTM-Based Text Generation: A Study on Historical Datasets}},
  author = {Mustafa Abbas Hussein Hussein and Serkan Savacs},
  year = {2024},
  doi = {10.5281/zenodo.10776102},
  eprint = {2403.07087},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.5281/zenodo.10776102}
}
```

- [← previous LLMs of Catan: Exploring Pragmatic Capabilities of Generative Chatbots Through Prediction and Classification of Dialogue Acts in Boardgames' Multi-party Dialogues](https://md.research.elamur.ai/free/en/papers/martinenghi-2024-llms-catan-exploring-pragmatic-capabilities/)
- [next → Methods for Matching English Language Addresses](https://md.research.elamur.ai/free/en/papers/ramani-2024-methods-matching-english-language-addresses/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/)

## References

21 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ayan, E. (2022). Using a Convolutional Neural Network as Feature Extractor for Different Machine Learning Classifiers to Diagnose Pneumonia. Sakarya University Journal of Computer and Information Sciences, 5(1), 48–61. https://doi.org/10.35377/saucis.5.69696.1019187 [↗](https://doi.org/10.35377/saucis.5.69696.1019187)
2. Bilen, B., & Horasan, F. (2022). LSTM Network based Sentiment Analysis for Customer Reviews. Politeknik Dergisi, 25(3), 959–966. https://doi.org/10.2339/politeknik.844019 [↗](https://doi.org/10.2339/politeknik.844019)
3. Bütüner, R., & Calp, M. H. (2022). Diagnosis and Detection of COVID-19 from Lung Tomography Images Using Deep Learning and Machine Learning Methods. International Journal of Intelligent Systems and Applications in Engineering, 10(2), 190–200. [https://ijisae.org/index.php/IJISAE/article/view/1843](https://ijisae.org/index.php/IJISAE/article/view/1843) [↗](https://ijisae.org/index.php/IJISAE/article/view/1843)
4. Buyrukoğlu, G., Buyrukoğlu, S., & Topalcengiz, Z. (2021). Comparing Regression Models with Count Data to Artificial Neural Network and Ensemble Models for Prediction of Generic Escherichia coli Population in Agricultural Ponds Based on Weather Station Measurements. Microbial Risk Analysis, 19, 100171. https://doi.org/10.1016/J.MRAN.2021.100171 [↗](https://doi.org/10.1016/j.mran.2021.100171)
5. Colon, R. S., Patra, P. K., & Elleithy, K. M. (2014). Random word retrieval for automatic story generation. Proceedings of the 2014 Zone 1 Conference of the American Society for Engineering Education, 1–6. https://doi.org/10.1109/ASEEZone1.2014.6820655 [↗](https://doi.org/10.1109/aseezone1.2014.6820655)
6. Gao, Y., & Glowacka, D. (2016). Deep Gate Recurrent Neural Network. In R. J. Durrant & K.-E. Kim (Eds.), Proceedings of The 8th Asian Conference on Machine Learning (Vol. 63, pp. 350–365). PMLR. https://proceedings.mlr.press/v63/gao30.html Güler, O., & Polat, K. (2022). Classification Performance of Deep Transfer Learning Methods for Pneumonia Detection from Chest X-Ray Images. Journal of Artificial Intelligence and Systems, 4(1), 107–126. https://doi.org/10.33969/AIS.2022040107 [↗](https://doi.org/10.33969/ais.2022040107)
7. Hatipoglu, A., & Omurca, S. ilhan. (2016). A Turkish Wikipedia Text Summarization System for Mobile Devices. International Journal of Information Technology and Computer Science, 8(1), 1. https://doi.org/10.5815/ IJITCS.2016.01.01 [↗](https://doi.org/10.5815/)
8. Hussain, H., Maliyakal, J. J., Philip, D., & Yadav, O. (2021). Shakespeare like Text Generation using LSTM and Tensorflow. International Journal of Emerging Technologies and Innovative Research, 8(5), b991–b993. [https://www.jetir.org/view?paper=JETIR2105252](https://www.jetir.org/view?paper=JETIR2105252) [↗](https://www.jetir.org/view?paper=JETIR2105252)
9. Jain, P., Agrawal, P., Mishra, A., Sukhwani, M., Laha, A., & Sankaranarayanan, K. (2017, July 18). Story Generation from Sequence of Independent Short Descriptions. Story Generation from Sequence of Independent Short Descriptions. https://doi.org/10.475/123 [↗](https://doi.org/10.475/123)
10. Jaya, A., & Uma, G. V. (2011). An Intelligent Automatic Story Generation System by Revising Proppian’s System. In B. K. and N. D. Meghanathan Natarajan and Kaushik (Ed.), Advances in Computer Science and Information Technology (pp. 594–603). Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Jaya%2C+A%2E%2C+%26+Uma%2C+G%2E+V%2E+%282011%29%2E+An+Intelligent+Automatic+Story+Generation+System+by+Revising+Proppian%E2%80%99s+System%2E+In+B%2E+K%2E+and+N%2E+D%2E+Meghanathan+Natarajan+and+Kaushik+%28Ed%2E%29%2C+Advances+in+Computer+Science+)
11. Jaya, A., & Uma, G. V. (2010). An intelligent system for semi-automatic story generation for kids using ontology. Proceedings of the Third Annual ACM Bangalore Conference. https://doi.org/10.1145/1754288.1754296 [↗](https://doi.org/10.1145/1754288.1754296)
12. Kavila, S. D., & Radhika, Y. (2015). Extractive Text Summarization Using Modified Weighing and Sentence Symmetric Feature Methods. International Journal of Modern Education and Computer Science, 7(10), 33. https://doi.org/10.5815/IJMECS.2015.10.05 DOI: 10.5281/ zenODO .10776102 ISBN: 978-625-6879-50-8 16 TH INTERNATIONAL İSTANBUL SCIENTIFIC RESEARCH CONGRESS ON LIFE, ENGINEERING, ARCHITECTURE, AND MATHEMATICAL SCIENCES PROCEEDINGS BOOK Khatri, C., Voleti, S., Veeraraghavan, S., Parikh, N., Islam, A., Mahmood, S., Garg, N., & Singh, V. (2015). Algorithmic content generation for products. 2015 IEEE International Conference on Big Data (Big Data), 2945–2947. https://doi.org/10.1109/BigData.2015.7364131 [↗](https://doi.org/10.5815/ijmecs.2015.10.05)
13. Kris. (2018). Nietzsche Texts. Kaggle - Nietzsche Texts. [https://www.kaggle.com/datasets/pankrzysiu/nietzsche-texts](https://www.kaggle.com/datasets/pankrzysiu/nietzsche-texts) [↗](https://www.kaggle.com/datasets/pankrzysiu/nietzsche-texts)
14. Le, H. T., & Le, T. M. (2013). An approach to abstractive text summarization. International Conference of Soft Computing and Pattern Recognition, 371–376. https://doi.org/10.1109/SOCPAR.2013.7054161 [↗](https://doi.org/10.1109/socpar.2013.7054161)
15. Li, B., Lee-Urban, S., Johnston, G., & Riedl, M. (2013). Story Generation with Crowdsourced Plot Graphs. Proceedings of the AAAI Conference on Artificial Intelligence, 27(1), 598–604. https://doi.org/10.1609/aaai. v27i1.8649 [↗](https://doi.org/10.1609/aaai)
16. LiamLarsen. (2017). Shakespeare plays. Kaggle - Shakespeare Plays. [https://www.kaggle.com/datasets/kingbur-rito666/shakespeare-plays/data](https://www.kaggle.com/datasets/kingbur-rito666/shakespeare-plays/data) [↗](https://www.kaggle.com/datasets/kingbur-rito666/shakespeare-plays/data)
17. Mehta, A., Gala, R., & Kurup, L. (2016). A roadmap to auto story generation. 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), 2306–2310. [find](https://scholar.google.com/scholar?q=Mehta%2C+A%2E%2C+Gala%2C+R%2E%2C+%26+Kurup%2C+L%2E+%282016%29%2E+A+roadmap+to+auto+story+generation%2E+2016+3rd+International+Conference+on+Computing+for+Sustainable+Global+Development+%28INDIACom%29%2C+2306%E2%80%932310%2E)
18. Nallapati, R., Zhou, B., santos, C. N. dos, Gulcehre, C., & Xiang, B. (2016, February 18). [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://md.research.elamur.ai/free/en/papers/nallapati-2016-abstractive-text-summarization-sequence-sequence/). Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond.
19. Pawade, D., Sakhapara, A., Jain, M., Jain, N., & Gada, K. (2018). Story Scrambler - Automatic Text Generation Using Word Level RNN-LSTM. International Journal of Information Technology and Computer Science, 10(6), 44–53. https://doi.org/10.5815/IJITCS.2018.06.05 [↗](https://doi.org/10.5815/ijitcs.2018.06.05)
20. Sakhare, D. Y., & Kumar, R. (2014). Syntactic and Sentence Feature Based Hybrid Approach for Text Summarization. International Journal of Information Technology and Computer Science, 6(3), 38–46. https://doi.org/10.5815/IJITCS.2014.03.05 [↗](https://doi.org/10.5815/ijitcs.2014.03.05)
21. Thomaidou, S., Lourentzou, I., Katsivelis-Perakis, P., & Vazirgiannis, M. (2013). Automated snippet generation for online advertising. Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, 1841–1844. https://doi.org/10.1145/2505515.2507876 [↗](https://doi.org/10.1145/2505515.2507876)

*Updated 29 October 2026*