--- title: "Temporal Tokenization Strategies for Event Sequence Modeling" description: "The study offers practical guidance on choosing a time encoding. 2 references, cited by 1 paper here." authors: ["Zefang Liu", "Nam H Nguyen", "Yinzhu Quan", "Shi-Xiong Zhang"] year: 2026 venue: "Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM)" source: "https://aclanthology.org/2026.gem-main.5/" doi: "10.18653/v1/2026.gem-main.5" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/liu-2026-temporal-tokenization-strategies-event-sequence/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2026-temporal-tokenization-strategies-event-sequence/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/) / Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models # Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models **Authors:** Zefang Liu, Nam H Nguyen, Yinzhu Quan and [Shi-Xiong Zhang](https://md.research.elamur.ai/free/en/authors/shi-xiong-zhang/) · **Year:** 2026 · **Venue:** [Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM)](https://md.research.elamur.ai/free/en/venues/gem/) ([GEM 2026](https://md.research.elamur.ai/free/en/venues/gem/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.gem-main.5/) · [PDF](https://aclanthology.org/2026.gem-main.5.pdf) · [DOI 10.18653/v1/2026.gem-main.5](https://doi.org/10.18653/v1/2026.gem-main.5) · [OpenAlex](https://openalex.org/W4417469573) **Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read ## Summary LLMs process discrete tokens, but events in real sequences happen in continuous time. How to turn timestamps or inter-event gaps into tokens ("temporal tokenization") has received little systematic study. This paper compares five families of encodings: plain numeric strings, float32 byte tokens (as in Language-TPP), Gregorian calendar tokens, uniform scale binning in linear or log space, and a new residual scalar quantization (RSQ) scheme. RSQ stacks K-Means codebooks over successive residuals. Each variant is fine-tuned on Llama-3.2-1B with QLoRA. The data are five public TPP-LLM benchmarks whose interval distributions range from smooth log-normal to multi-modal to sharply spiky. Next-event-type accuracy barely changes across tokenizers, but time-prediction error depends strongly on how well the encoding fits the data's distribution. The study offers practical guidance on choosing a time encoding. It also shows that plain next-token fine-tuning, with no TPP heads or likelihood integrals, is a simpler and competitive alternative to hybrid point-process models. ## Key points - **Data:** five TPP-LLM datasets (hosted on Hugging Face under `tppllm`), covering Amazon Review, US Earthquake, NYC Taxi, Chicago Crime and Stack Overflow. They range from about 30K to about 362K events. - **Configurations and cost:** - Numeric strings use 6 decimals and take roughly 4 tokens. - Byte encoding always uses 4 tokens. - Calendar tokens come in day and second resolution, absolute or relative, using 3 or 6 tokens. - Scale binning uses 256 bins and a single token. - RSQ uses either 1 level of 256 bins or 4 levels of 64 bins. - **Evaluation:** metrics are next-type accuracy, interval RMSE and tokens per time value. Results are averaged over 5 seeds. - **Best time-prediction strategy by dataset:** - Log-scale strategies win on Stack Overflow, Chicago Crime and Amazon Review: Scale Bin (Log) on Chicago Crime, and RSQ (Log, L4) on the other two. - Absolute Calendar (Second) wins on NYC Taxi, since it captures daily and hourly cycles. - Byte wins on US Earthquake. - **Efficiency:** single-token log binning and RSQ offer the best balance between accuracy and token cost. Code is available at github.com/CapitalOne-Research/temporal-tokenization. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Georgia Institute of Technology](https://md.research.elamur.ai/free/en/universities/georgia-institute-of-technology/) — University, Atlanta, United States · 133 works · 24,497 citations - [Capital One](https://md.research.elamur.ai/free/en/companies/capital-one/) — Company, Tysons Corner, United States · 6 works · 16 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 452 works 🧭 Topics (1): [✂️ Tokenization & segmentation](https://md.research.elamur.ai/free/en/topics/tokenization/) · 119 works ## Cite Cite as: Zefang Liu, Nam H Nguyen, Yinzhu Quan and Shi-Xiong Zhang. 2026. Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models. Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{liu2026temporal, title = {{Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models}}, author = {Zefang Liu, Nam H Nguyen, Yinzhu Quan and Shi-Xiong Zhang}, booktitle = {Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM)}, year = {2026}, doi = {10.18653/v1/2026.gem-main.5}, url = {https://doi.org/10.18653/v1/2026.gem-main.5} } ``` - [← previous Template-assisted Contrastive Learning of Task-oriented Dialogue Sentence Embeddings](https://md.research.elamur.ai/free/en/papers/oh-2026-template-assisted-contrastive-learning-task/) - [next → TempPerturb-Eval: On the Joint Effects of Internal Temperature and External Perturbations in RAG Robustness](https://md.research.elamur.ai/free/en/papers/zhou-2026-tempperturb-eval-joint-effects-internal/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/) ## References 2 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Gagan Bhatia, Maxime Peyrard, and Wei Zhao. 2025. Date fragments: A hidden bottleneck of tokenization for temporal reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3201–3219, Suzhou, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Gagan+Bhatia%2C+Maxime+Peyrard%2C+and+Wei+Zhao%2E+2025%2E+Date+fragments%3A+A+hidden+bottleneck+of+tokenization+for+temporal+reasoning%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+Natural+Lang) 2. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. [QLoRA: Efficient finetuning of quantized LLMs](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/). In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 10088–10115. \#\# Limitations *Updated 1 November 2026*