---
title: "Time Waits for No One! Analysis and Challenges of Temporal"
description: "The authors build a benchmark of eight timestamped tasks from four domains: Twitter, scientific… 57 references, cited by 61 papers here."
authors: ["Kelvin Luu", "Daniel Khashabi", "Suchin Gururangan", "Karishma Mandyam", "Noah A. Smith"]
year: 2022
venue: "Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies"
source: "https://aclanthology.org/2022.naacl-main.435/"
doi: "10.18653/v1/2022.naacl-main.435"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/luu-2022-time-waits-no-one-analysis/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/luu-2022-time-waits-no-one-analysis/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) / Time Waits for No One! Analysis and Challenges of Temporal Misalignment

# Time Waits for No One! Analysis and Challenges of Temporal Misalignment

**Authors:** Kelvin Luu, [Daniel Khashabi](https://md.research.elamur.ai/free/en/authors/daniel-khashabi/), [Suchin Gururangan](https://md.research.elamur.ai/free/en/authors/suchin-gururangan/), Karishma Mandyam and [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/) · **Year:** 2022 · **Venue:** [Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2022](https://md.research.elamur.ai/free/en/venues/naacl/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.naacl-main.435/) · [PDF](https://aclanthology.org/2022.naacl-main.435.pdf) · [DOI 10.18653/v1/2022.naacl-main.435](https://doi.org/10.18653/v1/2022.naacl-main.435) · [OpenAlex](https://openalex.org/W3213460052)

**Citations:** 40 citations (OpenAlex, 2026-10-14) · [61](https://md.research.elamur.ai/free/en/papers/luu-2022-time-waits-no-one-analysis/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper examines temporal misalignment, the situation in which the text used to train an NLP model and the text it is evaluated or deployed on come from different periods. The authors build a benchmark of eight timestamped tasks from four domains: Twitter, scientific articles, news, and food reviews. Each task covers at least five years. They use a standard pipeline in which GPT-2 is pretrained, optionally adapted to a domain or time period (DAPT), and then finetuned on a task. To isolate the effect of time, they hold the test period fixed, vary the training period, and keep training sets the same size. They also propose the TD score, which estimates how much performance drops per unit of time gap. Degradation turns out to be stronger than earlier work reported, and it differs widely between tasks, even within a single domain. Continued pretraining on unlabeled text from the target period helps only slightly. Finetuning on labeled data from that period helps much more. The practical lesson is that temporal adaptation cannot replace fresh annotations, and benchmarks whose train and test data come from the same period may overstate real-world performance.

## Key points
- **Benchmark:** eight tasks across four domains, three of them newly defined by the authors.
  - Examples include political affiliation of tweets (PoliAff), Twitter entity typing (TwiERC), publisher classification, summarization, and review rating classification.
  - Roughly 12M unlabeled tweets from 2015–2020 were gathered for temporal adaptation.
- **TD score:** for each test period, a least-squares line is fitted to a sign-adjusted performance gap plotted against training period. The line's slope gives the degradation rate, and these slopes are averaged across test periods.
- **Size of the effect:** over five years, F1 fell by about 40 points for political affiliation on Twitter but only about 1 point for Yelp review ratings.
- **Temporal adaptation vs. finetuning:** continued GPT-2 pretraining on data from the target period produced small gains compared with finetuning on labeled data from that period. This agrees with Röttger and Pierrehumbert (2021) and extends their findings to more tasks and longer time spans.
- **Language model drift:** GPT-2 perplexity also rises as the gap between training and evaluation periods grows, and the size of that rise depends on the domain.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 263 works · 180,581 citations
- [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) — Research institute, Seattle, United States · 219 works · 89,712 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Kelvin Luu, Daniel Khashabi, Suchin Gururangan, Karishma Mandyam and Noah A. Smith. 2022. Time Waits for No One! Analysis and Challenges of Temporal Misalignment. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{luu2022time,
  title = {{Time Waits for No One! Analysis and Challenges of Temporal Misalignment}},
  author = {Kelvin Luu, Daniel Khashabi, Suchin Gururangan, Karishma Mandyam and Noah A. Smith},
  booktitle = {Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2022},
  doi = {10.18653/v1/2022.naacl-main.435},
  url = {https://doi.org/10.18653/v1/2022.naacl-main.435}
}
```

- [← previous Text-Free Prosody-Aware Generative Spoken Language Modeling](https://md.research.elamur.ai/free/en/papers/kharitonov-2022-text-free-prosody-aware-generative/)
- [next → On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations](https://md.research.elamur.ai/free/en/papers/cao-2022-intrinsic-extrinsic-fairness-evaluation-metrics/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/)

## References

57 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eduardo G Altmann, Janet B Pierrehumbert, and Adilson E Motter. 2009. Beyond word frequency: Bursts, lulls, and scaling in the temporal distributions of words. PLOS one, 4(11):e7678. [find](https://scholar.google.com/scholar?q=Eduardo+G+Altmann%2C+Janet+B+Pierrehumbert%2C+and+Adilson+E+Motter%2E+2009%2E+Beyond+word+frequency%3A+Bursts%2C+lulls%2C+and+scaling+in+the+temporal+distributions+of+words%2E+PLOS+one%2C+4%2811%29%3Ae7678%2E)
2. Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, et al. 2018. [Construction of the literature graph in semantic scholar](https://md.research.elamur.ai/free/en/papers/ammar-2018-construction-literature-graph-semantic-scholar/). In NAACL.
3. Fan Bai, Alan Ritter, and Wei Xu. 2021. Pre-train or annotate? domain adaptation with a constrained budget. In EMNLP, pages 5002–5015. [find](https://scholar.google.com/scholar?q=Fan+Bai%2C+Alan+Ritter%2C+and+Wei+Xu%2E+2021%2E+Pre%2Dtrain+or+annotate%3F+domain+adaptation+with+a+constrained+budget%2E+In+EMNLP%2C+pages+5002%E2%80%935015%2E)
4. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. [Scibert: A pretrained language model for scientific text](https://md.research.elamur.ai/free/en/papers/beltagy-2019-scibert-pretrained-language-model-scientific/). In EMNLP.
5. Andrew Eliot Borthwick. 1999. A maximum entropy approach to named entity recognition. New York University. [find](https://scholar.google.com/scholar?q=Andrew+Eliot+Borthwick%2E+1999%2E+A+maximum+entropy+approach+to+named+entity+recognition%2E+New+York+University%2E)
6. Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). arXiv preprint arXiv:2005.14165.
7. Ivy Cao, Zizhou Liu, Giannis Karamanolakis, Daniel Hsu, and Luis Gravano. 2021. Quantifying the effects of COVID-19 on restaurant reviews. In Proceedings of the International Workshop on Natural Language Processing for Social Media, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ivy+Cao%2C+Zizhou+Liu%2C+Giannis+Karamanolakis%2C+Daniel+Hsu%2C+and+Luis+Gravano%2E+2021%2E+Quantifying+the+effects+of+COVID%2D19+on+restaurant+reviews%2E+In+Proceedings+of+the+International+Workshop+on+Natural+Langu)
8. Dallas Card, Amber E. Boydstun, Justin H. Gross, Philip Resnik, and Noah A. Smith. 2015. [The media frames corpus: Annotations of frames across issues](https://md.research.elamur.ai/free/en/papers/card-2015-media-frames-corpus-annotations-frames/). In ACL.
9. Alexandra Chronopoulou, Matthew E. Peters, and Jesse Dodge. 2021. Efficient hierarchical domain adaptation for pretrained language models. ArXiv, abs/2112.08786. [find](https://scholar.google.com/scholar?q=Alexandra+Chronopoulou%2C+Matthew+E%2E+Peters%2C+and+Jesse+Dodge%2E+2021%2E+Efficient+hierarchical+domain+adaptation+for+pretrained+language+models%2E+ArXiv%2C+abs%2F2112%2E08786%2E)
10. Hal Daumé III. 2007. [Frustratingly easy domain adaptation](https://md.research.elamur.ai/free/en/papers/iii-2007-frustratingly-easy-domain-adaptation/). In ACL, pages 256–263.
11. Kushal Dave, Steve Lawrence, and David M. Pennock. 2003. Mining the peanut gallery: opinion extraction and semantic classification of product reviews. In WWW. [find](https://scholar.google.com/scholar?q=Kushal+Dave%2C+Steve+Lawrence%2C+and+David+M%2E+Pennock%2E+2003%2E+Mining+the+peanut+gallery%3A+opinion+extraction+and+semantic+classification+of+product+reviews%2E+In+WWW%2E)
12. Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. 2019. Episodic memory in lifelong language learning. In nips. [find](https://scholar.google.com/scholar?q=Cyprien+de+Masson+d%27Autume%2C+Sebastian+Ruder%2C+Lingpeng+Kong%2C+and+Dani+Yogatama%2E+2019%2E+Episodic+memory+in+lifelong+language+learning%2E+In+nips%2E)
13. Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W. Cohen. 2021. [Time-aware language models as temporal knowledge bases](https://md.research.elamur.ai/free/en/papers/dhingra-2022-time-aware-language-models-as/). CoRR, abs/2106.15110.
14. Jesse Dodge, Maarten Sap, Ana Marasovic, William Agnew, Gabriel Ilharco, Dirk Groeneveld, and Matt Gardner. 2021. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In EMNLP. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Maarten+Sap%2C+Ana+Marasovic%2C+William+Agnew%2C+Gabriel+Ilharco%2C+Dirk+Groeneveld%2C+and+Matt+Gardner%2E+2021%2E+Documenting+large+webtext+corpora%3A+A+case+study+on+the+colossal+clean+crawled+corpus%2E+)
15. Jacob Eisenstein, Brendan O’Connor, Noah A Smith, and Eric P Xing. 2014. Diffusion of lexical change in social media. PloS one, 9(11). [find](https://scholar.google.com/scholar?q=Jacob+Eisenstein%2C+Brendan+O%E2%80%99Connor%2C+Noah+A+Smith%2C+and+Eric+P+Xing%2E+2014%2E+Diffusion+of+lexical+change+in+social+media%2E+PloS+one%2C+9%2811%29%2E)
16. Robert M. Entman. 1983. Framing: Toward clarification of a fractured paradigm. Journal of Communications. [find](https://scholar.google.com/scholar?q=Robert+M%2E+Entman%2E+1983%2E+Framing%3A+Toward+clarification+of+a+fractured+paradigm%2E+Journal+of+Communications%2E)
17. Hila Gonen, Ganesh Jawahar, Djamé Seddah, and Yoav Goldberg. 2020. Simple, interpretable and stable method for detecting words with usage change across corpora. In ACL. [find](https://scholar.google.com/scholar?q=Hila+Gonen%2C+Ganesh+Jawahar%2C+Djam%C3%A9+Seddah%2C+and+Yoav+Goldberg%2E+2020%2E+Simple%2C+interpretable+and+stable+method+for+detecting+words+with+usage+change+across+corpora%2E+In+ACL%2E)
18. Max Grusky, Mor Naaman, and Yoav Artzi. 2018. [Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies](https://md.research.elamur.ai/free/en/papers/grusky-2018-newsroom-dataset-1-3-million/). In NAACL.
19. Suchin Gururangan, Mike Lewis, Ari Holtzman, Noah A. Smith, and Luke Zettlemoyer. 2021. Demix layers: Disentangling domains for modular language modeling. CoRR, abs/2108.05036. [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Mike+Lewis%2C+Ari+Holtzman%2C+Noah+A%2E+Smith%2C+and+Luke+Zettlemoyer%2E+2021%2E+Demix+layers%3A+Disentangling+domains+for+modular+language+modeling%2E+CoRR%2C+abs%2F2108%2E05036%2E)
20. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In ACL. [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Ana+Marasovic%CC%81%2C+Swabha+Swayamdipta%2C+Kyle+Lo%2C+Iz+Beltagy%2C+Doug+Downey%2C+and+Noah+A+Smith%2E+2020%2E+Don%E2%80%99t+stop+pretraining%3A+Adapt+language+models+to+domains+and+tasks%2E+In+ACL%2E)
21. William L. Hamilton, Jure Leskovec, and Dan Jurafsky. 2016. [Diachronic word embeddings reveal statistical laws of semantic change](https://md.research.elamur.ai/free/en/papers/hamilton-2016-diachronic-word-embeddings-reveal-statistical/). In ACL.
22. Xiaochuang Han and Jacob Eisenstein. 2019. [Unsupervised domain adaptation of contextualized embeddings for sequence labeling](https://md.research.elamur.ai/free/en/papers/han-2019-unsupervised-domain-adaptation-contextualized-embeddings/). In EMNLP.
23. Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang, and Diyi Yang. 2021. Continual learning for text classification with information disentanglement based regularization. In ACL. [find](https://scholar.google.com/scholar?q=Yufan+Huang%2C+Yanzhe+Zhang%2C+Jiaao+Chen%2C+Xuezhi+Wang%2C+and+Diyi+Yang%2E+2021%2E+Continual+learning+for+text+classification+with+information+disentanglement+based+regularization%2E+In+ACL%2E)
24. Mohit Iyyer, Peter Enns, Jordan Boyd-Graber, and Philip Resnik. 2014. [Political ideology detection using recursive neural networks](https://md.research.elamur.ai/free/en/papers/iyyer-2014-political-ideology-detection-recursive-neural/). In ACL.
25. Kokil Jaidka, Niyati Chhaya, and Lyle Ungar. 2018. Diachronic degradation of language models: Insights from social media. In ACL. [find](https://scholar.google.com/scholar?q=Kokil+Jaidka%2C+Niyati+Chhaya%2C+and+Lyle+Ungar%2E+2018%2E+Diachronic+degradation+of+language+models%3A+Insights+from+social+media%2E+In+ACL%2E)
26. Jing Jiang and ChengXiang Zhai. 2007. [Instance weighting for domain adaptation in nlp](https://md.research.elamur.ai/free/en/papers/jiang-2007-instance-weighting-domain-adaptation-nlp/). In ACL.
27. Xisen Jin, Dejiao Zhang, Henghui Zhu, Wei Xiao, Shang-Wen Li, Xiaokai Wei, Andrew Arnold, and Xiang Ren. 2021. Lifelong pretraining: Continually adapting language models to emerging corpora. arXiv preprint arXiv:2110.08534. [find](https://scholar.google.com/scholar?q=Xisen+Jin%2C+Dejiao+Zhang%2C+Henghui+Zhu%2C+Wei+Xiao%2C+Shang%2DWen+Li%2C+Xiaokai+Wei%2C+Andrew+Arnold%2C+and+Xiang+Ren%2E+2021%2E+Lifelong+pretraining%3A+Continually+adapting+language+models+to+emerging+corpora%2E+arXiv+pre)
28. William Labov. 2011. Principles of linguistic change, volume 3: Cognitive and cultural factors, volume 36. John Wiley & Sons. [find](https://scholar.google.com/scholar?q=William+Labov%2E+2011%2E+Principles+of+linguistic+change%2C+volume+3%3A+Cognitive+and+cultural+factors%2C+volume+36%2E+John+Wiley+%26+Sons%2E)
29. Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d’Autume, Sebastian Ruder, Dani Yogatama, et al. 2021. Pitfalls of static language modelling. arXiv preprint arXiv:2102.01951. [find](https://scholar.google.com/scholar?q=Angeliki+Lazaridou%2C+Adhiguna+Kuncoro%2C+Elena+Gribovskaya%2C+Devang+Agrawal%2C+Adam+Liska%2C+Tayfun+Terzi%2C+Mai+Gimenez%2C+Cyprien+de+Masson+d%E2%80%99Autume%2C+Sebastian+Ruder%2C+Dani+Yogatama%2C+et+al%2E+2021%2E+Pitfalls+of+sta)
30. Chin-Yew Lin. 2004. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Proc. of Text Summarization Branches Out.
31. Wei-Hao Lin, Eric P. Xing, and Alexander Hauptmann. 2008. A joint topic and perspective model for ideological discourse. In ECML/PKDD. [find](https://scholar.google.com/scholar?q=Wei%2DHao+Lin%2C+Eric+P%2E+Xing%2C+and+Alexander+Hauptmann%2E+2008%2E+A+joint+topic+and+perspective+model+for+ideological+discourse%2E+In+ECML%2FPKDD%2E)
32. Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel S Weld. 2020. [S2orc: The semantic scholar open research corpus](https://md.research.elamur.ai/free/en/papers/lo-2019-s2orc-semantic-scholar-open-research/). In ACL.
33. Jie Lu, Anjin Liu, Fan Dong, Feng Gu, João Gama, and Guangquan Zhang. 2019. Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering. [find](https://scholar.google.com/scholar?q=Jie+Lu%2C+Anjin+Liu%2C+Fan+Dong%2C+Feng+Gu%2C+Jo%C3%A3o+Gama%2C+and+Guangquan+Zhang%2E+2019%2E+Learning+under+concept+drift%3A+A+review%2E+IEEE+Transactions+on+Knowledge+and+Data+Engineering%2E)
34. Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018. [Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction](https://md.research.elamur.ai/free/en/papers/luan-2018-multi-task-identification-entities-relations/). In EMNLP.
35. Afonso Mendes, Shashi Narayan, Sebastião Miranda, Zita Marinho, André F. T. Martins, and Shay B. Cohen. 2019. Jointly extracting and compressing documents with summary state representations. In NAACL. [find](https://scholar.google.com/scholar?q=Afonso+Mendes%2C+Shashi+Narayan%2C+Sebasti%C3%A3o+Miranda%2C+Zita+Marinho%2C+Andr%C3%A9+F%2E+T%2E+Martins%2C+and+Shay+B%2E+Cohen%2E+2019%2E+Jointly+extracting+and+compressing+documents+with+summary+state+representations%2E+In+NAACL%2E)
36. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. [Thumbs up? sentiment classification using machine learning techniques](https://md.research.elamur.ai/free/en/papers/pang-2002-thumbs-up-sentiment-classification-machine/). In EMNLP.
37. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In NAACL.
38. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). JMLR, 21:1–67.
39. Shruti Rijhwani and Daniel Preoţiuc-Pietro. 2020. Temporally-informed analysis of named entity recognition. In ACL. [find](https://scholar.google.com/scholar?q=Shruti+Rijhwani+and+Daniel+Preot%CC%A7iuc%2DPietro%2E+2020%2E+Temporally%2Dinformed+analysis+of+named+entity+recognition%2E+In+ACL%2E)
40. Paul Röttger and Janet Pierrehumbert. 2021. Temporal adaptation of bert and performance on downstream document classification: Insights from social media. In Findings of EMNLP, pages 2400–2412. [find](https://scholar.google.com/scholar?q=Paul+R%C3%B6ttger+and+Janet+Pierrehumbert%2E+2021%2E+Temporal+adaptation+of+bert+and+performance+on+downstream+document+classification%3A+Insights+from+social+media%2E+In+Findings+of+EMNLP%2C+pages+2400%E2%80%932412%2E)
41. Maja R. Rudolph and David M. Blei. 2018. Dynamic embeddings for language evolution. WWW. [find](https://scholar.google.com/scholar?q=Maja+R%2E+Rudolph+and+David+M%2E+Blei%2E+2018%2E+Dynamic+embeddings+for+language+evolution%2E+WWW%2E)
42. Tian Shi, Yaser Keneshloo, Naren Ramakrishnan, and Chandan K. Reddy. 2021. Neural abstractive text summarization with sequence-to-sequence models. ACM Transactions on Data Science. [find](https://scholar.google.com/scholar?q=Tian+Shi%2C+Yaser+Keneshloo%2C+Naren+Ramakrishnan%2C+and+Chandan+K%2E+Reddy%2E+2021%2E+Neural+abstractive+text+summarization+with+sequence%2Dto%2Dsequence+models%2E+ACM+Transactions+on+Data+Science%2E)
43. Tian Shi, Ping Wang, and Chandan K. Reddy. 2019. LeafNATS: An open-source toolkit and live demo system for neural abstractive text summarization. In NAACL. [find](https://scholar.google.com/scholar?q=Tian+Shi%2C+Ping+Wang%2C+and+Chandan+K%2E+Reddy%2E+2019%2E+LeafNATS%3A+An+open%2Dsource+toolkit+and+live+demo+system+for+neural+abstractive+text+summarization%2E+In+NAACL%2E)
44. Hidetoshi Shimodaira. 2000. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference. [find](https://scholar.google.com/scholar?q=Hidetoshi+Shimodaira%2E+2000%2E+Improving+predictive+inference+under+covariate+shift+by+weighting+the+log%2Dlikelihood+function%2E+Journal+of+Statistical+Planning+and+Inference%2E)
45. Ian Stewart and Jacob Eisenstein. 2018. Making “fetch” happen: The influence of social and linguistic context on nonstandard word growth and decline. In EMNLP. [find](https://scholar.google.com/scholar?q=Ian+Stewart+and+Jacob+Eisenstein%2E+2018%2E+Making+%E2%80%9Cfetch%E2%80%9D+happen%3A+The+influence+of+social+and+linguistic+context+on+nonstandard+word+growth+and+decline%2E+In+EMNLP%2E)
46. Shivashankar Subramanian, Daniel King, Doug Downey, and Sergey Feldman. 2021. S2and: A benchmark and evaluation system for author name disambiguation. In ACM/IEEE Joint Conference on Digital Libraries (JCDL), pages 170–179. IEEE. [find](https://scholar.google.com/scholar?q=Shivashankar+Subramanian%2C+Daniel+King%2C+Doug+Downey%2C+and+Sergey+Feldman%2E+2021%2E+S2and%3A+A+benchmark+and+evaluation+system+for+author+name+disambiguation%2E+In+ACM%2FIEEE+Joint+Conference+on+Digital+Libraries)
47. Nadine Tamburrini, Marco Cinnirella, Vincent AA Jansen, and John Bryden. 2015. Twitter users change word usage according to conversation-partner social identity. Social Networks, 40:84–89. [find](https://scholar.google.com/scholar?q=Nadine+Tamburrini%2C+Marco+Cinnirella%2C+Vincent+AA+Jansen%2C+and+John+Bryden%2E+2015%2E+Twitter+users+change+word+usage+according+to+conversation%2Dpartner+social+identity%2E+Social+Networks%2C+40%3A84%E2%80%9389%2E)
48. David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. [Fact or fiction: Verifying scientific claims](https://md.research.elamur.ai/free/en/papers/wadden-2020-fact-or-fiction-verifying-scientific/). In EMNLP.
49. Yunli Wang and Cyril Goutte. 2017. Detecting changes in twitter streams using temporal clusters of hashtags. In Proceedings of the Events and Stories in the News Workshop. [find](https://scholar.google.com/scholar?q=Yunli+Wang+and+Cyril+Goutte%2E+2017%2E+Detecting+changes+in+twitter+streams+using+temporal+clusters+of+hashtags%2E+In+Proceedings+of+the+Events+and+Stories+in+the+News+Workshop%2E)
50. Geoffrey I. Webb, Loong Kuan Lee, Bart Goethals, and François Petitjean. 2018. Analyzing concept drift and shift from sample data. Data Mining and Knowledge Discovery. [find](https://scholar.google.com/scholar?q=Geoffrey+I%2E+Webb%2C+Loong+Kuan+Lee%2C+Bart+Goethals%2C+and+Fran%C3%A7ois+Petitjean%2E+2018%2E+Analyzing+concept+drift+and+shift+from+sample+data%2E+Data+Mining+and+Knowledge+Discovery%2E)
51. Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, and Dan Klein. 2021. [Detoxifying language models risks marginalizing minority voices](https://md.research.elamur.ai/free/en/papers/xu-2021-detoxifying-language-models-risks-marginalizing/). In NAACL.
52. Yi Yang and Jacob Eisenstein. 2016. [Part-of-speech tagging for historical english](https://md.research.elamur.ai/free/en/papers/yang-2016-part-speech-tagging-historical-english/). In NAACL.
53. Justine Zhang, Jonathan Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, and Nithum Thain. 2018. [Conversations gone awry: Detecting early signs of conversational failure](https://md.research.elamur.ai/free/en/papers/zhang-2018-conversations-gone-awry-detecting-early/). In ACL.
54. Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. 2013. Domain adaptation under target and conditional shift. In ICML. [find](https://scholar.google.com/scholar?q=Kun+Zhang%2C+Bernhard+Sch%C3%B6lkopf%2C+Krikamol+Muandet%2C+and+Zhikun+Wang%2E+2013%2E+Domain+adaptation+under+target+and+conditional+shift%2E+In+ICML%2E)
55. Michael J.Q. Zhang and Eunsol Choi. 2021. SituatedQA: Incorporating extra-linguistic contexts into QA. EMNLP. [find](https://scholar.google.com/scholar?q=Michael+J%2EQ%2E+Zhang+and+Eunsol+Choi%2E+2021%2E+SituatedQA%3A+Incorporating+extra%2Dlinguistic+contexts+into+QA%2E+EMNLP%2E)
56. Yi Zhang, Zachary Ives, and Dan Roth. 2020. “who said it, and why?” provenance for natural language claims. In ACL. [find](https://scholar.google.com/scholar?q=Yi+Zhang%2C+Zachary+Ives%2C+and+Dan+Roth%2E+2020%2E+%E2%80%9Cwho+said+it%2C+and+why%3F%E2%80%9D+provenance+for+natural+language+claims%2E+In+ACL%2E)
57. Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Yejin Choi, and Noah A. Smith. 2021. [Challenges in automated debiasing for toxic language detection](https://md.research.elamur.ai/free/en/papers/zhou-2021-challenges-automated-debiasing-toxic-language/). In EACL.

*Updated 24 October 2026*