---
title: "Lexical Normalisation of Short Text Messages — Han 2011"
description: "The paper addresses lexical normalisation of noisy English short messages: mapping out-of-vocabulary (OOV) tokens such as typos, phonetic spellings (b4…"
authors: ["Bo Han", "Timothy Baldwin"]
year: 2011
venue: "Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies"
source: "https://aclanthology.org/P11-1038/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/han-2011-lexical-normalisation-short-text-messages/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/han-2011-lexical-normalisation-short-text-messages/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2011](https://md.research.elamur.ai/free/en/papers/2011/) / Lexical Normalisation of Short Text Messages: Makn Sens a #twitter

# Lexical Normalisation of Short Text Messages: Makn Sens a #twitter

**Authors:** [Bo Han](https://md.research.elamur.ai/free/en/authors/bo-han/), [Timothy Baldwin](https://md.research.elamur.ai/free/en/authors/timothy-baldwin/) · **Year:** 2011 · **Venue:** Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P11-1038/) · [PDF](https://aclanthology.org/P11-1038.pdf) · [OpenAlex](https://openalex.org/W2146867136)

**Citations:** 441 citations (openalex-snapshot · 2026-10-14) · [145](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper addresses lexical normalisation of noisy English short messages: mapping out-of-vocabulary (OOV) tokens such as typos, phonetic spellings (b4, 2morrow) and lengthened words (goooood) to a single standard word, so that downstream tools like parsers work better. A pilot study shows Twitter has many more OOV tokens per message than edited news, and the non-standard forms vary widely. Because of this long tail, supervised learning would suffer from sparse data. The authors therefore propose a cascade that needs no annotated training data. A classifier first decides whether an OOV token is ill-formed. Candidate corrections are then generated from morphological and phonetic similarity. Finally, candidates are ranked using word-similarity and context features, with a slang dictionary consulted first. The authors also build a new Twitter normalisation dataset. Evaluated on an SMS corpus and on this Twitter set, the full combination beats a noisy channel baseline and an SMT baseline in both F-score and BLEU. This shows that context-aware, annotation-free normalisation can be practical for social media text.

## Key points
- OOV status is decided with the GNU aspell dictionary (v0.60.6). Slang lookup uses a 5021-entry list gathered from noslang.com. Mentions, hashtags and URLs are not normalised but are kept as context.
- In the pilot study, about 15% of tweets have half or more of their tokens OOV. NYT text from English Gigaword shows a much more Zipfian pattern.
- The authors manually analysed 449 tweets and found 254 tokens needing normalisation:
  - simple letter insertion/omission accounts for 72.44%;
  - slang accounts for 12.20%;
  - number substitution accounts for 2.76%;
  - mixed letter-and-number forms account for 2.36%.
- The best configuration, DL+WS+CS, reaches F-score 0.755 and BLEU 0.876 on SMS, and F-score 0.753 and BLEU 0.934 on Twitter. By comparison, Cook and Stevenson's (2009) noisy-channel approach achieves roughly 0.46 F on each dataset.
- Dictionary lookup alone has the highest precision (0.961 on Twitter) but weak recall (0.460). Context support alone is weak, yet adding it to word similarity gives a statistically significant gain (p \< 0.05). Only 32.6% of ill-formed words had context windows made up entirely of IV words.

## Interesting ideas
- **Narrow, single-token task definition.** The task only covers OOV tokens with one-word targets, so multi-word expansions like imo are out of scope. This keeps the problem tractable and evaluation clean, and the same scoping approach could fit normalisation for other languages, such as Ukrainian social media.
- **Profiling OOV density per message.** Binning messages by their share of OOV tokens and comparing the distributions across genres is a cheap diagnostic. It shows how noisy a domain is and whether surrounding context can be trusted before you design a model.
- **High-precision lookup first, then a general model.** A slang dictionary handles frequent shorthands, and the similarity-plus-context ranker handles everything else. This is a good pattern wherever a small curated lexicon exists but cannot cover the long tail.
- **Token-level rather than type-level decisions.** The paper notes that the same form can need different corrections in different contexts, for example hw meaning "how" in one message and "homework" in another. Context-sensitive, per-occurrence normalisation therefore beats a fixed mapping table.
- **Keep non-normalisable tokens as context.** Mentions, hashtags and URLs are left in place rather than deleted, so they can still inform context modelling. This is useful in any pipeline that would otherwise strip such tokens and lose signal.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cited by

Cited by

145 papers of our library cite it (of the 140884 whose bibliographies we read); 13 of them have a page here, 132 not yet.

- [Semi-supervised Contextual Historical Text Normalization](https://md.research.elamur.ai/free/en/papers/acl-2020-makarov-semi-supervised-contextual-historical-text/)
- [How Noisy Social Media Text, How Diffrnt Social Media Sources?](https://md.research.elamur.ai/free/en/papers/baldwin-2013-how-noisy-social-media-text/)
- [Shared Tasks of the 2015 Workshop on Noisy User-generated Text: Twitter Lexical Normalization and Named Entity Recognition](https://md.research.elamur.ai/free/en/papers/baldwin-2015-shared-tasks-2015-workshop-noisy/)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://md.research.elamur.ai/free/en/papers/barbieri-2020-tweeteval-unified-benchmark-comparative-evaluation/)
- [Synthetic Data for English Lexical Normalization: How Close Can We Get to Manually Annotated Data?](https://md.research.elamur.ai/free/en/papers/dekker-2020-synthetic-data-english-lexical-normalization/)
- [What to do about bad language on the internet](https://md.research.elamur.ai/free/en/papers/eisenstein-2013-what-do-about-bad-language/)
- [Learning a POS tagger for AAVE-like language](https://md.research.elamur.ai/free/en/papers/jorgensen-2016-learning-pos-tagger-aave-like/)
- [Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words](https://md.research.elamur.ai/free/en/papers/lex-2021-hofmann-superbizarre-derivational/)
- [The Denoised Web Treebank: Evaluating Dependency Parsing under Noisy Input Conditions](https://md.research.elamur.ai/free/en/papers/lingbulk-2016-daiber-denoised-web-treebank-evaluating-dependency/)
- [Variation Is the Norm: Embracing Sociolinguistics in NLP](https://md.research.elamur.ai/free/en/papers/lutgen-2026-variation-is-norm-embracing-sociolinguistics/)
- [Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters](https://md.research.elamur.ai/free/en/papers/owoputi-2013-improved-part-speech-tagging-online/)
- [Named Entity Recognition in Tweets: An Experimental Study](https://md.research.elamur.ai/free/en/papers/ritter-2011-named-entity-recognition-tweets-experimental/)
- [From Shakespeare to Twitter: What are Language Styles all about?](https://md.research.elamur.ai/free/en/papers/xu-2017-shakespeare-twitter-what-are-language/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{han2011lexical,
  title = {{Lexical Normalisation of Short Text Messages: Makn Sens a \#twitter}},
  author = {Han, Bo and Baldwin, Timothy},
  booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
  year = {2011},
  url = {https://aclanthology.org/P11-1038/}
}
```

- [← previous Lexical Generalization in CCG Grammar Induction for Semantic Parsing](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2011-lexical-generalization-ccg-grammar-induction/)
- [next → Lexicon-Based Methods for Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/taboada-2011-lexicon-based-methods-sentiment-analysis/)
- [↑ back to papers of 2011](https://md.research.elamur.ai/free/en/papers/2011/)

## References

30 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. AiTi Aw, Min Zhang, Juan Xiao, and Jian Su. 2006. A phrase-based statistical model for SMS text normalization. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 33–40, Sydney, Australia. [find](https://scholar.google.com/scholar?q=AiTi+Aw%2C+Min+Zhang%2C+Juan+Xiao%2C+and+Jian+Su%2E+2006%2E+A+phrase%2Dbased+statistical+model+for+SMS+text+normalization%2E+In+Proceedings+of+the+21st+International+Conference+on+Computational+Linguistics+and+44th)
2. Timothy Baldwin and Marco Lui. 2010. Language identification: The long and the short of the matter. In HLT ’10: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 229–237, Los Angeles, USA. [find](https://scholar.google.com/scholar?q=Timothy+Baldwin+and+Marco+Lui%2E+2010%2E+Language+identification%3A+The+long+and+the+short+of+the+matter%2E+In+HLT+%E2%80%9910%3A+Human+Language+Technologies%3A+The+2010+Annual+Conference+of+the+North+American+Chapter+of)
3. Richard Beaufort, Sophie Roekhaut, Louise-Amélie Cougnon, and Cédrick Fairon. 2010. A hybrid rule/model-based finite-state framework for normalizing SMS messages. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 770–779, Uppsala, Sweden. [find](https://scholar.google.com/scholar?q=Richard+Beaufort%2C+Sophie+Roekhaut%2C+Louise%2DAme%CC%81lie+Cougnon%2C+and+Ce%CC%81drick+Fairon%2E+2010%2E+A+hybrid+rule%2Fmodel%2Dbased+finite%2Dstate+framework+for+normalizing+SMS+messages%2E+In+Proceedings+of+the+48th+Annual+M)
4. Eric Brill and Robert C. Moore. 2000. An improved error model for noisy channel spelling correction. In ACL ’00: Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, pages 286–293, Hong Kong. [find](https://scholar.google.com/scholar?q=Eric+Brill+and+Robert+C%2E+Moore%2E+2000%2E+An+improved+error+model+for+noisy+channel+spelling+correction%2E+In+ACL+%E2%80%9900%3A+Proceedings+of+the+38th+Annual+Meeting+on+Association+for+Computational+Linguistics%2C+pa) [our card](https://md.research.elamur.ai/free/en/papers/brill-2000-improved-error-model-noisy-channel/)
5. Kevin Burton, Akshay Java, and Ian Soboroff. 2009. [find](https://scholar.google.com/scholar?q=Kevin+Burton%2C+Akshay+Java%2C+and+Ian+Soboroff%2E+2009%2E)
6. The ICWSM 2009 Spinn3r Dataset. In Proceedings of the Third Annual Conference on Weblogs and Social Media, San Jose, USA. [find](https://scholar.google.com/scholar?q=The+ICWSM+2009+Spinn3r+Dataset%2E+In+Proceedings+of+the+Third+Annual+Conference+on+Weblogs+and+Social+Media%2C+San+Jose%2C+USA%2E)
7. Monojit Choudhury, Rahul Saraf, Vijit Jain, Animesh Mukherjee, Sudeshna Sarkar, and Anupam Basu. 2007. Investigation and modeling of the structure of texting language. International Journal on Document Analysis and Recognition, 10:157–174. [find](https://scholar.google.com/scholar?q=Monojit+Choudhury%2C+Rahul+Saraf%2C+Vijit+Jain%2C+Animesh+Mukherjee%2C+Sudeshna+Sarkar%2C+and+Anupam+Basu%2E+2007%2E+Investigation+and+modeling+of+the+structure+of+texting+language%2E+International+Journal+on+Documen)
8. Paul Cook and Suzanne Stevenson. 2009. An unsupervised model for text message normalization. In CALC ’09: Proceedings of the Workshop on Computational Approaches to Linguistic Creativity, pages 71– 78, Boulder, USA. [find](https://scholar.google.com/scholar?q=Paul+Cook+and+Suzanne+Stevenson%2E+2009%2E+An+unsupervised+model+for+text+message+normalization%2E+In+CALC+%E2%80%9909%3A+Proceedings+of+the+Workshop+on+Computational+Approaches+to+Linguistic+Creativity%2C+pages+71%E2%80%93+78)
9. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC 2006), Genoa, Italy. [find](https://scholar.google.com/scholar?q=Marie%2DCatherine+de+Marneffe%2C+Bill+MacCartney%2C+and+Christopher+D%2E+Manning%2E+2006%2E+Generating+typed+dependency+parses+from+phrase+structure+parses%2E+In+Proceedings+of+the+5th+International+Conference+on+L) [our card](https://md.research.elamur.ai/free/en/papers/map-2006-marneffe-generating-typed-dependency-parses-phrase/)
10. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 9:1871–1874. [find](https://scholar.google.com/scholar?q=Rong%2DEn+Fan%2C+Kai%2DWei+Chang%2C+Cho%2DJui+Hsieh%2C+Xiang%2DRui+Wang%2C+and+Chih%2DJen+Lin%2E+2008%2E+LIBLINEAR%3A+A+library+for+large+linear+classification%2E+Journal+of+Machine+Learning+Research%2C+9%3A1871%E2%80%931874%2E)
11. Graeme Hirst and Alexander Budanitsky. 2005. Correcting real-word spelling errors by restoring lexical cohesion. Natural Language Engineering, 11:87–111. [find](https://scholar.google.com/scholar?q=Graeme+Hirst+and+Alexander+Budanitsky%2E+2005%2E+Correcting+real%2Dword+spelling+errors+by+restoring+lexical+cohesion%2E+Natural+Language+Engineering%2C+11%3A87%E2%80%93111%2E)
12. Yijue How and Min-Yen Kan. 2005. Optimizing predictive text entry for short message service on mobile phones. In Human Computer Interfaces International (HCII 05), Las Vegas, USA. [find](https://scholar.google.com/scholar?q=Yijue+How+and+Min%2DYen+Kan%2E+2005%2E+Optimizing+predictive+text+entry+for+short+message+service+on+mobile+phones%2E+In+Human+Computer+Interfaces+International+%28HCII+05%29%2C+Las+Vegas%2C+USA%2E)
13. Emi Izumi, Kiyotaka Uchimoto, Toyomi Saiga, Thepchai Supnithi, and Hitoshi Isahara. 2003. Automatic error detection in the Japanese learners’ English spoken data. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 2, pages 145–148, Sapporo, Japan. [find](https://scholar.google.com/scholar?q=Emi+Izumi%2C+Kiyotaka+Uchimoto%2C+Toyomi+Saiga%2C+Thepchai+Supnithi%2C+and+Hitoshi+Isahara%2E+2003%2E+Automatic+error+detection+in+the+Japanese+learners%E2%80%99+English+spoken+data%2E+In+Proceedings+of+the+41st+Annual+Mee)
14. Joseph Kaufmann and Jugal Kalita. 2010. Syntactic normalization of Twitter messages. In International Conference on Natural Language Processing, Kharagpur, India. [find](https://scholar.google.com/scholar?q=Joseph+Kaufmann+and+Jugal+Kalita%2E+2010%2E+Syntactic+normalization+of+Twitter+messages%2E+In+International+Conference+on+Natural+Language+Processing%2C+Kharagpur%2C+India%2E)
15. Dan Klein and Christopher D. Manning. 2003. Fast exact inference with a factored model for natural language parsing. In Advances in Neural Information Processing Systems 15 (NIPS 2002), pages 3–10, Whistler, Canada. [find](https://scholar.google.com/scholar?q=Dan+Klein+and+Christopher+D%2E+Manning%2E+2003%2E+Fast+exact+inference+with+a+factored+model+for+natural+language+parsing%2E+In+Advances+in+Neural+Information+Processing+Systems+15+%28NIPS+2002%29%2C+pages+3%E2%80%9310%2C+Wh)
16. Catherine Kobus, Franois Yvon, and Graldine Damnati. 2008. Transcrire les SMS comme on reconnat la parole. In Actes de la Confrence sur le Traitement Automatique des Langues (TALN’08), pages 128–138. [find](https://scholar.google.com/scholar?q=Catherine+Kobus%2C+Franois+Yvon%2C+and+Graldine+Damnati%2E+2008%2E+Transcrire+les+SMS+comme+on+reconnat+la+parole%2E+In+Actes+de+la+Confrence+sur+le+Traitement+Automatique+des+Langues+%28TALN%E2%80%9908%29%2C+pages+128%E2%80%93138%2E)
17. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: open source toolkit for statistical machine translation. In Pro… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondr%CC%8Cej+Bojar%2C+Alexandra+Const)
18. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 311–318, Philadelphia, USA. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computat) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
19. James L. Peterson. 1980. Computer programs for detecting and correcting spelling errors. Commun. ACM, 23:676–687, December. [find](https://scholar.google.com/scholar?q=James+L%2E+Peterson%2E+1980%2E+Computer+programs+for+detecting+and+correcting+spelling+errors%2E+Commun%2E+ACM%2C+23%3A676%E2%80%93687%2C+December%2E)
20. Lawrence Philips. 2000. The double metaphone search algorithm. C/C++ Users Journal, 18:38–43. [find](https://scholar.google.com/scholar?q=Lawrence+Philips%2E+2000%2E+The+double+metaphone+search+algorithm%2E+C%2FC%2B%2B+Users+Journal%2C+18%3A38%E2%80%9343%2E)
21. Lawrence R. Rabiner. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257–286. [find](https://scholar.google.com/scholar?q=Lawrence+R%2E+Rabiner%2E+1989%2E+A+tutorial+on+hidden+Markov+models+and+selected+applications+in+speech+recognition%2E+Proceedings+of+the+IEEE%2C+77%282%29%3A257%E2%80%93286%2E)
22. Alan Ritter, Colin Cherry, and Bill Dolan. 2010. Unsupervised modeling of Twitter conversations. In HLT ’10: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 172–180, Los Angeles, USA. [find](https://scholar.google.com/scholar?q=Alan+Ritter%2C+Colin+Cherry%2C+and+Bill+Dolan%2E+2010%2E+Unsupervised+modeling+of+Twitter+conversations%2E+In+HLT+%E2%80%9910%3A+Human+Language+Technologies%3A+The+2010+Annual+Conference+of+the+North+American+Chapter+of+th) [our card](https://md.research.elamur.ai/free/en/papers/ritter-2010-unsupervised-modeling-twitter-conversations/)
23. Claude Elwood Shannon. 1948. A mathematical theory of communication. Bell System Technical Journal, 27:379–423, 623–656. [find](https://scholar.google.com/scholar?q=Claude+Elwood+Shannon%2E+1948%2E+A+mathematical+theory+of+communication%2E+Bell+System+Technical+Journal%2C+27%3A379%E2%80%93423%2C+623%E2%80%93656%2E)
24. Richard Sproat, Alan W. Black, Stanley Chen, Shankar Kumar, Mari Ostendorf, and Christopher Richards. 2001. Normalization of non-standard words. Computer Speech and Language, 15(3):287 – 333. [find](https://scholar.google.com/scholar?q=Richard+Sproat%2C+Alan+W%2E+Black%2C+Stanley+Chen%2C+Shankar+Kumar%2C+Mari+Ostendorf%2C+and+Christopher+Richards%2E+2001%2E+Normalization+of+non%2Dstandard+words%2E+Computer+Speech+and+Language%2C+15%283%29%3A287+%E2%80%93+333%2E)
25. Andreas Stolcke. 2002. Srilm - an extensible language modeling toolkit. In International Conference on Spoken Language Processing, pages 901–904, Denver, USA. [find](https://scholar.google.com/scholar?q=Andreas+Stolcke%2E+2002%2E+Srilm+%2D+an+extensible+language+modeling+toolkit%2E+In+International+Conference+on+Spoken+Language+Processing%2C+pages+901%E2%80%93904%2C+Denver%2C+USA%2E)
26. Guihua Sun, Gao Cong, Xiaohua Liu, Chin-Yew Lin, and Ming Zhou. 2007. Mining sequential patterns and tree patterns to detect erroneous sentences. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 81–88, Prague, Czech Republic. [find](https://scholar.google.com/scholar?q=Guihua+Sun%2C+Gao+Cong%2C+Xiaohua+Liu%2C+Chin%2DYew+Lin%2C+and+Ming+Zhou%2E+2007%2E+Mining+sequential+patterns+and+tree+patterns+to+detect+erroneous+sentences%2E+In+Proceedings+of+the+45th+Annual+Meeting+of+the+Assoc)
27. Kristina Toutanova and Robert C. Moore. 2002. Pronunciation modeling for improved spelling correction. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 144–151, Philadelphia, USA. [find](https://scholar.google.com/scholar?q=Kristina+Toutanova+and+Robert+C%2E+Moore%2E+2002%2E+Pronunciation+modeling+for+improved+spelling+correction%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computational+Linguistics%2C+ACL+%E2%80%9902%2C+)
28. Twitter. 2010. Big goals, big game, big records. [http://blog.twitter.com/2010/06/big-goals-big-game-big-records.html](http://blog.twitter.com/2010/06/big-goals-big-game-big-records.html). Retrieved 4 August 2010. [↗](http://blog.twitter.com/2010/06/big-goals-big-game-big-records.html)
29. Wilson Wong, Wei Liu, and Mohammed Bennamoun. 2006. Integrated scoring for spelling error correction, abbreviation expansion and case restoration in dirty text. In Proceedings of the Fifth Australasian Conference on Data Mining and Analytics, pages 83–89, Sydney, Australia. [find](https://scholar.google.com/scholar?q=Wilson+Wong%2C+Wei+Liu%2C+and+Mohammed+Bennamoun%2E+2006%2E+Integrated+scoring+for+spelling+error+correction%2C+abbreviation+expansion+and+case+restoration+in+dirty+text%2E+In+Proceedings+of+the+Fifth+Australasia)
30. Alexander Yeh. 2000. More accurate tests for the statistical significance of result differences. In Proceedings of the 18th Conference on Computational Linguistics - Volume 2, COLING ’00, pages 947–953, Saarbrücken, Germany. [find](https://scholar.google.com/scholar?q=Alexander+Yeh%2E+2000%2E+More+accurate+tests+for+the+statistical+significance+of+result+differences%2E+In+Proceedings+of+the+18th+Conference+on+Computational+Linguistics+%2D+Volume+2%2C+COLING+%E2%80%9900%2C+pages+947%E2%80%9395)