---
title: "Learning to Explain Non-Standard English Words and Phrases"
description: "Its key design is a dual encoder: a word-level LSTM reads the surrounding sentence; a… 20 references, cited by 38 papers here."
authors: ["Ke Ni", "William Yang Wang"]
year: 2017
venue: "Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)"
source: "https://aclanthology.org/I17-2070/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ni-2017-learning-explain-non-standard-english/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ni-2017-learning-explain-non-standard-english/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/4/) / Learning to Explain Non-Standard English Words and Phrases

# Learning to Explain Non-Standard English Words and Phrases

**Authors:** Ke Ni and [William Yang Wang](https://md.research.elamur.ai/free/en/authors/william-yang-wang/) · **Year:** 2017 · **Venue:** [Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)](https://md.research.elamur.ai/free/en/venues/ijcnlp/) ([IJCNLP 2017](https://md.research.elamur.ai/free/en/venues/ijcnlp/2017/)) · **Publisher:** [Asian Federation of Natural Language Processing](https://md.research.elamur.ai/free/en/publishers/asian-federation-of-natural-language-processing/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/I17-2070/) · [PDF](https://aclanthology.org/I17-2070.pdf) · [OpenAlex](https://openalex.org/W2756725603)

**Citations:** 29 citations (OpenAlex, 2026-10-14) · [38](https://md.research.elamur.ai/free/en/papers/ni-2017-learning-explain-non-standard-english/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses how to automatically explain slang and other non-standard English expressions that a fixed slang dictionary has not yet recorded. Earlier systems looked words up in small lists of about 5K terms, so coverage was poor and the lists went stale quickly. The authors instead treat explanation as generation. They built a corpus from Urban Dictionary (UD; here this means the slang website, not Universal Dependencies): 421K crowdsourced entries from 1999–2014, each with definitions and usage examples. On this corpus they trained an attentive LSTM sequence-to-sequence model that writes a definition for a marked expression in a sentence. Its key design is a dual encoder:
- a word-level LSTM reads the surrounding sentence;
- a character-level LSTM reads only the target expression;
- the two hidden states are combined linearly before decoding.

Test expressions never appear in training. On this split, the large dual encoder reached the best BLEU, ahead of a single word-level encoder and a fully character-level model. This shows a learned, open-vocabulary slang explainer is feasible, and the dataset was released.

## Key points
- **Corpus:** 421K Urban Dictionary entries (1999–2014), released publicly. Training uses 371,028 entries (907,624 example–definition pairs). Testing uses 50,000 entries (61,330 sentences). Training and test share no target expressions.
- **Architecture:**
  - word-level context encoder plus character-level target encoder, merged as h₁W₁ + h₂W₂ + B;
  - LSTM decoder with attention following Vinyals et al. (2015);
  - implemented in Tensorflow, trained with SGD on cross-entropy.
- **BLEU-1 / BLEU-2 results:**

  | Model | Hidden units | BLEU-1 | BLEU-2 |
  |---|---|---|---|
  | single encoder | 1024 | 21.06 | 2.1 |
  | small dual encoder | 512 | 21.84 | 2.2 |
  | large dual encoder | 1024 | **24.58** | **2.37** |
  | full char-level model | 256 | 21.13 | 1.8 |

  The char-level model was capped at 256 units by GPU memory.
- **Other findings:** LSTM cells beat GRUs, and attention helped.
- **Qualitative examples:** blends such as "loltastic" and "adoucheous" got more sensible glosses from the dual model. The authors attribute this partly to the model knowing which span it must explain.

## Interesting ideas
- Encoding the span to be explained separately (characters) from its sentence context (words) tells the model which of several odd tokens is the target, and the character view handles blends and misspellings.
- A crowdsourced dictionary whose entries carry example sentences doubles as a parallel corpus of (usage in context → definition) pairs, with vote-based moderation as a light quality filter.
- Splitting train/test by dictionary entry rather than by sentence guarantees the system is judged on genuinely unseen expressions, not memorised ones.
- The very low BLEU-2 (about 2) next to BLEU-1 (about 21–25) suggests n-gram overlap is a weak judge of free-text definitions. The authors therefore backed their numbers with inspected examples.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): an unknown-token module could pair a character-level view of the OOV form with its sentence context to propose a lemma, POS and standard-English counterpart, rather than guessing from context alone.
- **I2** (world): Urban Dictionary entries could feed a flagged slang lexicon in the compiled knowledge layer, kept as low-trust facts that the reader cites when it uses them.
- **I3** (learn): evaluations of lexical components should hold out whole lexemes, so that apparent gains on new words cannot come from leakage of seen forms.
- **I4** (answer): when Mova explains a word, grading should check meaning, for example by parsing the gloss and comparing its frames to the reference, rather than relying on surface overlap.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Ke Ni and William Yang Wang. 2017. Learning to Explain Non-Standard English Words and Phrases. Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ni2017learning,
  title = {{Learning to Explain Non-Standard English Words and Phrases}},
  author = {Ke Ni and William Yang Wang},
  booktitle = {Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
  year = {2017},
  url = {https://aclanthology.org/I17-2070/}
}
```

- [← previous A Characterization Study of Arabic Twitter Data with a Benchmarking for State-of-the-Art Opinion Mining Models](https://md.research.elamur.ai/free/en/papers/baly-2017-characterization-study-arabic-twitter-data/)
- [next → Macro Grammars and Holistic Triggering for Efficient Semantic Parsing](https://md.research.elamur.ai/free/en/papers/zhang-2017-macro-grammars-holistic-triggering-efficient/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/4/)

## References

20 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, pages 311–318.
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). arXiv preprint arXiv:1409.0473 . [↗](https://arxiv.org/abs/1409.0473)
3. Clint Burfoot and Timothy Baldwin. 2009. Automatic satire detection: Are you having a laugh? In Proceedings of the ACL-IJCNLP 2009 conference short papers. Association for Computational Linguistics, pages 161–164. [find](https://scholar.google.com/scholar?q=Clint+Burfoot+and+Timothy+Baldwin%2E+2009%2E+Automatic+satire+detection%3A+Are+you+having+a+laugh%3F+In+Proceedings+of+the+ACL%2DIJCNLP+2009+conference+short+papers%2E+Association+for+Computational+Linguistics%2C+p)
4. Sara Rosenthal and Kathleen McKeown. 2011. Age prediction in blogs: A study of style, content, and online behavior in pre-and post-social media generations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, pages 763–772. [↗](https://doi.org/10.7916/d8w384nb)
5. Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. [On the properties of neural machine translation: Encoder–decoder approaches](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/). Syntax, Semantics and Structure in Statistical Translation page 103.
6. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. [Sequence to sequence learning with neural networks](https://md.research.elamur.ai/free/en/papers/sutskever-2014-sequence-sequence-learning-neural-networks/). In Advances in neural information processing systems. pages 3104–3112. [↗](https://doi.org/10.48550/arxiv.1409.3215)
7. Johnny Davis. 2011. In praise of urban dictionaries. The Guardian. [find](https://scholar.google.com/scholar?q=Johnny+Davis%2E+2011%2E+In+praise+of+urban+dictionaries%2E+The+Guardian%2E)
8. Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Advances in Neural Information Processing Systems. pages 2773–2781. [↗](https://doi.org/10.48550/arxiv.1412.7449)
9. Jacob Eisenstein, Brendan O’Connor, Noah A Smith, and Eric P Xing. 2010. [A latent variable model for geographic lexical variation](https://md.research.elamur.ai/free/en/papers/eisenstein-2010-latent-variable-model-geographic-lexical/). In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 1277–1287.
10. William Yang Wang and Kathleen McKeown. 2010. ”got you!”: Automatic vandalism detection in wikipedia with web-based shallow syntactic-semantic modeling. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010). [↗](https://doi.org/10.7916/d8kp89j4)
11. Stephan Gouws, Donald Metzler, Congxing Cai, and Eduard Hovy. 2011. Contextual bearing on linguistic variation in social media. In Proceedings of the Workshop on Languages in Social Media. Association for Computational Linguistics, pages 20–29. [find](https://scholar.google.com/scholar?q=Stephan+Gouws%2C+Donald+Metzler%2C+Congxing+Cai%2C+and+Eduard+Hovy%2E+2011%2E+Contextual+bearing+on+linguistic+variation+in+social+media%2E+In+Proceedings+of+the+Workshop+on+Languages+in+Social+Media%2E+Association)
12. Michele Zappavigna. 2012. Discourse of Twitter and social media: How we use language to create affiliation on the web. A&C Black. [find](https://scholar.google.com/scholar?q=Michele+Zappavigna%2E+2012%2E+Discourse+of+Twitter+and+social+media%3A+How+we+use+language+to+create+affiliation+on+the+web%2E+A%26C+Black%2E)
13. Bo Han and Timothy Baldwin. 2011. [Lexical normalisation of short text messages: Makn sens a# twitter](https://md.research.elamur.ai/free/en/papers/han-2011-lexical-normalisation-short-text-messages/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, pages 368–378.
14. Leo Hickman. 2013. Why IBM’s Watson supercomputer can’t speak slang. The Guardian. [find](https://scholar.google.com/scholar?q=Leo+Hickman%2E+2013%2E+Why+IBM%E2%80%99s+Watson+supercomputer+can%E2%80%99t+speak+slang%2E+The+Guardian%2E)
15. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735–1780. [↗](https://doi.org/10.1162/neco.1997.9.8.1735)
16. Joe Jackson. 2011. Feds Check Urban Dictionary to Crack Gun-Store Death Threat. TIME. [find](https://scholar.google.com/scholar?q=Joe+Jackson%2E+2011%2E+Feds+Check+Urban+Dictionary+to+Crack+Gun%2DStore+Death+Threat%2E+TIME%2E)
17. Leslie Kaufman. 2013. For the Word on the Street, Courts Call Up an Online Witness. The New York Times. [find](https://scholar.google.com/scholar?q=Leslie+Kaufman%2E+2013%2E+For+the+Word+on+the+Street%2C+Courts+Call+Up+an+Online+Witness%2E+The+New+York+Times%2E)
18. Leah Kolt and Matt Lazier. 2009. Alumni in the News: Summer & Fall 2009. Cal Poly Magazine. [find](https://scholar.google.com/scholar?q=Leah+Kolt+and+Matt+Lazier%2E+2009%2E+Alumni+in+the+News%3A+Summer+%26+Fall+2009%2E+Cal+Poly+Magazine%2E)
19. JoAnn Lloyd. 2011. Alum Aaron Peckham’s Urban Dictionary Redefines Language. Cal Poly Magazine. [find](https://scholar.google.com/scholar?q=JoAnn+Lloyd%2E+2011%2E+Alum+Aaron+Peckham%E2%80%99s+Urban+Dictionary+Redefines+Language%2E+Cal+Poly+Magazine%2E)
20. Thanapon Noraset, Liang Chen, Larry Birnbaum, and Doug Downey. 2016. Definition modeling: Learning to define word embeddings in natural language. In AAAI. pages 3259–3266. [↗](https://doi.org/10.1609/aaai.v31i1.10996)

*Updated 1 November 2026*