---
title: "The University of Texas System Submission for the"
description: "This paper describes the BATs group's entry to the named entity recognition shared task at CALCS 2018. 24 references, cited by 2 papers here."
authors: ["Florian Janke", "Tongrui Li", "Eric Rincón", "Gualberto Guzmán", "Barbara Bullock", "Almeida Jacqueline Toribio"]
year: 2018
venue: "Proceedings of the Third Workshop on Computational Approaches to Linguistic Code-Switching"
source: "https://aclanthology.org/W18-3216/"
doi: "10.18653/v1/w18-3216"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/janke-2018-university-texas-system-submission-code/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/janke-2018-university-texas-system-submission-code/)
[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/7/) / The University of Texas System Submission for the Code-Switching Workshop Shared Task 2018
# The University of Texas System Submission for the Code-Switching Workshop Shared Task 2018
**Authors:** Florian Janke, Tongrui Li, Eric Rincón, Gualberto Guzmán, [Barbara Bullock](https://md.research.elamur.ai/free/en/authors/barbara-bullock/) and [Almeida Jacqueline Toribio](https://md.research.elamur.ai/free/en/authors/almeida-jacqueline-toribio/) · **Year:** 2018 · **Venue:** [Proceedings of the Third Workshop on Computational Approaches to Linguistic Code-Switching](https://md.research.elamur.ai/free/en/venues/calcs/) ([Workshop on Computational Approaches to Linguistic Code-Switching 2018](https://md.research.elamur.ai/free/en/venues/calcs/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)
**Links:** [Publisher's page](https://aclanthology.org/W18-3216/) · [PDF](https://aclanthology.org/W18-3216.pdf) · [DOI 10.18653/v1/w18-3216](https://doi.org/10.18653/v1/w18-3216) · [OpenAlex](https://openalex.org/W2885028086)
**Citations:** 2 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/janke-2018-university-texas-system-submission-code/cited-by/) in our library, of the 149,439 papers whose bibliographies we read
## Summary
This paper describes the BATs group's entry to the named entity recognition shared task at CALCS 2018. The task was to find and classify names in code-switched Twitter posts. Mixed-language tweets are difficult for several reasons. A name may take different forms in each language, be shared across both, or match an ordinary noun in the other language. Tweets also often lack reliable spelling and capitalization. The authors used the roughly 62k annotated tweets, labelled with ten entity types in IOB format. On them they trained a conditional random field with hand-crafted features: word form and suffixes, capitalization, NLTK POS and NE tags, tweet-level language probabilities from langdetect, and Twitter cues such as URLs, mentions, hashtags and emoji, plus features of neighbouring words. The CRF reached weighted F-scores of 65.62 on MSA-EGY and 54.16 on ENG-SPA, beating the baseline. A TensorFlow wide-and-deep network built on fewer features did much worse, mainly because its recall was very low. Later ablations showed that a feature's value depends on the language pair, which is a useful caution for anyone designing features for several languages at once.
## Key points
- The data were about 62k tweets annotated for the CALCS 2018 NER task. Tagging followed the IOB scheme over ten classes: eight entity types (TIME, EVENT, PERSON, GROUP, TITLE, PRODUCT, LOCATION, ORGANIZATION) plus OTHER and NOT-NE, and @-mentions and hashtags always received the O label.
- The CRF was built with python-crfsuite (a wrapper over CRFsuite), using L1 = 1.0, L2 = 0.001 and 150 training iterations. Features fell into five groups: word, capitalization, NLTK tags, langdetect language probabilities (ENG-SPA only) and Twitter-specific cues.
- Official submission scores were 65.62 on MSA-EGY and 54.16 on ENG-SPA. Both pairs scored above the baseline, and MSA-EGY scored higher despite the difference in data size.
- Post-deadline ablations used scikit-learn F1 on the test set:
- ENG-SPA was best (F1 0.34) when POS/NE tags were dropped or an ASCII check was added.
- MSA-EGY was best (0.77) with a combined "special" configuration.
- The ASCII feature helped ENG-SPA but hurt MSA-EGY.
- The wide-and-deep model (DNNLinearCombinedClassifier) reached only about 0.03–0.07 F1 on ENG-SPA, and the wide-only variant beat the deep+wide one. Deep training was roughly 20 times slower per iteration, so the team discarded the network.
## Interesting ideas
- Running the same feature toggles on two language pairs showed features that help one dataset and hurt the other, so a single global feature set can mislead.
- Outputs of ready-made NLTK taggers (POS, NE) were added as input features to a sequence model, letting a weak external tool act as one signal among many.
- A whole-tweet language probability was attached to every token, falling back to zero when the detector guessed an unrelated language. This bounds the damage from detector errors.
- With few features, a plain linear model beat the deep network and trained about twenty times faster, so model capacity cannot make up for missing features.
- The annotation scheme decided in advance that @-mentions and hashtags are never entities, removing a whole class of ambiguous decisions.
## For Mova
How these ideas could serve Mova, our small open language model — its files, stages and tests:
- **I1** (learn): Mova's evaluation could report ablations separately per treebank or text genre, flagging repair rules or features whose sign flips between them.
- **I2** (tokens): the en tagger could take tags proposed by other taggers as extra perceptron features, with each one's weight visible for explanation.
- **I3** (tokens): document-level signals such as register or dialect could be passed to token features softly, with a neutral default when the detector is unsure.
- **I4** (tree): Mova should keep the perceptron parser as a cheap reference whenever gparse is retrained, to catch cases where the transformer underperforms for lack of useful input.
- **I5** (world): Mova could fix explicit rules for which token types are allowed to become world entities, so entity creation stays predictable and explainable.
*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*
## Institutions
- [The University of Texas at Austin](https://md.research.elamur.ai/free/en/universities/the-university-of-texas-at-austin/) — University, Austin, United States · 119 works · 57,950 citations
📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works
🧭 Topics (1): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 882 works
## Cite
Cite as: Florian Janke, Tongrui Li, Eric Rincón, Gualberto Guzmán, Barbara Bullock and Almeida Jacqueline Toribio. 2018. The University of Texas System Submission for the Code-Switching Workshop Shared Task 2018. Proceedings of the Third Workshop on Computational Approaches to Linguistic Code-Switching.
The paper itself, not this page. BibTeX from the library's record of the work.
```bibtex
@inproceedings{janke2018university,
title = {{The University of Texas System Submission for the Code-Switching Workshop Shared Task 2018}},
author = {Florian Janke, Tongrui Li, Eric Rincón, Gualberto Guzmán, Barbara Bullock and Almeida Jacqueline Toribio},
booktitle = {Proceedings of the Third Workshop on Computational Approaches to Linguistic Code-Switching},
year = {2018},
doi = {10.18653/v1/w18-3216},
url = {https://doi.org/10.18653/v1/w18-3216}
}
```
- [← previous The Evolution of Biblical Hebrew Linguistics in South Africa: The Last 60 Years](https://md.research.elamur.ai/free/en/papers/naude-2018-evolution-biblical-hebrew-linguistics-south/)
- [next → To accommodate or to ignore?: The presuppositions of again and continue across contexts](https://md.research.elamur.ai/free/en/papers/bacovcin-2018-accommodate-or-ignore-presuppositions-i/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/7/)
## References
24 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.
1. Anupam Jamatia, Björn Gambäck, and Amitava Das. 2015. Part-of-speech tagging for code-mixed english-hindi twitter and facebook chat messages. In Proceedings of the International Conference Recent Advances in Natural Language Processing, pages 239–248. [find](https://scholar.google.com/scholar?q=Anupam+Jamatia%2C+Bjo%CC%88rn+Gamba%CC%88ck%2C+and+Amitava+Das%2E+2015%2E+Part%2Dof%2Dspeech+tagging+for+code%2Dmixed+english%2Dhindi+twitter+and+facebook+chat+messages%2E+In+Proceedings+of+the+International+Conference+Recent+Ad)
2. Martı́n Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. [Tensorflow: A system for large-scale machine learning](https://md.research.elamur.ai/free/en/papers/abadi-2016-tensorflow-system-large-scale-machine/). In OSDI, volume 16, pages 265– 283.
3. Anupam Jamatia, Björn Gambäck, and Amitava Das. 2016. Collecting and annotating indian social media code-mixed corpora. In the 17th International Conference on Intelligent Text Processing and Computational Linguistics, pages 3–9. [find](https://scholar.google.com/scholar?q=Anupam+Jamatia%2C+Bjo%CC%88rn+Gamba%CC%88ck%2C+and+Amitava+Das%2E+2016%2E+Collecting+and+annotating+indian+social+media+code%2Dmixed+corpora%2E+In+the+17th+International+Conference+on+Intelligent+Text+Processing+and+Comput)
4. Gustavo Aguilar, Fahad AlGhamdi, Victor Soto, Mona Diab, Julia Hirschberg, and Thamar Solorio. 2018. Overview of the CALCS 2018 Shared Task: Named Entity Recognition on Code-switched Data. In Proceedings of the Third Workshop on Computational Approaches to Linguistic Code-Switching, Melbourne, Australia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Gustavo+Aguilar%2C+Fahad+AlGhamdi%2C+Victor+Soto%2C+Mona+Diab%2C+Julia+Hirschberg%2C+and+Thamar+Solorio%2E+2018%2E+Overview+of+the+CALCS+2018+Shared+Task%3A+Named+Entity+Recognition+on+Code%2Dswitched+Data%2E+In+Proceedi)
5. David Jurgens, Stefan Dimitrov, and Derek Ruths. 2014. Twitter users# codeswitch hashtags!# moltoimportante# wow. In Proceedings of the First Workshop on Computational Approaches to Code Switching, pages 51–61. [find](https://scholar.google.com/scholar?q=David+Jurgens%2C+Stefan+Dimitrov%2C+and+Derek+Ruths%2E+2014%2E+Twitter+users%23+codeswitch+hashtags%21%23+moltoimportante%23+wow%2E+In+Proceedings+of+the+First+Workshop+on+Computational+Approaches+to+Code+Switching%2C+pa)
6. Steven Bird and Edward Loper. 2004. [Nltk: the natural language toolkit](https://md.research.elamur.ai/free/en/papers/bird-2006-nltk-natural-language-toolkit/). In Proceedings of the ACL 2004 on Interactive poster and demonstration sessions, page 31. Association for Computational Linguistics.
7. John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. [find](https://scholar.google.com/scholar?q=John+Lafferty%2C+Andrew+McCallum%2C+and+Fernando+CN+Pereira%2E+2001%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E)
8. Chu-Cheng Lin, Waleed Ammar, Lori Levin, and Chris Dyer. 2014. The cmu submission for the shared task on language identification in code-switched data. In Proceedings of the First Workshop on Computational Approaches to Code Switching, pages 80–86. [find](https://scholar.google.com/scholar?q=Chu%2DCheng+Lin%2C+Waleed+Ammar%2C+Lori+Levin%2C+and+Chris+Dyer%2E+2014%2E+The+cmu+submission+for+the+shared+task+on+language+identification+in+code%2Dswitched+data%2E+In+Proceedings+of+the+First+Workshop+on+Computat)
9. Özlem Çetinoglu. 2016. A turkish-german code-switching corpus. In LREC. [find](https://scholar.google.com/scholar?q=O%CC%88zlem+C%CC%A7etinoglu%2E+2016%2E+A+turkish%2Dgerman+code%2Dswitching+corpus%2E+In+LREC%2E)
10. Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, DLRS 2016, pages 7–10, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Heng%2DTze+Cheng%2C+Levent+Koc%2C+Jeremiah+Harmsen%2C+Tal+Shaked%2C+Tushar+Chandra%2C+Hrishi+Aradhye%2C+Glen+Anderson%2C+Greg+Corrado%2C+Wei+Chai%2C+Mustafa+Ispir%2C+Rohan+Anil%2C+Zakaria+Haque%2C+Lichan+Hong%2C+Vihan+Jain%2C+Xiao)
11. David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Lingvisticae Investigationes, 30(1):3–26. [find](https://scholar.google.com/scholar?q=David+Nadeau+and+Satoshi+Sekine%2E+2007%2E+A+survey+of+named+entity+recognition+and+classification%2E+Lingvisticae+Investigationes%2C+30%281%29%3A3%E2%80%9326%2E)
12. Shuyo Nakatani. 2010. Language detection library for java. [find](https://scholar.google.com/scholar?q=Shuyo+Nakatani%2E+2010%2E+Language+detection+library+for+java%2E)
13. Naoaki Okazaki. 2007. Crfsuite: a fast implementation of conditional random fields (crfs). [find](https://scholar.google.com/scholar?q=Naoaki+Okazaki%2E+2007%2E+Crfsuite%3A+a+fast+implementation+of+conditional+random+fields+%28crfs%29%2E)
14. Gokul Chittaranjan, Yogarshi Vyas, Kalika Bali, and Monojit Choudhury. 2014. Word-level language identification using crf: Code-switching shared task report of msr india system. In Proceedings of The First Workshop on Computational Approaches to Code Switching, pages 73–79. [find](https://scholar.google.com/scholar?q=Gokul+Chittaranjan%2C+Yogarshi+Vyas%2C+Kalika+Bali%2C+and+Monojit+Choudhury%2E+2014%2E+Word%2Dlevel+language+identification+using+crf%3A+Code%2Dswitching+shared+task+report+of+msr+india+system%2E+In+Proceedings+of+The+)
15. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. [find](https://scholar.google.com/scholar?q=F%2E+Pedregosa%2C+G%2E+Varoquaux%2C+A%2E+Gramfort%2C+V%2E+Michel%2C+B%2E+Thirion%2C+O%2E+Grisel%2C+M%2E+Blondel%2C+P%2E+Prettenhofer%2C+R%2E+Weiss%2C+V%2E+Dubourg%2C+J%2E+Vanderplas%2C+A%2E+Passos%2C+D%2E+Cournapeau%2C+M%2E+Brucher%2C+M%2E+Perrot%2C+and+E%2E+Duc)
16. Michal Danilák. 2017. Python port of google language detection library. [find](https://scholar.google.com/scholar?q=Michal+Danila%CC%81k%2E+2017%2E+Python+port+of+google+language+detection+library%2E)
17. Leon Derczynski, Eric Nichols, Marieke van Erp, and Nut Limsopatham. 2017. [Results of the wnut2017 shared task on novel and emerging entity recognition](https://md.research.elamur.ai/free/en/papers/derczynski-2017-results-wnut2017-shared-task-novel/). In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 140–147. Association for Computational Linguistics.
18. Terry Peng and Mikhail Korobov. 2014. python-crfsuite. [https://python-crfsuite.readthedocs.org/](https://python-crfsuite.readthedocs.org/). [↗](https://python-crfsuite.readthedocs.org/)
19. Kashif Riaz. 2010. Rule-based named entity recognition in urdu. In Proceedings of the 2010 named entities workshop, pages 126–135. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kashif+Riaz%2E+2010%2E+Rule%2Dbased+named+entity+recognition+in+urdu%2E+In+Proceedings+of+the+2010+named+entities+workshop%2C+pages+126%E2%80%93135%2E+Association+for+Computational+Linguistics%2E)
20. Gualberto Guzmán, Joseph Ricard, Jacqueline Serigos, Barbara E Bullock, and Almeida Jacqueline Toribio. 2017. Metrics for modeling code-switching across corpora. Proc. Interspeech 2017, pages 67–71. [find](https://scholar.google.com/scholar?q=Gualberto+Guzma%CC%81n%2C+Joseph+Ricard%2C+Jacqueline+Serigos%2C+Barbara+E+Bullock%2C+and+Almeida+Jacqueline+Toribio%2E+2017%2E+Metrics+for+modeling+code%2Dswitching+across+corpora%2E+Proc%2E+Interspeech+2017%2C+pages+67%E2%80%9371%2E)
21. Sunayana Sitaram and Alan W Black. 2016. Speech synthesis of code-mixed text. In LREC. [find](https://scholar.google.com/scholar?q=Sunayana+Sitaram+and+Alan+W+Black%2E+2016%2E+Speech+synthesis+of+code%2Dmixed+text%2E+In+LREC%2E)
22. David Vilares, Miguel A Alonso, and Carlos Gómez-Rodrı́guez. 2015. Sentiment analysis on monolingual, multilingual and code-switching twitter corpora. In Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 2–8. [find](https://scholar.google.com/scholar?q=David+Vilares%2C+Miguel+A+Alonso%2C+and+Carlos+Go%CC%81mez%2DRodr%C4%B1%CC%81guez%2E+2015%2E+Sentiment+analysis+on+monolingual%2C+multilingual+and+code%2Dswitching+twitter+corpora%2E+In+Proceedings+of+the+6th+Workshop+on+Computatio)
23. Gualberto A Guzman, Jacqueline Serigos, Barbara E Bullock, and Almeida Jacqueline Toribio. 2016. Simple tools for exploring variation in code-switching for linguists. In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 12–20. [find](https://scholar.google.com/scholar?q=Gualberto+A+Guzman%2C+Jacqueline+Serigos%2C+Barbara+E+Bullock%2C+and+Almeida+Jacqueline+Toribio%2E+2016%2E+Simple+tools+for+exploring+variation+in+code%2Dswitching+for+linguists%2E+In+Proceedings+of+the+Second+Work)
24. Ayah Zirikly and Mona Diab. 2015. [Named entity recognition for arabic social media](https://md.research.elamur.ai/free/en/papers/zirikly-2015-named-entity-recognition-arabic-social/). In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pages 176–185.
*Updated 2 November 2026*