---
title: "Detecting Foodborne Illness Complaints in Multiple Languages"
description: "Health departments in several US cities run classifiers that scan online posts, such as Yelp reviews, for reports of food… 14 references."
authors: ["Ziyi Liu", "Giannis Karamanolakis", "Daniel Hsu", "Luis Gravano"]
year: 2020
venue: "Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis"
source: "https://aclanthology.org/2020.louhi-1.15/"
doi: "10.18653/v1/2020.louhi-1.15"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2020-detecting-foodborne-illness-complaints-multiple/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2020-detecting-foodborne-illness-complaints-multiple/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/10/) / Detecting Foodborne Illness Complaints in Multiple Languages Using English Annotations Only

# Detecting Foodborne Illness Complaints in Multiple Languages Using English Annotations Only

**Authors:** [Ziyi Liu](https://md.research.elamur.ai/free/en/authors/ziyi-liu/), Giannis Karamanolakis, [Daniel Hsu](https://md.research.elamur.ai/free/en/authors/daniel-hsu/) and Luis Gravano · **Year:** 2020 · **Venue:** [Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis](https://md.research.elamur.ai/free/en/venues/international-workshop-on-health-text-mining-and-information-analysis-louhi/) ([International Workshop on Health Text Mining and Information Analysis 2020](https://md.research.elamur.ai/free/en/venues/international-workshop-on-health-text-mining-and-information-analysis-louhi/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.louhi-1.15/) · [PDF](https://aclanthology.org/2020.louhi-1.15.pdf) · [DOI 10.18653/v1/2020.louhi-1.15](https://doi.org/10.18653/v1/2020.louhi-1.15) · [OpenAlex](https://openalex.org/W3098266722)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Health departments in several US cities run classifiers that scan online posts, such as Yelp reviews, for reports of food poisoning so that epidemiologists can follow up. These systems work only on English because they were trained on thousands of reviews hand-labeled "Sick" or "Not Sick," and repeating that labeling for each new language is costly. Positive cases are also rare, which makes collecting data even harder. The authors explore cross-lingual text classification that reuses only the existing English labels.

They first show that zero-shot transfer with multilingual BERT (mBERT) handles foodborne-illness nuances less well than it handles tasks like sentiment. To close that gap, they machine-translate the labeled English reviews into target languages and train mBERT on English and translated data together. Experiments cover Yelp reviews in seven languages.

Joint training beats both zero-shot and target-only training. Monolingual BERT on translated data gets the best average F1, but mBERT usually has higher recall and is easier to deploy. Run on unlabeled reviews, the multilingual models catch complaints that English-only pipelines miss.

## Key points
- The task is binary "Sick" / "Not Sick" classification of restaurant reviews, with English labels supplied by epidemiologists. Similar English systems are already used in Chicago, Nevada, New York City and St. Louis.
- Compared setups:
  - LogReg and monolingual BERT trained on translated target-language data.
  - LogReg* and BERT*, which are trained on English and applied after translating test reviews into English.
  - mBERT in several training configurations: English only (zero-shot), target language only, and joint training.
- Translating test reviews into English (translate-test) was both weaker and more expensive. It would require translating every incoming review at deployment time.
- Adding translations into several languages to the training set helped some target languages further. mBERT also showed promise on languages absent from training.
- The model was applied to unlabeled reviews from the NYC area, the LA area and the Yelp Challenge dataset. Spanish, Chinese and German complaints were flagged "Sick" by mBERT but missed by English LogReg on their machine translations.

## Interesting ideas
- Creating training data by translating existing labeled examples, then training on source and translated data together, gives labels in a new language without new annotation.
- Translate-test (convert input to the training language at inference) is weaker than translate-train, partly because machine-translated text drifts from the training distribution.
- The authors note that test sets built by translating English data may overstate quality, since native speakers phrase complaints differently.
- The translation-garbled Chinese and German examples show how mistranslation alters the event: "not fresh" became "very fresh," and "vomit" became "break." Keyword cues like these mislead a bag-of-words classifier.
- A single multilingual model avoids pipeline steps such as language detection that can introduce errors. The authors weigh this deployment simplicity against a small accuracy loss.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Our English labeled frames or silver trees could be projected through translation into extra training views, then checked by our witness parsers before use.
- **I2** (learn): Treat MT-produced or paraphrased input as a separate dialect in evaluation, and include it as a negative-control condition rather than assuming it behaves like native text.
- **I3** (learn): Label any evaluation built from transformed data as such, and keep a small native-text test set to confirm that gains hold beyond the transformation.
- **I4** (frames): Frame-level checks on who experienced what symptom after eating what could expose such meaning flips, instead of trusting surface sentiment words.
- **I5** (tokens): When deciding whether to add a pre-routing stage such as a dialect or register detector, count its error contribution and maintenance cost, not just raw accuracy.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Columbia University](https://md.research.elamur.ai/free/en/universities/columbia-university/) — University, New York, United States · 254 works · 56,747 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 882 works

## Cite

Cite as: Ziyi Liu, Giannis Karamanolakis, Daniel Hsu and Luis Gravano. 2020. Detecting Foodborne Illness Complaints in Multiple Languages Using English Annotations Only. Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2020detecting,
  title = {{Detecting Foodborne Illness Complaints in Multiple Languages Using English Annotations Only}},
  author = {Ziyi Liu, Giannis Karamanolakis, Daniel Hsu and Luis Gravano},
  booktitle = {Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis},
  year = {2020},
  doi = {10.18653/v1/2020.louhi-1.15},
  url = {https://doi.org/10.18653/v1/2020.louhi-1.15}
}
```

- [← previous Deep segmental phonetic posterior-grams based discovery of non-categories in L2 English speech](https://md.research.elamur.ai/free/en/papers/li-2020-deep-segmental-phonetic-posterior-grams/)
- [next → Embed More Ignore Less (EMIL): Exploiting Enriched Representations for Arabic NLP](https://md.research.elamur.ai/free/en/papers/younes-2020-embed-more-ignore-less-emil/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/10/)

## References

14 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. [How multilingual is multilingual BERT?](https://md.research.elamur.ai/free/en/papers/pires-2019-how-multilingual-is-multilingual-bert/) In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
2. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. [A primer in BERTology: What we know about how BERT works](https://md.research.elamur.ai/free/en/papers/rogers-2020-primer-bertology-what-we-know/). arXiv preprint arXiv:2002.12327. [↗](https://arxiv.org/abs/2002.12327)
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
4. Sebastian Ruder, Ivan Vulić, and Anders Søgaard. 2019. A survey of cross-lingual word embedding models. Journal of Artificial Intelligence Research, 65:569–631. [find](https://scholar.google.com/scholar?q=Sebastian+Ruder%2C+Ivan+Vulic%CC%81%2C+and+Anders+S%C3%B8gaard%2E+2019%2E+A+survey+of+cross%2Dlingual+word+embedding+models%2E+Journal+of+Artificial+Intelligence+Research%2C+65%3A569%E2%80%93631%2E)
5. Thomas Effland, Anna Lawson, Sharon Balter, Kate-lynn Devinney, Vasudha Reddy, HaeNa Waechter, Luis Gravano, and Daniel Hsu. 2018. Discovering foodborne illness in online restaurant reviews. Journal of the American Medical Informatics Association. [find](https://scholar.google.com/scholar?q=Thomas+Effland%2C+Anna+Lawson%2C+Sharon+Balter%2C+Kate%2Dlynn+Devinney%2C+Vasudha+Reddy%2C+HaeNa+Waechter%2C+Luis+Gravano%2C+and+Daniel+Hsu%2E+2018%2E+Discovering+foodborne+illness+in+online+restaurant+reviews%2E+Journal+o)
6. Adam Sadilek, Henry A Kautz, Lauren DiPrete, Brian Labus, Eric Portman, Jack Teitel, and Vincent Silenzio. 2016. Deploying nEmesis: Preventing foodborne illness by data mining social media. In Proceedings of the AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Adam+Sadilek%2C+Henry+A+Kautz%2C+Lauren+DiPrete%2C+Brian+Labus%2C+Eric+Portman%2C+Jack+Teitel%2C+and+Vincent+Silenzio%2E+2016%2E+Deploying+nEmesis%3A+Preventing+foodborne+illness+by+data+mining+social+media%2E+In+Proceed)
7. Stephan Gouws and Anders Søgaard. 2015. Simple task-specific bilingual word embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. [find](https://scholar.google.com/scholar?q=Stephan+Gouws+and+Anders+S%C3%B8gaard%2E+2015%2E+Simple+task%2Dspecific+bilingual+word+embeddings%2E+In+Proceedings+of+the+2015+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguis)
8. Shijie Wu and Mark Dredze. 2019. [Beto, Bentz, Becas: The surprising cross-lingual effectiveness of BERT](https://md.research.elamur.ai/free/en/papers/wu-2019-beto-bentz-becas-surprising-cross/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.
9. Jenine K Harris, Leslie Hinyard, Kate Beatty, Jared B Hawkins, Elaine O Nsoesie, Raed Mansour, and John S Brownstein. 2018. Evaluating the implementation of a Twitter-based foodborne illness reporting tool in the City of St. Louis Department of Health. International Journal of Environmental Research and Public Health, 15(5). [find](https://scholar.google.com/scholar?q=Jenine+K+Harris%2C+Leslie+Hinyard%2C+Kate+Beatty%2C+Jared+B+Hawkins%2C+Elaine+O+Nsoesie%2C+Raed+Mansour%2C+and+John+S+Brownstein%2E+2018%2E+Evaluating+the+implementation+of+a+Twitter%2Dbased+foodborne+illness+reporting)
10. Jenine K Harris, Raed Mansour, Bechara Choucair, Joe Olson, Cory Nissen, and Jay Bhatt. 2014. Health department use of social media to identify foodborne illness-Chicago, Illinois, 2013-2014. Morbidity and Mortality Weekly Report, 63(32):681–685. [find](https://scholar.google.com/scholar?q=Jenine+K+Harris%2C+Raed+Mansour%2C+Bechara+Choucair%2C+Joe+Olson%2C+Cory+Nissen%2C+and+Jay+Bhatt%2E+2014%2E+Health+department+use+of+social+media+to+identify+foodborne+illness%2DChicago%2C+Illinois%2C+2013%2D2014%2E+Morbidit)
11. Giannis Karamanolakis, Daniel Hsu, and Luis Gravano. 2019. Weakly supervised attention networks for fine-grained opinion mining and public health. In Proceedings of the 5th Workshop on Noisy User-Generated Text. [find](https://scholar.google.com/scholar?q=Giannis+Karamanolakis%2C+Daniel+Hsu%2C+and+Luis+Gravano%2E+2019%2E+Weakly+supervised+attention+networks+for+fine%2Dgrained+opinion+mining+and+public+health%2E+In+Proceedings+of+the+5th+Workshop+on+Noisy+User%2DGene)
12. Giannis Karamanolakis, Daniel Hsu, and Luis Gravano. 2020. Cross-lingual text classification with minimal resources by transferring a sparse teacher. In Proceedings of the 2020 Findings of Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Giannis+Karamanolakis%2C+Daniel+Hsu%2C+and+Luis+Gravano%2E+2020%2E+Cross%2Dlingual+text+classification+with+minimal+resources+by+transferring+a+sparse+teacher%2E+In+Proceedings+of+the+2020+Findings+of+Empirical+M)
13. Kaliyaperumal Karthikeyan, Zihan Wang, Stephen Mayhew, and Dan Roth. 2019. Cross-lingual ability of multilingual BERT: An empirical study. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Kaliyaperumal+Karthikeyan%2C+Zihan+Wang%2C+Stephen+Mayhew%2C+and+Dan+Roth%2E+2019%2E+Cross%2Dlingual+ability+of+multilingual+BERT%3A+An+empirical+study%2E+In+International+Conference+on+Learning+Representations%2E)
14. Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, et al. 2019. [find](https://scholar.google.com/scholar?q=Yu%2DHsiang+Lin%2C+Chian%2DYu+Chen%2C+Jean+Lee%2C+Zirui+Li%2C+Yuyan+Zhang%2C+Mengzhou+Xia%2C+Shruti+Rijhwani%2C+Junxian+He%2C+Zhisong+Zhang%2C+Xuezhe+Ma%2C+et+al%2E+2019%2E)

*Updated 2 November 2026*