--- title: "HaleLab_NITK@SMM4H'24 — paper by Mahajan and S (2024)" description: "This paper is a short system description for Task 5 of the SMM4H 2024 shared task. 0.88 precision. 9 references." authors: ["Ritik Mahajan", "Sowmya S."] year: 2024 venue: "Proceedings of the 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks" source: "https://aclanthology.org/2024.smm4h-1.31/" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/mahajan-2024-halelab-nitk-smm4h-24-binary/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mahajan-2024-halelab-nitk-smm4h-24-binary/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/) / HaleLab\_NITK@SMM4H'24: Binary classification of English tweets reporting children's medical disorders # HaleLab\_NITK@SMM4H'24: Binary classification of English tweets reporting children's medical disorders **Authors:** Ritik Mahajan and Sowmya S. · **Year:** 2024 · **Venue:** [Proceedings of the 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks](https://md.research.elamur.ai/free/en/venues/smm4h/) ([Social Media Mining for Health Research and Applications Workshop and Shared Tasks 2024](https://md.research.elamur.ai/free/en/venues/smm4h/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2024.smm4h-1.31/) · [PDF](https://aclanthology.org/2024.smm4h-1.31.pdf) · [OpenAlex](https://openalex.org/W4402683370) **Citations:** 1 citations (OpenAlex, 2026-10-14) ## Summary This paper is a short system description for Task 5 of the SMM4H 2024 shared task. The task is to label English tweets from users who have previously announced a pregnancy. A tweet is positive when the parent clearly states that their own child has ADHD, ASD, delayed speech or asthma. It is negative when the tweet only mentions a disorder or does not concern the parent's child at all. The authors used the corpus by Klein et al. (2023) and normalised the tweets heavily before modelling: - unified disorder names, digits and kinship words - stripped Twitter-specific noise - expanded slang and contractions - lemmatised the text with spaCy They then fine-tuned RoBERTa-base with an extra dense ReLU layer and a sigmoid output. On the official 10,000-tweet test set the system reached F1 0.87, close to its validation score of 0.88. This is below the published RoBERTa-large baseline (0.927), which was measured on a smaller test set. The work matters because large-scale filtering of parents' first-hand health reports can support pediatric epidemiology and targeted family support. ## Key points - Data split: 7,398 training tweets, 389 validation tweets and 10,000 test tweets (Klein et al., 2023). - Preprocessing steps: - mapped son, daughter and similar words to "child" - removed URLs, usernames, hashtags and emojis with the tweet-preprocessor library - expanded items like "lol", "thx" and "we're" - lowercased the text - limited repeated characters to two - lemmatised with spaCy - Model and training: RoBERTa-base via Huggingface, max length 128, Adam, batch size 8, learning rate 1e-5. Training was capped at 10 epochs and halted early if there was no improvement for 4 epochs, with dropout 0.3, on a Colab T4 GPU. - Validation scores: F1 0.88, precision 0.89, recall 0.88. - Test scores: F1 0.87 (0.868), precision 0.86, recall 0.88. - Comparison: the RoBERTa-large baseline from Klein et al. (2024) scored 0.927 on 1,947 tweets, so the two numbers are not directly comparable. ## Interesting ideas - Kinship words with the same task role (son, daughter, child) are collapsed into one placeholder, so the model sees one consistent signal for "the poster's offspring". - Character elongation is capped at two repeats, which reduces noisy spellings to a small set of variants that can be looked up. - Chat abbreviations and contractions are expanded into full words before analysis, turning informal input into something closer to standard text. - The task asks whether the poster's own child has the condition, not whether a disorder is named, which is a question of who holds the condition. - The authors point out that their score and the baseline score come from test sets of different sizes and from different model sizes. This is a reminder that headline numbers can mislead. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (world): Kin nouns could link to a shared ontology class such as child-of-speaker while keeping their surface lemmas, so entity tracking treats "my son" and "my daughter" alike. - **I2** (tokens): The en crate could apply the same cap before lemma and morphology lookup, so forms like "sooo" or "loooove" still map to dictionary entries. - **I3** (tokens): A small expansion table run before tagging would give the UD parser cleaner tokens for social-media input without retraining. - **I4** (frames): prag could answer this explicitly by checking that the holder of a has-condition frame is coreferent with the speaker's child. This gives a traceable decision instead of keyword matching. - **I5** (learn): Evaluation reports should always pair a system with its baselines on identical test sets and note any mismatch next to the numbers. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [National Institute of Technology Karnataka](https://md.research.elamur.ai/free/en/universities/national-institute-of-technology-karnataka/) — University, Mangaluru, India · 2 works · 5 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Social Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/social-psychology/) · 306 works 🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works ## Cite Cite as: Ritik Mahajan and Sowmya S. 2024. HaleLab\_NITK@SMM4H'24: Binary classification of English tweets reporting children's medical disorders. Proceedings of the 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{mahajan2024halelab, title = {{HaleLab\_NITK@SMM4H'24: Binary classification of English tweets reporting children's medical disorders}}, author = {Ritik Mahajan and Sowmya S.}, booktitle = {Proceedings of the 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks}, year = {2024}, doi = {10.18653/v1/2024.smm4h-1.31}, url = {https://doi.org/10.18653/v1/2024.smm4h-1.31} } ``` - [← previous GAOKAO-Eval: Does high scores truly reflect strong capabilities in LLMs?](https://md.research.elamur.ai/free/en/papers/lei-2024-gaokao-eval-high-scores/) - [next → How Much Do Robots Understand Rudeness? Challenges in Human-Robot Interaction](https://md.research.elamur.ai/free/en/papers/orme-2024-how-much-do-robots-understand/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/) ## References 9 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Sony Bachina, Spandana Balumuri, and Sowmya Kamath. 2021. Ensemble albert and roberta for span prediction in question answering. In Proceedings of the 1st workshop on document-grounded dialogue and conversational question answering (DialDoc 2021), pages 63–68. [find](https://scholar.google.com/scholar?q=Sony+Bachina%2C+Spandana+Balumuri%2C+and+Sowmya+Kamath%2E+2021%2E+Ensemble+albert+and+roberta+for+span+prediction+in+question+answering%2E+In+Proceedings+of+the+1st+workshop+on+document%2Dgrounded+dialogue+and+co) 2. Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear. [find](https://scholar.google.com/scholar?q=Matthew+Honnibal+and+Ines+Montani%2E+2017%2E+spaCy+2%3A+Natural+language+understanding+with+Bloom+embeddings%2C+convolutional+neural+networks+and+incremental+parsing%2E+To+appear%2E) 3. Ari Z Klein, José Agustín Gutiérrez Gómez, Lisa D Levine, and Graciela Gonzalez-Hernandez. 2024. Using longitudinal twitter data for digital epidemiology of childhood health outcomes: An annotated data set and deep neural network classifiers. J Med Internet Res, 26:e50652. [↗](https://doi.org/10.2196/50652) 4. Ari Z Klein, Shriya Kunatharaju, Karen O’Connor, and Graciela Gonzalez-Hernandez. 2023. Pregex: rule-based detection and extraction of twitter data in pregnancy. Journal of Medical Internet Research, 25:e40569. [↗](https://doi.org/10.2196/40569) 5. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692. [↗](https://arxiv.org/abs/1907.11692) 6. Gurdeep Saini and Naveen Yadav. 2022. Ensemble neural models for depressive tendency prediction based on social media activity of twitter users. In Security, Privacy and Data Analytics: Select Proceedings of ISPDA 2021, pages 211–226. Springer. [↗](https://doi.org/10.1007/978-981-16-9089-1_18) 7. Reshma Unnikrishnan et al. 2023. Efficient parameter tuning of neural foundation models for drug perspective prediction from unstructured socio-medical data. Engineering Applications of Artificial Intelligence, 123:106214. [↗](https://doi.org/10.1016/j.engappai.2023.106214) 8. Guido Van Rossum and Fred L Drake Jr. 1995. Python tutorial. [find](https://scholar.google.com/scholar?q=Guido+Van+Rossum+and+Fred+L+Drake+Jr%2E+1995%2E+Python+tutorial%2E) 9. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019. [Huggingface’s transformers: State-ofthe-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771. [↗](https://arxiv.org/abs/1910.03771) *Updated 1 November 2026*