---
title: "Overview of the HASOC Subtrack at FIRE 2023: Identification of"
description: "The organizers argue that pinpointing these fragments has two uses. 23 references, cited by 1 paper here."
authors: ["Sarah Masud", "Mohammad Aflah Khan", "Md. Shad Akhtar", "Tanmoy Chakraborty"]
year: 2023
venue: "arXiv:2311.09834"
source: "https://arxiv.org/abs/2311.09834"
doi: "10.48550/arxiv.2311.09834"
arxiv: "2311.09834"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/masud-2023-overview-hasoc-subtrack-fire-2023/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/masud-2023-overview-hasoc-subtrack-fire-2023/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/16/) / Overview of the HASOC Subtrack at FIRE 2023: Identification of Tokens Contributing to Explicit Hate in English by Span Detection

# Overview of the HASOC Subtrack at FIRE 2023: Identification of Tokens Contributing to Explicit Hate in English by Span Detection

**Authors:** Sarah Masud, Mohammad Aflah Khan, [Md. Shad Akhtar](https://md.research.elamur.ai/free/en/authors/md-shad-akhtar/) and [Tanmoy Chakraborty](https://md.research.elamur.ai/free/en/authors/tanmoy-chakraborty/) · **Year:** 2023 · **Venue:** arXiv:2311.09834 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2311.09834) · [PDF](https://arxiv.org/pdf/2311.09834) · [DOI 10.48550/arxiv.2311.09834](https://doi.org/10.48550/arxiv.2311.09834) · [arXiv 2311.09834](https://arxiv.org/abs/2311.09834) · [OpenAlex](https://openalex.org/W4388788709)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
This overview covers HateNorm, a shared task run within HASOC at FIRE 2023. It asks systems to locate the exact words that make an English tweet explicitly hateful, rather than only labelling the whole post as hateful. The organizers argue that pinpointing these fragments has two uses. It can explain to users why automatic moderation flagged their post, and it can support rephrasing suggestions before a post goes public. The data comes from the existing Hate Normalization dataset of Masud et al. The organizers took only the 3027 explicitly hateful sentences that carry span annotations. The task is framed as sequence tagging with BIO labels and scored with seqeval macro-F1. The competition ran on Kaggle and drew 72 submissions from 12 teams. The winner, FiRC-NLP, used an ensemble of SpanBERT and CRF and reached 0.58. That score only matches the organizers' earlier SpanBERT+BiLSTM+CRF baseline. The authors conclude that span-level hate detection is still hard and deserves more attention than plain classification.

## Key points
- Data split: 2421 training sentences containing 4695 unique spans (about 1.94 spans per sentence) and 606 test sentences. The test set was divided into 182 public-leaderboard items and 424 private-leaderboard items.
- Single-word spans are the most frequent, and most spans are five tokens or shorter. A lone token is tagged B, and every I must follow a B.
- The competition ran from July 13 to August 16, 2023. The final ranking was computed on the hidden private portion (about 70% of the test set), and one team moved up three places when the private scores were revealed.
- Reference scores: the starter kit (DistilBERT + feed-forward head) scored about 0.34–0.36. The earlier baselines scored 0.44 for BiLSTM+CRF and 0.58 for SpanBERT+BiLSTM+CRF. Among participants, FiRC-NLP scored 0.576, Mohammadmostafa78 0.514 (XLM-RoBERTa + FNN), and CNLP-NITS-PP and IRLab@IITBHU about 0.509 each.
- Systems based on GloVe embeddings with BiLSTM+CRF stayed close to transformer systems. Half of the teams used no CRF layer and relied only on contextual encoders.

## Interesting ideas
- Scoring with seqeval counts a span as correct only when its boundaries and type match exactly, so a model gets no credit for tagging scattered tokens correctly.
- Rankings were published on a small public slice, while the decisive score came from a larger hidden slice; the resulting rank shifts expose teams that overfit to the visible numbers.
- The runner-up handled label imbalance by collapsing BIO into a simpler in/out scheme and then rebuilding spans afterwards, which made the learning problem easier.
- Each tag is tied one-to-one to an input token, so participants were told not to preprocess text in ways that would break the alignment.
- The task motivates spans as rationales: a minimal stretch of text that justifies a label, which serves both explanation and rewriting.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Evaluate span-like outputs (multiword expressions, argument spans of frames) with exact-boundary span F1 instead of per-token accuracy, so partial overlaps cannot inflate scores.
- **I2** (learn): Keep a sealed held-out portion of each test suite that is checked only at milestones, so repair rules and tuning cannot quietly overfit the development set.
- **I3** (tokens): When a tagger struggles with rare boundary labels (e.g., multiword names), train on a binary inside/outside label and recover span starts deterministically in post-processing.
- **I4** (tree): Treat token alignment as a hard invariant when converting treebanks or UD dialects, with a check that flags any normalization step that changes token counts.
- **I5** (frames): Attach to each speech-act or attitude judgment (insult, threat, irony) the exact token span that triggered it, so the judgment remains explainable and can feed rephrasing in generation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Indian Institute of Technology Delhi](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-delhi/) — University, New Delhi, India · 24 works · 6,933 citations
- [Indraprastha Institute of Information Technology Delhi](https://md.research.elamur.ai/free/en/universities/indraprastha-institute-of-information-technology-delhi/) — University, Delhi, India · 10 works · 1,663 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 626 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works

## Cite

Cite as: Sarah Masud, Mohammad Aflah Khan, Md. Shad Akhtar and Tanmoy Chakraborty. 2023. Overview of the HASOC Subtrack at FIRE 2023: Identification of Tokens Contributing to Explicit Hate in English by Span Detection. arXiv:2311.09834.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{masud2023overview,
  title = {{Overview of the HASOC Subtrack at FIRE 2023: Identification of Tokens Contributing to Explicit Hate in English by Span Detection}},
  author = {Sarah Masud, Mohammad Aflah Khan, Md. Shad Akhtar and Tanmoy Chakraborty},
  year = {2023},
  doi = {10.48550/arxiv.2311.09834},
  eprint = {2311.09834},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2311.09834}
}
```

- [← previous OLEA: Tool and Infrastructure for Offensive Language Error Analysis in English](https://md.research.elamur.ai/free/en/papers/grace-2023-olea-tool-infrastructure-offensive-language/)
- [next → Performance Analysis of Arabic Pre-trained Models on Named Entity Recognition Task](https://md.research.elamur.ai/free/en/papers/dahou-2023-performance-analysis-arabic-pre-trained/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/16/)

## References

23 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. J. Pavlopoulos, J. Sorensen, L. Laugier, I. Androutsopoulos, [SemEval-2021 task 5: Toxic spans detection](https://md.research.elamur.ai/free/en/papers/pavlopoulos-2021-semeval-2021-task-5-toxic/), in: Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), Association for Computational Linguistics, Online, 2021, pp. 59–69. URL: https://aclanthology.org/2021.semeval-1.6. doi: 10.18653/v1/2021.semeval-1. 6 . [↗](https://doi.org/10.18653/v1/2021.semeval-1)
2. M. Sundriyal, A. Kulkarni, V. Pulastya, M. S. Akhtar, T. Chakraborty, Empowering the fact-checkers! automatic identification of claim spans on Twitter, in: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 2022, pp. 7701–7715. URL: https://aclanthology.org/2022.emnlp-main.525. doi: 10.18653/v1/2022.emnlp-main. 525 . [↗](https://doi.org/10.18653/v1/2022.emnlp-main)
3. S. Masud, M. Bedi, M. A. Khan, M. S. Akhtar, T. Chakraborty, Proactively reducing the hate intensity of online posts via hate speech normalization, in: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’22, Association for Computing Machinery, New York, NY, USA, 2022, p. 3524–3534. URL: https://doi.org/10.1145/3534678.3539161. doi: 10.1145/3534678.3539161 . [↗](https://doi.org/10.1145/3534678.3539161)
4. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, I. Polosukhin, [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/), in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems, volume 30, Curran Associates, Inc., 2017. URL: [https://proceedings.neurips.cc/](https://proceedings.neurips.cc/) paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf. [↗](https://proceedings.neurips.cc/)
5. Z. Waseem, D. Hovy, [Hateful symbols or hateful people? predictive features for hate speech detection on Twitter](https://md.research.elamur.ai/free/en/papers/waseem-2016-hateful-symbols-or-hateful-people/), in: Proceedings of the NAACL Student Research Workshop, Association for Computational Linguistics, San Diego, California, 2016, pp. 88–93. URL: https://aclanthology.org/N16-2013. doi: 10.18653/v1/N16-2013 . [↗](https://doi.org/10.18653/v1/n16-2013)
6. T. Davidson, D. Warmsley, M. Macy, I. Weber, [Automated hate speech detection and the problem of offensive language](https://md.research.elamur.ai/free/en/papers/davidson-2017-automated-hate-speech-detection-problem/), Proceedings of the International AAAI Conference on Web and Social Media 11 (2017) 512–515. URL: [https://ojs.aaai.org/index.php/ICWSM/article/](https://ojs.aaai.org/index.php/ICWSM/article/) view/14955. [↗](https://ojs.aaai.org/index.php/ICWSM/article/)
7. A. Kulkarni, S. Masud, V. Goyal, T. Chakraborty, Revisiting hate speech benchmarks: From data curation to system deployment, in: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’23, Association for Computing Machinery, New York, NY, USA, 2023, p. 4333–4345. URL: https://doi.org/10.1145/3580305.3599896. doi: 10.1145/3580305.3599896 . [↗](https://doi.org/10.1145/3580305.3599896)
8. A. Schmidt, M. Wiegand, [A survey on hate speech detection using natural language processing](https://md.research.elamur.ai/free/en/papers/schmidt-2017-survey-hate-speech-detection-natural/), in: L.-W. Ku, C.-T. Li (Eds.), Proc. of the Fifth International Workshop on Natural Language Processing for Social Media, ACL, Valencia, Spain, 2017, pp. 1–10. URL: https://aclanthology.org/W17-1101. doi: 10.18653/v1/W17-1101 . [↗](https://doi.org/10.18653/v1/w17-1101)
9. A. Founta, L. Specia, A survey of online hate speech through the causal lens, in: A. Feder, K. Keith, E. Manzoor, R. Pryzant, D. Sridhar, Z. Wood-Doughty, J. Eisenstein, J. Grimmer, R. Reichart, M. Roberts, U. Shalit, B. Stewart, V. Veitch, D. Yang (Eds.), Proceedings of the First Workshop on Causal Inference and NLP, Association for Computational Linguistics, Punta Cana, Dominican Republic, 2021, pp. 74–82. URL: https://aclanthology.org/2021. cinlp-1.6. doi: 10.18653/v1/2021.cinlp-1.6 . [↗](https://doi.org/10.18653/v1/2021.cinlp-1.6)
10. M. Polignano, G. Colavito, C. Musto, M. de Gemmis, G. Semeraro, Lexicon enriched hybrid hate speech detection with human-centered explanations, in: Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization, UMAP ’22 Adjunct, Association for Computing Machinery, New York, NY, USA, 2022, p. 184–191. URL: https://doi.org/10.1145/3511047.3537688. doi: 10.1145/3511047.3537688 . [↗](https://doi.org/10.1145/3511047.3537688)
11. V. Stamou, I. Alexiou, A. Klimi, E. Molou, A. Saivanidou, S. Markantonatou, Cleansing & expanding the HURTLEX(el) with a multidimensional categorization of offensive words, in: K. Narang, A. Mostafazadeh Davani, L. Mathias, B. Vidgen, Z. Talat (Eds.), Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH), Association for Computational Linguistics, Seattle, Washington (Hybrid), 2022, pp. 102–108. URL: https://aclanthology.org/2022.woah-1.10. doi: 10.18653/v1/2022.woah-1.10 . [↗](https://doi.org/10.18653/v1/2022.woah-1.10)
12. J. Pavlopoulos, L. Laugier, A. Xenos, J. Sorensen, I. Androutsopoulos, From the detection of toxic spans in online discussions to the analysis of toxic-to-civil transfer, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Dublin, Ireland, 2022, pp. 3721–3734. URL: https://aclanthology.org/2022.acl-long.259. doi: 10.18653/v1/2022. acl-long.259 . [↗](https://doi.org/10.18653/v1/2022)
13. V. Agarwal, Y. Chen, N. Sastry, Haterephrase: Zero- and few-shot reduction of hate intensity in online posts using large language models, 2023. arXiv:2310.13985 . [↗](https://arxiv.org/abs/2310.13985)
14. B. Mathew, P. Saha, S. M. Yimam, C. Biemann, P. Goyal, A. Mukherjee, Hatexplain: A benchmark dataset for explainable hate speech detection, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 2021, pp. 14867–14875. [find](https://scholar.google.com/scholar?q=B%2E+Mathew%2C+P%2E+Saha%2C+S%2E+M%2E+Yimam%2C+C%2E+Biemann%2C+P%2E+Goyal%2C+A%2E+Mukherjee%2C+Hatexplain%3A+A+benchmark+dataset+for+explainable+hate+speech+detection%2C+in%3A+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelli)
15. P. G. Hoang, C. D. Luu, K. Q. Tran, K. V. Nguyen, N. L.-T. Nguyen, ViHOS: Hate speech spans detection for Vietnamese, in: Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, Association for Computational Linguistics, Dubrovnik, Croatia, 2023, pp. 652–669. URL: https://aclanthology.org/2023. eacl-main.47. doi: 10.18653/v1/2023.eacl-main.47 . [↗](https://doi.org/10.18653/v1/2023.eacl-main.47)
16. T. Ranasinghe, M. Zampieri, MUDES: Multilingual detection of offensive spans, in: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstrations, Association for Computational Linguistics, Online, 2021, pp. 144–152. URL: https://aclanthology.org/ 2021.naacl-demos.17. doi: 10.18653/v1/2021.naacl-demos.17 . [↗](https://doi.org/10.18653/v1/2021.naacl-demos.17)
17. M. Das, R. Raj, P. Saha, B. Mathew, M. Gupta, A. Mukherjee, Hatemm: A multi-modal dataset for hate video classification, Proceedings of the International AAAI Conference on Web and Social Media 17 (2023) 1014–1023. URL: https://ojs.aaai.org/index.php/ICWSM/ article/view/22209. doi: 10.1609/icwsm.v17i1.22209 . [↗](https://doi.org/10.1609/icwsm.v17i1.22209)
18. L. Zhou, A. Caines, I. Pete, A. Hutchings, Automated hate speech detection and span extraction in underground hacking and extremist forums, Natural Language Engineering 29 (2023) 1247–1274. doi: 10.1017/S1351324922000262 . [↗](https://doi.org/10.1017/s1351324922000262)
19. S. Mittal, M. Sundriyal, P. Nakov, Lost in translation, found in spans: Identifying claims in multilingual social media, arXiv:2310.18205 (2023). [↗](https://arxiv.org/abs/2310.18205)
20. B. Kennedy, X. Jin, A. Mostafazadeh Davani, M. Dehghani, X. Ren, [Contextualizing hate speech classifiers with post-hoc explanation](https://md.research.elamur.ai/free/en/papers/kennedy-2020-contextualizing-hate-speech-classifiers-post/), in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, 2020, pp. 5435–5442. URL: https://aclanthology.org/2020.acl-main.483. doi: 10.18653/v1/2020.acl-main.483 . [↗](https://doi.org/10.18653/v1/2020.acl-main.483)
21. H. Nakayama, seqeval: A python framework for sequence labeling evaluation, 2018. URL: [https://github.com/chakki-works/seqeval](https://github.com/chakki-works/seqeval), software available from https://github.com/chakki-works/seqeval. [↗](https://github.com/chakki-works/seqeval)
22. M. Joshi, D. Chen, Y. Liu, D. S. Weld, L. Zettlemoyer, O. Levy, [SpanBERT: Improving pre-training by representing and predicting spans](https://md.research.elamur.ai/free/en/papers/joshi-2019-spanbert-improving-pre-training-representing/), Transactions of the Association for Computational Linguistics 8 (2020) 64–77. URL: https://aclanthology.org/2020.tacl-1.5. doi: 10.1162/tacl_a_00300 . [↗](https://doi.org/10.1162/tacl_a_00300)
23. A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, V. Stoyanov, [Unsupervised cross-lingual representation learning at scale](https://md.research.elamur.ai/free/en/papers/conneau-2020-unsupervised-cross-lingual-representation-learning/), in: D. Jurafsky, J. Chai, N. Schluter, J. Tetreault (Eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, 2020, pp. 8440–8451. URL: https://aclanthology.org/2020.acl-main.747. doi: 10.18653/v1/2020.acl-main.747 . [↗](https://doi.org/10.18653/v1/2020.acl-main.747)

*Updated 2 November 2026*