--- title: "ISNA-Set: A novel English Corpus of Iran NEWS (2018)" description: "Earlier corpora built from Iranian outlets are almost all in Persian, so the authors release an English-language collection. 1 reference." authors: ["Mohammad Kamel", "Hadi Sadoghi-Yazdi"] year: 2018 venue: "arXiv:1808.07046" source: "https://arxiv.org/abs/1808.07046" doi: "10.48550/arxiv.1808.07046" arxiv: "1808.07046" licence: "arXiv — non-exclusive license to distribute" licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/kamel-2018-isna-set-novel-english-corpus/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kamel-2018-isna-set-novel-english-corpus/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/6/) / ISNA-Set: A novel English Corpus of Iran NEWS # ISNA-Set: A novel English Corpus of Iran NEWS **Authors:** Mohammad Kamel and Hadi Sadoghi-Yazdi · **Year:** 2018 · **Venue:** arXiv:1808.07046 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/) **Links:** [Publisher's page](https://arxiv.org/abs/1808.07046) · [PDF](https://arxiv.org/pdf/1808.07046) · [DOI 10.48550/arxiv.1808.07046](https://doi.org/10.48550/arxiv.1808.07046) · [arXiv 1808.07046](https://arxiv.org/abs/1808.07046) · [OpenAlex](https://openalex.org/W2888294989) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary The paper addresses a gap in resources for Iranian news. Earlier corpora built from Iranian outlets are almost all in Persian, so the authors release an English-language collection. ISNA-Set contains every English article they could gather from the website of the Iranian Students News Agency (ISNA): 1015 items in six topical categories. The pipeline has five steps. They build a URL list from the site archive, download the pages with wget, and use BeautifulSoup to separate the article from site chrome such as menus, headers and footers. Each article is stored as a JSON record. The text is then cleaned by converting dates to Unix timestamps, removing non-ASCII characters and applying stemming and lemmatization, and a Word2Vec model is trained on the result. Each record also carries automatically computed sentiment scores for title and body, counts of named-entity types and counts of part-of-speech tags. Most of the paper is descriptive statistics on article length and on publication by weekday, day of month, month and category. The contribution is a small, clean, metadata-rich English news resource from a domain that is rarely covered. ## Key points - **Size and categories:** the corpus has 1015 English ISNA articles in six categories: Politics, Economy, Culture and Art, Science, Social and Sports. - Politics is by far the largest and Economy is a distant second. - Social and Sports are the smallest. - **JSON record fields:** - text: title, body and editorial tags; - metadata: URL, news code, category, journalist ID, original date string and datetimestamp; - annotations: sentiment (body_s, title_s), entity-type counts and POS-tag counts. - **Length:** - Most articles are under 400 words, and articles over 1200 words are rare. - Most documents are below 2K characters, and documents above 5K are rare. - The largest length bin, 500–1000, holds 415 items. - **Weekly pattern:** publication nearly stops on Thursday and Friday, which are the Iranian weekend. Saturday and Monday are the busiest days. - **Yearly pattern:** months 8 and 10 have many economic and political stories, which the authors link to Iran's budget approval period. - Volume rises through summer. - Volume falls sharply through winter. ## Interesting ideas - Headline and body get separate sentiment scores, so a framing gap between a neutral title and a more charged body becomes measurable. - Free-text, human-formatted dates are normalized into machine timestamps at ingestion rather than left for later stages. - Each document carries compact aggregate profiles, such as an entity-type histogram and POS-tag counts, alongside its raw text. - Publication patterns that match real-world calendars, like the weekend dip and budget-season peaks, act as an external sanity check that the crawl is complete and the date parsing is correct. - Collection is split into separate stages (URL listing, crawling, parsing, cleaning), with site boilerplate stripped before any linguistic processing. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (frames): compute stance or speech-act features separately for headline and body, and flag documents whose headline frame disagrees with what the body actually asserts. - **I2** (world): resolve publication dates into absolute times up front so relative expressions in the text ("yesterday", "last month") can be anchored when ordering events. - **I3** (learn): store per-document tag and entity distributions with silver data, so shifts in these profiles between parser versions can serve as cheap drift alarms. - **I4** (learn): use known external regularities as negative controls, for example checking that extracted dates never cluster on impossible days, to catch silent pipeline bugs. - **I5** (tokens): keep markup and boilerplate removal as an explicit, inspectable step before tokenization, so stray menu text or non-ASCII debris can be traced instead of polluting tokens and lemmas. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works 🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works ## Cite Cite as: Mohammad Kamel and Hadi Sadoghi-Yazdi. 2018. ISNA-Set: A novel English Corpus of Iran NEWS. arXiv:1808.07046. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{kamel2018isna, title = {{ISNA-Set: A novel English Corpus of Iran NEWS}}, author = {Mohammad Kamel and Hadi Sadoghi-Yazdi}, year = {2018}, doi = {10.48550/arxiv.1808.07046}, eprint = {1808.07046}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.1808.07046} } ``` - [← previous Inflectional Variation in the Old English Participle. A Corpus-based Analysis](https://md.research.elamur.ai/free/en/papers/lopez-2018-inflectional-variation-old-english-participle/) - [next → Making sense: The glamorous history of English grammar (a review)](https://md.research.elamur.ai/free/en/papers/tomalin-2018-making-sense-glamorous-history-english/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/6/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. [1500 - 2000] Social [1500 - 2000] Science [1000 - 1500] Science [1500 - 2000] Sports [1000 - 1500] Sports [500 - 1000] Sports [1500 - 2000] Politics [1000 - 1500] Politics [1500 - 2000] Economy [1000 - 1500] Economy [1500 - 2000] Culture and Art [find](https://scholar.google.com/scholar?q=%5B1500+%2D+2000%5D+Social+%5B1500+%2D+2000%5D+Science+%5B1000+%2D+1500%5D+Science+%5B1500+%2D+2000%5D+Sports+%5B1000+%2D+1500%5D+Sports+%5B500+%2D+1000%5D+Sports+%5B1500+%2D+2000%5D+Politics+%5B1000+%2D+1500%5D+Politics+%5B1500+%2D+2000%5D+Economy+%5B1000) *Updated 24 October 2026*