---
title: "Language Independent Named Entity Recognition via Orthogonal"
description: "The authors build a standard pipeline: word embeddings feed a Bidirectional LSTM, and a CRF layer produces IOB tags. 23 references."
authors: ["Omar E. Rakha", "Hazem M. Abbas"]
year: 2025
venue: "arXiv:2503.14755"
source: "https://arxiv.org/abs/2503.14755"
doi: "10.48550/arxiv.2503.14755"
arxiv: "2503.14755"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rakha-2025-language-independent-named-entity-recognition/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rakha-2025-language-independent-named-entity-recognition/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/) / Language Independent Named Entity Recognition via Orthogonal Transformation of Word Vectors

# Language Independent Named Entity Recognition via Orthogonal Transformation of Word Vectors

**Authors:** Omar E. Rakha and [Hazem M. Abbas](https://md.research.elamur.ai/free/en/authors/hazem-m-abbas/) · **Year:** 2025 · **Venue:** arXiv:2503.14755 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2503.14755) · [PDF](https://arxiv.org/pdf/2503.14755) · [DOI 10.48550/arxiv.2503.14755](https://doi.org/10.48550/arxiv.2503.14755) · [arXiv 2503.14755](https://arxiv.org/abs/2503.14755) · [OpenAlex](https://openalex.org/W6910713492)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses the lack of labelled NER data in most languages. It asks whether a tagger trained only on English can find entities in another language with no target-language training or fine-tuning. The authors build a standard pipeline: word embeddings feed a Bidirectional LSTM, and a CRF layer produces IOB tags. They add one extra module before the encoder. When the input is not English, each word vector is multiplied by an orthogonal linear matrix that maps it into the English embedding space, so the frozen English model sees familiar inputs. The model is trained on WikiNER English and tested on the Arabic ANER dataset. On Arabic, raw vectors give near-zero scores, while aligned vectors reach a usable average F1 of 0.43. The main point is scalability: adding a language needs only a mapping matrix, not new annotation or retraining. Open-source matrices already exist for 78 languages.

## Key points
- Architecture: word embedding → embedding transformation (orthogonal matrix, target → English) → BiLSTM → CRF. It was implemented in Keras, and the code was released on GitHub.
- On WikiNER English, average P/R/F1 is 0.84/0.82/0.83. PER is the strongest class (F1 0.91), and MISC and ORG are the weakest (both 0.76).
- On ANER Arabic without alignment, average F1 is 0.02, which is essentially chance.
- On ANER Arabic with alignment, average P/R/F1 is 0.58/0.36/0.43. PER reaches F1 0.59 and LOC 0.57, but ORG (0.12) and MISC (0.06) nearly fail, and recall is consistently lower than precision.
- A grid search compared BiLSTM with a unidirectional LSTM and varied the number of units. BiLSTM was clearly better on English but unstable on aligned Arabic, and about 256 units was chosen.

## Interesting ideas
- Put a swappable vector-mapping step in front of a frozen trained model, so new input spaces can be adapted without touching the model weights.
- Always report the "no alignment" run alongside the aligned one, so the gain is measured against a model that should know nothing.
- Break the scores down per class, because the average hides that some categories (ORG, MISC) collapse under transfer while others (PER, LOC) survive.
- Restrict the mapping to be orthogonal so that distances and angles between word vectors are preserved and only the orientation of the space changes.
- Tune hyperparameters on both the source data and the transferred target data, since the best choice for one (BiLSTM on English) can be unstable on the other.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): gparse could accept embeddings from a newer or different vector model through a learned orthogonal map, avoiding a full retrain each time the embedding source changes.
- **I2** (learn): every transfer or repair experiment could include a similar deliberately broken input condition as a negative control, which shows that improvements are not artefacts of the scorer.
- **I3** (world): entity-type accuracy for the world of a text should be tracked per type, to show which kinds of entities lose quality when the text domain shifts.
- **I4** (back): when aligning vector representations of meta-world concepts with surface vocabularies for generation or translation, a distance-preserving map keeps neighbourhood structure explainable and easy to check.
- **I5** (learn): model selection for parsers trained on silver data could be checked on an out-of-domain set too, not only on the training-distribution dev set.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · 101 works

## Cite

Cite as: Omar E. Rakha and Hazem M. Abbas. 2025. Language Independent Named Entity Recognition via Orthogonal Transformation of Word Vectors. arXiv:2503.14755.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{rakha2025language,
  title = {{Language Independent Named Entity Recognition via Orthogonal Transformation of Word Vectors}},
  author = {Omar E. Rakha and Hazem M. Abbas},
  year = {2025},
  doi = {10.48550/arxiv.2503.14755},
  eprint = {2503.14755},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2503.14755}
}
```

- [← previous Is It JUST Semantics? A Case Study of Discourse Particle Understanding in LLMs](https://md.research.elamur.ai/free/en/papers/sheffield-2025-is-it-textit-just-semantics/)
- [next → Leveraging AI to Bridge Classical Arabic and Modern Standard Arabic for Text Simplification](https://md.research.elamur.ai/free/en/papers/altammami-2025-leveraging-ai-bridge-classical-arabic/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/)

## References

23 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jiafeng Guo, Gu Xu, Xueqi Cheng, and Hang Li. Named entity recognition in query. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 267–274. ACM, 2009. [find](https://scholar.google.com/scholar?q=Jiafeng+Guo%2C+Gu+Xu%2C+Xueqi+Cheng%2C+and+Hang+Li%2E+Named+entity+recognition+in+query%2E+In+Proceedings+of+the+32nd+international+ACM+SIGIR+conference+on+Research+and+development+in+information+retrieval%2C+pag)
2. Diego Mollá, Menno van Zaanen, and Steve Cassidy. Named entity recognition in question answering of speech data. In Nathalie Colineau and Mark Dras, editors, Proceedings of the Australasian Language Technology Workshop 2007, pages 57–65, Melbourne, Australia, December 2007. [find](https://scholar.google.com/scholar?q=Diego+Molla%CC%81%2C+Menno+van+Zaanen%2C+and+Steve+Cassidy%2E+Named+entity+recognition+in+question+answering+of+speech+data%2E+In+Nathalie+Colineau+and+Mark+Dras%2C+editors%2C+Proceedings+of+the+Australasian+Language+)
3. Yassine Benajiba, Paolo Rosso, and José Benedı́ruiz. Anersys: An arabic named entity recognition system based on maximum entropy. Computational Linguistics and Intelligent Text Processing, pages 143–153, 2007. [find](https://scholar.google.com/scholar?q=Yassine+Benajiba%2C+Paolo+Rosso%2C+and+Jose%CC%81+Bened%C4%B1%CC%81ruiz%2E+Anersys%3A+An+arabic+named+entity+recognition+system+based+on+maximum+entropy%2E+Computational+Linguistics+and+Intelligent+Text+Processing%2C+pages+143%E2%80%93)
4. Zhiheng Huang, Wei Xu, and Kai Yu. [Bidirectional lstm-crf models for sequence tagging](https://md.research.elamur.ai/free/en/papers/huang-2015-bidirectional-lstm-crf-models-sequence/). arXiv preprint arXiv:1508.01991, 2015. [↗](https://arxiv.org/abs/1508.01991)
5. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. [Neural architectures for named entity recognition](https://md.research.elamur.ai/free/en/papers/lample-2016-neural-architectures-named-entity-recognition/). arXiv preprint arXiv:1603.01360, 2016. [↗](https://arxiv.org/abs/1603.01360)
6. Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. Speech recognition with deep recurrent neural networks. In Acoustics, speech and signal processing (icassp), 2013 ieee international conference on, pages 6645–6649. IEEE, 2013. [↗](https://doi.org/10.48550/arxiv.1303.5778)
7. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, page 282–289, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc. [find](https://scholar.google.com/scholar?q=John+D%2E+Lafferty%2C+Andrew+McCallum%2C+and+Fernando+C%2E+N%2E+Pereira%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E+In+Proceedings+of+the+Eighteenth+International)
8. Lance A. Ramshaw and Mitchell P. Marcus. [Text chunking using transformation-based learning](https://md.research.elamur.ai/free/en/papers/ramshaw-1995-text-chunking-transformation-based-learning/). CoRR, cmp-lg/9505040, 1995.
9. Michal Ziemski, Marcin Junczys-Dowmunt, and Bruno Pouliquen. The united nations parallel corpus v1. 0. In LREC, 2016. — [The United Nations Parallel Corpus v1.0](https://md.research.elamur.ai/free/en/papers/ziemski-2016-united-nations-parallel-corpus-v1/)
10. Kareem Darwish. Named entity recognition using cross-lingual resources: Arabic as an example. In ACL (1), pages 1558–1567, 2013. [find](https://scholar.google.com/scholar?q=Kareem+Darwish%2E+Named+entity+recognition+using+cross%2Dlingual+resources%3A+Arabic+as+an+example%2E+In+ACL+%281%29%2C+pages+1558%E2%80%931567%2C+2013%2E)
11. Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, Sören Auer, and Chris Bizer. DBpedia - a large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web Journal, 2014. — [DBpedia – A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia](https://md.research.elamur.ai/free/en/papers/lehmann-2015-dbpedia-large-scale-multilingual-knowledge/) [↗](https://doi.org/10.3233/sw-140134)
12. Zhilin Yang, Ruslan Salakhutdinov, and William Cohen. Multi-task cross-lingual sequence tagging from scratch. arXiv preprint arXiv:1603.06270, 2016. [↗](https://arxiv.org/abs/1603.06270)
13. Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014. [↗](https://arxiv.org/abs/1412.3555)
14. Zellig S Harris. Distributional structure. Word, 10(2-3):146–162, 1954. [find](https://scholar.google.com/scholar?q=Zellig+S+Harris%2E+Distributional+structure%2E+Word%2C+10%282%2D3%29%3A146%E2%80%93162%2C+1954%2E)
15. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In Advances in neural information processing systems, pages 3111–3119, 2013. [↗](https://doi.org/10.48550/arxiv.1310.4546)
16. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. [Enriching word vectors with subword information](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/). arXiv preprint arXiv:1607.04606, 2016. [↗](https://arxiv.org/abs/1607.04606)
17. Samuel L Smith, David HP Turban, Steven Hamblin, and Nils Y Hammerla. [Offline bilingual word vectors, orthogonal transformations and the inverted softmax](https://md.research.elamur.ai/free/en/papers/smith-2017-offline-bilingual-word-vectors-orthogonal/). arXiv preprint arXiv:1702.03859, 2017. [↗](https://arxiv.org/abs/1702.03859)
18. Joel Nothman, Nicky Ringland, Will Radford, Tara Murphy, and James R. Curran. Learning multilingual named entity recognition from Wikipedia. Artificial Intelligence, 194:151–175, 2012. [↗](https://doi.org/10.1016/j.artint.2012.03.006)
19. Chao Xing, Dong Wang, Chao Liu, and Yiye Lin. [Normalized word embedding and orthogonal transform for bilingual word translation](https://md.research.elamur.ai/free/en/papers/xing-2015-normalized-word-embedding-orthogonal-transform/). In HLT-NAACL, pages 1006–1011, 2015.
20. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. [↗](https://doi.org/10.1162/neco.1997.9.8.1735)
21. Alex Graves, Navdeep Jaitly, and Abdel rahman Mohamed. Hybrid speech recognition with deep bidirectional lstm. 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, pages 273–278, 2013. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Navdeep+Jaitly%2C+and+Abdel+rahman+Mohamed%2E+Hybrid+speech+recognition+with+deep+bidirectional+lstm%2E+2013+IEEE+Workshop+on+Automatic+Speech+Recognition+and+Understanding%2C+pages+273%E2%80%93278%2C+2013)
22. G David Forney. The viterbi algorithm. Proceedings of the IEEE, 61(3):268–278, 1973. [↗](https://doi.org/10.1109/proc.1973.9030)
23. François Chollet et al. Keras. [https://github.com/fchollet/keras](https://github.com/fchollet/keras), 2015. [↗](https://github.com/fchollet/keras)

*Updated 24 October 2026*