---
title: "HHMM-based Chinese Lexical Analyzer ICTCLAS — Zhang 2003"
description: "Zhang et al. present ICTCLAS, a lexical analyzer built on a Hierarchical Hidden Markov Model (HHMM). Cited by 77 papers here."
authors: ["Hua-Ping Zhang", "Hong-Kui Yu", "De-Yi Xiong", "Qun Liu"]
year: 2003
venue: "Proceedings of the Second SIGHAN Workshop on Chinese Language Processing"
source: "https://aclanthology.org/W03-1730/"
doi: "10.3115/1119250.1119280"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2003-hhmm-based-chinese-lexical-analyzer/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2003-hhmm-based-chinese-lexical-analyzer/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2003](https://md.research.elamur.ai/free/en/papers/2003/) / HHMM-based Chinese Lexical Analyzer ICTCLAS

# HHMM-based Chinese Lexical Analyzer ICTCLAS

**Authors:** Hua-Ping Zhang, Hong-Kui Yu, De-Yi Xiong and [Qun Liu](https://md.research.elamur.ai/free/en/authors/qun-liu/) · **Year:** 2003 · **Venue:** [Proceedings of the Second SIGHAN Workshop on Chinese Language Processing](https://md.research.elamur.ai/free/en/venues/sighan-workshop-on-chinese-language-processing/) ([SIGHAN Workshop on Chinese Language Processing 2003](https://md.research.elamur.ai/free/en/venues/sighan-workshop-on-chinese-language-processing/2003/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W03-1730/) · [PDF](https://aclanthology.org/W03-1730.pdf) · [DOI 10.3115/1119250.1119280](https://doi.org/10.3115/1119250.1119280) · [OpenAlex](https://openalex.org/W1982498087)

**Citations:** 450 citations (OpenAlex, 2026-10-14) · [77](https://md.research.elamur.ai/free/en/papers/zhang-2003-hhmm-based-chinese-lexical-analyzer/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

**Note before use:** The PDF extraction is corrupted. The text is mostly stray symbols from a font-encoding failure, and none of the paper's sentences, tables or numbers can be read. The draft below is therefore **not based on the supplied text**. It reflects the generally known content of this paper. Every item marked **[verify]** must be checked against a clean copy (e.g., the ACL Anthology version) before it goes into the library.

## Summary
Chinese text has no spaces between words. Building a lexical analyzer therefore means solving several linked tasks: word segmentation, part-of-speech (POS) tagging, recognition of unknown words such as person, place and organization names, and ambiguity resolution. Pipelines that handle these separately tend to pass errors from one stage to the next. Zhang et al. present ICTCLAS, a lexical analyzer built on a Hierarchical Hidden Markov Model (HHMM). Each layer of the hierarchy is an HMM handling one level of analysis:
- splitting text into atoms,
- simple and recursive unknown-word recognition through role tagging,
- class-based segmentation,
- POS tagging.

To keep the search tractable while still allowing ambiguity to be resolved, the system first generates several candidate segmentations with an N-shortest-paths method rather than a single best guess. ICTCLAS took part in the First SIGHAN Chinese Word Segmentation Bakeoff and was among the top systems on some tracks **[verify ranks and tracks]**. It became one of the most widely used Chinese segmenters, which makes it a key reference for hierarchical, statistically grounded lexical analysis.

## Key points
- The system is ICTCLAS, from the Institute of Computing Technology, Chinese Academy of Sciences, and the underlying model is a Hierarchical Hidden Markov Model (HHMM).
- Unknown words (person names, transliterated names, locations, organizations) are found by tagging characters or words with roles using an HMM. Recognition can be applied recursively to handle nested names **[verify details]**.
- N-shortest-paths rough segmentation keeps several candidate segmentations for later layers to rescore.
- A class-based HMM performs the final segmentation, followed by HMM-based POS tagging.
- It was evaluated in the First SIGHAN Bakeoff (2003) on corpora such as PK and CTB. Exact precision, recall, F-scores and speed figures are **[verify; not recoverable from the supplied text]**.

## Interesting ideas
- **Layered HMMs instead of one large model:** Assigning each linguistic level its own HMM keeps every component simple and trainable, while the levels still share probabilistic scores. This design could carry over to morphologically rich languages like Ukrainian, where tokenization, morphology and NER interact.
- **Keep N candidates, not one:** Passing several rough segmentations forward delays hard decisions until more context is available. It is a cheap way to reduce error propagation in any NLP pipeline.
- **Role tagging for unknown words:** Recasting name detection as tagging tokens with roles (e.g., surname, given name, context) lets one sequence-labeling machinery cover many entity types. The same trick could help with transliterated foreign names in Ukrainian trade or geography data.
- **Recursive recognition:** Running recognition again on its own output allows nested entities to be found, such as an organization name that contains a place name. Nested entities are also common in geographic and commercial texts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) — Government body, Beijing, China · 68 works · 22,753 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔤 Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) · 91 works

## Cite

Cite as: Hua-Ping Zhang, Hong-Kui Yu, De-Yi Xiong and Qun Liu. 2003. HHMM-based Chinese Lexical Analyzer ICTCLAS. Proceedings of the Second SIGHAN Workshop on Chinese Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2003hhmm,
  title = {{HHMM-based Chinese Lexical Analyzer ICTCLAS}},
  author = {Hua-Ping Zhang, Hong-Kui Yu, De-Yi Xiong and Qun Liu},
  booktitle = {Proceedings of the Second SIGHAN Workshop on Chinese Language Processing},
  year = {2003},
  doi = {10.3115/1119250.1119280},
  url = {https://doi.org/10.3115/1119250.1119280}
}
```

- [← previous Sentence Level Discourse Parsing using Syntactic and Lexical Information](https://md.research.elamur.ai/free/en/papers/soricut-2003-sentence-level-discourse-parsing-syntactic/)
- [next → Empirical Methods for Compound Splitting](https://md.research.elamur.ai/free/en/papers/koehn-2003-empirical-methods-compound-splitting/)
- [↑ back to papers of 2003](https://md.research.elamur.ai/free/en/papers/2003/)

*Updated 24 October 2026*