---
title: "Methodology of Adapting Large English Language Models for"
description: "The authors propose a cheap way to adapt an English model to a target culture using instruction tuning… 15 references, cited by 1 paper here."
authors: ["Wenjing Zhang", "Siqi Xiao", "Xuejiao Lei", "Ning Wang", "Huazheng Zhang", "Meijuan An", "Bikun Yang", "Zhaoxiang Liu", "Kai Wang", "Shiguo Lian"]
year: 2024
venue: "arXiv:2406.18192"
source: "https://arxiv.org/abs/2406.18192"
doi: "10.48550/arxiv.2406.18192"
arxiv: "2406.18192"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2024-methodology-adapting-large-english-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2024-methodology-adapting-large-english-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/14/) / Methodology of Adapting Large English Language Models for Specific Cultural Contexts

# Methodology of Adapting Large English Language Models for Specific Cultural Contexts

**Authors:** Wenjing Zhang, Siqi Xiao, Xuejiao Lei, [Ning Wang](https://md.research.elamur.ai/free/en/authors/ning-wang/), Huazheng Zhang, Meijuan An, Bikun Yang, [Zhaoxiang Liu](https://md.research.elamur.ai/free/en/authors/zhaoxiang-liu/), [Kai Wang](https://md.research.elamur.ai/free/en/authors/kai-wang/) and [Shiguo Lian](https://md.research.elamur.ai/free/en/authors/shiguo-lian/) · **Year:** 2024 · **Venue:** arXiv:2406.18192 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2406.18192) · [PDF](https://arxiv.org/pdf/2406.18192) · [DOI 10.48550/arxiv.2406.18192](https://doi.org/10.48550/arxiv.2406.18192) · [arXiv 2406.18192](https://arxiv.org/abs/2406.18192) · [OpenAlex](https://openalex.org/W4400142110)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/zhang-2024-methodology-adapting-large-english-language/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Leading LLMs are trained mostly on English, so they lack local knowledge and can clash with local values when used in another culture. The authors propose a cheap way to adapt an English model to a target culture using instruction tuning alone, with no further pre-training. Their case study adapts LLaMA3-8B to Chinese. They build instruction data from open-source Chinese sets, filter it by hand, and have GPT rewrite weak answers. The model then goes through two gated stages. Each stage starts with an evaluation, and full-parameter fine-tuning runs only if a deficit shows up. The first stage targets knowledge and capabilities, the second safety and values. Both stages improved the target-culture scores, and the model reportedly kept its original strengths. Safety tuning, however, slightly reduced the knowledge gains. Knowledge tuning on its own already improved safety by a large margin. The work suggests that a mid-sized English model can be localized quickly on modest hardware.

## Key points
- Knowledge data comes from alpaca_gpt4_data_zh.json (52K GPT-4 instruction examples built on Alpaca prompts translated into Chinese) and RefGPTDatasetV1CN.json. Capability is evaluated on the A-Eval benchmark.
- There are three model variants: LLaMA3-8B (base), LLaMA3-8B-KG (knowledge-tuned) and LLaMA3-8B-SAFE (knowledge plus safety tuning).
- On multiple-choice risk identification, overall accuracy rose from 19.63% (base) to 69.30% (KG) and 72.06% (SAFE). The test spans five kinds of risk: discriminatory content, conflicts with core values, illegal business practices, breaches of people's rights, and the safety rules that particular services must meet.
- On open risky questions, the models are scored with two refusal-rate metrics (RR-1, RR-2) and a harm rate (HR).
  - SAFE had the best refusal and harm rates.
  - KG was best at responsible, guiding answers, likely because safety tuning makes the model more conservative.
- Knowledge tuning used 100K examples for 2 epochs on 4 servers with 8 A100 GPUs each, and took 15.67 hours.

## Interesting ideas
- Each fine-tuning stage runs only after an evaluation finds a gap, and it repeats until a preset bar is reached, so effort goes only where it is needed.
- Existing translated or machine-made data is reused, but humans first remove items tied to the source culture, and an LLM regenerates only the answers that fail. This avoids building a dataset from scratch.
- The authors check whether a later adaptation erodes an earlier one by comparing per-subtask scores across successive model versions, and they find real interference.
- Safety is tested in two separate ways: recognizing risky content (multiple choice) and producing a good response (refusal, harm and guidance rates). This separates "knows" from "does".

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Training or repair-rule induction for any component could be triggered and stopped by per-component evaluation thresholds rather than run on a fixed schedule.
- **I2** (learn): Silver data could be cleaned the same way: drop items that do not fit the target domain, then send only the failing items to an LLM teacher tier instead of relabeling everything.
- **I3** (learn): Each new training or repair step should be scored per subtask against the previous version, so that regressions caused by the new step are caught as a negative control.
- **I4** (answer): The readers could be scored separately on spotting an unanswerable or tricky question and on giving the right kind of answer to it, which shows where a failure happens.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [China United Network Communications Group](https://md.research.elamur.ai/free/en/companies/china-united-network-communications-group/) — Company, Beijing, China · 2 works · 2 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works

## Cite

Cite as: Wenjing Zhang, Siqi Xiao, Xuejiao Lei, Ning Wang, Huazheng Zhang, Meijuan An, Bikun Yang, Zhaoxiang Liu, Kai Wang and Shiguo Lian. 2024. Methodology of Adapting Large English Language Models for Specific Cultural Contexts. arXiv:2406.18192.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zhang2024methodology,
  title = {{Methodology of Adapting Large English Language Models for Specific Cultural Contexts}},
  author = {Wenjing Zhang, Siqi Xiao, Xuejiao Lei, Ning Wang, Huazheng Zhang, Meijuan An, Bikun Yang, Zhaoxiang Liu, Kai Wang and Shiguo Lian},
  year = {2024},
  doi = {10.48550/arxiv.2406.18192},
  eprint = {2406.18192},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2406.18192}
}
```

- [← previous MenakBERT — Hebrew Diacriticizer](https://md.research.elamur.ai/free/en/papers/cohen-2024-menakbert-hebrew-diacriticizer/)
- [next → NLP-LISAC at SemEval-2024 Task 1: Transformer-based approaches for Determining Semantic Textual Relatedness](https://md.research.elamur.ai/free/en/papers/benlahbib-2024-nlp-lisac-semeval-2024-task/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/14/)

## References

15 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774 (2023) [↗](https://arxiv.org/abs/2303.08774)
2. Cheng, Q., Sun, T., Zhang, W., Wang, S., Liu, X., Zhang, M., He, J., Huang, M., Yin, Z., Chen, K., Qiu, X.: Evaluating hallucinations in chinese large language models. CoRR abs/2310.03368 (2023). https://doi.org/10.48550/arXiv.2310.03368, https://doi.org/10.48550/arXiv.2310.03368 [↗](https://doi.org/10.48550/arxiv.2310.03368)
3. Chiang, W.L., Zheng, L., Sheng, Y., Angelopoulos, A.N., Li, T., Li, D., Zhang, H., Zhu, B., Jordan, M., Gonzalez, J.E., Stoica, I.: [Chatbot arena: An open platform for evaluating llms by human preference](https://md.research.elamur.ai/free/en/papers/chiang-2024-chatbot-arena-open-platform-evaluating/). ArXiv abs/2403.04132 (2024) [↗](https://arxiv.org/abs/2403.04132)
4. Conover, M., Hayes, M., Mathur, A., Xie, J., Wan, J., Shah, S., Ghodsi, A., Wendell, P., Zaharia, M., Xin, R.: Free dolly: Introducing the world’s first truly open instruction-tuned llm, [https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm) [↗](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm)
5. Honovich, O., Scialom, T., Levy, O., Schick, T.: [Unnatural instructions: Tuning language models with (almost) no human labor](https://md.research.elamur.ai/free/en/papers/honovich-2023-unnatural-instructions-tuning-language-models/). ArXiv abs/2212.09689 (2022) Adapting Large English Language Models for Specific Cultural Contexts [↗](https://arxiv.org/abs/2212.09689)
6. Ji, Y., Deng, Y., Gong, Y., Peng, Y., Niu, Q., Zhang, L., Ma, B., Li, X.: [Exploring the impact of instruction data scaling on large language models: An empirical study on real-world use cases](https://md.research.elamur.ai/free/en/papers/ji-2023-exploring-impact-instruction-data/). ArXiv abs/2303.14742 (2023) [↗](https://arxiv.org/abs/2303.14742)
7. Lian, S., Zhao, K., Liu, X., Lei, X., Yang, B., Zhang, W., Wang, K., Liu, Z.: What is the best model? application-driven evaluation for large language models. arXiv preprint arXiv:2406.10307 (2024) [↗](https://arxiv.org/abs/2406.10307)
8. Peng, B., Li, C., He, P., Michel Galley, J.G.: [Instruction tuning with gpt-4](https://md.research.elamur.ai/free/en/papers/peng-2023-instruction-tuning-gpt-4/). arXiv preprint arXiv:2304.03277 (2023) [↗](https://arxiv.org/abs/2304.03277)
9. Sun, H., Zhang, Z., Deng, J., Cheng, J., Huang, M.: [Safety assessment of chinese large language models](https://md.research.elamur.ai/free/en/papers/sun-2023-safety-assessment-chinese-large-language/). arXiv preprint arXiv:2304.10436 (2023) [↗](https://arxiv.org/abs/2304.10436)
10. Team, G., Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A.M., Hauth, A., et al.: [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805 (2023) [↗](https://arxiv.org/abs/2312.11805)
11. Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N.A., Khashabi, D., Hajishirzi, H.: Self-instruct: Aligning language models with self-generated instructions. In: Annual Meeting of the Association for Computational Linguistics (2022) [find](https://scholar.google.com/scholar?q=Wang%2C+Y%2E%2C+Kordi%2C+Y%2E%2C+Mishra%2C+S%2E%2C+Liu%2C+A%2E%2C+Smith%2C+N%2EA%2E%2C+Khashabi%2C+D%2E%2C+Hajishirzi%2C+H%2E%3A+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+In%3A+Annual+Meeting+of+the+Association+for)
12. Xu, C., Sun, Q., Zheng, K., Geng, X., Zhao, P., Feng, J., Tao, C., Jiang, D.: Wizardlm: Empowering large language models to follow complex instructions. ArXiv abs/2304.[12244 (2023)](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/) [↗](https://arxiv.org/abs/2304.12244)
13. Xu, C., Guo, D., Duan, N., McAuley, J.: [Baize: An open-source chat model with parameter-efficient tuning on self-chat data](https://md.research.elamur.ai/free/en/papers/xu-2023-baize-open-source-chat-model/). arXiv preprint arXiv:2304.01196 (2023) [↗](https://arxiv.org/abs/2304.01196)
14. Yang, D., Yuan, R., Fan, Y., , Yang, Y., Wang, Z., Wang, S.: Refgpt: Reference-to-dialogue by gpt and for gpt. [https://github.com/ziliwangnlp/RefGPT](https://github.com/ziliwangnlp/RefGPT) (2023) [↗](https://github.com/ziliwangnlp/RefGPT)
15. Zhang, W., Lei, X., Liu, Z., An, M., Yang, B., Zhao, K., Wang, K., Lian, S.: Chisafetybench: A chinese hierarchical safety benchmark for large language models. arXiv preprint arXiv:2406.10311 (2024) [↗](https://arxiv.org/abs/2406.10311)

*Updated 2 November 2026*