---
title: "Defensive Dual Masking for Robust Adversarial Defense (2026)"
description: "This paper presents Defensive Dual Masking (DDM), which applies [MASK] tokens at two stages. 11 references."
authors: ["Wangli Yang", "Jie Yang", "Yi Guo", "Johan Barthelemy"]
year: 2026
venue: "Computational Linguistics"
source: "https://aclanthology.org/2026.cl-1.5/"
doi: "10.1162/coli.a.574"
licence: "CC BY-NC-ND 4.0 (Computational Linguistics, MIT Press)"
licence_url: "https://creativecommons.org/licenses/by-nc-nd/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yang-2026-defensive-dual-masking-robust-adversarial/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yang-2026-defensive-dual-masking-robust-adversarial/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) / Defensive Dual Masking for Robust Adversarial Defense

# Defensive Dual Masking for Robust Adversarial Defense

**Authors:** Wangli Yang, [Jie Yang](https://md.research.elamur.ai/free/en/authors/jie-yang/), Yi Guo and Johan Barthelemy · **Year:** 2026 · **Venue:** [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/) ([Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/cl/2026/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY-NC-ND 4.0 (Computational Linguistics, MIT Press)](https://creativecommons.org/licenses/by-nc-nd/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.cl-1.5/) · [PDF](https://aclanthology.org/2026.cl-1.5.pdf) · [DOI 10.1162/coli.a.574](https://doi.org/10.1162/coli.a.574) · [OpenAlex](https://openalex.org/W4414788354)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Fine-tuned language models can lose most of their accuracy when an attacker makes small edits to the input, such as swapping a character or substituting a synonym. This paper presents Defensive Dual Masking (DDM), which applies [MASK] tokens at two stages. During fine-tuning, extra [MASK] tokens are inserted into each training sample without overwriting any words, and the model is trained only on these padded versions. At inference, a detector scores tokens by how suspicious they are, and the highest-scoring ones are replaced with [MASK]. The model never tries to reconstruct the masked words. It classifies the masked input in a single pass, with no ensembling and no changes to the architecture or loss. The authors support the approach with a theoretical analysis of stability and defense guarantees. They test it on four text-classification benchmarks against four attacks, and also with LLMs. DDM beats strong baselines by 1.4–12.1 accuracy points on average. Because it is cheap and plug-and-play, it is a practical way to harden NLP classifiers.

## Key points
- **Experimental setup:** The benchmarks are SST2, AGNEWS, IMDB and MR. The attacks are TextFooler, BERT-Attack, DeepWordBug and TextBugger, and the base encoder is BERT-base.
- **Masking budget:** The default budget is 20%. Training adds [MASK] tokens equal to 20% of the sequence length, and inference masks the top 20% of suspicious tokens, identified with FGWS.
- **Robustness:** On AGNEWS, accuracy under attack reaches about 82–86% depending on the attack, with attack success rates of roughly 9–11%. Improvements over the strongest baseline are significant at p \< 10⁻³.
- **Cost:** On AGNEWS, training is about 10.9% slower and uses about 18.1% more FLOPs. Classification FLOPs at inference fall, for example from 7.01G to 4.21G on AGNEWS.
- **Origin and code:** DDM extends the authors' earlier MI4D (Hu et al. 2023), which used random masking. The code is available at github.com/wlyang538/DDM.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Western Sydney University](https://md.research.elamur.ai/free/en/universities/western-sydney-university/) — University, Richmond, Australia · 4 works · 63 citations

Also on the byline: [University of Wollongong](https://ror.org/00jtmb277).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Wangli Yang, Jie Yang, Yi Guo and Johan Barthelemy. 2026. Defensive Dual Masking for Robust Adversarial Defense. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{yang2026defensive,
  title = {{Defensive Dual Masking for Robust Adversarial Defense}},
  author = {Wangli Yang, Jie Yang, Yi Guo and Johan Barthelemy},
  journal = {Computational Linguistics},
  year = {2026},
  doi = {10.1162/coli.a.574},
  url = {https://doi.org/10.1162/coli.a.574}
}
```

- [← previous DeepSemantics at SemEval-2026 Task 9: Label-Wise Optimization with Adaptive Focal Loss for Polarization Manifestation Identification](https://md.research.elamur.ai/free/en/papers/tiao-2026-deepsemantics-semeval-2026-task-9/)
- [next → DELTA: Dynamic Layer-Aware Token Attention for Efficient Long-Context Reasoning](https://md.research.elamur.ai/free/en/papers/zarch-2026-delta-dynamic-layer-aware-token/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/)

## References

11 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ankner, Zachary, Naomi Saphra, Davis Blalock, Jonathan Frankle, and Matthew Leavitt. 2024. Dynamic masking rate schedules for MLM pretraining. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 477–487. https… [find](https://scholar.google.com/scholar?q=Ankner%2C+Zachary%2C+Naomi+Saphra%2C+Davis+Blalock%2C+Jonathan+Frankle%2C+and+Matthew+Leavitt%2E+2024%2E+Dynamic+masking+rate+schedules+for+MLM+pretraining%2E+In+Proceedings+of+the+18th+Conference+of+the+European+Cha)
2. Ebrahimi, Javid, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. [HotFlip: White-box adversarial examples for text classification](https://md.research.elamur.ai/free/en/papers/ebrahimi-2018-hotflip-white-box-adversarial-examples/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 31–36. https://doi.org/10.18653/v1/P18 -2006 Freit… [↗](https://doi.org/10.18653/v1/P18)
3. Maas, Andrew L., Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. [Learning word vectors for sentiment analysis](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150.
4. Meng, Zhao, Yihan Dong, Mrinmaya Sachan, and Roger Wattenhofer. 2022. Self-supervised contrastive learning with adversarial perturbations for defending word substitution-based attacks. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 87–101. https://doi.org/10.18653/v1… [↗](https://doi.org/10.18653/v1…)
5. [TextAttack: A framework for adversarial attacks, data augmentation, and adversarial training in NLP](https://md.research.elamur.ai/free/en/papers/morris-2020-textattack-framework-adversarial-attacks-data/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 119–126. https://doi.org/10.18653/v1/2020.emnlp -demos.16 Mosca, Edoardo, Shre… [↗](https://doi.org/10.18653/v1/2020.emnlp)
6. Wang, Boxin, Chejian Xu, Xiangyu Liu, Yu Cheng, and Bo Li. 2022. [SemAttack: Natural textual attacks via different semantic spaces](https://md.research.elamur.ai/free/en/papers/williams-2018-broad-coverage-challenge-corpus-sentence/). In Findings of the Association for Computational Linguistics: NAACL 2022, pages 176–205. https://doi.org/10.18653/v1/2022.findings -naacl.14 Wang, Xiaosen, Yichen Yang,… [↗](https://doi.org/10.18653/v1/2022.findings)
7. Yang, Wangli, Jie Yang, Wanqing Li, and Yi Guo. 2024. ConClue: Conditional clue extraction formultiple choice question answering. In Document Analysis and Recognition - ICDAR 2024, pages 183–198. https://doi.org/10.1007/978-3-031 -70552-6 11 Yang, Yahan, Soham Dan, Dan Roth, and Insup Lee. 2023. In… [↗](https://doi.org/10.1007/978-3-031)
8. Zeng, Jiehang, Jianhan Xu, Xiaoqing Zheng, and Xuanjing Huang. 2023. [Certified robustness to text adversarial attacks by randomized MASK](https://md.research.elamur.ai/free/en/papers/zeng-2023-certified-robustness-text-adversarial-attacks/). Computational Linguistics, 49(2):395–427. https://doi.org/10.1162/coli a 00476 Zhan, Pengwei, Jing Yang, He Wang, Chao Zheng, Xiao Huang, and Liming Wang. 2023… [↗](https://doi.org/10.1162/coli)
9. Zhang, Cenyuan, Xiang Zhou, Yixin Wan, Xiaoqing Zheng, Kai-Wei Chang, and Cho-Jui Hsieh. 2022. Improving the adversarial robustness of NLP models by information bottleneck. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3588–3598. https://doi.org/10.18653/v1/2022.find… [↗](https://doi.org/10.18653/v1/2022.find…)
10. Zhang, Xiaomei, Zhaoxi Zhang, Qi Zhong, Xufei Zheng, Yanjun Zhang, Shengshan Hu, and Leo Yu Zhang. 2023. Masked language model based textual adversarial example detection. In Proceedings of the 2023 ACM Asia Conference on Computer and Communications Security, pages 925–937. https://doi.org/10.1145/3… [find](https://scholar.google.com/scholar?q=Zhang%2C+Xiaomei%2C+Zhaoxi+Zhang%2C+Qi+Zhong%2C+Xufei+Zheng%2C+Yanjun+Zhang%2C+Shengshan+Hu%2C+and+Leo+Yu+Zhang%2E+2023%2E+Masked+language+model+based+textual+adversarial+example+detection%2E+In+Proceedings+of+the+2023+A)
11. Zhao, Jiahao and Wenji Mao. 2023. Generative adversarial training with perturbed token detection for model robustness. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13012–13025. https://doi.org/10.18653/v1/2023 .emnlp-main.804 Zheng, Rui, Bao Rong,… [↗](https://doi.org/10.18653/v1/2023)

*Updated 1 November 2026*