Research · LLM Safety, Security & Privacy

Safety and security across the LLM lifecycle

From the data models learn from, to the models themselves, to the systems they are deployed in.

Large language models can be exploited at every stage of their lifecycle. My research characterizes these attack surfaces, including unauthorized training on private data, capability extraction through APIs, and attacks on retrieval, multi-agent, and reasoning pipelines, and develops safeguards designed to remain effective against adaptive adversaries while interfering minimally with benign use.

01 Data 02 Model 03 Deployment

6featured papers
3lifecycle layers: data, model, deployment
2data modalities protected: vision-language and text
5media outlets covering the CoT study

Overview

Three layers, three attack surfaces

I organize my work around the lifecycle of an LLM. Each layer exposes a different attack surface and calls for different safeguards. Select a paper to jump to it.

Layer 01 Data

Can data owners prevent their data from being learned without consent?

Threat Unauthorized fine-tuning on scraped image–text and text data
Safeguard Unlearnable examples that stay useful to people but not to models MMGuard TextCloak

Layer 02 Model

Can a deployed model's capabilities be protected from extraction?

Threat Capability extraction by distilling API responses into a student model
Safeguard Responses a student can fit without acquiring the underlying capability DistilShield

Layer 03 Deployment

Do LLM systems behave as intended in adversarial, open-ended settings?

Retrieval Poisoned knowledge bases LIAR
Multi-agent systems Harmful content propagating between agents The Wolf Within
Reasoning Chain-of-thought that fails out of distribution CoT Mirage

Layer 01 · Data security & privacy

Protecting data from unauthorized learning

Watermarking and machine unlearning act only after a model has already been trained on the data. I instead study unlearnable examples: perturbations that preserve the data's utility for people while preventing models trained on it without authorization from generalizing. The aim is protection that holds across modalities, model architectures, and threat models, from white-box to black-box.

Preprint 2026 Defense Vision-language models

MMGuard

To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model

Chengshuai Zhao, Zhen Tan, Dawei Li, Zhiyuan Yu, Huan Liu

Can data owners protect image–text data before unauthorized fine-tuning happens, rather than after?

Threat: Image–text pairs scraped from the web are used to fine-tune large vision-language models (LVLMs) without consent. Unlearning and watermarking act only after the infringement has occurred.
Method: Imperceptible perturbations create an optimization shortcut, and a cross-modal binding disruption shifts LVLM attention so the training target is bound to the noise, with theoretical guarantees. Ensemble training improves cross-model transferability.
Result: Effective, stealthy, and robust protection across 9 open-source LVLMs and 6 datasets under white-box, gray-box, and black-box threat models.

arXiv PDF

MMGuard framework: image and text perturbations with cross-modal bind disruption inside an LVLM — Framework overview · click to zoom

Preprint 2026 Defense Language models

TextCloak

Safeguarding Data from Unauthorized LLM Fine-Tuning via RL-Driven Unlearnable Examples

Chengshuai Zhao, Pingchuan Ma, Dawei Li, Bohan Jiang, Zhiyuan Yu, Zhen Tan, Huan Liu

Can text remain natural and useful to readers while providing no benefit to an LLM fine-tuned on it without authorization?

Threat: Existing unlearnable text targets discriminative classifiers through class-specific cues and does not carry over to open-ended LLM generation.
Method: A generative policy rewrites clean text into unlearnable examples while preserving semantic fidelity and fluency. GRPO-UE rewards each rewrite by the degradation it induces in fine-tuned surrogate LLMs, a bi-level optimization that discovers protective patterns beyond class-specific cues.
Result: Consistently impairs unauthorized fine-tuning across 6 datasets and 9 LLMs while preserving utility for legitimate use, and remains effective across architectures, training configurations, and adaptive attacks.

arXiv PDF

TextCloak: an LLM fine-tuned on original text performs well, while one fine-tuned on protected text performs poorly — Protected text retains utility but derails unauthorized fine-tuning

Layer 02 · Model security

Protecting model capabilities from extraction

Frontier models are costly to build, yet every API response exposes part of their capability. Rather than making responses harder to fit, I study how to let a student fit them while preventing the underlying capability from transferring, with minimal impact on benign use.

Under Review Defense Anti-distillation

DistilShield

Rethinking Anti-distillation of Large Language Models: An Unlearnable Examples Perspective

Chengshuai Zhao, Fan Zhang, Dawei Li, Bohan Jiang, Pingchuan Ma, Zhiyuan Yu, Zhen Tan, Huan Liu

Can a student fit a teacher's responses yet fail to acquire the capability behind them?

Threat: Adversaries query a proprietary LLM and distill its responses into a student model. Existing anti-distillation methods try to make those responses difficult to fit.
Method: An Unlearnable Converter embeds easy-to-fit shortcuts in the teacher's reasoning and acts across the distillation trajectory: shortcut induction, shortcut binding (steering student updates away from clean knowledge transfer), and shortcut consolidation under the student's own rollouts. The converter is optimized with Anti-GDPO, a bi-level algorithm.
Result: The first anti-distillation approach from the unlearnable-examples perspective: a plug-and-play defense layer that suppresses knowledge distillation while preserving the utility of teacher responses, evaluated on 6 datasets and 8 LLMs.

OpenReview

DistilShield pipeline: an unlearnable converter rewrites teacher reasoning, optimized by Anti-GDPO with shortcut induction, binding, consolidation, and utility rewards — Pipeline and Anti-GDPO reward design · click to zoom

Layer 03 · Deployment safety & security

Stress-testing LLM systems in deployment

Deployed LLMs operate inside larger systems: they retrieve from open knowledge bases, communicate with other agents, and present their reasoning to users. Each interface is a potential attack surface or a source of misplaced trust. I characterize these failure modes through realistic attacks and controlled evaluations.

Retrieval-augmented generation

EMNLP 2024 Attack Knowledge poisoning

LIAR

Glue Pizza and Eat Rocks: Exploiting Vulnerabilities in Retrieval-Augmented Generative Models

Zhen Tan*, Chengshuai Zhao*, Raha Moraffah, Yifan Li, Song Wang, Jundong Li, Tianlong Chen, Huan Liu

Can an adversary with no knowledge of user queries, the knowledge base, or model parameters still steer what a RAG system generates?

Threat: RAG systems retrieve from openly writable sources such as web scrapes and user-contributed data pools, exposing a data-poisoning attack surface.
Method: LIAR (expLoitative bI-level rAg tRaining) jointly optimizes adversarial content against the retriever and the generator through bi-level training, using access to the retriever alone.
Result: Crafted uploads alone can change model behavior in this realistic setting, motivating integrity safeguards in the design and deployment of RAG systems.

Paper PDF

LIAR threat model: an adversary injects documents into a knowledge base that a RAG retriever and LLM generator consume — Threat model and bi-level training · click to zoom

Multi-agent systems

CVPR 2024 Workshop Attack Multi-agent MLLMs

The Wolf Within

Covert Injection of Malice into MLLM Societies via an MLLM Operative

Zhen Tan*, Chengshuai Zhao*, Raha Moraffah, Yifan Li, Yu Kong, Tianlong Chen, Huan Liu

Can a single compromised agent induce harmful behavior throughout a society of multimodal agents?

Threat: Multimodal LLM agents increasingly collaborate in networks, exchanging prompts and images to complete complex tasks.
Method: Manipulate one MLLM agent so that it generates prompts that, in turn, induce other agents to produce harmful content: an indirect, covert propagation path rather than a direct jailbreak.
Result: The induced prompts transfer across agents, circulating dangerous instructions and misinformation through ordinary inter-agent communication, a multi-agent failure mode that calls for dedicated detection and mitigation.

arXiv PDF Code

The Wolf Within: a manipulated MLLM agent infects other agents in an MLLM society — Malice propagating through an agent society · click to zoom

Reasoning · chain-of-thought faithfulness

ACL 2026 Findings Evaluation Chain-of-thought In the press

CoT Mirage

Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

Chengshuai Zhao, Zhen Tan, Pingchuan Ma, Dawei Li, Bohan Jiang, Yancheng Wang, Yingzhen Yang, Huan Liu

Does a model's chain of thought reflect reasoning that generalizes, or patterns that only hold in distribution?

Setting: Chain-of-thought outputs resemble deliberate inference, which encourages users to trust them. We examine whether that reasoning holds up under distribution shift.
Method: DataAlchemy, an isolated and controlled environment for training LLMs from scratch, systematically probes CoT along three dimensions: task, length, and format.
Result: CoT reasoning is a brittle mirage: its effectiveness is bounded by the discrepancy between training data and test queries, and it degrades into fluent but incorrect reasoning beyond the training distribution.

PDF arXiv Code 🤗 HF Papers Video

DataAlchemy: a controlled environment probing chain-of-thought reasoning under task, length, and format shifts — The DataAlchemy environment · click to zoom

Covered by The New Yorker Ars Technica VentureBeat SF Examiner The Decoder

Outlook

Open questions

Directions I am pursuing next, one for each layer of the lifecycle.

Layer 01 · Data
Can data protections remain effective against adaptive adversaries and against models that have not been trained yet?
Layer 02 · Model
Can we prevent capability extraction without degrading the experience of legitimate users?
Layer 03 · Deployment
How can we detect and contain failures that propagate across retrieval, agents, and reasoning before they compound?

If you work on related problems, I would be glad to hear from you.

All publications CV