Object Hallucination-Free Reinforcement Unlearning for Vision-Language Models
Kaidi Jia, Yujie Lin, Chengyi Yang, Jiayao Ma, Jinsong Su
TLDR
HFRU is a reinforcement unlearning framework for VLMs that deeply removes sensitive visual knowledge from the vision encoder, preventing object hallucination.
Key contributions
- Proposes HFRU, a reinforcement unlearning framework for Vision-Language Models.
- Operates on the vision encoder for deep, semantic knowledge removal, unlike prior superficial methods.
- Uses a two-stage GRPO-based optimization with a composite reward, including an abstraction reward.
- Achieves over 98% forgetting and retention performance with negligible object hallucination.
Why it matters
Existing VLM unlearning methods are superficial and often cause object hallucination. HFRU offers a novel reinforcement unlearning approach that deeply removes sensitive visual knowledge from the vision encoder, preventing hallucinations. This enables more effective and safer unlearning, addressing critical privacy and bias concerns.
Original Abstract
Vision-language models (VLMs) raise growing concerns about privacy, copyright, and bias, motivating machine unlearning to remove sensitive knowledge. However, existing methods primarily fine-tune the language decoder, leading to superficial forgetting that fails to erase underlying visual representations and often introduces object hallucination. We propose HFRU, a reinforcement unlearning framework that operates on the vision encoder for deep semantic removal. Our two-stage approach combines alignment disruption with GRPO-based optimization using a composite reward, including an abstraction reward that encourages semantically valid substitutions and mitigates hallucinations. Experiments on object recognition and face identity tasks show that HFRU achieves over 98% forgetting and retention performance, while introducing negligible object hallucination, significantly outperforming prior methods.Our code and implementation details are available at https://github.com/XMUDeepLIT/HFRU.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.