ESC: Erasing Space Concept for Knowledge Deletion
Journal:
arXiv
Published Date:
Apr 3, 2025
Abstract
As concerns regarding privacy in deep learning continue to grow, individuals
are increasingly apprehensive about the potential exploitation of their
personal knowledge in trained models. Despite several research efforts to
address this, they often fail to consider the real-world demand from users for
complete knowledge erasure. Furthermore, our investigation reveals that
existing methods have a risk of leaking personal knowledge through embedding
features. To address these issues, we introduce a novel concept of Knowledge
Deletion (KD), an advanced task that considers both concerns, and provides an
appropriate metric, named Knowledge Retention score (KR), for assessing
knowledge retention in feature space. To achieve this, we propose a novel
training-free erasing approach named Erasing Space Concept (ESC), which
restricts the important subspace for the forgetting knowledge by eliminating
the relevant activations in the feature. In addition, we suggest ESC with
Training (ESC-T), which uses a learnable mask to better balance the trade-off
between forgetting and preserving knowledge in KD. Our extensive experiments on
various datasets and models demonstrate that our proposed methods achieve the
fastest and state-of-the-art performance. Notably, our methods are applicable
to diverse forgetting scenarios, such as facial domain setting, demonstrating
the generalizability of our methods. The code is available at
http://github.com/KU-VGI/ESC .
