SGC: Soft Gradient Collaboration for Backdoor Attacks in Self-Supervised Distillation
TL;DR
Results show that security assessment of self-supervised distillation should include executable training logic in addition to model weights and visible data.
Abstract
Self-supervised knowledge distillation is widely used to compress reusable encoders, but an untrusted distillation implementation can itself become an attack surface. We study an algorithm-level threat in which the teacher encoder and user-visible distillation dataset remain unchanged, while malicious code internally generates trigger-bearing views and optimizes an additional backdoor objective. To instantiate this threat, we propose soft gradient collaboration (SGC), which combines distribution-alignment-based distillation, target-representation-based backdoor design, and conflict-avoidance gradient collaboration to reduce interference with benign representation transfer while embedding a trigger-to-target association in the student encoder. Experiments on CIFAR-10 and STL-10 show that SGC maintains competitive downstream accuracy and effective non-target attack success. Quantitative CKA, feature-distribution, and class-structure analyses further indicate that SGC retains clean representations closer to benign distillation than fixed scalarization or removal of distribution alignment. Its no-defense attack success is not the highest among the compared attacks; instead, its main empirical advantage is stronger residual attack persistence after MIMIC, MKD, and SSLDefender. Under SSLDefender, SGC retains 9.12% non-target ASR on CIFAR-10 and 9.06% on STL-10, the highest residual values among the compared attacks. Additional experiments with a compact ResNet-18 student, multiple target classes and trigger configurations, and a supplemental CIFAR-100 setting broaden the empirical evaluation across student capacity, target semantics, trigger configurations, and label-space complexity. These results show that security assessment of self-supervised distillation should include executable training logic in addition to model weights and visible data. The concealment considered here is limited to dataset-only inspection and clean-output validation; SGC is not claimed to evade source-code auditing, runtime data-flow monitoring, or training-log inspection.