Skip to content
Preprint

DP-VOXLET: Provable Speaker Anonymization for Disentangled Speech Representations

Aug 2026 · 0 citations · 25 references
Computer Science

TL;DR

This work introduces speaker differential privacy, a formal definition of speaker anonymization based on the framework of differential privacy, and a mechanism for speaker anonymization that provably satisfies the definition.

Abstract

Systems for speaker anonymization obfuscate the speaker of an utterance, while maintaining its original semantic contents and prosody. Recent solutions for speaker anonymization rely on learned representations that disentangle an utterance into semantic contents and speaker properties. To anonymize an utterance, these systems replace the speaker properties while leaving the semantic contents unchanged---an approach that can produce strong results on empirical measures of privacy. In this work, we introduce speaker differential privacy, a formal definition of speaker anonymization based on the framework of differential privacy, and a mechanism for speaker anonymization that provably satisfies the definition. In contrast to prior heuristic-based anonymization systems, our approach enables a provable lower bound on re-identification success rate (e.g. equal error rate) for any possible adversary. We implement our approach in a framework that is compatible with existing disentangled representations. Compared to the prior work on differential privacy for speaker anonymization, our approach achieves significantly higher utility.

View source

Similar papers

#machine learning Preprint Oct 2026

LiveAnon: Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models

Speaker anonymization (SA) aims to conceal speaker identity while preserving linguistic content and emotion, but existing streaming systems often lose emotion information through discrete content bottlenecks or require emotion-specific modeling. We present LiveAnon, a voice-conversion-based streaming SA system that dir...

Yejin Kim, Kyungsu Kim, Seungu Han et al. · 0 citations
Preprint Sep 2026

VoxTubeS: Distributable Speaker-Anonymized Synthetic Speech Corpora and Their Analysis

VoxTubeS is presented, a family of speaker-anonymized synthetic speech corpora designed for redistribution, comprising three method families and seven variants derived from the VoxTube corpus, which is distributed under CC BY-NC-SA 4.0.

Zhe Zhang, Ye-Xin Lu, Junichi Yamagishi · 0 citations
Preprint Oct 2026

VoxAnon: A balanced dataset for a fair evaluation of speech anonymization

VoxAnon is introduced, a dataset built from VoxCeleb1 and 2 with training and evaluation splits balanced across gender, age, accent, and pitch, and it is shown that attribute imbalance changes which subgroups appear hardest to re-identify, in directions that depend on the attribute.

Orane Dufour, Paul Magron, Mickael Rouvier · 0 citations
Preprint Sep 2026

Reducing Speaker Residual by Considering Pinhole Effect in Voice Anonymization

A fine-tuning strategy with a pinhole loss for well-trained voice anonymization frameworks to further reduce residual speaker attributes in non-identity representations to improve privacy protection.

Ze-Yan Liu, Wei Jiang, Li-Ping Chen et al. · 0 citations
Preprint Aug 2026

Your Voice Cloning System is Secretly a Voice Anonymizer

This work proposes repurposing XTTSv2, a multilingual voice cloning model trained on 27k hours of speech, for speaker anonymization without retraining, and introduces an iterative refinement strategy that balances privacy and utility by maximizing a harmonic mean of speaker dissimilarity and intelligibility.

Romolo Muletta, Felix Matthias Saaro, Mark Cieliebak et al. · 0 citations
Open access Sep 2026

Towards Privacy-Preserving ASR: Speaker-Level Machine Unlearning

This work conducts a layer-wise analysis of Self-Supervised Learning based ASR representations, revealing that speaker identity is primarily encoded in intermediate encoder layers, and explores a feature-level unlearning strategy that targets these identity structures.

Seaone Ok, Seungu Han, Eungbeom Kim et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.