Differential privacy of noisy (S)GD under heavy-tailed perturbations

Umut Şimşekli; Mert Gürbüzbalaban; Sinan Yıldırım; Lingjiong Zhu

Publications · Preprint · arXiv 2024

Differential privacy of noisy (S)GD under heavy-tailed perturbations

Umut Şimşekli, Mert Gürbüzbalaban, Sinan Yıldırım, Lingjiong Zhu

arXiv preprint, 2024.

arXiv:2403.02051

In brief

Differential privacy is usually obtained by adding Gaussian noise to gradients; this paper asks what happens when the noise is heavy-tailed instead, as it naturally is in SGD. It establishes differential-privacy guarantees for noisy (stochastic) gradient descent under heavy-tailed perturbations and works out the resulting privacy–utility trade-off.

Topics

Differential privacy Heavy tails in SGD

Cite

@misc{simsekli2024dp,
  title   = {Differential privacy of noisy (S)GD under heavy-tailed perturbations},
  author  = {Umut Şimşekli and Mert Gürbüzbalaban and Sinan Yıldırım and Lingjiong Zhu},
  year    = {2024},
  howpublished = {arXiv preprint arXiv:2403.02051},
}

Relatedsame topics

Rényi differential privacy for heavy-tailed SDEs via fractional Poincaré inequalities (arXiv, 2025)
Differentially private accelerated optimization algorithms (SIAM J. Optim., 2022)
Algorithmic stability of stochastic gradient descent with momentum under heavy-tailed noise (arXiv, 2025)
Heavy-tail phenomenon in decentralized SGD (IISE Trans., 2025)
Algorithmic stability of heavy-tailed SGD with general loss functions (ICML, 2023)

← All publications · Research program · Mert Gürbüzbalaban