Privacy of SGD under Gaussian or heavy-tailed noise: guarantees without gradient clipping

Journal of Machine Learning Research, 2026. Accepted October 18, 2026.

arXiv:2403.02051

Differential privacy is usually obtained by adding Gaussian noise to the gradients and clipping them so that no single data point can move an update too far. This paper shows that the clipping is not needed: for noisy SGD with α-stable injected noise, which covers the Gaussian case as well as heavy-tailed noise with infinite variance, (ε, δ)-differential privacy holds with ε = 0 and δ of order 1/n for a broad class of possibly non-convex losses, with n the number of data points. Heavy-tailed noise gives privacy guarantees comparable to the Gaussian mechanism, so it is a viable alternative to it.

Differential privacyHeavy tails in SGD

• Rényi differential privacy for heavy-tailed SDEs via fractional Poincaré inequalities (arXiv, 2025)
• Differentially private accelerated optimization algorithms (SIAM J. Optim., 2022)
• Algorithmic stability of stochastic gradient descent with momentum under heavy-tailed noise (arXiv, 2025)
• Heavy-tail phenomenon in decentralized SGD (IISE Trans., 2025)
• Algorithmic stability of heavy-tailed SGD with general loss functions (ICML, 2023)