Publications · Preprint · arXiv 2025
Algorithmic stability of stochastic gradient descent with momentum under heavy-tailed noise
arXiv preprint, 2025.
In brief
Algorithmic stability — how much a trained model changes when one training example is replaced — controls generalization, and under heavy-tailed gradient noise it depends on the tail index. The paper extends this analysis from plain SGD to SGD with momentum, quantifying how the momentum parameter and the heaviness of the tails together govern stability and hence generalization.
Cite
@misc{dang2025stability,
title = {Algorithmic stability of stochastic gradient descent with momentum under heavy-tailed noise},
author = {Thanh Dang and Melih Barsbey and A K. M. Rokonuzzaman Sonet and Mert Gürbüzbalaban and Umut Şimşekli and Lingjiong Zhu},
year = {2025},
howpublished = {arXiv preprint arXiv:2502.00885},
}Relatedsame topics