First exit time analysis of stochastic gradient descent under heavy-tailed gradient noise

Thanh Huy Nguyen; Umut Şimşekli; Mert Gürbüzbalaban; Gaël Richard

Publications · Conference paper · NeurIPS 2019

First exit time analysis of stochastic gradient descent under heavy-tailed gradient noise

Thanh Huy Nguyen, Umut Şimşekli, Mert Gürbüzbalaban, Gaël Richard

Advances in Neural Information Processing Systems (NeurIPS), 2019.

Publisher page arXiv:1906.09069

Go further

Interactive

Heavy tails and basin exits → — a playground built around the ideas in this paper.

Topics

Heavy tails in SGD

Cite

@inproceedings{nguyen2019exittime,
  title   = {First exit time analysis of stochastic gradient descent under heavy-tailed gradient noise},
  author  = {Thanh Huy Nguyen and Umut Şimşekli and Mert Gürbüzbalaban and Gaël Richard},
  year    = {2019},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  url     = {https://proceedings.neurips.cc/paper/2019/hash/a97da629b098b75c294dffdc3e463904-Abstract.html},
  note    = {arXiv:1906.09069},
}

Relatedsame topics

Algorithmic stability of stochastic gradient descent with momentum under heavy-tailed noise (arXiv, 2025)
Heavy-tail phenomenon in decentralized SGD (IISE Trans., 2025)
Rényi differential privacy for heavy-tailed SDEs via fractional Poincaré inequalities (arXiv, 2025)
Differential privacy of noisy (S)GD under heavy-tailed perturbations (arXiv, 2024)
Algorithmic stability of heavy-tailed SGD with general loss functions (ICML, 2023)

← All publications · Research program · Mert Gürbüzbalaban