Publications · Conference paper · ICML 2019
Accelerated linear convergence of stochastic momentum methods in Wasserstein distances
International Conference on Machine Learning (ICML), PMLR 97, pp. 891–901, 2019.
In brief
Shows that with persistent gradient noise, momentum methods converge in distribution: the iterates contract linearly, at the accelerated rate, to a unique stationary law in the 1-Wasserstein distance — the analytical foundation for measuring and designing the stationary "noise cloud" of accelerated methods.
Cite
@inproceedings{can2019wasserstein,
title = {Accelerated linear convergence of stochastic momentum methods in Wasserstein distances},
author = {Bugra Can and Mert Gürbüzbalaban and Lingjiong Zhu},
year = {2019},
booktitle = {International Conference on Machine Learning (ICML)},
series = {Proceedings of Machine Learning Research},
volume = {97},
pages = {891--901},
url = {http://proceedings.mlr.press/v97/can19a.html},
note = {arXiv:1901.07445},
}Relatedsame topics