arXiv preprint, 2024.
In brief
Decentralized stochastic gradient Langevin dynamics lets agents sample a posterior over a network without sharing data, but network effects bias the samples, most visibly with full-batch gradients. Borrowing the EXTRA correction from decentralized optimization removes that bias: the method converges to the target posterior in 2-Wasserstein distance under strong convexity and smoothness, and outperforms plain decentralized SGLD when communication is constrained.
Cite
@misc{gurbuzbalaban2024extra,
title = {Generalized EXTRA stochastic gradient Langevin dynamics},
author = {Mert Gürbüzbalaban and Mohammad Rafiqul Islam and Xiaoyu Wang and Lingjiong Zhu},
year = {2024},
howpublished = {arXiv preprint arXiv:2412.01993},
}Relatedsame topics