Global convergence rate of proximal incremental aggregated gradient methods

SIAM Journal on Optimization, 28(2), pp. 1282–1300, 2018.

Publisher (DOI)

Linear convergence for the proximal incremental aggregated gradient method, which minimizes a sum of smooth components plus a nonsmooth regularizer using one new component gradient per step and stale gradients for the rest. The rate is global, holds under strong convexity, and depends explicitly on how stale the aggregated gradients are allowed to be.

Incremental methods & data ordering

• Why random reshuffling beats stochastic gradient descent (Math. Program., 2021)
• Randomness and permutations in coordinate descent methods (Math. Program., 2020)
• Convergence rate of incremental gradient and incremental Newton methods (SIAM J. Optim., 2019)
• Surpassing gradient descent provably: a cyclic incremental method with linear convergence rate (SIAM J. Optim., 2018)
• A double incremental aggregated gradient method with linear convergence rate for large-scale optimization (ICASSP, 2017)