DAve-QN: a distributed averaged quasi-Newton method with local superlinear convergence rate

Saeed Soori; Konstantin Mishchenko; Aryan Mokhtari; Maryam Mehri Dehnavi; Mert Gürbüzbalaban

Conference paper, 2020

DAve-QN: a distributed averaged quasi-Newton method with local superlinear convergence rate

Saeed Soori, Konstantin Mishchenko, Aryan Mokhtari, Maryam Mehri Dehnavi, Mert Gürbüzbalaban

International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 108, pp. 1965–1976, 2020.

Publisher page

In brief

A distributed quasi-Newton method in which each worker maintains its own curvature estimate and the primary node keeps an average, so that no single node has to handle the full problem. DAve-QN converges locally at a superlinear rate — the hallmark of quasi-Newton methods — even though information arrives from workers with delays, and the experiments compare it with first-order distributed methods.

Topics

Distributed & decentralized learning

Cite

@inproceedings{soori2020daveqn,
  title   = {DAve-QN: a distributed averaged quasi-Newton method with local superlinear convergence rate},
  author  = {Saeed Soori and Konstantin Mishchenko and Aryan Mokhtari and Maryam Mehri Dehnavi and Mert Gürbüzbalaban},
  year    = {2020},
  booktitle = {International Conference on Artificial Intelligence and Statistics (AISTATS)},
  series  = {Proceedings of Machine Learning Research},
  volume  = {108},
  pages   = {1965--1976},
  url     = {http://proceedings.mlr.press/v108/soori20a.html},
}