ADONIS - Asynchronous Decentralized Optimization of MachiNe LearnIng ModelS

Welcome to the homepage of the ADONIS research project

The ADONIS project explores asynchronous decentralized optimization for both convex and non-convex problems. In the convex setting, we can solve well-posed mathematical problems and obtain theoretical guarantees regarding the convergence of optimization algorithms. However, deep learning poses a unique challenge in the non-convex setting, where practical performance often deviates significantly from theoretical predictions. In light of this, the ADONIS project adopts a principled approach to optimization research, guided by practical considerations and real-world applications.

Job offers

We are currently proposing an internship on Adaptive optimization for Decentralized DNNs.

news

Jul 29, 2023	The ADONIS team is giving a talk at the Localized Learning Workshop of ICML 2023.
Jun 26, 2023	Louis Fournier gave a talk on Forward Gradients to the MLIA team.
Jun 15, 2023	Edouard Oyallon is giving a talk at Centre Borelli about the recent advances of the project.
Jun 2, 2023	Welcoming Belilovsky Lab director, Eugene Belilovsky, for a month.
Apr 24, 2023	Two papers accepted at ICML 2023, on accelerated gossip and forward gradient methods.

selected publications

DADAO: Decoupled Accelerated Decentralized Asynchronous Optimization

Adel Nabli, and Edouard Oyallon

In Proceedings of the 40th International Conference on Machine Learning, 2023

Bib HTML PDF

@inproceedings{nabli2023dadao,
  title = {DADAO: Decoupled Accelerated Decentralized Asynchronous Optimization},
  author = {Nabli, Adel and Oyallon, Edouard},
  booktitle = {Proceedings of the 40th International Conference on Machine Learning},
  year = {2023},
  series = {Proceedings of Machine Learning Research},
  publisher = {PMLR},
}

Can Forward Gradient match Backpropagation?

Louis Fournier, Stéphane Rivaud, Eugene Belilovsky, and 2 more authors

In Proceedings of the 40th International Conference on Machine Learning, 2023

Bib HTML PDF

@inproceedings{fournier2023can,
  title = {Can Forward Gradient match Backpropagation?},
  author = {Fournier, Louis and Rivaud, Stéphane and Belilovsky, Eugene and Eickenberg, Michael and Oyallon, Edouard},
  booktitle = {Proceedings of the 40th International Conference on Machine Learning},
  year = {2023},
  series = {Proceedings of Machine Learning Research},
  publisher = {PMLR},
}