Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Under construction.

Description

Code for Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks.

Prerequisites

• python >= 3.6
• cython

Structure

In this repository we provide the code and some guided example to help the reader to reproduce the figures of the paper [1]. The repository is structured as follows.

The notebooks are self-explanatory.

Building Cython code

Both /sim and /ode use Cython code. To build, run python setup.py build_ext --inplace on the respective folder. Then simply start a Python session and do whether from sim import sim or from ode import ode and use the imported function as described in the how_to.ipynb notebooks.

Reference

[1] Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks, R. Veiga, L. Stephan, B. Loureiro, F. Krzakala and L. Zdeborová, arXiv:2202.00293 [stat.ML]