Stefano Sarao Mannelli presented work at NeurIPS 2019 on spiked matrix-tensor models. The work analyzed the landscape of the loss function for these models and found a phase transition between a trivial phase where gradient descent easily finds the global minimum, and a hard phase with spurious local minima. They developed techniques like the Kac-Rice formula to analyze the landscape and dynamics using dynamical mean field theory.

1. Stefano Sarao Mannelli - NeurIPS 2019
Stefano Sarao Mannelli
in collaboration with
Giulio Biroli, Chiara Cammarota,
Florent Krzakala and Lenka Zdeborová
NeurIPS 2019
IPhT
SACLAY
Who is Afraid of Big Bad
Minima?

2. Stefano Sarao Mannelli - NeurIPS 2019
Spiked Matrix-Tensor Model

3. Stefano Sarao Mannelli - NeurIPS 2019
Spiked Matrix-Tensor Model
minus log-likelihood

4. Stefano Sarao Mannelli - NeurIPS 2019
Spiked Matrix-Tensor Model
Global minimum

5. Stefano Sarao Mannelli - NeurIPS 2019
Spiked Matrix-Tensor Model
Global minimum
presence
of spurious
minima

6. Stefano Sarao Mannelli - NeurIPS 2019
Trivial
GF does not
work
Phase Diagram
G
F
works
spuriosity
with

7. Stefano Sarao Mannelli - NeurIPS 2019
Trivial
GF does not
work
Phase Diagram
G
F
works
spuriosity
with
Landscape
Trivialization
via Kac-Rice
formula

8. Stefano Sarao Mannelli - NeurIPS 2019
Trivial
GF does not
work
Phase Diagram
G
F
works
spuriosity
with
Landscape
Trivialization
via Kac-Rice
formula
Dynamics via DMFT

9. Stefano Sarao Mannelli - NeurIPS 2019
Trivial
GF does not
work
Phase Diagram
G
F
works
spuriosity
with

10. Stefano Sarao Mannelli - NeurIPS 2019
Explanation
trivialgradient-flow easy
gradient-flow
hard
Loss
snr : 1/
Fix

11. Stefano Sarao Mannelli - NeurIPS 2019
Explanation
trivialgradient-flow easy
gradient-flow
hard
Loss
snr : 1/
Fix

12. Stefano Sarao Mannelli - NeurIPS 2019
Thank you! :)