Your SlideShare is downloading. ×

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

DataScienceConferenc1

•

Nov. 30, 2022

• •

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Sketch of the proof:

Sketch of the proof:

Sketch of the proof:

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Over-parametrization is necessary for robustness!

Over-parametrization is necessary for robustness!

Over-parametrization is necessary for robustness!

Curse of dimensionality!

This is huge over-parametrization!

This is huge over-parametrization!

This is huge over-parametrization!

Source: nilesjohnson.net

•Thank you!

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

Next SlideShares

Upcoming SlideShare

[DSC Europe 22] The nuts and bolts of music source separation - Stipe Kabic

[DSC Europe 22] The nuts and bolts of music source separation - Stipe Kabic

Loading in …3

×

Check these out next

[DSC Europe 22] Avoid mistakes building AI products - Karol Przystalski

DataScienceConferenc1

[DSC Europe 22] The Making of a Data Organization - Denys Holovatyi

DataScienceConferenc1

[DSC Europe 22] Building the future, a founder's perspective - Ben Taylor

DataScienceConferenc1

[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Produ...

DataScienceConferenc1

[DSC Europe 22] Reverse Engineering Cloud Native: Interoperability and Commun...

DataScienceConferenc1

[DSC Europe 22] Smart approach in development and deployment process for vari...

DataScienceConferenc1

[DSC Europe 22] Engineers guide for shepherding models in to production - Mar...

DataScienceConferenc1

[DSC Europe 22] Anti-Money Laundering ML Modeling approach - Gizem Akar

DataScienceConferenc1

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

Nov. 30, 2022

• •

Download to read offline

Data & Analytics

We will present some recent fundamental developments in understanding how neural networks work, in particular why the overparametrized models with the billions or even trilions of parameters perform so well even in cases when the number of parameters outnumbers the number of training samples, contrary to classical machine learning paradigm. We will present the BLN Conjecture (2020) and demonstrate few extreme cases of Bubeck-Sellke Theorem (2021). In order to illustrate highly technical theorem and it's proof we will use an original pictorial approach to represent the behaviour of neural networks from the point of view of the higher-dimensional geometry.

DataScienceConferenc1

We will present some recent fundamental developments in understanding how neural networks work, in particular why the overparametrized models with the billions or even trilions of parameters perform so well even in cases when the number of parameters outnumbers the number of training samples, contrary to classical machine learning paradigm. We will present the BLN Conjecture (2020) and demonstrate few extreme cases of Bubeck-Sellke Theorem (2021). In order to illustrate highly technical theorem and it's proof we will use an original pictorial approach to represent the behaviour of neural networks from the point of view of the higher-dimensional geometry.

Data & Analytics

More Related Content

[DSC Europe 22] Avoid mistakes building AI products - Karol Przystalski

DataScienceConferenc1

[DSC Europe 22] The Making of a Data Organization - Denys Holovatyi

DataScienceConferenc1

[DSC Europe 22] Building the future, a founder's perspective - Ben Taylor

DataScienceConferenc1

[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Produ...

DataScienceConferenc1

[DSC Europe 22] Reverse Engineering Cloud Native: Interoperability and Commun...

DataScienceConferenc1

[DSC Europe 22] Smart approach in development and deployment process for vari...

DataScienceConferenc1

[DSC Europe 22] Engineers guide for shepherding models in to production - Mar...

DataScienceConferenc1

[DSC Europe 22] Anti-Money Laundering ML Modeling approach - Gizem Akar

DataScienceConferenc1

[DSC Europe 22] The ways AI will change your workplace - Iliada Kothra

DataScienceConferenc1

[DSC Europe 22] Similarity search: Take your search engine to the next level ...

DataScienceConferenc1

[DSC Europe 22] Equality alert in newsrooms: Providing transparency about the...

DataScienceConferenc1

[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar

DataScienceConferenc1

[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...

DataScienceConferenc1

[DSC Europe 22] AIOPS – How can machine learning help in IT operations - Dam...

DataScienceConferenc1

[DSC Europe 22] AI Ethics and AI Quality By Design - Muthu Ramachandran

DataScienceConferenc1

[DSC Europe 22] Machine learning algorithms as tools for student success pred...

DataScienceConferenc1

[DSC Europe 22] Parakeet - a parrot that can repeat words very well - Tomisla...

DataScienceConferenc1

[DSC Europe 22] Disrupting Supply Chain with Data Science - Arthur Jansen

DataScienceConferenc1

[DSC Europe 22] How to enable enterprises for DevSecOps without fighting the ...

DataScienceConferenc1

[DSC Europe 22] Starting deep learning projects without sufficient amount of ...

DataScienceConferenc1

[DSC Europe 22] Avoid mistakes building AI products - Karol Przystalski

DataScienceConferenc1

[DSC Europe 22] The Making of a Data Organization - Denys Holovatyi

DataScienceConferenc1

[DSC Europe 22] Building the future, a founder's perspective - Ben Taylor

DataScienceConferenc1

[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Produ...

DataScienceConferenc1

[DSC Europe 22] Reverse Engineering Cloud Native: Interoperability and Commun...

DataScienceConferenc1

[DSC Europe 22] Smart approach in development and deployment process for vari...

DataScienceConferenc1

Data Engineer's Lunch #85: Designing a Modern Data Stack

Anant Corporation

february-2023-calendar-landscape-large-numerals.pdf

Mohammed Saddam Hussain

New Project 168 [0E2B84A]_000.ppt

Enabling a Data Mesh Architecture and Data Sharing Culture with Denodo

design principles for visualization

Modelado Dimensional 4 etapas.ppt

introduction meeting 1 - tugas.pdf

ARTUROGOMEZGARCIA2

Chapter_1_Lecture.pptx

ZelalemGebreegziabhe

EDA Assignment.pptx

ShakirAhmadBarbhuiya

CRITICAL PATH METHOD IN NETWORK.pptx

TOURISM_Chapter_14.pptx

Time Complexity Analysis in Data Structure.docx

Mani .S (Specialization in Semantic Web)

THE STUDY OF EFFECT OF IMMUNOSUPPRESSANTS DURING ORGAN.pptx

SQL (Scratch to Advance).pptx

Case 8. SQL Basic 2 and Analyzing Business Data Using SQL.pptx

TAHUN 2 WANG.pptx

Data Engineer's Lunch #85: Designing a Modern Data Stack

Anant Corporation

february-2023-calendar-landscape-large-numerals.pdf

Mohammed Saddam Hussain

New Project 168 [0E2B84A]_000.ppt

Enabling a Data Mesh Architecture and Data Sharing Culture with Denodo

design principles for visualization

Modelado Dimensional 4 etapas.ppt

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

1. Sketch of the proof:
2. Sketch of the proof:
3. Sketch of the proof:
4. Sketch of the proof (d=2):
5. Sketch of the proof (d=2):
6. Sketch of the proof (d=2):
7. Sketch of the proof (d=2):
8. Sketch of the proof (d=2):
9. Sketch of the proof (d=2):
10. Sketch of the proof (d=2):
11. Over-parametrization is necessary for robustness!
12. Over-parametrization is necessary for robustness!
13. Over-parametrization is necessary for robustness!
14. Curse of dimensionality!
15. This is huge over-parametrization!
16. This is huge over-parametrization!
17. This is huge over-parametrization!
18. Source: nilesjohnson.net
19. •Thank you!