SlideShare a Scribd company logo
1 of 24
Download to read offline
Gert-Jan Both, Remy Kusters
CRI Research, Université Paris Descartes, Paris, France
DeepMoD
Model discovery
neural networks
DeepMoD
Model discovery
neural networks
@GJ_Both
@RemyKusters
AI, ML and statistics
Symbolic MLStatistical ML vs.
@sandserif
Main criticisms …
ML provides uninterpretable
solutions
Goal:
Develop tools to discover interpretable models for quantitative
science
Combine the statistical power of neural networks with the
symbolic power of regression
Approach:
Goal:
Combine the statistical power of neural networks with the
symbolic power of regression
Develop tools to discover interpretable models for quantitative
science
<latexit sha1_base64="OuHvrtTUD9r4pBI06iD0mJxKAcs=">AAACGnicbVDLSgMxFM34rPVVdekmWIQWyjAjgm6EoiAuFWwrtMOQSTNtMPMguZGWYb7Djb/ixoUi7sSNf2NaZ6HWA8k9nHMvyT1BKrgCx/m05uYXFpeWSyvl1bX1jc3K1nZbJVpS1qKJSORNQBQTPGYt4CDYTSoZiQLBOsHt2cTv3DGpeBJfwzhlXkQGMQ85JWAkv+JqH2qjBtTxCc56lIjsPM9xTTew9kfmKoqfjUZ5A9u2XfcrVcd2psCzxC1IFRW49CvvvX5CdcRioIIo1XWdFLyMSOBUsLzc04qlhN6SAesaGpOIKS+brpbjfaP0cZhIc2LAU/XnREYipcZRYDojAkP115uI/3ldDeGxl/E41cBi+v1QqAWGBE9ywn0uGQUxNoRQyc1fMR0SSSiYNMsmBPfvyrOkfWC7hl8dVpunRRwltIv2UA256Ag10QW6RC1E0T16RM/oxXqwnqxX6+27dc4qZnbQL1gfX5Tang4=</latexit>
Differential equation
Space
Time
Noisy Data
Underlying distribution
P(x, t)<latexit sha1_base64="3XtpKznpBSb+nMyPAQPZfV01TDk=">AAAB7XicbZBNSwMxEIZn61etX1WPXhaLUEHKrgh6LHrxWMF+QLuUbJq2sdlkSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZecNYcIOe9+3kVlbX1jfym4Wt7Z3dveL+QcOoRFNWp0oo3QqJYYJLVkeOgrVizUgUCtYMRzfTevORacOVvMdxzIKIDCTvc0rQWo1a+ekMT7vFklfxZnKXwc+gBJlq3eJXp6doEjGJVBBj2r4XY5ASjZwKNil0EsNiQkdkwNoWJYmYCdLZthP3xDo9t6+0fRLdmft7IiWRMeMotJ0RwaFZrE3N/2rtBPtXQcplnCCTdP5RPxEuKnd6utvjmlEUYwuEam53demQaELRBlSwIfiLJy9D47ziW767KFWvszjycATHUAYfLqEKt1CDOlB4gGd4hTdHOS/Ou/Mxb8052cwh/JHz+QOJ2I5v</latexit><latexit sha1_base64="3XtpKznpBSb+nMyPAQPZfV01TDk=">AAAB7XicbZBNSwMxEIZn61etX1WPXhaLUEHKrgh6LHrxWMF+QLuUbJq2sdlkSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZecNYcIOe9+3kVlbX1jfym4Wt7Z3dveL+QcOoRFNWp0oo3QqJYYJLVkeOgrVizUgUCtYMRzfTevORacOVvMdxzIKIDCTvc0rQWo1a+ekMT7vFklfxZnKXwc+gBJlq3eJXp6doEjGJVBBj2r4XY5ASjZwKNil0EsNiQkdkwNoWJYmYCdLZthP3xDo9t6+0fRLdmft7IiWRMeMotJ0RwaFZrE3N/2rtBPtXQcplnCCTdP5RPxEuKnd6utvjmlEUYwuEam53demQaELRBlSwIfiLJy9D47ziW767KFWvszjycATHUAYfLqEKt1CDOlB4gGd4hTdHOS/Ou/Mxb8052cwh/JHz+QOJ2I5v</latexit><latexit sha1_base64="3XtpKznpBSb+nMyPAQPZfV01TDk=">AAAB7XicbZBNSwMxEIZn61etX1WPXhaLUEHKrgh6LHrxWMF+QLuUbJq2sdlkSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZecNYcIOe9+3kVlbX1jfym4Wt7Z3dveL+QcOoRFNWp0oo3QqJYYJLVkeOgrVizUgUCtYMRzfTevORacOVvMdxzIKIDCTvc0rQWo1a+ekMT7vFklfxZnKXwc+gBJlq3eJXp6doEjGJVBBj2r4XY5ASjZwKNil0EsNiQkdkwNoWJYmYCdLZthP3xDo9t6+0fRLdmft7IiWRMeMotJ0RwaFZrE3N/2rtBPtXQcplnCCTdP5RPxEuKnd6utvjmlEUYwuEam53demQaELRBlSwIfiLJy9D47ziW767KFWvszjycATHUAYfLqEKt1CDOlB4gGd4hTdHOS/Ou/Mxb8052cwh/JHz+QOJ2I5v</latexit><latexit sha1_base64="3XtpKznpBSb+nMyPAQPZfV01TDk=">AAAB7XicbZBNSwMxEIZn61etX1WPXhaLUEHKrgh6LHrxWMF+QLuUbJq2sdlkSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZecNYcIOe9+3kVlbX1jfym4Wt7Z3dveL+QcOoRFNWp0oo3QqJYYJLVkeOgrVizUgUCtYMRzfTevORacOVvMdxzIKIDCTvc0rQWo1a+ekMT7vFklfxZnKXwc+gBJlq3eJXp6doEjGJVBBj2r4XY5ASjZwKNil0EsNiQkdkwNoWJYmYCdLZthP3xDo9t6+0fRLdmft7IiWRMeMotJ0RwaFZrE3N/2rtBPtXQcplnCCTdP5RPxEuKnd6utvjmlEUYwuEam53demQaELRBlSwIfiLJy9D47ziW767KFWvszjycATHUAYfLqEKt1CDOlB4gGd4hTdHOS/Ou/Mxb8052cwh/JHz+QOJ2I5v</latexit>
Approach:
<latexit sha1_base64="OuHvrtTUD9r4pBI06iD0mJxKAcs=">AAACGnicbVDLSgMxFM34rPVVdekmWIQWyjAjgm6EoiAuFWwrtMOQSTNtMPMguZGWYb7Djb/ixoUi7sSNf2NaZ6HWA8k9nHMvyT1BKrgCx/m05uYXFpeWSyvl1bX1jc3K1nZbJVpS1qKJSORNQBQTPGYt4CDYTSoZiQLBOsHt2cTv3DGpeBJfwzhlXkQGMQ85JWAkv+JqH2qjBtTxCc56lIjsPM9xTTew9kfmKoqfjUZ5A9u2XfcrVcd2psCzxC1IFRW49CvvvX5CdcRioIIo1XWdFLyMSOBUsLzc04qlhN6SAesaGpOIKS+brpbjfaP0cZhIc2LAU/XnREYipcZRYDojAkP115uI/3ldDeGxl/E41cBi+v1QqAWGBE9ywn0uGQUxNoRQyc1fMR0SSSiYNMsmBPfvyrOkfWC7hl8dVpunRRwltIv2UA256Ag10QW6RC1E0T16RM/oxXqwnqxX6+27dc4qZnbQL1gfX5Tang4=</latexit>
Analytic solution
Space
Time
<latexit sha1_base64="f64HEF6znFEZoH5kmjcDbs/59Jo=">AAAB7XicbZBNSwMxEIZn61etX1WPXoJFqCBlVwQ9Fr14rGA/oF1KNk3b2OxmSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZeYNYCoOu++3kVlbX1jfym4Wt7Z3dveL+QcOoRDNeZ0oq3Qqo4VJEvI4CJW/FmtMwkLwZjG6m9eYj10ao6B7HMfdDOohEXzCK1mok5aczPO0WS27FnYksg5dBCTLVusWvTk+xJOQRMkmNaXtujH5KNQom+aTQSQyPKRvRAW9bjGjIjZ/Otp2QE+v0SF9p+yIkM/f3REpDY8ZhYDtDikOzWJua/9XaCfav/FREcYI8YvOP+okkqMj0dNITmjOUYwuUaWF3JWxINWVoAyrYELzFk5ehcV7xLN9dlKrXWRx5OIJjKIMHl1CFW6hBHRg8wDO8wpujnBfn3fmYt+acbOYQ/sj5/AHCpY6U</latexit>
Noisy Data
DeepMoD
Model discovery
neural networks
DeepMoD
Model discovery
neural networks
Code available:
github.com/PhIMaL
<latexit sha1_base64="1Gq7mrW1/ZKuQjtOjsnM1gDsGDU=">AAACBHicbZDJSgNBEIZ74hbjFvWYS2MQIkiYEUEvQtCLxwjZIBNCT6eSNOlZ6K4JCUMOXnwVLx4U8epDePNt7CwHTfyh4eOvKqrr9yIpNNr2t5VaW9/Y3EpvZ3Z29/YPsodHNR3GikOVhzJUDY9pkCKAKgqU0IgUMN+TUPcGd9N6fQhKizCo4DiCls96gegKztBY7WwubmNhdI5n9Ia6lT4go+4QeOKOxKSdzdtFeya6Cs4C8mShcjv75XZCHvsQIJdM66ZjR9hKmELBJUwybqwhYnzAetA0GDAfdCuZHTGhp8bp0G6ozAuQztzfEwnztR77nun0Gfb1cm1q/ldrxti9biUiiGKEgM8XdWNJMaTTRGhHKOAoxwYYV8L8lfI+U4yjyS1jQnCWT16F2kXRMfxwmS/dLuJIkxw5IQXikCtSIvekTKqEk0fyTF7Jm/VkvVjv1se8NWUtZo7JH1mfPysqlyg=</latexit>
Dataset Neural Network Thresh
ˆuu
Auto. Diff.
∂t ˆu = Θξ
Library
Include in cost function
NN converged
ξ∗
= ξ ·
||
||
×
×
×
×
× ×
×
×
ξ∗
Update cost f
Space
Time
Burgers equation
∂tu = uux − 0.1uxx
L = MSE(u, ˆu)
+ MSE(∂t ˆu, Θξ)
+ L1(ξ)
= ...
u
ux
uux
u2
uxx
Space
Time
<latexit sha1_base64="f64HEF6znFEZoH5kmjcDbs/59Jo=">AAAB7XicbZBNSwMxEIZn61etX1WPXoJFqCBlVwQ9Fr14rGA/oF1KNk3b2OxmSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZeYNYCoOu++3kVlbX1jfym4Wt7Z3dveL+QcOoRDNeZ0oq3Qqo4VJEvI4CJW/FmtMwkLwZjG6m9eYj10ao6B7HMfdDOohEXzCK1mok5aczPO0WS27FnYksg5dBCTLVusWvTk+xJOQRMkmNaXtujH5KNQom+aTQSQyPKRvRAW9bjGjIjZ/Otp2QE+v0SF9p+yIkM/f3REpDY8ZhYDtDikOzWJua/9XaCfav/FREcYI8YvOP+okkqMj0dNITmjOUYwuUaWF3JWxINWVoAyrYELzFk5ehcV7xLN9dlKrXWRx5OIJjKIMHl1CFW6hBHRg8wDO8wpujnBfn3fmYt+acbOYQ/sj5/AHCpY6U</latexit>
SolutionNoisy Data
DeepMoD
Model discovery
neural networks
DeepMoD
Model discovery
neural networks
Code available:
github.com/PhIMaL
<latexit sha1_base64="1Gq7mrW1/ZKuQjtOjsnM1gDsGDU=">AAACBHicbZDJSgNBEIZ74hbjFvWYS2MQIkiYEUEvQtCLxwjZIBNCT6eSNOlZ6K4JCUMOXnwVLx4U8epDePNt7CwHTfyh4eOvKqrr9yIpNNr2t5VaW9/Y3EpvZ3Z29/YPsodHNR3GikOVhzJUDY9pkCKAKgqU0IgUMN+TUPcGd9N6fQhKizCo4DiCls96gegKztBY7WwubmNhdI5n9Ia6lT4go+4QeOKOxKSdzdtFeya6Cs4C8mShcjv75XZCHvsQIJdM66ZjR9hKmELBJUwybqwhYnzAetA0GDAfdCuZHTGhp8bp0G6ozAuQztzfEwnztR77nun0Gfb1cm1q/ldrxti9biUiiGKEgM8XdWNJMaTTRGhHKOAoxwYYV8L8lfI+U4yjyS1jQnCWT16F2kXRMfxwmS/dLuJIkxw5IQXikCtSIvekTKqEk0fyTF7Jm/VkvVjv1se8NWUtZo7JH1mfPysqlyg=</latexit>
aset Neural Network Threshold
ˆuu
Auto. Diff.
∂t ˆu = Θξ
Library
Include in cost function
NN converged
ξ∗
= ξ ·
||Θ||
||ut||
So
Sparsity converge
×
×
×
×
× ×
×
×
×
×
×
keep
set to
zero
keep
ξ∗
Update cost function
ace
equation
x − 0.1uxx
L = MSE(u, ˆu)
+ MSE(∂t ˆu, Θξ)
+ L1(ξ)
= ...
u
ux
uux
u2
uxx
Fou
∂tu =
Space
Time
<latexit sha1_base64="f64HEF6znFEZoH5kmjcDbs/59Jo=">AAAB7XicbZBNSwMxEIZn61etX1WPXoJFqCBlVwQ9Fr14rGA/oF1KNk3b2OxmSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZeYNYCoOu++3kVlbX1jfym4Wt7Z3dveL+QcOoRDNeZ0oq3Qqo4VJEvI4CJW/FmtMwkLwZjG6m9eYj10ao6B7HMfdDOohEXzCK1mok5aczPO0WS27FnYksg5dBCTLVusWvTk+xJOQRMkmNaXtujH5KNQom+aTQSQyPKRvRAW9bjGjIjZ/Otp2QE+v0SF9p+yIkM/f3REpDY8ZhYDtDikOzWJua/9XaCfav/FREcYI8YvOP+okkqMj0dNITmjOUYwuUaWF3JWxINWVoAyrYELzFk5ehcV7xLN9dlKrXWRx5OIJjKIMHl1CFW6hBHRg8wDO8wpujnBfn3fmYt+acbOYQ/sj5/AHCpY6U</latexit>
⇠⇤
=<latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit>
Neural Network Threshold
ˆuu
Auto. Diff.
∂t ˆu = Θξ
Library
Include in cost function
NN converged
ξ∗
= ξ ·
||Θ||
||ut||
Solution
Sparsity converged
×
×
×
×
× ×
×
×
×
×
×
keep
set to
zero
keep
ξ∗
Update cost function
L = MSE(u, ˆu)
+ MSE(∂t ˆu, Θξ)
+ L1(ξ)
= ...
u
ux
uux
u2
uxx
Found solution
∂tu = uux − 0.1uxx
0
0
1
0
...
-0.1
0
uux
uxx
u
ux
u2
uxx
<latexit sha1_base64="1Gq7mrW1/ZKuQjtOjsnM1gDsGDU=">AAACBHicbZDJSgNBEIZ74hbjFvWYS2MQIkiYEUEvQtCLxwjZIBNCT6eSNOlZ6K4JCUMOXnwVLx4U8epDePNt7CwHTfyh4eOvKqrr9yIpNNr2t5VaW9/Y3EpvZ3Z29/YPsodHNR3GikOVhzJUDY9pkCKAKgqU0IgUMN+TUPcGd9N6fQhKizCo4DiCls96gegKztBY7WwubmNhdI5n9Ia6lT4go+4QeOKOxKSdzdtFeya6Cs4C8mShcjv75XZCHvsQIJdM66ZjR9hKmELBJUwybqwhYnzAetA0GDAfdCuZHTGhp8bp0G6ozAuQztzfEwnztR77nun0Gfb1cm1q/ldrxti9biUiiGKEgM8XdWNJMaTTRGhHKOAoxwYYV8L8lfI+U4yjyS1jQnCWT16F2kXRMfxwmS/dLuJIkxw5IQXikCtSIvekTKqEk0fyTF7Jm/VkvVjv1se8NWUtZo7JH1mfPysqlyg=</latexit>
SolutionNoisy Data Regression
Code available:
github.com/PhIMaL
DeepMoD
Model discovery
neural networks
DeepMoD
Model discovery
neural networks
5% white noise …
Why does it fail with noise?
Θ =
! ! !
u ux uxu
! ! !
! ! !
uxx uxxu uxxux
! ! !
"
⎛
⎝
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
Numerically calculating library impossible …
uxx(x)<latexit sha1_base64="QNA18qBfAJrCCEwMxK/B/I1M350=">AAAB8HicbZDLSgMxFIbP1Futt6pLN8Ei1E2ZEUGXRTcuK9iLtEPJpJk2NMkMSUZahj6FGxeKuPVx3Pk2ZtpZaOsPgY//nEPO+YOYM21c99sprK1vbG4Vt0s7u3v7B+XDo5aOEkVok0Q8Up0Aa8qZpE3DDKedWFEsAk7bwfg2q7efqNIskg9mGlNf4KFkISPYWOsx6aeTyaw6Oe+XK27NnQutgpdDBXI1+uWv3iAiiaDSEI617npubPwUK8MIp7NSL9E0xmSMh7RrUWJBtZ/OF56hM+sMUBgp+6RBc/f3RIqF1lMR2E6BzUgv1zLzv1o3MeG1nzIZJ4ZKsvgoTDgyEcquRwOmKDF8agETxeyuiIywwsTYjEo2BG/55FVoXdQ8y/eXlfpNHkcRTuAUquDBFdThDhrQBAICnuEV3hzlvDjvzseiteDkM8fwR87nD707kFk=</latexit><latexit sha1_base64="QNA18qBfAJrCCEwMxK/B/I1M350=">AAAB8HicbZDLSgMxFIbP1Futt6pLN8Ei1E2ZEUGXRTcuK9iLtEPJpJk2NMkMSUZahj6FGxeKuPVx3Pk2ZtpZaOsPgY//nEPO+YOYM21c99sprK1vbG4Vt0s7u3v7B+XDo5aOEkVok0Q8Up0Aa8qZpE3DDKedWFEsAk7bwfg2q7efqNIskg9mGlNf4KFkISPYWOsx6aeTyaw6Oe+XK27NnQutgpdDBXI1+uWv3iAiiaDSEI617npubPwUK8MIp7NSL9E0xmSMh7RrUWJBtZ/OF56hM+sMUBgp+6RBc/f3RIqF1lMR2E6BzUgv1zLzv1o3MeG1nzIZJ4ZKsvgoTDgyEcquRwOmKDF8agETxeyuiIywwsTYjEo2BG/55FVoXdQ8y/eXlfpNHkcRTuAUquDBFdThDhrQBAICnuEV3hzlvDjvzseiteDkM8fwR87nD707kFk=</latexit><latexit sha1_base64="QNA18qBfAJrCCEwMxK/B/I1M350=">AAAB8HicbZDLSgMxFIbP1Futt6pLN8Ei1E2ZEUGXRTcuK9iLtEPJpJk2NMkMSUZahj6FGxeKuPVx3Pk2ZtpZaOsPgY//nEPO+YOYM21c99sprK1vbG4Vt0s7u3v7B+XDo5aOEkVok0Q8Up0Aa8qZpE3DDKedWFEsAk7bwfg2q7efqNIskg9mGlNf4KFkISPYWOsx6aeTyaw6Oe+XK27NnQutgpdDBXI1+uWv3iAiiaDSEI617npubPwUK8MIp7NSL9E0xmSMh7RrUWJBtZ/OF56hM+sMUBgp+6RBc/f3RIqF1lMR2E6BzUgv1zLzv1o3MeG1nzIZJ4ZKsvgoTDgyEcquRwOmKDF8agETxeyuiIywwsTYjEo2BG/55FVoXdQ8y/eXlfpNHkcRTuAUquDBFdThDhrQBAICnuEV3hzlvDjvzseiteDkM8fwR87nD707kFk=</latexit><latexit sha1_base64="QNA18qBfAJrCCEwMxK/B/I1M350=">AAAB8HicbZDLSgMxFIbP1Futt6pLN8Ei1E2ZEUGXRTcuK9iLtEPJpJk2NMkMSUZahj6FGxeKuPVx3Pk2ZtpZaOsPgY//nEPO+YOYM21c99sprK1vbG4Vt0s7u3v7B+XDo5aOEkVok0Q8Up0Aa8qZpE3DDKedWFEsAk7bwfg2q7efqNIskg9mGlNf4KFkISPYWOsx6aeTyaw6Oe+XK27NnQutgpdDBXI1+uWv3iAiiaDSEI617npubPwUK8MIp7NSL9E0xmSMh7RrUWJBtZ/OF56hM+sMUBgp+6RBc/f3RIqF1lMR2E6BzUgv1zLzv1o3MeG1nzIZJ4ZKsvgoTDgyEcquRwOmKDF8agETxeyuiIywwsTYjEo2BG/55FVoXdQ8y/eXlfpNHkcRTuAUquDBFdThDhrQBAICnuEV3hzlvDjvzseiteDkM8fwR87nD707kFk=</latexit>
ux(x)<latexit sha1_base64="tjArAx6NhzxT95XjS0tfOz6ea7o=">AAAB73icbZBNSwMxEIZn61etX1WPXoJFqJeyK4Iei148VrAf0C4lm07b0Gx2TbLSsvRPePGgiFf/jjf/jWm7B219IfDwzgyZeYNYcG1c99vJra1vbG7ltws7u3v7B8XDo4aOEsWwziIRqVZANQousW64EdiKFdIwENgMRrezevMJleaRfDCTGP2QDiTvc0aNtVpJNx1Py+PzbrHkVty5yCp4GZQgU61b/Or0IpaEKA0TVOu258bGT6kynAmcFjqJxpiyER1g26KkIWo/ne87JWfW6ZF+pOyThszd3xMpDbWehIHtDKkZ6uXazPyv1k5M/9pPuYwTg5ItPuongpiIzI4nPa6QGTGxQJnidlfChlRRZmxEBRuCt3zyKjQuKp7l+8tS9SaLIw8ncApl8OAKqnAHNagDAwHP8ApvzqPz4rw7H4vWnJPNHMMfOZ8/3rOP1w==</latexit><latexit sha1_base64="tjArAx6NhzxT95XjS0tfOz6ea7o=">AAAB73icbZBNSwMxEIZn61etX1WPXoJFqJeyK4Iei148VrAf0C4lm07b0Gx2TbLSsvRPePGgiFf/jjf/jWm7B219IfDwzgyZeYNYcG1c99vJra1vbG7ltws7u3v7B8XDo4aOEsWwziIRqVZANQousW64EdiKFdIwENgMRrezevMJleaRfDCTGP2QDiTvc0aNtVpJNx1Py+PzbrHkVty5yCp4GZQgU61b/Or0IpaEKA0TVOu258bGT6kynAmcFjqJxpiyER1g26KkIWo/ne87JWfW6ZF+pOyThszd3xMpDbWehIHtDKkZ6uXazPyv1k5M/9pPuYwTg5ItPuongpiIzI4nPa6QGTGxQJnidlfChlRRZmxEBRuCt3zyKjQuKp7l+8tS9SaLIw8ncApl8OAKqnAHNagDAwHP8ApvzqPz4rw7H4vWnJPNHMMfOZ8/3rOP1w==</latexit><latexit sha1_base64="tjArAx6NhzxT95XjS0tfOz6ea7o=">AAAB73icbZBNSwMxEIZn61etX1WPXoJFqJeyK4Iei148VrAf0C4lm07b0Gx2TbLSsvRPePGgiFf/jjf/jWm7B219IfDwzgyZeYNYcG1c99vJra1vbG7ltws7u3v7B8XDo4aOEsWwziIRqVZANQousW64EdiKFdIwENgMRrezevMJleaRfDCTGP2QDiTvc0aNtVpJNx1Py+PzbrHkVty5yCp4GZQgU61b/Or0IpaEKA0TVOu258bGT6kynAmcFjqJxpiyER1g26KkIWo/ne87JWfW6ZF+pOyThszd3xMpDbWehIHtDKkZ6uXazPyv1k5M/9pPuYwTg5ItPuongpiIzI4nPa6QGTGxQJnidlfChlRRZmxEBRuCt3zyKjQuKp7l+8tS9SaLIw8ncApl8OAKqnAHNagDAwHP8ApvzqPz4rw7H4vWnJPNHMMfOZ8/3rOP1w==</latexit><latexit sha1_base64="tjArAx6NhzxT95XjS0tfOz6ea7o=">AAAB73icbZBNSwMxEIZn61etX1WPXoJFqJeyK4Iei148VrAf0C4lm07b0Gx2TbLSsvRPePGgiFf/jjf/jWm7B219IfDwzgyZeYNYcG1c99vJra1vbG7ltws7u3v7B8XDo4aOEsWwziIRqVZANQousW64EdiKFdIwENgMRrezevMJleaRfDCTGP2QDiTvc0aNtVpJNx1Py+PzbrHkVty5yCp4GZQgU61b/Or0IpaEKA0TVOu258bGT6kynAmcFjqJxpiyER1g26KkIWo/ne87JWfW6ZF+pOyThszd3xMpDbWehIHtDKkZ6uXazPyv1k5M/9pPuYwTg5ItPuongpiIzI4nPa6QGTGxQJnidlfChlRRZmxEBRuCt3zyKjQuKp7l+8tS9SaLIw8ncApl8OAKqnAHNagDAwHP8ApvzqPz4rw7H4vWnJPNHMMfOZ8/3rOP1w==</latexit>
u(x)<latexit sha1_base64="xyz5fYmv7DAMjqhOp6NQw7olzvk=">AAAB63icbZDNSgMxFIXv1L9a/6ou3QSLUDdlRgRdFt24rOC0hXYomTTThiaZIcmIZegruHGhiFtfyJ1vY6adhbYeCHycey+594QJZ9q47rdTWlvf2Nwqb1d2dvf2D6qHR20dp4pQn8Q8Vt0Qa8qZpL5hhtNuoigWIaedcHKb1zuPVGkWywczTWgg8EiyiBFsciutP50PqjW34c6FVsEroAaFWoPqV38Yk1RQaQjHWvc8NzFBhpVhhNNZpZ9qmmAywSPasyixoDrI5rvO0Jl1hiiKlX3SoLn7eyLDQuupCG2nwGasl2u5+V+tl5roOsiYTFJDJVl8FKUcmRjlh6MhU5QYPrWAiWJ2V0TGWGFibDwVG4K3fPIqtC8anuX7y1rzpoijDCdwCnXw4AqacAct8IHAGJ7hFd4c4bw4787HorXkFDPH8EfO5w+DSo3g</latexit><latexit sha1_base64="xyz5fYmv7DAMjqhOp6NQw7olzvk=">AAAB63icbZDNSgMxFIXv1L9a/6ou3QSLUDdlRgRdFt24rOC0hXYomTTThiaZIcmIZegruHGhiFtfyJ1vY6adhbYeCHycey+594QJZ9q47rdTWlvf2Nwqb1d2dvf2D6qHR20dp4pQn8Q8Vt0Qa8qZpL5hhtNuoigWIaedcHKb1zuPVGkWywczTWgg8EiyiBFsciutP50PqjW34c6FVsEroAaFWoPqV38Yk1RQaQjHWvc8NzFBhpVhhNNZpZ9qmmAywSPasyixoDrI5rvO0Jl1hiiKlX3SoLn7eyLDQuupCG2nwGasl2u5+V+tl5roOsiYTFJDJVl8FKUcmRjlh6MhU5QYPrWAiWJ2V0TGWGFibDwVG4K3fPIqtC8anuX7y1rzpoijDCdwCnXw4AqacAct8IHAGJ7hFd4c4bw4787HorXkFDPH8EfO5w+DSo3g</latexit><latexit sha1_base64="xyz5fYmv7DAMjqhOp6NQw7olzvk=">AAAB63icbZDNSgMxFIXv1L9a/6ou3QSLUDdlRgRdFt24rOC0hXYomTTThiaZIcmIZegruHGhiFtfyJ1vY6adhbYeCHycey+594QJZ9q47rdTWlvf2Nwqb1d2dvf2D6qHR20dp4pQn8Q8Vt0Qa8qZpL5hhtNuoigWIaedcHKb1zuPVGkWywczTWgg8EiyiBFsciutP50PqjW34c6FVsEroAaFWoPqV38Yk1RQaQjHWvc8NzFBhpVhhNNZpZ9qmmAywSPasyixoDrI5rvO0Jl1hiiKlX3SoLn7eyLDQuupCG2nwGasl2u5+V+tl5roOsiYTFJDJVl8FKUcmRjlh6MhU5QYPrWAiWJ2V0TGWGFibDwVG4K3fPIqtC8anuX7y1rzpoijDCdwCnXw4AqacAct8IHAGJ7hFd4c4bw4787HorXkFDPH8EfO5w+DSo3g</latexit><latexit sha1_base64="xyz5fYmv7DAMjqhOp6NQw7olzvk=">AAAB63icbZDNSgMxFIXv1L9a/6ou3QSLUDdlRgRdFt24rOC0hXYomTTThiaZIcmIZegruHGhiFtfyJ1vY6adhbYeCHycey+594QJZ9q47rdTWlvf2Nwqb1d2dvf2D6qHR20dp4pQn8Q8Vt0Qa8qZpL5hhtNuoigWIaedcHKb1zuPVGkWywczTWgg8EiyiBFsciutP50PqjW34c6FVsEroAaFWoPqV38Yk1RQaQjHWvc8NzFBhpVhhNNZpZ9qmmAywSPasyixoDrI5rvO0Jl1hiiKlX3SoLn7eyLDQuupCG2nwGasl2u5+V+tl5roOsiYTFJDJVl8FKUcmRjlh6MhU5QYPrWAiWJ2V0TGWGFibDwVG4K3fPIqtC8anuX7y1rzpoijDCdwCnXw4AqacAct8IHAGJ7hFd4c4bw4787HorXkFDPH8EfO5w+DSo3g</latexit>
Our approach …
Input
The Neural Network
(~x, t)<latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit>
ˆu<latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit>
Output
Space
Time
<latexit sha1_base64="f64HEF6znFEZoH5kmjcDbs/59Jo=">AAAB7XicbZBNSwMxEIZn61etX1WPXoJFqCBlVwQ9Fr14rGA/oF1KNk3b2OxmSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZeYNYCoOu++3kVlbX1jfym4Wt7Z3dveL+QcOoRDNeZ0oq3Qqo4VJEvI4CJW/FmtMwkLwZjG6m9eYj10ao6B7HMfdDOohEXzCK1mok5aczPO0WS27FnYksg5dBCTLVusWvTk+xJOQRMkmNaXtujH5KNQom+aTQSQyPKRvRAW9bjGjIjZ/Otp2QE+v0SF9p+yIkM/f3REpDY8ZhYDtDikOzWJua/9XaCfav/FREcYI8YvOP+okkqMj0dNITmjOUYwuUaWF3JWxINWVoAyrYELzFk5ehcV7xLN9dlKrXWRx5OIJjKIMHl1CFW6hBHRg8wDO8wpujnBfn3fmYt+acbOYQ/sj5/AHCpY6U</latexit>
Construct the library w.r.t. the inferred solution
Θ =
! ! !
u ux uxu
! ! !
! ! !
uxx uxxu uxxux
! ! !
"
⎛
⎝
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
^ ^ ^ ^ ^ ^ ^ ^ ^
Include the regression task within the neural network
How do we train a neural network
Optimise loss function:
L = LMSE + LReg + LL1<latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit>
Learn the mapping
(~x, t) ! ~u<latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit>
Approximate the data as good as possible …
How do we train a neural network
Optimise loss function:
L = LMSE + LReg + LL1<latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit>
Learn the mapping
(~x, t) ! ~u<latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit>
Approximate the data as good as possible …
Regression
(Seek the PDE)
Discover the differential equation
L1 Penalty
Promoting sparsity
How do we train a neural network
Optimise loss function:
L = LMSE + LReg + LL1<latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit>
Learn the mapping
(~x, t) ! ~u<latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit>
Approximate the data as good as possible …
Regression
(Seek the PDE)
Discover the differential equation
Convergence
ξ
optimised
MSE
minimized
c)
Training the neural network
c)
<latexit sha1_base64="1Gq7mrW1/ZKuQjtOjsnM1gDsGDU=">AAACBHicbZDJSgNBEIZ74hbjFvWYS2MQIkiYEUEvQtCLxwjZIBNCT6eSNOlZ6K4JCUMOXnwVLx4U8epDePNt7CwHTfyh4eOvKqrr9yIpNNr2t5VaW9/Y3EpvZ3Z29/YPsodHNR3GikOVhzJUDY9pkCKAKgqU0IgUMN+TUPcGd9N6fQhKizCo4DiCls96gegKztBY7WwubmNhdI5n9Ia6lT4go+4QeOKOxKSdzdtFeya6Cs4C8mShcjv75XZCHvsQIJdM66ZjR9hKmELBJUwybqwhYnzAetA0GDAfdCuZHTGhp8bp0G6ozAuQztzfEwnztR77nun0Gfb1cm1q/ldrxti9biUiiGKEgM8XdWNJMaTTRGhHKOAoxwYYV8L8lfI+U4yjyS1jQnCWT16F2kXRMfxwmS/dLuJIkxw5IQXikCtSIvekTKqEk0fyTF7Jm/VkvVjv1se8NWUtZo7JH1mfPysqlyg=</latexit>
⇠⇤
=<latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit>
Neural Network Threshold
ˆuu
Auto. Diff.
∂t ˆu = Θξ
Library
Include in cost function
NN converged
ξ∗
= ξ ·
||Θ||
||ut||
Solution
Sparsity converged
×
×
×
×
× ×
×
×
×
×
×
keep
set to
zero
keep
ξ∗
Update cost function
L = MSE(u, ˆu)
+ MSE(∂t ˆu, Θξ)
+ L1(ξ)
= ...
u
ux
uux
u2
uxx
Found solution
∂tu = uux − 0.1uxx
0
0
1
0
...
-0.1
0
uux
uxx
u
ux
u2
uxx
Convergence
ξ
optimised
MSE
minimized
c)
Training the coefficient vector
<latexit sha1_base64="1Gq7mrW1/ZKuQjtOjsnM1gDsGDU=">AAACBHicbZDJSgNBEIZ74hbjFvWYS2MQIkiYEUEvQtCLxwjZIBNCT6eSNOlZ6K4JCUMOXnwVLx4U8epDePNt7CwHTfyh4eOvKqrr9yIpNNr2t5VaW9/Y3EpvZ3Z29/YPsodHNR3GikOVhzJUDY9pkCKAKgqU0IgUMN+TUPcGd9N6fQhKizCo4DiCls96gegKztBY7WwubmNhdI5n9Ia6lT4go+4QeOKOxKSdzdtFeya6Cs4C8mShcjv75XZCHvsQIJdM66ZjR9hKmELBJUwybqwhYnzAetA0GDAfdCuZHTGhp8bp0G6ozAuQztzfEwnztR77nun0Gfb1cm1q/ldrxti9biUiiGKEgM8XdWNJMaTTRGhHKOAoxwYYV8L8lfI+U4yjyS1jQnCWT16F2kXRMfxwmS/dLuJIkxw5IQXikCtSIvekTKqEk0fyTF7Jm/VkvVjv1se8NWUtZo7JH1mfPysqlyg=</latexit>
Dataset Neural Network Threshold
ˆuu
Auto. Diff.
∂t ˆu = Θξ
Library
Include in cost function
NN converged
ξ∗
= ξ ·
||Θ||
||ut||
Sparsity c
×
×
×
×
× ×
×
×
×
×
×
keep
set to
zero
keep
ξ∗
Update cost function
Space
Time
Burgers equation
∂tu = uux − 0.1uxx
L = MSE(u, ˆu)
+ MSE(∂t ˆu, Θξ)
+ L1(ξ)
= ...
u
ux
uux
u2
uxx
Space
Time
<latexit sha1_base64="f64HEF6znFEZoH5kmjcDbs/59Jo=">AAAB7XicbZBNSwMxEIZn61etX1WPXoJFqCBlVwQ9Fr14rGA/oF1KNk3b2OxmSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZeYNYCoOu++3kVlbX1jfym4Wt7Z3dveL+QcOoRDNeZ0oq3Qqo4VJEvI4CJW/FmtMwkLwZjG6m9eYj10ao6B7HMfdDOohEXzCK1mok5aczPO0WS27FnYksg5dBCTLVusWvTk+xJOQRMkmNaXtujH5KNQom+aTQSQyPKRvRAW9bjGjIjZ/Otp2QE+v0SF9p+yIkM/f3REpDY8ZhYDtDikOzWJua/9XaCfav/FREcYI8YvOP+okkqMj0dNITmjOUYwuUaWF3JWxINWVoAyrYELzFk5ehcV7xLN9dlKrXWRx5OIJjKIMHl1CFW6hBHRg8wDO8wpujnBfn3fmYt+acbOYQ/sj5/AHCpY6U</latexit>
SolutionNoisy Data Regression
Code available:
github.com/PhIMaL
L = LMSE + LReg + LL1<latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit>
Neural network
(~x, t)<latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit>
ˆu<latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit>
⇠⇤
=<latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit>
Neural Network Threshold
ˆuu
Auto. Diff.
∂t ˆu = Θξ
Library
Include in cost function
NN converged
ξ∗
= ξ ·
||Θ||
||ut||
Solution
Sparsity converged
×
×
×
×
× ×
×
×
×
×
×
keep
set to
zero
keep
ξ∗
Update cost function
L = MSE(u, ˆu)
+ MSE(∂t ˆu, Θξ)
+ L1(ξ)
= ...
u
ux
uux
u2
uxx
Found solution
∂tu = uux − 0.1uxx
0
0
1
0
...
-0.1
0
uux
uxx
u
ux
u2
uxx
DeepMoD
Model discovery
neural networks
DeepMoD
Model discovery
neural networks
u
x
<latexit sha1_base64="6yAhDb2lEN8ufJi3rjVOCio5Mf8=">AAACAnicbZDLSgMxFIYz9VbrbdSVuAkWQRDLjAi6EYpuXFawF2iHIZOmbWgmMyQn0jIUN76KGxeKuPUp3Pk2ppeFVn8IfPznHE7OH6WCa/C8Lye3sLi0vJJfLaytb2xuuds7NZ0YRVmVJiJRjYhoJrhkVeAgWCNVjMSRYPWofz2u1++Z0jyRdzBMWRCTruQdTglYK3T3TAj4Ep8YbMIBPsYtOaZsMBiFbtEreRPhv+DPoIhmqoTuZ6udUBMzCVQQrZu+l0KQEQWcCjYqtIxmKaF90mVNi5LETAfZ5IQRPrROG3cSZZ8EPHF/TmQk1noYR7YzJtDT87Wx+V+taaBzEWRcpgaYpNNFHSMwJHicB25zxSiIoQVCFbd/xbRHFKFgUyvYEPz5k/9C7bTkW749K5avZnHk0T46QEfIR+eojG5QBVURRQ/oCb2gV+fReXbenPdpa86ZzeyiX3I+vgEk2JX+</latexit>
Why regression inside neural network?
Resilience to Noise/ sampling size
Relative error on coefficients
<latexit sha1_base64="6yAhDb2lEN8ufJi3rjVOCio5Mf8=">AAACAnicbZDLSgMxFIYz9VbrbdSVuAkWQRDLjAi6EYpuXFawF2iHIZOmbWgmMyQn0jIUN76KGxeKuPUp3Pk2ppeFVn8IfPznHE7OH6WCa/C8Lye3sLi0vJJfLaytb2xuuds7NZ0YRVmVJiJRjYhoJrhkVeAgWCNVjMSRYPWofz2u1++Z0jyRdzBMWRCTruQdTglYK3T3TAj4Ep8YbMIBPsYtOaZsMBiFbtEreRPhv+DPoIhmqoTuZ6udUBMzCVQQrZu+l0KQEQWcCjYqtIxmKaF90mVNi5LETAfZ5IQRPrROG3cSZZ8EPHF/TmQk1noYR7YzJtDT87Wx+V+taaBzEWRcpgaYpNNFHSMwJHicB25zxSiIoQVCFbd/xbRHFKFgUyvYEPz5k/9C7bTkW749K5avZnHk0T46QEfIR+eojG5QBVURRQ/oCb2gV+fReXbenPdpa86ZzeyiX3I+vgEk2JX+</latexit>
Burgers’ equation
Applications in physics
Higher order PDEs
<latexit sha1_base64="89mN2eImzrIm0QzNLSbmbxofcBY=">AAACAHicbZDLSsNAFIYn9VbrLerChZvBIrixJCLqRii6cVnBXqANYTKdtEMnkzAXSQnZ+CpuXCji1sdw59s4bbPQ1gMzfPz/OcycP0gYlcpxvq3S0vLK6lp5vbKxubW9Y+/utWSsBSZNHLNYdAIkCaOcNBVVjHQSQVAUMNIORrcTv/1IhKQxf1DjhHgRGnAaUoyUkXz7QPsKXsMLqKH2U3hq7ixN09y3q07NmRZcBLeAKiiq4dtfvX6MdUS4wgxJ2XWdRHkZEopiRvJKT0uSIDxCA9I1yFFEpJdNF8jhsVH6MIyFOVzBqfp7IkORlOMoMJ0RUkM5703E/7yuVuGVl1GeaEU4nj0UagZVDCdpwD4VBCs2NoCwoOavEA+RQFiZzComBHd+5UVondVcw/fn1fpNEUcZHIIjcAJccAnq4A40QBNgkINn8ArerCfrxXq3PmatJauY2Qd/yvr8Aa06lS4=</latexit>
t = 5
t = 7
t = 9
Ground truth Sampled Inferred
y
y
y
Coupled equations
<latexit sha1_base64="oJZFq0o7Dpv+0FhHmocEH1HfGbM=">AAACJ3icbZDLSgMxFIYzXmu9jbp0EyxCXVhmRNCNUtSFywr2Ap0yZNJMG5rJDMkZtZS+jRtfxY2gIrr0TUzbWWjrgZCP/z+H5PxBIrgGx/my5uYXFpeWcyv51bX1jU17a7um41RRVqWxiFUjIJoJLlkVOAjWSBQjUSBYPehdjvz6HVOax/IW+glrRaQjecgpASP59nnqAz7DniSBIB5tx4A9wUIo4iufZzJO8SH2Hri5M+Eee4p3unDg2wWn5IwLz4KbQQFlVfHtV68d0zRiEqggWjddJ4HWgCjgVLBh3ks1SwjtkQ5rGpQkYro1GO85xPtGaeMwVuZIwGP198SARFr3o8B0RgS6etobif95zRTC09aAyyQFJunkoTAVGGI8Cg23uWIURN8AoYqbv2LaJYpQMNHmTQju9MqzUDsquYZvjgvliyyOHNpFe6iIXHSCyugaVVAVUfSIntEbereerBfrw/qctM5Z2cwO+lPW9w97rKPZ</latexit>
<latexit sha1_base64="TZ48ZV1oiCE1tUaFN922vZ54mwQ=">AAACCXicbZDLSgMxFIYz9VbrrerSzcEiCGKZEUE3QtEuXFawF2iHIZOmbWjmQnLGUkq3bnwVNy4UcesbuPNtTNtZaOuBhI//P4fk/H4shUbb/rYyS8srq2vZ9dzG5tb2Tn53r6ajRDFeZZGMVMOnmksR8ioKlLwRK04DX/K637+Z+PUHrrSIwnscxtwNaDcUHcEoGsnLw8BDuAIoewNolblECgM4hb65T6AHCXj5gl20pwWL4KRQIGlVvPxXqx2xJOAhMkm1bjp2jO6IKhRM8nGulWgeU9anXd40GNKAa3c03WQMR0ZpQydS5oQIU/X3xIgGWg8D33QGFHt63puI/3nNBDuX7kiEcYI8ZLOHOokEjGASC7SF4gzl0ABlSpi/AutRRRma8HImBGd+5UWonRUdw3fnhdJ1GkeWHJBDckwcckFK5JZUSJUw8kieySt5s56sF+vd+pi1Zqx0Zp/8KevzB0yCluA=</latexit>
u<latexit sha1_base64="0ivuOvsA1zGj29z/88YG+b8aLnI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjC/YD2lA220m7drMJuxuhhP4CLx4U8epP8ua/cdvmoK0vLDy8M8POvEEiuDau++0UNja3tneKu6W9/YPDo/LxSVvHqWLYYrGIVTegGgWX2DLcCOwmCmkUCOwEk7t5vfOESvNYPphpgn5ER5KHnFFjrWY6KFfcqrsQWQcvhwrkagzKX/1hzNIIpWGCat3z3MT4GVWGM4GzUj/VmFA2oSPsWZQ0Qu1ni0Vn5MI6QxLGyj5pyML9PZHRSOtpFNjOiJqxXq3Nzf9qvdSEN37GZZIalGz5UZgKYmIyv5oMuUJmxNQCZYrbXQkbU0WZsdmUbAje6snr0L6qepab15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh85nz/hr4z5</latexit><latexit sha1_base64="0ivuOvsA1zGj29z/88YG+b8aLnI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjC/YD2lA220m7drMJuxuhhP4CLx4U8epP8ua/cdvmoK0vLDy8M8POvEEiuDau++0UNja3tneKu6W9/YPDo/LxSVvHqWLYYrGIVTegGgWX2DLcCOwmCmkUCOwEk7t5vfOESvNYPphpgn5ER5KHnFFjrWY6KFfcqrsQWQcvhwrkagzKX/1hzNIIpWGCat3z3MT4GVWGM4GzUj/VmFA2oSPsWZQ0Qu1ni0Vn5MI6QxLGyj5pyML9PZHRSOtpFNjOiJqxXq3Nzf9qvdSEN37GZZIalGz5UZgKYmIyv5oMuUJmxNQCZYrbXQkbU0WZsdmUbAje6snr0L6qepab15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh85nz/hr4z5</latexit><latexit sha1_base64="0ivuOvsA1zGj29z/88YG+b8aLnI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjC/YD2lA220m7drMJuxuhhP4CLx4U8epP8ua/cdvmoK0vLDy8M8POvEEiuDau++0UNja3tneKu6W9/YPDo/LxSVvHqWLYYrGIVTegGgWX2DLcCOwmCmkUCOwEk7t5vfOESvNYPphpgn5ER5KHnFFjrWY6KFfcqrsQWQcvhwrkagzKX/1hzNIIpWGCat3z3MT4GVWGM4GzUj/VmFA2oSPsWZQ0Qu1ni0Vn5MI6QxLGyj5pyML9PZHRSOtpFNjOiJqxXq3Nzf9qvdSEN37GZZIalGz5UZgKYmIyv5oMuUJmxNQCZYrbXQkbU0WZsdmUbAje6snr0L6qepab15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh85nz/hr4z5</latexit><latexit sha1_base64="0ivuOvsA1zGj29z/88YG+b8aLnI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjC/YD2lA220m7drMJuxuhhP4CLx4U8epP8ua/cdvmoK0vLDy8M8POvEEiuDau++0UNja3tneKu6W9/YPDo/LxSVvHqWLYYrGIVTegGgWX2DLcCOwmCmkUCOwEk7t5vfOESvNYPphpgn5ER5KHnFFjrWY6KFfcqrsQWQcvhwrkagzKX/1hzNIIpWGCat3z3MT4GVWGM4GzUj/VmFA2oSPsWZQ0Qu1ni0Vn5MI6QxLGyj5pyML9PZHRSOtpFNjOiJqxXq3Nzf9qvdSEN37GZZIalGz5UZgKYmIyv5oMuUJmxNQCZYrbXQkbU0WZsdmUbAje6snr0L6qepab15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh85nz/hr4z5</latexit>
w<latexit sha1_base64="sGagZRpsBDQQXLOTtGoRB0Vi9Sc=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OKxBfsBbSib7aRdu9mE3Y1SQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOyZl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJr/2MyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVR9Sw3Liu1mzyOIpzAKZyDB1dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A5LeM+w==</latexit><latexit sha1_base64="sGagZRpsBDQQXLOTtGoRB0Vi9Sc=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OKxBfsBbSib7aRdu9mE3Y1SQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOyZl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJr/2MyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVR9Sw3Liu1mzyOIpzAKZyDB1dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A5LeM+w==</latexit><latexit sha1_base64="sGagZRpsBDQQXLOTtGoRB0Vi9Sc=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OKxBfsBbSib7aRdu9mE3Y1SQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOyZl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJr/2MyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVR9Sw3Liu1mzyOIpzAKZyDB1dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A5LeM+w==</latexit><latexit sha1_base64="sGagZRpsBDQQXLOTtGoRB0Vi9Sc=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OKxBfsBbSib7aRdu9mE3Y1SQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOyZl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJr/2MyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVR9Sw3Liu1mzyOIpzAKZyDB1dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A5LeM+w==</latexit>
:particle density
:attractant density
2D equations
Raw images Density field
ut = 0.3uy + 0.01uxx + 0.01uyy<latexit sha1_base64="giCCDNoPIOmus3F40Yxqtlm2jVw=">AAACEnicbZBLS8NAEMc39VXrK+rRy2IRFKEkKuhFKHrxWME+oA1hs922SzebsA9pCPkMXvwqXjwo4tWTN7+N2zaH2jqw8Jv/zDA7/yBmVCrH+bEKS8srq2vF9dLG5tb2jr2715CRFpjUccQi0QqQJIxyUldUMdKKBUFhwEgzGN6O681HIiSN+INKYuKFqM9pj2KkjOTbJ9pX8Bo6lXOo/QSeGnJcg+lolM1kSZL5dtlkk4CL4OZQBnnUfPu7042wDglXmCEp264TKy9FQlHMSFbqaElihIeoT9oGOQqJ9NLJSRk8MkoX9iJhHldwos5OpCiUMgkD0xkiNZDztbH4X62tVe/KSymPtSIcTxf1NIMqgmN/YJcKghVLDCAsqPkrxAMkEFbGxZIxwZ0/eREaZxXX8P1FuXqT21EEB+AQHAMXXIIquAM1UAcYPIEX8AberWfr1fqwPqetBSuf2Qd/wvr6BTy+mhY=</latexit><latexit sha1_base64="giCCDNoPIOmus3F40Yxqtlm2jVw=">AAACEnicbZBLS8NAEMc39VXrK+rRy2IRFKEkKuhFKHrxWME+oA1hs922SzebsA9pCPkMXvwqXjwo4tWTN7+N2zaH2jqw8Jv/zDA7/yBmVCrH+bEKS8srq2vF9dLG5tb2jr2715CRFpjUccQi0QqQJIxyUldUMdKKBUFhwEgzGN6O681HIiSN+INKYuKFqM9pj2KkjOTbJ9pX8Bo6lXOo/QSeGnJcg+lolM1kSZL5dtlkk4CL4OZQBnnUfPu7042wDglXmCEp264TKy9FQlHMSFbqaElihIeoT9oGOQqJ9NLJSRk8MkoX9iJhHldwos5OpCiUMgkD0xkiNZDztbH4X62tVe/KSymPtSIcTxf1NIMqgmN/YJcKghVLDCAsqPkrxAMkEFbGxZIxwZ0/eREaZxXX8P1FuXqT21EEB+AQHAMXXIIquAM1UAcYPIEX8AberWfr1fqwPqetBSuf2Qd/wvr6BTy+mhY=</latexit><latexit sha1_base64="giCCDNoPIOmus3F40Yxqtlm2jVw=">AAACEnicbZBLS8NAEMc39VXrK+rRy2IRFKEkKuhFKHrxWME+oA1hs922SzebsA9pCPkMXvwqXjwo4tWTN7+N2zaH2jqw8Jv/zDA7/yBmVCrH+bEKS8srq2vF9dLG5tb2jr2715CRFpjUccQi0QqQJIxyUldUMdKKBUFhwEgzGN6O681HIiSN+INKYuKFqM9pj2KkjOTbJ9pX8Bo6lXOo/QSeGnJcg+lolM1kSZL5dtlkk4CL4OZQBnnUfPu7042wDglXmCEp264TKy9FQlHMSFbqaElihIeoT9oGOQqJ9NLJSRk8MkoX9iJhHldwos5OpCiUMgkD0xkiNZDztbH4X62tVe/KSymPtSIcTxf1NIMqgmN/YJcKghVLDCAsqPkrxAMkEFbGxZIxwZ0/eREaZxXX8P1FuXqT21EEB+AQHAMXXIIquAM1UAcYPIEX8AberWfr1fqwPqetBSuf2Qd/wvr6BTy+mhY=</latexit><latexit sha1_base64="giCCDNoPIOmus3F40Yxqtlm2jVw=">AAACEnicbZBLS8NAEMc39VXrK+rRy2IRFKEkKuhFKHrxWME+oA1hs922SzebsA9pCPkMXvwqXjwo4tWTN7+N2zaH2jqw8Jv/zDA7/yBmVCrH+bEKS8srq2vF9dLG5tb2jr2715CRFpjUccQi0QqQJIxyUldUMdKKBUFhwEgzGN6O681HIiSN+INKYuKFqM9pj2KkjOTbJ9pX8Bo6lXOo/QSeGnJcg+lolM1kSZL5dtlkk4CL4OZQBnnUfPu7042wDglXmCEp264TKy9FQlHMSFbqaElihIeoT9oGOQqJ9NLJSRk8MkoX9iJhHldwos5OpCiUMgkD0xkiNZDztbH4X62tVe/KSymPtSIcTxf1NIMqgmN/YJcKghVLDCAsqPkrxAMkEFbGxZIxwZ0/eREaZxXX8P1FuXqT21EEB+AQHAMXXIIquAM1UAcYPIEX8AberWfr1fqwPqetBSuf2Qd/wvr6BTy+mhY=</latexit>
Underlying AD equation
Apply to real data …
Model discovery from experimental data
Single ParticleTracking
Time
xx
Time
Time
Time
Trajectory
<latexit sha1_base64="A2yY/FcKHQddACwl6l5TaATnAyI=">AAACFXicbZDLSsQwFIZT7463qks3wUEQFGkHQTeC6MalguMMTGtJM+kYTNOSnMgMpS/hxldx40IRt4I738bMZaEz/hD4+M85nJw/zgXX4HnfztT0zOzc/MJiZWl5ZXXNXd+40ZlRlNVpJjLVjIlmgktWBw6CNXPFSBoL1ojvz/v1xgNTmmfyGno5C1PSkTzhlIC1InffRAVAifdwkChCC78sAiAmoiU2EeATHEhzW7NcdLtl5Fa9A28gPAn+CKpopMvI/QraGTUpk0AF0brlezmEBVHAqWBlJTCa5YTekw5rWZQkZTosBleVeMc6bZxkyj4JeOD+nihIqnUvjW1nSuBOj9f65n+1loHkOCy4zA0wSYeLEiMwZLgfEW5zxSiIngVCFbd/xfSO2HTABlmxIfjjJ0/CTe3At3x1WD09G8WxgLbQNtpFPjpCp+gCXaI6ougRPaNX9OY8OS/Ou/MxbJ1yRjOb6I+czx/MZJ6T</latexit>
<latexit sha1_base64="l3iwXAx+BJVV3L+FIIt6mI0zRgc=">AAACAnicbZDLSsNAFIYn9VbrLepK3AwWQRBKIoJuhKIuXFawF2hDmEym7dDJJMyltITgxldx40IRtz6FO9/GaZuFtv4w8PGfczhz/iBhVCrH+bYKS8srq2vF9dLG5tb2jr2715CxFpjUccxi0QqQJIxyUldUMdJKBEFRwEgzGNxM6s0hEZLG/EGNE+JFqMdpl2KkjOXbB9pPVQav4C00NBpl8BQO/RBq6Ntlp+JMBRfBzaEMctV8+6sTxlhHhCvMkJRt10mUlyKhKGYkK3W0JAnCA9QjbYMcRUR66fSEDB4bJ4TdWJjHFZy6vydSFEk5jgLTGSHVl/O1iflfra1V99JLKU+0IhzPFnU1gyqGkzxgSAXBio0NICyo+SvEfSQQVia1kgnBnT95ERpnFdfw/Xm5ep3HUQSH4AicABdcgCq4AzVQBxg8gmfwCt6sJ+vFerc+Zq0FK5/ZB39kff4A+62V2w==</latexit>
<latexit sha1_base64="q8SSRp0DvPa9XRci5Cdg/mAJUPE=">AAAB+nicbZDLSsNAFIZP6q3WW6pLN4NFcFUSEXQjFHXhsoK9QBvCZDpth04mYWaiLTGP4saFIm59Ene+jdM2C239YeDjP+dwzvxBzJnSjvNtFVZW19Y3ipulre2d3T27vN9UUSIJbZCIR7IdYEU5E7Shmea0HUuKw4DTVjC6ntZbD1QqFol7PYmpF+KBYH1GsDaWb5cTP9UZukQ3yNB4nCHfrjhVZya0DG4OFchV9+2vbi8iSUiFJhwr1XGdWHsplpoRTrNSN1E0xmSEB7RjUOCQKi+dnZ6hY+P0UD+S5gmNZu7viRSHSk3CwHSGWA/VYm1q/lfrJLp/4aVMxImmgswX9ROOdISmOaAek5RoPjGAiWTmVkSGWGKiTVolE4K7+OVlaJ5WXcN3Z5XaVR5HEQ7hCE7AhXOowS3UoQEEHuEZXuHNerJerHfrY95asPKZA/gj6/MHPD2TUg==</latexit>
Random walk
Advected random walk
Persistant random walk
x
Density
estimation
Model
discovery
Time
xx
Time
Time
Time
Trajectory
<latexit sha1_base64="A2yY/FcKHQddACwl6l5TaATnAyI=">AAACFXicbZDLSsQwFIZT7463qks3wUEQFGkHQTeC6MalguMMTGtJM+kYTNOSnMgMpS/hxldx40IRt4I738bMZaEz/hD4+M85nJw/zgXX4HnfztT0zOzc/MJiZWl5ZXXNXd+40ZlRlNVpJjLVjIlmgktWBw6CNXPFSBoL1ojvz/v1xgNTmmfyGno5C1PSkTzhlIC1InffRAVAifdwkChCC78sAiAmoiU2EeATHEhzW7NcdLtl5Fa9A28gPAn+CKpopMvI/QraGTUpk0AF0brlezmEBVHAqWBlJTCa5YTekw5rWZQkZTosBleVeMc6bZxkyj4JeOD+nihIqnUvjW1nSuBOj9f65n+1loHkOCy4zA0wSYeLEiMwZLgfEW5zxSiIngVCFbd/xfSO2HTABlmxIfjjJ0/CTe3At3x1WD09G8WxgLbQNtpFPjpCp+gCXaI6ougRPaNX9OY8OS/Ou/MxbJ1yRjOb6I+czx/MZJ6T</latexit>
<latexit sha1_base64="l3iwXAx+BJVV3L+FIIt6mI0zRgc=">AAACAnicbZDLSsNAFIYn9VbrLepK3AwWQRBKIoJuhKIuXFawF2hDmEym7dDJJMyltITgxldx40IRtz6FO9/GaZuFtv4w8PGfczhz/iBhVCrH+bYKS8srq2vF9dLG5tb2jr2715CxFpjUccxi0QqQJIxyUldUMdJKBEFRwEgzGNxM6s0hEZLG/EGNE+JFqMdpl2KkjOXbB9pPVQav4C00NBpl8BQO/RBq6Ntlp+JMBRfBzaEMctV8+6sTxlhHhCvMkJRt10mUlyKhKGYkK3W0JAnCA9QjbYMcRUR66fSEDB4bJ4TdWJjHFZy6vydSFEk5jgLTGSHVl/O1iflfra1V99JLKU+0IhzPFnU1gyqGkzxgSAXBio0NICyo+SvEfSQQVia1kgnBnT95ERpnFdfw/Xm5ep3HUQSH4AicABdcgCq4AzVQBxg8gmfwCt6sJ+vFerc+Zq0FK5/ZB39kff4A+62V2w==</latexit>
<latexit sha1_base64="q8SSRp0DvPa9XRci5Cdg/mAJUPE=">AAAB+nicbZDLSsNAFIZP6q3WW6pLN4NFcFUSEXQjFHXhsoK9QBvCZDpth04mYWaiLTGP4saFIm59Ene+jdM2C239YeDjP+dwzvxBzJnSjvNtFVZW19Y3ipulre2d3T27vN9UUSIJbZCIR7IdYEU5E7Shmea0HUuKw4DTVjC6ntZbD1QqFol7PYmpF+KBYH1GsDaWb5cTP9UZukQ3yNB4nCHfrjhVZya0DG4OFchV9+2vbi8iSUiFJhwr1XGdWHsplpoRTrNSN1E0xmSEB7RjUOCQKi+dnZ6hY+P0UD+S5gmNZu7viRSHSk3CwHSGWA/VYm1q/lfrJLp/4aVMxImmgswX9ROOdISmOaAek5RoPjGAiWTmVkSGWGKiTVolE4K7+OVlaJ5WXcN3Z5XaVR5HEQ7hCE7AhXOowS3UoQEEHuEZXuHNerJerHfrY95asPKZA/gj6/MHPD2TUg==</latexit>
Random walk
Advected random walk
Persistant random walk
x
Density
estimation
Model
discovery
Single ParticleTracking
Density estimation:Temporal normalizing flows
t
z
t
x
Latent spaceReal space
x
t
Positional data
Temporal Normalizing Flows (tNFs) extend NFs with a temporal
component to estimate a time-dependent density evolution
Code available on Github:
https://github.com/PhIMaL
Paper available on arXiv:
1912.09092
Density estimation:Temporal normalizing flows
t
z
t
x
Latent spaceReal space
x
t
Positional data
Temporal Normalizing Flows (tNFs) extend NFs with a temporal
component to estimate a time-dependent density evolution
tNFs allow discovery of PDEs with conservation
constraints, e.g. Fokker-Planck, …
Code available on Github:
https://github.com/PhIMaL
Paper available on arXiv:
1912.09092
Code available on Github:
https://github.com/PhIMaL
DeepMoD
Model discovery
neural networks
DeepMoD
Model discovery
neural networks
Paper available on arXiv:
1904.08406
T. Chrysostomou
(Intern)
C. Le Scao
( Intern)
A, Brandon-Bravo
( Engineer)
G.-J. Both
( PhD)
@GJ_Both
@RemyKusters
Postdoc position available !

More Related Content

Similar to DeepMoD: Deep Learning Model discovery

The current state of prediction in neuroimaging
The current state of prediction in neuroimagingThe current state of prediction in neuroimaging
The current state of prediction in neuroimagingSaigeRutherford
 
IRJET- Improved Model for Big Data Analytics using Dynamic Multi-Swarm Op...
IRJET-  	  Improved Model for Big Data Analytics using Dynamic Multi-Swarm Op...IRJET-  	  Improved Model for Big Data Analytics using Dynamic Multi-Swarm Op...
IRJET- Improved Model for Big Data Analytics using Dynamic Multi-Swarm Op...IRJET Journal
 
AI - history and recent breakthroughs
AI - history and recent breakthroughs AI - history and recent breakthroughs
AI - history and recent breakthroughs Armando Vieira
 
Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!
Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!
Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!Sri Ambati
 
Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Aalto University
 
A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...
A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...
A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...Editor IJCATR
 
AI Market Size, Strategy
AI Market Size, StrategyAI Market Size, Strategy
AI Market Size, StrategyPurpose+
 
Train, explain, acclaim. Build a good model in three steps
Train, explain, acclaim.  Build a good model in three stepsTrain, explain, acclaim.  Build a good model in three steps
Train, explain, acclaim. Build a good model in three stepsPrzemek Biecek
 
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062Wael Alawsey
 
Yuri van Geest, Exponential Organizations - DMX Dublin 2016
Yuri van Geest,  Exponential Organizations - DMX Dublin 2016Yuri van Geest,  Exponential Organizations - DMX Dublin 2016
Yuri van Geest, Exponential Organizations - DMX Dublin 2016DMX Dublin
 
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...Amit Sheth
 
M.Sc. Thesis Topics and Proposals @ Polimi Data Science Lab - 2024 - prof. Br...
M.Sc. Thesis Topics and Proposals @ Polimi Data Science Lab - 2024 - prof. Br...M.Sc. Thesis Topics and Proposals @ Polimi Data Science Lab - 2024 - prof. Br...
M.Sc. Thesis Topics and Proposals @ Polimi Data Science Lab - 2024 - prof. Br...Marco Brambilla
 
The Science of Data Science
The Science of Data Science The Science of Data Science
The Science of Data Science James Hendler
 
Discover How Scientific Data is Used for the Public Good with Natural Languag...
Discover How Scientific Data is Used for the Public Good with Natural Languag...Discover How Scientific Data is Used for the Public Good with Natural Languag...
Discover How Scientific Data is Used for the Public Good with Natural Languag...BaoTramDuong2
 
CompetitiveIntelligenceNetworks
CompetitiveIntelligenceNetworksCompetitiveIntelligenceNetworks
CompetitiveIntelligenceNetworksEva Ortoll
 
Machine learning ppt unit one syllabuspptx
Machine learning ppt unit one syllabuspptxMachine learning ppt unit one syllabuspptx
Machine learning ppt unit one syllabuspptxVenkateswaraBabuRavi
 
Cyber bullying detection and analysis.ppt.pdf
Cyber bullying detection and analysis.ppt.pdfCyber bullying detection and analysis.ppt.pdf
Cyber bullying detection and analysis.ppt.pdfHunais Abdul Nafi
 

Similar to DeepMoD: Deep Learning Model discovery (20)

The current state of prediction in neuroimaging
The current state of prediction in neuroimagingThe current state of prediction in neuroimaging
The current state of prediction in neuroimaging
 
IRJET- Improved Model for Big Data Analytics using Dynamic Multi-Swarm Op...
IRJET-  	  Improved Model for Big Data Analytics using Dynamic Multi-Swarm Op...IRJET-  	  Improved Model for Big Data Analytics using Dynamic Multi-Swarm Op...
IRJET- Improved Model for Big Data Analytics using Dynamic Multi-Swarm Op...
 
AI - history and recent breakthroughs
AI - history and recent breakthroughs AI - history and recent breakthroughs
AI - history and recent breakthroughs
 
Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!
Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!
Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!
 
Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"
 
A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...
A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...
A Comparative Study of Various Data Mining Techniques: Statistics, Decision T...
 
AI Market Size, Strategy
AI Market Size, StrategyAI Market Size, Strategy
AI Market Size, Strategy
 
Train, explain, acclaim. Build a good model in three steps
Train, explain, acclaim.  Build a good model in three stepsTrain, explain, acclaim.  Build a good model in three steps
Train, explain, acclaim. Build a good model in three steps
 
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062
 
Yuri van Geest, Exponential Organizations - DMX Dublin 2016
Yuri van Geest,  Exponential Organizations - DMX Dublin 2016Yuri van Geest,  Exponential Organizations - DMX Dublin 2016
Yuri van Geest, Exponential Organizations - DMX Dublin 2016
 
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...
 
M.Sc. Thesis Topics and Proposals @ Polimi Data Science Lab - 2024 - prof. Br...
M.Sc. Thesis Topics and Proposals @ Polimi Data Science Lab - 2024 - prof. Br...M.Sc. Thesis Topics and Proposals @ Polimi Data Science Lab - 2024 - prof. Br...
M.Sc. Thesis Topics and Proposals @ Polimi Data Science Lab - 2024 - prof. Br...
 
Ai in finance
Ai in financeAi in finance
Ai in finance
 
ODSC APAC 2022 - Explainable AI
ODSC APAC 2022 - Explainable AIODSC APAC 2022 - Explainable AI
ODSC APAC 2022 - Explainable AI
 
The Science of Data Science
The Science of Data Science The Science of Data Science
The Science of Data Science
 
Machine learning
Machine learningMachine learning
Machine learning
 
Discover How Scientific Data is Used for the Public Good with Natural Languag...
Discover How Scientific Data is Used for the Public Good with Natural Languag...Discover How Scientific Data is Used for the Public Good with Natural Languag...
Discover How Scientific Data is Used for the Public Good with Natural Languag...
 
CompetitiveIntelligenceNetworks
CompetitiveIntelligenceNetworksCompetitiveIntelligenceNetworks
CompetitiveIntelligenceNetworks
 
Machine learning ppt unit one syllabuspptx
Machine learning ppt unit one syllabuspptxMachine learning ppt unit one syllabuspptx
Machine learning ppt unit one syllabuspptx
 
Cyber bullying detection and analysis.ppt.pdf
Cyber bullying detection and analysis.ppt.pdfCyber bullying detection and analysis.ppt.pdf
Cyber bullying detection and analysis.ppt.pdf
 

Recently uploaded

GBSN - Microbiology (Unit 5) Concept of isolation
GBSN - Microbiology (Unit 5) Concept of isolationGBSN - Microbiology (Unit 5) Concept of isolation
GBSN - Microbiology (Unit 5) Concept of isolationAreesha Ahmad
 
Film Coated Tablet and Film Coating raw materials.pdf
Film Coated Tablet and Film Coating raw materials.pdfFilm Coated Tablet and Film Coating raw materials.pdf
Film Coated Tablet and Film Coating raw materials.pdfPharmatech-rx
 
Taphonomy and Quality of the Fossil Record
Taphonomy and Quality of the  Fossil RecordTaphonomy and Quality of the  Fossil Record
Taphonomy and Quality of the Fossil RecordSangram Sahoo
 
PHOTOSYNTHETIC BACTERIA (OXYGENIC AND ANOXYGENIC)
PHOTOSYNTHETIC BACTERIA  (OXYGENIC AND ANOXYGENIC)PHOTOSYNTHETIC BACTERIA  (OXYGENIC AND ANOXYGENIC)
PHOTOSYNTHETIC BACTERIA (OXYGENIC AND ANOXYGENIC)kushbuR
 
GBSN - Biochemistry (Unit 3) Metabolism
GBSN - Biochemistry (Unit 3) MetabolismGBSN - Biochemistry (Unit 3) Metabolism
GBSN - Biochemistry (Unit 3) MetabolismAreesha Ahmad
 
Warming the earth and the atmosphere.pptx
Warming the earth and the atmosphere.pptxWarming the earth and the atmosphere.pptx
Warming the earth and the atmosphere.pptxGlendelCaroz
 
MSC IV_Forensic medicine - Mechanical injuries.pdf
MSC IV_Forensic medicine - Mechanical injuries.pdfMSC IV_Forensic medicine - Mechanical injuries.pdf
MSC IV_Forensic medicine - Mechanical injuries.pdfSuchita Rawat
 
Adaptive Restore algorithm & importance Monte Carlo
Adaptive Restore algorithm & importance Monte CarloAdaptive Restore algorithm & importance Monte Carlo
Adaptive Restore algorithm & importance Monte CarloChristian Robert
 
PARENTAL CARE IN FISHES.pptx for 5th sem
PARENTAL CARE IN FISHES.pptx for 5th semPARENTAL CARE IN FISHES.pptx for 5th sem
PARENTAL CARE IN FISHES.pptx for 5th semborkhotudu123
 
Heads-Up Multitasker: CHI 2024 Presentation.pdf
Heads-Up Multitasker: CHI 2024 Presentation.pdfHeads-Up Multitasker: CHI 2024 Presentation.pdf
Heads-Up Multitasker: CHI 2024 Presentation.pdfbyp19971001
 
Terpineol and it's characterization pptx
Terpineol and it's characterization pptxTerpineol and it's characterization pptx
Terpineol and it's characterization pptxMuhammadRazzaq31
 
SaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptx
SaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptxSaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptx
SaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptxPat (JS) Heslop-Harrison
 
FORENSIC CHEMISTRY ARSON INVESTIGATION.pdf
FORENSIC CHEMISTRY ARSON INVESTIGATION.pdfFORENSIC CHEMISTRY ARSON INVESTIGATION.pdf
FORENSIC CHEMISTRY ARSON INVESTIGATION.pdfSuchita Rawat
 
Factor Causing low production and physiology of mamary Gland
Factor Causing low production and physiology of mamary GlandFactor Causing low production and physiology of mamary Gland
Factor Causing low production and physiology of mamary GlandRcvets
 
Molecular and Cellular Mechanism of Action of Hormones such as Growth Hormone...
Molecular and Cellular Mechanism of Action of Hormones such as Growth Hormone...Molecular and Cellular Mechanism of Action of Hormones such as Growth Hormone...
Molecular and Cellular Mechanism of Action of Hormones such as Growth Hormone...Ansari Aashif Raza Mohd Imtiyaz
 
GBSN - Microbiology (Unit 4) Concept of Asepsis
GBSN - Microbiology (Unit 4) Concept of AsepsisGBSN - Microbiology (Unit 4) Concept of Asepsis
GBSN - Microbiology (Unit 4) Concept of AsepsisAreesha Ahmad
 
Costs to heap leach gold ore tailings in Karamoja region of Uganda
Costs to heap leach gold ore tailings in Karamoja region of UgandaCosts to heap leach gold ore tailings in Karamoja region of Uganda
Costs to heap leach gold ore tailings in Karamoja region of UgandaTimothyOkuna
 
GBSN - Biochemistry (Unit 8) Enzymology
GBSN - Biochemistry (Unit 8) EnzymologyGBSN - Biochemistry (Unit 8) Enzymology
GBSN - Biochemistry (Unit 8) EnzymologyAreesha Ahmad
 
Vital Signs of Animals Presentation By Aftab Ahmed Rahimoon
Vital Signs of Animals Presentation By Aftab Ahmed RahimoonVital Signs of Animals Presentation By Aftab Ahmed Rahimoon
Vital Signs of Animals Presentation By Aftab Ahmed RahimoonAftabAhmedRahimoon
 

Recently uploaded (20)

GBSN - Microbiology (Unit 5) Concept of isolation
GBSN - Microbiology (Unit 5) Concept of isolationGBSN - Microbiology (Unit 5) Concept of isolation
GBSN - Microbiology (Unit 5) Concept of isolation
 
Film Coated Tablet and Film Coating raw materials.pdf
Film Coated Tablet and Film Coating raw materials.pdfFilm Coated Tablet and Film Coating raw materials.pdf
Film Coated Tablet and Film Coating raw materials.pdf
 
Taphonomy and Quality of the Fossil Record
Taphonomy and Quality of the  Fossil RecordTaphonomy and Quality of the  Fossil Record
Taphonomy and Quality of the Fossil Record
 
PHOTOSYNTHETIC BACTERIA (OXYGENIC AND ANOXYGENIC)
PHOTOSYNTHETIC BACTERIA  (OXYGENIC AND ANOXYGENIC)PHOTOSYNTHETIC BACTERIA  (OXYGENIC AND ANOXYGENIC)
PHOTOSYNTHETIC BACTERIA (OXYGENIC AND ANOXYGENIC)
 
GBSN - Biochemistry (Unit 3) Metabolism
GBSN - Biochemistry (Unit 3) MetabolismGBSN - Biochemistry (Unit 3) Metabolism
GBSN - Biochemistry (Unit 3) Metabolism
 
Warming the earth and the atmosphere.pptx
Warming the earth and the atmosphere.pptxWarming the earth and the atmosphere.pptx
Warming the earth and the atmosphere.pptx
 
MSC IV_Forensic medicine - Mechanical injuries.pdf
MSC IV_Forensic medicine - Mechanical injuries.pdfMSC IV_Forensic medicine - Mechanical injuries.pdf
MSC IV_Forensic medicine - Mechanical injuries.pdf
 
Adaptive Restore algorithm & importance Monte Carlo
Adaptive Restore algorithm & importance Monte CarloAdaptive Restore algorithm & importance Monte Carlo
Adaptive Restore algorithm & importance Monte Carlo
 
PARENTAL CARE IN FISHES.pptx for 5th sem
PARENTAL CARE IN FISHES.pptx for 5th semPARENTAL CARE IN FISHES.pptx for 5th sem
PARENTAL CARE IN FISHES.pptx for 5th sem
 
Heads-Up Multitasker: CHI 2024 Presentation.pdf
Heads-Up Multitasker: CHI 2024 Presentation.pdfHeads-Up Multitasker: CHI 2024 Presentation.pdf
Heads-Up Multitasker: CHI 2024 Presentation.pdf
 
Terpineol and it's characterization pptx
Terpineol and it's characterization pptxTerpineol and it's characterization pptx
Terpineol and it's characterization pptx
 
SaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptx
SaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptxSaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptx
SaffronCrocusGenomicsThessalonikiOnlineMay2024TalkOnline.pptx
 
FORENSIC CHEMISTRY ARSON INVESTIGATION.pdf
FORENSIC CHEMISTRY ARSON INVESTIGATION.pdfFORENSIC CHEMISTRY ARSON INVESTIGATION.pdf
FORENSIC CHEMISTRY ARSON INVESTIGATION.pdf
 
Factor Causing low production and physiology of mamary Gland
Factor Causing low production and physiology of mamary GlandFactor Causing low production and physiology of mamary Gland
Factor Causing low production and physiology of mamary Gland
 
Molecular and Cellular Mechanism of Action of Hormones such as Growth Hormone...
Molecular and Cellular Mechanism of Action of Hormones such as Growth Hormone...Molecular and Cellular Mechanism of Action of Hormones such as Growth Hormone...
Molecular and Cellular Mechanism of Action of Hormones such as Growth Hormone...
 
GBSN - Microbiology (Unit 4) Concept of Asepsis
GBSN - Microbiology (Unit 4) Concept of AsepsisGBSN - Microbiology (Unit 4) Concept of Asepsis
GBSN - Microbiology (Unit 4) Concept of Asepsis
 
Costs to heap leach gold ore tailings in Karamoja region of Uganda
Costs to heap leach gold ore tailings in Karamoja region of UgandaCosts to heap leach gold ore tailings in Karamoja region of Uganda
Costs to heap leach gold ore tailings in Karamoja region of Uganda
 
GBSN - Biochemistry (Unit 8) Enzymology
GBSN - Biochemistry (Unit 8) EnzymologyGBSN - Biochemistry (Unit 8) Enzymology
GBSN - Biochemistry (Unit 8) Enzymology
 
Vital Signs of Animals Presentation By Aftab Ahmed Rahimoon
Vital Signs of Animals Presentation By Aftab Ahmed RahimoonVital Signs of Animals Presentation By Aftab Ahmed Rahimoon
Vital Signs of Animals Presentation By Aftab Ahmed Rahimoon
 
HIV AND INFULENZA VIRUS PPT HIV PPT INFULENZA VIRUS PPT
HIV AND INFULENZA VIRUS PPT HIV PPT  INFULENZA VIRUS PPTHIV AND INFULENZA VIRUS PPT HIV PPT  INFULENZA VIRUS PPT
HIV AND INFULENZA VIRUS PPT HIV PPT INFULENZA VIRUS PPT
 

DeepMoD: Deep Learning Model discovery

  • 1. Gert-Jan Both, Remy Kusters CRI Research, Université Paris Descartes, Paris, France DeepMoD Model discovery neural networks DeepMoD Model discovery neural networks @GJ_Both @RemyKusters
  • 2. AI, ML and statistics Symbolic MLStatistical ML vs. @sandserif Main criticisms … ML provides uninterpretable solutions
  • 3. Goal: Develop tools to discover interpretable models for quantitative science Combine the statistical power of neural networks with the symbolic power of regression Approach:
  • 4. Goal: Combine the statistical power of neural networks with the symbolic power of regression Develop tools to discover interpretable models for quantitative science <latexit sha1_base64="OuHvrtTUD9r4pBI06iD0mJxKAcs=">AAACGnicbVDLSgMxFM34rPVVdekmWIQWyjAjgm6EoiAuFWwrtMOQSTNtMPMguZGWYb7Djb/ixoUi7sSNf2NaZ6HWA8k9nHMvyT1BKrgCx/m05uYXFpeWSyvl1bX1jc3K1nZbJVpS1qKJSORNQBQTPGYt4CDYTSoZiQLBOsHt2cTv3DGpeBJfwzhlXkQGMQ85JWAkv+JqH2qjBtTxCc56lIjsPM9xTTew9kfmKoqfjUZ5A9u2XfcrVcd2psCzxC1IFRW49CvvvX5CdcRioIIo1XWdFLyMSOBUsLzc04qlhN6SAesaGpOIKS+brpbjfaP0cZhIc2LAU/XnREYipcZRYDojAkP115uI/3ldDeGxl/E41cBi+v1QqAWGBE9ywn0uGQUxNoRQyc1fMR0SSSiYNMsmBPfvyrOkfWC7hl8dVpunRRwltIv2UA256Ag10QW6RC1E0T16RM/oxXqwnqxX6+27dc4qZnbQL1gfX5Tang4=</latexit> Differential equation Space Time Noisy Data Underlying distribution P(x, t)<latexit sha1_base64="3XtpKznpBSb+nMyPAQPZfV01TDk=">AAAB7XicbZBNSwMxEIZn61etX1WPXhaLUEHKrgh6LHrxWMF+QLuUbJq2sdlkSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZecNYcIOe9+3kVlbX1jfym4Wt7Z3dveL+QcOoRFNWp0oo3QqJYYJLVkeOgrVizUgUCtYMRzfTevORacOVvMdxzIKIDCTvc0rQWo1a+ekMT7vFklfxZnKXwc+gBJlq3eJXp6doEjGJVBBj2r4XY5ASjZwKNil0EsNiQkdkwNoWJYmYCdLZthP3xDo9t6+0fRLdmft7IiWRMeMotJ0RwaFZrE3N/2rtBPtXQcplnCCTdP5RPxEuKnd6utvjmlEUYwuEam53demQaELRBlSwIfiLJy9D47ziW767KFWvszjycATHUAYfLqEKt1CDOlB4gGd4hTdHOS/Ou/Mxb8052cwh/JHz+QOJ2I5v</latexit><latexit sha1_base64="3XtpKznpBSb+nMyPAQPZfV01TDk=">AAAB7XicbZBNSwMxEIZn61etX1WPXhaLUEHKrgh6LHrxWMF+QLuUbJq2sdlkSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZecNYcIOe9+3kVlbX1jfym4Wt7Z3dveL+QcOoRFNWp0oo3QqJYYJLVkeOgrVizUgUCtYMRzfTevORacOVvMdxzIKIDCTvc0rQWo1a+ekMT7vFklfxZnKXwc+gBJlq3eJXp6doEjGJVBBj2r4XY5ASjZwKNil0EsNiQkdkwNoWJYmYCdLZthP3xDo9t6+0fRLdmft7IiWRMeMotJ0RwaFZrE3N/2rtBPtXQcplnCCTdP5RPxEuKnd6utvjmlEUYwuEam53demQaELRBlSwIfiLJy9D47ziW767KFWvszjycATHUAYfLqEKt1CDOlB4gGd4hTdHOS/Ou/Mxb8052cwh/JHz+QOJ2I5v</latexit><latexit sha1_base64="3XtpKznpBSb+nMyPAQPZfV01TDk=">AAAB7XicbZBNSwMxEIZn61etX1WPXhaLUEHKrgh6LHrxWMF+QLuUbJq2sdlkSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZecNYcIOe9+3kVlbX1jfym4Wt7Z3dveL+QcOoRFNWp0oo3QqJYYJLVkeOgrVizUgUCtYMRzfTevORacOVvMdxzIKIDCTvc0rQWo1a+ekMT7vFklfxZnKXwc+gBJlq3eJXp6doEjGJVBBj2r4XY5ASjZwKNil0EsNiQkdkwNoWJYmYCdLZthP3xDo9t6+0fRLdmft7IiWRMeMotJ0RwaFZrE3N/2rtBPtXQcplnCCTdP5RPxEuKnd6utvjmlEUYwuEam53demQaELRBlSwIfiLJy9D47ziW767KFWvszjycATHUAYfLqEKt1CDOlB4gGd4hTdHOS/Ou/Mxb8052cwh/JHz+QOJ2I5v</latexit><latexit sha1_base64="3XtpKznpBSb+nMyPAQPZfV01TDk=">AAAB7XicbZBNSwMxEIZn61etX1WPXhaLUEHKrgh6LHrxWMF+QLuUbJq2sdlkSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZecNYcIOe9+3kVlbX1jfym4Wt7Z3dveL+QcOoRFNWp0oo3QqJYYJLVkeOgrVizUgUCtYMRzfTevORacOVvMdxzIKIDCTvc0rQWo1a+ekMT7vFklfxZnKXwc+gBJlq3eJXp6doEjGJVBBj2r4XY5ASjZwKNil0EsNiQkdkwNoWJYmYCdLZthP3xDo9t6+0fRLdmft7IiWRMeMotJ0RwaFZrE3N/2rtBPtXQcplnCCTdP5RPxEuKnd6utvjmlEUYwuEam53demQaELRBlSwIfiLJy9D47ziW767KFWvszjycATHUAYfLqEKt1CDOlB4gGd4hTdHOS/Ou/Mxb8052cwh/JHz+QOJ2I5v</latexit> Approach:
  • 5. <latexit sha1_base64="OuHvrtTUD9r4pBI06iD0mJxKAcs=">AAACGnicbVDLSgMxFM34rPVVdekmWIQWyjAjgm6EoiAuFWwrtMOQSTNtMPMguZGWYb7Djb/ixoUi7sSNf2NaZ6HWA8k9nHMvyT1BKrgCx/m05uYXFpeWSyvl1bX1jc3K1nZbJVpS1qKJSORNQBQTPGYt4CDYTSoZiQLBOsHt2cTv3DGpeBJfwzhlXkQGMQ85JWAkv+JqH2qjBtTxCc56lIjsPM9xTTew9kfmKoqfjUZ5A9u2XfcrVcd2psCzxC1IFRW49CvvvX5CdcRioIIo1XWdFLyMSOBUsLzc04qlhN6SAesaGpOIKS+brpbjfaP0cZhIc2LAU/XnREYipcZRYDojAkP115uI/3ldDeGxl/E41cBi+v1QqAWGBE9ywn0uGQUxNoRQyc1fMR0SSSiYNMsmBPfvyrOkfWC7hl8dVpunRRwltIv2UA256Ag10QW6RC1E0T16RM/oxXqwnqxX6+27dc4qZnbQL1gfX5Tang4=</latexit> Analytic solution Space Time <latexit sha1_base64="f64HEF6znFEZoH5kmjcDbs/59Jo=">AAAB7XicbZBNSwMxEIZn61etX1WPXoJFqCBlVwQ9Fr14rGA/oF1KNk3b2OxmSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZeYNYCoOu++3kVlbX1jfym4Wt7Z3dveL+QcOoRDNeZ0oq3Qqo4VJEvI4CJW/FmtMwkLwZjG6m9eYj10ao6B7HMfdDOohEXzCK1mok5aczPO0WS27FnYksg5dBCTLVusWvTk+xJOQRMkmNaXtujH5KNQom+aTQSQyPKRvRAW9bjGjIjZ/Otp2QE+v0SF9p+yIkM/f3REpDY8ZhYDtDikOzWJua/9XaCfav/FREcYI8YvOP+okkqMj0dNITmjOUYwuUaWF3JWxINWVoAyrYELzFk5ehcV7xLN9dlKrXWRx5OIJjKIMHl1CFW6hBHRg8wDO8wpujnBfn3fmYt+acbOYQ/sj5/AHCpY6U</latexit> Noisy Data DeepMoD Model discovery neural networks DeepMoD Model discovery neural networks Code available: github.com/PhIMaL
  • 6. <latexit sha1_base64="1Gq7mrW1/ZKuQjtOjsnM1gDsGDU=">AAACBHicbZDJSgNBEIZ74hbjFvWYS2MQIkiYEUEvQtCLxwjZIBNCT6eSNOlZ6K4JCUMOXnwVLx4U8epDePNt7CwHTfyh4eOvKqrr9yIpNNr2t5VaW9/Y3EpvZ3Z29/YPsodHNR3GikOVhzJUDY9pkCKAKgqU0IgUMN+TUPcGd9N6fQhKizCo4DiCls96gegKztBY7WwubmNhdI5n9Ia6lT4go+4QeOKOxKSdzdtFeya6Cs4C8mShcjv75XZCHvsQIJdM66ZjR9hKmELBJUwybqwhYnzAetA0GDAfdCuZHTGhp8bp0G6ozAuQztzfEwnztR77nun0Gfb1cm1q/ldrxti9biUiiGKEgM8XdWNJMaTTRGhHKOAoxwYYV8L8lfI+U4yjyS1jQnCWT16F2kXRMfxwmS/dLuJIkxw5IQXikCtSIvekTKqEk0fyTF7Jm/VkvVjv1se8NWUtZo7JH1mfPysqlyg=</latexit> Dataset Neural Network Thresh ˆuu Auto. Diff. ∂t ˆu = Θξ Library Include in cost function NN converged ξ∗ = ξ · || || × × × × × × × × ξ∗ Update cost f Space Time Burgers equation ∂tu = uux − 0.1uxx L = MSE(u, ˆu) + MSE(∂t ˆu, Θξ) + L1(ξ) = ... u ux uux u2 uxx Space Time <latexit sha1_base64="f64HEF6znFEZoH5kmjcDbs/59Jo=">AAAB7XicbZBNSwMxEIZn61etX1WPXoJFqCBlVwQ9Fr14rGA/oF1KNk3b2OxmSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZeYNYCoOu++3kVlbX1jfym4Wt7Z3dveL+QcOoRDNeZ0oq3Qqo4VJEvI4CJW/FmtMwkLwZjG6m9eYj10ao6B7HMfdDOohEXzCK1mok5aczPO0WS27FnYksg5dBCTLVusWvTk+xJOQRMkmNaXtujH5KNQom+aTQSQyPKRvRAW9bjGjIjZ/Otp2QE+v0SF9p+yIkM/f3REpDY8ZhYDtDikOzWJua/9XaCfav/FREcYI8YvOP+okkqMj0dNITmjOUYwuUaWF3JWxINWVoAyrYELzFk5ehcV7xLN9dlKrXWRx5OIJjKIMHl1CFW6hBHRg8wDO8wpujnBfn3fmYt+acbOYQ/sj5/AHCpY6U</latexit> SolutionNoisy Data DeepMoD Model discovery neural networks DeepMoD Model discovery neural networks Code available: github.com/PhIMaL
  • 7. <latexit sha1_base64="1Gq7mrW1/ZKuQjtOjsnM1gDsGDU=">AAACBHicbZDJSgNBEIZ74hbjFvWYS2MQIkiYEUEvQtCLxwjZIBNCT6eSNOlZ6K4JCUMOXnwVLx4U8epDePNt7CwHTfyh4eOvKqrr9yIpNNr2t5VaW9/Y3EpvZ3Z29/YPsodHNR3GikOVhzJUDY9pkCKAKgqU0IgUMN+TUPcGd9N6fQhKizCo4DiCls96gegKztBY7WwubmNhdI5n9Ia6lT4go+4QeOKOxKSdzdtFeya6Cs4C8mShcjv75XZCHvsQIJdM66ZjR9hKmELBJUwybqwhYnzAetA0GDAfdCuZHTGhp8bp0G6ozAuQztzfEwnztR77nun0Gfb1cm1q/ldrxti9biUiiGKEgM8XdWNJMaTTRGhHKOAoxwYYV8L8lfI+U4yjyS1jQnCWT16F2kXRMfxwmS/dLuJIkxw5IQXikCtSIvekTKqEk0fyTF7Jm/VkvVjv1se8NWUtZo7JH1mfPysqlyg=</latexit> aset Neural Network Threshold ˆuu Auto. Diff. ∂t ˆu = Θξ Library Include in cost function NN converged ξ∗ = ξ · ||Θ|| ||ut|| So Sparsity converge × × × × × × × × × × × keep set to zero keep ξ∗ Update cost function ace equation x − 0.1uxx L = MSE(u, ˆu) + MSE(∂t ˆu, Θξ) + L1(ξ) = ... u ux uux u2 uxx Fou ∂tu = Space Time <latexit sha1_base64="f64HEF6znFEZoH5kmjcDbs/59Jo=">AAAB7XicbZBNSwMxEIZn61etX1WPXoJFqCBlVwQ9Fr14rGA/oF1KNk3b2OxmSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZeYNYCoOu++3kVlbX1jfym4Wt7Z3dveL+QcOoRDNeZ0oq3Qqo4VJEvI4CJW/FmtMwkLwZjG6m9eYj10ao6B7HMfdDOohEXzCK1mok5aczPO0WS27FnYksg5dBCTLVusWvTk+xJOQRMkmNaXtujH5KNQom+aTQSQyPKRvRAW9bjGjIjZ/Otp2QE+v0SF9p+yIkM/f3REpDY8ZhYDtDikOzWJua/9XaCfav/FREcYI8YvOP+okkqMj0dNITmjOUYwuUaWF3JWxINWVoAyrYELzFk5ehcV7xLN9dlKrXWRx5OIJjKIMHl1CFW6hBHRg8wDO8wpujnBfn3fmYt+acbOYQ/sj5/AHCpY6U</latexit> ⇠⇤ =<latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit> Neural Network Threshold ˆuu Auto. Diff. ∂t ˆu = Θξ Library Include in cost function NN converged ξ∗ = ξ · ||Θ|| ||ut|| Solution Sparsity converged × × × × × × × × × × × keep set to zero keep ξ∗ Update cost function L = MSE(u, ˆu) + MSE(∂t ˆu, Θξ) + L1(ξ) = ... u ux uux u2 uxx Found solution ∂tu = uux − 0.1uxx 0 0 1 0 ... -0.1 0 uux uxx u ux u2 uxx <latexit sha1_base64="1Gq7mrW1/ZKuQjtOjsnM1gDsGDU=">AAACBHicbZDJSgNBEIZ74hbjFvWYS2MQIkiYEUEvQtCLxwjZIBNCT6eSNOlZ6K4JCUMOXnwVLx4U8epDePNt7CwHTfyh4eOvKqrr9yIpNNr2t5VaW9/Y3EpvZ3Z29/YPsodHNR3GikOVhzJUDY9pkCKAKgqU0IgUMN+TUPcGd9N6fQhKizCo4DiCls96gegKztBY7WwubmNhdI5n9Ia6lT4go+4QeOKOxKSdzdtFeya6Cs4C8mShcjv75XZCHvsQIJdM66ZjR9hKmELBJUwybqwhYnzAetA0GDAfdCuZHTGhp8bp0G6ozAuQztzfEwnztR77nun0Gfb1cm1q/ldrxti9biUiiGKEgM8XdWNJMaTTRGhHKOAoxwYYV8L8lfI+U4yjyS1jQnCWT16F2kXRMfxwmS/dLuJIkxw5IQXikCtSIvekTKqEk0fyTF7Jm/VkvVjv1se8NWUtZo7JH1mfPysqlyg=</latexit> SolutionNoisy Data Regression Code available: github.com/PhIMaL DeepMoD Model discovery neural networks DeepMoD Model discovery neural networks
  • 8. 5% white noise … Why does it fail with noise? Θ = ! ! ! u ux uxu ! ! ! ! ! ! uxx uxxu uxxux ! ! ! " ⎛ ⎝ ⎜ ⎜ ⎜ ⎞ ⎠ ⎟ ⎟ ⎟ Numerically calculating library impossible … uxx(x)<latexit sha1_base64="QNA18qBfAJrCCEwMxK/B/I1M350=">AAAB8HicbZDLSgMxFIbP1Futt6pLN8Ei1E2ZEUGXRTcuK9iLtEPJpJk2NMkMSUZahj6FGxeKuPVx3Pk2ZtpZaOsPgY//nEPO+YOYM21c99sprK1vbG4Vt0s7u3v7B+XDo5aOEkVok0Q8Up0Aa8qZpE3DDKedWFEsAk7bwfg2q7efqNIskg9mGlNf4KFkISPYWOsx6aeTyaw6Oe+XK27NnQutgpdDBXI1+uWv3iAiiaDSEI617npubPwUK8MIp7NSL9E0xmSMh7RrUWJBtZ/OF56hM+sMUBgp+6RBc/f3RIqF1lMR2E6BzUgv1zLzv1o3MeG1nzIZJ4ZKsvgoTDgyEcquRwOmKDF8agETxeyuiIywwsTYjEo2BG/55FVoXdQ8y/eXlfpNHkcRTuAUquDBFdThDhrQBAICnuEV3hzlvDjvzseiteDkM8fwR87nD707kFk=</latexit><latexit sha1_base64="QNA18qBfAJrCCEwMxK/B/I1M350=">AAAB8HicbZDLSgMxFIbP1Futt6pLN8Ei1E2ZEUGXRTcuK9iLtEPJpJk2NMkMSUZahj6FGxeKuPVx3Pk2ZtpZaOsPgY//nEPO+YOYM21c99sprK1vbG4Vt0s7u3v7B+XDo5aOEkVok0Q8Up0Aa8qZpE3DDKedWFEsAk7bwfg2q7efqNIskg9mGlNf4KFkISPYWOsx6aeTyaw6Oe+XK27NnQutgpdDBXI1+uWv3iAiiaDSEI617npubPwUK8MIp7NSL9E0xmSMh7RrUWJBtZ/OF56hM+sMUBgp+6RBc/f3RIqF1lMR2E6BzUgv1zLzv1o3MeG1nzIZJ4ZKsvgoTDgyEcquRwOmKDF8agETxeyuiIywwsTYjEo2BG/55FVoXdQ8y/eXlfpNHkcRTuAUquDBFdThDhrQBAICnuEV3hzlvDjvzseiteDkM8fwR87nD707kFk=</latexit><latexit sha1_base64="QNA18qBfAJrCCEwMxK/B/I1M350=">AAAB8HicbZDLSgMxFIbP1Futt6pLN8Ei1E2ZEUGXRTcuK9iLtEPJpJk2NMkMSUZahj6FGxeKuPVx3Pk2ZtpZaOsPgY//nEPO+YOYM21c99sprK1vbG4Vt0s7u3v7B+XDo5aOEkVok0Q8Up0Aa8qZpE3DDKedWFEsAk7bwfg2q7efqNIskg9mGlNf4KFkISPYWOsx6aeTyaw6Oe+XK27NnQutgpdDBXI1+uWv3iAiiaDSEI617npubPwUK8MIp7NSL9E0xmSMh7RrUWJBtZ/OF56hM+sMUBgp+6RBc/f3RIqF1lMR2E6BzUgv1zLzv1o3MeG1nzIZJ4ZKsvgoTDgyEcquRwOmKDF8agETxeyuiIywwsTYjEo2BG/55FVoXdQ8y/eXlfpNHkcRTuAUquDBFdThDhrQBAICnuEV3hzlvDjvzseiteDkM8fwR87nD707kFk=</latexit><latexit sha1_base64="QNA18qBfAJrCCEwMxK/B/I1M350=">AAAB8HicbZDLSgMxFIbP1Futt6pLN8Ei1E2ZEUGXRTcuK9iLtEPJpJk2NMkMSUZahj6FGxeKuPVx3Pk2ZtpZaOsPgY//nEPO+YOYM21c99sprK1vbG4Vt0s7u3v7B+XDo5aOEkVok0Q8Up0Aa8qZpE3DDKedWFEsAk7bwfg2q7efqNIskg9mGlNf4KFkISPYWOsx6aeTyaw6Oe+XK27NnQutgpdDBXI1+uWv3iAiiaDSEI617npubPwUK8MIp7NSL9E0xmSMh7RrUWJBtZ/OF56hM+sMUBgp+6RBc/f3RIqF1lMR2E6BzUgv1zLzv1o3MeG1nzIZJ4ZKsvgoTDgyEcquRwOmKDF8agETxeyuiIywwsTYjEo2BG/55FVoXdQ8y/eXlfpNHkcRTuAUquDBFdThDhrQBAICnuEV3hzlvDjvzseiteDkM8fwR87nD707kFk=</latexit> ux(x)<latexit sha1_base64="tjArAx6NhzxT95XjS0tfOz6ea7o=">AAAB73icbZBNSwMxEIZn61etX1WPXoJFqJeyK4Iei148VrAf0C4lm07b0Gx2TbLSsvRPePGgiFf/jjf/jWm7B219IfDwzgyZeYNYcG1c99vJra1vbG7ltws7u3v7B8XDo4aOEsWwziIRqVZANQousW64EdiKFdIwENgMRrezevMJleaRfDCTGP2QDiTvc0aNtVpJNx1Py+PzbrHkVty5yCp4GZQgU61b/Or0IpaEKA0TVOu258bGT6kynAmcFjqJxpiyER1g26KkIWo/ne87JWfW6ZF+pOyThszd3xMpDbWehIHtDKkZ6uXazPyv1k5M/9pPuYwTg5ItPuongpiIzI4nPa6QGTGxQJnidlfChlRRZmxEBRuCt3zyKjQuKp7l+8tS9SaLIw8ncApl8OAKqnAHNagDAwHP8ApvzqPz4rw7H4vWnJPNHMMfOZ8/3rOP1w==</latexit><latexit sha1_base64="tjArAx6NhzxT95XjS0tfOz6ea7o=">AAAB73icbZBNSwMxEIZn61etX1WPXoJFqJeyK4Iei148VrAf0C4lm07b0Gx2TbLSsvRPePGgiFf/jjf/jWm7B219IfDwzgyZeYNYcG1c99vJra1vbG7ltws7u3v7B8XDo4aOEsWwziIRqVZANQousW64EdiKFdIwENgMRrezevMJleaRfDCTGP2QDiTvc0aNtVpJNx1Py+PzbrHkVty5yCp4GZQgU61b/Or0IpaEKA0TVOu258bGT6kynAmcFjqJxpiyER1g26KkIWo/ne87JWfW6ZF+pOyThszd3xMpDbWehIHtDKkZ6uXazPyv1k5M/9pPuYwTg5ItPuongpiIzI4nPa6QGTGxQJnidlfChlRRZmxEBRuCt3zyKjQuKp7l+8tS9SaLIw8ncApl8OAKqnAHNagDAwHP8ApvzqPz4rw7H4vWnJPNHMMfOZ8/3rOP1w==</latexit><latexit sha1_base64="tjArAx6NhzxT95XjS0tfOz6ea7o=">AAAB73icbZBNSwMxEIZn61etX1WPXoJFqJeyK4Iei148VrAf0C4lm07b0Gx2TbLSsvRPePGgiFf/jjf/jWm7B219IfDwzgyZeYNYcG1c99vJra1vbG7ltws7u3v7B8XDo4aOEsWwziIRqVZANQousW64EdiKFdIwENgMRrezevMJleaRfDCTGP2QDiTvc0aNtVpJNx1Py+PzbrHkVty5yCp4GZQgU61b/Or0IpaEKA0TVOu258bGT6kynAmcFjqJxpiyER1g26KkIWo/ne87JWfW6ZF+pOyThszd3xMpDbWehIHtDKkZ6uXazPyv1k5M/9pPuYwTg5ItPuongpiIzI4nPa6QGTGxQJnidlfChlRRZmxEBRuCt3zyKjQuKp7l+8tS9SaLIw8ncApl8OAKqnAHNagDAwHP8ApvzqPz4rw7H4vWnJPNHMMfOZ8/3rOP1w==</latexit><latexit sha1_base64="tjArAx6NhzxT95XjS0tfOz6ea7o=">AAAB73icbZBNSwMxEIZn61etX1WPXoJFqJeyK4Iei148VrAf0C4lm07b0Gx2TbLSsvRPePGgiFf/jjf/jWm7B219IfDwzgyZeYNYcG1c99vJra1vbG7ltws7u3v7B8XDo4aOEsWwziIRqVZANQousW64EdiKFdIwENgMRrezevMJleaRfDCTGP2QDiTvc0aNtVpJNx1Py+PzbrHkVty5yCp4GZQgU61b/Or0IpaEKA0TVOu258bGT6kynAmcFjqJxpiyER1g26KkIWo/ne87JWfW6ZF+pOyThszd3xMpDbWehIHtDKkZ6uXazPyv1k5M/9pPuYwTg5ItPuongpiIzI4nPa6QGTGxQJnidlfChlRRZmxEBRuCt3zyKjQuKp7l+8tS9SaLIw8ncApl8OAKqnAHNagDAwHP8ApvzqPz4rw7H4vWnJPNHMMfOZ8/3rOP1w==</latexit> u(x)<latexit sha1_base64="xyz5fYmv7DAMjqhOp6NQw7olzvk=">AAAB63icbZDNSgMxFIXv1L9a/6ou3QSLUDdlRgRdFt24rOC0hXYomTTThiaZIcmIZegruHGhiFtfyJ1vY6adhbYeCHycey+594QJZ9q47rdTWlvf2Nwqb1d2dvf2D6qHR20dp4pQn8Q8Vt0Qa8qZpL5hhtNuoigWIaedcHKb1zuPVGkWywczTWgg8EiyiBFsciutP50PqjW34c6FVsEroAaFWoPqV38Yk1RQaQjHWvc8NzFBhpVhhNNZpZ9qmmAywSPasyixoDrI5rvO0Jl1hiiKlX3SoLn7eyLDQuupCG2nwGasl2u5+V+tl5roOsiYTFJDJVl8FKUcmRjlh6MhU5QYPrWAiWJ2V0TGWGFibDwVG4K3fPIqtC8anuX7y1rzpoijDCdwCnXw4AqacAct8IHAGJ7hFd4c4bw4787HorXkFDPH8EfO5w+DSo3g</latexit><latexit sha1_base64="xyz5fYmv7DAMjqhOp6NQw7olzvk=">AAAB63icbZDNSgMxFIXv1L9a/6ou3QSLUDdlRgRdFt24rOC0hXYomTTThiaZIcmIZegruHGhiFtfyJ1vY6adhbYeCHycey+594QJZ9q47rdTWlvf2Nwqb1d2dvf2D6qHR20dp4pQn8Q8Vt0Qa8qZpL5hhtNuoigWIaedcHKb1zuPVGkWywczTWgg8EiyiBFsciutP50PqjW34c6FVsEroAaFWoPqV38Yk1RQaQjHWvc8NzFBhpVhhNNZpZ9qmmAywSPasyixoDrI5rvO0Jl1hiiKlX3SoLn7eyLDQuupCG2nwGasl2u5+V+tl5roOsiYTFJDJVl8FKUcmRjlh6MhU5QYPrWAiWJ2V0TGWGFibDwVG4K3fPIqtC8anuX7y1rzpoijDCdwCnXw4AqacAct8IHAGJ7hFd4c4bw4787HorXkFDPH8EfO5w+DSo3g</latexit><latexit sha1_base64="xyz5fYmv7DAMjqhOp6NQw7olzvk=">AAAB63icbZDNSgMxFIXv1L9a/6ou3QSLUDdlRgRdFt24rOC0hXYomTTThiaZIcmIZegruHGhiFtfyJ1vY6adhbYeCHycey+594QJZ9q47rdTWlvf2Nwqb1d2dvf2D6qHR20dp4pQn8Q8Vt0Qa8qZpL5hhtNuoigWIaedcHKb1zuPVGkWywczTWgg8EiyiBFsciutP50PqjW34c6FVsEroAaFWoPqV38Yk1RQaQjHWvc8NzFBhpVhhNNZpZ9qmmAywSPasyixoDrI5rvO0Jl1hiiKlX3SoLn7eyLDQuupCG2nwGasl2u5+V+tl5roOsiYTFJDJVl8FKUcmRjlh6MhU5QYPrWAiWJ2V0TGWGFibDwVG4K3fPIqtC8anuX7y1rzpoijDCdwCnXw4AqacAct8IHAGJ7hFd4c4bw4787HorXkFDPH8EfO5w+DSo3g</latexit><latexit sha1_base64="xyz5fYmv7DAMjqhOp6NQw7olzvk=">AAAB63icbZDNSgMxFIXv1L9a/6ou3QSLUDdlRgRdFt24rOC0hXYomTTThiaZIcmIZegruHGhiFtfyJ1vY6adhbYeCHycey+594QJZ9q47rdTWlvf2Nwqb1d2dvf2D6qHR20dp4pQn8Q8Vt0Qa8qZpL5hhtNuoigWIaedcHKb1zuPVGkWywczTWgg8EiyiBFsciutP50PqjW34c6FVsEroAaFWoPqV38Yk1RQaQjHWvc8NzFBhpVhhNNZpZ9qmmAywSPasyixoDrI5rvO0Jl1hiiKlX3SoLn7eyLDQuupCG2nwGasl2u5+V+tl5roOsiYTFJDJVl8FKUcmRjlh6MhU5QYPrWAiWJ2V0TGWGFibDwVG4K3fPIqtC8anuX7y1rzpoijDCdwCnXw4AqacAct8IHAGJ7hFd4c4bw4787HorXkFDPH8EfO5w+DSo3g</latexit>
  • 9. Our approach … Input The Neural Network (~x, t)<latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit> ˆu<latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit> Output Space Time <latexit sha1_base64="f64HEF6znFEZoH5kmjcDbs/59Jo=">AAAB7XicbZBNSwMxEIZn61etX1WPXoJFqCBlVwQ9Fr14rGA/oF1KNk3b2OxmSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZeYNYCoOu++3kVlbX1jfym4Wt7Z3dveL+QcOoRDNeZ0oq3Qqo4VJEvI4CJW/FmtMwkLwZjG6m9eYj10ao6B7HMfdDOohEXzCK1mok5aczPO0WS27FnYksg5dBCTLVusWvTk+xJOQRMkmNaXtujH5KNQom+aTQSQyPKRvRAW9bjGjIjZ/Otp2QE+v0SF9p+yIkM/f3REpDY8ZhYDtDikOzWJua/9XaCfav/FREcYI8YvOP+okkqMj0dNITmjOUYwuUaWF3JWxINWVoAyrYELzFk5ehcV7xLN9dlKrXWRx5OIJjKIMHl1CFW6hBHRg8wDO8wpujnBfn3fmYt+acbOYQ/sj5/AHCpY6U</latexit> Construct the library w.r.t. the inferred solution Θ = ! ! ! u ux uxu ! ! ! ! ! ! uxx uxxu uxxux ! ! ! " ⎛ ⎝ ⎜ ⎜ ⎜ ⎞ ⎠ ⎟ ⎟ ⎟ ^ ^ ^ ^ ^ ^ ^ ^ ^ Include the regression task within the neural network
  • 10. How do we train a neural network Optimise loss function: L = LMSE + LReg + LL1<latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit> Learn the mapping (~x, t) ! ~u<latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit> Approximate the data as good as possible …
  • 11. How do we train a neural network Optimise loss function: L = LMSE + LReg + LL1<latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit> Learn the mapping (~x, t) ! ~u<latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit> Approximate the data as good as possible … Regression (Seek the PDE) Discover the differential equation
  • 12. L1 Penalty Promoting sparsity How do we train a neural network Optimise loss function: L = LMSE + LReg + LL1<latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit> Learn the mapping (~x, t) ! ~u<latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit><latexit sha1_base64="yJgnElJriwfv2D53AURbemLgi/I=">AAACCHicbZDLSsNAFIYn9VbrLerShYNFqCAlEUGXRTcuK9gLNKFMptN26OTCzEm1hCzd+CpuXCji1kdw59s4TbPQ1h8GPv5zDmfO70WCK7Csb6OwtLyyulZcL21sbm3vmLt7TRXGkrIGDUUo2x5RTPCANYCDYO1IMuJ7grW80fW03hozqXgY3MEkYq5PBgHvc0pAW13zsOKMGU0e0lM4wY7kgyEQKcN7nNlx2jXLVtXKhBfBzqGMctW75pfTC2nsswCoIEp1bCsCNyESOBUsLTmxYhGhIzJgHY0B8Zlyk+yQFB9rp4f7odQvAJy5vycS4is18T3d6RMYqvna1Pyv1omhf+kmPIhiYAGdLerHAkOIp6ngHpeMgphoIFRy/VdMh0QSCjq7kg7Bnj95EZpnVVvz7Xm5dpXHUUQH6AhVkI0uUA3doDpqIIoe0TN6RW/Gk/FivBsfs9aCkc/soz8yPn8AtuSZyA==</latexit> Approximate the data as good as possible … Regression (Seek the PDE) Discover the differential equation
  • 14. <latexit sha1_base64="1Gq7mrW1/ZKuQjtOjsnM1gDsGDU=">AAACBHicbZDJSgNBEIZ74hbjFvWYS2MQIkiYEUEvQtCLxwjZIBNCT6eSNOlZ6K4JCUMOXnwVLx4U8epDePNt7CwHTfyh4eOvKqrr9yIpNNr2t5VaW9/Y3EpvZ3Z29/YPsodHNR3GikOVhzJUDY9pkCKAKgqU0IgUMN+TUPcGd9N6fQhKizCo4DiCls96gegKztBY7WwubmNhdI5n9Ia6lT4go+4QeOKOxKSdzdtFeya6Cs4C8mShcjv75XZCHvsQIJdM66ZjR9hKmELBJUwybqwhYnzAetA0GDAfdCuZHTGhp8bp0G6ozAuQztzfEwnztR77nun0Gfb1cm1q/ldrxti9biUiiGKEgM8XdWNJMaTTRGhHKOAoxwYYV8L8lfI+U4yjyS1jQnCWT16F2kXRMfxwmS/dLuJIkxw5IQXikCtSIvekTKqEk0fyTF7Jm/VkvVjv1se8NWUtZo7JH1mfPysqlyg=</latexit> ⇠⇤ =<latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit> Neural Network Threshold ˆuu Auto. Diff. ∂t ˆu = Θξ Library Include in cost function NN converged ξ∗ = ξ · ||Θ|| ||ut|| Solution Sparsity converged × × × × × × × × × × × keep set to zero keep ξ∗ Update cost function L = MSE(u, ˆu) + MSE(∂t ˆu, Θξ) + L1(ξ) = ... u ux uux u2 uxx Found solution ∂tu = uux − 0.1uxx 0 0 1 0 ... -0.1 0 uux uxx u ux u2 uxx Convergence ξ optimised MSE minimized c) Training the coefficient vector
  • 15. <latexit sha1_base64="1Gq7mrW1/ZKuQjtOjsnM1gDsGDU=">AAACBHicbZDJSgNBEIZ74hbjFvWYS2MQIkiYEUEvQtCLxwjZIBNCT6eSNOlZ6K4JCUMOXnwVLx4U8epDePNt7CwHTfyh4eOvKqrr9yIpNNr2t5VaW9/Y3EpvZ3Z29/YPsodHNR3GikOVhzJUDY9pkCKAKgqU0IgUMN+TUPcGd9N6fQhKizCo4DiCls96gegKztBY7WwubmNhdI5n9Ia6lT4go+4QeOKOxKSdzdtFeya6Cs4C8mShcjv75XZCHvsQIJdM66ZjR9hKmELBJUwybqwhYnzAetA0GDAfdCuZHTGhp8bp0G6ozAuQztzfEwnztR77nun0Gfb1cm1q/ldrxti9biUiiGKEgM8XdWNJMaTTRGhHKOAoxwYYV8L8lfI+U4yjyS1jQnCWT16F2kXRMfxwmS/dLuJIkxw5IQXikCtSIvekTKqEk0fyTF7Jm/VkvVjv1se8NWUtZo7JH1mfPysqlyg=</latexit> Dataset Neural Network Threshold ˆuu Auto. Diff. ∂t ˆu = Θξ Library Include in cost function NN converged ξ∗ = ξ · ||Θ|| ||ut|| Sparsity c × × × × × × × × × × × keep set to zero keep ξ∗ Update cost function Space Time Burgers equation ∂tu = uux − 0.1uxx L = MSE(u, ˆu) + MSE(∂t ˆu, Θξ) + L1(ξ) = ... u ux uux u2 uxx Space Time <latexit sha1_base64="f64HEF6znFEZoH5kmjcDbs/59Jo=">AAAB7XicbZBNSwMxEIZn61etX1WPXoJFqCBlVwQ9Fr14rGA/oF1KNk3b2OxmSWbFsvQ/ePGgiFf/jzf/jWm7B219IfDwzgyZeYNYCoOu++3kVlbX1jfym4Wt7Z3dveL+QcOoRDNeZ0oq3Qqo4VJEvI4CJW/FmtMwkLwZjG6m9eYj10ao6B7HMfdDOohEXzCK1mok5aczPO0WS27FnYksg5dBCTLVusWvTk+xJOQRMkmNaXtujH5KNQom+aTQSQyPKRvRAW9bjGjIjZ/Otp2QE+v0SF9p+yIkM/f3REpDY8ZhYDtDikOzWJua/9XaCfav/FREcYI8YvOP+okkqMj0dNITmjOUYwuUaWF3JWxINWVoAyrYELzFk5ehcV7xLN9dlKrXWRx5OIJjKIMHl1CFW6hBHRg8wDO8wpujnBfn3fmYt+acbOYQ/sj5/AHCpY6U</latexit> SolutionNoisy Data Regression Code available: github.com/PhIMaL L = LMSE + LReg + LL1<latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit><latexit sha1_base64="gUOXx4d83oxXFYgklynsP3UnEo8=">AAACMHicbVDLSsNAFJ3UV62vqEs3g0UQhJKIoBuhKKKLCvXRB7QhTKbTduhkEmYmQgn5JDd+im4UFHHrVzhps+jDAwNnzrmXe+/xQkalsqwPI7ewuLS8kl8trK1vbG6Z2zt1GUQCkxoOWCCaHpKEUU5qiipGmqEgyPcYaXiDy9RvPBEhacAf1TAkjo96nHYpRkpLrnnd9pHqY8TiSgLP4cTPjW8frhJ4NK3dk96cVnHtxDWLVskaAc4TOyNFkKHqmq/tToAjn3CFGZKyZVuhcmIkFMWMJIV2JEmI8AD1SEtTjnwinXh0cAIPtNKB3UDoxxUcqZMdMfKlHPqerkwXlbNeKv7ntSLVPXNiysNIEY7Hg7oRgyqAaXqwQwXBig01QVhQvSvEfSQQVjrjgg7Bnj15ntSPS7bmdyfF8kUWRx7sgX1wCGxwCsrgBlRBDWDwDN7AJ/gyXox349v4GZfmjKxnF0zB+P0DA3+pkA==</latexit> Neural network (~x, t)<latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit><latexit sha1_base64="lT4SfvKTT4nZSef6YVRDL9M85fI=">AAAB8nicbZDLSgMxFIYz9VbrrerSTbAIFaTMiKDLohuXFewFpkPJpJk2NJMMyZliGfoYblwo4tancefbmLaz0NYfAh//OYec84eJ4AZc99sprK1vbG4Vt0s7u3v7B+XDo5ZRqaasSZVQuhMSwwSXrAkcBOskmpE4FKwdju5m9faYacOVfIRJwoKYDCSPOCVgLb/aHTOaPU0v4LxXrrg1dy68Cl4OFZSr0St/dfuKpjGTQAUxxvfcBIKMaOBUsGmpmxqWEDoiA+ZblCRmJsjmK0/xmXX6OFLaPgl47v6eyEhszCQObWdMYGiWazPzv5qfQnQTZFwmKTBJFx9FqcCg8Ox+3OeaURATC4RqbnfFdEg0oWBTKtkQvOWTV6F1WfMsP1xV6rd5HEV0gk5RFXnoGtXRPWqgJqJIoWf0it4ccF6cd+dj0Vpw8plj9EfO5w++jZDj</latexit> ˆu<latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit><latexit sha1_base64="BGeEfkLOCpJ34NMiu4knM6UlhOc=">AAAB7nicbZBNSwMxEIZn/az1q+rRS7AInsquCHosevFYwX5Au5Rsmm1Ds9klmQhl6Y/w4kERr/4eb/4b03YP2vpC4OGdGTLzRpkUBn3/21tb39jc2i7tlHf39g8OK0fHLZNazXiTpTLVnYgaLoXiTRQoeSfTnCaR5O1ofDert5+4NiJVjzjJeJjQoRKxYBSd1e6NKOZ22q9U/Zo/F1mFoIAqFGr0K1+9QcpswhUySY3pBn6GYU41Cib5tNyzhmeUjemQdx0qmnAT5vN1p+TcOQMSp9o9hWTu/p7IaWLMJIlcZ0JxZJZrM/O/WtdifBPmQmUWuWKLj2IrCaZkdjsZCM0ZyokDyrRwuxI2opoydAmVXQjB8smr0LqsBY4frqr12yKOEpzCGVxAANdQh3toQBMYjOEZXuHNy7wX7937WLSuecXMCfyR9/kDq6+Pxg==</latexit> ⇠⇤ =<latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit><latexit sha1_base64="waOymetEs+AzOO0/zGH/BLksit4=">AAAB7XicbZDLSgMxFIbP1Futt6pLN8EiiIsyI4JuhKIblxXsBdqxZNJMG5tJhiQjlqHv4MaFIm59H3e+jZl2Ftr6Q+DjP+eQc/4g5kwb1/12CkvLK6trxfXSxubW9k55d6+pZaIIbRDJpWoHWFPOBG0YZjhtx4riKOC0FYyus3rrkSrNpLgz45j6ER4IFjKCjbWa3Sd2f3LZK1fcqjsVWgQvhwrkqvfKX92+JElEhSEca93x3Nj4KVaGEU4npW6iaYzJCA9ox6LAEdV+Ot12go6s00ehVPYJg6bu74kUR1qPo8B2RtgM9XwtM/+rdRITXvgpE3FiqCCzj8KEIyNRdjrqM0WJ4WMLmChmd0VkiBUmxgZUsiF48ycvQvO06lm+PavUrvI4inAAh3AMHpxDDW6gDg0g8ADP8ApvjnRenHfnY9ZacPKZffgj5/MH+eWOuA==</latexit> Neural Network Threshold ˆuu Auto. Diff. ∂t ˆu = Θξ Library Include in cost function NN converged ξ∗ = ξ · ||Θ|| ||ut|| Solution Sparsity converged × × × × × × × × × × × keep set to zero keep ξ∗ Update cost function L = MSE(u, ˆu) + MSE(∂t ˆu, Θξ) + L1(ξ) = ... u ux uux u2 uxx Found solution ∂tu = uux − 0.1uxx 0 0 1 0 ... -0.1 0 uux uxx u ux u2 uxx DeepMoD Model discovery neural networks DeepMoD Model discovery neural networks
  • 17. Resilience to Noise/ sampling size Relative error on coefficients <latexit sha1_base64="6yAhDb2lEN8ufJi3rjVOCio5Mf8=">AAACAnicbZDLSgMxFIYz9VbrbdSVuAkWQRDLjAi6EYpuXFawF2iHIZOmbWgmMyQn0jIUN76KGxeKuPUp3Pk2ppeFVn8IfPznHE7OH6WCa/C8Lye3sLi0vJJfLaytb2xuuds7NZ0YRVmVJiJRjYhoJrhkVeAgWCNVjMSRYPWofz2u1++Z0jyRdzBMWRCTruQdTglYK3T3TAj4Ep8YbMIBPsYtOaZsMBiFbtEreRPhv+DPoIhmqoTuZ6udUBMzCVQQrZu+l0KQEQWcCjYqtIxmKaF90mVNi5LETAfZ5IQRPrROG3cSZZ8EPHF/TmQk1noYR7YzJtDT87Wx+V+taaBzEWRcpgaYpNNFHSMwJHicB25zxSiIoQVCFbd/xbRHFKFgUyvYEPz5k/9C7bTkW749K5avZnHk0T46QEfIR+eojG5QBVURRQ/oCb2gV+fReXbenPdpa86ZzeyiX3I+vgEk2JX+</latexit> Burgers’ equation Applications in physics Higher order PDEs <latexit sha1_base64="89mN2eImzrIm0QzNLSbmbxofcBY=">AAACAHicbZDLSsNAFIYn9VbrLerChZvBIrixJCLqRii6cVnBXqANYTKdtEMnkzAXSQnZ+CpuXCji1sdw59s4bbPQ1gMzfPz/OcycP0gYlcpxvq3S0vLK6lp5vbKxubW9Y+/utWSsBSZNHLNYdAIkCaOcNBVVjHQSQVAUMNIORrcTv/1IhKQxf1DjhHgRGnAaUoyUkXz7QPsKXsMLqKH2U3hq7ixN09y3q07NmRZcBLeAKiiq4dtfvX6MdUS4wgxJ2XWdRHkZEopiRvJKT0uSIDxCA9I1yFFEpJdNF8jhsVH6MIyFOVzBqfp7IkORlOMoMJ0RUkM5703E/7yuVuGVl1GeaEU4nj0UagZVDCdpwD4VBCs2NoCwoOavEA+RQFiZzComBHd+5UVondVcw/fn1fpNEUcZHIIjcAJccAnq4A40QBNgkINn8ArerCfrxXq3PmatJauY2Qd/yvr8Aa06lS4=</latexit> t = 5 t = 7 t = 9 Ground truth Sampled Inferred y y y Coupled equations <latexit sha1_base64="oJZFq0o7Dpv+0FhHmocEH1HfGbM=">AAACJ3icbZDLSgMxFIYzXmu9jbp0EyxCXVhmRNCNUtSFywr2Ap0yZNJMG5rJDMkZtZS+jRtfxY2gIrr0TUzbWWjrgZCP/z+H5PxBIrgGx/my5uYXFpeWcyv51bX1jU17a7um41RRVqWxiFUjIJoJLlkVOAjWSBQjUSBYPehdjvz6HVOax/IW+glrRaQjecgpASP59nnqAz7DniSBIB5tx4A9wUIo4iufZzJO8SH2Hri5M+Eee4p3unDg2wWn5IwLz4KbQQFlVfHtV68d0zRiEqggWjddJ4HWgCjgVLBh3ks1SwjtkQ5rGpQkYro1GO85xPtGaeMwVuZIwGP198SARFr3o8B0RgS6etobif95zRTC09aAyyQFJunkoTAVGGI8Cg23uWIURN8AoYqbv2LaJYpQMNHmTQju9MqzUDsquYZvjgvliyyOHNpFe6iIXHSCyugaVVAVUfSIntEbereerBfrw/qctM5Z2cwO+lPW9w97rKPZ</latexit> <latexit sha1_base64="TZ48ZV1oiCE1tUaFN922vZ54mwQ=">AAACCXicbZDLSgMxFIYz9VbrrerSzcEiCGKZEUE3QtEuXFawF2iHIZOmbWjmQnLGUkq3bnwVNy4UcesbuPNtTNtZaOuBhI//P4fk/H4shUbb/rYyS8srq2vZ9dzG5tb2Tn53r6ajRDFeZZGMVMOnmksR8ioKlLwRK04DX/K637+Z+PUHrrSIwnscxtwNaDcUHcEoGsnLw8BDuAIoewNolblECgM4hb65T6AHCXj5gl20pwWL4KRQIGlVvPxXqx2xJOAhMkm1bjp2jO6IKhRM8nGulWgeU9anXd40GNKAa3c03WQMR0ZpQydS5oQIU/X3xIgGWg8D33QGFHt63puI/3nNBDuX7kiEcYI8ZLOHOokEjGASC7SF4gzl0ABlSpi/AutRRRma8HImBGd+5UWonRUdw3fnhdJ1GkeWHJBDckwcckFK5JZUSJUw8kieySt5s56sF+vd+pi1Zqx0Zp/8KevzB0yCluA=</latexit> u<latexit sha1_base64="0ivuOvsA1zGj29z/88YG+b8aLnI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjC/YD2lA220m7drMJuxuhhP4CLx4U8epP8ua/cdvmoK0vLDy8M8POvEEiuDau++0UNja3tneKu6W9/YPDo/LxSVvHqWLYYrGIVTegGgWX2DLcCOwmCmkUCOwEk7t5vfOESvNYPphpgn5ER5KHnFFjrWY6KFfcqrsQWQcvhwrkagzKX/1hzNIIpWGCat3z3MT4GVWGM4GzUj/VmFA2oSPsWZQ0Qu1ni0Vn5MI6QxLGyj5pyML9PZHRSOtpFNjOiJqxXq3Nzf9qvdSEN37GZZIalGz5UZgKYmIyv5oMuUJmxNQCZYrbXQkbU0WZsdmUbAje6snr0L6qepab15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh85nz/hr4z5</latexit><latexit sha1_base64="0ivuOvsA1zGj29z/88YG+b8aLnI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjC/YD2lA220m7drMJuxuhhP4CLx4U8epP8ua/cdvmoK0vLDy8M8POvEEiuDau++0UNja3tneKu6W9/YPDo/LxSVvHqWLYYrGIVTegGgWX2DLcCOwmCmkUCOwEk7t5vfOESvNYPphpgn5ER5KHnFFjrWY6KFfcqrsQWQcvhwrkagzKX/1hzNIIpWGCat3z3MT4GVWGM4GzUj/VmFA2oSPsWZQ0Qu1ni0Vn5MI6QxLGyj5pyML9PZHRSOtpFNjOiJqxXq3Nzf9qvdSEN37GZZIalGz5UZgKYmIyv5oMuUJmxNQCZYrbXQkbU0WZsdmUbAje6snr0L6qepab15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh85nz/hr4z5</latexit><latexit sha1_base64="0ivuOvsA1zGj29z/88YG+b8aLnI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjC/YD2lA220m7drMJuxuhhP4CLx4U8epP8ua/cdvmoK0vLDy8M8POvEEiuDau++0UNja3tneKu6W9/YPDo/LxSVvHqWLYYrGIVTegGgWX2DLcCOwmCmkUCOwEk7t5vfOESvNYPphpgn5ER5KHnFFjrWY6KFfcqrsQWQcvhwrkagzKX/1hzNIIpWGCat3z3MT4GVWGM4GzUj/VmFA2oSPsWZQ0Qu1ni0Vn5MI6QxLGyj5pyML9PZHRSOtpFNjOiJqxXq3Nzf9qvdSEN37GZZIalGz5UZgKYmIyv5oMuUJmxNQCZYrbXQkbU0WZsdmUbAje6snr0L6qepab15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh85nz/hr4z5</latexit><latexit sha1_base64="0ivuOvsA1zGj29z/88YG+b8aLnI=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEeqx6MVjC/YD2lA220m7drMJuxuhhP4CLx4U8epP8ua/cdvmoK0vLDy8M8POvEEiuDau++0UNja3tneKu6W9/YPDo/LxSVvHqWLYYrGIVTegGgWX2DLcCOwmCmkUCOwEk7t5vfOESvNYPphpgn5ER5KHnFFjrWY6KFfcqrsQWQcvhwrkagzKX/1hzNIIpWGCat3z3MT4GVWGM4GzUj/VmFA2oSPsWZQ0Qu1ni0Vn5MI6QxLGyj5pyML9PZHRSOtpFNjOiJqxXq3Nzf9qvdSEN37GZZIalGz5UZgKYmIyv5oMuUJmxNQCZYrbXQkbU0WZsdmUbAje6snr0L6qepab15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh85nz/hr4z5</latexit> w<latexit sha1_base64="sGagZRpsBDQQXLOTtGoRB0Vi9Sc=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OKxBfsBbSib7aRdu9mE3Y1SQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOyZl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJr/2MyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVR9Sw3Liu1mzyOIpzAKZyDB1dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A5LeM+w==</latexit><latexit sha1_base64="sGagZRpsBDQQXLOTtGoRB0Vi9Sc=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OKxBfsBbSib7aRdu9mE3Y1SQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOyZl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJr/2MyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVR9Sw3Liu1mzyOIpzAKZyDB1dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A5LeM+w==</latexit><latexit sha1_base64="sGagZRpsBDQQXLOTtGoRB0Vi9Sc=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OKxBfsBbSib7aRdu9mE3Y1SQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOyZl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJr/2MyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVR9Sw3Liu1mzyOIpzAKZyDB1dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A5LeM+w==</latexit><latexit sha1_base64="sGagZRpsBDQQXLOTtGoRB0Vi9Sc=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tF8FQSEfRY9OKxBfsBbSib7aRdu9mE3Y1SQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOyZl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJr/2MyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVR9Sw3Liu1mzyOIpzAKZyDB1dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A5LeM+w==</latexit> :particle density :attractant density 2D equations
  • 18. Raw images Density field ut = 0.3uy + 0.01uxx + 0.01uyy<latexit sha1_base64="giCCDNoPIOmus3F40Yxqtlm2jVw=">AAACEnicbZBLS8NAEMc39VXrK+rRy2IRFKEkKuhFKHrxWME+oA1hs922SzebsA9pCPkMXvwqXjwo4tWTN7+N2zaH2jqw8Jv/zDA7/yBmVCrH+bEKS8srq2vF9dLG5tb2jr2715CRFpjUccQi0QqQJIxyUldUMdKKBUFhwEgzGN6O681HIiSN+INKYuKFqM9pj2KkjOTbJ9pX8Bo6lXOo/QSeGnJcg+lolM1kSZL5dtlkk4CL4OZQBnnUfPu7042wDglXmCEp264TKy9FQlHMSFbqaElihIeoT9oGOQqJ9NLJSRk8MkoX9iJhHldwos5OpCiUMgkD0xkiNZDztbH4X62tVe/KSymPtSIcTxf1NIMqgmN/YJcKghVLDCAsqPkrxAMkEFbGxZIxwZ0/eREaZxXX8P1FuXqT21EEB+AQHAMXXIIquAM1UAcYPIEX8AberWfr1fqwPqetBSuf2Qd/wvr6BTy+mhY=</latexit><latexit sha1_base64="giCCDNoPIOmus3F40Yxqtlm2jVw=">AAACEnicbZBLS8NAEMc39VXrK+rRy2IRFKEkKuhFKHrxWME+oA1hs922SzebsA9pCPkMXvwqXjwo4tWTN7+N2zaH2jqw8Jv/zDA7/yBmVCrH+bEKS8srq2vF9dLG5tb2jr2715CRFpjUccQi0QqQJIxyUldUMdKKBUFhwEgzGN6O681HIiSN+INKYuKFqM9pj2KkjOTbJ9pX8Bo6lXOo/QSeGnJcg+lolM1kSZL5dtlkk4CL4OZQBnnUfPu7042wDglXmCEp264TKy9FQlHMSFbqaElihIeoT9oGOQqJ9NLJSRk8MkoX9iJhHldwos5OpCiUMgkD0xkiNZDztbH4X62tVe/KSymPtSIcTxf1NIMqgmN/YJcKghVLDCAsqPkrxAMkEFbGxZIxwZ0/eREaZxXX8P1FuXqT21EEB+AQHAMXXIIquAM1UAcYPIEX8AberWfr1fqwPqetBSuf2Qd/wvr6BTy+mhY=</latexit><latexit sha1_base64="giCCDNoPIOmus3F40Yxqtlm2jVw=">AAACEnicbZBLS8NAEMc39VXrK+rRy2IRFKEkKuhFKHrxWME+oA1hs922SzebsA9pCPkMXvwqXjwo4tWTN7+N2zaH2jqw8Jv/zDA7/yBmVCrH+bEKS8srq2vF9dLG5tb2jr2715CRFpjUccQi0QqQJIxyUldUMdKKBUFhwEgzGN6O681HIiSN+INKYuKFqM9pj2KkjOTbJ9pX8Bo6lXOo/QSeGnJcg+lolM1kSZL5dtlkk4CL4OZQBnnUfPu7042wDglXmCEp264TKy9FQlHMSFbqaElihIeoT9oGOQqJ9NLJSRk8MkoX9iJhHldwos5OpCiUMgkD0xkiNZDztbH4X62tVe/KSymPtSIcTxf1NIMqgmN/YJcKghVLDCAsqPkrxAMkEFbGxZIxwZ0/eREaZxXX8P1FuXqT21EEB+AQHAMXXIIquAM1UAcYPIEX8AberWfr1fqwPqetBSuf2Qd/wvr6BTy+mhY=</latexit><latexit sha1_base64="giCCDNoPIOmus3F40Yxqtlm2jVw=">AAACEnicbZBLS8NAEMc39VXrK+rRy2IRFKEkKuhFKHrxWME+oA1hs922SzebsA9pCPkMXvwqXjwo4tWTN7+N2zaH2jqw8Jv/zDA7/yBmVCrH+bEKS8srq2vF9dLG5tb2jr2715CRFpjUccQi0QqQJIxyUldUMdKKBUFhwEgzGN6O681HIiSN+INKYuKFqM9pj2KkjOTbJ9pX8Bo6lXOo/QSeGnJcg+lolM1kSZL5dtlkk4CL4OZQBnnUfPu7042wDglXmCEp264TKy9FQlHMSFbqaElihIeoT9oGOQqJ9NLJSRk8MkoX9iJhHldwos5OpCiUMgkD0xkiNZDztbH4X62tVe/KSymPtSIcTxf1NIMqgmN/YJcKghVLDCAsqPkrxAMkEFbGxZIxwZ0/eREaZxXX8P1FuXqT21EEB+AQHAMXXIIquAM1UAcYPIEX8AberWfr1fqwPqetBSuf2Qd/wvr6BTy+mhY=</latexit> Underlying AD equation Apply to real data … Model discovery from experimental data
  • 19. Single ParticleTracking Time xx Time Time Time Trajectory <latexit sha1_base64="A2yY/FcKHQddACwl6l5TaATnAyI=">AAACFXicbZDLSsQwFIZT7463qks3wUEQFGkHQTeC6MalguMMTGtJM+kYTNOSnMgMpS/hxldx40IRt4I738bMZaEz/hD4+M85nJw/zgXX4HnfztT0zOzc/MJiZWl5ZXXNXd+40ZlRlNVpJjLVjIlmgktWBw6CNXPFSBoL1ojvz/v1xgNTmmfyGno5C1PSkTzhlIC1InffRAVAifdwkChCC78sAiAmoiU2EeATHEhzW7NcdLtl5Fa9A28gPAn+CKpopMvI/QraGTUpk0AF0brlezmEBVHAqWBlJTCa5YTekw5rWZQkZTosBleVeMc6bZxkyj4JeOD+nihIqnUvjW1nSuBOj9f65n+1loHkOCy4zA0wSYeLEiMwZLgfEW5zxSiIngVCFbd/xfSO2HTABlmxIfjjJ0/CTe3At3x1WD09G8WxgLbQNtpFPjpCp+gCXaI6ougRPaNX9OY8OS/Ou/MxbJ1yRjOb6I+czx/MZJ6T</latexit> <latexit sha1_base64="l3iwXAx+BJVV3L+FIIt6mI0zRgc=">AAACAnicbZDLSsNAFIYn9VbrLepK3AwWQRBKIoJuhKIuXFawF2hDmEym7dDJJMyltITgxldx40IRtz6FO9/GaZuFtv4w8PGfczhz/iBhVCrH+bYKS8srq2vF9dLG5tb2jr2715CxFpjUccxi0QqQJIxyUldUMdJKBEFRwEgzGNxM6s0hEZLG/EGNE+JFqMdpl2KkjOXbB9pPVQav4C00NBpl8BQO/RBq6Ntlp+JMBRfBzaEMctV8+6sTxlhHhCvMkJRt10mUlyKhKGYkK3W0JAnCA9QjbYMcRUR66fSEDB4bJ4TdWJjHFZy6vydSFEk5jgLTGSHVl/O1iflfra1V99JLKU+0IhzPFnU1gyqGkzxgSAXBio0NICyo+SvEfSQQVia1kgnBnT95ERpnFdfw/Xm5ep3HUQSH4AicABdcgCq4AzVQBxg8gmfwCt6sJ+vFerc+Zq0FK5/ZB39kff4A+62V2w==</latexit> <latexit sha1_base64="q8SSRp0DvPa9XRci5Cdg/mAJUPE=">AAAB+nicbZDLSsNAFIZP6q3WW6pLN4NFcFUSEXQjFHXhsoK9QBvCZDpth04mYWaiLTGP4saFIm59Ene+jdM2C239YeDjP+dwzvxBzJnSjvNtFVZW19Y3ipulre2d3T27vN9UUSIJbZCIR7IdYEU5E7Shmea0HUuKw4DTVjC6ntZbD1QqFol7PYmpF+KBYH1GsDaWb5cTP9UZukQ3yNB4nCHfrjhVZya0DG4OFchV9+2vbi8iSUiFJhwr1XGdWHsplpoRTrNSN1E0xmSEB7RjUOCQKi+dnZ6hY+P0UD+S5gmNZu7viRSHSk3CwHSGWA/VYm1q/lfrJLp/4aVMxImmgswX9ROOdISmOaAek5RoPjGAiWTmVkSGWGKiTVolE4K7+OVlaJ5WXcN3Z5XaVR5HEQ7hCE7AhXOowS3UoQEEHuEZXuHNerJerHfrY95asPKZA/gj6/MHPD2TUg==</latexit> Random walk Advected random walk Persistant random walk x Density estimation Model discovery
  • 20. Time xx Time Time Time Trajectory <latexit sha1_base64="A2yY/FcKHQddACwl6l5TaATnAyI=">AAACFXicbZDLSsQwFIZT7463qks3wUEQFGkHQTeC6MalguMMTGtJM+kYTNOSnMgMpS/hxldx40IRt4I738bMZaEz/hD4+M85nJw/zgXX4HnfztT0zOzc/MJiZWl5ZXXNXd+40ZlRlNVpJjLVjIlmgktWBw6CNXPFSBoL1ojvz/v1xgNTmmfyGno5C1PSkTzhlIC1InffRAVAifdwkChCC78sAiAmoiU2EeATHEhzW7NcdLtl5Fa9A28gPAn+CKpopMvI/QraGTUpk0AF0brlezmEBVHAqWBlJTCa5YTekw5rWZQkZTosBleVeMc6bZxkyj4JeOD+nihIqnUvjW1nSuBOj9f65n+1loHkOCy4zA0wSYeLEiMwZLgfEW5zxSiIngVCFbd/xfSO2HTABlmxIfjjJ0/CTe3At3x1WD09G8WxgLbQNtpFPjpCp+gCXaI6ougRPaNX9OY8OS/Ou/MxbJ1yRjOb6I+czx/MZJ6T</latexit> <latexit sha1_base64="l3iwXAx+BJVV3L+FIIt6mI0zRgc=">AAACAnicbZDLSsNAFIYn9VbrLepK3AwWQRBKIoJuhKIuXFawF2hDmEym7dDJJMyltITgxldx40IRtz6FO9/GaZuFtv4w8PGfczhz/iBhVCrH+bYKS8srq2vF9dLG5tb2jr2715CxFpjUccxi0QqQJIxyUldUMdJKBEFRwEgzGNxM6s0hEZLG/EGNE+JFqMdpl2KkjOXbB9pPVQav4C00NBpl8BQO/RBq6Ntlp+JMBRfBzaEMctV8+6sTxlhHhCvMkJRt10mUlyKhKGYkK3W0JAnCA9QjbYMcRUR66fSEDB4bJ4TdWJjHFZy6vydSFEk5jgLTGSHVl/O1iflfra1V99JLKU+0IhzPFnU1gyqGkzxgSAXBio0NICyo+SvEfSQQVia1kgnBnT95ERpnFdfw/Xm5ep3HUQSH4AicABdcgCq4AzVQBxg8gmfwCt6sJ+vFerc+Zq0FK5/ZB39kff4A+62V2w==</latexit> <latexit sha1_base64="q8SSRp0DvPa9XRci5Cdg/mAJUPE=">AAAB+nicbZDLSsNAFIZP6q3WW6pLN4NFcFUSEXQjFHXhsoK9QBvCZDpth04mYWaiLTGP4saFIm59Ene+jdM2C239YeDjP+dwzvxBzJnSjvNtFVZW19Y3ipulre2d3T27vN9UUSIJbZCIR7IdYEU5E7Shmea0HUuKw4DTVjC6ntZbD1QqFol7PYmpF+KBYH1GsDaWb5cTP9UZukQ3yNB4nCHfrjhVZya0DG4OFchV9+2vbi8iSUiFJhwr1XGdWHsplpoRTrNSN1E0xmSEB7RjUOCQKi+dnZ6hY+P0UD+S5gmNZu7viRSHSk3CwHSGWA/VYm1q/lfrJLp/4aVMxImmgswX9ROOdISmOaAek5RoPjGAiWTmVkSGWGKiTVolE4K7+OVlaJ5WXcN3Z5XaVR5HEQ7hCE7AhXOowS3UoQEEHuEZXuHNerJerHfrY95asPKZA/gj6/MHPD2TUg==</latexit> Random walk Advected random walk Persistant random walk x Density estimation Model discovery Single ParticleTracking
  • 21. Density estimation:Temporal normalizing flows t z t x Latent spaceReal space x t Positional data Temporal Normalizing Flows (tNFs) extend NFs with a temporal component to estimate a time-dependent density evolution Code available on Github: https://github.com/PhIMaL Paper available on arXiv: 1912.09092
  • 22. Density estimation:Temporal normalizing flows t z t x Latent spaceReal space x t Positional data Temporal Normalizing Flows (tNFs) extend NFs with a temporal component to estimate a time-dependent density evolution tNFs allow discovery of PDEs with conservation constraints, e.g. Fokker-Planck, … Code available on Github: https://github.com/PhIMaL Paper available on arXiv: 1912.09092
  • 23. Code available on Github: https://github.com/PhIMaL DeepMoD Model discovery neural networks DeepMoD Model discovery neural networks Paper available on arXiv: 1904.08406 T. Chrysostomou (Intern) C. Le Scao ( Intern) A, Brandon-Bravo ( Engineer) G.-J. Both ( PhD) @GJ_Both @RemyKusters