I will discuss paradigmatic statistical models of inference and learning from high dimensional data, such as sparse PCA and the perceptron neural network, in the sub-linear sparsity regime. In this limit the underlying hidden signal, i.e., the low-rank matrix in PCA or the neural network weights, has a number of non-zero components that scales sub-linearly with the total dimension of the vector. I will provide explicit low-dimensional variational formulas for the asymptotic mutual information between the signal and the data in suitable sparse limits. In the setting of support recovery these formulas imply sharp 0-1 phase transitions for the asymptotic minimum mean-square-error (or generalization error in the neural network setting). A similar phase transition was analyzed recently in the context of sparse high-dimensional linear regression by Reeves et al.
2019 Fall Series: Special Guest Lecture - 0-1 Phase Transitions in High Dimensional Inference and Learning - Jean Barbier, December 3, 2019
1. 0-1 Phase Transitions
in High-Dimensional Inference & Learning
Jean Barbier
International Center for Theoretical Physics, Trieste, Italy
Error
SNR
Joint work with Nicolas Macris & Clément Luneau, EPFL
2. Spiked matrix/tensor models
W =
q
n X ⌦ X ⌦ . . . ⌦ X + Z
<latexit sha1_base64="qQseBedGJiS1x3H0QMD/U8sLiZ8=">AAADInicjVFBS+QwGP2surqz7jrqZcFLcRDEhaFVRC/CoBePLjjOsHaQNJNxg2lTk1SQ0p/hwbv/xJs38SAL+wPWq/4Bv8QK7siyprR5ed97r/mSOBNcmyD4NeKNjo1/mJj8WPs09fnLdH1mdl/LXFHWplJI1Y2JZoKnrG24EaybKUaSWLBOfLxt651TpjSX6Z45y1gvIUcpH3BKDFKH9V4RxQO/U276kT5RpogGitAiEpjQJ2WRlqUTdMtIGp4w7Q8tI9GXRg8Vv7n5R1k7rDeCZuCG/xaEFWi01vbOvy5fhLuyfgcR9EEChRwSYJCCQSyAgMbnAEIIIEOuBwVyChF3dQYl1NCbo4qhgiB7jN8jXB1UbIprm6mdm+JfBL4KnT4sokeiTiG2f/NdPXfJlv1XduEy7d7OcI6rrARZAz+R/Z/vRflen+3FwAA2XA8ce8ocY7ujVUruTsXu3H/VlcGEDDmL+1hXiKlzvpyz7zza9W7Plrj6H6e0rF3TSpvDvd0lXnA4fJ1vwf5KM1xtrnwPG60teB6TMA8LsIT3uQ4t2IFdaGP2FaY+wKN36V17N97ts9QbqTxz8Nfwfj8BUHS4ig==</latexit>
W =
q
n X(1)
⌦ X(2)
⌦ . . . ⌦ X(k)
+ Z
<latexit sha1_base64="qJQ/YjrsM4mDdQhMZTDaBmsbUZc=">AAADNHicjVHdShwxGP12qtZu/Vnbm0Jvhi6CVlhmtojeFJb2ppcWXHepo5LJZjVsZjImGUGGPEYvfBHfpJRC70qhvekL9KZf4ojapdQMM3NyvnNO8iVpIbg2UfSlETyYmZ17OP+o+XhhcWm5tfJkT8tSUdanUkg1TIlmguesb7gRbFgoRrJUsEE6eevqgzOmNJf5rjkv2EFGjnM+5pQYpI5askrScTiwr8NEnypTJWNFaJUITBgRW+XWesHQHlZr8bpNpOEZ0+EN2b0hEzGSRk9LJut2w88+2OZRqx11Ij/CaRDXoN3b3P347OVFvCNbnyGBEUigUEIGDHIwiAUQ0PjsQwwRFMgdQIWcQsR9nYGFJnpLVDFUEGQn+D3G2X7N5jh3mdq7Ka4i8FXoDGEVPRJ1CrFbLfT10ic79l/Zlc90ezvHf1pnZcgaOEH2f75r5X19rhcDY9j2PXDsqfCM647WKaU/Fbfz8FZXBhMK5BweYV0hpt55fc6h92jfuztb4us/vNKxbk5rbQk/3S7xguO/r3Ma7HU78atO933c7r2BqzEPz+EFrOF9bkEP3sEO9DH7E/xuzDRmg8vga/At+H4lDRq15yncGcGvP/C5vQM=</latexit>
[Korada Macris 09], [Deshpande Abbe Montanari 16],
[B. Macris Dia Lesieur Krzakala Zdeborova 16], [Miolane Lelarge 17], [Miolane 17],
[B. Macris 17], [B. Miolane Macris 18], [El Alaoui Krzakala 18], [Mourrat 19], [B. Luneau Macris 19]
High-dimensional inference:
what we DO know (a statistical physics perspective)
lim
n!1
1
n
I(X; W) = inf
q 0
i(pot)
(q, ) lim
n!1
1
n
EkX ⌦ X E[X ⌦ X|W]k2
= mmse(q⇤
, )
<latexit sha1_base64="pIKHaSk8Z7lgVJpQc5YuEfzJci8=">AAADpnicjVHbjtMwEJ1suCzLrcAjLxYVUougShYQSGilFQgBL2iR6LZS3RbHdbvWxkmaOIuq1N+JeOQN/oKx8XIrCBwlGZ8553jGkxSprHQUfQy2wjNnz53fvrBz8dLlK1db164fVnldctHneZqXw4RVIpWZ6GupUzEsSsFUkopBcvzM5gcnoqxknr3Vq0KMFVtkci450whNW59oKtW0yajOqczmemUInZeMxxl51WloMidD88T9B6a7ZynTZkkXgkSGyEmDlFKRItemazrLuzTFk2esS+hyWbMZ2TB33k1smsxQxfRRkjTPDV0TfxLNtVSi8rt7PyijPxLWvrAxXU9291wpSlUCK5nc+V7LtNWOepFbZDOIfdAGvw7y1gegMIMcONSgQEAGGuMUGFT4jCCGCArExtAgVmIkXV6AgR3U1sgSyGCIHuN3gbuRRzPcW8/KqTmekuJbopLAbdTkyCsxtqcRl6+ds0X/5t04T1vbCv+J91KIajhC9F+6U+b/6mwvGubw2PUgsafCIbY77l1qdyu2cvJTVxodCsRsPMN8iTF3ytN7Jk5Tud7t3TKX/+yYFrV77rk1fLFV4oDj38e5GRzu9uL7vYdvHrT3n/pRb8NNuAUdnOcj2IeXcAB94MGLQAUnwfuwE74O++HgG3Ur8Job8MsK330FrM3jAA==</latexit>
W =
q
n X ⌦ X + Z
<latexit sha1_base64="9IeZQWNKlVBlkBsQEQH4TzFjOn8=">AAADBXicjVHNThRBGCxGEFxEVr2YeJmwITGQbGYwBC4mG7x4xIRlNzKE9PT2Ymfnz+4eEjKZs0/ggRfw5o1w9QW8qvEN5C34uhkSgRDpyUxX11dV0193XCRSmyD4M+U9mJ55ODv3qDX/eOHJYvvps12dl4qLPs+TXA1jpkUiM9E30iRiWCjB0jgRg3jy1tYHR0JpmWc75rgQ+yk7zORYcmaIOmj3qige+4P6jR/pT8pU0VgxXkUJJYxYXWV17QTDOsqNTIX2m+Wqmz/UrYN2J+gGbvi3QdiATm9958uLlZNwO2//RoQRcnCUSCGQwRBOwKDp2UOIAAVx+6iIU4SkqwvUaJG3JJUgBSN2Qt9DWu01bEZrm6mdm9NfEnoVOX0skycnnSJs/+a7eumSLXtXduUy7d6OaY6brJRYg4/E/s93pbyvz/ZiMMam60FST4VjbHe8SSndqdid+/90ZSihIM7iEdUVYe6cV+fsO492vduzZa7+1ykta9e80ZY4t7ukCw5vXudtsLvWDV9319+Hnd4WLsccXmIJr+g+N9DDO2yjT9lf8QM/8cv77H3zTr2zS6k31Xie49rwvl8A8iKsOw==</latexit>
2 groups SBM
Groups asymmetry
HardImpossible Easy
3. W =
q
n X + Z
<latexit sha1_base64="6uMoEBvpE9gq6pVu6P9pW95FBUY=">AAADB3icjVHLbtQwFD1NeZThNZQNEpuIERICaZQUIdggjdpNl1Op0xnRVJXj8bRWnQe2gzSK8gH9gC74A3bdoW77A922/AH8BdduKgEVAkeJj88958TXTksljY2ibwvB4o2bt24v3encvXf/wcPuo+UtU1SaixEvVKEnKTNCyVyMrLRKTEotWJYqMU4P1lx9/EloI4t8085LsZOxvVzOJGeWqN3uap2ks3DcvA8T81HbOplpxutEUcKUNXXeNElaqKmZZzTVyXBfNt4xaV75+UPT2e32on7kR3gdxC3oDd5sHj15+TkeFt1zJJiiAEeFDAI5LGEFBkPPNmJEKInbQU2cJiR9XaBBh7wVqQQpGLEH9N2j1XbL5rR2mca7Of1F0avJGeI5eQrSacLub6GvVz7ZsX/Lrn2m29uc5rTNyoi12Cf2X74r5f/6XC8WM7zzPUjqqfSM6463KZU/Fbfz8JeuLCWUxDk8pbomzL3z6pxD7zG+d3e2zNe/e6Vj3Zq32go/3C7pguM/r/M62Frpx6/7Kxtxb7CKy7GEp3iGF3SfbzHAOoYYUfYXnOEcF8FhcBx8DU4upcFC63mM30Zw+hPTKa2z</latexit>
Random linear estimation
Generalized linear models
[Reeves Pfister 16], [B. Macris Dia Krzakala 16]
[B. Miolane Macris Krzakala Zdeborovà 18]
W = '
⇣q
1
n X
⌘
<latexit sha1_base64="zK/PJNSEGcUY313MhuC/grI8fmI=">AAADCnicjVHNTtwwGBwCLXT7ty299RJ1VYleVgk90AvSiqpSj1upy65EEHK83l2L/NV2kFZR3qAnxFtw663qtS/ACRVegL5FP5sgURBqHSUezzcz8WfHRSK1CYJfC97i0r37yysPWg8fPX7ytP3s+bbOS8XFgOdJrkYx0yKRmRgYaRIxKpRgaZyIYbz/3taHB0JpmWefzbwQuymbZnIiOTNE7bU/VFE88Yf1ph8dMFXMZLQlp2uR/qJMFU0U41VYV1ldR3GejPU8pamK+jNZO9+otvI3rb12J+gGbvi3QdiATm/j6HC+evGin7dPEWGMHBwlUghkMIQTMGh6dhAiQEHcLiriFCHp6gI1WuQtSSVIwYjdp++UVjsNm9HaZmrn5vSXhF5FTh+vyZOTThG2f/NdvXTJlr0ru3KZdm9zmuMmKyXWYEbsv3xXyv/12V4MJnjnepDUU+EY2x1vUkp3Knbn/rWuDCUUxFk8proizJ3z6px959Gud3u2zNUvnNKyds0bbYnfdpd0weHN67wNtte74dvu+qew09vC5VjBS7zCGt3nBnr4iD4GlH2ME5zh3PvqffO+ez8upd5C41nFX8P7+Qfxjq9S</latexit>
W = sign( X)<latexit sha1_base64="WX5GPt3XTil6001t4w8zQyKp+Ms=">AAAC8XicjVHLSsQwFD3W93vUpZviICjI0FFBN4LoxuUIjjNgRdpOZgz2RZIKQ+lPuHMnbv0Bt/oT4h/oX3gTK/hANKXNybnnnOYmfhpyqRznecAaHBoeGR0bn5icmp6ZrczNH8skEwFrBkmYiLbvSRbymDUVVyFrp4J5kR+yln+xr+utSyYkT+Ij1U/ZaeT1Yt7lgaeIOqus5a7ftVvFTu6KyJa8Fxcrrp+EHdmPaMrdxjkvjKZdrJ5Vqk7NMcP+CeolqKIcjaTyBBcdJAiQIQJDDEU4hAdJzwnqcJASd4qcOEGImzpDgQnyZqRipPCIvaBvj1YnJRvTWmdK4w7oLyG9gpw2lsmTkE4Q1n+zTT0zyZr9LTs3mXpvfZr9MisiVuGc2L98H8r/+nQvCl1smx449ZQaRncXlCmZORW9c/tTV4oSUuI07lBdEA6M8+OcbeORpnd9tp6pvxilZvU6KLUZXvUu6YLr36/zJzher9U3auuHm9XdvfKqx7CIJazQfW5hFwdooEnZV7jHAx4taV1bN9btu9QaKD0L+DKsuzd5SqGp</latexit>
X
±1<latexit sha1_base64="nmXQk1ldbUwYZLkkbnpMKZHj/ZE=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVZIq6LLoRlxVsLXQFknSaR2axzCZKKW48Qfc6peJf6B/4Z0xBbWITkhy5tx7zsy91xchT5XjvBasufmFxaXicmlldW19o7y51UqTTAasGSRhItu+l7KQx6ypuApZW0jmRX7IrvzRqY5f3TKZ8iS+VGPBepE3jPmAB54iqtkVke1elytO1THLngVuDirIVyMpv6CLPhIEyBCBIYYiHMJDSk8HLhwI4nqYECcJcRNnuEeJtBllMcrwiB3Rd0i7Ts7GtNeeqVEHdEpIrySljT3SJJQnCevTbBPPjLNmf/OeGE99tzH9/dwrIlbhhti/dNPM/+p0LQoDHJsaONUkDKOrC3KXzHRF39z+UpUiB0Gcxn2KS8KBUU77bBtNamrXvfVM/M1kalbvgzw3w7u+JQ3Y/TnOWdCqVd2Dau3isFI/yUddxA52sU/zPEIdZ2igSd4cj3jCs3VuCevOGn+mWoVcs41vy3r4AJIdkLs=</latexit>
sign(·)<latexit sha1_base64="ychA8oOiRO4WejO0Ftjv52RsrFk=">AAAC1nicjVHLSsNAFD2Nr/quunQTLIJuSqqCLotuXFawD2hLSaZjHZoXk4lSiu7ErT/gVj9J/AP9C++MKahFdEKSM+fec2buvV7si0Q5zmvOmpqemZ3Lzy8sLi2vrBbW1utJlErGayzyI9n03IT7IuQ1JZTPm7HkbuD5vOENTnS8ccVlIqLwXA1j3gncfiguBHMVUd3C2qgtAzsR/fBmp816kdrtFopOyTHLngTlDBSRrWpUeEEbPURgSBGAI4Qi7MNFQk8LZTiIietgRJwkJEyc4wYLpE0pi1OGS+yAvn3atTI2pL32TIya0Sk+vZKUNrZJE1GeJKxPs008Nc6a/c17ZDz13Yb09zKvgFiFS2L/0o0z/6vTtShc4MjUIKim2DC6Opa5pKYr+ub2l6oUOcTEadyjuCTMjHLcZ9toElO77q1r4m8mU7N6z7LcFO/6ljTg8s9xToL6Xqm8X9o7OyhWjrNR57GJLezQPA9RwSmqqJH3NR7xhGerad1ad9b9Z6qVyzQb+Lashw+MtJY1</latexit>
{ µ, Yµ}data<latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit>
Perceptron NN
Sampling rate = # data points / # parameters to learn, infer
4. 1-bit compressed sensing (signal processing)
Perceptron (statistics/ML)
Y = sign( X + Z
p
)
Compressed sensing (signal processing)
CDMA (multi-user communication theory)
Superposition codes (coding theory)
Y = X + Z
p
Phase retrieval (signal processing)Y = | X|
EstimationLearning
Rectified Linear Unit (ReLU) (ML/neural nets)Y = max(0, X)<latexit sha1_base64="Ru6lXUxXyCnS3Ry1iRRkoY4QEPY=">AAAC93icjVHLSsQwFD3W93vUpZviICjI0FFBXQiiG5cjODpiRdpORsP0RZqKY+mHuHMlbv0Ct/oF4h/oX3gTO+AD0ZQ2J+eec5qbuLHPE2lZLz1Gb1//wODQ8Mjo2PjEZGlq+iCJUuGxuhf5kWi4TsJ8HrK65NJnjVgwJ3B9dui2d1T98IKJhEfhvuzE7CRwzkLe4p4jiTotrdiSXUq3lR3lm5ktAjNwLvMFa8l2I7+ZdAKaMrt2zvOurpEvnpbKVsXSw/wJqgUooxi1qPQMG01E8JAiAEMISdiHg4SeY1RhISbuBBlxghDXdYYcI+RNScVI4RDbpu8ZrY4LNqS1yky026O/+PQKcpqYJ09EOkFY/c3U9VQnK/a37Exnqr11aHaLrIBYiXNi//J1lf/1qV4kWljXPXDqKdaM6s4rUlJ9Kmrn5qeuJCXExCncpLog7Gln95xN7Ul07+psHV1/1UrFqrVXaFO8qV3SBVe/X+dPUF+ubFSsvdXy1nZx00OYxRwW6DrXsIVd1FCn6Gs84BFPxpVxY9wadx9So6fwzODLMO7fARXQpPs=</latexit><latexit sha1_base64="Ru6lXUxXyCnS3Ry1iRRkoY4QEPY=">AAAC93icjVHLSsQwFD3W93vUpZviICjI0FFBXQiiG5cjODpiRdpORsP0RZqKY+mHuHMlbv0Ct/oF4h/oX3gTO+AD0ZQ2J+eec5qbuLHPE2lZLz1Gb1//wODQ8Mjo2PjEZGlq+iCJUuGxuhf5kWi4TsJ8HrK65NJnjVgwJ3B9dui2d1T98IKJhEfhvuzE7CRwzkLe4p4jiTotrdiSXUq3lR3lm5ktAjNwLvMFa8l2I7+ZdAKaMrt2zvOurpEvnpbKVsXSw/wJqgUooxi1qPQMG01E8JAiAEMISdiHg4SeY1RhISbuBBlxghDXdYYcI+RNScVI4RDbpu8ZrY4LNqS1yky026O/+PQKcpqYJ09EOkFY/c3U9VQnK/a37Exnqr11aHaLrIBYiXNi//J1lf/1qV4kWljXPXDqKdaM6s4rUlJ9Kmrn5qeuJCXExCncpLog7Gln95xN7Ul07+psHV1/1UrFqrVXaFO8qV3SBVe/X+dPUF+ubFSsvdXy1nZx00OYxRwW6DrXsIVd1FCn6Gs84BFPxpVxY9wadx9So6fwzODLMO7fARXQpPs=</latexit><latexit sha1_base64="Ru6lXUxXyCnS3Ry1iRRkoY4QEPY=">AAAC93icjVHLSsQwFD3W93vUpZviICjI0FFBXQiiG5cjODpiRdpORsP0RZqKY+mHuHMlbv0Ct/oF4h/oX3gTO+AD0ZQ2J+eec5qbuLHPE2lZLz1Gb1//wODQ8Mjo2PjEZGlq+iCJUuGxuhf5kWi4TsJ8HrK65NJnjVgwJ3B9dui2d1T98IKJhEfhvuzE7CRwzkLe4p4jiTotrdiSXUq3lR3lm5ktAjNwLvMFa8l2I7+ZdAKaMrt2zvOurpEvnpbKVsXSw/wJqgUooxi1qPQMG01E8JAiAEMISdiHg4SeY1RhISbuBBlxghDXdYYcI+RNScVI4RDbpu8ZrY4LNqS1yky026O/+PQKcpqYJ09EOkFY/c3U9VQnK/a37Exnqr11aHaLrIBYiXNi//J1lf/1qV4kWljXPXDqKdaM6s4rUlJ9Kmrn5qeuJCXExCncpLog7Gln95xN7Ul07+psHV1/1UrFqrVXaFO8qV3SBVe/X+dPUF+ubFSsvdXy1nZx00OYxRwW6DrXsIVd1FCn6Gs84BFPxpVxY9wadx9So6fwzODLMO7fARXQpPs=</latexit>
Sigmoid/Logistic regression (ML/neural nets)P(Yµ = 1) =
1
1 + exp( µ · x)X<latexit sha1_base64="ZHW7OZbi5udCyj0kx2zC6hkQFRA=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkVwVVIR1F3RjTsrGFu0RZLptA5Nk5BMxBK69Qvc6m+Jf6B/4Z1xCmoRnZDkzLn3nJl7rx8HIpWO81qwZmbn5heKi6Wl5ZXVtfL6xmUaZQnjLouCKGn5XsoDEXJXChnwVpxwb+gHvOkPTlS8eceTVEThhRzFvDP0+qHoCeZJoq7akt9Lv5e3xjflilN19LKnQc2ACsxqROUXtNFFBIYMQ3CEkIQDeEjpuUYNDmLiOsiJSwgJHecYo0TajLI4ZXjEDujbp921YUPaK89UqxmdEtCbkNLGDmkiyksIq9NsHc+0s2J/8861p7rbiP6+8RoSK3FL7F+6SeZ/daoWiR4OdQ2Caoo1o6pjxiXTXVE3t79UJckhJk7hLsUTwkwrJ322tSbVtaveejr+pjMVq/bM5GZ4V7ekAdd+jnMauHvVo6pzvl+pH5tJF7GFbezSOA9QxykacMk6xCOe8GydWZmVW+PPVKtgNJv4tqyHDyPvk2s=</latexit><latexit sha1_base64="ZHW7OZbi5udCyj0kx2zC6hkQFRA=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkVwVVIR1F3RjTsrGFu0RZLptA5Nk5BMxBK69Qvc6m+Jf6B/4Z1xCmoRnZDkzLn3nJl7rx8HIpWO81qwZmbn5heKi6Wl5ZXVtfL6xmUaZQnjLouCKGn5XsoDEXJXChnwVpxwb+gHvOkPTlS8eceTVEThhRzFvDP0+qHoCeZJoq7akt9Lv5e3xjflilN19LKnQc2ACsxqROUXtNFFBIYMQ3CEkIQDeEjpuUYNDmLiOsiJSwgJHecYo0TajLI4ZXjEDujbp921YUPaK89UqxmdEtCbkNLGDmkiyksIq9NsHc+0s2J/8861p7rbiP6+8RoSK3FL7F+6SeZ/daoWiR4OdQ2Caoo1o6pjxiXTXVE3t79UJckhJk7hLsUTwkwrJ322tSbVtaveejr+pjMVq/bM5GZ4V7ekAdd+jnMauHvVo6pzvl+pH5tJF7GFbezSOA9QxykacMk6xCOe8GydWZmVW+PPVKtgNJv4tqyHDyPvk2s=</latexit><latexit sha1_base64="ZHW7OZbi5udCyj0kx2zC6hkQFRA=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkVwVVIR1F3RjTsrGFu0RZLptA5Nk5BMxBK69Qvc6m+Jf6B/4Z1xCmoRnZDkzLn3nJl7rx8HIpWO81qwZmbn5heKi6Wl5ZXVtfL6xmUaZQnjLouCKGn5XsoDEXJXChnwVpxwb+gHvOkPTlS8eceTVEThhRzFvDP0+qHoCeZJoq7akt9Lv5e3xjflilN19LKnQc2ACsxqROUXtNFFBIYMQ3CEkIQDeEjpuUYNDmLiOsiJSwgJHecYo0TajLI4ZXjEDujbp921YUPaK89UqxmdEtCbkNLGDmkiyksIq9NsHc+0s2J/8861p7rbiP6+8RoSK3FL7F+6SeZ/daoWiR4OdQ2Caoo1o6pjxiXTXVE3t79UJckhJk7hLsUTwkwrJ322tSbVtaveejr+pjMVq/bM5GZ4V7ekAdd+jnMauHvVo6pzvl+pH5tJF7GFbezSOA9QxykacMk6xCOe8GydWZmVW+PPVKtgNJv4tqyHDyPvk2s=</latexit>
SKA telescope
Super-Resolution
5. [Gabrie Manoel Luneau B. Macris Krzakala Zdeborová 18]
Models of deep neural networks in a pre-trained phase:
[Aubin Maillard B. Macris Krzakala Zdeborová 18]
Models of shallow networks, committee machines:
W = '(1)
⇣r
1
n
(1)
'(2)
⇣r
1
n
(2)
'(3)
⇣
. . . '(L)
⇣r
1
n
(L)
X
⌘
. . .
⌘
<latexit sha1_base64="aKzzv3leqrpf9BVZ4UDbaldzcRo=">AAADs3iclVFdb9MwFL1Z+Bjlq7BHXiwqpO6lSroheEGaxgsPeyjSulZqxuQ4bmvNiYPtTCpRnuA/Iv7B9sJv4NqkqDAhmKPEx+eec+Jrp6UUxkbRt2ArvHX7zt3te537Dx4+etx98vTEqEozPmZKKj1NqeFSFHxshZV8WmpO81TySXr+1tUnF1wboYpjuyr5aU4XhZgLRi1SZ93vdZLOyaR5Q0hyQXW5FB/qfrzbJIdi0U/MR23rZK4pq+OmLpomSZXMzCrHqU5GS9G06l/W4Y2sw03r3toqM2XNxnaObpSJat/TtCHOtbuOc7hz1u1Fg8gPch3ELehBO0aq+xUSyEABgwpy4FCARSyBgsFnBjFEUCJ3CjVyGpHwdQ4NdNBboYqjgiJ7jt8FrmYtW+DaZRrvZvgXia9GJ4EX6FGo04jd34ivVz7ZsX/Lrn2m29sK57TNypG1sET2X7618n99rhcLc3jtexDYU+kZ1x1rUyp/Km7nZKMriwklcg5nWNeImXeuz5l4j/G9u7Olvn7plY51a9ZqK7hyu8QLjv+8zuvgZDiI9wbD9/u9g8P2qrfhGTyHPt7nKziAdzCCMbDgOPgUfA6+hC/DWZiG2U/pVtB6duC3EeY/AHNx6N4=</latexit>
6. [Aubin Loureiro Maillard Krzakala Zdeborová 19]
High-dimensional inference with structured/generative priors:
with V ⇠ Pout
⇣
·
r
1
n
WZ
⌘
<latexit sha1_base64="abRLwfGt9aoXKbn/T4RJA6epZB4=">AAADI3icjVFNb9QwEH0NX2X5WuDIxWKFVKRqlbRIcKzKpcdFYncr6rJKvN7FahIH20Fapfk5/SfcekNIiAN34Ao/gLFJJaBC4CjxmzfzXjyerMqVdXH8cS26cPHS5SvrV3vXrt+4eat/+87E6toIORY612Y/S63MVSnHTrlc7ldGpkWWy2l29NTnp2+ksUqXz92qkodFuizVQonUETXrv2x4tmCTlltVMDaaNdwUTNeu5btqucHFXDvGN31wzDcZt6+Na/jCpKJJ2qZsW57pfG5XBW3BadqG7UXQP+zN+oN4GIfFzoOkAwN0a6T7H8Axh4ZAjQISJRzhHCksPQdIEKMi7hANcYaQCnmJFj3S1lQlqSIl9oi+S4oOOrak2HvaoBb0l5xeQ0qGB6TRVGcI+7+xkK+Ds2f/5t0ET3+2Fe1Z51UQ6/CK2H/pzir/V+d7cVjgSehBUU9VYHx3onOpw634k7NfunLkUBHn8ZzyhrAIyrN7ZkFjQ+/+btOQ/xwqPetj0dXW+OJPSQNO/hzneTDZGibbw61njwY7u92o13EP97FB83yMHexhhDF5v8VXfMP36CQ6jd5F73+WRmud5i5+W9GnH2HatYU=</latexit>
Y ⇠ P
⇣
·
r
n
U ⌦ V
⌘
<latexit sha1_base64="xXdOqv1gr7Nyh6KlCP5HUAlr70o=">AAADGXicjVFNT9wwEB1CgS1QWOiRi9UVEkholeVD9Ijg0uMi7S5UGCHH691aJHGwnUqrkF/CP+HGDXHlgHrrse2vYGyCBEUIHCV582bes8cTZbE0Ngx/jQXjHyYmp2ofp2dmP83N1xcWe0blmosuV7HShxEzIpap6FppY3GYacGSKBYH0emeyx/8FNpIlXbsKBPHCRumciA5s0id1DsFjQbke0mNTEib7srhCuV9ZQldc8E5XSPUnGlb0IFmvKAxWvdZWaRl6ZXdkiorE2F81CudaPWk3giboV/kJWhVoAHVaqv6HVDogwIOOSQgIAWLOAYGBp8jaEEIGXLHUCCnEUmfF1DCNGpzrBJYwZA9xe8Qo6OKTTF2nsarOe4S46tRSWAZNQrrNGK3G/H53Ds79jXvwnu6s43wH1VeCbIWfiD7lu6x8r0614uFAXz1PUjsKfOM645XLrm/FXdy8qQriw4Zcg73Ma8Rc698vGfiNcb37u6W+fwfX+lYF/OqNoe/7pQ44Nb/43wJeuvN1kZza3+zsbNbjboGS/AFVnCe27AD36ANXfS+hN/o+y+4CK6C6+DmoTQYqzSf4dkKbu8Bt4Ox0w==</latexit>
[Tramel Manoel Caltagirone Gabrie Krzakala 16]
7. High-dimensional inference:
what we DON’T know (yet)
• Non Bayes optimal setting
• Structure in the data
• New statistical regimes
• Zooming on the corners of the phase diagrams
( , P0) 6= (˜, ˜P0)<latexit sha1_base64="iutlYq+xxQQXk4biHbNfleAcIeQ=">AAAC9XicjVHLSsNAFD3GV62vqks3g0WoICWtgi6LblxWsFWwUpJ01MFpEpOJUoq/4c6duPUH3OoviH+gf+GdcRQfiE5Icu+555yZO9ePpUiV6z4NOINDwyOjubH8+MTk1HRhZraZRlkS8EYQySjZ872USxHyhhJK8r044V7Xl3zXP9nU9d0znqQiCndUL+YHXe8oFIci8BRB7YJbaklid7xlVm+7S6wV8lNWaikhO5x9lGyuGe1C0S27ZrGfQcUGRdhVjwqPaKGDCAEydMERQlEs4SGlZx8VuIgJO0CfsIQiYeocF8iTNiMWJ4ZH6Al9jyjbt2hIufZMjTqgXSS9CSkZFkkTES+hWO/GTD0zzhr9zbtvPPXZevT3rVeXUIVjQv/SvTP/q9O9KBxi3fQgqKfYILq7wLpk5lb0ydmnrhQ5xITpuEP1hOLAKN/vmRlNanrXd+uZ+rNhalTngeVmeNGnpAFXvo/zZ9Cslisr5er2arG2YUedwzwWUKJ5rqGGLdTRIO9L3OEeD865c+VcOzdvVGfAaubwZTm3r+5+oH4=</latexit>
[Goldt Mézard Krzakala Zdeborová 19]
8. What about the
CORNERS
of phase diagrams
(in inference problems)?
⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit>
=
1
<latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit>
noise level
sparsity
9. Compressed sensing
↵ =
m
n<latexit sha1_base64="wnw1yuYGkmkdhkUuYnakUAai+cc=">AAAC13icjVHLSsNAFD2Nr/qudekmWARXJVVBN0LRjcsK9iGtyGSctsG8mEzEUoo7cesPuNU/Ev9A/8I7Ywo+EJ2Q5My595yZe68b+16iHOclZ01MTk3P5Gfn5hcWl5YLK8VGEqWSizqP/Ei2XJYI3wtFXXnKF61YCha4vmi6l4c63rwSMvGi8EQNYnEWsF7odT3OFFHnhWKH+XGf7Xe6kvFhMBqGo/NCySk7Ztk/QSUDJWSrFhWe0cEFInCkCCAQQhH2wZDQ00YFDmLizjAkThLyTFxghDnSppQlKIMRe0nfHu3aGRvSXnsmRs3pFJ9eSUobG6SJKE8S1qfZJp4aZ83+5j00nvpuA/q7mVdArEKf2L9048z/6nQtCl3smRo8qik2jK6OZy6p6Yq+uf2pKkUOMXEaX1BcEuZGOe6zbTSJqV33lpn4q8nUrN7zLDfFm74lDbjyfZw/QWOrXNkubx3vlKoH2ajzWMM6Nmmeu6jiCDXUyfsaD3jEk3Vq3Vi31t1HqpXLNKv4sqz7dyqcl0Y=</latexit>
⇢ = ⇥(1)<latexit sha1_base64="Y7PvJGlHAI+BpBFatinfXoOjCBI=">AAAC0XicjVHLSsNAFD2Nr1pfVZdugkWom5JUQTdC0Y3Lin1BWyVJp21oXkwmQimCuPUH3OpPiX+gf+GdMQW1iE5Icubce87MvdeOPDcWhvGa0ebmFxaXssu5ldW19Y385lYjDhPusLoTeiFv2VbMPDdgdeEKj7Uizizf9ljTHp3JePOG8dgNg5oYR6zrW4PA7buOJYi66vBheNKpDZmwiub+db5glAy19FlgpqCAdFXD/As66CGEgwQ+GAIIwh4sxPS0YcJARFwXE+I4IVfFGW6RI21CWYwyLGJH9B3Qrp2yAe2lZ6zUDp3i0ctJqWOPNCHlccLyNF3FE+Us2d+8J8pT3m1Mfzv18okVGBL7l26a+V+drEWgj2NVg0s1RYqR1TmpS6K6Im+uf6lKkENEnMQ9inPCjlJO+6wrTaxql721VPxNZUpW7p00N8G7vCUN2Pw5zlnQKJfMg1L54rBQOU1HncUOdlGkeR6hgnNUUSdvjkc84Vm71MbanXb/maplUs02vi3t4QONupRB</latexit>
⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit>
↵ = ⇥(1)<latexit sha1_base64="wDwxDp8Jtqu98sFt/qBe5OvtuX8=">AAAC1XicjVHLSsNAFD2Nr1pfUZdugkWom5KooBuh6MZlhb6gLWWSTtvQNAnJpFBKd+LWH3CrvyT+gf6Fd8YU1CI6IcmZc+85M/deO/TcWJjma0ZbWl5ZXcuu5zY2t7Z39N29WhwkkcOrTuAFUcNmMfdcn1eFKzzeCCPORrbH6/bwWsbrYx7FbuBXxCTk7RHr+27PdZggqqPrLeaFA3bZqgy4YAXruKPnzaKplrEIrBTkka5yoL+ghS4COEgwAocPQdgDQ0xPExZMhMS1MSUuIuSqOMcMOdImlMUpgxE7pG+fds2U9WkvPWOldugUj96IlAaOSBNQXkRYnmaoeKKcJfub91R5yrtN6G+nXiNiBQbE/qWbZ/5XJ2sR6OFC1eBSTaFiZHVO6pKorsibG1+qEuQQEidxl+IRYUcp5302lCZWtcveMhV/U5mSlXsnzU3wLm9JA7Z+jnMR1E6K1mnx5PYsX7pKR53FAQ5RoHmeo4QblFEl7zEe8YRnra7NtDvt/jNVy6SafXxb2sMHN9OVQw==</latexit>
= ⇥(1)<latexit sha1_base64="2CD7f7ymTgUC3T0VN00wswlLVAc=">AAAC1nicjVHLSsNAFD2Nr1pfrS7dBItQNyWpgm6EohuXFfqCtpRJOm1D0yQkE6UU3Ylbf8CtfpL4B/oX3hlTUIvohCRnzj3nzNwZK3CdSBjGa0pbWFxaXkmvZtbWNza3srnteuTHoc1rtu/6YdNiEXcdj9eEI1zeDELOxpbLG9boXNYbVzyMHN+riknAO2M28Jy+YzNBVDeba7sk7rHTdnXIBSuYB91s3igaaujzwExAHsmo+NkXtNGDDxsxxuDwIAi7YIjoacGEgYC4DqbEhYQcVee4QYa8Mak4KRixI/oOaNZKWI/mMjNSbptWcekNyaljnzw+6ULCcjVd1WOVLNnfsqcqU+5tQn8ryRoTKzAk9i/fTPlfn+xFoI8T1YNDPQWKkd3ZSUqsTkXuXP/SlaCEgDiJe1QPCdvKOTtnXXki1bs8W6bqb0opWTm3E22Md7lLumDz53XOg3qpaB4WS5dH+fJZctVp7GIPBbrPY5RxgQpqlH2NRzzhWWtqt9qddv8p1VKJZwffhvbwATvolag=</latexit>
1<latexit sha1_base64="GM0DyeF3AwEXlxUmliDnfXT5ehQ=">AAACz3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl047IF+4CmyCSdxqFpEpKJUori1h9wq38l/oH+hXfGFNQiOiHJmXPPOTN3xo0DkUrLei0Yc/MLi0vF5dLK6tr6Rnlzq5VGWeLxphcFUdJxWcoDEfKmFDLgnTjhbOQGvO0Oz1S9fc2TVEThhRzHvDdifigGwmOSKMcJSNpnju+b9mW5YlUtPcxZYOeggnzUo/ILHPQRwUOGEThCSMIBGFJ6urBhISauhwlxCSGh6xy3KJE3IxUnBSN2SF+fZt2cDWmuMlPt9miVgN6EnCb2yBORLiGsVjN1PdPJiv0te6Iz1d7G9HfzrBGxElfE/uWbKv/rU71IDHCiexDUU6wZ1Z2Xp2T6VNTOzS9dSUqIiVO4T/WEsKed03M2tSfVvauzZbr+ppWKVXMv12Z4V7ukC7Z/XucsaB1U7cPqQeOoUjvNr7qIHexin+7zGDWco44mZcd4xBOejYZxY9wZ959So5B7tvFtGA8fDjiTrw==</latexit>
Wµ =
r
n
µ · X + Zµ µ = 1, . . . , m
Xi ⇠ Ber(⇢) i = 1, . . . , n<latexit sha1_base64="G+Ucf8j6N1Ci6INlgn7132TkJXc=">AAADUXicjVFNb9QwEJ1k+ShboFu4IHExXVEVUa2SUgkulapy4bhIbHdFXa0cx9u16sSp7SCtovwD/h0HxD+AKzdujE0qPioEjhI/v3lvMuPJKiWtS5JPUdy7dv3GzbVb/fXbd+5uDDbvHVtdGy4mXCttZhmzQslSTJx0SswqI1iRKTHNzl/6+PSdMFbq8o1bVeK0YGelXEjOHFLzwfvpnBb19gG1F8Y1dGEYb6hCf87apmxbmmmV21WBW0PHS9l6OeW5dqSh2YLMWvKUvPUkoRc1y4lHB+kuVSixuwWl/dlcblMri4aaghwJ0+5Qs9RPOr38KcZyhskoCYtcBWkHhtCtsR58BAo5aOBQQwECSnCIFTCw+JxACglUyJ1Cg5xBJENcQAt99NaoEqhgyJ7j9wxPJx1b4tnntMHN8S8KX4NOAo/Ro1FnEPu/kRCvQ2bP/i13E3L62la4Z12uAlkHS2T/5btU/q/P9+JgAS9CDxJ7qgLju+Ndljrciq+c/NKVwwwVch7nGDeIeXBe3jMJHht693fLQvxzUHrWn3mnreGLrxIHnP45zqvgeG+UPhvtvd4fHh51o16Dh7AFOzjP53AIr2AME8z9NXoQPYq24g/xtx704h/SOOo89+G31Vv/Dr2XwcY=</latexit>
sparsity
measurement rate
[Reeves Xu Zadik 19]
11. ↵n =
m
n<latexit sha1_base64="HNyXCSzlEWzm0asCuCcvSGCErl4=">AAAC2nicjVHLSsNAFD3GV62vqLhyEyyCq5KqoBuh6MZlBVsLVspkOtVgXkwmQgnduBO3/oBb/SDxD/QvvDOm4APRCUnOnHvPmbn3ekngp8p1X8as8YnJqenSTHl2bn5h0V5abqVxJrlo8jiIZdtjqQj8SDSVrwLRTqRgoReIU+/qUMdPr4VM/Tg6UYNEnIfsIvL7PmeKqK692mFBcsm6kbPf6UvG83CYR8OuXXGrrlnOT1ArQAXFasT2MzroIQZHhhACERThAAwpPWeowUVC3Dly4iQh38QFhiiTNqMsQRmM2Cv6XtDurGAj2mvP1Kg5nRLQK0npYIM0MeVJwvo0x8Qz46zZ37xz46nvNqC/V3iFxCpcEvuXbpT5X52uRaGPPVODTzUlhtHV8cIlM13RN3c+VaXIISFO4x7FJWFulKM+O0aTmtp1b5mJv5pMzeo9L3IzvOlb0oBr38f5E7S2qrXt6tbxTqV+UIy6hDWsY5PmuYs6jtBAk7xzPOART1bHurFurbuPVGus0Kzgy7Lu3wHjEZhR</latexit>
⇢n<latexit sha1_base64="2XVURmQqOQ2kJDHceYZs4UyFluk=">AAACyXicjVHLTsJAFD3UF+ILdemmkZi4Ii2a6JLoxsQNJvJIgJB2GKBS2jqdGpG48gfc6o8Z/0D/wjtjSVRidJq2Z86958zce93I92JpWa8ZY25+YXEpu5xbWV1b38hvbtXiMBGMV1noh6LhOjH3vYBXpSd93ogEd0auz+vu8FTF6zdcxF4YXMpxxNsjpx94PY85kqhaSwzCTtDJF6yipZc5C+wUFJCuSph/QQtdhGBIMAJHAEnYh4OYniZsWIiIa2NCnCDk6TjHPXKkTSiLU4ZD7JC+fdo1UzagvfKMtZrRKT69gpQm9kgTUp4grE4zdTzRzor9zXuiPdXdxvR3U68RsRIDYv/STTP/q1O1SPRwrGvwqKZIM6o6lrokuivq5uaXqiQ5RMQp3KW4IMy0ctpnU2tiXbvqraPjbzpTsWrP0twE7+qWNGD75zhnQa1UtA+KpYvDQvkkHXUWO9jFPs3zCGWcoYIqeV/hEU94Ns6Na+PWuPtMNTKpZhvflvHwAeFSka0=</latexit>
(informal) Theorem [Reeves Xu Zadik 19]: « All-or-nothing » phenomenon
Weak recovery impossible:
⇢n = o(n 1/2
) and ↵ < ↵IT,n with ↵IT,n =
2⇢n| ln ⇢n|
ln(1 + n)<latexit sha1_base64="lnscvIQZKnJefNiQ5egIenv7Xfo=">AAADWXicjVFdb9MwFL1p+CjlY4U98mJRgVpgXVKQ4GGTJniBtyGt26RlRI7rrtYcJ3IcYMryL/hvCPEH4C/wxLXrIaBC4CjJueeec+3rm5VSVCaKPged8NLlK1e713rXb9y8tda/fWe/KmrN+JQVstCHGa24FIpPjTCSH5aa0zyT/CA7fWnzB++4rkSh9sxZyY9zeqLEXDBqkEr7Hx8kelGkimyTYqjeNhvx5qQdkYQkhn8wDVWz1gZUlgu65f9p0yQ6J6/32seqJT+174VZLMNV1TZJ5pqyZrLc7TyRinjYNhgM40eJxFPPaKpGbdofROPILbIKYg8G4Ndu0f8ECcygAAY15MBBgUEsgUKFzxHEEEGJ3DE0yGlEwuU5tNBDb40qjgqK7Cl+TzA68qzC2NasnJvhLhJfjU4C99FToE4jtrsRl69dZcv+rXbjatqzneE/87VyZA0skP2X70L5vz7bi4E5PHc9COypdIztjvkqtbsVe3LyS1cGK5TIWTzDvEbMnPPinonzVK53e7fU5b86pWVtzLy2hm/2lDjg+M9xroL9yTh+Mp68eTrYeeFH3YW7cA+GOM9nsAOvYBemWPt7QIJR8LDzJQzCbthbSjuB96zDbytc/wFqPsKC</latexit>
lim inf
n!1
1
⇢nn
EkX ˆXoptk2
= 1
<latexit sha1_base64="uqseclRG6+r42WOepxt/Hs2XdP8=">AAADG3icjVFNTxUxFL0MqAh+PHDppuGFxI0vM6DRDQnBmLDExPd4CcVJp3R4DTPtpNMheRnmp/BP3Lkzbg1hywqiP4LbUhKVGO1kZk7PPee0t82qQtY2js9notm5e/cfzD9cWHz0+MnT3tLyqNaN4WLIdaHNOGO1KKQSQyttIcaVEazMCrGbHb1z9d1jYWqp1Uc7rcR+yQ6VzCVnFqm0N6KFLKXK01ZRqykiO+0IzQ3jbdK11Ex0qlRHS2YnWda+7+gJaWmWk3H3kk6YDThFZUl0ZbH+aW0jSXv9eBD7Qe6CJIA+hLGje2dA4QA0cGigBAEKLOICGNT47EECMVTI7UOLnEEkfV1ABwvobVAlUMGQPcLvIc72Aqtw7jJr7+a4SoGvQSeBVfRo1BnEbjXi641Pduzfsluf6fY2xX8WskpkLUyQ/ZfvVvm/PteLhRze+h4k9lR5xnXHQ0rjT8XtnPzSlcWECjmHD7BuEHPvvD1n4j21792dLfP1S690rJvzoG3gyu0SLzj58zrvgtHaIFkfvP7wqr+5Fa56Hp7DCrzA+3wDm7ANOzDE7M9wAT/gZ3QafYm+Rt9upNFM8DyD30b0/RpLkLPV</latexit>
Strong recovery possible: lim sup
n!1
1
⇢nn
EkX ˆXoptk2
= 0
<latexit sha1_base64="c96bpEitY67uc2VlQIQbGInOYrc=">AAADG3icjVHPaxQxGP061VrXatf26CW4CL10mVmFeiksiuCxgvsDmnbIpJlu6EwyZDKFZZw/xf/EmzfxKsWrJ0X/CL/EFNQiNsPMvLzvvZd8SVYVsrZx/HklWr1xc+3W+u3enY279zb797emtW4MFxOuC23mGatFIZWYWGkLMa+MYGVWiFl29tzVZ+fC1FKr13ZZiaOSnSqZS84sUml/SgtZ1k2VtopaTaXK7bIjNDeMt0nXUrPQqVIdLZldZFn7oqNvSEuznMy7XbpgNuAUlSXRlcX68Wg/TvuDeBj7Qa6CJIABhHGg+xdA4QQ0cGigBAEKLOICGNT4HEICMVTIHUGLnEEkfV1ABz30NqgSqGDInuH3FGeHgVU4d5m1d3NcpcDXoJPAI/Ro1BnEbjXi641Pduy/sluf6fa2xH8WskpkLSyQ/Z/vUnldn+vFQg5PfQ8Se6o847rjIaXxp+J2Tn7rymJChZzDJ1g3iLl3Xp4z8Z7a9+7Olvn6V690rJvzoG3gm9slXnDy93VeBdPRMHk8HL16Mhg/C1e9Dg/gIezgfe7BGF7CAUww+x18ge/wI3obvY8+RB9/SaOV4NmGP0b06SeQyrPs</latexit>
⇢n = o(1) and ↵ > ↵IT,n<latexit sha1_base64="Ln0kLNdmUwdFBu8hKQqvctdC95s=">AAAC/nicjVFdS9xAFD2mH3601rV97Mvg0mKhLIktqA+K2Jf6prCrC0aWSXZ0g0kmTCaihAX/Sd/6Vnz1D/hqH0v/gf4L70xnQbsUOyGZc8+952Tu3KhIk1L7/u8J78nTZ88np6ZnXrycfTXXmH+9W8pKxaITy1SqbsRLkSa56OhEp6JbKMGzKBV70fEXk987EapMZN7WZ4U4yPhRnhwmMddE9Rqr70M1kL2crTG5GHxgIQu1ONU1z/tDE/C0GPB1t/fqOlQZ22oPP+bDXqPpt3y72DgIHGjCrW3Z+IUQfUjEqJBBIIcmnIKjpGcfAXwUxB2gJk4RSmxeYIgZ0lZUJaiCE3tM3yOK9h2bU2w8S6uO6S8pvYqUDO9II6lOETZ/YzZfWWfD/su7tp7mbGe0R84rI1ZjQOxjulHl/+pMLxqHWLE9JNRTYRnTXexcKnsr5uTsXleaHAriDO5TXhGOrXJ0z8xqStu7uVtu8ze20rAmjl1thVtzShpw8Pc4x8HuUiv41Fra+dzc2HSjnsJbLGCR5rmMDXzFNjrk/Q1XuMZP79z77v3wLv6UehNO8wYPlnd5B7FVpPc=</latexit>
Wµ =
s
n
⇢nn
µ · X + Zµ µ = 1, . . . , ↵nn
Xi ⇠ Ber(⇢n) i = 1, . . . , n n = ⌦(1)<latexit sha1_base64="1I5kPe7Eij+cSnQwga68y7AnmK8=">AAADwHicjVFdb9MwFL1Z+Bjlq4NHXiwqpk5MVTOQxkulqbzwRpHoWjFPleO4rbUkTm0HqUR55hV+HeIfwG/ghWsvE9ANgaMkx+fec66vb1yk0th+/2uwFV67fuPm9q3W7Tt3791v7zw4NqrUXIy5SpWexsyIVOZibKVNxbTQgmVxKibx2UsXn7wX2kiVv7XrQpxmbJHLueTMIjVr/5jMaFbuDqhZaVvRuWa8oinqEzarqPevtEjqvK4rqpdqg0SaxipNzDrDX0VHS1k7Q8oTZUlF4zmZ1k/fOYrQVckSgmgQ7dMU42afsrRYbhYiOaWt6UzuUiMz9NAZGQpdd68ov9eYyl+WOVLnha7sggwIfZ2JBetGe7N2p9/r+0Uug6gBHWjWSLW/AIUEFHAoIQMBOVjEKTAw+JxABH0okDuFCjmNSPq4gBpaqC0xS2AGQ/YMvwvcnTRsjnvnabyaY5UUX41KAk9QozBPI3bViI+X3tmxf/OuvKc72xr/ceOVIWthiey/dBeZ/6tzvViYwwvfg8SeCs+47njjUvpbcScnv3Vl0aFAzuEE4xox98qLeyZeY3zv7m6Zj3/zmY51e97klvDdnRIHHG2O8zI4PuhFz3oHb553jobNqLfhETyGLs7zEI7gFYxgDDxgwcfgU/A5HIbLUIWr89StoNE8hD9W+OEnxOrv/w==</latexit>
Replica prediction at ⇢ ! 0+ agrees with ⇢n ! 0+ for ⇢n = o(n 1/2
)<latexit sha1_base64="52bXL9v9NgK4PAqRjfi5jyXoWfU=">AAADOHicjVFNTxsxEJ0shSbhK6VHLlYjJBAi7KZIcKkUlQtHWjWAxELkdZzEyma9sr2lUZQ/xh9BuXEDrvwBOjYbUYoQeLW7M2/ee/Z4ojQW2vj+pODNfJid+1gslecXFpeWK59WjrTMFONNJmOpTiKqeSwS3jTCxPwkVZwOopgfR/19Wz/+zZUWMvllhik/G9BuIjqCUYNQq6JDw/+Y0U+OmzFKUNsWzJYINWMSklD1JAmNJH5r06aOTbuKc00uhOlNOa3kiVV+ZHWkeqp+k+vJ+Wgr2K6PN1qVql/z3SIvgyAPqpCvQ1m5ghDaIIFBBgPgkIDBOAYKGp9TCMCHFLEzGCGmMBKuzmEMZdRmyOLIoIj28dvF7DRHE8ytp3ZqhrvE+CpUElhDjUSewtjuRlw9c84Wfc175Dzt2Yb4j3KvAaIGeoi+pZsy36uzvRjowJ7rQWBPqUNsdyx3ydyt2JOTf7oy6JAiZuM21hXGzCmn90ycRrve7d1SV791TIvanOXcDO7sKXHAwf/jfBkc1WvB11r9x0618T0fdRFW4Qus4zx3oQEHcAhN9J7AQ6FYKHmX3rV34909Ur1CrvkMz5Z3/xfE5roM</latexit>
12. [Lesieur Krzakala Zdeborová 15]⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit>
=
1
<latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit>
Sparse principal components analysis: high sparsity regime
W =
r
n
X ⌦ X + Z
Xi ⇠ Ber(⇢) i = 1, . . . , n<latexit sha1_base64="1z0nYv4qYQG5iyOMuyNR5/W5QP0=">AAADMnicjVHPTxQxFH47oOL6g1WPXho2EoxkM4MkeCEhePEICctupIR0Ol1o6EyHtmNCJvNf+Z94MHozXjThH+DGaymJSox2MjPf+977vvb15bWS1qXpl14yN3/n7r2F+/0HDx89Xhw8ebpvdWO4GHOttJnmzAolKzF20ikxrY1gZa7EJD996/OTD8JYqas9d16Lw5IdV3ImOXNIHQ3KluYzMumWN6k9M66lM8N4SxU6FKxrq64LBdOOaidLYUkMX4X/+47S/vRILlMrS0yZkmwL061Qc6JfEnrWsILIzWyVqkI7u4r7DdNRGha5DbIIhhDXjh58BgoFaODQQAkCKnCIFTCw+BxABinUyB1Ci5xBJENeQAd91DZYJbCCIXuK32OMDiJbYew9bVBz3EXha1BJ4AVqNNYZxH43EvJNcPbs37zb4OnPdo7/PHqVyDo4QfZfupvK/9X5XhzM4E3oQWJPdWB8dzy6NOFW/MnJL105dKiR87jAvEHMg/LmnknQ2NC7v1sW8j9CpWd9zGNtAz/9KXHA2Z/jvA3210bZ69Ha7vpwazuOegGewxKs4Dw3YAvewQ6M0fsTXPaS3lzyMfmafEu+X5cmvah5Br+t5OIKpZ65Jg==</latexit>
« Bernoulli spiked model »
14. All-or-nothing in sparse PCA
I-MMSE relation:
d
d
1
n
I(X; W) =
1
4n2
EkX ⌦ X E[X ⌦ X|W]k2
F
<latexit sha1_base64="7sDds8HKzzKXCvjLcngkqhQ1OFE=">AAADYnicjVFdT9RAFL3dosD6wSKP8jBxYwIPbtpCgokxIRoNvEHisptsl810OgsT+pXp1IR056f4u4zvPsi/8M4wBBSNTtP23HPPuTN3blJlolZB8M3r+EsPHi6vrHYfPX7ydK23/uykLhvJ+JCVWSnHCa15Jgo+VEJlfFxJTvMk46Pk4r3Jjz5zWYuy+KQuKz7N6Vkh5oJRhdSs9yWeS8raVLdpnKEtpZpcU6FuC3241cbJnIz1G/sf6W3yltwKdovTCPU5VedJ0n7Q8cLJ41KJnNfEha/uaCZ/lizcBtN4MWtjmZOP+jSa9frBILCL3AehA31w66jsfYUYUiiBQQM5cChAIc6AQo3PBEIIoEJuCi1yEpGweQ4auuhtUMVRQZG9wO8ZRhPHFhibmrV1M9wlw1eik8BL9JSok4jNbsTmG1vZsH+r3dqa5myX+E9crRxZBefI/st3o/xfn+lFwRxe2x4E9lRZxnTHXJXG3oo5ObnTlcIKFXIGp5iXiJl13twzsZ7a9m7ultr8D6s0rImZ0zZwZU6JAw5/H+d9cBINwp1BdLzb33/nRr0Cz+EFbOE892AfDuAIhsA839v2Im+n893v+uv+xrW04znPBvyy/M2fXdTHNg==</latexit>
IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit>
IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit>
⇢ ! 0+ gives IT(⇢)
⇢!0
!
4| ln ⇢|
⇢
[Lesieur Krzakala Zdeborová 15]
15. ⇢n ! 0+<latexit sha1_base64="CsLOCzcFzETb5Ob3d8nGwZTOSr4=">AAAC4nicjVHLSsNAFD2N7/qKuhQhWARBKGkVdCm6calgbaEtJUmnbWiaCZOJUEJX7tyJW3/ArX6M+Af6F94ZU1CL6IQkZ86958zce90o8GNp2685Y2p6ZnZufiG/uLS8smqurV/FPBEeq3g84KLmOjEL/JBVpC8DVosEcwZuwKpu/1TFq9dMxD4PL+UwYs2B0w39ju85kqiWudUQPd5KG9opFaw9SsPRyGpIbtmtvZZZsIu2XtYkKGWggGydc/MFDbTB4SHBAAwhJOEADmJ66ijBRkRcEylxgpCv4wwj5EmbUBajDIfYPn27tKtnbEh75RlrtUenBPQKUlrYIQ2nPEFYnWbpeKKdFfubd6o91d2G9HczrwGxEj1i/9KNM/+rU7VIdHCka/Cppkgzqjovc0l0V9TNrS9VSXKIiFO4TXFB2NPKcZ8trYl17aq3jo6/6UzFqr2X5SZ4V7ekAZd+jnMSXJWLpf1i+eKgcHySjXoem9jGLs3zEMc4wzkq5H2DRzzh2Wgbt8adcf+ZauQyzQa+LePhA38Am5s=</latexit>
n ! +1<latexit sha1_base64="Haf+8ljzRuATXGTIzWyIZOsLN5E=">AAAC6XicjVHLSsNAFD2Nr1pfVZduglUQhJJWQZdFNy4r2FZoSknSqQ5NkzCZCCX0B9y5E7f+gFv9EfEP9C+8M6bgA9EJSc6ce86ZuTNu5PNYWtZLzpianpmdy88XFhaXlleKq2vNOEyExxpe6Ifi3HVi5vOANSSXPjuPBHOGrs9a7uBY1VtXTMQ8DM7kKGKdoXMR8D73HElUt7hl+yTuOd3U1mGpYL1xGozHpi1Dc9fmQV+OusWSVbb0MH+CSgZKyEY9LD7DRg8hPCQYgiGAJOzDQUxPGxVYiIjrICVOEOK6zjBGgbwJqRgpHGIH9L2gWTtjA5qrzFi7PVrFp1eQ08Q2eULSCcJqNVPXE52s2N+yU52p9jaiv5tlDYmVuCT2L99E+V+f6kWij0PdA6eeIs2o7rwsJdGnonZufupKUkJEnMI9qgvCnnZOztnUnlj3rs7W0fVXrVSsmnuZNsGb2iVdcOX7df4EzWq5sleunu6XakfZVeexgU3s0H0eoIYT1NGg7Gs84BFPxsC4MW6Nuw+pkcs86/gyjPt3I/qe8A==</latexit>
⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit>
VERY high sparsity
Linked to planted clique
=
1
<latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit>
Sparse PCA: VERY high sparsity regime
W =
r
n
n
X ⌦ X + Z
Xi ⇠ Ber(⇢n) i = 1, . . . , n<latexit sha1_base64="zDIqNDRQ8LcLOTNSia4xfXI5HVo=">AAADVnicjVFfaxQxEJ+9s7ae/876KELw8KxYjt0q2JdCqS8+VvB6h005stlcG5rdbJOsUJb9EH469RvoJ/BNnMQU1EM0yyYzv5nfLzOZvFbSujT9nPT619aur2/cGNy8dfvO3eG9zSOrG8PFlGulzTxnVihZiamTTol5bQQrcyVm+fkrH5+9F8ZKXb11l7U4KdlpJZeSM4fQYvihpfmSzLrxHrUXxrV0aRhvqUKFgi3aloYrWiOKjlRd17W4Bcq8o9rJUlgS3WfhfNdROpgv5JhaWWLIlORAmG6LmjO9KveU0IuGFUTuZdtUFdrZbaxplE7SsMiqkUVjBHEd6uFHoFCABg4NlCCgAoe2AgYWv2PIIIUasRNoETNoyRAX0MEAuQ1mCcxgiJ7jforecUQr9L2mDWyOtyj8DTIJPEaOxjyDtr+NhHgTlD36N+02aPraLvHMo1aJqIMzRP/Fu8r8X57vxcESdkMPEnuqA+K741GlCa/iKye/dOVQoUbM2wXGDdo8MK/emQSODb37t2Uh/iVketT7POY28NVXiQPO/hznqnG0M8meT3bevBjtH8RRb8ADeARbOM+XsA+v4RCmqP0teZiMkye9T73v/bX++s/UXhI59+G31R/+AJqtxag=</latexit>
[Berthet Rigollet 13]
[Deshpande Montanari 13]
16. Finite size bound for the mutual information
Theorem (B. Macris 19): based on the adaptive interpolation method
Let P0,n = ⇢n p0 + (1 ⇢n) 0. Let n = ⇥(| ln ⇢n|/⇢n) which is the appro-
priate scaling to observe a phase transition, and ⇢n = ⇥(n ) with 2 [0, 1/6)
Then
1
⇢n| ln ⇢n|
1
n
I(X; W) inf
q2[0,1]
ipot
n (q; n, ⇢n) C
(ln n)1/3
n(1 6 )/7
with i(pot)
n (q; n, ⇢n) =
n⇢2
n
4
(q 1)2
+ In(X;
p
n⇢nq X + Z) X ⇠ P0,n, Z ⇠ N(0, 1).
<latexit sha1_base64="fQUbEiIQan28f7ueBu1sSyqjneA=">AAAFDHicjVLbbtNAEN2GACXcWnjkZURLldBc7BRapCpS1UqISggVqZeIbmNtnE2yqrN27E2hcvcX+BPeeEO88gM8ICR4h79gduMUaEHgKPHsmTPnzMymHQUiUY7zeSp3IX/x0uXpK4Wr167fuDkze2s3CUexz3f8MAjjZpslPBCS7yihAt6MYs4G7YDvtQ83TH7viMeJCOW2Oo74wYD1pOgKnymEvNnc46dcwfyWlzplqRs07oeeBFqGyHMWi25lDJRohweKec58FSyfBmjRYZ5sAN3uc8WKJzSQY/JJLSuah5d94fdBJKD6HFgUxWEUC6Y4JD7DjnugQgjbCY+PMA1RHwcBFTOZCNNdGZjsoJdVa2Q+spVWaBsjbfSF6iPBHKmQ+07ZrS0jjExZQLQnZIo+PXlfFxZoN2Z+6uo06/Jnv5qui97JaV7qzWJK211o6lX73tMlqKB+10uH+ALjc6BBtFBqAFGotCeLw9XTnZQnSzOyQAMOG2PxInqCLLVSt7ak0aiV4oaXbful2orWlBao4q9UaubSQIcj1oEF4RniqVdJ/8mrMXY4xcdwq67TB8ivuKVWfXET22yu0mQYq7NEGGq88+bii9LYtEkTMYDsX1EGCi8MkFK8Nnimi7iAUrVAuexM9uvNzDlVxz5wPnCzYI5kz1Y484lQ0iEh8cmIDAgnkiiMA8JIgp994hKHRIgdkBSxGCNh85xoUsDaEbI4Mhiih/jbw9N+hko8G83EVvvoEuA3xkog97AmRF6MsXEDmx9ZZYP+TTu1mqa3Y3y3M60Boor0Ef1X3YT5v3VmFkW65JGdQeBMkUXMdH6mMrJbMZ3DL1MpVIgQM3EH8zHGvq2c7BlsTWJnN7tlNv/NMg1qzn7GHZHvpku8YPfsdZ4PdutVd6n68Hl9bm09u+ppcofcJUW8zxWyRp6QLbJD/Nyb3Mfcl9zX/Ov82/y7/PsxNTeV1dwmvz35Dz8AiglePw==</latexit>
C1n 1/2
< ⇢n < C2<latexit sha1_base64="GYXdMrydX8eJ3XaVm05eFoqGSSU=">AAAC3XicjVHLSsNAFD3G97vqRnATLIIba1IFXbgQu3GpYG2hrSFJRw2mmTCZCFJ0507c+gNu9XfEP9C/8M44BbWITkhy5tx7zsy9N0jjKJOO8zpgDQ4Nj4yOjU9MTk3PzBbm5o8znouQVUMec1EP/IzFUcKqMpIxq6eC+Z0gZrXgoqLitUsmsognR/IqZa2Of5ZEp1HoS6K8wmLFc+3kpLvmrpevd5rinHvJjl3xyl6h6JQcvex+4BpQhFkHvPCCJtrgCJGjA4YEknAMHxk9DbhwkBLXQpc4QSjScYZrTJA2pyxGGT6xF/Q9o13DsAntlWem1SGdEtMrSGljhTSc8gRhdZqt47l2Vuxv3l3tqe52Rf/AeHWIlTgn9i9dL/O/OlWLxCm2dQ0R1ZRqRlUXGpdcd0Xd3P5SlSSHlDiF2xQXhEOt7PXZ1ppM16566+v4m85UrNqHJjfHu7olDdj9Oc5+cFwuuRul8uFmcXfPjHoMS1jGKs1zC7vYxwGq5H2DRzzh2fKsW+vOuv9MtQaMZgHflvXwAQUnl3Q=</latexit>
left open
Replica prediction at ⇢ ! 0+ agrees with ⇢n ! 0+ for ⇢n = o(n 1/2
)<latexit sha1_base64="52bXL9v9NgK4PAqRjfi5jyXoWfU=">AAADOHicjVFNTxsxEJ0shSbhK6VHLlYjJBAi7KZIcKkUlQtHWjWAxELkdZzEyma9sr2lUZQ/xh9BuXEDrvwBOjYbUYoQeLW7M2/ee/Z4ojQW2vj+pODNfJid+1gslecXFpeWK59WjrTMFONNJmOpTiKqeSwS3jTCxPwkVZwOopgfR/19Wz/+zZUWMvllhik/G9BuIjqCUYNQq6JDw/+Y0U+OmzFKUNsWzJYINWMSklD1JAmNJH5r06aOTbuKc00uhOlNOa3kiVV+ZHWkeqp+k+vJ+Wgr2K6PN1qVql/z3SIvgyAPqpCvQ1m5ghDaIIFBBgPgkIDBOAYKGp9TCMCHFLEzGCGmMBKuzmEMZdRmyOLIoIj28dvF7DRHE8ytp3ZqhrvE+CpUElhDjUSewtjuRlw9c84Wfc175Dzt2Yb4j3KvAaIGeoi+pZsy36uzvRjowJ7rQWBPqUNsdyx3ydyt2JOTf7oy6JAiZuM21hXGzCmn90ycRrve7d1SV791TIvanOXcDO7sKXHAwf/jfBkc1WvB11r9x0618T0fdRFW4Qus4zx3oQEHcAhN9J7AQ6FYKHmX3rV34909Ur1CrvkMz5Z3/xfE5roM</latexit>
Compressed sensing [Reeves Xu Zadik 19]:
17. 1
⇢n| ln ⇢n|
inf
q2[0,1]
ipot
n (q; n, ⇢n) !
n!+1
I 1 + I > 1
<latexit sha1_base64="pfzcBo5SrMvAvhK3FxHgaDjx7Rw=">AAADeXicjVHdbtMwGP3S8LONvzLu4MZQIXVsqpINaUhIqIIbxtWQ6DapKZHjuq01x84cB6hC3op3QbwBPAM3fPZSCRgIHCU5Pt85x/7srJCitFH0JeiEly5fubq2vnHt+o2bt7q3N49KXRnGR0xLbU4yWnIpFB9ZYSU/KQyneSb5cXb6wtWP33FTCq3e2GXBJzmdKzETjFqk0u6nZGYoq+OmTsxCp+pjIhVpYZMINUvrM/yRcbQTTxoi3qIuJ4W2Tar6Z08TiUtNaap2zj1bJPlgxHxhqTH6/RijrCbbLscuJ3WznsxpnlOS5NQusqw+aJJMzPsrVnISO2Jr+8+CZ2057faiQeQHuQjiFvSgHYe6+xkSmIIGBhXkwEGBRSyBQonPGGKIoEBuAjVyBpHwdQ4NbKC3QhVHBUX2FL9znI1bVuHcZZbezXAVia9BJ4GH6NGoM4jdasTXK5/s2L9l1z7T7W2J/6zNypG1sED2X76V8n99rhcLM3jiexDYU+EZ1x1rUyp/Km7n5KeuLCYUyDk8xbpBzLxzdc7Ee0rfuztb6utfvdKxbs5abQXf3C7xguPfr/MiONodxHuD3dePe8Pn7VWvwT14AH28z30Ywks4hBGw4G4wDA6CV53v4f2wHz46l3aC1nMHfhnh3g8qRtA8</latexit>
Setting n ⌘ n,IT(⇢n) where n,IT(⇢n) ⌘ 4| ln ⇢n|
⇢n<latexit sha1_base64="RQtxbdebaThUKCNpnc/TcDPahlo=">AAADS3icjVFBT9swGP0SYGOwjQ6OcLDWTmLSVKVl0nZE48JuoFFAIqhyXLe1cJzgOEwo5Lp/xxnxDzZpt92mHfbZdcU2QMxRkuf3vffsz05yKQoTRddBODM79+jx/JOFxafPni81XizvF1mpGe+xTGb6MKEFl0LxnhFG8sNcc5omkh8kJ1u2fnDGdSEytWfOc36c0pESQ8GoQarf+PKJGyPUiLRiia4B7auYn5bijMQjmqaUTOlKvalinZKPe3W9Hutx1levW+TzmGt+471LRHze23ioKasuYqnIpHRRVxNQt/qNZtSO3CC3QceDJvixkzWuIIYBZMCghBQ4KDCIJVAo8DmCDkSQI3cMFXIakXB1DjUsoLdEFUcFRfYEvyOcHXlW4dxmFs7NcBWJr0YngVfoyVCnEdvViKuXLtmy92VXLtPu7Rz/ic9KkTUwRvYh31T5vz7bi4EhvHc9COwpd4ztjvmU0p2K3Tn5oyuDCTlyFg+wrhEz55yeM3GewvVuz5a6+lentKydM68t4ZvdJV5w59/rvA32u+3ORru7221ufvBXPQ+r8BLW8T7fwSZsww70MPt7sBysBmvhZfgj/Bn+mkjDwHtW4K8xM/cbePrB3A==</latexit>
All-or-nothing in sparse PCA
1
(n⇢n)2
EkX ⌦ X E[X ⌦ X|W]k2
F !
n!1
I( 1)
<latexit sha1_base64="sJaHnwbzC0ucbLGuGU+Q1r3AbWo=">AAADZXicjVFdb9MwFL1pYIzBoAPECw9YVEidBFVSxsfjBALB25DoWqnuKsdzWmuJHTkOUGX5L/wrxC8A/gXXXsbXQOAoybnnnnPt65sUmSxtFH0KOuG582sX1i9uXLq8eeVqd+vafqkrw8WI60ybScJKkUklRlbaTEwKI1ieZGKcHD1z+fFbYUqp1Ru7KsQsZwslU8mZRWre/UBTw3gdN3VfUbPUc7V9MGxozuwySernDT2uaZKSSUO1lbkoSRveJz800z9LTpzjZkaP5zU1OXnRHAzpeyMXS8uM0e+milpNpUrtalZ/3/NV06cLlueMZoLE2/NuLxpEfpGzIG5BD9q1p7sfgcIhaOBQQQ4CFFjEGTAo8ZlCDBEUyM2gRs4gkj4voIEN9FaoEqhgyB7hd4HRtGUVxq5m6d0cd8nwNegkcBc9GnUGsduN+HzlKzv2b7VrX9OdbYX/pK2VI2thiey/fKfK//W5Xiyk8MT3ILGnwjOuO95WqfytuJOTn7qyWKFAzuFDzBvE3DtP75l4T+l7d3fLfP6LVzrWxbzVVvDVnRIHHP8+zrNgfziIHwyGr3d6u0/bUa/DLbgDfZznY9iFl7AHI+DBWnAveBg86nwON8Mb4c0TaSdoPdfhlxXe/gYbSspP</latexit>
[The overlap gap property, Gamarnik Jagannath Sen 19]
In agreement with the recent study of the
Maximum Likelihood Estimator for FIXED but small ⇢ ! 0+<latexit sha1_base64="RM3Ida9m8zVRPO2cA0dUwxlRUtA=">AAACznicjVHLSsNAFD2Nr1pfVZdugkUQhJJUQZdFNy4r2FZoS0nSaRuaZMJkUiiluPUH3OpniX+gf+GdMQW1iE5Icubcc+7MvdeNAz+RlvWaM5aWV1bX8uuFjc2t7Z3i7l4j4anwWN3jARd3rpOwwI9YXfoyYHexYE7oBqzpjq5UvDlmIvF5dCsnMeuEziDy+77nSKJabTHkbclNq3vSLZassqWXuQjsDJSQrRovvqCNHjg8pAjBEEESDuAgoacFGxZi4jqYEicI+TrOMEOBvCmpGCkcYkf0HdCulbER7VXORLs9OiWgV5DTxBF5OOkEYXWaqeOpzqzY33JPdU51twn93SxXSKzEkNi/fHPlf32qFok+LnQNPtUUa0ZV52VZUt0VdXPzS1WSMsTEKdyjuCDsaee8z6b2JLp21VtHx9+0UrFq72XaFO/qljRg++c4F0GjUrZPy5Wbs1L1Mht1Hgc4xDHN8xxVXKOGuu74I57wbNSMsTEz7j+lRi7z7OPbMh4+AMI1kys=</latexit>
MLE recovers a constant fraction of the support of X i↵ > C | ln ⇢|
⇢ .<latexit sha1_base64="M8HoApg3dBZlTLYJIWcUjk20uLQ=">AAADKnicjVHBThRBEC0GFUTUFY9eKu6acNrMLgc9GSIh4aAJJi5swhDS09vDduidnnT3kJBl/4g/4caFEK8e9OgVLla1Q6ISoz2ZmVev3qvu6soro31I06u5ZP7e/QcLiw+XHi0/fvK09Wxlx9vaSTWQ1lg3zIVXRpdqEHQwalg5JSa5Ubv50Qbnd4+V89qWn8JJpfYn4rDUhZYiEHXQUh/eb6JT0rIIBUpb+iDKgIUTkiVoCwxjhb6uKusCh50sL3DYQV0wNrTZSOBb3MCMTdPTzJSYubE9nU35N+t0D1rttJvGhXdBrwFtaNa2bV1CBiOwIKGGCSgoIRA2IMDTswc9SKEibh+mxDlCOuYVzGCJvDWpFCkEsUf0PaRor2FLirmmj25Juxh6HTkRXpHHks4R5t0w5utYmdm/1Z7Gmny2E/rnTa0JsQHGxP7Ld6v8Xx/3EqCAN7EHTT1VkeHuZFOljrfCJ8dfugpUoSKO8YjyjrCMztt7xujxsXe+WxHzX6OSWY5lo63hG5+SBtz7c5x3wU6/21vr9j/22+vvmlEvwgt4Cas0z9ewDluwDQOqfQ7f4RpukrPkIrlKPv+UJnON5zn8tpIvPwAPebad</latexit>
= 1<latexit sha1_base64="qFglFRl5V+IA2gC5aE1LqSuQEAw=">AAACy3icjVHLSsNAFD3GV62vqks3wSK4KokKuhGKbtwIFewDbJFJOq2heTGZCLW69Afc6n+Jf6B/4Z1xCmoRnZDkzLnn3Jl7r5eGQSYd53XKmp6ZnZsvLBQXl5ZXVktr640syYXP634SJqLlsYyHQczrMpAhb6WCs8gLedMbnKh484aLLEjiCzlMeSdi/TjoBT6TRLXafRZF7Mi9KpWdiqOXPQlcA8owq5aUXtBGFwl85IjAEUMSDsGQ0XMJFw5S4joYEScIBTrOcY8ieXNScVIwYgf07dPu0rAx7VXOTLt9OiWkV5DTxjZ5EtIJwuo0W8dznVmxv+Ue6ZzqbkP6eyZXRKzENbF/+cbK//pULRI9HOoaAqop1YyqzjdZct0VdXP7S1WSMqTEKdyluCDsa+e4z7b2ZLp21Vum429aqVi19402x7u6JQ3Y/TnOSdDYrbh7ld3z/XL12Iy6gE1sYYfmeYAqTlFDXc/xEU94ts6szLq17j6l1pTxbODbsh4+AAnqkhw=</latexit>
Replica picture valid when ⇢n = ⇥(n ) with 2 [0, 1/6).<latexit sha1_base64="DmgcLFf/+9pNk8Sk0Om4CtZd8xc=">AAADE3icjVFBT9RAFP6oCggCqx69TNg1gUSWFiJwISF48WSAsECyxc10dqATutNmOoWQDT/Df+LNG/Hq2ehVD/gveDOUBCVGp2n7ve+975t585IiU6UNw28jwYOHj0bHxh9PTD6Zmp5pPH22V+aVEbIj8iw3BwkvZaa07FhlM3lQGMkHSSb3k5M3Lr9/Kk2pcr1rzwt5OODHWh0pwS1Rvca7HUnbCM4KJWxlJDvlmeqzs1Rq1opNmvf0erybSsvn9PvhQpwQuphvsTNlUypwYax0N3wVLa7Mt9q9RjNsh36x+yCqQRP12sobXxGjjxwCFQaQ0LCEM3CU9HQRIURB3CGGxBlCyuclLjBB2oqqJFVwYk/oe0xRt2Y1xc6z9GpBu2T0GlIyvCRNTnWGsNuN+XzlnR37N++h93RnO6d/UnsNiLVIif2X7rbyf3WuF4sjrPkeFPVUeMZ1J2qXyt+KOzm705Ulh4I4h/uUN4SFV97eM/Oa0vfu7pb7/JWvdKyLRV1b4Zc7JQ04+nOc98HeUjtabr/eXmpubNajHscLzGKO5rmKDbzFFjrk/RHf8QM/gw/Bp+Ay+HxTGozUmuf4bQVfrgG+qazR</latexit>
19. All-or-nothing in sparse PCA: Wishart case
W =
r
n
n
U ⌦ V + Z
<latexit sha1_base64="QQmmu/mvgT+bHyK0p3O/5TzzQag=">AAADBXicjVHLShxBFD22eaiJycQs3TQZhEBg6MmDZBMYdJOlQmZGtGWorqkxhf1KVXVAml7nT9xlF9zmB7JNxD8wf+GtawuJIqaa7jr33HtO162blKm2LopO54L5O3fv3V9YXHrwcPnR486TlZEtKiPVUBZpYbYTYVWqczV02qVquzRKZEmqxsnBhs+PvyhjdZF/dIel2svEfq5nWgpH1KQzqONkFo6b97H9bFwdz4yQdZySwVRM8qbOm4Yrhk1cOJ0pG3I4al7wvtNMOt2oF/EKr4N+C7po12bROUGMKQpIVMigkMMRTiFg6dlFHxFK4vZQE2cIac4rNFgibUVViioEsQf03adot2Vzir2nZbWkv6T0GlKGWCNNQXWGsP9byPmKnT17k3fNnv5sh7QnrVdGrMMnYm/TXVb+r8734jDDO+5BU08lM7472bpUfCv+5OFfXTlyKInzeEp5Q1iy8vKeQ9ZY7t3freD8GVd61seyra3wx5+SBty/Os7rYPSy13/Ve7P1ujtYb0e9gFU8w3Oa51sM8AGbGJL3EX7iF34HX4Nvwffg+KI0mGs1T/HPCn6cAydXqnI=</latexit>
lim
n!1
1
m2
MMSE(U ⌦ U|W)
<latexit sha1_base64="rQcaoCEbwAEhn6NlOtX6kY10PPs=">AAADDnicjVFNT9VAFD1U+RABn7p0M/HFBDYvLWhkSTQmuiDB6OORUHhphylMXttpplOTl9r/4D9x58649Q8Y40a3+i+8Mw6JQAhM0865595zOnduWuWyNmH4Yya4cXN2bn7h1uLtpeWVO72793Zr1WguhlzlSu+lSS1yWYqhkSYXe5UWSZHmYpROntv86J3QtVTlWzOtxEGRHJcykzwxRI17r+JcFuO2jI1isSwzM+3iTCe8jbq2OFzv2lgXbHv7zYtutY3TjA27WBlZiJr58L3bR93auNcPB6Fb7CKIPOjDrx3V+44YR1DgaFBAoIQhnCNBTc8+IoSoiDtAS5wmJF1eoMMiaRuqElSREDuh7zFF+54tKbaetVNz+ktOryYlwyPSKKrThO3fmMs3ztmyl3m3ztOebUp76r0KYg1OiL1Kd1p5XZ3txSDDputBUk+VY2x33Ls07lbsydl/XRlyqIiz+IjymjB3ytN7Zk5Tu97t3SYu/9tVWtbG3Nc2+GNPSQOOzo/zIthdH0QbgyevH/e3nvlRL+ABHmKV5vkUW3iJHQzJ+yO+4Sd+BR+CT8Hn4Mu/0mDGa+7jzAq+/gXGKq1/</latexit>
lim
n!1
1
(n⇢)2
MMSE(V ⌦ V|W)
<latexit sha1_base64="4egtK9Z9u3TX01Q29gJAntlvm+A=">AAADFHicjVFNb9QwEH0NX6V8dIEjF4sV0vaySgoVHCtQJS4VRbC7leqySlyntZrEkeMgrUL+Rv8JN26IK1eEuIIE/4KxcSWgQuAo8Zs38148nqwuVGPj+NNSdO78hYuXli+vXLl67frq4MbNaaNbI+RE6EKb3SxtZKEqObHKFnK3NjIts0LOsuPHLj97JU2jdPXCLmq5X6aHlcqVSC1R88FTXqhy3lXcasZVldtFz3OTii7pu1HFzZFee7ned9yUbHv7+VY/6niWs2nPtVWlbFgIX/t91q/NB8N4HPvFzoIkgCHC2tGDj+A4gIZAixISFSzhAikaevaQIEZN3D464gwh5fMSPVZI21KVpIqU2GP6HlK0F9iKYufZeLWgvxT0GlIy3CWNpjpD2P2N+XzrnR37N+/Oe7qzLWjPgldJrMURsf/SnVb+r871YpHjoe9BUU+1Z1x3Iri0/lbcydkvXVlyqIlz+IDyhrDwytN7Zl7T+N7d3aY+/81XOtbFItS2+O5OSQNO/hznWTBdHyf3xhvP7g83H4VRL+M27mBE83yATTzBDibk/Qaf8QVfo5PobfQuev+zNFoKmlv4bUUffgD9na+0</latexit>
Ui ⇠ N(0, 1) i = 1 . . . , m
Vj ⇠ Ber(⇢) i = 1 . . . , n
Zij ⇠ N(0, 1)<latexit sha1_base64="4c4HJih7GNY/srfVTqFV2E4NQtI=">AAADN3icjVFdTxQxFL07frCuX6s++tKwkawJ2cyoRF9MCL7wRCBhFyIlk063QKEzHdqOCZnsD/OXqE++EXj0Dxhua0nQjdFOZub03HNOe9uiVtK6NP3WSW7dvnN3oXuvd//Bw0eP+0+eTqxuDBdjrpU2uwWzQslKjJ10SuzWRrCyUGKnOPng6zufhLFSV9vurBb7JTus5IHkzCGV983SOJfUyrKlnCmyMRumy9lLQk8bNiXyfUbVVDu7XFLaW5rkx15JWmpKsibMbEjNkZ4XV178MW/l8YxEw43sXt4fpKM0DDIPsggGEMem7n8BClPQwKGBEgRU4BArYGDx2YMMUqiR24cWOYNIhrqAGfTQ26BKoIIhe4LfQ5ztRbbCuc+0wc1xFYWvQSeBF+jRqDOI/Wok1JuQ7Nm/Zbch0+/tDP9FzCqRdXCE7L9818r/9fleHBzAu9CDxJ7qwPjueExpwqn4nZMbXTlMqJHzeIp1g5gH5/U5k+CxoXd/tizUL4LSs37Oo7aBS79LvODsz+ucB5NXo+z1aGXrzWB1LV51F57DIgzxPt/CKqzDJowx+yv87Cx0usnn5Htynlz8kiad6HkGv43kxxUAirct</latexit>
IT(⇢) !
s
4
| ln ⇢|
↵⇢<latexit sha1_base64="5GWkqPGzV16w66Ro/Q2kIfYGKM0=">AAADBnicjVFNT9wwEH2kH9AtbZdy7CVihQSXVbbdih6hXNrbVmIBCaOV4/WyEU6cOk4lFHLvP+mtN9Qrf6DXrvoPyr9gbIJUQBV1lOTNm3nPHk+cq6SwUfR7Lnjw8NHj+YUnraeLz56/aC+93C10aYQcCq202Y95IVWSyaFNrJL7uZE8jZXci4+3XX7vizRForMde5LLw5QfZckkEdwSNWpvMUXFYz6qmEnDjzv1GjNTvc6sDlnx2diqzyaGi+qUqSx0qdO6YlzlU+6jum6N2p2oG/kV3gW9BnTQrIFuz8AwhoZAiRQSGSxhBY6CngP0ECEn7hAVcYZQ4vMSNVqkLalKUgUn9pi+RxQdNGxGsfMsvFrQLopeQ8oQq6TRVGcIu91Cny+9s2P/5V15T3e2E/rHjVdKrMWU2Pt015X/q3O9WEzwzveQUE+5Z1x3onEp/a24k4d/dWXJISfO4THlDWHhldf3HHpN4Xt3d8t9/o+vdKyLRVNb4sKdkgbcuz3Ou2D3dbf3pvv2U7+z+b4Z9QJeYQVrNM8NbOIDBhiS9zf8xC/Mgq/B9+As+HFVGsw1mmXcWMH5JUR1qnw=</latexit>
IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit>
IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit>
20. Generalizationerror
W = sign( X)<latexit sha1_base64="WX5GPt3XTil6001t4w8zQyKp+Ms=">AAAC8XicjVHLSsQwFD3W93vUpZviICjI0FFBN4LoxuUIjjNgRdpOZgz2RZIKQ+lPuHMnbv0Bt/oT4h/oX3gTK/hANKXNybnnnOYmfhpyqRznecAaHBoeGR0bn5icmp6ZrczNH8skEwFrBkmYiLbvSRbymDUVVyFrp4J5kR+yln+xr+utSyYkT+Ij1U/ZaeT1Yt7lgaeIOqus5a7ftVvFTu6KyJa8Fxcrrp+EHdmPaMrdxjkvjKZdrJ5Vqk7NMcP+CeolqKIcjaTyBBcdJAiQIQJDDEU4hAdJzwnqcJASd4qcOEGImzpDgQnyZqRipPCIvaBvj1YnJRvTWmdK4w7oLyG9gpw2lsmTkE4Q1n+zTT0zyZr9LTs3mXpvfZr9MisiVuGc2L98H8r/+nQvCl1smx449ZQaRncXlCmZORW9c/tTV4oSUuI07lBdEA6M8+OcbeORpnd9tp6pvxilZvU6KLUZXvUu6YLr36/zJzher9U3auuHm9XdvfKqx7CIJazQfW5hFwdooEnZV7jHAx4taV1bN9btu9QaKD0L+DKsuzd5SqGp</latexit>
X
±1<latexit sha1_base64="nmXQk1ldbUwYZLkkbnpMKZHj/ZE=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVZIq6LLoRlxVsLXQFknSaR2axzCZKKW48Qfc6peJf6B/4Z0xBbWITkhy5tx7zsy91xchT5XjvBasufmFxaXicmlldW19o7y51UqTTAasGSRhItu+l7KQx6ypuApZW0jmRX7IrvzRqY5f3TKZ8iS+VGPBepE3jPmAB54iqtkVke1elytO1THLngVuDirIVyMpv6CLPhIEyBCBIYYiHMJDSk8HLhwI4nqYECcJcRNnuEeJtBllMcrwiB3Rd0i7Ts7GtNeeqVEHdEpIrySljT3SJJQnCevTbBPPjLNmf/OeGE99tzH9/dwrIlbhhti/dNPM/+p0LQoDHJsaONUkDKOrC3KXzHRF39z+UpUiB0Gcxn2KS8KBUU77bBtNamrXvfVM/M1kalbvgzw3w7u+JQ3Y/TnOWdCqVd2Dau3isFI/yUddxA52sU/zPEIdZ2igSd4cj3jCs3VuCevOGn+mWoVcs41vy3r4AJIdkLs=</latexit>
sign(·)<latexit sha1_base64="ychA8oOiRO4WejO0Ftjv52RsrFk=">AAAC1nicjVHLSsNAFD2Nr/quunQTLIJuSqqCLotuXFawD2hLSaZjHZoXk4lSiu7ErT/gVj9J/AP9C++MKahFdEKSM+fec2buvV7si0Q5zmvOmpqemZ3Lzy8sLi2vrBbW1utJlErGayzyI9n03IT7IuQ1JZTPm7HkbuD5vOENTnS8ccVlIqLwXA1j3gncfiguBHMVUd3C2qgtAzsR/fBmp816kdrtFopOyTHLngTlDBSRrWpUeEEbPURgSBGAI4Qi7MNFQk8LZTiIietgRJwkJEyc4wYLpE0pi1OGS+yAvn3atTI2pL32TIya0Sk+vZKUNrZJE1GeJKxPs008Nc6a/c17ZDz13Yb09zKvgFiFS2L/0o0z/6vTtShc4MjUIKim2DC6Opa5pKYr+ub2l6oUOcTEadyjuCTMjHLcZ9toElO77q1r4m8mU7N6z7LcFO/6ljTg8s9xToL6Xqm8X9o7OyhWjrNR57GJLezQPA9RwSmqqJH3NR7xhGerad1ad9b9Z6qVyzQb+Lashw+MtJY1</latexit>
{ µ, Yµ}data<latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit>
0-1 transition in the perceptron generalization performance
↵IT(⇢) !
⇢| ln ⇢|
ln 2<latexit sha1_base64="uIQF3NEkW1i+d5bBLeCJJC88RCU=">AAAC+3icjVHBTttAEH24pQ0pbQMcuViNkOglcqAtHCO40FsqEYiEUbReNomVtddarysh4z/prbeKa3+g1/Ze8Qf0Lzq7dSRoVJW1bL95M+/tzk6UyTg3QXCz5D16vPzkaWOl+Wz1+YuXrbX1k1wVmosBV1LpYcRyIeNUDExspBhmWrAkkuI0mh3a/OlHofNYpcfmMhPnCZuk8TjmzBA1ar0LmcymbFSGOvHfH1fboZ6q16FRfjjWjJc2vApl6jtQlRbuVM1Rqx10Arf8RdCtQRv16qvWT4S4gAJHgQQCKQxhCYacnjN0ESAj7hwlcZpQ7PICFZqkLahKUAUjdkbfCUVnNZtSbD1zp+a0i6RXk9LHFmkU1WnCdjff5QvnbNl/eZfO057tkv5R7ZUQazAl9n+6eeVDdbYXgzH2XQ8x9ZQ5xnbHa5fC3Yo9uX+nK0MOGXEWX1BeE+ZOOb9n32ly17u9W+byt67SsjbmdW2BX/aUNODu3+NcBCc7ne5u5+2HN+3eQT3qBjbxCts0zz30cIQ+BuT9Cd/wHT+8yvvsffGu/5R6S7VmA/eW9/U3mT2lZw==</latexit>
21. Proof idea: adaptive interpolation method [B. Macris 16]
(
Wij(t) =
q
(1 t) n
n XiXj + Zij , 1 i < j n ,
fWi(t, ✏) =
p
Rn(t, ✏) Xi + eZi , 1 i n ,<latexit sha1_base64="Cg3NYN1mLrUUqg5ElqB/esfJD9o=">AAADqXicjVFdb9MwFL1Z+Bjhq4NHXiwqUKuVKhlI4wGkCV4QTxuia7V5ihLH7dy5Tpo4oBLldyLeeYB/wbXnaoMJgaMk1+fec46vb1pIUekw/OZt+Neu37i5eSu4fefuvfudrQeHVV6XjI9YLvNykiYVl0LxkRZa8klR8mSRSj5Oz96a/PgTLyuRq496VfCTRTJTYipYohGKO99pymdCNQw1qjYYx42Ytz3dJ09f02pZ6qYXPdN9Oi0T1lCJulkSq7ZRbUsHZBKLSTwn2+TI0uhgQOhyWScZiajkRLwic/NXmKA0oJ9FxrWQGW/GbSx6ekB5UQmZqz5BO3Lu9yFWlzMoii5occFGN3Fh5Qyps1z7BZSrzHVF4k43HIZ2katB5IIuuLWfd74ChQxyYFDDAjgo0BhLSKDC5xgiCKFA7AQaxEqMhM1zaCFAbo1VHCsSRM/wO8PdsUMV7o1mZdkMXSS+JTIJPEFOjnUlxsaN2HxtlQ36N+3GapqzrfCfOq0FohpOEf0Xb135vzzTi4YpvLQ9COypsIjpjjmV2t6KOTm51JVGhQIxE2eYLzFmlrm+Z2I5le3d3G1i8z9spUHNnrnaGn6aU+KAoz/HeTU43BlGz4c7By+6e2/cqDfhETyGHs5zF/bgHezDCJj33iu8lffF3/YP/Il/dF664TnOQ/ht+ewXdGLeQQ==</latexit>
Interpolating model:
in(t, ✏) ⌘
1
n
I(X; W(t), fW(t, ✏))
<latexit sha1_base64="lG/fuMgZHpabDG/n3b0IKlOYje0=">AAADG3icjVFPS9xAHH2mWv/XtT16CS6WXZAlUUGhl6W9tDcF9w8YWZLZWR2cTeJkoixhP0q/SW+9lV5L6dWTpX4IfzNGWxWxE5J5837vvcxvJkqlyLTn/ZpwXkxOvZyemZ2bX1h8tVRZft3Oklwx3mKJTFQ3CjMuRcxbWmjJu6ni4TCSvBOdfDD1zhlXmUjifT1K+eEwPIrFQLBQE9WrtEWviMc1vR7wNBMyietvA36ai7NgoEJW+GOqfqoVQTRwu+N3du6QvL4enIs+10L2+R35N6Peq1S9hmeH+xj4JaiiHLtJ5ScC9JGAIccQHDE0YYkQGT0H8OEhJe4QBXGKkLB1jjHmyJuTipMiJPaEvke0OijZmNYmM7NuRn+R9CpyulgjT0I6Rdj8zbX13CYb9qnswmaavY1ojsqsIbEax8Q+57tV/q/P9KIxwI7tQVBPqWVMd6xMye2pmJ27/3SlKSElzuA+1RVhZp235+xaT2Z7N2cb2vqlVRrWrFmpzfHb7JIu2H94nY9Be6PhbzY29raqzfflVc9gBauo0X1uo4mP2EWLsr/gAn9w5Xx2vjrfnO83Umei9LzBveH8uAbHl7IS</latexit>
in(0, ✏) = in(1, ✏)
Z 1
0
dt
d
dt
in(t, ✏)
<latexit sha1_base64="mO6HXMT4JTGeFWNiFq05NbH+GSM=">AAADEXicjVHLSsQwFD3W93vUpZviICjo0KqgG0F048KFgqOCo0ObZjTYaUuaClLmK/wTd+7ErR+gLnWlf+FNpuIL0ZS2J+eec5Kb+EkoUuU4jx1WZ1d3T29f/8Dg0PDIaGlsfC+NM8l4lcVhLA98L+WhiHhVCRXyg0Ryr+mHfN8/29D1/XMuUxFHu+oi4UdN7yQSDcE8RVS9tCXqedSaceZqPElFGEezq23G/WDmayJSdefYDZRda0iP5UErD1SrLVQfwnqp7FQcM+yfwC1AGcXYjkv3qCFADIYMTXBEUIRDeEjpOYQLBwlxR8iJk4SEqXO0MEDejFScFB6xZ/Q9odlhwUY015mpcTNaJaRXktPGNHli0knCejXb1DOTrNnfsnOTqfd2QX+/yGoSq3BK7F++d+V/fboXhQZWTA+CekoMo7tjRUpmTkXv3P7UlaKEhDiNA6pLwsw438/ZNp7U9K7P1jP1F6PUrJ6zQpvhVe+SLtj9fp0/wd5CxV2sLOwsldfWi6vuwySmMEP3uYw1bGIbVcq+wgOe8GxdWtfWjXXbllodhWcCX4Z19wbyiq5L</latexit>
1
n
I(X; W) = ipot
n Rn(t, ✏); n, ⇢n + Remainder
⇣
Rn(t, ✏),
1
n
EhX · xit,✏
⌘
<latexit sha1_base64="Qof9htjpZRZsAU57rA1eF30B1BM=">AAADk3icjVFdb9MwFL1Z+BjjYx2IJ14sKlArqqoZk4ZUIU0FJHhAGhNdK+YROY7bWkvsyHEQU5S/yCviH8C/4NrLxMdA4CjJuefec3yvnRSZLO1o9CVYCy9dvnJ1/drG9Rs3b212tm4flroyXEy5zrSZJ6wUmVRiaqXNxLwwguVJJmbJyTOXn30QppRavbWnhTjO2VLJheTMIhV3PtGFYTxS5FWvpsmCzJux/8+aPnn4lBAiY/W+piYnhbYNTeSydxDXqunZARVFKTOt+mOa4YYpi9WAmpWOlSvrk0dediByJlUqTEMnfxIPfAN11CBPc2ZXSVK/aNBRLTNB2p4I5am2Z9FHjIzPxvUPHyTRvh93uqPhyC9yEUQt6EK79nXnM1BIQQOHCnIQoMAizoBBic8RRDCCArljqJEziKTPC2hgA7UVVgmsYMie4HeJ0VHLKoydZ+nVHHfJ8DWoJPAANRrrDGK3G/H5yjs79m/etfd0vZ3iP2m9cmQtrJD9l+688n91bhYLC3jiZ5A4U+EZNx1vXSp/Kq5z8tNUFh0K5BxOMW8Qc688P2fiNaWf3Z0t8/mvvtKxLuZtbQXfXJd4wdHv13kRHG4Po8fD7Tc73b1Je9XrcA/uQw/vcxf24CXswxR4sBO8C3iQhnfDcTgJn5+VrgWt5g78ssLX3wFQhtic</latexit>
Sum rule:
Rn(t, ✏) = ✏ + nt ⇥ argminq i(pot)
(q; n, ⇢n)
)
1
n
I(X; W) . inf
q 0
ipot
n (q; n, ⇢n)
<latexit sha1_base64="Husz+qtKzIoHMOlptlG6LYPDcLU=">AAADmnicjVFdb9MwFL1Z+Bjlq4NHeLComFpRVemYBFKFNI0XJl7GtK6V5i1yUjc1TZzUdkBTlJ+JEP8A/gXXJp2AgcBRkuNz7znX1zcqUqFNEHzxNvxr12/c3LzVun3n7r377a0HJzovVczHcZ7mahoxzVMh+dgIk/JpoTjLopRPouVrG5984EqLXB6bi4KfZSyRYi5iZpAK25+2j8JK1l3Tp7zQIs1l79UaPaMpGs1YKImhRmRcExqJpEsqqjLCVJIJWYcr2ifivOo6sshN3au7q9GltE/VIg9lzyp7lLa26ZFIFoYplX8kdK5YTIaSHKA8mpNpPXL/Sd0jNOVaa5GRFqFCzsNqRRNOgtqWC+X5Zbk/FQvbnWAQuEWugmEDOtCsw7z9GSjMIIcYSsiAgwSDOAUGGp9TGEIABXJnUCGnEAkX51BDC7UlZnHMYMgu8Zvg7rRhJe6tp3bqGKuk+CpUEniKmhzzFGJbjbh46Zwt+zfvynnas13gP2q8MmQNLJD9l26d+b8624uBObx0PQjsqXCM7S5uXEp3K/bk5KeuDDoUyFk8w7hCHDvl+p6J02jXu71b5uJfXaZl7T5uckv4Zk+JAx7+Ps6r4GRnMHw+2Hm329nbb0a9CY/gCXRxni9gD97AIYwh9kYe8957S/+xv+8f+G9/pG54jeYh/LL84+8rbNpA</latexit>
Upper bound
« trivial path »
d
dt
Rn(t, ✏) = E
D 1
n
X · x
E
t,✏
(Rn(t, ✏))
)
1
n
I(X; W) & inf
q 0
ipot
n (q; n, ⇢n)
<latexit sha1_base64="9L/XtJe5/TRni1/uA4AES/h/Qss=">AAADrHicjVFri9QwFL3d+ljH16gf/RIcXGZgGDqroLAIy4qgH4R12dkdmIwlbdNO2Dbppqm61P5O8Reo/8Kb2MHHIJrS9t5zzznJzY3KXFQmCD57W/6ly1eubl/rXb9x89bt/p27J5WqdcxnscqVnkes4rmQfGaEyfm81JwVUc5Po7Pntn76jutKKHlsLkq+LFgmRSpiZhAK+192aKpZ3CRtkxjSHoWNbIdmTHlZiVzJEXlGaMHMKoqaFy09EBnNmcxy7lRkKhsapWTeEhonyhCXfcDMErUjhs1Pt5YMNzcYUdrboUciWxmmtXpP1tbk1bBz33P/03ZEaGawl4JQIdOwOacZJ0FLx0SE8m1DdUFKZdrh+R6esogSFsox1SsVylHYHwSTwC2yGUy7YADdOlT9T0AhAQUx1FAABwkG4xwYVPgsYAoBlIgtoUFMYyRcnUMLPdTWyOLIYIie4TfDbNGhEnPrWTl1jLvk+GpUEniIGoU8jbHdjbh67Zwt+jfvxnnas13gP+q8CkQNrBD9l27N/F+d7cVACk9dDwJ7Kh1iu4s7l9rdij05+aUrgw4lYjZOsK4xjp1yfc/EaSrXu71b5upfHdOiNo87bg3f7ClxwNM/x7kZnOxOpo8mu28eD/YPulFvw314AEOc5xPYh5dwCDOIvdde5X30Wn/iH/sLf/mDuuV1mnvw2/LT7+ak4h4=</latexit>
Lower bound
« smart path »
22. Thank
You
• Statistical physics valid in corners of phase diagrams.
• Interpolation techniques allow to naturally catch the finite
size corrections.
• « All-or-nothing » seems quite generic: « Universal »
phenomenon in reconstruction of very sparse information
(i.e., with « effective low dimension ») from much higher-
dimensional data?
• What about more complex models (multi layers etc)?
• « All-or-nothing » for algorithms (AMP, gradient-based: on-
going)?