SlideShare a Scribd company logo
1 of 22
Download to read offline
0-1 Phase Transitions
in High-Dimensional Inference & Learning
Jean Barbier
International Center for Theoretical Physics, Trieste, Italy
Error
SNR
Joint work with Nicolas Macris & Clément Luneau, EPFL
Spiked matrix/tensor models
W =
q
n X ⌦ X ⌦ . . . ⌦ X + Z
<latexit sha1_base64="qQseBedGJiS1x3H0QMD/U8sLiZ8=">AAADInicjVFBS+QwGP2surqz7jrqZcFLcRDEhaFVRC/CoBePLjjOsHaQNJNxg2lTk1SQ0p/hwbv/xJs38SAL+wPWq/4Bv8QK7siyprR5ed97r/mSOBNcmyD4NeKNjo1/mJj8WPs09fnLdH1mdl/LXFHWplJI1Y2JZoKnrG24EaybKUaSWLBOfLxt651TpjSX6Z45y1gvIUcpH3BKDFKH9V4RxQO/U276kT5RpogGitAiEpjQJ2WRlqUTdMtIGp4w7Q8tI9GXRg8Vv7n5R1k7rDeCZuCG/xaEFWi01vbOvy5fhLuyfgcR9EEChRwSYJCCQSyAgMbnAEIIIEOuBwVyChF3dQYl1NCbo4qhgiB7jN8jXB1UbIprm6mdm+JfBL4KnT4sokeiTiG2f/NdPXfJlv1XduEy7d7OcI6rrARZAz+R/Z/vRflen+3FwAA2XA8ce8ocY7ujVUruTsXu3H/VlcGEDDmL+1hXiKlzvpyz7zza9W7Plrj6H6e0rF3TSpvDvd0lXnA4fJ1vwf5KM1xtrnwPG60teB6TMA8LsIT3uQ4t2IFdaGP2FaY+wKN36V17N97ts9QbqTxz8Nfwfj8BUHS4ig==</latexit>
W =
q
n X(1)
⌦ X(2)
⌦ . . . ⌦ X(k)
+ Z
<latexit sha1_base64="qJQ/YjrsM4mDdQhMZTDaBmsbUZc=">AAADNHicjVHdShwxGP12qtZu/Vnbm0Jvhi6CVlhmtojeFJb2ppcWXHepo5LJZjVsZjImGUGGPEYvfBHfpJRC70qhvekL9KZf4ojapdQMM3NyvnNO8iVpIbg2UfSlETyYmZ17OP+o+XhhcWm5tfJkT8tSUdanUkg1TIlmguesb7gRbFgoRrJUsEE6eevqgzOmNJf5rjkv2EFGjnM+5pQYpI5askrScTiwr8NEnypTJWNFaJUITBgRW+XWesHQHlZr8bpNpOEZ0+EN2b0hEzGSRk9LJut2w88+2OZRqx11Ij/CaRDXoN3b3P347OVFvCNbnyGBEUigUEIGDHIwiAUQ0PjsQwwRFMgdQIWcQsR9nYGFJnpLVDFUEGQn+D3G2X7N5jh3mdq7Ka4i8FXoDGEVPRJ1CrFbLfT10ic79l/Zlc90ezvHf1pnZcgaOEH2f75r5X19rhcDY9j2PXDsqfCM647WKaU/Fbfz8FZXBhMK5BweYV0hpt55fc6h92jfuztb4us/vNKxbk5rbQk/3S7xguO/r3Ma7HU78atO933c7r2BqzEPz+EFrOF9bkEP3sEO9DH7E/xuzDRmg8vga/At+H4lDRq15yncGcGvP/C5vQM=</latexit>
[Korada Macris 09], [Deshpande Abbe Montanari 16],
[B. Macris Dia Lesieur Krzakala Zdeborova 16], [Miolane Lelarge 17], [Miolane 17],
[B. Macris 17], [B. Miolane Macris 18], [El Alaoui Krzakala 18], [Mourrat 19], [B. Luneau Macris 19]
High-dimensional inference:
what we DO know (a statistical physics perspective)
lim
n!1
1
n
I(X; W) = inf
q 0
i(pot)
(q, ) lim
n!1
1
n
EkX ⌦ X E[X ⌦ X|W]k2
= mmse(q⇤
, )
<latexit sha1_base64="pIKHaSk8Z7lgVJpQc5YuEfzJci8=">AAADpnicjVHbjtMwEJ1suCzLrcAjLxYVUougShYQSGilFQgBL2iR6LZS3RbHdbvWxkmaOIuq1N+JeOQN/oKx8XIrCBwlGZ8553jGkxSprHQUfQy2wjNnz53fvrBz8dLlK1db164fVnldctHneZqXw4RVIpWZ6GupUzEsSsFUkopBcvzM5gcnoqxknr3Vq0KMFVtkci450whNW59oKtW0yajOqczmemUInZeMxxl51WloMidD88T9B6a7ZynTZkkXgkSGyEmDlFKRItemazrLuzTFk2esS+hyWbMZ2TB33k1smsxQxfRRkjTPDV0TfxLNtVSi8rt7PyijPxLWvrAxXU9291wpSlUCK5nc+V7LtNWOepFbZDOIfdAGvw7y1gegMIMcONSgQEAGGuMUGFT4jCCGCArExtAgVmIkXV6AgR3U1sgSyGCIHuN3gbuRRzPcW8/KqTmekuJbopLAbdTkyCsxtqcRl6+ds0X/5t04T1vbCv+J91KIajhC9F+6U+b/6mwvGubw2PUgsafCIbY77l1qdyu2cvJTVxodCsRsPMN8iTF3ytN7Jk5Tud7t3TKX/+yYFrV77rk1fLFV4oDj38e5GRzu9uL7vYdvHrT3n/pRb8NNuAUdnOcj2IeXcAB94MGLQAUnwfuwE74O++HgG3Ur8Job8MsK330FrM3jAA==</latexit>
W =
q
n X ⌦ X + Z
<latexit sha1_base64="9IeZQWNKlVBlkBsQEQH4TzFjOn8=">AAADBXicjVHNThRBGCxGEFxEVr2YeJmwITGQbGYwBC4mG7x4xIRlNzKE9PT2Ymfnz+4eEjKZs0/ggRfw5o1w9QW8qvEN5C34uhkSgRDpyUxX11dV0193XCRSmyD4M+U9mJ55ODv3qDX/eOHJYvvps12dl4qLPs+TXA1jpkUiM9E30iRiWCjB0jgRg3jy1tYHR0JpmWc75rgQ+yk7zORYcmaIOmj3qige+4P6jR/pT8pU0VgxXkUJJYxYXWV17QTDOsqNTIX2m+Wqmz/UrYN2J+gGbvi3QdiATm9958uLlZNwO2//RoQRcnCUSCGQwRBOwKDp2UOIAAVx+6iIU4SkqwvUaJG3JJUgBSN2Qt9DWu01bEZrm6mdm9NfEnoVOX0skycnnSJs/+a7eumSLXtXduUy7d6OaY6brJRYg4/E/s93pbyvz/ZiMMam60FST4VjbHe8SSndqdid+/90ZSihIM7iEdUVYe6cV+fsO492vduzZa7+1ykta9e80ZY4t7ukCw5vXudtsLvWDV9319+Hnd4WLsccXmIJr+g+N9DDO2yjT9lf8QM/8cv77H3zTr2zS6k31Xie49rwvl8A8iKsOw==</latexit>
2 groups SBM
Groups asymmetry
HardImpossible Easy
W =
q
n X + Z
<latexit sha1_base64="6uMoEBvpE9gq6pVu6P9pW95FBUY=">AAADB3icjVHLbtQwFD1NeZThNZQNEpuIERICaZQUIdggjdpNl1Op0xnRVJXj8bRWnQe2gzSK8gH9gC74A3bdoW77A922/AH8BdduKgEVAkeJj88958TXTksljY2ibwvB4o2bt24v3encvXf/wcPuo+UtU1SaixEvVKEnKTNCyVyMrLRKTEotWJYqMU4P1lx9/EloI4t8085LsZOxvVzOJGeWqN3uap2ks3DcvA8T81HbOplpxutEUcKUNXXeNElaqKmZZzTVyXBfNt4xaV75+UPT2e32on7kR3gdxC3oDd5sHj15+TkeFt1zJJiiAEeFDAI5LGEFBkPPNmJEKInbQU2cJiR9XaBBh7wVqQQpGLEH9N2j1XbL5rR2mca7Of1F0avJGeI5eQrSacLub6GvVz7ZsX/Lrn2m29uc5rTNyoi12Cf2X74r5f/6XC8WM7zzPUjqqfSM6463KZU/Fbfz8JeuLCWUxDk8pbomzL3z6pxD7zG+d3e2zNe/e6Vj3Zq32go/3C7pguM/r/M62Frpx6/7Kxtxb7CKy7GEp3iGF3SfbzHAOoYYUfYXnOEcF8FhcBx8DU4upcFC63mM30Zw+hPTKa2z</latexit>
Random linear estimation
Generalized linear models
[Reeves Pfister 16], [B. Macris Dia Krzakala 16]
[B. Miolane Macris Krzakala Zdeborovà 18]
W = '
⇣q
1
n X
⌘
<latexit sha1_base64="zK/PJNSEGcUY313MhuC/grI8fmI=">AAADCnicjVHNTtwwGBwCLXT7ty299RJ1VYleVgk90AvSiqpSj1upy65EEHK83l2L/NV2kFZR3qAnxFtw663qtS/ACRVegL5FP5sgURBqHSUezzcz8WfHRSK1CYJfC97i0r37yysPWg8fPX7ytP3s+bbOS8XFgOdJrkYx0yKRmRgYaRIxKpRgaZyIYbz/3taHB0JpmWefzbwQuymbZnIiOTNE7bU/VFE88Yf1ph8dMFXMZLQlp2uR/qJMFU0U41VYV1ldR3GejPU8pamK+jNZO9+otvI3rb12J+gGbvi3QdiATm/j6HC+evGin7dPEWGMHBwlUghkMIQTMGh6dhAiQEHcLiriFCHp6gI1WuQtSSVIwYjdp++UVjsNm9HaZmrn5vSXhF5FTh+vyZOTThG2f/NdvXTJlr0ru3KZdm9zmuMmKyXWYEbsv3xXyv/12V4MJnjnepDUU+EY2x1vUkp3Knbn/rWuDCUUxFk8proizJ3z6px959Gud3u2zNUvnNKyds0bbYnfdpd0weHN67wNtte74dvu+qew09vC5VjBS7zCGt3nBnr4iD4GlH2ME5zh3PvqffO+ez8upd5C41nFX8P7+Qfxjq9S</latexit>
W = sign( X)<latexit sha1_base64="WX5GPt3XTil6001t4w8zQyKp+Ms=">AAAC8XicjVHLSsQwFD3W93vUpZviICjI0FFBN4LoxuUIjjNgRdpOZgz2RZIKQ+lPuHMnbv0Bt/oT4h/oX3gTK/hANKXNybnnnOYmfhpyqRznecAaHBoeGR0bn5icmp6ZrczNH8skEwFrBkmYiLbvSRbymDUVVyFrp4J5kR+yln+xr+utSyYkT+Ij1U/ZaeT1Yt7lgaeIOqus5a7ftVvFTu6KyJa8Fxcrrp+EHdmPaMrdxjkvjKZdrJ5Vqk7NMcP+CeolqKIcjaTyBBcdJAiQIQJDDEU4hAdJzwnqcJASd4qcOEGImzpDgQnyZqRipPCIvaBvj1YnJRvTWmdK4w7oLyG9gpw2lsmTkE4Q1n+zTT0zyZr9LTs3mXpvfZr9MisiVuGc2L98H8r/+nQvCl1smx449ZQaRncXlCmZORW9c/tTV4oSUuI07lBdEA6M8+OcbeORpnd9tp6pvxilZvU6KLUZXvUu6YLr36/zJzher9U3auuHm9XdvfKqx7CIJazQfW5hFwdooEnZV7jHAx4taV1bN9btu9QaKD0L+DKsuzd5SqGp</latexit>
X
±1<latexit sha1_base64="nmXQk1ldbUwYZLkkbnpMKZHj/ZE=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVZIq6LLoRlxVsLXQFknSaR2axzCZKKW48Qfc6peJf6B/4Z0xBbWITkhy5tx7zsy91xchT5XjvBasufmFxaXicmlldW19o7y51UqTTAasGSRhItu+l7KQx6ypuApZW0jmRX7IrvzRqY5f3TKZ8iS+VGPBepE3jPmAB54iqtkVke1elytO1THLngVuDirIVyMpv6CLPhIEyBCBIYYiHMJDSk8HLhwI4nqYECcJcRNnuEeJtBllMcrwiB3Rd0i7Ts7GtNeeqVEHdEpIrySljT3SJJQnCevTbBPPjLNmf/OeGE99tzH9/dwrIlbhhti/dNPM/+p0LQoDHJsaONUkDKOrC3KXzHRF39z+UpUiB0Gcxn2KS8KBUU77bBtNamrXvfVM/M1kalbvgzw3w7u+JQ3Y/TnOWdCqVd2Dau3isFI/yUddxA52sU/zPEIdZ2igSd4cj3jCs3VuCevOGn+mWoVcs41vy3r4AJIdkLs=</latexit>
sign(·)<latexit sha1_base64="ychA8oOiRO4WejO0Ftjv52RsrFk=">AAAC1nicjVHLSsNAFD2Nr/quunQTLIJuSqqCLotuXFawD2hLSaZjHZoXk4lSiu7ErT/gVj9J/AP9C++MKahFdEKSM+fec2buvV7si0Q5zmvOmpqemZ3Lzy8sLi2vrBbW1utJlErGayzyI9n03IT7IuQ1JZTPm7HkbuD5vOENTnS8ccVlIqLwXA1j3gncfiguBHMVUd3C2qgtAzsR/fBmp816kdrtFopOyTHLngTlDBSRrWpUeEEbPURgSBGAI4Qi7MNFQk8LZTiIietgRJwkJEyc4wYLpE0pi1OGS+yAvn3atTI2pL32TIya0Sk+vZKUNrZJE1GeJKxPs008Nc6a/c17ZDz13Yb09zKvgFiFS2L/0o0z/6vTtShc4MjUIKim2DC6Opa5pKYr+ub2l6oUOcTEadyjuCTMjHLcZ9toElO77q1r4m8mU7N6z7LcFO/6ljTg8s9xToL6Xqm8X9o7OyhWjrNR57GJLezQPA9RwSmqqJH3NR7xhGerad1ad9b9Z6qVyzQb+Lashw+MtJY1</latexit>
{ µ, Yµ}data<latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit>
Perceptron NN
Sampling rate = # data points / # parameters to learn, infer
1-bit compressed sensing (signal processing)
Perceptron (statistics/ML)
Y = sign( X + Z
p
)
Compressed sensing (signal processing)
CDMA (multi-user communication theory)
Superposition codes (coding theory)
Y = X + Z
p
Phase retrieval  (signal processing)Y = | X|
EstimationLearning
Rectified Linear Unit (ReLU) (ML/neural nets)Y = max(0, X)<latexit sha1_base64="Ru6lXUxXyCnS3Ry1iRRkoY4QEPY=">AAAC93icjVHLSsQwFD3W93vUpZviICjI0FFBXQiiG5cjODpiRdpORsP0RZqKY+mHuHMlbv0Ct/oF4h/oX3gTO+AD0ZQ2J+eec5qbuLHPE2lZLz1Gb1//wODQ8Mjo2PjEZGlq+iCJUuGxuhf5kWi4TsJ8HrK65NJnjVgwJ3B9dui2d1T98IKJhEfhvuzE7CRwzkLe4p4jiTotrdiSXUq3lR3lm5ktAjNwLvMFa8l2I7+ZdAKaMrt2zvOurpEvnpbKVsXSw/wJqgUooxi1qPQMG01E8JAiAEMISdiHg4SeY1RhISbuBBlxghDXdYYcI+RNScVI4RDbpu8ZrY4LNqS1yky026O/+PQKcpqYJ09EOkFY/c3U9VQnK/a37Exnqr11aHaLrIBYiXNi//J1lf/1qV4kWljXPXDqKdaM6s4rUlJ9Kmrn5qeuJCXExCncpLog7Gln95xN7Ul07+psHV1/1UrFqrVXaFO8qV3SBVe/X+dPUF+ubFSsvdXy1nZx00OYxRwW6DrXsIVd1FCn6Gs84BFPxpVxY9wadx9So6fwzODLMO7fARXQpPs=</latexit><latexit sha1_base64="Ru6lXUxXyCnS3Ry1iRRkoY4QEPY=">AAAC93icjVHLSsQwFD3W93vUpZviICjI0FFBXQiiG5cjODpiRdpORsP0RZqKY+mHuHMlbv0Ct/oF4h/oX3gTO+AD0ZQ2J+eec5qbuLHPE2lZLz1Gb1//wODQ8Mjo2PjEZGlq+iCJUuGxuhf5kWi4TsJ8HrK65NJnjVgwJ3B9dui2d1T98IKJhEfhvuzE7CRwzkLe4p4jiTotrdiSXUq3lR3lm5ktAjNwLvMFa8l2I7+ZdAKaMrt2zvOurpEvnpbKVsXSw/wJqgUooxi1qPQMG01E8JAiAEMISdiHg4SeY1RhISbuBBlxghDXdYYcI+RNScVI4RDbpu8ZrY4LNqS1yky026O/+PQKcpqYJ09EOkFY/c3U9VQnK/a37Exnqr11aHaLrIBYiXNi//J1lf/1qV4kWljXPXDqKdaM6s4rUlJ9Kmrn5qeuJCXExCncpLog7Gln95xN7Ul07+psHV1/1UrFqrVXaFO8qV3SBVe/X+dPUF+ubFSsvdXy1nZx00OYxRwW6DrXsIVd1FCn6Gs84BFPxpVxY9wadx9So6fwzODLMO7fARXQpPs=</latexit><latexit sha1_base64="Ru6lXUxXyCnS3Ry1iRRkoY4QEPY=">AAAC93icjVHLSsQwFD3W93vUpZviICjI0FFBXQiiG5cjODpiRdpORsP0RZqKY+mHuHMlbv0Ct/oF4h/oX3gTO+AD0ZQ2J+eec5qbuLHPE2lZLz1Gb1//wODQ8Mjo2PjEZGlq+iCJUuGxuhf5kWi4TsJ8HrK65NJnjVgwJ3B9dui2d1T98IKJhEfhvuzE7CRwzkLe4p4jiTotrdiSXUq3lR3lm5ktAjNwLvMFa8l2I7+ZdAKaMrt2zvOurpEvnpbKVsXSw/wJqgUooxi1qPQMG01E8JAiAEMISdiHg4SeY1RhISbuBBlxghDXdYYcI+RNScVI4RDbpu8ZrY4LNqS1yky026O/+PQKcpqYJ09EOkFY/c3U9VQnK/a37Exnqr11aHaLrIBYiXNi//J1lf/1qV4kWljXPXDqKdaM6s4rUlJ9Kmrn5qeuJCXExCncpLog7Gln95xN7Ul07+psHV1/1UrFqrVXaFO8qV3SBVe/X+dPUF+ubFSsvdXy1nZx00OYxRwW6DrXsIVd1FCn6Gs84BFPxpVxY9wadx9So6fwzODLMO7fARXQpPs=</latexit>
Sigmoid/Logistic regression (ML/neural nets)P(Yµ = 1) =
1
1 + exp( µ · x)X<latexit sha1_base64="ZHW7OZbi5udCyj0kx2zC6hkQFRA=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkVwVVIR1F3RjTsrGFu0RZLptA5Nk5BMxBK69Qvc6m+Jf6B/4Z1xCmoRnZDkzLn3nJl7rx8HIpWO81qwZmbn5heKi6Wl5ZXVtfL6xmUaZQnjLouCKGn5XsoDEXJXChnwVpxwb+gHvOkPTlS8eceTVEThhRzFvDP0+qHoCeZJoq7akt9Lv5e3xjflilN19LKnQc2ACsxqROUXtNFFBIYMQ3CEkIQDeEjpuUYNDmLiOsiJSwgJHecYo0TajLI4ZXjEDujbp921YUPaK89UqxmdEtCbkNLGDmkiyksIq9NsHc+0s2J/8861p7rbiP6+8RoSK3FL7F+6SeZ/daoWiR4OdQ2Caoo1o6pjxiXTXVE3t79UJckhJk7hLsUTwkwrJ322tSbVtaveejr+pjMVq/bM5GZ4V7ekAdd+jnMauHvVo6pzvl+pH5tJF7GFbezSOA9QxykacMk6xCOe8GydWZmVW+PPVKtgNJv4tqyHDyPvk2s=</latexit><latexit sha1_base64="ZHW7OZbi5udCyj0kx2zC6hkQFRA=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkVwVVIR1F3RjTsrGFu0RZLptA5Nk5BMxBK69Qvc6m+Jf6B/4Z1xCmoRnZDkzLn3nJl7rx8HIpWO81qwZmbn5heKi6Wl5ZXVtfL6xmUaZQnjLouCKGn5XsoDEXJXChnwVpxwb+gHvOkPTlS8eceTVEThhRzFvDP0+qHoCeZJoq7akt9Lv5e3xjflilN19LKnQc2ACsxqROUXtNFFBIYMQ3CEkIQDeEjpuUYNDmLiOsiJSwgJHecYo0TajLI4ZXjEDujbp921YUPaK89UqxmdEtCbkNLGDmkiyksIq9NsHc+0s2J/8861p7rbiP6+8RoSK3FL7F+6SeZ/daoWiR4OdQ2Caoo1o6pjxiXTXVE3t79UJckhJk7hLsUTwkwrJ322tSbVtaveejr+pjMVq/bM5GZ4V7ekAdd+jnMauHvVo6pzvl+pH5tJF7GFbezSOA9QxykacMk6xCOe8GydWZmVW+PPVKtgNJv4tqyHDyPvk2s=</latexit><latexit sha1_base64="ZHW7OZbi5udCyj0kx2zC6hkQFRA=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkVwVVIR1F3RjTsrGFu0RZLptA5Nk5BMxBK69Qvc6m+Jf6B/4Z1xCmoRnZDkzLn3nJl7rx8HIpWO81qwZmbn5heKi6Wl5ZXVtfL6xmUaZQnjLouCKGn5XsoDEXJXChnwVpxwb+gHvOkPTlS8eceTVEThhRzFvDP0+qHoCeZJoq7akt9Lv5e3xjflilN19LKnQc2ACsxqROUXtNFFBIYMQ3CEkIQDeEjpuUYNDmLiOsiJSwgJHecYo0TajLI4ZXjEDujbp921YUPaK89UqxmdEtCbkNLGDmkiyksIq9NsHc+0s2J/8861p7rbiP6+8RoSK3FL7F+6SeZ/daoWiR4OdQ2Caoo1o6pjxiXTXVE3t79UJckhJk7hLsUTwkwrJ322tSbVtaveejr+pjMVq/bM5GZ4V7ekAdd+jnMauHvVo6pzvl+pH5tJF7GFbezSOA9QxykacMk6xCOe8GydWZmVW+PPVKtgNJv4tqyHDyPvk2s=</latexit>
SKA telescope 

Super-Resolution
[Gabrie Manoel Luneau B. Macris Krzakala Zdeborová 18]
Models of deep neural networks in a pre-trained phase:
[Aubin Maillard B. Macris Krzakala Zdeborová 18]
Models of shallow networks, committee machines:
W = '(1)
⇣r
1
n
(1)
'(2)
⇣r
1
n
(2)
'(3)
⇣
. . . '(L)
⇣r
1
n
(L)
X
⌘
. . .
⌘
<latexit sha1_base64="aKzzv3leqrpf9BVZ4UDbaldzcRo=">AAADs3iclVFdb9MwFL1Z+Bjlq7BHXiwqpO6lSroheEGaxgsPeyjSulZqxuQ4bmvNiYPtTCpRnuA/Iv7B9sJv4NqkqDAhmKPEx+eec+Jrp6UUxkbRt2ArvHX7zt3te537Dx4+etx98vTEqEozPmZKKj1NqeFSFHxshZV8WmpO81TySXr+1tUnF1wboYpjuyr5aU4XhZgLRi1SZ93vdZLOyaR5Q0hyQXW5FB/qfrzbJIdi0U/MR23rZK4pq+OmLpomSZXMzCrHqU5GS9G06l/W4Y2sw03r3toqM2XNxnaObpSJat/TtCHOtbuOc7hz1u1Fg8gPch3ELehBO0aq+xUSyEABgwpy4FCARSyBgsFnBjFEUCJ3CjVyGpHwdQ4NdNBboYqjgiJ7jt8FrmYtW+DaZRrvZvgXia9GJ4EX6FGo04jd34ivVz7ZsX/Lrn2m29sK57TNypG1sET2X7618n99rhcLc3jtexDYU+kZ1x1rUyp/Km7nZKMriwklcg5nWNeImXeuz5l4j/G9u7Olvn7plY51a9ZqK7hyu8QLjv+8zuvgZDiI9wbD9/u9g8P2qrfhGTyHPt7nKziAdzCCMbDgOPgUfA6+hC/DWZiG2U/pVtB6duC3EeY/AHNx6N4=</latexit>
[Aubin Loureiro Maillard Krzakala Zdeborová 19]
High-dimensional inference with structured/generative priors:
with V ⇠ Pout
⇣
·
r
1
n
WZ
⌘
<latexit sha1_base64="abRLwfGt9aoXKbn/T4RJA6epZB4=">AAADI3icjVFNb9QwEH0NX2X5WuDIxWKFVKRqlbRIcKzKpcdFYncr6rJKvN7FahIH20Fapfk5/SfcekNIiAN34Ao/gLFJJaBC4CjxmzfzXjyerMqVdXH8cS26cPHS5SvrV3vXrt+4eat/+87E6toIORY612Y/S63MVSnHTrlc7ldGpkWWy2l29NTnp2+ksUqXz92qkodFuizVQonUETXrv2x4tmCTlltVMDaaNdwUTNeu5btqucHFXDvGN31wzDcZt6+Na/jCpKJJ2qZsW57pfG5XBW3BadqG7UXQP+zN+oN4GIfFzoOkAwN0a6T7H8Axh4ZAjQISJRzhHCksPQdIEKMi7hANcYaQCnmJFj3S1lQlqSIl9oi+S4oOOrak2HvaoBb0l5xeQ0qGB6TRVGcI+7+xkK+Ds2f/5t0ET3+2Fe1Z51UQ6/CK2H/pzir/V+d7cVjgSehBUU9VYHx3onOpw634k7NfunLkUBHn8ZzyhrAIyrN7ZkFjQ+/+btOQ/xwqPetj0dXW+OJPSQNO/hzneTDZGibbw61njwY7u92o13EP97FB83yMHexhhDF5v8VXfMP36CQ6jd5F73+WRmud5i5+W9GnH2HatYU=</latexit>
Y ⇠ P
⇣
·
r
n
U ⌦ V
⌘
<latexit sha1_base64="xXdOqv1gr7Nyh6KlCP5HUAlr70o=">AAADGXicjVFNT9wwEB1CgS1QWOiRi9UVEkholeVD9Ijg0uMi7S5UGCHH691aJHGwnUqrkF/CP+HGDXHlgHrrse2vYGyCBEUIHCV582bes8cTZbE0Ngx/jQXjHyYmp2ofp2dmP83N1xcWe0blmosuV7HShxEzIpap6FppY3GYacGSKBYH0emeyx/8FNpIlXbsKBPHCRumciA5s0id1DsFjQbke0mNTEib7srhCuV9ZQldc8E5XSPUnGlb0IFmvKAxWvdZWaRl6ZXdkiorE2F81CudaPWk3giboV/kJWhVoAHVaqv6HVDogwIOOSQgIAWLOAYGBp8jaEEIGXLHUCCnEUmfF1DCNGpzrBJYwZA9xe8Qo6OKTTF2nsarOe4S46tRSWAZNQrrNGK3G/H53Ds79jXvwnu6s43wH1VeCbIWfiD7lu6x8r0614uFAXz1PUjsKfOM645XLrm/FXdy8qQriw4Zcg73Ma8Rc698vGfiNcb37u6W+fwfX+lYF/OqNoe/7pQ44Nb/43wJeuvN1kZza3+zsbNbjboGS/AFVnCe27AD36ANXfS+hN/o+y+4CK6C6+DmoTQYqzSf4dkKbu8Bt4Ox0w==</latexit>
[Tramel Manoel Caltagirone Gabrie Krzakala 16]
High-dimensional inference:
what we DON’T know (yet)
• Non Bayes optimal setting
• Structure in the data
• New statistical regimes
• Zooming on the corners of the phase diagrams
( , P0) 6= (˜, ˜P0)<latexit sha1_base64="iutlYq+xxQQXk4biHbNfleAcIeQ=">AAAC9XicjVHLSsNAFD3GV62vqks3g0WoICWtgi6LblxWsFWwUpJ01MFpEpOJUoq/4c6duPUH3OoviH+gf+GdcRQfiE5Icu+555yZO9ePpUiV6z4NOINDwyOjubH8+MTk1HRhZraZRlkS8EYQySjZ872USxHyhhJK8r044V7Xl3zXP9nU9d0znqQiCndUL+YHXe8oFIci8BRB7YJbaklid7xlVm+7S6wV8lNWaikhO5x9lGyuGe1C0S27ZrGfQcUGRdhVjwqPaKGDCAEydMERQlEs4SGlZx8VuIgJO0CfsIQiYeocF8iTNiMWJ4ZH6Al9jyjbt2hIufZMjTqgXSS9CSkZFkkTES+hWO/GTD0zzhr9zbtvPPXZevT3rVeXUIVjQv/SvTP/q9O9KBxi3fQgqKfYILq7wLpk5lb0ydmnrhQ5xITpuEP1hOLAKN/vmRlNanrXd+uZ+rNhalTngeVmeNGnpAFXvo/zZ9Cslisr5er2arG2YUedwzwWUKJ5rqGGLdTRIO9L3OEeD865c+VcOzdvVGfAaubwZTm3r+5+oH4=</latexit>
[Goldt Mézard Krzakala Zdeborová 19]
What about the
CORNERS
of phase diagrams
(in inference problems)?
⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit>
=
1
<latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit>
noise level
sparsity
Compressed sensing
↵ =
m
n<latexit sha1_base64="wnw1yuYGkmkdhkUuYnakUAai+cc=">AAAC13icjVHLSsNAFD2Nr/qudekmWARXJVVBN0LRjcsK9iGtyGSctsG8mEzEUoo7cesPuNU/Ev9A/8I7Ywo+EJ2Q5My595yZe68b+16iHOclZ01MTk3P5Gfn5hcWl5YLK8VGEqWSizqP/Ei2XJYI3wtFXXnKF61YCha4vmi6l4c63rwSMvGi8EQNYnEWsF7odT3OFFHnhWKH+XGf7Xe6kvFhMBqGo/NCySk7Ztk/QSUDJWSrFhWe0cEFInCkCCAQQhH2wZDQ00YFDmLizjAkThLyTFxghDnSppQlKIMRe0nfHu3aGRvSXnsmRs3pFJ9eSUobG6SJKE8S1qfZJp4aZ83+5j00nvpuA/q7mVdArEKf2L9048z/6nQtCl3smRo8qik2jK6OZy6p6Yq+uf2pKkUOMXEaX1BcEuZGOe6zbTSJqV33lpn4q8nUrN7zLDfFm74lDbjyfZw/QWOrXNkubx3vlKoH2ajzWMM6Nmmeu6jiCDXUyfsaD3jEk3Vq3Vi31t1HqpXLNKv4sqz7dyqcl0Y=</latexit>
⇢ = ⇥(1)<latexit sha1_base64="Y7PvJGlHAI+BpBFatinfXoOjCBI=">AAAC0XicjVHLSsNAFD2Nr1pfVZdugkWom5JUQTdC0Y3Lin1BWyVJp21oXkwmQimCuPUH3OpPiX+gf+GdMQW1iE5Icubce87MvdeOPDcWhvGa0ebmFxaXssu5ldW19Y385lYjDhPusLoTeiFv2VbMPDdgdeEKj7Uizizf9ljTHp3JePOG8dgNg5oYR6zrW4PA7buOJYi66vBheNKpDZmwiub+db5glAy19FlgpqCAdFXD/As66CGEgwQ+GAIIwh4sxPS0YcJARFwXE+I4IVfFGW6RI21CWYwyLGJH9B3Qrp2yAe2lZ6zUDp3i0ctJqWOPNCHlccLyNF3FE+Us2d+8J8pT3m1Mfzv18okVGBL7l26a+V+drEWgj2NVg0s1RYqR1TmpS6K6Im+uf6lKkENEnMQ9inPCjlJO+6wrTaxql721VPxNZUpW7p00N8G7vCUN2Pw5zlnQKJfMg1L54rBQOU1HncUOdlGkeR6hgnNUUSdvjkc84Vm71MbanXb/maplUs02vi3t4QONupRB</latexit>
⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit>
↵ = ⇥(1)<latexit sha1_base64="wDwxDp8Jtqu98sFt/qBe5OvtuX8=">AAAC1XicjVHLSsNAFD2Nr1pfUZdugkWom5KooBuh6MZlhb6gLWWSTtvQNAnJpFBKd+LWH3CrvyT+gf6Fd8YU1CI6IcmZc+85M/deO/TcWJjma0ZbWl5ZXcuu5zY2t7Z39N29WhwkkcOrTuAFUcNmMfdcn1eFKzzeCCPORrbH6/bwWsbrYx7FbuBXxCTk7RHr+27PdZggqqPrLeaFA3bZqgy4YAXruKPnzaKplrEIrBTkka5yoL+ghS4COEgwAocPQdgDQ0xPExZMhMS1MSUuIuSqOMcMOdImlMUpgxE7pG+fds2U9WkvPWOldugUj96IlAaOSBNQXkRYnmaoeKKcJfub91R5yrtN6G+nXiNiBQbE/qWbZ/5XJ2sR6OFC1eBSTaFiZHVO6pKorsibG1+qEuQQEidxl+IRYUcp5302lCZWtcveMhV/U5mSlXsnzU3wLm9JA7Z+jnMR1E6K1mnx5PYsX7pKR53FAQ5RoHmeo4QblFEl7zEe8YRnra7NtDvt/jNVy6SafXxb2sMHN9OVQw==</latexit>
= ⇥(1)<latexit sha1_base64="2CD7f7ymTgUC3T0VN00wswlLVAc=">AAAC1nicjVHLSsNAFD2Nr1pfrS7dBItQNyWpgm6EohuXFfqCtpRJOm1D0yQkE6UU3Ylbf8CtfpL4B/oX3hlTUIvohCRnzj3nzNwZK3CdSBjGa0pbWFxaXkmvZtbWNza3srnteuTHoc1rtu/6YdNiEXcdj9eEI1zeDELOxpbLG9boXNYbVzyMHN+riknAO2M28Jy+YzNBVDeba7sk7rHTdnXIBSuYB91s3igaaujzwExAHsmo+NkXtNGDDxsxxuDwIAi7YIjoacGEgYC4DqbEhYQcVee4QYa8Mak4KRixI/oOaNZKWI/mMjNSbptWcekNyaljnzw+6ULCcjVd1WOVLNnfsqcqU+5tQn8ryRoTKzAk9i/fTPlfn+xFoI8T1YNDPQWKkd3ZSUqsTkXuXP/SlaCEgDiJe1QPCdvKOTtnXXki1bs8W6bqb0opWTm3E22Md7lLumDz53XOg3qpaB4WS5dH+fJZctVp7GIPBbrPY5RxgQpqlH2NRzzhWWtqt9qddv8p1VKJZwffhvbwATvolag=</latexit>
1<latexit sha1_base64="GM0DyeF3AwEXlxUmliDnfXT5ehQ=">AAACz3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl047IF+4CmyCSdxqFpEpKJUori1h9wq38l/oH+hXfGFNQiOiHJmXPPOTN3xo0DkUrLei0Yc/MLi0vF5dLK6tr6Rnlzq5VGWeLxphcFUdJxWcoDEfKmFDLgnTjhbOQGvO0Oz1S9fc2TVEThhRzHvDdifigGwmOSKMcJSNpnju+b9mW5YlUtPcxZYOeggnzUo/ILHPQRwUOGEThCSMIBGFJ6urBhISauhwlxCSGh6xy3KJE3IxUnBSN2SF+fZt2cDWmuMlPt9miVgN6EnCb2yBORLiGsVjN1PdPJiv0te6Iz1d7G9HfzrBGxElfE/uWbKv/rU71IDHCiexDUU6wZ1Z2Xp2T6VNTOzS9dSUqIiVO4T/WEsKed03M2tSfVvauzZbr+ppWKVXMv12Z4V7ukC7Z/XucsaB1U7cPqQeOoUjvNr7qIHexin+7zGDWco44mZcd4xBOejYZxY9wZ959So5B7tvFtGA8fDjiTrw==</latexit>
Wµ =
r
n
µ · X + Zµ µ = 1, . . . , m
Xi ⇠ Ber(⇢) i = 1, . . . , n<latexit sha1_base64="G+Ucf8j6N1Ci6INlgn7132TkJXc=">AAADUXicjVFNb9QwEJ1k+ShboFu4IHExXVEVUa2SUgkulapy4bhIbHdFXa0cx9u16sSp7SCtovwD/h0HxD+AKzdujE0qPioEjhI/v3lvMuPJKiWtS5JPUdy7dv3GzbVb/fXbd+5uDDbvHVtdGy4mXCttZhmzQslSTJx0SswqI1iRKTHNzl/6+PSdMFbq8o1bVeK0YGelXEjOHFLzwfvpnBb19gG1F8Y1dGEYb6hCf87apmxbmmmV21WBW0PHS9l6OeW5dqSh2YLMWvKUvPUkoRc1y4lHB+kuVSixuwWl/dlcblMri4aaghwJ0+5Qs9RPOr38KcZyhskoCYtcBWkHhtCtsR58BAo5aOBQQwECSnCIFTCw+JxACglUyJ1Cg5xBJENcQAt99NaoEqhgyJ7j9wxPJx1b4tnntMHN8S8KX4NOAo/Ro1FnEPu/kRCvQ2bP/i13E3L62la4Z12uAlkHS2T/5btU/q/P9+JgAS9CDxJ7qgLju+Ndljrciq+c/NKVwwwVch7nGDeIeXBe3jMJHht693fLQvxzUHrWn3mnreGLrxIHnP45zqvgeG+UPhvtvd4fHh51o16Dh7AFOzjP53AIr2AME8z9NXoQPYq24g/xtx704h/SOOo89+G31Vv/Dr2XwcY=</latexit>
sparsity
measurement rate
[Reeves Xu Zadik 19]
[Reeves Xu Zadik 19]
n = = 2<latexit sha1_base64="C2FTjtFB8XOW/DSfx9H/vVfrqME=">AAAC2HicjVHLSsNAFD2Nr1pf0S7dBIvgqiRV0E2h6MZlBfvAtpQkndZgXiQToRTBnbj1B9zqF4l/oH/hnXEKahGdkOTMueecmTvjxL6XctN8zWlz8wuLS/nlwsrq2vqGvrnVTKMscVnDjfwoaTt2ynwvZA3ucZ+144TZgeOzlnN1Iuqta5akXhSe83HMeoE9Cr2h59qcqL5e7PokHtj9sKpQtdLXS2bZlMOYBZYCJahRj/QXdDFABBcZAjCE4IR92Ejp6cCCiZi4HibEJYQ8WWe4QYG8GakYKWxir+g7ollHsSHNRWYq3S6t4tObkNPALnki0iWExWqGrGcyWbC/ZU9kptjbmP6OygqI5bgk9i/fVPlfn+iFY4gj2YNHPcWSEd25KiWTpyJ2bnzpilNCTJzAA6onhF3pnJ6zIT2p7F2crS3rb1IpWDF3lTbDu9glXbD18zpnQbNStvbLlbODUu1YXXUe29jBHt3nIWo4RR0Nyh7jEU941i60W+1Ou/+UajnlKeLb0B4+AE61luE=</latexit>
⇢ &<latexit sha1_base64="7984W2rrhwZWY+76OORG7U9Xucg=">AAAC0HicjVHLSsNAFD2Nr1pfVZdugkVwVZIq6LLoxmUV+4C2SJJO29A0EycTtRQRt/6AW/0q8Q/0L7wzpqAW0QlJzpx7z5m597pR4MfSsl4zxszs3PxCdjG3tLyyupZf36jFPBEeq3o84KLhOjEL/JBVpS8D1ogEc4ZuwOru4FjF61dMxD4Pz+UoYu2h0wv9ru85kqh2S/S52YqZIwS/vsgXrKKllzkN7BQUkK4Kz7+ghQ44PCQYgiGEJBzAQUxPEzYsRMS1MSZOEPJ1nOEWOdImlMUowyF2QN8e7ZopG9JeecZa7dEpAb2ClCZ2SMMpTxBWp5k6nmhnxf7mPdae6m4j+rup15BYiT6xf+kmmf/VqVokujjUNfhUU6QZVZ2XuiS6K+rm5peqJDlExCncobgg7GnlpM+m1sS6dtVbR8ffdKZi1d5LcxO8q1vSgO2f45wGtVLR3iuWTvcL5aN01FlsYRu7NM8DlHGCCqrkfYlHPOHZODNujDvj/jPVyKSaTXxbxsMHaLqUpQ==</latexit>
⇢ &<latexit sha1_base64="7984W2rrhwZWY+76OORG7U9Xucg=">AAAC0HicjVHLSsNAFD2Nr1pfVZdugkVwVZIq6LLoxmUV+4C2SJJO29A0EycTtRQRt/6AW/0q8Q/0L7wzpqAW0QlJzpx7z5m597pR4MfSsl4zxszs3PxCdjG3tLyyupZf36jFPBEeq3o84KLhOjEL/JBVpS8D1ogEc4ZuwOru4FjF61dMxD4Pz+UoYu2h0wv9ru85kqh2S/S52YqZIwS/vsgXrKKllzkN7BQUkK4Kz7+ghQ44PCQYgiGEJBzAQUxPEzYsRMS1MSZOEPJ1nOEWOdImlMUowyF2QN8e7ZopG9JeecZa7dEpAb2ClCZ2SMMpTxBWp5k6nmhnxf7mPdae6m4j+rup15BYiT6xf+kmmf/VqVokujjUNfhUU6QZVZ2XuiS6K+rm5peqJDlExCncobgg7GnlpM+m1sS6dtVbR8ffdKZi1d5LcxO8q1vSgO2f45wGtVLR3iuWTvcL5aN01FlsYRu7NM8DlHGCCqrkfYlHPOHZODNujDvj/jPVyKSaTXxbxsMHaLqUpQ==</latexit>
(informal) Theorem [B. Macris Dia Krzakala 16] [Reeves Pfister 16]:
Compressed sensing: low measurement-rate regime
lim
n!1
1
n
I(X; W) = inf
q 0
i(pot)
(q, ) lim
n!1
1
⇢n
EkX E[X|W]k2
=
1
⇢
mmse(q⇤
, )
<latexit sha1_base64="CjDg2F4pFEFilYEvhZWW0IWjFxs=">AAADunicjVHbjtMwEJ1suCzLZQs88mJRIbUIqmQBgQQrrUBI8LZIdFupbisncVurudVxkKrUD7zCByL+AD6Ad8bGy63i4ijJ+Mw5xzOeqExFpYLgo7fjnzl77vzuhb2Lly5f2W9dvXZSFbWMeT8u0kIOI1bxVOS8r4RK+bCUnGVRygfR8pnJD95wWYkif63WJR9nbJ6LmYiZQmja+kJTkU2bnKqCinym1prQmWRxmJOXnYZGMzLUj+1/oLuHhjJtVnTOSaCJmDRIkRkpC6W7urO6Q1M8OWFdQlermiVky9x6N6FuGmprbyRPNHosCq1zTTOmFlHUPNd0Q9zhd3+AIwdtXD1jupkcHP7NUtvysqziWN3k9vf6pq120AvsIttB6II2uHVctD4AhQQKiKGGDDjkoDBOgUGFzwhCCKBEbAwNYhIjYfMcNOyhtkYWRwZDdInfOe5GDs1xbzwrq47xlBRfiUoCt1BTIE9ibE4jNl9bZ4P+ybuxnqa2Nf4j55UhqmCB6L90p8z/1ZleFMzgke1BYE+lRUx3sXOp7a2YyslPXSl0KBEzcYJ5iXFslaf3TKymsr2bu2U2/8kyDWr2sePW8NlUiQMOfx/ndnBy0Avv9R68ut8+eupGvQs34CZ0cJ4P4QhewDH0IfZG3lvvnffef+JHvvCX36g7ntNch1+Wr74C/zDtZQ==</latexit>
lim
n!1
1
⇢n
EkX E[X|W]k2
=
1
⇢
mmse(q⇤
, )
↵/↵IT(⇢)<latexit sha1_base64="V9ZJRXKm6dvB3MskAo2n1hNqnF4=">AAAC4nicjVHLSgMxFD0d3/U16lKEwSLUTZ36QJdFN7qr0KpgS8lM03bovMhkhFJcuXMnbv0Bt/ox4h/oX3gTR/CBaIZJTs695yQ314l9L5G2/ZwzRkbHxicmp/LTM7Nz8+bC4kkSpcLldTfyI3HmsIT7Xsjr0pM+P4sFZ4Hj81Onf6DipxdcJF4U1uQg5s2AdUOv47lMEtUyVxrMj3ts431pDRsisI5ql8WG6EXr+ZZZsEu2HtZPUM5AAdmoRuYTGmgjgosUAThCSMI+GBL6zlGGjZi4JobECUKejnNcIk/alLI4ZTBi+zR3aXeesSHtlWei1S6d4tMvSGlhjTQR5QnC6jRLx1PtrNjfvIfaU91tQKuTeQXESvSI/Uv3kflfnapFooM9XYNHNcWaUdW5mUuqX0Xd3PpUlSSHmDiF2xQXhF2t/HhnS2sSXbt6W6bjLzpTsWrvZrkpXtUtqcHl7+38CU42S+Wt0s7xdqGyn7V6EstYRZH6uYsKDlFFnbyvcI8HPBpt49q4MW7fU41cplnCl2HcvQH/EJqX</latexit>
1
↵n =
m
n<latexit sha1_base64="HNyXCSzlEWzm0asCuCcvSGCErl4=">AAAC2nicjVHLSsNAFD3GV62vqLhyEyyCq5KqoBuh6MZlBVsLVspkOtVgXkwmQgnduBO3/oBb/SDxD/QvvDOm4APRCUnOnHvPmbn3ekngp8p1X8as8YnJqenSTHl2bn5h0V5abqVxJrlo8jiIZdtjqQj8SDSVrwLRTqRgoReIU+/qUMdPr4VM/Tg6UYNEnIfsIvL7PmeKqK692mFBcsm6kbPf6UvG83CYR8OuXXGrrlnOT1ArQAXFasT2MzroIQZHhhACERThAAwpPWeowUVC3Dly4iQh38QFhiiTNqMsQRmM2Cv6XtDurGAj2mvP1Kg5nRLQK0npYIM0MeVJwvo0x8Qz46zZ37xz46nvNqC/V3iFxCpcEvuXbpT5X52uRaGPPVODTzUlhtHV8cIlM13RN3c+VaXIISFO4x7FJWFulKM+O0aTmtp1b5mJv5pMzeo9L3IzvOlb0oBr38f5E7S2qrXt6tbxTqV+UIy6hDWsY5PmuYs6jtBAk7xzPOART1bHurFurbuPVGus0Kzgy7Lu3wHjEZhR</latexit>
⇢n<latexit sha1_base64="2XVURmQqOQ2kJDHceYZs4UyFluk=">AAACyXicjVHLTsJAFD3UF+ILdemmkZi4Ii2a6JLoxsQNJvJIgJB2GKBS2jqdGpG48gfc6o8Z/0D/wjtjSVRidJq2Z86958zce93I92JpWa8ZY25+YXEpu5xbWV1b38hvbtXiMBGMV1noh6LhOjH3vYBXpSd93ogEd0auz+vu8FTF6zdcxF4YXMpxxNsjpx94PY85kqhaSwzCTtDJF6yipZc5C+wUFJCuSph/QQtdhGBIMAJHAEnYh4OYniZsWIiIa2NCnCDk6TjHPXKkTSiLU4ZD7JC+fdo1UzagvfKMtZrRKT69gpQm9kgTUp4grE4zdTzRzor9zXuiPdXdxvR3U68RsRIDYv/STTP/q1O1SPRwrGvwqKZIM6o6lrokuivq5uaXqiQ5RMQp3KW4IMy0ctpnU2tiXbvqraPjbzpTsWrP0twE7+qWNGD75zhnQa1UtA+KpYvDQvkkHXUWO9jFPs3zCGWcoYIqeV/hEU94Ns6Na+PWuPtMNTKpZhvflvHwAeFSka0=</latexit>
(informal) Theorem [Reeves Xu Zadik 19]: « All-or-nothing » phenomenon
Weak recovery impossible:
⇢n = o(n 1/2
) and ↵ < ↵IT,n with ↵IT,n =
2⇢n| ln ⇢n|
ln(1 + n)<latexit sha1_base64="lnscvIQZKnJefNiQ5egIenv7Xfo=">AAADWXicjVFdb9MwFL1p+CjlY4U98mJRgVpgXVKQ4GGTJniBtyGt26RlRI7rrtYcJ3IcYMryL/hvCPEH4C/wxLXrIaBC4CjJueeec+3rm5VSVCaKPged8NLlK1e713rXb9y8tda/fWe/KmrN+JQVstCHGa24FIpPjTCSH5aa0zyT/CA7fWnzB++4rkSh9sxZyY9zeqLEXDBqkEr7Hx8kelGkimyTYqjeNhvx5qQdkYQkhn8wDVWz1gZUlgu65f9p0yQ6J6/32seqJT+174VZLMNV1TZJ5pqyZrLc7TyRinjYNhgM40eJxFPPaKpGbdofROPILbIKYg8G4Ndu0f8ECcygAAY15MBBgUEsgUKFzxHEEEGJ3DE0yGlEwuU5tNBDb40qjgqK7Cl+TzA68qzC2NasnJvhLhJfjU4C99FToE4jtrsRl69dZcv+rXbjatqzneE/87VyZA0skP2X70L5vz7bi4E5PHc9COypdIztjvkqtbsVe3LyS1cGK5TIWTzDvEbMnPPinonzVK53e7fU5b86pWVtzLy2hm/2lDjg+M9xroL9yTh+Mp68eTrYeeFH3YW7cA+GOM9nsAOvYBemWPt7QIJR8LDzJQzCbthbSjuB96zDbytc/wFqPsKC</latexit>
lim inf
n!1
1
⇢nn
EkX ˆXoptk2
= 1
<latexit sha1_base64="uqseclRG6+r42WOepxt/Hs2XdP8=">AAADG3icjVFNTxUxFL0MqAh+PHDppuGFxI0vM6DRDQnBmLDExPd4CcVJp3R4DTPtpNMheRnmp/BP3Lkzbg1hywqiP4LbUhKVGO1kZk7PPee0t82qQtY2js9notm5e/cfzD9cWHz0+MnT3tLyqNaN4WLIdaHNOGO1KKQSQyttIcaVEazMCrGbHb1z9d1jYWqp1Uc7rcR+yQ6VzCVnFqm0N6KFLKXK01ZRqykiO+0IzQ3jbdK11Ex0qlRHS2YnWda+7+gJaWmWk3H3kk6YDThFZUl0ZbH+aW0jSXv9eBD7Qe6CJIA+hLGje2dA4QA0cGigBAEKLOICGNT47EECMVTI7UOLnEEkfV1ABwvobVAlUMGQPcLvIc72Aqtw7jJr7+a4SoGvQSeBVfRo1BnEbjXi641Pduzfsluf6fY2xX8WskpkLUyQ/ZfvVvm/PteLhRze+h4k9lR5xnXHQ0rjT8XtnPzSlcWECjmHD7BuEHPvvD1n4j21792dLfP1S690rJvzoG3gyu0SLzj58zrvgtHaIFkfvP7wqr+5Fa56Hp7DCrzA+3wDm7ANOzDE7M9wAT/gZ3QafYm+Rt9upNFM8DyD30b0/RpLkLPV</latexit>
Strong recovery possible: lim sup
n!1
1
⇢nn
EkX ˆXoptk2
= 0
<latexit sha1_base64="c96bpEitY67uc2VlQIQbGInOYrc=">AAADG3icjVHPaxQxGP061VrXatf26CW4CL10mVmFeiksiuCxgvsDmnbIpJlu6EwyZDKFZZw/xf/EmzfxKsWrJ0X/CL/EFNQiNsPMvLzvvZd8SVYVsrZx/HklWr1xc+3W+u3enY279zb797emtW4MFxOuC23mGatFIZWYWGkLMa+MYGVWiFl29tzVZ+fC1FKr13ZZiaOSnSqZS84sUml/SgtZ1k2VtopaTaXK7bIjNDeMt0nXUrPQqVIdLZldZFn7oqNvSEuznMy7XbpgNuAUlSXRlcX68Wg/TvuDeBj7Qa6CJIABhHGg+xdA4QQ0cGigBAEKLOICGNT4HEICMVTIHUGLnEEkfV1ABz30NqgSqGDInuH3FGeHgVU4d5m1d3NcpcDXoJPAI/Ro1BnEbjXi641Pduy/sluf6fa2xH8WskpkLSyQ/Z/vUnldn+vFQg5PfQ8Se6o847rjIaXxp+J2Tn7rymJChZzDJ1g3iLl3Xp4z8Z7a9+7Olvn6V690rJvzoG3gm9slXnDy93VeBdPRMHk8HL16Mhg/C1e9Dg/gIezgfe7BGF7CAUww+x18ge/wI3obvY8+RB9/SaOV4NmGP0b06SeQyrPs</latexit>
⇢n = o(1) and ↵ > ↵IT,n<latexit sha1_base64="Ln0kLNdmUwdFBu8hKQqvctdC95s=">AAAC/nicjVFdS9xAFD2mH3601rV97Mvg0mKhLIktqA+K2Jf6prCrC0aWSXZ0g0kmTCaihAX/Sd/6Vnz1D/hqH0v/gf4L70xnQbsUOyGZc8+952Tu3KhIk1L7/u8J78nTZ88np6ZnXrycfTXXmH+9W8pKxaITy1SqbsRLkSa56OhEp6JbKMGzKBV70fEXk987EapMZN7WZ4U4yPhRnhwmMddE9Rqr70M1kL2crTG5GHxgIQu1ONU1z/tDE/C0GPB1t/fqOlQZ22oPP+bDXqPpt3y72DgIHGjCrW3Z+IUQfUjEqJBBIIcmnIKjpGcfAXwUxB2gJk4RSmxeYIgZ0lZUJaiCE3tM3yOK9h2bU2w8S6uO6S8pvYqUDO9II6lOETZ/YzZfWWfD/su7tp7mbGe0R84rI1ZjQOxjulHl/+pMLxqHWLE9JNRTYRnTXexcKnsr5uTsXleaHAriDO5TXhGOrXJ0z8xqStu7uVtu8ze20rAmjl1thVtzShpw8Pc4x8HuUiv41Fra+dzc2HSjnsJbLGCR5rmMDXzFNjrk/Q1XuMZP79z77v3wLv6UehNO8wYPlnd5B7FVpPc=</latexit>
Wµ =
s
n
⇢nn
µ · X + Zµ µ = 1, . . . , ↵nn
Xi ⇠ Ber(⇢n) i = 1, . . . , n n = ⌦(1)<latexit sha1_base64="1I5kPe7Eij+cSnQwga68y7AnmK8=">AAADwHicjVFdb9MwFL1Z+Bjlq4NHXiwqpk5MVTOQxkulqbzwRpHoWjFPleO4rbUkTm0HqUR55hV+HeIfwG/ghWsvE9ANgaMkx+fec66vb1yk0th+/2uwFV67fuPm9q3W7Tt3791v7zw4NqrUXIy5SpWexsyIVOZibKVNxbTQgmVxKibx2UsXn7wX2kiVv7XrQpxmbJHLueTMIjVr/5jMaFbuDqhZaVvRuWa8oinqEzarqPevtEjqvK4rqpdqg0SaxipNzDrDX0VHS1k7Q8oTZUlF4zmZ1k/fOYrQVckSgmgQ7dMU42afsrRYbhYiOaWt6UzuUiMz9NAZGQpdd68ov9eYyl+WOVLnha7sggwIfZ2JBetGe7N2p9/r+0Uug6gBHWjWSLW/AIUEFHAoIQMBOVjEKTAw+JxABH0okDuFCjmNSPq4gBpaqC0xS2AGQ/YMvwvcnTRsjnvnabyaY5UUX41KAk9QozBPI3bViI+X3tmxf/OuvKc72xr/ceOVIWthiey/dBeZ/6tzvViYwwvfg8SeCs+47njjUvpbcScnv3Vl0aFAzuEE4xox98qLeyZeY3zv7m6Zj3/zmY51e97klvDdnRIHHG2O8zI4PuhFz3oHb553jobNqLfhETyGLs7zEI7gFYxgDDxgwcfgU/A5HIbLUIWr89StoNE8hD9W+OEnxOrv/w==</latexit>
Replica prediction at ⇢ ! 0+ agrees with ⇢n ! 0+ for ⇢n = o(n 1/2
)<latexit sha1_base64="52bXL9v9NgK4PAqRjfi5jyXoWfU=">AAADOHicjVFNTxsxEJ0shSbhK6VHLlYjJBAi7KZIcKkUlQtHWjWAxELkdZzEyma9sr2lUZQ/xh9BuXEDrvwBOjYbUYoQeLW7M2/ee/Z4ojQW2vj+pODNfJid+1gslecXFpeWK59WjrTMFONNJmOpTiKqeSwS3jTCxPwkVZwOopgfR/19Wz/+zZUWMvllhik/G9BuIjqCUYNQq6JDw/+Y0U+OmzFKUNsWzJYINWMSklD1JAmNJH5r06aOTbuKc00uhOlNOa3kiVV+ZHWkeqp+k+vJ+Wgr2K6PN1qVql/z3SIvgyAPqpCvQ1m5ghDaIIFBBgPgkIDBOAYKGp9TCMCHFLEzGCGmMBKuzmEMZdRmyOLIoIj28dvF7DRHE8ytp3ZqhrvE+CpUElhDjUSewtjuRlw9c84Wfc175Dzt2Yb4j3KvAaIGeoi+pZsy36uzvRjowJ7rQWBPqUNsdyx3ydyt2JOTf7oy6JAiZuM21hXGzCmn90ycRrve7d1SV791TIvanOXcDO7sKXHAwf/jfBkc1WvB11r9x0618T0fdRFW4Qus4zx3oQEHcAhN9J7AQ6FYKHmX3rV34909Ur1CrvkMz5Z3/xfE5roM</latexit>
[Lesieur Krzakala Zdeborová 15]⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit>
=
1
<latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit>
Sparse principal components analysis: high sparsity regime
W =
r
n
X ⌦ X + Z
Xi ⇠ Ber(⇢) i = 1, . . . , n<latexit sha1_base64="1z0nYv4qYQG5iyOMuyNR5/W5QP0=">AAADMnicjVHPTxQxFH47oOL6g1WPXho2EoxkM4MkeCEhePEICctupIR0Ol1o6EyHtmNCJvNf+Z94MHozXjThH+DGaymJSox2MjPf+977vvb15bWS1qXpl14yN3/n7r2F+/0HDx89Xhw8ebpvdWO4GHOttJnmzAolKzF20ikxrY1gZa7EJD996/OTD8JYqas9d16Lw5IdV3ImOXNIHQ3KluYzMumWN6k9M66lM8N4SxU6FKxrq64LBdOOaidLYUkMX4X/+47S/vRILlMrS0yZkmwL061Qc6JfEnrWsILIzWyVqkI7u4r7DdNRGha5DbIIhhDXjh58BgoFaODQQAkCKnCIFTCw+BxABinUyB1Ci5xBJENeQAd91DZYJbCCIXuK32OMDiJbYew9bVBz3EXha1BJ4AVqNNYZxH43EvJNcPbs37zb4OnPdo7/PHqVyDo4QfZfupvK/9X5XhzM4E3oQWJPdWB8dzy6NOFW/MnJL105dKiR87jAvEHMg/LmnknQ2NC7v1sW8j9CpWd9zGNtAz/9KXHA2Z/jvA3210bZ69Ha7vpwazuOegGewxKs4Dw3YAvewQ6M0fsTXPaS3lzyMfmafEu+X5cmvah5Br+t5OIKpZ65Jg==</latexit>
« Bernoulli spiked model »
⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit>
=
1
<latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit>
lim
n!1
1
n
I(X; W) = inf
q 0
i(pot)
(q; ⇢, ) lim
n!1
1
⇢n
EkX
<latexit sha1_base64="Lulp+ygbLGh2RfzxN1GNl08w/UU=">AAADi3icjVHtbtMwFL1Z+BiFsQI/+WNRIbVoVEmHBGJMmkCT4N9AdK1Ur5GTOq21JE4dB6kKfj1eAfEG8BZcm0wCJj4cJTk+95x7fX3jMhOVDoIv3pZ/5eq169s3Ojdv7dze7d65e1rJWiV8nMhMqmnMKp6Jgo+10BmfloqzPM74JD5/ZeOTD1xVQhbv9abkZzlbFiIVCdNIRd1PNBN51BRUSyqKVG8MoaliSROapjBv+g2NUzI1B+4/MYNDq4qaNV1yEhgi5g1KVE5Kqc3A9NcHVK3kHs3wBAs2IHS9rtmC/KWI1ZMCiZzpVRw3x4Z+bIs+piumWxy5KrLUGJ6PDt0uzyuOJeePBp2o2wuGgVvkMghb0IN2ncjuZ6CwAAkJ1JADhwI04gwYVPjMIIQASuTOoEFOIRIuzsFAB701qjgqGLLn+F3ibtayBe5tzsq5E6yS4avQSeAheiTqFGJbjbh47TJb9k+5G5fTnm2D/7jNlSOrYYXsv3wXyv/12V40pPDM9SCwp9IxtrukzVK7W7EnJz91pTFDiZzFC4wrxIlzXtwzcZ7K9W7vlrn4V6e0rN0nrbaGb/aUOODw93FeBqejYbg/HL190jt62Y56G+7DA+jjPJ/CEbyGExhD4u1577yZR/0df99/7r/4Id3yWs89+GX5x98BfXbXcw==</latexit>
Theorem [Korada Macris 09], [Deshpande Montanari 14],
[B. Macris Dia Lesieur Krzakala Zdeborová 16], [Miolane Lelarge 17]:
Sparse PCA: high sparsity regime
lim
n!1
1
(⇢n)2
EkX ⌦ X E[X ⌦ X|W]k2
F = mmse(q⇤
)
<latexit sha1_base64="XU1cnkMkItBJZdPCKdPu2tKN48o=">AAADVXicjVFdb9MwFL1pxxjjYx08sgeLaqhDokoKE7wgTUxDPA6JrpXqtkoyZ7UWx8F2kKos/4F/N/EP2B/gDYlrz5MGA4GjJOeee861r29S5lybMPwatNort1Zvr91Zv3vv/oONzubDIy0rlbJhKnOpxkmsWc4LNjTc5GxcKhaLJGej5HTf5kefmdJcFh/NsmRTEZ8UPONpbJCad77QnIt5XVAjKS8ys2xopuK0jpq6R9VCkmJnNmgIFbFZJEl90NCzmiYZGTdUGi6YJj58fk0z+bPk0jlqpvRsXlMlyLtmNnjjkBCaNb1Ps2c780437IdukZsg8qALfh3KzjlQOAYJKVQggEEBBnEOMWh8JhBBCCVyU6iRU4i4yzNoYB29FaoYKmJkT/F7gtHEswXGtqZ27hR3yfFV6CSwjR6JOoXY7kZcvnKVLfu32rWrac+2xH/iawlkDSyQ/ZfvSvm/PtuLgQxeux449lQ6xnaX+iqVuxV7cnKtK4MVSuQsPsa8Qpw659U9E+fRrnd7t7HLf3NKy9o49doKLuwpccDR7+O8CY4G/ehFf/fDy+7eWz/qNXgMT6CH83wFe/AeDmGItb8HW8F28LR13vrRXmmvXkpbgfc8gl9We+Mno27Eng==</latexit>
All-or-nothing in sparse PCA
I-MMSE relation:
d
d
1
n
I(X; W) =
1
4n2
EkX ⌦ X E[X ⌦ X|W]k2
F
<latexit sha1_base64="7sDds8HKzzKXCvjLcngkqhQ1OFE=">AAADYnicjVFdT9RAFL3dosD6wSKP8jBxYwIPbtpCgokxIRoNvEHisptsl810OgsT+pXp1IR056f4u4zvPsi/8M4wBBSNTtP23HPPuTN3blJlolZB8M3r+EsPHi6vrHYfPX7ydK23/uykLhvJ+JCVWSnHCa15Jgo+VEJlfFxJTvMk46Pk4r3Jjz5zWYuy+KQuKz7N6Vkh5oJRhdSs9yWeS8raVLdpnKEtpZpcU6FuC3241cbJnIz1G/sf6W3yltwKdovTCPU5VedJ0n7Q8cLJ41KJnNfEha/uaCZ/lizcBtN4MWtjmZOP+jSa9frBILCL3AehA31w66jsfYUYUiiBQQM5cChAIc6AQo3PBEIIoEJuCi1yEpGweQ4auuhtUMVRQZG9wO8ZRhPHFhibmrV1M9wlw1eik8BL9JSok4jNbsTmG1vZsH+r3dqa5myX+E9crRxZBefI/st3o/xfn+lFwRxe2x4E9lRZxnTHXJXG3oo5ObnTlcIKFXIGp5iXiJl13twzsZ7a9m7ultr8D6s0rImZ0zZwZU6JAw5/H+d9cBINwp1BdLzb33/nRr0Cz+EFbOE892AfDuAIhsA839v2Im+n893v+uv+xrW04znPBvyy/M2fXdTHNg==</latexit>
IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit>
IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit>
⇢ ! 0+ gives IT(⇢)
⇢!0
!
4| ln ⇢|
⇢
[Lesieur Krzakala Zdeborová 15]
⇢n ! 0+<latexit sha1_base64="CsLOCzcFzETb5Ob3d8nGwZTOSr4=">AAAC4nicjVHLSsNAFD2N7/qKuhQhWARBKGkVdCm6calgbaEtJUmnbWiaCZOJUEJX7tyJW3/ArX6M+Af6F94ZU1CL6IQkZ86958zce90o8GNp2685Y2p6ZnZufiG/uLS8smqurV/FPBEeq3g84KLmOjEL/JBVpC8DVosEcwZuwKpu/1TFq9dMxD4PL+UwYs2B0w39ju85kqiWudUQPd5KG9opFaw9SsPRyGpIbtmtvZZZsIu2XtYkKGWggGydc/MFDbTB4SHBAAwhJOEADmJ66ijBRkRcEylxgpCv4wwj5EmbUBajDIfYPn27tKtnbEh75RlrtUenBPQKUlrYIQ2nPEFYnWbpeKKdFfubd6o91d2G9HczrwGxEj1i/9KNM/+rU7VIdHCka/Cppkgzqjovc0l0V9TNrS9VSXKIiFO4TXFB2NPKcZ8trYl17aq3jo6/6UzFqr2X5SZ4V7ekAZd+jnMSXJWLpf1i+eKgcHySjXoem9jGLs3zEMc4wzkq5H2DRzzh2Wgbt8adcf+ZauQyzQa+LePhA38Am5s=</latexit>
n ! +1<latexit sha1_base64="Haf+8ljzRuATXGTIzWyIZOsLN5E=">AAAC6XicjVHLSsNAFD2Nr1pfVZduglUQhJJWQZdFNy4r2FZoSknSqQ5NkzCZCCX0B9y5E7f+gFv9EfEP9C+8M6bgA9EJSc6ce86ZuTNu5PNYWtZLzpianpmdy88XFhaXlleKq2vNOEyExxpe6Ifi3HVi5vOANSSXPjuPBHOGrs9a7uBY1VtXTMQ8DM7kKGKdoXMR8D73HElUt7hl+yTuOd3U1mGpYL1xGozHpi1Dc9fmQV+OusWSVbb0MH+CSgZKyEY9LD7DRg8hPCQYgiGAJOzDQUxPGxVYiIjrICVOEOK6zjBGgbwJqRgpHGIH9L2gWTtjA5qrzFi7PVrFp1eQ08Q2eULSCcJqNVPXE52s2N+yU52p9jaiv5tlDYmVuCT2L99E+V+f6kWij0PdA6eeIs2o7rwsJdGnonZufupKUkJEnMI9qgvCnnZOztnUnlj3rs7W0fVXrVSsmnuZNsGb2iVdcOX7df4EzWq5sleunu6XakfZVeexgU3s0H0eoIYT1NGg7Gs84BFPxsC4MW6Nuw+pkcs86/gyjPt3I/qe8A==</latexit>
⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit>
VERY high sparsity
Linked to planted clique
=
1
<latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit>
Sparse PCA: VERY high sparsity regime
W =
r
n
n
X ⌦ X + Z
Xi ⇠ Ber(⇢n) i = 1, . . . , n<latexit sha1_base64="zDIqNDRQ8LcLOTNSia4xfXI5HVo=">AAADVnicjVFfaxQxEJ+9s7ae/876KELw8KxYjt0q2JdCqS8+VvB6h005stlcG5rdbJOsUJb9EH469RvoJ/BNnMQU1EM0yyYzv5nfLzOZvFbSujT9nPT619aur2/cGNy8dfvO3eG9zSOrG8PFlGulzTxnVihZiamTTol5bQQrcyVm+fkrH5+9F8ZKXb11l7U4KdlpJZeSM4fQYvihpfmSzLrxHrUXxrV0aRhvqUKFgi3aloYrWiOKjlRd17W4Bcq8o9rJUlgS3WfhfNdROpgv5JhaWWLIlORAmG6LmjO9KveU0IuGFUTuZdtUFdrZbaxplE7SsMiqkUVjBHEd6uFHoFCABg4NlCCgAoe2AgYWv2PIIIUasRNoETNoyRAX0MEAuQ1mCcxgiJ7jforecUQr9L2mDWyOtyj8DTIJPEaOxjyDtr+NhHgTlD36N+02aPraLvHMo1aJqIMzRP/Fu8r8X57vxcESdkMPEnuqA+K741GlCa/iKye/dOVQoUbM2wXGDdo8MK/emQSODb37t2Uh/iVketT7POY28NVXiQPO/hznqnG0M8meT3bevBjtH8RRb8ADeARbOM+XsA+v4RCmqP0teZiMkye9T73v/bX++s/UXhI59+G31R/+AJqtxag=</latexit>
[Berthet Rigollet 13]
[Deshpande Montanari 13]
Finite size bound for the mutual information
Theorem (B. Macris 19): based on the adaptive interpolation method
Let P0,n = ⇢n p0 + (1 ⇢n) 0. Let n = ⇥(| ln ⇢n|/⇢n) which is the appro-
priate scaling to observe a phase transition, and ⇢n = ⇥(n ) with 2 [0, 1/6)
Then
1
⇢n| ln ⇢n|
1
n
I(X; W) inf
q2[0,1]
ipot
n (q; n, ⇢n)  C
(ln n)1/3
n(1 6 )/7
with i(pot)
n (q; n, ⇢n) =
n⇢2
n
4
(q 1)2
+ In(X;
p
n⇢nq X + Z) X ⇠ P0,n, Z ⇠ N(0, 1).
<latexit sha1_base64="fQUbEiIQan28f7ueBu1sSyqjneA=">AAAFDHicjVLbbtNAEN2GACXcWnjkZURLldBc7BRapCpS1UqISggVqZeIbmNtnE2yqrN27E2hcvcX+BPeeEO88gM8ICR4h79gduMUaEHgKPHsmTPnzMymHQUiUY7zeSp3IX/x0uXpK4Wr167fuDkze2s3CUexz3f8MAjjZpslPBCS7yihAt6MYs4G7YDvtQ83TH7viMeJCOW2Oo74wYD1pOgKnymEvNnc46dcwfyWlzplqRs07oeeBFqGyHMWi25lDJRohweKec58FSyfBmjRYZ5sAN3uc8WKJzSQY/JJLSuah5d94fdBJKD6HFgUxWEUC6Y4JD7DjnugQgjbCY+PMA1RHwcBFTOZCNNdGZjsoJdVa2Q+spVWaBsjbfSF6iPBHKmQ+07ZrS0jjExZQLQnZIo+PXlfFxZoN2Z+6uo06/Jnv5qui97JaV7qzWJK211o6lX73tMlqKB+10uH+ALjc6BBtFBqAFGotCeLw9XTnZQnSzOyQAMOG2PxInqCLLVSt7ak0aiV4oaXbful2orWlBao4q9UaubSQIcj1oEF4RniqVdJ/8mrMXY4xcdwq67TB8ivuKVWfXET22yu0mQYq7NEGGq88+bii9LYtEkTMYDsX1EGCi8MkFK8Nnimi7iAUrVAuexM9uvNzDlVxz5wPnCzYI5kz1Y484lQ0iEh8cmIDAgnkiiMA8JIgp994hKHRIgdkBSxGCNh85xoUsDaEbI4Mhiih/jbw9N+hko8G83EVvvoEuA3xkog97AmRF6MsXEDmx9ZZYP+TTu1mqa3Y3y3M60Boor0Ef1X3YT5v3VmFkW65JGdQeBMkUXMdH6mMrJbMZ3DL1MpVIgQM3EH8zHGvq2c7BlsTWJnN7tlNv/NMg1qzn7GHZHvpku8YPfsdZ4PdutVd6n68Hl9bm09u+ppcofcJUW8zxWyRp6QLbJD/Nyb3Mfcl9zX/Ov82/y7/PsxNTeV1dwmvz35Dz8AiglePw==</latexit>
C1n 1/2
< ⇢n < C2<latexit sha1_base64="GYXdMrydX8eJ3XaVm05eFoqGSSU=">AAAC3XicjVHLSsNAFD3G97vqRnATLIIba1IFXbgQu3GpYG2hrSFJRw2mmTCZCFJ0507c+gNu9XfEP9C/8M44BbWITkhy5tx7zsy9N0jjKJOO8zpgDQ4Nj4yOjU9MTk3PzBbm5o8znouQVUMec1EP/IzFUcKqMpIxq6eC+Z0gZrXgoqLitUsmsognR/IqZa2Of5ZEp1HoS6K8wmLFc+3kpLvmrpevd5rinHvJjl3xyl6h6JQcvex+4BpQhFkHvPCCJtrgCJGjA4YEknAMHxk9DbhwkBLXQpc4QSjScYZrTJA2pyxGGT6xF/Q9o13DsAntlWem1SGdEtMrSGljhTSc8gRhdZqt47l2Vuxv3l3tqe52Rf/AeHWIlTgn9i9dL/O/OlWLxCm2dQ0R1ZRqRlUXGpdcd0Xd3P5SlSSHlDiF2xQXhEOt7PXZ1ppM16566+v4m85UrNqHJjfHu7olDdj9Oc5+cFwuuRul8uFmcXfPjHoMS1jGKs1zC7vYxwGq5H2DRzzh2fKsW+vOuv9MtQaMZgHflvXwAQUnl3Q=</latexit>
left open
Replica prediction at ⇢ ! 0+ agrees with ⇢n ! 0+ for ⇢n = o(n 1/2
)<latexit sha1_base64="52bXL9v9NgK4PAqRjfi5jyXoWfU=">AAADOHicjVFNTxsxEJ0shSbhK6VHLlYjJBAi7KZIcKkUlQtHWjWAxELkdZzEyma9sr2lUZQ/xh9BuXEDrvwBOjYbUYoQeLW7M2/ee/Z4ojQW2vj+pODNfJid+1gslecXFpeWK59WjrTMFONNJmOpTiKqeSwS3jTCxPwkVZwOopgfR/19Wz/+zZUWMvllhik/G9BuIjqCUYNQq6JDw/+Y0U+OmzFKUNsWzJYINWMSklD1JAmNJH5r06aOTbuKc00uhOlNOa3kiVV+ZHWkeqp+k+vJ+Wgr2K6PN1qVql/z3SIvgyAPqpCvQ1m5ghDaIIFBBgPgkIDBOAYKGp9TCMCHFLEzGCGmMBKuzmEMZdRmyOLIoIj28dvF7DRHE8ytp3ZqhrvE+CpUElhDjUSewtjuRlw9c84Wfc175Dzt2Yb4j3KvAaIGeoi+pZsy36uzvRjowJ7rQWBPqUNsdyx3ydyt2JOTf7oy6JAiZuM21hXGzCmn90ycRrve7d1SV791TIvanOXcDO7sKXHAwf/jfBkc1WvB11r9x0618T0fdRFW4Qus4zx3oQEHcAhN9J7AQ6FYKHmX3rV34909Ur1CrvkMz5Z3/xfE5roM</latexit>
Compressed sensing [Reeves Xu Zadik 19]:
1
⇢n| ln ⇢n|
inf
q2[0,1]
ipot
n (q; n, ⇢n) !
n!+1
I  1 + I > 1
<latexit sha1_base64="pfzcBo5SrMvAvhK3FxHgaDjx7Rw=">AAADeXicjVHdbtMwGP3S8LONvzLu4MZQIXVsqpINaUhIqIIbxtWQ6DapKZHjuq01x84cB6hC3op3QbwBPAM3fPZSCRgIHCU5Pt85x/7srJCitFH0JeiEly5fubq2vnHt+o2bt7q3N49KXRnGR0xLbU4yWnIpFB9ZYSU/KQyneSb5cXb6wtWP33FTCq3e2GXBJzmdKzETjFqk0u6nZGYoq+OmTsxCp+pjIhVpYZMINUvrM/yRcbQTTxoi3qIuJ4W2Tar6Z08TiUtNaap2zj1bJPlgxHxhqTH6/RijrCbbLscuJ3WznsxpnlOS5NQusqw+aJJMzPsrVnISO2Jr+8+CZ2057faiQeQHuQjiFvSgHYe6+xkSmIIGBhXkwEGBRSyBQonPGGKIoEBuAjVyBpHwdQ4NbKC3QhVHBUX2FL9znI1bVuHcZZbezXAVia9BJ4GH6NGoM4jdasTXK5/s2L9l1z7T7W2J/6zNypG1sED2X76V8n99rhcLM3jiexDYU+EZ1x1rUyp/Km7n5KeuLCYUyDk8xbpBzLxzdc7Ee0rfuztb6utfvdKxbs5abQXf3C7xguPfr/MiONodxHuD3dePe8Pn7VWvwT14AH28z30Ywks4hBGw4G4wDA6CV53v4f2wHz46l3aC1nMHfhnh3g8qRtA8</latexit>
Setting n ⌘ n,IT(⇢n) where n,IT(⇢n) ⌘ 4| ln ⇢n|
⇢n<latexit sha1_base64="RQtxbdebaThUKCNpnc/TcDPahlo=">AAADS3icjVFBT9swGP0SYGOwjQ6OcLDWTmLSVKVl0nZE48JuoFFAIqhyXLe1cJzgOEwo5Lp/xxnxDzZpt92mHfbZdcU2QMxRkuf3vffsz05yKQoTRddBODM79+jx/JOFxafPni81XizvF1mpGe+xTGb6MKEFl0LxnhFG8sNcc5omkh8kJ1u2fnDGdSEytWfOc36c0pESQ8GoQarf+PKJGyPUiLRiia4B7auYn5bijMQjmqaUTOlKvalinZKPe3W9Hutx1levW+TzmGt+471LRHze23ioKasuYqnIpHRRVxNQt/qNZtSO3CC3QceDJvixkzWuIIYBZMCghBQ4KDCIJVAo8DmCDkSQI3cMFXIakXB1DjUsoLdEFUcFRfYEvyOcHXlW4dxmFs7NcBWJr0YngVfoyVCnEdvViKuXLtmy92VXLtPu7Rz/ic9KkTUwRvYh31T5vz7bi4EhvHc9COwpd4ztjvmU0p2K3Tn5oyuDCTlyFg+wrhEz55yeM3GewvVuz5a6+lentKydM68t4ZvdJV5w59/rvA32u+3ORru7221ufvBXPQ+r8BLW8T7fwSZsww70MPt7sBysBmvhZfgj/Bn+mkjDwHtW4K8xM/cbePrB3A==</latexit>
All-or-nothing in sparse PCA
1
(n⇢n)2
EkX ⌦ X E[X ⌦ X|W]k2
F !
n!1
I(  1)
<latexit sha1_base64="sJaHnwbzC0ucbLGuGU+Q1r3AbWo=">AAADZXicjVFdb9MwFL1pYIzBoAPECw9YVEidBFVSxsfjBALB25DoWqnuKsdzWmuJHTkOUGX5L/wrxC8A/gXXXsbXQOAoybnnnnPt65sUmSxtFH0KOuG582sX1i9uXLq8eeVqd+vafqkrw8WI60ybScJKkUklRlbaTEwKI1ieZGKcHD1z+fFbYUqp1Ru7KsQsZwslU8mZRWre/UBTw3gdN3VfUbPUc7V9MGxozuwySernDT2uaZKSSUO1lbkoSRveJz800z9LTpzjZkaP5zU1OXnRHAzpeyMXS8uM0e+milpNpUrtalZ/3/NV06cLlueMZoLE2/NuLxpEfpGzIG5BD9q1p7sfgcIhaOBQQQ4CFFjEGTAo8ZlCDBEUyM2gRs4gkj4voIEN9FaoEqhgyB7hd4HRtGUVxq5m6d0cd8nwNegkcBc9GnUGsduN+HzlKzv2b7VrX9OdbYX/pK2VI2thiey/fKfK//W5Xiyk8MT3ILGnwjOuO95WqfytuJOTn7qyWKFAzuFDzBvE3DtP75l4T+l7d3fLfP6LVzrWxbzVVvDVnRIHHP8+zrNgfziIHwyGr3d6u0/bUa/DLbgDfZznY9iFl7AHI+DBWnAveBg86nwON8Mb4c0TaSdoPdfhlxXe/gYbSspP</latexit>
[The overlap gap property, Gamarnik Jagannath Sen 19]
In agreement with the recent study of the
Maximum Likelihood Estimator for FIXED but small ⇢ ! 0+<latexit sha1_base64="RM3Ida9m8zVRPO2cA0dUwxlRUtA=">AAACznicjVHLSsNAFD2Nr1pfVZdugkUQhJJUQZdFNy4r2FZoS0nSaRuaZMJkUiiluPUH3OpniX+gf+GdMQW1iE5Icubcc+7MvdeNAz+RlvWaM5aWV1bX8uuFjc2t7Z3i7l4j4anwWN3jARd3rpOwwI9YXfoyYHexYE7oBqzpjq5UvDlmIvF5dCsnMeuEziDy+77nSKJabTHkbclNq3vSLZassqWXuQjsDJSQrRovvqCNHjg8pAjBEEESDuAgoacFGxZi4jqYEicI+TrOMEOBvCmpGCkcYkf0HdCulbER7VXORLs9OiWgV5DTxBF5OOkEYXWaqeOpzqzY33JPdU51twn93SxXSKzEkNi/fHPlf32qFok+LnQNPtUUa0ZV52VZUt0VdXPzS1WSMsTEKdyjuCDsaee8z6b2JLp21VtHx9+0UrFq72XaFO/qljRg++c4F0GjUrZPy5Wbs1L1Mht1Hgc4xDHN8xxVXKOGuu74I57wbNSMsTEz7j+lRi7z7OPbMh4+AMI1kys=</latexit>
MLE recovers a constant fraction of the support of X i↵ > C | ln ⇢|
⇢ .<latexit sha1_base64="M8HoApg3dBZlTLYJIWcUjk20uLQ=">AAADKnicjVHBThRBEC0GFUTUFY9eKu6acNrMLgc9GSIh4aAJJi5swhDS09vDduidnnT3kJBl/4g/4caFEK8e9OgVLla1Q6ISoz2ZmVev3qvu6soro31I06u5ZP7e/QcLiw+XHi0/fvK09Wxlx9vaSTWQ1lg3zIVXRpdqEHQwalg5JSa5Ubv50Qbnd4+V89qWn8JJpfYn4rDUhZYiEHXQUh/eb6JT0rIIBUpb+iDKgIUTkiVoCwxjhb6uKusCh50sL3DYQV0wNrTZSOBb3MCMTdPTzJSYubE9nU35N+t0D1rttJvGhXdBrwFtaNa2bV1CBiOwIKGGCSgoIRA2IMDTswc9SKEibh+mxDlCOuYVzGCJvDWpFCkEsUf0PaRor2FLirmmj25Juxh6HTkRXpHHks4R5t0w5utYmdm/1Z7Gmny2E/rnTa0JsQHGxP7Ld6v8Xx/3EqCAN7EHTT1VkeHuZFOljrfCJ8dfugpUoSKO8YjyjrCMztt7xujxsXe+WxHzX6OSWY5lo63hG5+SBtz7c5x3wU6/21vr9j/22+vvmlEvwgt4Cas0z9ewDluwDQOqfQ7f4RpukrPkIrlKPv+UJnON5zn8tpIvPwAPebad</latexit>
= 1<latexit sha1_base64="qFglFRl5V+IA2gC5aE1LqSuQEAw=">AAACy3icjVHLSsNAFD3GV62vqks3wSK4KokKuhGKbtwIFewDbJFJOq2heTGZCLW69Afc6n+Jf6B/4Z1xCmoRnZDkzLnn3Jl7r5eGQSYd53XKmp6ZnZsvLBQXl5ZXVktr640syYXP634SJqLlsYyHQczrMpAhb6WCs8gLedMbnKh484aLLEjiCzlMeSdi/TjoBT6TRLXafRZF7Mi9KpWdiqOXPQlcA8owq5aUXtBGFwl85IjAEUMSDsGQ0XMJFw5S4joYEScIBTrOcY8ieXNScVIwYgf07dPu0rAx7VXOTLt9OiWkV5DTxjZ5EtIJwuo0W8dznVmxv+Ue6ZzqbkP6eyZXRKzENbF/+cbK//pULRI9HOoaAqop1YyqzjdZct0VdXP7S1WSMqTEKdyluCDsa+e4z7b2ZLp21Vum429aqVi19402x7u6JQ3Y/TnOSdDYrbh7ld3z/XL12Iy6gE1sYYfmeYAqTlFDXc/xEU94ts6szLq17j6l1pTxbODbsh4+AAnqkhw=</latexit>
Replica picture valid when ⇢n = ⇥(n ) with 2 [0, 1/6).<latexit sha1_base64="DmgcLFf/+9pNk8Sk0Om4CtZd8xc=">AAADE3icjVFBT9RAFP6oCggCqx69TNg1gUSWFiJwISF48WSAsECyxc10dqATutNmOoWQDT/Df+LNG/Hq2ehVD/gveDOUBCVGp2n7ve+975t585IiU6UNw28jwYOHj0bHxh9PTD6Zmp5pPH22V+aVEbIj8iw3BwkvZaa07FhlM3lQGMkHSSb3k5M3Lr9/Kk2pcr1rzwt5OODHWh0pwS1Rvca7HUnbCM4KJWxlJDvlmeqzs1Rq1opNmvf0erybSsvn9PvhQpwQuphvsTNlUypwYax0N3wVLa7Mt9q9RjNsh36x+yCqQRP12sobXxGjjxwCFQaQ0LCEM3CU9HQRIURB3CGGxBlCyuclLjBB2oqqJFVwYk/oe0xRt2Y1xc6z9GpBu2T0GlIyvCRNTnWGsNuN+XzlnR37N++h93RnO6d/UnsNiLVIif2X7rbyf3WuF4sjrPkeFPVUeMZ1J2qXyt+KOzm705Ulh4I4h/uUN4SFV97eM/Oa0vfu7pb7/JWvdKyLRV1b4Zc7JQ04+nOc98HeUjtabr/eXmpubNajHscLzGKO5rmKDbzFFjrk/RHf8QM/gw/Bp+Ay+HxTGozUmuf4bQVfrgG+qazR</latexit>
All-or-nothing transition:
a generic phenomenon?
Error
SNR
All-or-nothing in sparse PCA: Wishart case
W =
r
n
n
U ⌦ V + Z
<latexit sha1_base64="QQmmu/mvgT+bHyK0p3O/5TzzQag=">AAADBXicjVHLShxBFD22eaiJycQs3TQZhEBg6MmDZBMYdJOlQmZGtGWorqkxhf1KVXVAml7nT9xlF9zmB7JNxD8wf+GtawuJIqaa7jr33HtO162blKm2LopO54L5O3fv3V9YXHrwcPnR486TlZEtKiPVUBZpYbYTYVWqczV02qVquzRKZEmqxsnBhs+PvyhjdZF/dIel2svEfq5nWgpH1KQzqONkFo6b97H9bFwdz4yQdZySwVRM8qbOm4Yrhk1cOJ0pG3I4al7wvtNMOt2oF/EKr4N+C7po12bROUGMKQpIVMigkMMRTiFg6dlFHxFK4vZQE2cIac4rNFgibUVViioEsQf03adot2Vzir2nZbWkv6T0GlKGWCNNQXWGsP9byPmKnT17k3fNnv5sh7QnrVdGrMMnYm/TXVb+r8734jDDO+5BU08lM7472bpUfCv+5OFfXTlyKInzeEp5Q1iy8vKeQ9ZY7t3freD8GVd61seyra3wx5+SBty/Os7rYPSy13/Ve7P1ujtYb0e9gFU8w3Oa51sM8AGbGJL3EX7iF34HX4Nvwffg+KI0mGs1T/HPCn6cAydXqnI=</latexit>
lim
n!1
1
m2
MMSE(U ⌦ U|W)
<latexit sha1_base64="rQcaoCEbwAEhn6NlOtX6kY10PPs=">AAADDnicjVFNT9VAFD1U+RABn7p0M/HFBDYvLWhkSTQmuiDB6OORUHhphylMXttpplOTl9r/4D9x58649Q8Y40a3+i+8Mw6JQAhM0865595zOnduWuWyNmH4Yya4cXN2bn7h1uLtpeWVO72793Zr1WguhlzlSu+lSS1yWYqhkSYXe5UWSZHmYpROntv86J3QtVTlWzOtxEGRHJcykzwxRI17r+JcFuO2jI1isSwzM+3iTCe8jbq2OFzv2lgXbHv7zYtutY3TjA27WBlZiJr58L3bR93auNcPB6Fb7CKIPOjDrx3V+44YR1DgaFBAoIQhnCNBTc8+IoSoiDtAS5wmJF1eoMMiaRuqElSREDuh7zFF+54tKbaetVNz+ktOryYlwyPSKKrThO3fmMs3ztmyl3m3ztOebUp76r0KYg1OiL1Kd1p5XZ3txSDDputBUk+VY2x33Ls07lbsydl/XRlyqIiz+IjymjB3ytN7Zk5Tu97t3SYu/9tVWtbG3Nc2+GNPSQOOzo/zIthdH0QbgyevH/e3nvlRL+ABHmKV5vkUW3iJHQzJ+yO+4Sd+BR+CT8Hn4Mu/0mDGa+7jzAq+/gXGKq1/</latexit>
lim
n!1
1
(n⇢)2
MMSE(V ⌦ V|W)
<latexit sha1_base64="4egtK9Z9u3TX01Q29gJAntlvm+A=">AAADFHicjVFNb9QwEH0NX6V8dIEjF4sV0vaySgoVHCtQJS4VRbC7leqySlyntZrEkeMgrUL+Rv8JN26IK1eEuIIE/4KxcSWgQuAo8Zs38148nqwuVGPj+NNSdO78hYuXli+vXLl67frq4MbNaaNbI+RE6EKb3SxtZKEqObHKFnK3NjIts0LOsuPHLj97JU2jdPXCLmq5X6aHlcqVSC1R88FTXqhy3lXcasZVldtFz3OTii7pu1HFzZFee7ned9yUbHv7+VY/6niWs2nPtVWlbFgIX/t91q/NB8N4HPvFzoIkgCHC2tGDj+A4gIZAixISFSzhAikaevaQIEZN3D464gwh5fMSPVZI21KVpIqU2GP6HlK0F9iKYufZeLWgvxT0GlIy3CWNpjpD2P2N+XzrnR37N+/Oe7qzLWjPgldJrMURsf/SnVb+r871YpHjoe9BUU+1Z1x3Iri0/lbcydkvXVlyqIlz+IDyhrDwytN7Zl7T+N7d3aY+/81XOtbFItS2+O5OSQNO/hznWTBdHyf3xhvP7g83H4VRL+M27mBE83yATTzBDibk/Qaf8QVfo5PobfQuev+zNFoKmlv4bUUffgD9na+0</latexit>
Ui ⇠ N(0, 1) i = 1 . . . , m
Vj ⇠ Ber(⇢) i = 1 . . . , n
Zij ⇠ N(0, 1)<latexit sha1_base64="4c4HJih7GNY/srfVTqFV2E4NQtI=">AAADN3icjVFdTxQxFL07frCuX6s++tKwkawJ2cyoRF9MCL7wRCBhFyIlk063QKEzHdqOCZnsD/OXqE++EXj0Dxhua0nQjdFOZub03HNOe9uiVtK6NP3WSW7dvnN3oXuvd//Bw0eP+0+eTqxuDBdjrpU2uwWzQslKjJ10SuzWRrCyUGKnOPng6zufhLFSV9vurBb7JTus5IHkzCGV983SOJfUyrKlnCmyMRumy9lLQk8bNiXyfUbVVDu7XFLaW5rkx15JWmpKsibMbEjNkZ4XV178MW/l8YxEw43sXt4fpKM0DDIPsggGEMem7n8BClPQwKGBEgRU4BArYGDx2YMMUqiR24cWOYNIhrqAGfTQ26BKoIIhe4LfQ5ztRbbCuc+0wc1xFYWvQSeBF+jRqDOI/Wok1JuQ7Nm/Zbch0+/tDP9FzCqRdXCE7L9818r/9fleHBzAu9CDxJ7qwPjueExpwqn4nZMbXTlMqJHzeIp1g5gH5/U5k+CxoXd/tizUL4LSs37Oo7aBS79LvODsz+ucB5NXo+z1aGXrzWB1LV51F57DIgzxPt/CKqzDJowx+yv87Cx0usnn5Htynlz8kiad6HkGv43kxxUAirct</latexit>
IT(⇢) !
s
4
| ln ⇢|
↵⇢<latexit sha1_base64="5GWkqPGzV16w66Ro/Q2kIfYGKM0=">AAADBnicjVFNT9wwEH2kH9AtbZdy7CVihQSXVbbdih6hXNrbVmIBCaOV4/WyEU6cOk4lFHLvP+mtN9Qrf6DXrvoPyr9gbIJUQBV1lOTNm3nPHk+cq6SwUfR7Lnjw8NHj+YUnraeLz56/aC+93C10aYQcCq202Y95IVWSyaFNrJL7uZE8jZXci4+3XX7vizRForMde5LLw5QfZckkEdwSNWpvMUXFYz6qmEnDjzv1GjNTvc6sDlnx2diqzyaGi+qUqSx0qdO6YlzlU+6jum6N2p2oG/kV3gW9BnTQrIFuz8AwhoZAiRQSGSxhBY6CngP0ECEn7hAVcYZQ4vMSNVqkLalKUgUn9pi+RxQdNGxGsfMsvFrQLopeQ8oQq6TRVGcIu91Cny+9s2P/5V15T3e2E/rHjVdKrMWU2Pt015X/q3O9WEzwzveQUE+5Z1x3onEp/a24k4d/dWXJISfO4THlDWHhldf3HHpN4Xt3d8t9/o+vdKyLRVNb4sKdkgbcuz3Ou2D3dbf3pvv2U7+z+b4Z9QJeYQVrNM8NbOIDBhiS9zf8xC/Mgq/B9+As+HFVGsw1mmXcWMH5JUR1qnw=</latexit>
IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit>
IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit>
Generalizationerror
W = sign( X)<latexit sha1_base64="WX5GPt3XTil6001t4w8zQyKp+Ms=">AAAC8XicjVHLSsQwFD3W93vUpZviICjI0FFBN4LoxuUIjjNgRdpOZgz2RZIKQ+lPuHMnbv0Bt/oT4h/oX3gTK/hANKXNybnnnOYmfhpyqRznecAaHBoeGR0bn5icmp6ZrczNH8skEwFrBkmYiLbvSRbymDUVVyFrp4J5kR+yln+xr+utSyYkT+Ij1U/ZaeT1Yt7lgaeIOqus5a7ftVvFTu6KyJa8Fxcrrp+EHdmPaMrdxjkvjKZdrJ5Vqk7NMcP+CeolqKIcjaTyBBcdJAiQIQJDDEU4hAdJzwnqcJASd4qcOEGImzpDgQnyZqRipPCIvaBvj1YnJRvTWmdK4w7oLyG9gpw2lsmTkE4Q1n+zTT0zyZr9LTs3mXpvfZr9MisiVuGc2L98H8r/+nQvCl1smx449ZQaRncXlCmZORW9c/tTV4oSUuI07lBdEA6M8+OcbeORpnd9tp6pvxilZvU6KLUZXvUu6YLr36/zJzher9U3auuHm9XdvfKqx7CIJazQfW5hFwdooEnZV7jHAx4taV1bN9btu9QaKD0L+DKsuzd5SqGp</latexit>
X
±1<latexit sha1_base64="nmXQk1ldbUwYZLkkbnpMKZHj/ZE=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVZIq6LLoRlxVsLXQFknSaR2axzCZKKW48Qfc6peJf6B/4Z0xBbWITkhy5tx7zsy91xchT5XjvBasufmFxaXicmlldW19o7y51UqTTAasGSRhItu+l7KQx6ypuApZW0jmRX7IrvzRqY5f3TKZ8iS+VGPBepE3jPmAB54iqtkVke1elytO1THLngVuDirIVyMpv6CLPhIEyBCBIYYiHMJDSk8HLhwI4nqYECcJcRNnuEeJtBllMcrwiB3Rd0i7Ts7GtNeeqVEHdEpIrySljT3SJJQnCevTbBPPjLNmf/OeGE99tzH9/dwrIlbhhti/dNPM/+p0LQoDHJsaONUkDKOrC3KXzHRF39z+UpUiB0Gcxn2KS8KBUU77bBtNamrXvfVM/M1kalbvgzw3w7u+JQ3Y/TnOWdCqVd2Dau3isFI/yUddxA52sU/zPEIdZ2igSd4cj3jCs3VuCevOGn+mWoVcs41vy3r4AJIdkLs=</latexit>
sign(·)<latexit sha1_base64="ychA8oOiRO4WejO0Ftjv52RsrFk=">AAAC1nicjVHLSsNAFD2Nr/quunQTLIJuSqqCLotuXFawD2hLSaZjHZoXk4lSiu7ErT/gVj9J/AP9C++MKahFdEKSM+fec2buvV7si0Q5zmvOmpqemZ3Lzy8sLi2vrBbW1utJlErGayzyI9n03IT7IuQ1JZTPm7HkbuD5vOENTnS8ccVlIqLwXA1j3gncfiguBHMVUd3C2qgtAzsR/fBmp816kdrtFopOyTHLngTlDBSRrWpUeEEbPURgSBGAI4Qi7MNFQk8LZTiIietgRJwkJEyc4wYLpE0pi1OGS+yAvn3atTI2pL32TIya0Sk+vZKUNrZJE1GeJKxPs008Nc6a/c17ZDz13Yb09zKvgFiFS2L/0o0z/6vTtShc4MjUIKim2DC6Opa5pKYr+ub2l6oUOcTEadyjuCTMjHLcZ9toElO77q1r4m8mU7N6z7LcFO/6ljTg8s9xToL6Xqm8X9o7OyhWjrNR57GJLezQPA9RwSmqqJH3NR7xhGerad1ad9b9Z6qVyzQb+Lashw+MtJY1</latexit>
{ µ, Yµ}data<latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit>
0-1 transition in the perceptron generalization performance
↵IT(⇢) !
⇢| ln ⇢|
ln 2<latexit sha1_base64="uIQF3NEkW1i+d5bBLeCJJC88RCU=">AAAC+3icjVHBTttAEH24pQ0pbQMcuViNkOglcqAtHCO40FsqEYiEUbReNomVtddarysh4z/prbeKa3+g1/Ze8Qf0Lzq7dSRoVJW1bL95M+/tzk6UyTg3QXCz5D16vPzkaWOl+Wz1+YuXrbX1k1wVmosBV1LpYcRyIeNUDExspBhmWrAkkuI0mh3a/OlHofNYpcfmMhPnCZuk8TjmzBA1ar0LmcymbFSGOvHfH1fboZ6q16FRfjjWjJc2vApl6jtQlRbuVM1Rqx10Arf8RdCtQRv16qvWT4S4gAJHgQQCKQxhCYacnjN0ESAj7hwlcZpQ7PICFZqkLahKUAUjdkbfCUVnNZtSbD1zp+a0i6RXk9LHFmkU1WnCdjff5QvnbNl/eZfO057tkv5R7ZUQazAl9n+6eeVDdbYXgzH2XQ8x9ZQ5xnbHa5fC3Yo9uX+nK0MOGXEWX1BeE+ZOOb9n32ly17u9W+byt67SsjbmdW2BX/aUNODu3+NcBCc7ne5u5+2HN+3eQT3qBjbxCts0zz30cIQ+BuT9Cd/wHT+8yvvsffGu/5R6S7VmA/eW9/U3mT2lZw==</latexit>
Proof idea: adaptive interpolation method [B. Macris 16]
(
Wij(t) =
q
(1 t) n
n XiXj + Zij , 1  i < j  n ,
fWi(t, ✏) =
p
Rn(t, ✏) Xi + eZi , 1  i  n ,<latexit sha1_base64="Cg3NYN1mLrUUqg5ElqB/esfJD9o=">AAADqXicjVFdb9MwFL1Z+Bjhq4NHXiwqUKuVKhlI4wGkCV4QTxuia7V5ihLH7dy5Tpo4oBLldyLeeYB/wbXnaoMJgaMk1+fec46vb1pIUekw/OZt+Neu37i5eSu4fefuvfudrQeHVV6XjI9YLvNykiYVl0LxkRZa8klR8mSRSj5Oz96a/PgTLyuRq496VfCTRTJTYipYohGKO99pymdCNQw1qjYYx42Ytz3dJ09f02pZ6qYXPdN9Oi0T1lCJulkSq7ZRbUsHZBKLSTwn2+TI0uhgQOhyWScZiajkRLwic/NXmKA0oJ9FxrWQGW/GbSx6ekB5UQmZqz5BO3Lu9yFWlzMoii5occFGN3Fh5Qyps1z7BZSrzHVF4k43HIZ2katB5IIuuLWfd74ChQxyYFDDAjgo0BhLSKDC5xgiCKFA7AQaxEqMhM1zaCFAbo1VHCsSRM/wO8PdsUMV7o1mZdkMXSS+JTIJPEFOjnUlxsaN2HxtlQ36N+3GapqzrfCfOq0FohpOEf0Xb135vzzTi4YpvLQ9COypsIjpjjmV2t6KOTm51JVGhQIxE2eYLzFmlrm+Z2I5le3d3G1i8z9spUHNnrnaGn6aU+KAoz/HeTU43BlGz4c7By+6e2/cqDfhETyGHs5zF/bgHezDCJj33iu8lffF3/YP/Il/dF664TnOQ/ht+ewXdGLeQQ==</latexit>
Interpolating model:
in(t, ✏) ⌘
1
n
I(X; W(t), fW(t, ✏))
<latexit sha1_base64="lG/fuMgZHpabDG/n3b0IKlOYje0=">AAADG3icjVFPS9xAHH2mWv/XtT16CS6WXZAlUUGhl6W9tDcF9w8YWZLZWR2cTeJkoixhP0q/SW+9lV5L6dWTpX4IfzNGWxWxE5J5837vvcxvJkqlyLTn/ZpwXkxOvZyemZ2bX1h8tVRZft3Oklwx3mKJTFQ3CjMuRcxbWmjJu6ni4TCSvBOdfDD1zhlXmUjifT1K+eEwPIrFQLBQE9WrtEWviMc1vR7wNBMyietvA36ai7NgoEJW+GOqfqoVQTRwu+N3du6QvL4enIs+10L2+R35N6Peq1S9hmeH+xj4JaiiHLtJ5ScC9JGAIccQHDE0YYkQGT0H8OEhJe4QBXGKkLB1jjHmyJuTipMiJPaEvke0OijZmNYmM7NuRn+R9CpyulgjT0I6Rdj8zbX13CYb9qnswmaavY1ojsqsIbEax8Q+57tV/q/P9KIxwI7tQVBPqWVMd6xMye2pmJ27/3SlKSElzuA+1RVhZp235+xaT2Z7N2cb2vqlVRrWrFmpzfHb7JIu2H94nY9Be6PhbzY29raqzfflVc9gBauo0X1uo4mP2EWLsr/gAn9w5Xx2vjrfnO83Umei9LzBveH8uAbHl7IS</latexit>
in(0, ✏) = in(1, ✏)
Z 1
0
dt
d
dt
in(t, ✏)
<latexit sha1_base64="mO6HXMT4JTGeFWNiFq05NbH+GSM=">AAADEXicjVHLSsQwFD3W93vUpZviICjo0KqgG0F048KFgqOCo0ObZjTYaUuaClLmK/wTd+7ErR+gLnWlf+FNpuIL0ZS2J+eec5Kb+EkoUuU4jx1WZ1d3T29f/8Dg0PDIaGlsfC+NM8l4lcVhLA98L+WhiHhVCRXyg0Ryr+mHfN8/29D1/XMuUxFHu+oi4UdN7yQSDcE8RVS9tCXqedSaceZqPElFGEezq23G/WDmayJSdefYDZRda0iP5UErD1SrLVQfwnqp7FQcM+yfwC1AGcXYjkv3qCFADIYMTXBEUIRDeEjpOYQLBwlxR8iJk4SEqXO0MEDejFScFB6xZ/Q9odlhwUY015mpcTNaJaRXktPGNHli0knCejXb1DOTrNnfsnOTqfd2QX+/yGoSq3BK7F++d+V/fboXhQZWTA+CekoMo7tjRUpmTkXv3P7UlaKEhDiNA6pLwsw438/ZNp7U9K7P1jP1F6PUrJ6zQpvhVe+SLtj9fp0/wd5CxV2sLOwsldfWi6vuwySmMEP3uYw1bGIbVcq+wgOe8GxdWtfWjXXbllodhWcCX4Z19wbyiq5L</latexit>
1
n
I(X; W) = ipot
n Rn(t, ✏); n, ⇢n + Remainder
⇣
Rn(t, ✏),
1
n
EhX · xit,✏
⌘
<latexit sha1_base64="Qof9htjpZRZsAU57rA1eF30B1BM=">AAADk3icjVFdb9MwFL1Z+BjjYx2IJ14sKlArqqoZk4ZUIU0FJHhAGhNdK+YROY7bWkvsyHEQU5S/yCviH8C/4NrLxMdA4CjJuefec3yvnRSZLO1o9CVYCy9dvnJ1/drG9Rs3b212tm4flroyXEy5zrSZJ6wUmVRiaqXNxLwwguVJJmbJyTOXn30QppRavbWnhTjO2VLJheTMIhV3PtGFYTxS5FWvpsmCzJux/8+aPnn4lBAiY/W+piYnhbYNTeSydxDXqunZARVFKTOt+mOa4YYpi9WAmpWOlSvrk0dediByJlUqTEMnfxIPfAN11CBPc2ZXSVK/aNBRLTNB2p4I5am2Z9FHjIzPxvUPHyTRvh93uqPhyC9yEUQt6EK79nXnM1BIQQOHCnIQoMAizoBBic8RRDCCArljqJEziKTPC2hgA7UVVgmsYMie4HeJ0VHLKoydZ+nVHHfJ8DWoJPAANRrrDGK3G/H5yjs79m/etfd0vZ3iP2m9cmQtrJD9l+688n91bhYLC3jiZ5A4U+EZNx1vXSp/Kq5z8tNUFh0K5BxOMW8Qc688P2fiNaWf3Z0t8/mvvtKxLuZtbQXfXJd4wdHv13kRHG4Po8fD7Tc73b1Je9XrcA/uQw/vcxf24CXswxR4sBO8C3iQhnfDcTgJn5+VrgWt5g78ssLX3wFQhtic</latexit>
Sum rule:
Rn(t, ✏) = ✏ + nt ⇥ argminq i(pot)
(q; n, ⇢n)
)
1
n
I(X; W) . inf
q 0
ipot
n (q; n, ⇢n)
<latexit sha1_base64="Husz+qtKzIoHMOlptlG6LYPDcLU=">AAADmnicjVFdb9MwFL1Z+Bjlq4NHeLComFpRVemYBFKFNI0XJl7GtK6V5i1yUjc1TZzUdkBTlJ+JEP8A/gXXJp2AgcBRkuNz7znX1zcqUqFNEHzxNvxr12/c3LzVun3n7r377a0HJzovVczHcZ7mahoxzVMh+dgIk/JpoTjLopRPouVrG5984EqLXB6bi4KfZSyRYi5iZpAK25+2j8JK1l3Tp7zQIs1l79UaPaMpGs1YKImhRmRcExqJpEsqqjLCVJIJWYcr2ifivOo6sshN3au7q9GltE/VIg9lzyp7lLa26ZFIFoYplX8kdK5YTIaSHKA8mpNpPXL/Sd0jNOVaa5GRFqFCzsNqRRNOgtqWC+X5Zbk/FQvbnWAQuEWugmEDOtCsw7z9GSjMIIcYSsiAgwSDOAUGGp9TGEIABXJnUCGnEAkX51BDC7UlZnHMYMgu8Zvg7rRhJe6tp3bqGKuk+CpUEniKmhzzFGJbjbh46Zwt+zfvynnas13gP2q8MmQNLJD9l26d+b8624uBObx0PQjsqXCM7S5uXEp3K/bk5KeuDDoUyFk8w7hCHDvl+p6J02jXu71b5uJfXaZl7T5uckv4Zk+JAx7+Ps6r4GRnMHw+2Hm329nbb0a9CY/gCXRxni9gD97AIYwh9kYe8957S/+xv+8f+G9/pG54jeYh/LL84+8rbNpA</latexit>
Upper bound
« trivial path »
d
dt
Rn(t, ✏) = E
D 1
n
X · x
E
t,✏
(Rn(t, ✏))
)
1
n
I(X; W) & inf
q 0
ipot
n (q; n, ⇢n)
<latexit sha1_base64="9L/XtJe5/TRni1/uA4AES/h/Qss=">AAADrHicjVFri9QwFL3d+ljH16gf/RIcXGZgGDqroLAIy4qgH4R12dkdmIwlbdNO2Dbppqm61P5O8Reo/8Kb2MHHIJrS9t5zzznJzY3KXFQmCD57W/6ly1eubl/rXb9x89bt/p27J5WqdcxnscqVnkes4rmQfGaEyfm81JwVUc5Po7Pntn76jutKKHlsLkq+LFgmRSpiZhAK+192aKpZ3CRtkxjSHoWNbIdmTHlZiVzJEXlGaMHMKoqaFy09EBnNmcxy7lRkKhsapWTeEhonyhCXfcDMErUjhs1Pt5YMNzcYUdrboUciWxmmtXpP1tbk1bBz33P/03ZEaGawl4JQIdOwOacZJ0FLx0SE8m1DdUFKZdrh+R6esogSFsox1SsVylHYHwSTwC2yGUy7YADdOlT9T0AhAQUx1FAABwkG4xwYVPgsYAoBlIgtoUFMYyRcnUMLPdTWyOLIYIie4TfDbNGhEnPrWTl1jLvk+GpUEniIGoU8jbHdjbh67Zwt+jfvxnnas13gP+q8CkQNrBD9l27N/F+d7cVACk9dDwJ7Kh1iu4s7l9rdij05+aUrgw4lYjZOsK4xjp1yfc/EaSrXu71b5upfHdOiNo87bg3f7ClxwNM/x7kZnOxOpo8mu28eD/YPulFvw314AEOc5xPYh5dwCDOIvdde5X30Wn/iH/sLf/mDuuV1mnvw2/LT7+ak4h4=</latexit>
Lower bound
« smart path »
Thank
You
• Statistical physics valid in corners of phase diagrams.
• Interpolation techniques allow to naturally catch the finite
size corrections.
• « All-or-nothing » seems quite generic: « Universal »
phenomenon in reconstruction of very sparse information
(i.e., with « effective low dimension ») from much higher-
dimensional data?
• What about more complex models (multi layers etc)?
• « All-or-nothing » for algorithms (AMP, gradient-based: on-
going)?

More Related Content

More from The Statistical and Applied Mathematical Sciences Institute

More from The Statistical and Applied Mathematical Sciences Institute (20)

Causal Inference Opening Workshop - Latent Variable Models, Causal Inference,...
Causal Inference Opening Workshop - Latent Variable Models, Causal Inference,...Causal Inference Opening Workshop - Latent Variable Models, Causal Inference,...
Causal Inference Opening Workshop - Latent Variable Models, Causal Inference,...
 
Causal Inference Opening Workshop - Causal Discovery in Neuroimaging Data - F...
Causal Inference Opening Workshop - Causal Discovery in Neuroimaging Data - F...Causal Inference Opening Workshop - Causal Discovery in Neuroimaging Data - F...
Causal Inference Opening Workshop - Causal Discovery in Neuroimaging Data - F...
 
Causal Inference Opening Workshop - Smooth Extensions to BART for Heterogeneo...
Causal Inference Opening Workshop - Smooth Extensions to BART for Heterogeneo...Causal Inference Opening Workshop - Smooth Extensions to BART for Heterogeneo...
Causal Inference Opening Workshop - Smooth Extensions to BART for Heterogeneo...
 
Causal Inference Opening Workshop - A Bracketing Relationship between Differe...
Causal Inference Opening Workshop - A Bracketing Relationship between Differe...Causal Inference Opening Workshop - A Bracketing Relationship between Differe...
Causal Inference Opening Workshop - A Bracketing Relationship between Differe...
 
Causal Inference Opening Workshop - Testing Weak Nulls in Matched Observation...
Causal Inference Opening Workshop - Testing Weak Nulls in Matched Observation...Causal Inference Opening Workshop - Testing Weak Nulls in Matched Observation...
Causal Inference Opening Workshop - Testing Weak Nulls in Matched Observation...
 
Causal Inference Opening Workshop - Difference-in-differences: more than meet...
Causal Inference Opening Workshop - Difference-in-differences: more than meet...Causal Inference Opening Workshop - Difference-in-differences: more than meet...
Causal Inference Opening Workshop - Difference-in-differences: more than meet...
 
Causal Inference Opening Workshop - New Statistical Learning Methods for Esti...
Causal Inference Opening Workshop - New Statistical Learning Methods for Esti...Causal Inference Opening Workshop - New Statistical Learning Methods for Esti...
Causal Inference Opening Workshop - New Statistical Learning Methods for Esti...
 
Causal Inference Opening Workshop - Bipartite Causal Inference with Interfere...
Causal Inference Opening Workshop - Bipartite Causal Inference with Interfere...Causal Inference Opening Workshop - Bipartite Causal Inference with Interfere...
Causal Inference Opening Workshop - Bipartite Causal Inference with Interfere...
 
Causal Inference Opening Workshop - Bridging the Gap Between Causal Literatur...
Causal Inference Opening Workshop - Bridging the Gap Between Causal Literatur...Causal Inference Opening Workshop - Bridging the Gap Between Causal Literatur...
Causal Inference Opening Workshop - Bridging the Gap Between Causal Literatur...
 
Causal Inference Opening Workshop - Some Applications of Reinforcement Learni...
Causal Inference Opening Workshop - Some Applications of Reinforcement Learni...Causal Inference Opening Workshop - Some Applications of Reinforcement Learni...
Causal Inference Opening Workshop - Some Applications of Reinforcement Learni...
 
Causal Inference Opening Workshop - Bracketing Bounds for Differences-in-Diff...
Causal Inference Opening Workshop - Bracketing Bounds for Differences-in-Diff...Causal Inference Opening Workshop - Bracketing Bounds for Differences-in-Diff...
Causal Inference Opening Workshop - Bracketing Bounds for Differences-in-Diff...
 
Causal Inference Opening Workshop - Assisting the Impact of State Polcies: Br...
Causal Inference Opening Workshop - Assisting the Impact of State Polcies: Br...Causal Inference Opening Workshop - Assisting the Impact of State Polcies: Br...
Causal Inference Opening Workshop - Assisting the Impact of State Polcies: Br...
 
Causal Inference Opening Workshop - Experimenting in Equilibrium - Stefan Wag...
Causal Inference Opening Workshop - Experimenting in Equilibrium - Stefan Wag...Causal Inference Opening Workshop - Experimenting in Equilibrium - Stefan Wag...
Causal Inference Opening Workshop - Experimenting in Equilibrium - Stefan Wag...
 
Causal Inference Opening Workshop - Targeted Learning for Causal Inference Ba...
Causal Inference Opening Workshop - Targeted Learning for Causal Inference Ba...Causal Inference Opening Workshop - Targeted Learning for Causal Inference Ba...
Causal Inference Opening Workshop - Targeted Learning for Causal Inference Ba...
 
Causal Inference Opening Workshop - Bayesian Nonparametric Models for Treatme...
Causal Inference Opening Workshop - Bayesian Nonparametric Models for Treatme...Causal Inference Opening Workshop - Bayesian Nonparametric Models for Treatme...
Causal Inference Opening Workshop - Bayesian Nonparametric Models for Treatme...
 
2019 Fall Series: Special Guest Lecture - Adversarial Risk Analysis of the Ge...
2019 Fall Series: Special Guest Lecture - Adversarial Risk Analysis of the Ge...2019 Fall Series: Special Guest Lecture - Adversarial Risk Analysis of the Ge...
2019 Fall Series: Special Guest Lecture - Adversarial Risk Analysis of the Ge...
 
2019 Fall Series: Professional Development, Writing Academic Papers…What Work...
2019 Fall Series: Professional Development, Writing Academic Papers…What Work...2019 Fall Series: Professional Development, Writing Academic Papers…What Work...
2019 Fall Series: Professional Development, Writing Academic Papers…What Work...
 
2019 GDRR: Blockchain Data Analytics - Machine Learning in/for Blockchain: Fu...
2019 GDRR: Blockchain Data Analytics - Machine Learning in/for Blockchain: Fu...2019 GDRR: Blockchain Data Analytics - Machine Learning in/for Blockchain: Fu...
2019 GDRR: Blockchain Data Analytics - Machine Learning in/for Blockchain: Fu...
 
2019 GDRR: Blockchain Data Analytics - QuTrack: Model Life Cycle Management f...
2019 GDRR: Blockchain Data Analytics - QuTrack: Model Life Cycle Management f...2019 GDRR: Blockchain Data Analytics - QuTrack: Model Life Cycle Management f...
2019 GDRR: Blockchain Data Analytics - QuTrack: Model Life Cycle Management f...
 
2019 GDRR: Blockchain Data Analytics - Modeling Cryptocurrency Markets with T...
2019 GDRR: Blockchain Data Analytics - Modeling Cryptocurrency Markets with T...2019 GDRR: Blockchain Data Analytics - Modeling Cryptocurrency Markets with T...
2019 GDRR: Blockchain Data Analytics - Modeling Cryptocurrency Markets with T...
 

Recently uploaded

Like-prefer-love -hate+verb+ing & silent letters & citizenship text.pdf
Like-prefer-love -hate+verb+ing & silent letters & citizenship text.pdfLike-prefer-love -hate+verb+ing & silent letters & citizenship text.pdf
Like-prefer-love -hate+verb+ing & silent letters & citizenship text.pdfMr Bounab Samir
 
Introduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher EducationIntroduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher Educationpboyjonauth
 
Full Stack Web Development Course for Beginners
Full Stack Web Development Course  for BeginnersFull Stack Web Development Course  for Beginners
Full Stack Web Development Course for BeginnersSabitha Banu
 
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...Nguyen Thanh Tu Collection
 
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPTECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPTiammrhaywood
 
Keynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-designKeynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-designMIPLM
 
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxMULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxAnupkumar Sharma
 
ROOT CAUSE ANALYSIS PowerPoint Presentation
ROOT CAUSE ANALYSIS PowerPoint PresentationROOT CAUSE ANALYSIS PowerPoint Presentation
ROOT CAUSE ANALYSIS PowerPoint PresentationAadityaSharma884161
 
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptxECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptxiammrhaywood
 
ACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdfACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdfSpandanaRallapalli
 
How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17Celine George
 
What is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERPWhat is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERPCeline George
 
Proudly South Africa powerpoint Thorisha.pptx
Proudly South Africa powerpoint Thorisha.pptxProudly South Africa powerpoint Thorisha.pptx
Proudly South Africa powerpoint Thorisha.pptxthorishapillay1
 
Types of Journalistic Writing Grade 8.pptx
Types of Journalistic Writing Grade 8.pptxTypes of Journalistic Writing Grade 8.pptx
Types of Journalistic Writing Grade 8.pptxEyham Joco
 
Romantic Opera MUSIC FOR GRADE NINE pptx
Romantic Opera MUSIC FOR GRADE NINE pptxRomantic Opera MUSIC FOR GRADE NINE pptx
Romantic Opera MUSIC FOR GRADE NINE pptxsqpmdrvczh
 

Recently uploaded (20)

Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝
Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝
Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝
 
Like-prefer-love -hate+verb+ing & silent letters & citizenship text.pdf
Like-prefer-love -hate+verb+ing & silent letters & citizenship text.pdfLike-prefer-love -hate+verb+ing & silent letters & citizenship text.pdf
Like-prefer-love -hate+verb+ing & silent letters & citizenship text.pdf
 
Introduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher EducationIntroduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher Education
 
Full Stack Web Development Course for Beginners
Full Stack Web Development Course  for BeginnersFull Stack Web Development Course  for Beginners
Full Stack Web Development Course for Beginners
 
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - CẢ NĂ...
 
Raw materials used in Herbal Cosmetics.pptx
Raw materials used in Herbal Cosmetics.pptxRaw materials used in Herbal Cosmetics.pptx
Raw materials used in Herbal Cosmetics.pptx
 
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPTECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
ECONOMIC CONTEXT - LONG FORM TV DRAMA - PPT
 
Keynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-designKeynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-design
 
Model Call Girl in Bikash Puri Delhi reach out to us at 🔝9953056974🔝
Model Call Girl in Bikash Puri  Delhi reach out to us at 🔝9953056974🔝Model Call Girl in Bikash Puri  Delhi reach out to us at 🔝9953056974🔝
Model Call Girl in Bikash Puri Delhi reach out to us at 🔝9953056974🔝
 
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxMULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
 
OS-operating systems- ch04 (Threads) ...
OS-operating systems- ch04 (Threads) ...OS-operating systems- ch04 (Threads) ...
OS-operating systems- ch04 (Threads) ...
 
ROOT CAUSE ANALYSIS PowerPoint Presentation
ROOT CAUSE ANALYSIS PowerPoint PresentationROOT CAUSE ANALYSIS PowerPoint Presentation
ROOT CAUSE ANALYSIS PowerPoint Presentation
 
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptxECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
 
ACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdfACC 2024 Chronicles. Cardiology. Exam.pdf
ACC 2024 Chronicles. Cardiology. Exam.pdf
 
How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17How to Configure Email Server in Odoo 17
How to Configure Email Server in Odoo 17
 
What is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERPWhat is Model Inheritance in Odoo 17 ERP
What is Model Inheritance in Odoo 17 ERP
 
Proudly South Africa powerpoint Thorisha.pptx
Proudly South Africa powerpoint Thorisha.pptxProudly South Africa powerpoint Thorisha.pptx
Proudly South Africa powerpoint Thorisha.pptx
 
TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
TataKelola dan KamSiber Kecerdasan Buatan v022.pdfTataKelola dan KamSiber Kecerdasan Buatan v022.pdf
TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
 
Types of Journalistic Writing Grade 8.pptx
Types of Journalistic Writing Grade 8.pptxTypes of Journalistic Writing Grade 8.pptx
Types of Journalistic Writing Grade 8.pptx
 
Romantic Opera MUSIC FOR GRADE NINE pptx
Romantic Opera MUSIC FOR GRADE NINE pptxRomantic Opera MUSIC FOR GRADE NINE pptx
Romantic Opera MUSIC FOR GRADE NINE pptx
 

2019 Fall Series: Special Guest Lecture - 0-1 Phase Transitions in High Dimensional Inference and Learning - Jean Barbier, December 3, 2019

  • 1. 0-1 Phase Transitions in High-Dimensional Inference & Learning Jean Barbier International Center for Theoretical Physics, Trieste, Italy Error SNR Joint work with Nicolas Macris & Clément Luneau, EPFL
  • 2. Spiked matrix/tensor models W = q n X ⌦ X ⌦ . . . ⌦ X + Z <latexit sha1_base64="qQseBedGJiS1x3H0QMD/U8sLiZ8=">AAADInicjVFBS+QwGP2surqz7jrqZcFLcRDEhaFVRC/CoBePLjjOsHaQNJNxg2lTk1SQ0p/hwbv/xJs38SAL+wPWq/4Bv8QK7siyprR5ed97r/mSOBNcmyD4NeKNjo1/mJj8WPs09fnLdH1mdl/LXFHWplJI1Y2JZoKnrG24EaybKUaSWLBOfLxt651TpjSX6Z45y1gvIUcpH3BKDFKH9V4RxQO/U276kT5RpogGitAiEpjQJ2WRlqUTdMtIGp4w7Q8tI9GXRg8Vv7n5R1k7rDeCZuCG/xaEFWi01vbOvy5fhLuyfgcR9EEChRwSYJCCQSyAgMbnAEIIIEOuBwVyChF3dQYl1NCbo4qhgiB7jN8jXB1UbIprm6mdm+JfBL4KnT4sokeiTiG2f/NdPXfJlv1XduEy7d7OcI6rrARZAz+R/Z/vRflen+3FwAA2XA8ce8ocY7ujVUruTsXu3H/VlcGEDDmL+1hXiKlzvpyz7zza9W7Plrj6H6e0rF3TSpvDvd0lXnA4fJ1vwf5KM1xtrnwPG60teB6TMA8LsIT3uQ4t2IFdaGP2FaY+wKN36V17N97ts9QbqTxz8Nfwfj8BUHS4ig==</latexit> W = q n X(1) ⌦ X(2) ⌦ . . . ⌦ X(k) + Z <latexit sha1_base64="qJQ/YjrsM4mDdQhMZTDaBmsbUZc=">AAADNHicjVHdShwxGP12qtZu/Vnbm0Jvhi6CVlhmtojeFJb2ppcWXHepo5LJZjVsZjImGUGGPEYvfBHfpJRC70qhvekL9KZf4ojapdQMM3NyvnNO8iVpIbg2UfSlETyYmZ17OP+o+XhhcWm5tfJkT8tSUdanUkg1TIlmguesb7gRbFgoRrJUsEE6eevqgzOmNJf5rjkv2EFGjnM+5pQYpI5askrScTiwr8NEnypTJWNFaJUITBgRW+XWesHQHlZr8bpNpOEZ0+EN2b0hEzGSRk9LJut2w88+2OZRqx11Ij/CaRDXoN3b3P347OVFvCNbnyGBEUigUEIGDHIwiAUQ0PjsQwwRFMgdQIWcQsR9nYGFJnpLVDFUEGQn+D3G2X7N5jh3mdq7Ka4i8FXoDGEVPRJ1CrFbLfT10ic79l/Zlc90ezvHf1pnZcgaOEH2f75r5X19rhcDY9j2PXDsqfCM647WKaU/Fbfz8FZXBhMK5BweYV0hpt55fc6h92jfuztb4us/vNKxbk5rbQk/3S7xguO/r3Ma7HU78atO933c7r2BqzEPz+EFrOF9bkEP3sEO9DH7E/xuzDRmg8vga/At+H4lDRq15yncGcGvP/C5vQM=</latexit> [Korada Macris 09], [Deshpande Abbe Montanari 16], [B. Macris Dia Lesieur Krzakala Zdeborova 16], [Miolane Lelarge 17], [Miolane 17], [B. Macris 17], [B. Miolane Macris 18], [El Alaoui Krzakala 18], [Mourrat 19], [B. Luneau Macris 19] High-dimensional inference: what we DO know (a statistical physics perspective) lim n!1 1 n I(X; W) = inf q 0 i(pot) (q, ) lim n!1 1 n EkX ⌦ X E[X ⌦ X|W]k2 = mmse(q⇤ , ) <latexit sha1_base64="pIKHaSk8Z7lgVJpQc5YuEfzJci8=">AAADpnicjVHbjtMwEJ1suCzLrcAjLxYVUougShYQSGilFQgBL2iR6LZS3RbHdbvWxkmaOIuq1N+JeOQN/oKx8XIrCBwlGZ8553jGkxSprHQUfQy2wjNnz53fvrBz8dLlK1db164fVnldctHneZqXw4RVIpWZ6GupUzEsSsFUkopBcvzM5gcnoqxknr3Vq0KMFVtkci450whNW59oKtW0yajOqczmemUInZeMxxl51WloMidD88T9B6a7ZynTZkkXgkSGyEmDlFKRItemazrLuzTFk2esS+hyWbMZ2TB33k1smsxQxfRRkjTPDV0TfxLNtVSi8rt7PyijPxLWvrAxXU9291wpSlUCK5nc+V7LtNWOepFbZDOIfdAGvw7y1gegMIMcONSgQEAGGuMUGFT4jCCGCArExtAgVmIkXV6AgR3U1sgSyGCIHuN3gbuRRzPcW8/KqTmekuJbopLAbdTkyCsxtqcRl6+ds0X/5t04T1vbCv+J91KIajhC9F+6U+b/6mwvGubw2PUgsafCIbY77l1qdyu2cvJTVxodCsRsPMN8iTF3ytN7Jk5Tud7t3TKX/+yYFrV77rk1fLFV4oDj38e5GRzu9uL7vYdvHrT3n/pRb8NNuAUdnOcj2IeXcAB94MGLQAUnwfuwE74O++HgG3Ur8Job8MsK330FrM3jAA==</latexit> W = q n X ⌦ X + Z <latexit sha1_base64="9IeZQWNKlVBlkBsQEQH4TzFjOn8=">AAADBXicjVHNThRBGCxGEFxEVr2YeJmwITGQbGYwBC4mG7x4xIRlNzKE9PT2Ymfnz+4eEjKZs0/ggRfw5o1w9QW8qvEN5C34uhkSgRDpyUxX11dV0193XCRSmyD4M+U9mJ55ODv3qDX/eOHJYvvps12dl4qLPs+TXA1jpkUiM9E30iRiWCjB0jgRg3jy1tYHR0JpmWc75rgQ+yk7zORYcmaIOmj3qige+4P6jR/pT8pU0VgxXkUJJYxYXWV17QTDOsqNTIX2m+Wqmz/UrYN2J+gGbvi3QdiATm9958uLlZNwO2//RoQRcnCUSCGQwRBOwKDp2UOIAAVx+6iIU4SkqwvUaJG3JJUgBSN2Qt9DWu01bEZrm6mdm9NfEnoVOX0skycnnSJs/+a7eumSLXtXduUy7d6OaY6brJRYg4/E/s93pbyvz/ZiMMam60FST4VjbHe8SSndqdid+/90ZSihIM7iEdUVYe6cV+fsO492vduzZa7+1ykta9e80ZY4t7ukCw5vXudtsLvWDV9319+Hnd4WLsccXmIJr+g+N9DDO2yjT9lf8QM/8cv77H3zTr2zS6k31Xie49rwvl8A8iKsOw==</latexit> 2 groups SBM Groups asymmetry HardImpossible Easy
  • 3. W = q n X + Z <latexit sha1_base64="6uMoEBvpE9gq6pVu6P9pW95FBUY=">AAADB3icjVHLbtQwFD1NeZThNZQNEpuIERICaZQUIdggjdpNl1Op0xnRVJXj8bRWnQe2gzSK8gH9gC74A3bdoW77A922/AH8BdduKgEVAkeJj88958TXTksljY2ibwvB4o2bt24v3encvXf/wcPuo+UtU1SaixEvVKEnKTNCyVyMrLRKTEotWJYqMU4P1lx9/EloI4t8085LsZOxvVzOJGeWqN3uap2ks3DcvA8T81HbOplpxutEUcKUNXXeNElaqKmZZzTVyXBfNt4xaV75+UPT2e32on7kR3gdxC3oDd5sHj15+TkeFt1zJJiiAEeFDAI5LGEFBkPPNmJEKInbQU2cJiR9XaBBh7wVqQQpGLEH9N2j1XbL5rR2mca7Of1F0avJGeI5eQrSacLub6GvVz7ZsX/Lrn2m29uc5rTNyoi12Cf2X74r5f/6XC8WM7zzPUjqqfSM6463KZU/Fbfz8JeuLCWUxDk8pbomzL3z6pxD7zG+d3e2zNe/e6Vj3Zq32go/3C7pguM/r/M62Frpx6/7Kxtxb7CKy7GEp3iGF3SfbzHAOoYYUfYXnOEcF8FhcBx8DU4upcFC63mM30Zw+hPTKa2z</latexit> Random linear estimation Generalized linear models [Reeves Pfister 16], [B. Macris Dia Krzakala 16] [B. Miolane Macris Krzakala Zdeborovà 18] W = ' ⇣q 1 n X ⌘ <latexit sha1_base64="zK/PJNSEGcUY313MhuC/grI8fmI=">AAADCnicjVHNTtwwGBwCLXT7ty299RJ1VYleVgk90AvSiqpSj1upy65EEHK83l2L/NV2kFZR3qAnxFtw663qtS/ACRVegL5FP5sgURBqHSUezzcz8WfHRSK1CYJfC97i0r37yysPWg8fPX7ytP3s+bbOS8XFgOdJrkYx0yKRmRgYaRIxKpRgaZyIYbz/3taHB0JpmWefzbwQuymbZnIiOTNE7bU/VFE88Yf1ph8dMFXMZLQlp2uR/qJMFU0U41VYV1ldR3GejPU8pamK+jNZO9+otvI3rb12J+gGbvi3QdiATm/j6HC+evGin7dPEWGMHBwlUghkMIQTMGh6dhAiQEHcLiriFCHp6gI1WuQtSSVIwYjdp++UVjsNm9HaZmrn5vSXhF5FTh+vyZOTThG2f/NdvXTJlr0ru3KZdm9zmuMmKyXWYEbsv3xXyv/12V4MJnjnepDUU+EY2x1vUkp3Knbn/rWuDCUUxFk8proizJ3z6px959Gud3u2zNUvnNKyds0bbYnfdpd0weHN67wNtte74dvu+qew09vC5VjBS7zCGt3nBnr4iD4GlH2ME5zh3PvqffO+ez8upd5C41nFX8P7+Qfxjq9S</latexit> W = sign( X)<latexit sha1_base64="WX5GPt3XTil6001t4w8zQyKp+Ms=">AAAC8XicjVHLSsQwFD3W93vUpZviICjI0FFBN4LoxuUIjjNgRdpOZgz2RZIKQ+lPuHMnbv0Bt/oT4h/oX3gTK/hANKXNybnnnOYmfhpyqRznecAaHBoeGR0bn5icmp6ZrczNH8skEwFrBkmYiLbvSRbymDUVVyFrp4J5kR+yln+xr+utSyYkT+Ij1U/ZaeT1Yt7lgaeIOqus5a7ftVvFTu6KyJa8Fxcrrp+EHdmPaMrdxjkvjKZdrJ5Vqk7NMcP+CeolqKIcjaTyBBcdJAiQIQJDDEU4hAdJzwnqcJASd4qcOEGImzpDgQnyZqRipPCIvaBvj1YnJRvTWmdK4w7oLyG9gpw2lsmTkE4Q1n+zTT0zyZr9LTs3mXpvfZr9MisiVuGc2L98H8r/+nQvCl1smx449ZQaRncXlCmZORW9c/tTV4oSUuI07lBdEA6M8+OcbeORpnd9tp6pvxilZvU6KLUZXvUu6YLr36/zJzher9U3auuHm9XdvfKqx7CIJazQfW5hFwdooEnZV7jHAx4taV1bN9btu9QaKD0L+DKsuzd5SqGp</latexit> X ±1<latexit sha1_base64="nmXQk1ldbUwYZLkkbnpMKZHj/ZE=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVZIq6LLoRlxVsLXQFknSaR2axzCZKKW48Qfc6peJf6B/4Z0xBbWITkhy5tx7zsy91xchT5XjvBasufmFxaXicmlldW19o7y51UqTTAasGSRhItu+l7KQx6ypuApZW0jmRX7IrvzRqY5f3TKZ8iS+VGPBepE3jPmAB54iqtkVke1elytO1THLngVuDirIVyMpv6CLPhIEyBCBIYYiHMJDSk8HLhwI4nqYECcJcRNnuEeJtBllMcrwiB3Rd0i7Ts7GtNeeqVEHdEpIrySljT3SJJQnCevTbBPPjLNmf/OeGE99tzH9/dwrIlbhhti/dNPM/+p0LQoDHJsaONUkDKOrC3KXzHRF39z+UpUiB0Gcxn2KS8KBUU77bBtNamrXvfVM/M1kalbvgzw3w7u+JQ3Y/TnOWdCqVd2Dau3isFI/yUddxA52sU/zPEIdZ2igSd4cj3jCs3VuCevOGn+mWoVcs41vy3r4AJIdkLs=</latexit> sign(·)<latexit sha1_base64="ychA8oOiRO4WejO0Ftjv52RsrFk=">AAAC1nicjVHLSsNAFD2Nr/quunQTLIJuSqqCLotuXFawD2hLSaZjHZoXk4lSiu7ErT/gVj9J/AP9C++MKahFdEKSM+fec2buvV7si0Q5zmvOmpqemZ3Lzy8sLi2vrBbW1utJlErGayzyI9n03IT7IuQ1JZTPm7HkbuD5vOENTnS8ccVlIqLwXA1j3gncfiguBHMVUd3C2qgtAzsR/fBmp816kdrtFopOyTHLngTlDBSRrWpUeEEbPURgSBGAI4Qi7MNFQk8LZTiIietgRJwkJEyc4wYLpE0pi1OGS+yAvn3atTI2pL32TIya0Sk+vZKUNrZJE1GeJKxPs008Nc6a/c17ZDz13Yb09zKvgFiFS2L/0o0z/6vTtShc4MjUIKim2DC6Opa5pKYr+ub2l6oUOcTEadyjuCTMjHLcZ9toElO77q1r4m8mU7N6z7LcFO/6ljTg8s9xToL6Xqm8X9o7OyhWjrNR57GJLezQPA9RwSmqqJH3NR7xhGerad1ad9b9Z6qVyzQb+Lashw+MtJY1</latexit> { µ, Yµ}data<latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit> Perceptron NN Sampling rate = # data points / # parameters to learn, infer
  • 4. 1-bit compressed sensing (signal processing) Perceptron (statistics/ML) Y = sign( X + Z p ) Compressed sensing (signal processing) CDMA (multi-user communication theory) Superposition codes (coding theory) Y = X + Z p Phase retrieval  (signal processing)Y = | X| EstimationLearning Rectified Linear Unit (ReLU) (ML/neural nets)Y = max(0, X)<latexit sha1_base64="Ru6lXUxXyCnS3Ry1iRRkoY4QEPY=">AAAC93icjVHLSsQwFD3W93vUpZviICjI0FFBXQiiG5cjODpiRdpORsP0RZqKY+mHuHMlbv0Ct/oF4h/oX3gTO+AD0ZQ2J+eec5qbuLHPE2lZLz1Gb1//wODQ8Mjo2PjEZGlq+iCJUuGxuhf5kWi4TsJ8HrK65NJnjVgwJ3B9dui2d1T98IKJhEfhvuzE7CRwzkLe4p4jiTotrdiSXUq3lR3lm5ktAjNwLvMFa8l2I7+ZdAKaMrt2zvOurpEvnpbKVsXSw/wJqgUooxi1qPQMG01E8JAiAEMISdiHg4SeY1RhISbuBBlxghDXdYYcI+RNScVI4RDbpu8ZrY4LNqS1yky026O/+PQKcpqYJ09EOkFY/c3U9VQnK/a37Exnqr11aHaLrIBYiXNi//J1lf/1qV4kWljXPXDqKdaM6s4rUlJ9Kmrn5qeuJCXExCncpLog7Gln95xN7Ul07+psHV1/1UrFqrVXaFO8qV3SBVe/X+dPUF+ubFSsvdXy1nZx00OYxRwW6DrXsIVd1FCn6Gs84BFPxpVxY9wadx9So6fwzODLMO7fARXQpPs=</latexit><latexit sha1_base64="Ru6lXUxXyCnS3Ry1iRRkoY4QEPY=">AAAC93icjVHLSsQwFD3W93vUpZviICjI0FFBXQiiG5cjODpiRdpORsP0RZqKY+mHuHMlbv0Ct/oF4h/oX3gTO+AD0ZQ2J+eec5qbuLHPE2lZLz1Gb1//wODQ8Mjo2PjEZGlq+iCJUuGxuhf5kWi4TsJ8HrK65NJnjVgwJ3B9dui2d1T98IKJhEfhvuzE7CRwzkLe4p4jiTotrdiSXUq3lR3lm5ktAjNwLvMFa8l2I7+ZdAKaMrt2zvOurpEvnpbKVsXSw/wJqgUooxi1qPQMG01E8JAiAEMISdiHg4SeY1RhISbuBBlxghDXdYYcI+RNScVI4RDbpu8ZrY4LNqS1yky026O/+PQKcpqYJ09EOkFY/c3U9VQnK/a37Exnqr11aHaLrIBYiXNi//J1lf/1qV4kWljXPXDqKdaM6s4rUlJ9Kmrn5qeuJCXExCncpLog7Gln95xN7Ul07+psHV1/1UrFqrVXaFO8qV3SBVe/X+dPUF+ubFSsvdXy1nZx00OYxRwW6DrXsIVd1FCn6Gs84BFPxpVxY9wadx9So6fwzODLMO7fARXQpPs=</latexit><latexit sha1_base64="Ru6lXUxXyCnS3Ry1iRRkoY4QEPY=">AAAC93icjVHLSsQwFD3W93vUpZviICjI0FFBXQiiG5cjODpiRdpORsP0RZqKY+mHuHMlbv0Ct/oF4h/oX3gTO+AD0ZQ2J+eec5qbuLHPE2lZLz1Gb1//wODQ8Mjo2PjEZGlq+iCJUuGxuhf5kWi4TsJ8HrK65NJnjVgwJ3B9dui2d1T98IKJhEfhvuzE7CRwzkLe4p4jiTotrdiSXUq3lR3lm5ktAjNwLvMFa8l2I7+ZdAKaMrt2zvOurpEvnpbKVsXSw/wJqgUooxi1qPQMG01E8JAiAEMISdiHg4SeY1RhISbuBBlxghDXdYYcI+RNScVI4RDbpu8ZrY4LNqS1yky026O/+PQKcpqYJ09EOkFY/c3U9VQnK/a37Exnqr11aHaLrIBYiXNi//J1lf/1qV4kWljXPXDqKdaM6s4rUlJ9Kmrn5qeuJCXExCncpLog7Gln95xN7Ul07+psHV1/1UrFqrVXaFO8qV3SBVe/X+dPUF+ubFSsvdXy1nZx00OYxRwW6DrXsIVd1FCn6Gs84BFPxpVxY9wadx9So6fwzODLMO7fARXQpPs=</latexit> Sigmoid/Logistic regression (ML/neural nets)P(Yµ = 1) = 1 1 + exp( µ · x)X<latexit sha1_base64="ZHW7OZbi5udCyj0kx2zC6hkQFRA=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkVwVVIR1F3RjTsrGFu0RZLptA5Nk5BMxBK69Qvc6m+Jf6B/4Z1xCmoRnZDkzLn3nJl7rx8HIpWO81qwZmbn5heKi6Wl5ZXVtfL6xmUaZQnjLouCKGn5XsoDEXJXChnwVpxwb+gHvOkPTlS8eceTVEThhRzFvDP0+qHoCeZJoq7akt9Lv5e3xjflilN19LKnQc2ACsxqROUXtNFFBIYMQ3CEkIQDeEjpuUYNDmLiOsiJSwgJHecYo0TajLI4ZXjEDujbp921YUPaK89UqxmdEtCbkNLGDmkiyksIq9NsHc+0s2J/8861p7rbiP6+8RoSK3FL7F+6SeZ/daoWiR4OdQ2Caoo1o6pjxiXTXVE3t79UJckhJk7hLsUTwkwrJ322tSbVtaveejr+pjMVq/bM5GZ4V7ekAdd+jnMauHvVo6pzvl+pH5tJF7GFbezSOA9QxykacMk6xCOe8GydWZmVW+PPVKtgNJv4tqyHDyPvk2s=</latexit><latexit sha1_base64="ZHW7OZbi5udCyj0kx2zC6hkQFRA=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkVwVVIR1F3RjTsrGFu0RZLptA5Nk5BMxBK69Qvc6m+Jf6B/4Z1xCmoRnZDkzLn3nJl7rx8HIpWO81qwZmbn5heKi6Wl5ZXVtfL6xmUaZQnjLouCKGn5XsoDEXJXChnwVpxwb+gHvOkPTlS8eceTVEThhRzFvDP0+qHoCeZJoq7akt9Lv5e3xjflilN19LKnQc2ACsxqROUXtNFFBIYMQ3CEkIQDeEjpuUYNDmLiOsiJSwgJHecYo0TajLI4ZXjEDujbp921YUPaK89UqxmdEtCbkNLGDmkiyksIq9NsHc+0s2J/8861p7rbiP6+8RoSK3FL7F+6SeZ/daoWiR4OdQ2Caoo1o6pjxiXTXVE3t79UJckhJk7hLsUTwkwrJ322tSbVtaveejr+pjMVq/bM5GZ4V7ekAdd+jnMauHvVo6pzvl+pH5tJF7GFbezSOA9QxykacMk6xCOe8GydWZmVW+PPVKtgNJv4tqyHDyPvk2s=</latexit><latexit sha1_base64="ZHW7OZbi5udCyj0kx2zC6hkQFRA=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkVwVVIR1F3RjTsrGFu0RZLptA5Nk5BMxBK69Qvc6m+Jf6B/4Z1xCmoRnZDkzLn3nJl7rx8HIpWO81qwZmbn5heKi6Wl5ZXVtfL6xmUaZQnjLouCKGn5XsoDEXJXChnwVpxwb+gHvOkPTlS8eceTVEThhRzFvDP0+qHoCeZJoq7akt9Lv5e3xjflilN19LKnQc2ACsxqROUXtNFFBIYMQ3CEkIQDeEjpuUYNDmLiOsiJSwgJHecYo0TajLI4ZXjEDujbp921YUPaK89UqxmdEtCbkNLGDmkiyksIq9NsHc+0s2J/8861p7rbiP6+8RoSK3FL7F+6SeZ/daoWiR4OdQ2Caoo1o6pjxiXTXVE3t79UJckhJk7hLsUTwkwrJ322tSbVtaveejr+pjMVq/bM5GZ4V7ekAdd+jnMauHvVo6pzvl+pH5tJF7GFbezSOA9QxykacMk6xCOe8GydWZmVW+PPVKtgNJv4tqyHDyPvk2s=</latexit> SKA telescope Super-Resolution
  • 5. [Gabrie Manoel Luneau B. Macris Krzakala Zdeborová 18] Models of deep neural networks in a pre-trained phase: [Aubin Maillard B. Macris Krzakala Zdeborová 18] Models of shallow networks, committee machines: W = '(1) ⇣r 1 n (1) '(2) ⇣r 1 n (2) '(3) ⇣ . . . '(L) ⇣r 1 n (L) X ⌘ . . . ⌘ <latexit sha1_base64="aKzzv3leqrpf9BVZ4UDbaldzcRo=">AAADs3iclVFdb9MwFL1Z+Bjlq7BHXiwqpO6lSroheEGaxgsPeyjSulZqxuQ4bmvNiYPtTCpRnuA/Iv7B9sJv4NqkqDAhmKPEx+eec+Jrp6UUxkbRt2ArvHX7zt3te537Dx4+etx98vTEqEozPmZKKj1NqeFSFHxshZV8WmpO81TySXr+1tUnF1wboYpjuyr5aU4XhZgLRi1SZ93vdZLOyaR5Q0hyQXW5FB/qfrzbJIdi0U/MR23rZK4pq+OmLpomSZXMzCrHqU5GS9G06l/W4Y2sw03r3toqM2XNxnaObpSJat/TtCHOtbuOc7hz1u1Fg8gPch3ELehBO0aq+xUSyEABgwpy4FCARSyBgsFnBjFEUCJ3CjVyGpHwdQ4NdNBboYqjgiJ7jt8FrmYtW+DaZRrvZvgXia9GJ4EX6FGo04jd34ivVz7ZsX/Lrn2m29sK57TNypG1sET2X7618n99rhcLc3jtexDYU+kZ1x1rUyp/Km7nZKMriwklcg5nWNeImXeuz5l4j/G9u7Olvn7plY51a9ZqK7hyu8QLjv+8zuvgZDiI9wbD9/u9g8P2qrfhGTyHPt7nKziAdzCCMbDgOPgUfA6+hC/DWZiG2U/pVtB6duC3EeY/AHNx6N4=</latexit>
  • 6. [Aubin Loureiro Maillard Krzakala Zdeborová 19] High-dimensional inference with structured/generative priors: with V ⇠ Pout ⇣ · r 1 n WZ ⌘ <latexit sha1_base64="abRLwfGt9aoXKbn/T4RJA6epZB4=">AAADI3icjVFNb9QwEH0NX2X5WuDIxWKFVKRqlbRIcKzKpcdFYncr6rJKvN7FahIH20Fapfk5/SfcekNIiAN34Ao/gLFJJaBC4CjxmzfzXjyerMqVdXH8cS26cPHS5SvrV3vXrt+4eat/+87E6toIORY612Y/S63MVSnHTrlc7ldGpkWWy2l29NTnp2+ksUqXz92qkodFuizVQonUETXrv2x4tmCTlltVMDaaNdwUTNeu5btqucHFXDvGN31wzDcZt6+Na/jCpKJJ2qZsW57pfG5XBW3BadqG7UXQP+zN+oN4GIfFzoOkAwN0a6T7H8Axh4ZAjQISJRzhHCksPQdIEKMi7hANcYaQCnmJFj3S1lQlqSIl9oi+S4oOOrak2HvaoBb0l5xeQ0qGB6TRVGcI+7+xkK+Ds2f/5t0ET3+2Fe1Z51UQ6/CK2H/pzir/V+d7cVjgSehBUU9VYHx3onOpw634k7NfunLkUBHn8ZzyhrAIyrN7ZkFjQ+/+btOQ/xwqPetj0dXW+OJPSQNO/hzneTDZGibbw61njwY7u92o13EP97FB83yMHexhhDF5v8VXfMP36CQ6jd5F73+WRmud5i5+W9GnH2HatYU=</latexit> Y ⇠ P ⇣ · r n U ⌦ V ⌘ <latexit sha1_base64="xXdOqv1gr7Nyh6KlCP5HUAlr70o=">AAADGXicjVFNT9wwEB1CgS1QWOiRi9UVEkholeVD9Ijg0uMi7S5UGCHH691aJHGwnUqrkF/CP+HGDXHlgHrrse2vYGyCBEUIHCV582bes8cTZbE0Ngx/jQXjHyYmp2ofp2dmP83N1xcWe0blmosuV7HShxEzIpap6FppY3GYacGSKBYH0emeyx/8FNpIlXbsKBPHCRumciA5s0id1DsFjQbke0mNTEib7srhCuV9ZQldc8E5XSPUnGlb0IFmvKAxWvdZWaRl6ZXdkiorE2F81CudaPWk3giboV/kJWhVoAHVaqv6HVDogwIOOSQgIAWLOAYGBp8jaEEIGXLHUCCnEUmfF1DCNGpzrBJYwZA9xe8Qo6OKTTF2nsarOe4S46tRSWAZNQrrNGK3G/H53Ds79jXvwnu6s43wH1VeCbIWfiD7lu6x8r0614uFAXz1PUjsKfOM645XLrm/FXdy8qQriw4Zcg73Ma8Rc698vGfiNcb37u6W+fwfX+lYF/OqNoe/7pQ44Nb/43wJeuvN1kZza3+zsbNbjboGS/AFVnCe27AD36ANXfS+hN/o+y+4CK6C6+DmoTQYqzSf4dkKbu8Bt4Ox0w==</latexit> [Tramel Manoel Caltagirone Gabrie Krzakala 16]
  • 7. High-dimensional inference: what we DON’T know (yet) • Non Bayes optimal setting • Structure in the data • New statistical regimes • Zooming on the corners of the phase diagrams ( , P0) 6= (˜, ˜P0)<latexit sha1_base64="iutlYq+xxQQXk4biHbNfleAcIeQ=">AAAC9XicjVHLSsNAFD3GV62vqks3g0WoICWtgi6LblxWsFWwUpJ01MFpEpOJUoq/4c6duPUH3OoviH+gf+GdcRQfiE5Icu+555yZO9ePpUiV6z4NOINDwyOjubH8+MTk1HRhZraZRlkS8EYQySjZ872USxHyhhJK8r044V7Xl3zXP9nU9d0znqQiCndUL+YHXe8oFIci8BRB7YJbaklid7xlVm+7S6wV8lNWaikhO5x9lGyuGe1C0S27ZrGfQcUGRdhVjwqPaKGDCAEydMERQlEs4SGlZx8VuIgJO0CfsIQiYeocF8iTNiMWJ4ZH6Al9jyjbt2hIufZMjTqgXSS9CSkZFkkTES+hWO/GTD0zzhr9zbtvPPXZevT3rVeXUIVjQv/SvTP/q9O9KBxi3fQgqKfYILq7wLpk5lb0ydmnrhQ5xITpuEP1hOLAKN/vmRlNanrXd+uZ+rNhalTngeVmeNGnpAFXvo/zZ9Cslisr5er2arG2YUedwzwWUKJ5rqGGLdTRIO9L3OEeD865c+VcOzdvVGfAaubwZTm3r+5+oH4=</latexit> [Goldt Mézard Krzakala Zdeborová 19]
  • 8. What about the CORNERS of phase diagrams (in inference problems)? ⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit> = 1 <latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit> noise level sparsity
  • 9. Compressed sensing ↵ = m n<latexit sha1_base64="wnw1yuYGkmkdhkUuYnakUAai+cc=">AAAC13icjVHLSsNAFD2Nr/qudekmWARXJVVBN0LRjcsK9iGtyGSctsG8mEzEUoo7cesPuNU/Ev9A/8I7Ywo+EJ2Q5My595yZe68b+16iHOclZ01MTk3P5Gfn5hcWl5YLK8VGEqWSizqP/Ei2XJYI3wtFXXnKF61YCha4vmi6l4c63rwSMvGi8EQNYnEWsF7odT3OFFHnhWKH+XGf7Xe6kvFhMBqGo/NCySk7Ztk/QSUDJWSrFhWe0cEFInCkCCAQQhH2wZDQ00YFDmLizjAkThLyTFxghDnSppQlKIMRe0nfHu3aGRvSXnsmRs3pFJ9eSUobG6SJKE8S1qfZJp4aZ83+5j00nvpuA/q7mVdArEKf2L9048z/6nQtCl3smRo8qik2jK6OZy6p6Yq+uf2pKkUOMXEaX1BcEuZGOe6zbTSJqV33lpn4q8nUrN7zLDfFm74lDbjyfZw/QWOrXNkubx3vlKoH2ajzWMM6Nmmeu6jiCDXUyfsaD3jEk3Vq3Vi31t1HqpXLNKv4sqz7dyqcl0Y=</latexit> ⇢ = ⇥(1)<latexit sha1_base64="Y7PvJGlHAI+BpBFatinfXoOjCBI=">AAAC0XicjVHLSsNAFD2Nr1pfVZdugkWom5JUQTdC0Y3Lin1BWyVJp21oXkwmQimCuPUH3OpPiX+gf+GdMQW1iE5Icubce87MvdeOPDcWhvGa0ebmFxaXssu5ldW19Y385lYjDhPusLoTeiFv2VbMPDdgdeEKj7Uizizf9ljTHp3JePOG8dgNg5oYR6zrW4PA7buOJYi66vBheNKpDZmwiub+db5glAy19FlgpqCAdFXD/As66CGEgwQ+GAIIwh4sxPS0YcJARFwXE+I4IVfFGW6RI21CWYwyLGJH9B3Qrp2yAe2lZ6zUDp3i0ctJqWOPNCHlccLyNF3FE+Us2d+8J8pT3m1Mfzv18okVGBL7l26a+V+drEWgj2NVg0s1RYqR1TmpS6K6Im+uf6lKkENEnMQ9inPCjlJO+6wrTaxql721VPxNZUpW7p00N8G7vCUN2Pw5zlnQKJfMg1L54rBQOU1HncUOdlGkeR6hgnNUUSdvjkc84Vm71MbanXb/maplUs02vi3t4QONupRB</latexit> ⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit> ↵ = ⇥(1)<latexit sha1_base64="wDwxDp8Jtqu98sFt/qBe5OvtuX8=">AAAC1XicjVHLSsNAFD2Nr1pfUZdugkWom5KooBuh6MZlhb6gLWWSTtvQNAnJpFBKd+LWH3CrvyT+gf6Fd8YU1CI6IcmZc+85M/deO/TcWJjma0ZbWl5ZXcuu5zY2t7Z39N29WhwkkcOrTuAFUcNmMfdcn1eFKzzeCCPORrbH6/bwWsbrYx7FbuBXxCTk7RHr+27PdZggqqPrLeaFA3bZqgy4YAXruKPnzaKplrEIrBTkka5yoL+ghS4COEgwAocPQdgDQ0xPExZMhMS1MSUuIuSqOMcMOdImlMUpgxE7pG+fds2U9WkvPWOldugUj96IlAaOSBNQXkRYnmaoeKKcJfub91R5yrtN6G+nXiNiBQbE/qWbZ/5XJ2sR6OFC1eBSTaFiZHVO6pKorsibG1+qEuQQEidxl+IRYUcp5302lCZWtcveMhV/U5mSlXsnzU3wLm9JA7Z+jnMR1E6K1mnx5PYsX7pKR53FAQ5RoHmeo4QblFEl7zEe8YRnra7NtDvt/jNVy6SafXxb2sMHN9OVQw==</latexit> = ⇥(1)<latexit sha1_base64="2CD7f7ymTgUC3T0VN00wswlLVAc=">AAAC1nicjVHLSsNAFD2Nr1pfrS7dBItQNyWpgm6EohuXFfqCtpRJOm1D0yQkE6UU3Ylbf8CtfpL4B/oX3hlTUIvohCRnzj3nzNwZK3CdSBjGa0pbWFxaXkmvZtbWNza3srnteuTHoc1rtu/6YdNiEXcdj9eEI1zeDELOxpbLG9boXNYbVzyMHN+riknAO2M28Jy+YzNBVDeba7sk7rHTdnXIBSuYB91s3igaaujzwExAHsmo+NkXtNGDDxsxxuDwIAi7YIjoacGEgYC4DqbEhYQcVee4QYa8Mak4KRixI/oOaNZKWI/mMjNSbptWcekNyaljnzw+6ULCcjVd1WOVLNnfsqcqU+5tQn8ryRoTKzAk9i/fTPlfn+xFoI8T1YNDPQWKkd3ZSUqsTkXuXP/SlaCEgDiJe1QPCdvKOTtnXXki1bs8W6bqb0opWTm3E22Md7lLumDz53XOg3qpaB4WS5dH+fJZctVp7GIPBbrPY5RxgQpqlH2NRzzhWWtqt9qddv8p1VKJZwffhvbwATvolag=</latexit> 1<latexit sha1_base64="GM0DyeF3AwEXlxUmliDnfXT5ehQ=">AAACz3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl047IF+4CmyCSdxqFpEpKJUori1h9wq38l/oH+hXfGFNQiOiHJmXPPOTN3xo0DkUrLei0Yc/MLi0vF5dLK6tr6Rnlzq5VGWeLxphcFUdJxWcoDEfKmFDLgnTjhbOQGvO0Oz1S9fc2TVEThhRzHvDdifigGwmOSKMcJSNpnju+b9mW5YlUtPcxZYOeggnzUo/ILHPQRwUOGEThCSMIBGFJ6urBhISauhwlxCSGh6xy3KJE3IxUnBSN2SF+fZt2cDWmuMlPt9miVgN6EnCb2yBORLiGsVjN1PdPJiv0te6Iz1d7G9HfzrBGxElfE/uWbKv/rU71IDHCiexDUU6wZ1Z2Xp2T6VNTOzS9dSUqIiVO4T/WEsKed03M2tSfVvauzZbr+ppWKVXMv12Z4V7ukC7Z/XucsaB1U7cPqQeOoUjvNr7qIHexin+7zGDWco44mZcd4xBOejYZxY9wZ959So5B7tvFtGA8fDjiTrw==</latexit> Wµ = r n µ · X + Zµ µ = 1, . . . , m Xi ⇠ Ber(⇢) i = 1, . . . , n<latexit sha1_base64="G+Ucf8j6N1Ci6INlgn7132TkJXc=">AAADUXicjVFNb9QwEJ1k+ShboFu4IHExXVEVUa2SUgkulapy4bhIbHdFXa0cx9u16sSp7SCtovwD/h0HxD+AKzdujE0qPioEjhI/v3lvMuPJKiWtS5JPUdy7dv3GzbVb/fXbd+5uDDbvHVtdGy4mXCttZhmzQslSTJx0SswqI1iRKTHNzl/6+PSdMFbq8o1bVeK0YGelXEjOHFLzwfvpnBb19gG1F8Y1dGEYb6hCf87apmxbmmmV21WBW0PHS9l6OeW5dqSh2YLMWvKUvPUkoRc1y4lHB+kuVSixuwWl/dlcblMri4aaghwJ0+5Qs9RPOr38KcZyhskoCYtcBWkHhtCtsR58BAo5aOBQQwECSnCIFTCw+JxACglUyJ1Cg5xBJENcQAt99NaoEqhgyJ7j9wxPJx1b4tnntMHN8S8KX4NOAo/Ro1FnEPu/kRCvQ2bP/i13E3L62la4Z12uAlkHS2T/5btU/q/P9+JgAS9CDxJ7qgLju+Ndljrciq+c/NKVwwwVch7nGDeIeXBe3jMJHht693fLQvxzUHrWn3mnreGLrxIHnP45zqvgeG+UPhvtvd4fHh51o16Dh7AFOzjP53AIr2AME8z9NXoQPYq24g/xtx704h/SOOo89+G31Vv/Dr2XwcY=</latexit> sparsity measurement rate [Reeves Xu Zadik 19]
  • 10. [Reeves Xu Zadik 19] n = = 2<latexit sha1_base64="C2FTjtFB8XOW/DSfx9H/vVfrqME=">AAAC2HicjVHLSsNAFD2Nr1pf0S7dBIvgqiRV0E2h6MZlBfvAtpQkndZgXiQToRTBnbj1B9zqF4l/oH/hnXEKahGdkOTMueecmTvjxL6XctN8zWlz8wuLS/nlwsrq2vqGvrnVTKMscVnDjfwoaTt2ynwvZA3ucZ+144TZgeOzlnN1Iuqta5akXhSe83HMeoE9Cr2h59qcqL5e7PokHtj9sKpQtdLXS2bZlMOYBZYCJahRj/QXdDFABBcZAjCE4IR92Ejp6cCCiZi4HibEJYQ8WWe4QYG8GakYKWxir+g7ollHsSHNRWYq3S6t4tObkNPALnki0iWExWqGrGcyWbC/ZU9kptjbmP6OygqI5bgk9i/fVPlfn+iFY4gj2YNHPcWSEd25KiWTpyJ2bnzpilNCTJzAA6onhF3pnJ6zIT2p7F2crS3rb1IpWDF3lTbDu9glXbD18zpnQbNStvbLlbODUu1YXXUe29jBHt3nIWo4RR0Nyh7jEU941i60W+1Ou/+UajnlKeLb0B4+AE61luE=</latexit> ⇢ &<latexit sha1_base64="7984W2rrhwZWY+76OORG7U9Xucg=">AAAC0HicjVHLSsNAFD2Nr1pfVZdugkVwVZIq6LLoxmUV+4C2SJJO29A0EycTtRQRt/6AW/0q8Q/0L7wzpqAW0QlJzpx7z5m597pR4MfSsl4zxszs3PxCdjG3tLyyupZf36jFPBEeq3o84KLhOjEL/JBVpS8D1ogEc4ZuwOru4FjF61dMxD4Pz+UoYu2h0wv9ru85kqh2S/S52YqZIwS/vsgXrKKllzkN7BQUkK4Kz7+ghQ44PCQYgiGEJBzAQUxPEzYsRMS1MSZOEPJ1nOEWOdImlMUowyF2QN8e7ZopG9JeecZa7dEpAb2ClCZ2SMMpTxBWp5k6nmhnxf7mPdae6m4j+rup15BYiT6xf+kmmf/VqVokujjUNfhUU6QZVZ2XuiS6K+rm5peqJDlExCncobgg7GnlpM+m1sS6dtVbR8ffdKZi1d5LcxO8q1vSgO2f45wGtVLR3iuWTvcL5aN01FlsYRu7NM8DlHGCCqrkfYlHPOHZODNujDvj/jPVyKSaTXxbxsMHaLqUpQ==</latexit> ⇢ &<latexit sha1_base64="7984W2rrhwZWY+76OORG7U9Xucg=">AAAC0HicjVHLSsNAFD2Nr1pfVZdugkVwVZIq6LLoxmUV+4C2SJJO29A0EycTtRQRt/6AW/0q8Q/0L7wzpqAW0QlJzpx7z5m597pR4MfSsl4zxszs3PxCdjG3tLyyupZf36jFPBEeq3o84KLhOjEL/JBVpS8D1ogEc4ZuwOru4FjF61dMxD4Pz+UoYu2h0wv9ru85kqh2S/S52YqZIwS/vsgXrKKllzkN7BQUkK4Kz7+ghQ44PCQYgiGEJBzAQUxPEzYsRMS1MSZOEPJ1nOEWOdImlMUowyF2QN8e7ZopG9JeecZa7dEpAb2ClCZ2SMMpTxBWp5k6nmhnxf7mPdae6m4j+rup15BYiT6xf+kmmf/VqVokujjUNfhUU6QZVZ2XuiS6K+rm5peqJDlExCncobgg7GnlpM+m1sS6dtVbR8ffdKZi1d5LcxO8q1vSgO2f45wGtVLR3iuWTvcL5aN01FlsYRu7NM8DlHGCCqrkfYlHPOHZODNujDvj/jPVyKSaTXxbxsMHaLqUpQ==</latexit> (informal) Theorem [B. Macris Dia Krzakala 16] [Reeves Pfister 16]: Compressed sensing: low measurement-rate regime lim n!1 1 n I(X; W) = inf q 0 i(pot) (q, ) lim n!1 1 ⇢n EkX E[X|W]k2 = 1 ⇢ mmse(q⇤ , ) <latexit sha1_base64="CjDg2F4pFEFilYEvhZWW0IWjFxs=">AAADunicjVHbjtMwEJ1suCzLZQs88mJRIbUIqmQBgQQrrUBI8LZIdFupbisncVurudVxkKrUD7zCByL+AD6Ad8bGy63i4ijJ+Mw5xzOeqExFpYLgo7fjnzl77vzuhb2Lly5f2W9dvXZSFbWMeT8u0kIOI1bxVOS8r4RK+bCUnGVRygfR8pnJD95wWYkif63WJR9nbJ6LmYiZQmja+kJTkU2bnKqCinym1prQmWRxmJOXnYZGMzLUj+1/oLuHhjJtVnTOSaCJmDRIkRkpC6W7urO6Q1M8OWFdQlermiVky9x6N6FuGmprbyRPNHosCq1zTTOmFlHUPNd0Q9zhd3+AIwdtXD1jupkcHP7NUtvysqziWN3k9vf6pq120AvsIttB6II2uHVctD4AhQQKiKGGDDjkoDBOgUGFzwhCCKBEbAwNYhIjYfMcNOyhtkYWRwZDdInfOe5GDs1xbzwrq47xlBRfiUoCt1BTIE9ibE4jNl9bZ4P+ybuxnqa2Nf4j55UhqmCB6L90p8z/1ZleFMzgke1BYE+lRUx3sXOp7a2YyslPXSl0KBEzcYJ5iXFslaf3TKymsr2bu2U2/8kyDWr2sePW8NlUiQMOfx/ndnBy0Avv9R68ut8+eupGvQs34CZ0cJ4P4QhewDH0IfZG3lvvnffef+JHvvCX36g7ntNch1+Wr74C/zDtZQ==</latexit> lim n!1 1 ⇢n EkX E[X|W]k2 = 1 ⇢ mmse(q⇤ , ) ↵/↵IT(⇢)<latexit sha1_base64="V9ZJRXKm6dvB3MskAo2n1hNqnF4=">AAAC4nicjVHLSgMxFD0d3/U16lKEwSLUTZ36QJdFN7qr0KpgS8lM03bovMhkhFJcuXMnbv0Bt/ox4h/oX3gTR/CBaIZJTs695yQ314l9L5G2/ZwzRkbHxicmp/LTM7Nz8+bC4kkSpcLldTfyI3HmsIT7Xsjr0pM+P4sFZ4Hj81Onf6DipxdcJF4U1uQg5s2AdUOv47lMEtUyVxrMj3ts431pDRsisI5ql8WG6EXr+ZZZsEu2HtZPUM5AAdmoRuYTGmgjgosUAThCSMI+GBL6zlGGjZi4JobECUKejnNcIk/alLI4ZTBi+zR3aXeesSHtlWei1S6d4tMvSGlhjTQR5QnC6jRLx1PtrNjfvIfaU91tQKuTeQXESvSI/Uv3kflfnapFooM9XYNHNcWaUdW5mUuqX0Xd3PpUlSSHmDiF2xQXhF2t/HhnS2sSXbt6W6bjLzpTsWrvZrkpXtUtqcHl7+38CU42S+Wt0s7xdqGyn7V6EstYRZH6uYsKDlFFnbyvcI8HPBpt49q4MW7fU41cplnCl2HcvQH/EJqX</latexit> 1
  • 11. ↵n = m n<latexit sha1_base64="HNyXCSzlEWzm0asCuCcvSGCErl4=">AAAC2nicjVHLSsNAFD3GV62vqLhyEyyCq5KqoBuh6MZlBVsLVspkOtVgXkwmQgnduBO3/oBb/SDxD/QvvDOm4APRCUnOnHvPmbn3ekngp8p1X8as8YnJqenSTHl2bn5h0V5abqVxJrlo8jiIZdtjqQj8SDSVrwLRTqRgoReIU+/qUMdPr4VM/Tg6UYNEnIfsIvL7PmeKqK692mFBcsm6kbPf6UvG83CYR8OuXXGrrlnOT1ArQAXFasT2MzroIQZHhhACERThAAwpPWeowUVC3Dly4iQh38QFhiiTNqMsQRmM2Cv6XtDurGAj2mvP1Kg5nRLQK0npYIM0MeVJwvo0x8Qz46zZ37xz46nvNqC/V3iFxCpcEvuXbpT5X52uRaGPPVODTzUlhtHV8cIlM13RN3c+VaXIISFO4x7FJWFulKM+O0aTmtp1b5mJv5pMzeo9L3IzvOlb0oBr38f5E7S2qrXt6tbxTqV+UIy6hDWsY5PmuYs6jtBAk7xzPOART1bHurFurbuPVGus0Kzgy7Lu3wHjEZhR</latexit> ⇢n<latexit sha1_base64="2XVURmQqOQ2kJDHceYZs4UyFluk=">AAACyXicjVHLTsJAFD3UF+ILdemmkZi4Ii2a6JLoxsQNJvJIgJB2GKBS2jqdGpG48gfc6o8Z/0D/wjtjSVRidJq2Z86958zce93I92JpWa8ZY25+YXEpu5xbWV1b38hvbtXiMBGMV1noh6LhOjH3vYBXpSd93ogEd0auz+vu8FTF6zdcxF4YXMpxxNsjpx94PY85kqhaSwzCTtDJF6yipZc5C+wUFJCuSph/QQtdhGBIMAJHAEnYh4OYniZsWIiIa2NCnCDk6TjHPXKkTSiLU4ZD7JC+fdo1UzagvfKMtZrRKT69gpQm9kgTUp4grE4zdTzRzor9zXuiPdXdxvR3U68RsRIDYv/STTP/q1O1SPRwrGvwqKZIM6o6lrokuivq5uaXqiQ5RMQp3KW4IMy0ctpnU2tiXbvqraPjbzpTsWrP0twE7+qWNGD75zhnQa1UtA+KpYvDQvkkHXUWO9jFPs3zCGWcoYIqeV/hEU94Ns6Na+PWuPtMNTKpZhvflvHwAeFSka0=</latexit> (informal) Theorem [Reeves Xu Zadik 19]: « All-or-nothing » phenomenon Weak recovery impossible: ⇢n = o(n 1/2 ) and ↵ < ↵IT,n with ↵IT,n = 2⇢n| ln ⇢n| ln(1 + n)<latexit sha1_base64="lnscvIQZKnJefNiQ5egIenv7Xfo=">AAADWXicjVFdb9MwFL1p+CjlY4U98mJRgVpgXVKQ4GGTJniBtyGt26RlRI7rrtYcJ3IcYMryL/hvCPEH4C/wxLXrIaBC4CjJueeec+3rm5VSVCaKPged8NLlK1e713rXb9y8tda/fWe/KmrN+JQVstCHGa24FIpPjTCSH5aa0zyT/CA7fWnzB++4rkSh9sxZyY9zeqLEXDBqkEr7Hx8kelGkimyTYqjeNhvx5qQdkYQkhn8wDVWz1gZUlgu65f9p0yQ6J6/32seqJT+174VZLMNV1TZJ5pqyZrLc7TyRinjYNhgM40eJxFPPaKpGbdofROPILbIKYg8G4Ndu0f8ECcygAAY15MBBgUEsgUKFzxHEEEGJ3DE0yGlEwuU5tNBDb40qjgqK7Cl+TzA68qzC2NasnJvhLhJfjU4C99FToE4jtrsRl69dZcv+rXbjatqzneE/87VyZA0skP2X70L5vz7bi4E5PHc9COypdIztjvkqtbsVe3LyS1cGK5TIWTzDvEbMnPPinonzVK53e7fU5b86pWVtzLy2hm/2lDjg+M9xroL9yTh+Mp68eTrYeeFH3YW7cA+GOM9nsAOvYBemWPt7QIJR8LDzJQzCbthbSjuB96zDbytc/wFqPsKC</latexit> lim inf n!1 1 ⇢nn EkX ˆXoptk2 = 1 <latexit sha1_base64="uqseclRG6+r42WOepxt/Hs2XdP8=">AAADG3icjVFNTxUxFL0MqAh+PHDppuGFxI0vM6DRDQnBmLDExPd4CcVJp3R4DTPtpNMheRnmp/BP3Lkzbg1hywqiP4LbUhKVGO1kZk7PPee0t82qQtY2js9notm5e/cfzD9cWHz0+MnT3tLyqNaN4WLIdaHNOGO1KKQSQyttIcaVEazMCrGbHb1z9d1jYWqp1Uc7rcR+yQ6VzCVnFqm0N6KFLKXK01ZRqykiO+0IzQ3jbdK11Ex0qlRHS2YnWda+7+gJaWmWk3H3kk6YDThFZUl0ZbH+aW0jSXv9eBD7Qe6CJIA+hLGje2dA4QA0cGigBAEKLOICGNT47EECMVTI7UOLnEEkfV1ABwvobVAlUMGQPcLvIc72Aqtw7jJr7+a4SoGvQSeBVfRo1BnEbjXi641Pduzfsluf6fY2xX8WskpkLUyQ/ZfvVvm/PteLhRze+h4k9lR5xnXHQ0rjT8XtnPzSlcWECjmHD7BuEHPvvD1n4j21792dLfP1S690rJvzoG3gyu0SLzj58zrvgtHaIFkfvP7wqr+5Fa56Hp7DCrzA+3wDm7ANOzDE7M9wAT/gZ3QafYm+Rt9upNFM8DyD30b0/RpLkLPV</latexit> Strong recovery possible: lim sup n!1 1 ⇢nn EkX ˆXoptk2 = 0 <latexit sha1_base64="c96bpEitY67uc2VlQIQbGInOYrc=">AAADG3icjVHPaxQxGP061VrXatf26CW4CL10mVmFeiksiuCxgvsDmnbIpJlu6EwyZDKFZZw/xf/EmzfxKsWrJ0X/CL/EFNQiNsPMvLzvvZd8SVYVsrZx/HklWr1xc+3W+u3enY279zb797emtW4MFxOuC23mGatFIZWYWGkLMa+MYGVWiFl29tzVZ+fC1FKr13ZZiaOSnSqZS84sUml/SgtZ1k2VtopaTaXK7bIjNDeMt0nXUrPQqVIdLZldZFn7oqNvSEuznMy7XbpgNuAUlSXRlcX68Wg/TvuDeBj7Qa6CJIABhHGg+xdA4QQ0cGigBAEKLOICGNT4HEICMVTIHUGLnEEkfV1ABz30NqgSqGDInuH3FGeHgVU4d5m1d3NcpcDXoJPAI/Ro1BnEbjXi641Pduy/sluf6fa2xH8WskpkLSyQ/Z/vUnldn+vFQg5PfQ8Se6o847rjIaXxp+J2Tn7rymJChZzDJ1g3iLl3Xp4z8Z7a9+7Olvn6V690rJvzoG3gm9slXnDy93VeBdPRMHk8HL16Mhg/C1e9Dg/gIezgfe7BGF7CAUww+x18ge/wI3obvY8+RB9/SaOV4NmGP0b06SeQyrPs</latexit> ⇢n = o(1) and ↵ > ↵IT,n<latexit sha1_base64="Ln0kLNdmUwdFBu8hKQqvctdC95s=">AAAC/nicjVFdS9xAFD2mH3601rV97Mvg0mKhLIktqA+K2Jf6prCrC0aWSXZ0g0kmTCaihAX/Sd/6Vnz1D/hqH0v/gf4L70xnQbsUOyGZc8+952Tu3KhIk1L7/u8J78nTZ88np6ZnXrycfTXXmH+9W8pKxaITy1SqbsRLkSa56OhEp6JbKMGzKBV70fEXk987EapMZN7WZ4U4yPhRnhwmMddE9Rqr70M1kL2crTG5GHxgIQu1ONU1z/tDE/C0GPB1t/fqOlQZ22oPP+bDXqPpt3y72DgIHGjCrW3Z+IUQfUjEqJBBIIcmnIKjpGcfAXwUxB2gJk4RSmxeYIgZ0lZUJaiCE3tM3yOK9h2bU2w8S6uO6S8pvYqUDO9II6lOETZ/YzZfWWfD/su7tp7mbGe0R84rI1ZjQOxjulHl/+pMLxqHWLE9JNRTYRnTXexcKnsr5uTsXleaHAriDO5TXhGOrXJ0z8xqStu7uVtu8ze20rAmjl1thVtzShpw8Pc4x8HuUiv41Fra+dzc2HSjnsJbLGCR5rmMDXzFNjrk/Q1XuMZP79z77v3wLv6UehNO8wYPlnd5B7FVpPc=</latexit> Wµ = s n ⇢nn µ · X + Zµ µ = 1, . . . , ↵nn Xi ⇠ Ber(⇢n) i = 1, . . . , n n = ⌦(1)<latexit sha1_base64="1I5kPe7Eij+cSnQwga68y7AnmK8=">AAADwHicjVFdb9MwFL1Z+Bjlq4NHXiwqpk5MVTOQxkulqbzwRpHoWjFPleO4rbUkTm0HqUR55hV+HeIfwG/ghWsvE9ANgaMkx+fec66vb1yk0th+/2uwFV67fuPm9q3W7Tt3791v7zw4NqrUXIy5SpWexsyIVOZibKVNxbTQgmVxKibx2UsXn7wX2kiVv7XrQpxmbJHLueTMIjVr/5jMaFbuDqhZaVvRuWa8oinqEzarqPevtEjqvK4rqpdqg0SaxipNzDrDX0VHS1k7Q8oTZUlF4zmZ1k/fOYrQVckSgmgQ7dMU42afsrRYbhYiOaWt6UzuUiMz9NAZGQpdd68ov9eYyl+WOVLnha7sggwIfZ2JBetGe7N2p9/r+0Uug6gBHWjWSLW/AIUEFHAoIQMBOVjEKTAw+JxABH0okDuFCjmNSPq4gBpaqC0xS2AGQ/YMvwvcnTRsjnvnabyaY5UUX41KAk9QozBPI3bViI+X3tmxf/OuvKc72xr/ceOVIWthiey/dBeZ/6tzvViYwwvfg8SeCs+47njjUvpbcScnv3Vl0aFAzuEE4xox98qLeyZeY3zv7m6Zj3/zmY51e97klvDdnRIHHG2O8zI4PuhFz3oHb553jobNqLfhETyGLs7zEI7gFYxgDDxgwcfgU/A5HIbLUIWr89StoNE8hD9W+OEnxOrv/w==</latexit> Replica prediction at ⇢ ! 0+ agrees with ⇢n ! 0+ for ⇢n = o(n 1/2 )<latexit sha1_base64="52bXL9v9NgK4PAqRjfi5jyXoWfU=">AAADOHicjVFNTxsxEJ0shSbhK6VHLlYjJBAi7KZIcKkUlQtHWjWAxELkdZzEyma9sr2lUZQ/xh9BuXEDrvwBOjYbUYoQeLW7M2/ee/Z4ojQW2vj+pODNfJid+1gslecXFpeWK59WjrTMFONNJmOpTiKqeSwS3jTCxPwkVZwOopgfR/19Wz/+zZUWMvllhik/G9BuIjqCUYNQq6JDw/+Y0U+OmzFKUNsWzJYINWMSklD1JAmNJH5r06aOTbuKc00uhOlNOa3kiVV+ZHWkeqp+k+vJ+Wgr2K6PN1qVql/z3SIvgyAPqpCvQ1m5ghDaIIFBBgPgkIDBOAYKGp9TCMCHFLEzGCGmMBKuzmEMZdRmyOLIoIj28dvF7DRHE8ytp3ZqhrvE+CpUElhDjUSewtjuRlw9c84Wfc175Dzt2Yb4j3KvAaIGeoi+pZsy36uzvRjowJ7rQWBPqUNsdyx3ydyt2JOTf7oy6JAiZuM21hXGzCmn90ycRrve7d1SV791TIvanOXcDO7sKXHAwf/jfBkc1WvB11r9x0618T0fdRFW4Qus4zx3oQEHcAhN9J7AQ6FYKHmX3rV34909Ur1CrvkMz5Z3/xfE5roM</latexit>
  • 12. [Lesieur Krzakala Zdeborová 15]⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit> = 1 <latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit> Sparse principal components analysis: high sparsity regime W = r n X ⌦ X + Z Xi ⇠ Ber(⇢) i = 1, . . . , n<latexit sha1_base64="1z0nYv4qYQG5iyOMuyNR5/W5QP0=">AAADMnicjVHPTxQxFH47oOL6g1WPXho2EoxkM4MkeCEhePEICctupIR0Ol1o6EyHtmNCJvNf+Z94MHozXjThH+DGaymJSox2MjPf+977vvb15bWS1qXpl14yN3/n7r2F+/0HDx89Xhw8ebpvdWO4GHOttJnmzAolKzF20ikxrY1gZa7EJD996/OTD8JYqas9d16Lw5IdV3ImOXNIHQ3KluYzMumWN6k9M66lM8N4SxU6FKxrq64LBdOOaidLYUkMX4X/+47S/vRILlMrS0yZkmwL061Qc6JfEnrWsILIzWyVqkI7u4r7DdNRGha5DbIIhhDXjh58BgoFaODQQAkCKnCIFTCw+BxABinUyB1Ci5xBJENeQAd91DZYJbCCIXuK32OMDiJbYew9bVBz3EXha1BJ4AVqNNYZxH43EvJNcPbs37zb4OnPdo7/PHqVyDo4QfZfupvK/9X5XhzM4E3oQWJPdWB8dzy6NOFW/MnJL105dKiR87jAvEHMg/LmnknQ2NC7v1sW8j9CpWd9zGNtAz/9KXHA2Z/jvA3210bZ69Ha7vpwazuOegGewxKs4Dw3YAvewQ6M0fsTXPaS3lzyMfmafEu+X5cmvah5Br+t5OIKpZ65Jg==</latexit> « Bernoulli spiked model »
  • 13. ⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit> = 1 <latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit> lim n!1 1 n I(X; W) = inf q 0 i(pot) (q; ⇢, ) lim n!1 1 ⇢n EkX <latexit sha1_base64="Lulp+ygbLGh2RfzxN1GNl08w/UU=">AAADi3icjVHtbtMwFL1Z+BiFsQI/+WNRIbVoVEmHBGJMmkCT4N9AdK1Ur5GTOq21JE4dB6kKfj1eAfEG8BZcm0wCJj4cJTk+95x7fX3jMhOVDoIv3pZ/5eq169s3Ojdv7dze7d65e1rJWiV8nMhMqmnMKp6Jgo+10BmfloqzPM74JD5/ZeOTD1xVQhbv9abkZzlbFiIVCdNIRd1PNBN51BRUSyqKVG8MoaliSROapjBv+g2NUzI1B+4/MYNDq4qaNV1yEhgi5g1KVE5Kqc3A9NcHVK3kHs3wBAs2IHS9rtmC/KWI1ZMCiZzpVRw3x4Z+bIs+piumWxy5KrLUGJ6PDt0uzyuOJeePBp2o2wuGgVvkMghb0IN2ncjuZ6CwAAkJ1JADhwI04gwYVPjMIIQASuTOoEFOIRIuzsFAB701qjgqGLLn+F3ibtayBe5tzsq5E6yS4avQSeAheiTqFGJbjbh47TJb9k+5G5fTnm2D/7jNlSOrYYXsv3wXyv/12V40pPDM9SCwp9IxtrukzVK7W7EnJz91pTFDiZzFC4wrxIlzXtwzcZ7K9W7vlrn4V6e0rN0nrbaGb/aUOODw93FeBqejYbg/HL190jt62Y56G+7DA+jjPJ/CEbyGExhD4u1577yZR/0df99/7r/4Id3yWs89+GX5x98BfXbXcw==</latexit> Theorem [Korada Macris 09], [Deshpande Montanari 14], [B. Macris Dia Lesieur Krzakala Zdeborová 16], [Miolane Lelarge 17]: Sparse PCA: high sparsity regime lim n!1 1 (⇢n)2 EkX ⌦ X E[X ⌦ X|W]k2 F = mmse(q⇤ ) <latexit sha1_base64="XU1cnkMkItBJZdPCKdPu2tKN48o=">AAADVXicjVFdb9MwFL1pxxjjYx08sgeLaqhDokoKE7wgTUxDPA6JrpXqtkoyZ7UWx8F2kKos/4F/N/EP2B/gDYlrz5MGA4GjJOeee861r29S5lybMPwatNort1Zvr91Zv3vv/oONzubDIy0rlbJhKnOpxkmsWc4LNjTc5GxcKhaLJGej5HTf5kefmdJcFh/NsmRTEZ8UPONpbJCad77QnIt5XVAjKS8ys2xopuK0jpq6R9VCkmJnNmgIFbFZJEl90NCzmiYZGTdUGi6YJj58fk0z+bPk0jlqpvRsXlMlyLtmNnjjkBCaNb1Ps2c780437IdukZsg8qALfh3KzjlQOAYJKVQggEEBBnEOMWh8JhBBCCVyU6iRU4i4yzNoYB29FaoYKmJkT/F7gtHEswXGtqZ27hR3yfFV6CSwjR6JOoXY7kZcvnKVLfu32rWrac+2xH/iawlkDSyQ/ZfvSvm/PtuLgQxeux449lQ6xnaX+iqVuxV7cnKtK4MVSuQsPsa8Qpw659U9E+fRrnd7t7HLf3NKy9o49doKLuwpccDR7+O8CY4G/ehFf/fDy+7eWz/qNXgMT6CH83wFe/AeDmGItb8HW8F28LR13vrRXmmvXkpbgfc8gl9We+Mno27Eng==</latexit>
  • 14. All-or-nothing in sparse PCA I-MMSE relation: d d 1 n I(X; W) = 1 4n2 EkX ⌦ X E[X ⌦ X|W]k2 F <latexit sha1_base64="7sDds8HKzzKXCvjLcngkqhQ1OFE=">AAADYnicjVFdT9RAFL3dosD6wSKP8jBxYwIPbtpCgokxIRoNvEHisptsl810OgsT+pXp1IR056f4u4zvPsi/8M4wBBSNTtP23HPPuTN3blJlolZB8M3r+EsPHi6vrHYfPX7ydK23/uykLhvJ+JCVWSnHCa15Jgo+VEJlfFxJTvMk46Pk4r3Jjz5zWYuy+KQuKz7N6Vkh5oJRhdSs9yWeS8raVLdpnKEtpZpcU6FuC3241cbJnIz1G/sf6W3yltwKdovTCPU5VedJ0n7Q8cLJ41KJnNfEha/uaCZ/lizcBtN4MWtjmZOP+jSa9frBILCL3AehA31w66jsfYUYUiiBQQM5cChAIc6AQo3PBEIIoEJuCi1yEpGweQ4auuhtUMVRQZG9wO8ZRhPHFhibmrV1M9wlw1eik8BL9JSok4jNbsTmG1vZsH+r3dqa5myX+E9crRxZBefI/st3o/xfn+lFwRxe2x4E9lRZxnTHXJXG3oo5ObnTlcIKFXIGp5iXiJl13twzsZ7a9m7ultr8D6s0rImZ0zZwZU6JAw5/H+d9cBINwp1BdLzb33/nRr0Cz+EFbOE892AfDuAIhsA839v2Im+n893v+uv+xrW04znPBvyy/M2fXdTHNg==</latexit> IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit> IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit> ⇢ ! 0+ gives IT(⇢) ⇢!0 ! 4| ln ⇢| ⇢ [Lesieur Krzakala Zdeborová 15]
  • 15. ⇢n ! 0+<latexit sha1_base64="CsLOCzcFzETb5Ob3d8nGwZTOSr4=">AAAC4nicjVHLSsNAFD2N7/qKuhQhWARBKGkVdCm6calgbaEtJUmnbWiaCZOJUEJX7tyJW3/ArX6M+Af6F94ZU1CL6IQkZ86958zce90o8GNp2685Y2p6ZnZufiG/uLS8smqurV/FPBEeq3g84KLmOjEL/JBVpC8DVosEcwZuwKpu/1TFq9dMxD4PL+UwYs2B0w39ju85kqiWudUQPd5KG9opFaw9SsPRyGpIbtmtvZZZsIu2XtYkKGWggGydc/MFDbTB4SHBAAwhJOEADmJ66ijBRkRcEylxgpCv4wwj5EmbUBajDIfYPn27tKtnbEh75RlrtUenBPQKUlrYIQ2nPEFYnWbpeKKdFfubd6o91d2G9HczrwGxEj1i/9KNM/+rU7VIdHCka/Cppkgzqjovc0l0V9TNrS9VSXKIiFO4TXFB2NPKcZ8trYl17aq3jo6/6UzFqr2X5SZ4V7ekAZd+jnMSXJWLpf1i+eKgcHySjXoem9jGLs3zEMc4wzkq5H2DRzzh2Wgbt8adcf+ZauQyzQa+LePhA38Am5s=</latexit> n ! +1<latexit sha1_base64="Haf+8ljzRuATXGTIzWyIZOsLN5E=">AAAC6XicjVHLSsNAFD2Nr1pfVZduglUQhJJWQZdFNy4r2FZoSknSqQ5NkzCZCCX0B9y5E7f+gFv9EfEP9C+8M6bgA9EJSc6ce86ZuTNu5PNYWtZLzpianpmdy88XFhaXlleKq2vNOEyExxpe6Ifi3HVi5vOANSSXPjuPBHOGrs9a7uBY1VtXTMQ8DM7kKGKdoXMR8D73HElUt7hl+yTuOd3U1mGpYL1xGozHpi1Dc9fmQV+OusWSVbb0MH+CSgZKyEY9LD7DRg8hPCQYgiGAJOzDQUxPGxVYiIjrICVOEOK6zjBGgbwJqRgpHGIH9L2gWTtjA5qrzFi7PVrFp1eQ08Q2eULSCcJqNVPXE52s2N+yU52p9jaiv5tlDYmVuCT2L99E+V+f6kWij0PdA6eeIs2o7rwsJdGnonZufupKUkJEnMI9qgvCnnZOztnUnlj3rs7W0fVXrVSsmnuZNsGb2iVdcOX7df4EzWq5sleunu6XakfZVeexgU3s0H0eoIYT1NGg7Gs84BFPxsC4MW6Nuw+pkcs86/gyjPt3I/qe8A==</latexit> ⇢<latexit sha1_base64="jB/4cBQZIOHbHf/E/Vfv7BykrbE=">AAACx3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdFl0o7sK9gFtkWQ6bUPTTJhMiqW48Afc6p+Jf6B/4Z0xBbWITkhy5tx7zsy914/DIFGO85qzFhaXllfyq4W19Y3NreL2Tj0RqWS8xkQoZNP3Eh4GEa+pQIW8GUvujfyQN/zhhY43xlwmgYhu1CTmnZHXj4JewDylqbYciNtiySk7ZtnzwM1ACdmqiuIL2uhCgCHFCBwRFOEQHhJ6WnDhICaugylxklBg4hz3KJA2pSxOGR6xQ/r2adfK2Ij22jMxakanhPRKUto4II2gPElYn2abeGqcNfub99R46rtN6O9nXiNiFQbE/qWbZf5Xp2tR6OHM1BBQTbFhdHUsc0lNV/TN7S9VKXKIidO4S3FJmBnlrM+20SSmdt1bz8TfTKZm9Z5luSne9S1pwO7Pcc6D+lHZPS4fXZ+UKufZqPPYwz4OaZ6nqOASVdTIe4BHPOHZurKENbbuPlOtXKbZxbdlPXwAqx2QzA==</latexit> VERY high sparsity Linked to planted clique = 1 <latexit sha1_base64="jXUSXRRJOQDAgDSJoWqmpgnmM6E=">AAAC2XicjVHLSsNAFD3GV33Xx85NsAiuSqKCboSiLlxWsFVopUymUw1OkzCZCLW4cCdu/QG3+kPiH+hfeGeMoBbRCUnOnHvPmbn3BokMU+15L0PO8Mjo2HhhYnJqemZ2rji/UE/jTHFR47GM1UnAUiHDSNR0qKU4SZRg3UCK4+Biz8SPL4VKwzg60r1EnHbZWRR2Qs40Ua3iUnNfSM12mh3FuN+UpGyzVrHklT273EHg56CEfFXj4jOaaCMGR4YuBCJowhIMKT0N+PCQEHeKPnGKUGjjAteYJG1GWYIyGLEX9D2jXSNnI9obz9SqOZ0i6VWkdLFKmpjyFGFzmmvjmXU27G/efetp7tajf5B7dYnVOCf2L91n5n91phaNDrZtDSHVlFjGVMdzl8x2xdzc/VKVJoeEOIPbFFeEuVV+9tm1mtTWbnrLbPzVZhrW7Hmem+HN3JIG7P8c5yCor5f9jfL64WapspuPuoBlrGCN5rmFCg5QRY28r/CARzw5DefGuXXuPlKdoVyziG/LuX8HkPmXYQ==</latexit> Sparse PCA: VERY high sparsity regime W = r n n X ⌦ X + Z Xi ⇠ Ber(⇢n) i = 1, . . . , n<latexit sha1_base64="zDIqNDRQ8LcLOTNSia4xfXI5HVo=">AAADVnicjVFfaxQxEJ+9s7ae/876KELw8KxYjt0q2JdCqS8+VvB6h005stlcG5rdbJOsUJb9EH469RvoJ/BNnMQU1EM0yyYzv5nfLzOZvFbSujT9nPT619aur2/cGNy8dfvO3eG9zSOrG8PFlGulzTxnVihZiamTTol5bQQrcyVm+fkrH5+9F8ZKXb11l7U4KdlpJZeSM4fQYvihpfmSzLrxHrUXxrV0aRhvqUKFgi3aloYrWiOKjlRd17W4Bcq8o9rJUlgS3WfhfNdROpgv5JhaWWLIlORAmG6LmjO9KveU0IuGFUTuZdtUFdrZbaxplE7SsMiqkUVjBHEd6uFHoFCABg4NlCCgAoe2AgYWv2PIIIUasRNoETNoyRAX0MEAuQ1mCcxgiJ7jforecUQr9L2mDWyOtyj8DTIJPEaOxjyDtr+NhHgTlD36N+02aPraLvHMo1aJqIMzRP/Fu8r8X57vxcESdkMPEnuqA+K741GlCa/iKye/dOVQoUbM2wXGDdo8MK/emQSODb37t2Uh/iVketT7POY28NVXiQPO/hznqnG0M8meT3bevBjtH8RRb8ADeARbOM+XsA+v4RCmqP0teZiMkye9T73v/bX++s/UXhI59+G31R/+AJqtxag=</latexit> [Berthet Rigollet 13] [Deshpande Montanari 13]
  • 16. Finite size bound for the mutual information Theorem (B. Macris 19): based on the adaptive interpolation method Let P0,n = ⇢n p0 + (1 ⇢n) 0. Let n = ⇥(| ln ⇢n|/⇢n) which is the appro- priate scaling to observe a phase transition, and ⇢n = ⇥(n ) with 2 [0, 1/6) Then 1 ⇢n| ln ⇢n| 1 n I(X; W) inf q2[0,1] ipot n (q; n, ⇢n)  C (ln n)1/3 n(1 6 )/7 with i(pot) n (q; n, ⇢n) = n⇢2 n 4 (q 1)2 + In(X; p n⇢nq X + Z) X ⇠ P0,n, Z ⇠ N(0, 1). <latexit sha1_base64="fQUbEiIQan28f7ueBu1sSyqjneA=">AAAFDHicjVLbbtNAEN2GACXcWnjkZURLldBc7BRapCpS1UqISggVqZeIbmNtnE2yqrN27E2hcvcX+BPeeEO88gM8ICR4h79gduMUaEHgKPHsmTPnzMymHQUiUY7zeSp3IX/x0uXpK4Wr167fuDkze2s3CUexz3f8MAjjZpslPBCS7yihAt6MYs4G7YDvtQ83TH7viMeJCOW2Oo74wYD1pOgKnymEvNnc46dcwfyWlzplqRs07oeeBFqGyHMWi25lDJRohweKec58FSyfBmjRYZ5sAN3uc8WKJzSQY/JJLSuah5d94fdBJKD6HFgUxWEUC6Y4JD7DjnugQgjbCY+PMA1RHwcBFTOZCNNdGZjsoJdVa2Q+spVWaBsjbfSF6iPBHKmQ+07ZrS0jjExZQLQnZIo+PXlfFxZoN2Z+6uo06/Jnv5qui97JaV7qzWJK211o6lX73tMlqKB+10uH+ALjc6BBtFBqAFGotCeLw9XTnZQnSzOyQAMOG2PxInqCLLVSt7ak0aiV4oaXbful2orWlBao4q9UaubSQIcj1oEF4RniqVdJ/8mrMXY4xcdwq67TB8ivuKVWfXET22yu0mQYq7NEGGq88+bii9LYtEkTMYDsX1EGCi8MkFK8Nnimi7iAUrVAuexM9uvNzDlVxz5wPnCzYI5kz1Y484lQ0iEh8cmIDAgnkiiMA8JIgp994hKHRIgdkBSxGCNh85xoUsDaEbI4Mhiih/jbw9N+hko8G83EVvvoEuA3xkog97AmRF6MsXEDmx9ZZYP+TTu1mqa3Y3y3M60Boor0Ef1X3YT5v3VmFkW65JGdQeBMkUXMdH6mMrJbMZ3DL1MpVIgQM3EH8zHGvq2c7BlsTWJnN7tlNv/NMg1qzn7GHZHvpku8YPfsdZ4PdutVd6n68Hl9bm09u+ppcofcJUW8zxWyRp6QLbJD/Nyb3Mfcl9zX/Ov82/y7/PsxNTeV1dwmvz35Dz8AiglePw==</latexit> C1n 1/2 < ⇢n < C2<latexit sha1_base64="GYXdMrydX8eJ3XaVm05eFoqGSSU=">AAAC3XicjVHLSsNAFD3G97vqRnATLIIba1IFXbgQu3GpYG2hrSFJRw2mmTCZCFJ0507c+gNu9XfEP9C/8M44BbWITkhy5tx7zsy9N0jjKJOO8zpgDQ4Nj4yOjU9MTk3PzBbm5o8znouQVUMec1EP/IzFUcKqMpIxq6eC+Z0gZrXgoqLitUsmsognR/IqZa2Of5ZEp1HoS6K8wmLFc+3kpLvmrpevd5rinHvJjl3xyl6h6JQcvex+4BpQhFkHvPCCJtrgCJGjA4YEknAMHxk9DbhwkBLXQpc4QSjScYZrTJA2pyxGGT6xF/Q9o13DsAntlWem1SGdEtMrSGljhTSc8gRhdZqt47l2Vuxv3l3tqe52Rf/AeHWIlTgn9i9dL/O/OlWLxCm2dQ0R1ZRqRlUXGpdcd0Xd3P5SlSSHlDiF2xQXhEOt7PXZ1ppM16566+v4m85UrNqHJjfHu7olDdj9Oc5+cFwuuRul8uFmcXfPjHoMS1jGKs1zC7vYxwGq5H2DRzzh2fKsW+vOuv9MtQaMZgHflvXwAQUnl3Q=</latexit> left open Replica prediction at ⇢ ! 0+ agrees with ⇢n ! 0+ for ⇢n = o(n 1/2 )<latexit sha1_base64="52bXL9v9NgK4PAqRjfi5jyXoWfU=">AAADOHicjVFNTxsxEJ0shSbhK6VHLlYjJBAi7KZIcKkUlQtHWjWAxELkdZzEyma9sr2lUZQ/xh9BuXEDrvwBOjYbUYoQeLW7M2/ee/Z4ojQW2vj+pODNfJid+1gslecXFpeWK59WjrTMFONNJmOpTiKqeSwS3jTCxPwkVZwOopgfR/19Wz/+zZUWMvllhik/G9BuIjqCUYNQq6JDw/+Y0U+OmzFKUNsWzJYINWMSklD1JAmNJH5r06aOTbuKc00uhOlNOa3kiVV+ZHWkeqp+k+vJ+Wgr2K6PN1qVql/z3SIvgyAPqpCvQ1m5ghDaIIFBBgPgkIDBOAYKGp9TCMCHFLEzGCGmMBKuzmEMZdRmyOLIoIj28dvF7DRHE8ytp3ZqhrvE+CpUElhDjUSewtjuRlw9c84Wfc175Dzt2Yb4j3KvAaIGeoi+pZsy36uzvRjowJ7rQWBPqUNsdyx3ydyt2JOTf7oy6JAiZuM21hXGzCmn90ycRrve7d1SV791TIvanOXcDO7sKXHAwf/jfBkc1WvB11r9x0618T0fdRFW4Qus4zx3oQEHcAhN9J7AQ6FYKHmX3rV34909Ur1CrvkMz5Z3/xfE5roM</latexit> Compressed sensing [Reeves Xu Zadik 19]:
  • 17. 1 ⇢n| ln ⇢n| inf q2[0,1] ipot n (q; n, ⇢n) ! n!+1 I  1 + I > 1 <latexit sha1_base64="pfzcBo5SrMvAvhK3FxHgaDjx7Rw=">AAADeXicjVHdbtMwGP3S8LONvzLu4MZQIXVsqpINaUhIqIIbxtWQ6DapKZHjuq01x84cB6hC3op3QbwBPAM3fPZSCRgIHCU5Pt85x/7srJCitFH0JeiEly5fubq2vnHt+o2bt7q3N49KXRnGR0xLbU4yWnIpFB9ZYSU/KQyneSb5cXb6wtWP33FTCq3e2GXBJzmdKzETjFqk0u6nZGYoq+OmTsxCp+pjIhVpYZMINUvrM/yRcbQTTxoi3qIuJ4W2Tar6Z08TiUtNaap2zj1bJPlgxHxhqTH6/RijrCbbLscuJ3WznsxpnlOS5NQusqw+aJJMzPsrVnISO2Jr+8+CZ2057faiQeQHuQjiFvSgHYe6+xkSmIIGBhXkwEGBRSyBQonPGGKIoEBuAjVyBpHwdQ4NbKC3QhVHBUX2FL9znI1bVuHcZZbezXAVia9BJ4GH6NGoM4jdasTXK5/s2L9l1z7T7W2J/6zNypG1sED2X76V8n99rhcLM3jiexDYU+EZ1x1rUyp/Km7n5KeuLCYUyDk8xbpBzLxzdc7Ee0rfuztb6utfvdKxbs5abQXf3C7xguPfr/MiONodxHuD3dePe8Pn7VWvwT14AH28z30Ywks4hBGw4G4wDA6CV53v4f2wHz46l3aC1nMHfhnh3g8qRtA8</latexit> Setting n ⌘ n,IT(⇢n) where n,IT(⇢n) ⌘ 4| ln ⇢n| ⇢n<latexit sha1_base64="RQtxbdebaThUKCNpnc/TcDPahlo=">AAADS3icjVFBT9swGP0SYGOwjQ6OcLDWTmLSVKVl0nZE48JuoFFAIqhyXLe1cJzgOEwo5Lp/xxnxDzZpt92mHfbZdcU2QMxRkuf3vffsz05yKQoTRddBODM79+jx/JOFxafPni81XizvF1mpGe+xTGb6MKEFl0LxnhFG8sNcc5omkh8kJ1u2fnDGdSEytWfOc36c0pESQ8GoQarf+PKJGyPUiLRiia4B7auYn5bijMQjmqaUTOlKvalinZKPe3W9Hutx1levW+TzmGt+471LRHze23ioKasuYqnIpHRRVxNQt/qNZtSO3CC3QceDJvixkzWuIIYBZMCghBQ4KDCIJVAo8DmCDkSQI3cMFXIakXB1DjUsoLdEFUcFRfYEvyOcHXlW4dxmFs7NcBWJr0YngVfoyVCnEdvViKuXLtmy92VXLtPu7Rz/ic9KkTUwRvYh31T5vz7bi4EhvHc9COwpd4ztjvmU0p2K3Tn5oyuDCTlyFg+wrhEz55yeM3GewvVuz5a6+lentKydM68t4ZvdJV5w59/rvA32u+3ORru7221ufvBXPQ+r8BLW8T7fwSZsww70MPt7sBysBmvhZfgj/Bn+mkjDwHtW4K8xM/cbePrB3A==</latexit> All-or-nothing in sparse PCA 1 (n⇢n)2 EkX ⌦ X E[X ⌦ X|W]k2 F ! n!1 I(  1) <latexit sha1_base64="sJaHnwbzC0ucbLGuGU+Q1r3AbWo=">AAADZXicjVFdb9MwFL1pYIzBoAPECw9YVEidBFVSxsfjBALB25DoWqnuKsdzWmuJHTkOUGX5L/wrxC8A/gXXXsbXQOAoybnnnnPt65sUmSxtFH0KOuG582sX1i9uXLq8eeVqd+vafqkrw8WI60ybScJKkUklRlbaTEwKI1ieZGKcHD1z+fFbYUqp1Ru7KsQsZwslU8mZRWre/UBTw3gdN3VfUbPUc7V9MGxozuwySernDT2uaZKSSUO1lbkoSRveJz800z9LTpzjZkaP5zU1OXnRHAzpeyMXS8uM0e+milpNpUrtalZ/3/NV06cLlueMZoLE2/NuLxpEfpGzIG5BD9q1p7sfgcIhaOBQQQ4CFFjEGTAo8ZlCDBEUyM2gRs4gkj4voIEN9FaoEqhgyB7hd4HRtGUVxq5m6d0cd8nwNegkcBc9GnUGsduN+HzlKzv2b7VrX9OdbYX/pK2VI2thiey/fKfK//W5Xiyk8MT3ILGnwjOuO95WqfytuJOTn7qyWKFAzuFDzBvE3DtP75l4T+l7d3fLfP6LVzrWxbzVVvDVnRIHHP8+zrNgfziIHwyGr3d6u0/bUa/DLbgDfZznY9iFl7AHI+DBWnAveBg86nwON8Mb4c0TaSdoPdfhlxXe/gYbSspP</latexit> [The overlap gap property, Gamarnik Jagannath Sen 19] In agreement with the recent study of the Maximum Likelihood Estimator for FIXED but small ⇢ ! 0+<latexit sha1_base64="RM3Ida9m8zVRPO2cA0dUwxlRUtA=">AAACznicjVHLSsNAFD2Nr1pfVZdugkUQhJJUQZdFNy4r2FZoS0nSaRuaZMJkUiiluPUH3OpniX+gf+GdMQW1iE5Icubcc+7MvdeNAz+RlvWaM5aWV1bX8uuFjc2t7Z3i7l4j4anwWN3jARd3rpOwwI9YXfoyYHexYE7oBqzpjq5UvDlmIvF5dCsnMeuEziDy+77nSKJabTHkbclNq3vSLZassqWXuQjsDJSQrRovvqCNHjg8pAjBEEESDuAgoacFGxZi4jqYEicI+TrOMEOBvCmpGCkcYkf0HdCulbER7VXORLs9OiWgV5DTxBF5OOkEYXWaqeOpzqzY33JPdU51twn93SxXSKzEkNi/fHPlf32qFok+LnQNPtUUa0ZV52VZUt0VdXPzS1WSMsTEKdyjuCDsaee8z6b2JLp21VtHx9+0UrFq72XaFO/qljRg++c4F0GjUrZPy5Wbs1L1Mht1Hgc4xDHN8xxVXKOGuu74I57wbNSMsTEz7j+lRi7z7OPbMh4+AMI1kys=</latexit> MLE recovers a constant fraction of the support of X i↵ > C | ln ⇢| ⇢ .<latexit sha1_base64="M8HoApg3dBZlTLYJIWcUjk20uLQ=">AAADKnicjVHBThRBEC0GFUTUFY9eKu6acNrMLgc9GSIh4aAJJi5swhDS09vDduidnnT3kJBl/4g/4caFEK8e9OgVLla1Q6ISoz2ZmVev3qvu6soro31I06u5ZP7e/QcLiw+XHi0/fvK09Wxlx9vaSTWQ1lg3zIVXRpdqEHQwalg5JSa5Ubv50Qbnd4+V89qWn8JJpfYn4rDUhZYiEHXQUh/eb6JT0rIIBUpb+iDKgIUTkiVoCwxjhb6uKusCh50sL3DYQV0wNrTZSOBb3MCMTdPTzJSYubE9nU35N+t0D1rttJvGhXdBrwFtaNa2bV1CBiOwIKGGCSgoIRA2IMDTswc9SKEibh+mxDlCOuYVzGCJvDWpFCkEsUf0PaRor2FLirmmj25Juxh6HTkRXpHHks4R5t0w5utYmdm/1Z7Gmny2E/rnTa0JsQHGxP7Ld6v8Xx/3EqCAN7EHTT1VkeHuZFOljrfCJ8dfugpUoSKO8YjyjrCMztt7xujxsXe+WxHzX6OSWY5lo63hG5+SBtz7c5x3wU6/21vr9j/22+vvmlEvwgt4Cas0z9ewDluwDQOqfQ7f4RpukrPkIrlKPv+UJnON5zn8tpIvPwAPebad</latexit> = 1<latexit sha1_base64="qFglFRl5V+IA2gC5aE1LqSuQEAw=">AAACy3icjVHLSsNAFD3GV62vqks3wSK4KokKuhGKbtwIFewDbJFJOq2heTGZCLW69Afc6n+Jf6B/4Z1xCmoRnZDkzLnn3Jl7r5eGQSYd53XKmp6ZnZsvLBQXl5ZXVktr640syYXP634SJqLlsYyHQczrMpAhb6WCs8gLedMbnKh484aLLEjiCzlMeSdi/TjoBT6TRLXafRZF7Mi9KpWdiqOXPQlcA8owq5aUXtBGFwl85IjAEUMSDsGQ0XMJFw5S4joYEScIBTrOcY8ieXNScVIwYgf07dPu0rAx7VXOTLt9OiWkV5DTxjZ5EtIJwuo0W8dznVmxv+Ue6ZzqbkP6eyZXRKzENbF/+cbK//pULRI9HOoaAqop1YyqzjdZct0VdXP7S1WSMqTEKdyluCDsa+e4z7b2ZLp21Vum429aqVi19402x7u6JQ3Y/TnOSdDYrbh7ld3z/XL12Iy6gE1sYYfmeYAqTlFDXc/xEU94ts6szLq17j6l1pTxbODbsh4+AAnqkhw=</latexit> Replica picture valid when ⇢n = ⇥(n ) with 2 [0, 1/6).<latexit sha1_base64="DmgcLFf/+9pNk8Sk0Om4CtZd8xc=">AAADE3icjVFBT9RAFP6oCggCqx69TNg1gUSWFiJwISF48WSAsECyxc10dqATutNmOoWQDT/Df+LNG/Hq2ehVD/gveDOUBCVGp2n7ve+975t585IiU6UNw28jwYOHj0bHxh9PTD6Zmp5pPH22V+aVEbIj8iw3BwkvZaa07FhlM3lQGMkHSSb3k5M3Lr9/Kk2pcr1rzwt5OODHWh0pwS1Rvca7HUnbCM4KJWxlJDvlmeqzs1Rq1opNmvf0erybSsvn9PvhQpwQuphvsTNlUypwYax0N3wVLa7Mt9q9RjNsh36x+yCqQRP12sobXxGjjxwCFQaQ0LCEM3CU9HQRIURB3CGGxBlCyuclLjBB2oqqJFVwYk/oe0xRt2Y1xc6z9GpBu2T0GlIyvCRNTnWGsNuN+XzlnR37N++h93RnO6d/UnsNiLVIif2X7rbyf3WuF4sjrPkeFPVUeMZ1J2qXyt+KOzm705Ulh4I4h/uUN4SFV97eM/Oa0vfu7pb7/JWvdKyLRV1b4Zc7JQ04+nOc98HeUjtabr/eXmpubNajHscLzGKO5rmKDbzFFjrk/RHf8QM/gw/Bp+Ay+HxTGozUmuf4bQVfrgG+qazR</latexit>
  • 18. All-or-nothing transition: a generic phenomenon? Error SNR
  • 19. All-or-nothing in sparse PCA: Wishart case W = r n n U ⌦ V + Z <latexit sha1_base64="QQmmu/mvgT+bHyK0p3O/5TzzQag=">AAADBXicjVHLShxBFD22eaiJycQs3TQZhEBg6MmDZBMYdJOlQmZGtGWorqkxhf1KVXVAml7nT9xlF9zmB7JNxD8wf+GtawuJIqaa7jr33HtO162blKm2LopO54L5O3fv3V9YXHrwcPnR486TlZEtKiPVUBZpYbYTYVWqczV02qVquzRKZEmqxsnBhs+PvyhjdZF/dIel2svEfq5nWgpH1KQzqONkFo6b97H9bFwdz4yQdZySwVRM8qbOm4Yrhk1cOJ0pG3I4al7wvtNMOt2oF/EKr4N+C7po12bROUGMKQpIVMigkMMRTiFg6dlFHxFK4vZQE2cIac4rNFgibUVViioEsQf03adot2Vzir2nZbWkv6T0GlKGWCNNQXWGsP9byPmKnT17k3fNnv5sh7QnrVdGrMMnYm/TXVb+r8734jDDO+5BU08lM7472bpUfCv+5OFfXTlyKInzeEp5Q1iy8vKeQ9ZY7t3freD8GVd61seyra3wx5+SBty/Os7rYPSy13/Ve7P1ujtYb0e9gFU8w3Oa51sM8AGbGJL3EX7iF34HX4Nvwffg+KI0mGs1T/HPCn6cAydXqnI=</latexit> lim n!1 1 m2 MMSE(U ⌦ U|W) <latexit sha1_base64="rQcaoCEbwAEhn6NlOtX6kY10PPs=">AAADDnicjVFNT9VAFD1U+RABn7p0M/HFBDYvLWhkSTQmuiDB6OORUHhphylMXttpplOTl9r/4D9x58649Q8Y40a3+i+8Mw6JQAhM0865595zOnduWuWyNmH4Yya4cXN2bn7h1uLtpeWVO72793Zr1WguhlzlSu+lSS1yWYqhkSYXe5UWSZHmYpROntv86J3QtVTlWzOtxEGRHJcykzwxRI17r+JcFuO2jI1isSwzM+3iTCe8jbq2OFzv2lgXbHv7zYtutY3TjA27WBlZiJr58L3bR93auNcPB6Fb7CKIPOjDrx3V+44YR1DgaFBAoIQhnCNBTc8+IoSoiDtAS5wmJF1eoMMiaRuqElSREDuh7zFF+54tKbaetVNz+ktOryYlwyPSKKrThO3fmMs3ztmyl3m3ztOebUp76r0KYg1OiL1Kd1p5XZ3txSDDputBUk+VY2x33Ls07lbsydl/XRlyqIiz+IjymjB3ytN7Zk5Tu97t3SYu/9tVWtbG3Nc2+GNPSQOOzo/zIthdH0QbgyevH/e3nvlRL+ABHmKV5vkUW3iJHQzJ+yO+4Sd+BR+CT8Hn4Mu/0mDGa+7jzAq+/gXGKq1/</latexit> lim n!1 1 (n⇢)2 MMSE(V ⌦ V|W) <latexit sha1_base64="4egtK9Z9u3TX01Q29gJAntlvm+A=">AAADFHicjVFNb9QwEH0NX6V8dIEjF4sV0vaySgoVHCtQJS4VRbC7leqySlyntZrEkeMgrUL+Rv8JN26IK1eEuIIE/4KxcSWgQuAo8Zs38148nqwuVGPj+NNSdO78hYuXli+vXLl67frq4MbNaaNbI+RE6EKb3SxtZKEqObHKFnK3NjIts0LOsuPHLj97JU2jdPXCLmq5X6aHlcqVSC1R88FTXqhy3lXcasZVldtFz3OTii7pu1HFzZFee7ned9yUbHv7+VY/6niWs2nPtVWlbFgIX/t91q/NB8N4HPvFzoIkgCHC2tGDj+A4gIZAixISFSzhAikaevaQIEZN3D464gwh5fMSPVZI21KVpIqU2GP6HlK0F9iKYufZeLWgvxT0GlIy3CWNpjpD2P2N+XzrnR37N+/Oe7qzLWjPgldJrMURsf/SnVb+r871YpHjoe9BUU+1Z1x3Iri0/lbcydkvXVlyqIlz+IDyhrDwytN7Zl7T+N7d3aY+/81XOtbFItS2+O5OSQNO/hznWTBdHyf3xhvP7g83H4VRL+M27mBE83yATTzBDibk/Qaf8QVfo5PobfQuev+zNFoKmlv4bUUffgD9na+0</latexit> Ui ⇠ N(0, 1) i = 1 . . . , m Vj ⇠ Ber(⇢) i = 1 . . . , n Zij ⇠ N(0, 1)<latexit sha1_base64="4c4HJih7GNY/srfVTqFV2E4NQtI=">AAADN3icjVFdTxQxFL07frCuX6s++tKwkawJ2cyoRF9MCL7wRCBhFyIlk063QKEzHdqOCZnsD/OXqE++EXj0Dxhua0nQjdFOZub03HNOe9uiVtK6NP3WSW7dvnN3oXuvd//Bw0eP+0+eTqxuDBdjrpU2uwWzQslKjJ10SuzWRrCyUGKnOPng6zufhLFSV9vurBb7JTus5IHkzCGV983SOJfUyrKlnCmyMRumy9lLQk8bNiXyfUbVVDu7XFLaW5rkx15JWmpKsibMbEjNkZ4XV178MW/l8YxEw43sXt4fpKM0DDIPsggGEMem7n8BClPQwKGBEgRU4BArYGDx2YMMUqiR24cWOYNIhrqAGfTQ26BKoIIhe4LfQ5ztRbbCuc+0wc1xFYWvQSeBF+jRqDOI/Wok1JuQ7Nm/Zbch0+/tDP9FzCqRdXCE7L9818r/9fleHBzAu9CDxJ7qwPjueExpwqn4nZMbXTlMqJHzeIp1g5gH5/U5k+CxoXd/tizUL4LSs37Oo7aBS79LvODsz+ucB5NXo+z1aGXrzWB1LV51F57DIgzxPt/CKqzDJowx+yv87Cx0usnn5Htynlz8kiad6HkGv43kxxUAirct</latexit> IT(⇢) ! s 4 | ln ⇢| ↵⇢<latexit sha1_base64="5GWkqPGzV16w66Ro/Q2kIfYGKM0=">AAADBnicjVFNT9wwEH2kH9AtbZdy7CVihQSXVbbdih6hXNrbVmIBCaOV4/WyEU6cOk4lFHLvP+mtN9Qrf6DXrvoPyr9gbIJUQBV1lOTNm3nPHk+cq6SwUfR7Lnjw8NHj+YUnraeLz56/aC+93C10aYQcCq202Y95IVWSyaFNrJL7uZE8jZXci4+3XX7vizRForMde5LLw5QfZckkEdwSNWpvMUXFYz6qmEnDjzv1GjNTvc6sDlnx2diqzyaGi+qUqSx0qdO6YlzlU+6jum6N2p2oG/kV3gW9BnTQrIFuz8AwhoZAiRQSGSxhBY6CngP0ECEn7hAVcYZQ4vMSNVqkLalKUgUn9pi+RxQdNGxGsfMsvFrQLopeQ8oQq6TRVGcIu91Cny+9s2P/5V15T3e2E/rHjVdKrMWU2Pt015X/q3O9WEzwzveQUE+5Z1x3onEp/a24k4d/dWXJISfO4THlDWHhldf3HHpN4Xt3d8t9/o+vdKyLRVNb4sKdkgbcuz3Ou2D3dbf3pvv2U7+z+b4Z9QJeYQVrNM8NbOIDBhiS9zf8xC/Mgq/B9+As+HFVGsw1mmXcWMH5JUR1qnw=</latexit> IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit> IT(⇢)<latexit sha1_base64="BOhMTTzJYGVOXKi1lzzb25hr/fw=">AAAC63icjVHLSsNAFD3GV31XXboJFqVuSloFXYpudFfBqmCkTKbTNjQvJhNBSv/AnTtx6w+41f8Q/0D/wjtjCj4QnZDkzLn3nJl7r5cEfqoc52XEGh0bn5gsTE3PzM7NLxQXl07SOJNcNHgcxPLMY6kI/Eg0lK8CcZZIwUIvEKdeb1/HTy+FTP04OlZXibgIWSfy2z5niqhmcd1tS8b7bkCSFhsMQbPvytA+PB6UXdmNNwbNYsmpOGbZP0E1ByXkqx4Xn+GihRgcGUIIRFCEAzCk9JyjCgcJcRfoEycJ+SYuMMA0aTPKEpTBiO3Rt0O785yNaK89U6PmdEpArySljTXSxJQnCevTbBPPjLNmf/PuG099tyv6e7lXSKxCl9i/dMPM/+p0LQpt7JgafKopMYyujucumemKvrn9qSpFDglxGrcoLglzoxz22Taa1NSue8tM/NVkalbveZ6b4U3fkgZc/T7On+CkVqluVmpHW6XdvXzUBaxgFWWa5zZ2cYA6GuR9jQc84skKrRvr1rr7SLVGcs0yvizr/h1E259T</latexit>
  • 20. Generalizationerror W = sign( X)<latexit sha1_base64="WX5GPt3XTil6001t4w8zQyKp+Ms=">AAAC8XicjVHLSsQwFD3W93vUpZviICjI0FFBN4LoxuUIjjNgRdpOZgz2RZIKQ+lPuHMnbv0Bt/oT4h/oX3gTK/hANKXNybnnnOYmfhpyqRznecAaHBoeGR0bn5icmp6ZrczNH8skEwFrBkmYiLbvSRbymDUVVyFrp4J5kR+yln+xr+utSyYkT+Ij1U/ZaeT1Yt7lgaeIOqus5a7ftVvFTu6KyJa8Fxcrrp+EHdmPaMrdxjkvjKZdrJ5Vqk7NMcP+CeolqKIcjaTyBBcdJAiQIQJDDEU4hAdJzwnqcJASd4qcOEGImzpDgQnyZqRipPCIvaBvj1YnJRvTWmdK4w7oLyG9gpw2lsmTkE4Q1n+zTT0zyZr9LTs3mXpvfZr9MisiVuGc2L98H8r/+nQvCl1smx449ZQaRncXlCmZORW9c/tTV4oSUuI07lBdEA6M8+OcbeORpnd9tp6pvxilZvU6KLUZXvUu6YLr36/zJzher9U3auuHm9XdvfKqx7CIJazQfW5hFwdooEnZV7jHAx4taV1bN9btu9QaKD0L+DKsuzd5SqGp</latexit> X ±1<latexit sha1_base64="nmXQk1ldbUwYZLkkbnpMKZHj/ZE=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVZIq6LLoRlxVsLXQFknSaR2axzCZKKW48Qfc6peJf6B/4Z0xBbWITkhy5tx7zsy91xchT5XjvBasufmFxaXicmlldW19o7y51UqTTAasGSRhItu+l7KQx6ypuApZW0jmRX7IrvzRqY5f3TKZ8iS+VGPBepE3jPmAB54iqtkVke1elytO1THLngVuDirIVyMpv6CLPhIEyBCBIYYiHMJDSk8HLhwI4nqYECcJcRNnuEeJtBllMcrwiB3Rd0i7Ts7GtNeeqVEHdEpIrySljT3SJJQnCevTbBPPjLNmf/OeGE99tzH9/dwrIlbhhti/dNPM/+p0LQoDHJsaONUkDKOrC3KXzHRF39z+UpUiB0Gcxn2KS8KBUU77bBtNamrXvfVM/M1kalbvgzw3w7u+JQ3Y/TnOWdCqVd2Dau3isFI/yUddxA52sU/zPEIdZ2igSd4cj3jCs3VuCevOGn+mWoVcs41vy3r4AJIdkLs=</latexit> sign(·)<latexit sha1_base64="ychA8oOiRO4WejO0Ftjv52RsrFk=">AAAC1nicjVHLSsNAFD2Nr/quunQTLIJuSqqCLotuXFawD2hLSaZjHZoXk4lSiu7ErT/gVj9J/AP9C++MKahFdEKSM+fec2buvV7si0Q5zmvOmpqemZ3Lzy8sLi2vrBbW1utJlErGayzyI9n03IT7IuQ1JZTPm7HkbuD5vOENTnS8ccVlIqLwXA1j3gncfiguBHMVUd3C2qgtAzsR/fBmp816kdrtFopOyTHLngTlDBSRrWpUeEEbPURgSBGAI4Qi7MNFQk8LZTiIietgRJwkJEyc4wYLpE0pi1OGS+yAvn3atTI2pL32TIya0Sk+vZKUNrZJE1GeJKxPs008Nc6a/c17ZDz13Yb09zKvgFiFS2L/0o0z/6vTtShc4MjUIKim2DC6Opa5pKYr+ub2l6oUOcTEadyjuCTMjHLcZ9toElO77q1r4m8mU7N6z7LcFO/6ljTg8s9xToL6Xqm8X9o7OyhWjrNR57GJLezQPA9RwSmqqJH3NR7xhGerad1ad9b9Z6qVyzQb+Lashw+MtJY1</latexit> { µ, Yµ}data<latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit><latexit sha1_base64="ySg0iifOf/801N1+P4n8TEQMVqI=">AAAC73icjVHLSsQwFD3W93vUpZviIIjI0Iqgy0E3LkdwnBErQ9qJGuzLNBWGMv/gzp249Qfc6l+If6B/4U2s4APRlCYn595zkpvrp6HIlOM8D1iDQ8Mjo2PjE5NT0zOzlbn5gyzJZcCbQRImsu2zjIci5k0lVMjbqeQs8kPe8s93dLx1yWUmknhf9VJ+HLHTWJyIgCmiOpVVr/D8JOxmvYiWwmuciX7Hi/K1Qz17/U7hycjuMsX6nUrVqTlm2D+BW4IqytFIKk/w0EWCADkicMRQhEMwZPQdwYWDlLhjFMRJQsLEOfqYIG1OWZwyGLHnNJ/S7qhkY9prz8yoAzolpF+S0sYyaRLKk4T1abaJ58ZZs795F8ZT361Hq196RcQqnBH7l+4j8786XYvCCbZMDYJqSg2jqwtKl9y8ir65/akqRQ4pcRp3KS4JB0b58c620WSmdv22zMRfTKZm9T4oc3O86ltSg93v7fwJDtZrrlNz9zaq9e2y1WNYxBJWqJ+bqGMXDTTJ+wr3eMCjdWFdWzfW7XuqNVBqFvBlWHdvI1Ghkw==</latexit> 0-1 transition in the perceptron generalization performance ↵IT(⇢) ! ⇢| ln ⇢| ln 2<latexit sha1_base64="uIQF3NEkW1i+d5bBLeCJJC88RCU=">AAAC+3icjVHBTttAEH24pQ0pbQMcuViNkOglcqAtHCO40FsqEYiEUbReNomVtddarysh4z/prbeKa3+g1/Ze8Qf0Lzq7dSRoVJW1bL95M+/tzk6UyTg3QXCz5D16vPzkaWOl+Wz1+YuXrbX1k1wVmosBV1LpYcRyIeNUDExspBhmWrAkkuI0mh3a/OlHofNYpcfmMhPnCZuk8TjmzBA1ar0LmcymbFSGOvHfH1fboZ6q16FRfjjWjJc2vApl6jtQlRbuVM1Rqx10Arf8RdCtQRv16qvWT4S4gAJHgQQCKQxhCYacnjN0ESAj7hwlcZpQ7PICFZqkLahKUAUjdkbfCUVnNZtSbD1zp+a0i6RXk9LHFmkU1WnCdjff5QvnbNl/eZfO057tkv5R7ZUQazAl9n+6eeVDdbYXgzH2XQ8x9ZQ5xnbHa5fC3Yo9uX+nK0MOGXEWX1BeE+ZOOb9n32ly17u9W+byt67SsjbmdW2BX/aUNODu3+NcBCc7ne5u5+2HN+3eQT3qBjbxCts0zz30cIQ+BuT9Cd/wHT+8yvvsffGu/5R6S7VmA/eW9/U3mT2lZw==</latexit>
  • 21. Proof idea: adaptive interpolation method [B. Macris 16] ( Wij(t) = q (1 t) n n XiXj + Zij , 1  i < j  n , fWi(t, ✏) = p Rn(t, ✏) Xi + eZi , 1  i  n ,<latexit sha1_base64="Cg3NYN1mLrUUqg5ElqB/esfJD9o=">AAADqXicjVFdb9MwFL1Z+Bjhq4NHXiwqUKuVKhlI4wGkCV4QTxuia7V5ihLH7dy5Tpo4oBLldyLeeYB/wbXnaoMJgaMk1+fec46vb1pIUekw/OZt+Neu37i5eSu4fefuvfudrQeHVV6XjI9YLvNykiYVl0LxkRZa8klR8mSRSj5Oz96a/PgTLyuRq496VfCTRTJTYipYohGKO99pymdCNQw1qjYYx42Ytz3dJ09f02pZ6qYXPdN9Oi0T1lCJulkSq7ZRbUsHZBKLSTwn2+TI0uhgQOhyWScZiajkRLwic/NXmKA0oJ9FxrWQGW/GbSx6ekB5UQmZqz5BO3Lu9yFWlzMoii5occFGN3Fh5Qyps1z7BZSrzHVF4k43HIZ2katB5IIuuLWfd74ChQxyYFDDAjgo0BhLSKDC5xgiCKFA7AQaxEqMhM1zaCFAbo1VHCsSRM/wO8PdsUMV7o1mZdkMXSS+JTIJPEFOjnUlxsaN2HxtlQ36N+3GapqzrfCfOq0FohpOEf0Xb135vzzTi4YpvLQ9COypsIjpjjmV2t6KOTm51JVGhQIxE2eYLzFmlrm+Z2I5le3d3G1i8z9spUHNnrnaGn6aU+KAoz/HeTU43BlGz4c7By+6e2/cqDfhETyGHs5zF/bgHezDCJj33iu8lffF3/YP/Il/dF664TnOQ/ht+ewXdGLeQQ==</latexit> Interpolating model: in(t, ✏) ⌘ 1 n I(X; W(t), fW(t, ✏)) <latexit sha1_base64="lG/fuMgZHpabDG/n3b0IKlOYje0=">AAADG3icjVFPS9xAHH2mWv/XtT16CS6WXZAlUUGhl6W9tDcF9w8YWZLZWR2cTeJkoixhP0q/SW+9lV5L6dWTpX4IfzNGWxWxE5J5837vvcxvJkqlyLTn/ZpwXkxOvZyemZ2bX1h8tVRZft3Oklwx3mKJTFQ3CjMuRcxbWmjJu6ni4TCSvBOdfDD1zhlXmUjifT1K+eEwPIrFQLBQE9WrtEWviMc1vR7wNBMyietvA36ai7NgoEJW+GOqfqoVQTRwu+N3du6QvL4enIs+10L2+R35N6Peq1S9hmeH+xj4JaiiHLtJ5ScC9JGAIccQHDE0YYkQGT0H8OEhJe4QBXGKkLB1jjHmyJuTipMiJPaEvke0OijZmNYmM7NuRn+R9CpyulgjT0I6Rdj8zbX13CYb9qnswmaavY1ojsqsIbEax8Q+57tV/q/P9KIxwI7tQVBPqWVMd6xMye2pmJ27/3SlKSElzuA+1RVhZp235+xaT2Z7N2cb2vqlVRrWrFmpzfHb7JIu2H94nY9Be6PhbzY29raqzfflVc9gBauo0X1uo4mP2EWLsr/gAn9w5Xx2vjrfnO83Umei9LzBveH8uAbHl7IS</latexit> in(0, ✏) = in(1, ✏) Z 1 0 dt d dt in(t, ✏) <latexit sha1_base64="mO6HXMT4JTGeFWNiFq05NbH+GSM=">AAADEXicjVHLSsQwFD3W93vUpZviICjo0KqgG0F048KFgqOCo0ObZjTYaUuaClLmK/wTd+7ErR+gLnWlf+FNpuIL0ZS2J+eec5Kb+EkoUuU4jx1WZ1d3T29f/8Dg0PDIaGlsfC+NM8l4lcVhLA98L+WhiHhVCRXyg0Ryr+mHfN8/29D1/XMuUxFHu+oi4UdN7yQSDcE8RVS9tCXqedSaceZqPElFGEezq23G/WDmayJSdefYDZRda0iP5UErD1SrLVQfwnqp7FQcM+yfwC1AGcXYjkv3qCFADIYMTXBEUIRDeEjpOYQLBwlxR8iJk4SEqXO0MEDejFScFB6xZ/Q9odlhwUY015mpcTNaJaRXktPGNHli0knCejXb1DOTrNnfsnOTqfd2QX+/yGoSq3BK7F++d+V/fboXhQZWTA+CekoMo7tjRUpmTkXv3P7UlaKEhDiNA6pLwsw438/ZNp7U9K7P1jP1F6PUrJ6zQpvhVe+SLtj9fp0/wd5CxV2sLOwsldfWi6vuwySmMEP3uYw1bGIbVcq+wgOe8GxdWtfWjXXbllodhWcCX4Z19wbyiq5L</latexit> 1 n I(X; W) = ipot n Rn(t, ✏); n, ⇢n + Remainder ⇣ Rn(t, ✏), 1 n EhX · xit,✏ ⌘ <latexit sha1_base64="Qof9htjpZRZsAU57rA1eF30B1BM=">AAADk3icjVFdb9MwFL1Z+BjjYx2IJ14sKlArqqoZk4ZUIU0FJHhAGhNdK+YROY7bWkvsyHEQU5S/yCviH8C/4NrLxMdA4CjJuefec3yvnRSZLO1o9CVYCy9dvnJ1/drG9Rs3b212tm4flroyXEy5zrSZJ6wUmVRiaqXNxLwwguVJJmbJyTOXn30QppRavbWnhTjO2VLJheTMIhV3PtGFYTxS5FWvpsmCzJux/8+aPnn4lBAiY/W+piYnhbYNTeSydxDXqunZARVFKTOt+mOa4YYpi9WAmpWOlSvrk0dediByJlUqTEMnfxIPfAN11CBPc2ZXSVK/aNBRLTNB2p4I5am2Z9FHjIzPxvUPHyTRvh93uqPhyC9yEUQt6EK79nXnM1BIQQOHCnIQoMAizoBBic8RRDCCArljqJEziKTPC2hgA7UVVgmsYMie4HeJ0VHLKoydZ+nVHHfJ8DWoJPAANRrrDGK3G/H5yjs79m/etfd0vZ3iP2m9cmQtrJD9l+688n91bhYLC3jiZ5A4U+EZNx1vXSp/Kq5z8tNUFh0K5BxOMW8Qc688P2fiNaWf3Z0t8/mvvtKxLuZtbQXfXJd4wdHv13kRHG4Po8fD7Tc73b1Je9XrcA/uQw/vcxf24CXswxR4sBO8C3iQhnfDcTgJn5+VrgWt5g78ssLX3wFQhtic</latexit> Sum rule: Rn(t, ✏) = ✏ + nt ⇥ argminq i(pot) (q; n, ⇢n) ) 1 n I(X; W) . inf q 0 ipot n (q; n, ⇢n) <latexit sha1_base64="Husz+qtKzIoHMOlptlG6LYPDcLU=">AAADmnicjVFdb9MwFL1Z+Bjlq4NHeLComFpRVemYBFKFNI0XJl7GtK6V5i1yUjc1TZzUdkBTlJ+JEP8A/gXXJp2AgcBRkuNz7znX1zcqUqFNEHzxNvxr12/c3LzVun3n7r377a0HJzovVczHcZ7mahoxzVMh+dgIk/JpoTjLopRPouVrG5984EqLXB6bi4KfZSyRYi5iZpAK25+2j8JK1l3Tp7zQIs1l79UaPaMpGs1YKImhRmRcExqJpEsqqjLCVJIJWYcr2ifivOo6sshN3au7q9GltE/VIg9lzyp7lLa26ZFIFoYplX8kdK5YTIaSHKA8mpNpPXL/Sd0jNOVaa5GRFqFCzsNqRRNOgtqWC+X5Zbk/FQvbnWAQuEWugmEDOtCsw7z9GSjMIIcYSsiAgwSDOAUGGp9TGEIABXJnUCGnEAkX51BDC7UlZnHMYMgu8Zvg7rRhJe6tp3bqGKuk+CpUEniKmhzzFGJbjbh46Zwt+zfvynnas13gP2q8MmQNLJD9l26d+b8624uBObx0PQjsqXCM7S5uXEp3K/bk5KeuDDoUyFk8w7hCHDvl+p6J02jXu71b5uJfXaZl7T5uckv4Zk+JAx7+Ps6r4GRnMHw+2Hm329nbb0a9CY/gCXRxni9gD97AIYwh9kYe8957S/+xv+8f+G9/pG54jeYh/LL84+8rbNpA</latexit> Upper bound « trivial path » d dt Rn(t, ✏) = E D 1 n X · x E t,✏ (Rn(t, ✏)) ) 1 n I(X; W) & inf q 0 ipot n (q; n, ⇢n) <latexit sha1_base64="9L/XtJe5/TRni1/uA4AES/h/Qss=">AAADrHicjVFri9QwFL3d+ljH16gf/RIcXGZgGDqroLAIy4qgH4R12dkdmIwlbdNO2Dbppqm61P5O8Reo/8Kb2MHHIJrS9t5zzznJzY3KXFQmCD57W/6ly1eubl/rXb9x89bt/p27J5WqdcxnscqVnkes4rmQfGaEyfm81JwVUc5Po7Pntn76jutKKHlsLkq+LFgmRSpiZhAK+192aKpZ3CRtkxjSHoWNbIdmTHlZiVzJEXlGaMHMKoqaFy09EBnNmcxy7lRkKhsapWTeEhonyhCXfcDMErUjhs1Pt5YMNzcYUdrboUciWxmmtXpP1tbk1bBz33P/03ZEaGawl4JQIdOwOacZJ0FLx0SE8m1DdUFKZdrh+R6esogSFsox1SsVylHYHwSTwC2yGUy7YADdOlT9T0AhAQUx1FAABwkG4xwYVPgsYAoBlIgtoUFMYyRcnUMLPdTWyOLIYIie4TfDbNGhEnPrWTl1jLvk+GpUEniIGoU8jbHdjbh67Zwt+jfvxnnas13gP+q8CkQNrBD9l27N/F+d7cVACk9dDwJ7Kh1iu4s7l9rdij05+aUrgw4lYjZOsK4xjp1yfc/EaSrXu71b5upfHdOiNo87bg3f7ClxwNM/x7kZnOxOpo8mu28eD/YPulFvw314AEOc5xPYh5dwCDOIvdde5X30Wn/iH/sLf/mDuuV1mnvw2/LT7+ak4h4=</latexit> Lower bound « smart path »
  • 22. Thank You • Statistical physics valid in corners of phase diagrams. • Interpolation techniques allow to naturally catch the finite size corrections. • « All-or-nothing » seems quite generic: « Universal » phenomenon in reconstruction of very sparse information (i.e., with « effective low dimension ») from much higher- dimensional data? • What about more complex models (multi layers etc)? • « All-or-nothing » for algorithms (AMP, gradient-based: on- going)?