This document discusses inverse problems in imaging such as inpainting, deblurring, superresolution, compressed sensing, MRI, and radar. It notes that inverse problems involve observing data y that is related to the desired signal β through a model y = Xβ + ε, and the goal is to recover β from the observed data y. One classical approach is Tikhonov regularization from 1943, which finds the least squares solution to deblurring problems. The document provides examples of how deblurring an image the "naive" way can lead to wildly different solutions compared to using Tikhonov regularization.
1. Learning to Solve Inverse
Problems in Imaging
Rebecca Willett, University of Chicago
Greg Ongie,
UChicago
Davis Gilton,
UW-Madison
2. β
Inverse problems in imaging
• Inpainting
• Deblurring
• Superresolution
• Compressed
Sensing
• MRI
• Radar
Observe: y = Xβ + ε
Goal: Recover β from y
y
3. Classical approach: Tikhonov regularization (1943)
• Example: deblurring
• Least squares solution:
Wildly Different Solutions
Blurred image with noiseBlurred image
Deblurred the “naïve” way Deblurred the “naïve” way
ˆβ = (X X) 1
X y<latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit><latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit><latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit><latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit>
Wildly different solutions
4. Classical approach: Tikhonov regularization (1943)
• Example: deblurring
• Least squares solution:
Wildly Different Solutions
Blurred image with noiseBlurred image
Deblurred the “naïve” way Deblurred the “naïve” way
• Tikhonov regularization
(aka “ridge regression”)
better conditioned; suppresses noise
ˆβ = arg min
β
y Xβ 2
2 + λ β 2
2
=(X X + λI) 1
X y<latexit sha1_base64="el+v4cMXAHNtxVmtmnE54ehzJBM=">AAAMaHiczVbdjhs1FE7KX0kX2MIFQty4TQtbmI0yES29iVRKRVuplPKz7Urx7sozcyYx6/FMPZ7dRO48Cc/Bw/AKPAXHdpLNJEvV9gIxq43sc75zfH7tExWCl7rf/6t94a2333n3vYvvdy5tffDhR9uXP35a5pWKYS/ORa72I1aC4BL2NNcC9gsFLIsEPIuOv7f8ZyegSp7L3/SsgIOMjSVPecw0ko62/6CPHzzi44kmtMyYEKRDIxhzaZjgY/lV3aEThrwINCPDLwhlapxxeeQJ9MVsd98t6YujweGAfE2owLMTy1qhU9pB2Z39Q6rzguyvwB7eODS7YT3nzDoUZLI8+mi72+/13Uc2F+F80W3NvydHlzt/0iSPqwykjgUry1EYFvrAlIInUKIvVQkFi4/ZGEa4lCyD8sC4INbkOlISkuYK/6UmjroqYVhWlrMsQmTG9KRc51niebxRpdPbB4bLotIgY39QWgmic2IzQhKuINZihgsWK655TOIJUyzWmLfmKSdManSjBMyzHOsJ1TDVpzzRE3Ord5PLhosmynCfQEp1aiiiQHORgEnrOTnSE0yRoVFmqFsuGFhAojTUuhJF5pcF2efe0/PC2D2hSFnqmzpd08V+WpjpIS0Uz6DuXHekBJ4bWmo0UIHAFfpb6PmvngmgWnEmxwJqM1yocTIL+nMkSjiN8yxjWCr0uxOoR+GBoQJSjVVlMaQbEqpsUVPllTVl2FJmBT4HdhoRnFaSx3nizF8lCz3VijWj7RJfQNyk2lry1Os2bRnj0pLMfY699iuT5ZKB0pazc4+PuS6DR9igMrivAI5vvCb6AYgTzHXMHkMFu667fdFgfy8MKF9PciE26hBCHqIiHv9gu8R+Q9IU8+zAIu/mIlniNpGWvcStaD0P59kHTRN9Yq1hKcu4mC0us1cx0wFf3dY9gRl3Mq9i8Rn6XLvfNLT/B5tXe8ndPyoTqW0ns4h+N6zrDRQ+TU0UxeqbsAI8WoB2sCgV6VzrhopqXUOUlqA4lOceWG2euMA3jl5r+UpA9vKXwt1K+Ny6Hbb9GF8bVhNi1DiqTe920A96t9dQhaiyM8gtC7m1BhljC8olxgKCfjPaeLuibaMB3lzWQS+GPhjTHdT1BjbKpw7coTqOsNBMqtAN4u7ZobHvrdMw7IYBiXKt82xJuPLNzSuRQP/rwL2GbrAYLuMXIM6TPKhDY4wBKDuEGHtaPh2BnDAZQxJwy+H4rE54koAMkImxHe6GWYaOeCdOJ1zjle/dqJsJ8X5aZ5wjlZ11IsXUzJTH3N6ITbdP/zO3A+80qYoCbFX4uenM3DcOgRuCVtyeuwjyhKtc2uHGlM8rXk7sfFh3DLb2/HBHMNdoVAlsqGu17XpiyNnAYLD0IcPDscIKXftrY5VdMOW4A+R2NtkYvjXhdelVxDkK7DOd4eCAU4MJezexnptsFoFwAw2ybSOeo8FC/BFeHv/mMOMi54JgY3YPcAhU8CM+lj9hipjGMFMbIBqUVfQ7Dl00wCmMBvW/Yj24p3svAeGwWhv3u96B+Ez7QcNnx+4Vn9bd0Nm52LpBN1wfazcXTwe9sN8Lfx5079ydj7wXW5+3rrZ2WmHr29ad1oPWk9ZeK26321+2++3w0t9b21ufbn3moRfac5lPWo1v6+o/Ctholg==</latexit><latexit sha1_base64="el+v4cMXAHNtxVmtmnE54ehzJBM=">AAAMaHiczVbdjhs1FE7KX0kX2MIFQty4TQtbmI0yES29iVRKRVuplPKz7Urx7sozcyYx6/FMPZ7dRO48Cc/Bw/AKPAXHdpLNJEvV9gIxq43sc75zfH7tExWCl7rf/6t94a2333n3vYvvdy5tffDhR9uXP35a5pWKYS/ORa72I1aC4BL2NNcC9gsFLIsEPIuOv7f8ZyegSp7L3/SsgIOMjSVPecw0ko62/6CPHzzi44kmtMyYEKRDIxhzaZjgY/lV3aEThrwINCPDLwhlapxxeeQJ9MVsd98t6YujweGAfE2owLMTy1qhU9pB2Z39Q6rzguyvwB7eODS7YT3nzDoUZLI8+mi72+/13Uc2F+F80W3NvydHlzt/0iSPqwykjgUry1EYFvrAlIInUKIvVQkFi4/ZGEa4lCyD8sC4INbkOlISkuYK/6UmjroqYVhWlrMsQmTG9KRc51niebxRpdPbB4bLotIgY39QWgmic2IzQhKuINZihgsWK655TOIJUyzWmLfmKSdManSjBMyzHOsJ1TDVpzzRE3Ord5PLhosmynCfQEp1aiiiQHORgEnrOTnSE0yRoVFmqFsuGFhAojTUuhJF5pcF2efe0/PC2D2hSFnqmzpd08V+WpjpIS0Uz6DuXHekBJ4bWmo0UIHAFfpb6PmvngmgWnEmxwJqM1yocTIL+nMkSjiN8yxjWCr0uxOoR+GBoQJSjVVlMaQbEqpsUVPllTVl2FJmBT4HdhoRnFaSx3nizF8lCz3VijWj7RJfQNyk2lry1Os2bRnj0pLMfY699iuT5ZKB0pazc4+PuS6DR9igMrivAI5vvCb6AYgTzHXMHkMFu667fdFgfy8MKF9PciE26hBCHqIiHv9gu8R+Q9IU8+zAIu/mIlniNpGWvcStaD0P59kHTRN9Yq1hKcu4mC0us1cx0wFf3dY9gRl3Mq9i8Rn6XLvfNLT/B5tXe8ndPyoTqW0ns4h+N6zrDRQ+TU0UxeqbsAI8WoB2sCgV6VzrhopqXUOUlqA4lOceWG2euMA3jl5r+UpA9vKXwt1K+Ny6Hbb9GF8bVhNi1DiqTe920A96t9dQhaiyM8gtC7m1BhljC8olxgKCfjPaeLuibaMB3lzWQS+GPhjTHdT1BjbKpw7coTqOsNBMqtAN4u7ZobHvrdMw7IYBiXKt82xJuPLNzSuRQP/rwL2GbrAYLuMXIM6TPKhDY4wBKDuEGHtaPh2BnDAZQxJwy+H4rE54koAMkImxHe6GWYaOeCdOJ1zjle/dqJsJ8X5aZ5wjlZ11IsXUzJTH3N6ITbdP/zO3A+80qYoCbFX4uenM3DcOgRuCVtyeuwjyhKtc2uHGlM8rXk7sfFh3DLb2/HBHMNdoVAlsqGu17XpiyNnAYLD0IcPDscIKXftrY5VdMOW4A+R2NtkYvjXhdelVxDkK7DOd4eCAU4MJezexnptsFoFwAw2ybSOeo8FC/BFeHv/mMOMi54JgY3YPcAhU8CM+lj9hipjGMFMbIBqUVfQ7Dl00wCmMBvW/Yj24p3svAeGwWhv3u96B+Ez7QcNnx+4Vn9bd0Nm52LpBN1wfazcXTwe9sN8Lfx5079ydj7wXW5+3rrZ2WmHr29ad1oPWk9ZeK26321+2++3w0t9b21ufbn3moRfac5lPWo1v6+o/Ctholg==</latexit><latexit sha1_base64="el+v4cMXAHNtxVmtmnE54ehzJBM=">AAAMaHiczVbdjhs1FE7KX0kX2MIFQty4TQtbmI0yES29iVRKRVuplPKz7Urx7sozcyYx6/FMPZ7dRO48Cc/Bw/AKPAXHdpLNJEvV9gIxq43sc75zfH7tExWCl7rf/6t94a2333n3vYvvdy5tffDhR9uXP35a5pWKYS/ORa72I1aC4BL2NNcC9gsFLIsEPIuOv7f8ZyegSp7L3/SsgIOMjSVPecw0ko62/6CPHzzi44kmtMyYEKRDIxhzaZjgY/lV3aEThrwINCPDLwhlapxxeeQJ9MVsd98t6YujweGAfE2owLMTy1qhU9pB2Z39Q6rzguyvwB7eODS7YT3nzDoUZLI8+mi72+/13Uc2F+F80W3NvydHlzt/0iSPqwykjgUry1EYFvrAlIInUKIvVQkFi4/ZGEa4lCyD8sC4INbkOlISkuYK/6UmjroqYVhWlrMsQmTG9KRc51niebxRpdPbB4bLotIgY39QWgmic2IzQhKuINZihgsWK655TOIJUyzWmLfmKSdManSjBMyzHOsJ1TDVpzzRE3Ord5PLhosmynCfQEp1aiiiQHORgEnrOTnSE0yRoVFmqFsuGFhAojTUuhJF5pcF2efe0/PC2D2hSFnqmzpd08V+WpjpIS0Uz6DuXHekBJ4bWmo0UIHAFfpb6PmvngmgWnEmxwJqM1yocTIL+nMkSjiN8yxjWCr0uxOoR+GBoQJSjVVlMaQbEqpsUVPllTVl2FJmBT4HdhoRnFaSx3nizF8lCz3VijWj7RJfQNyk2lry1Os2bRnj0pLMfY699iuT5ZKB0pazc4+PuS6DR9igMrivAI5vvCb6AYgTzHXMHkMFu667fdFgfy8MKF9PciE26hBCHqIiHv9gu8R+Q9IU8+zAIu/mIlniNpGWvcStaD0P59kHTRN9Yq1hKcu4mC0us1cx0wFf3dY9gRl3Mq9i8Rn6XLvfNLT/B5tXe8ndPyoTqW0ns4h+N6zrDRQ+TU0UxeqbsAI8WoB2sCgV6VzrhopqXUOUlqA4lOceWG2euMA3jl5r+UpA9vKXwt1K+Ny6Hbb9GF8bVhNi1DiqTe920A96t9dQhaiyM8gtC7m1BhljC8olxgKCfjPaeLuibaMB3lzWQS+GPhjTHdT1BjbKpw7coTqOsNBMqtAN4u7ZobHvrdMw7IYBiXKt82xJuPLNzSuRQP/rwL2GbrAYLuMXIM6TPKhDY4wBKDuEGHtaPh2BnDAZQxJwy+H4rE54koAMkImxHe6GWYaOeCdOJ1zjle/dqJsJ8X5aZ5wjlZ11IsXUzJTH3N6ITbdP/zO3A+80qYoCbFX4uenM3DcOgRuCVtyeuwjyhKtc2uHGlM8rXk7sfFh3DLb2/HBHMNdoVAlsqGu17XpiyNnAYLD0IcPDscIKXftrY5VdMOW4A+R2NtkYvjXhdelVxDkK7DOd4eCAU4MJezexnptsFoFwAw2ybSOeo8FC/BFeHv/mMOMi54JgY3YPcAhU8CM+lj9hipjGMFMbIBqUVfQ7Dl00wCmMBvW/Yj24p3svAeGwWhv3u96B+Ez7QcNnx+4Vn9bd0Nm52LpBN1wfazcXTwe9sN8Lfx5079ydj7wXW5+3rrZ2WmHr29ad1oPWk9ZeK26321+2++3w0t9b21ufbn3moRfac5lPWo1v6+o/Ctholg==</latexit><latexit sha1_base64="el+v4cMXAHNtxVmtmnE54ehzJBM=">AAAMaHiczVbdjhs1FE7KX0kX2MIFQty4TQtbmI0yES29iVRKRVuplPKz7Urx7sozcyYx6/FMPZ7dRO48Cc/Bw/AKPAXHdpLNJEvV9gIxq43sc75zfH7tExWCl7rf/6t94a2333n3vYvvdy5tffDhR9uXP35a5pWKYS/ORa72I1aC4BL2NNcC9gsFLIsEPIuOv7f8ZyegSp7L3/SsgIOMjSVPecw0ko62/6CPHzzi44kmtMyYEKRDIxhzaZjgY/lV3aEThrwINCPDLwhlapxxeeQJ9MVsd98t6YujweGAfE2owLMTy1qhU9pB2Z39Q6rzguyvwB7eODS7YT3nzDoUZLI8+mi72+/13Uc2F+F80W3NvydHlzt/0iSPqwykjgUry1EYFvrAlIInUKIvVQkFi4/ZGEa4lCyD8sC4INbkOlISkuYK/6UmjroqYVhWlrMsQmTG9KRc51niebxRpdPbB4bLotIgY39QWgmic2IzQhKuINZihgsWK655TOIJUyzWmLfmKSdManSjBMyzHOsJ1TDVpzzRE3Ord5PLhosmynCfQEp1aiiiQHORgEnrOTnSE0yRoVFmqFsuGFhAojTUuhJF5pcF2efe0/PC2D2hSFnqmzpd08V+WpjpIS0Uz6DuXHekBJ4bWmo0UIHAFfpb6PmvngmgWnEmxwJqM1yocTIL+nMkSjiN8yxjWCr0uxOoR+GBoQJSjVVlMaQbEqpsUVPllTVl2FJmBT4HdhoRnFaSx3nizF8lCz3VijWj7RJfQNyk2lry1Os2bRnj0pLMfY699iuT5ZKB0pazc4+PuS6DR9igMrivAI5vvCb6AYgTzHXMHkMFu667fdFgfy8MKF9PciE26hBCHqIiHv9gu8R+Q9IU8+zAIu/mIlniNpGWvcStaD0P59kHTRN9Yq1hKcu4mC0us1cx0wFf3dY9gRl3Mq9i8Rn6XLvfNLT/B5tXe8ndPyoTqW0ns4h+N6zrDRQ+TU0UxeqbsAI8WoB2sCgV6VzrhopqXUOUlqA4lOceWG2euMA3jl5r+UpA9vKXwt1K+Ny6Hbb9GF8bVhNi1DiqTe920A96t9dQhaiyM8gtC7m1BhljC8olxgKCfjPaeLuibaMB3lzWQS+GPhjTHdT1BjbKpw7coTqOsNBMqtAN4u7ZobHvrdMw7IYBiXKt82xJuPLNzSuRQP/rwL2GbrAYLuMXIM6TPKhDY4wBKDuEGHtaPh2BnDAZQxJwy+H4rE54koAMkImxHe6GWYaOeCdOJ1zjle/dqJsJ8X5aZ5wjlZ11IsXUzJTH3N6ITbdP/zO3A+80qYoCbFX4uenM3DcOgRuCVtyeuwjyhKtc2uHGlM8rXk7sfFh3DLb2/HBHMNdoVAlsqGu17XpiyNnAYLD0IcPDscIKXftrY5VdMOW4A+R2NtkYvjXhdelVxDkK7DOd4eCAU4MJezexnptsFoFwAw2ybSOeo8FC/BFeHv/mMOMi54JgY3YPcAhU8CM+lj9hipjGMFMbIBqUVfQ7Dl00wCmMBvW/Yj24p3svAeGwWhv3u96B+Ez7QcNnx+4Vn9bd0Nm52LpBN1wfazcXTwe9sN8Lfx5079ydj7wXW5+3rrZ2WmHr29ad1oPWk9ZeK26321+2++3w0t9b21ufbn3moRfac5lPWo1v6+o/Ctholg==</latexit>
ˆβ = (X X) 1
X y<latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit><latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit><latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit><latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit>
Wildly different solutions
5. Classical approach: Tikhonov regularization (1943)
• Example: deblurring
• Least squares solution:
Wildly Different Solutions
Blurred image with noiseBlurred image
Deblurred the “naïve” way Deblurred the “naïve” way
• Tikhonov regularization
(aka “ridge regression”)
better conditioned; suppresses noise
ˆβ = arg min
β
y Xβ 2
2 + λ β 2
2
=(X X + λI) 1
X y<latexit sha1_base64="el+v4cMXAHNtxVmtmnE54ehzJBM=">AAAMaHiczVbdjhs1FE7KX0kX2MIFQty4TQtbmI0yES29iVRKRVuplPKz7Urx7sozcyYx6/FMPZ7dRO48Cc/Bw/AKPAXHdpLNJEvV9gIxq43sc75zfH7tExWCl7rf/6t94a2333n3vYvvdy5tffDhR9uXP35a5pWKYS/ORa72I1aC4BL2NNcC9gsFLIsEPIuOv7f8ZyegSp7L3/SsgIOMjSVPecw0ko62/6CPHzzi44kmtMyYEKRDIxhzaZjgY/lV3aEThrwINCPDLwhlapxxeeQJ9MVsd98t6YujweGAfE2owLMTy1qhU9pB2Z39Q6rzguyvwB7eODS7YT3nzDoUZLI8+mi72+/13Uc2F+F80W3NvydHlzt/0iSPqwykjgUry1EYFvrAlIInUKIvVQkFi4/ZGEa4lCyD8sC4INbkOlISkuYK/6UmjroqYVhWlrMsQmTG9KRc51niebxRpdPbB4bLotIgY39QWgmic2IzQhKuINZihgsWK655TOIJUyzWmLfmKSdManSjBMyzHOsJ1TDVpzzRE3Ord5PLhosmynCfQEp1aiiiQHORgEnrOTnSE0yRoVFmqFsuGFhAojTUuhJF5pcF2efe0/PC2D2hSFnqmzpd08V+WpjpIS0Uz6DuXHekBJ4bWmo0UIHAFfpb6PmvngmgWnEmxwJqM1yocTIL+nMkSjiN8yxjWCr0uxOoR+GBoQJSjVVlMaQbEqpsUVPllTVl2FJmBT4HdhoRnFaSx3nizF8lCz3VijWj7RJfQNyk2lry1Os2bRnj0pLMfY699iuT5ZKB0pazc4+PuS6DR9igMrivAI5vvCb6AYgTzHXMHkMFu667fdFgfy8MKF9PciE26hBCHqIiHv9gu8R+Q9IU8+zAIu/mIlniNpGWvcStaD0P59kHTRN9Yq1hKcu4mC0us1cx0wFf3dY9gRl3Mq9i8Rn6XLvfNLT/B5tXe8ndPyoTqW0ns4h+N6zrDRQ+TU0UxeqbsAI8WoB2sCgV6VzrhopqXUOUlqA4lOceWG2euMA3jl5r+UpA9vKXwt1K+Ny6Hbb9GF8bVhNi1DiqTe920A96t9dQhaiyM8gtC7m1BhljC8olxgKCfjPaeLuibaMB3lzWQS+GPhjTHdT1BjbKpw7coTqOsNBMqtAN4u7ZobHvrdMw7IYBiXKt82xJuPLNzSuRQP/rwL2GbrAYLuMXIM6TPKhDY4wBKDuEGHtaPh2BnDAZQxJwy+H4rE54koAMkImxHe6GWYaOeCdOJ1zjle/dqJsJ8X5aZ5wjlZ11IsXUzJTH3N6ITbdP/zO3A+80qYoCbFX4uenM3DcOgRuCVtyeuwjyhKtc2uHGlM8rXk7sfFh3DLb2/HBHMNdoVAlsqGu17XpiyNnAYLD0IcPDscIKXftrY5VdMOW4A+R2NtkYvjXhdelVxDkK7DOd4eCAU4MJezexnptsFoFwAw2ybSOeo8FC/BFeHv/mMOMi54JgY3YPcAhU8CM+lj9hipjGMFMbIBqUVfQ7Dl00wCmMBvW/Yj24p3svAeGwWhv3u96B+Ez7QcNnx+4Vn9bd0Nm52LpBN1wfazcXTwe9sN8Lfx5079ydj7wXW5+3rrZ2WmHr29ad1oPWk9ZeK26321+2++3w0t9b21ufbn3moRfac5lPWo1v6+o/Ctholg==</latexit><latexit sha1_base64="el+v4cMXAHNtxVmtmnE54ehzJBM=">AAAMaHiczVbdjhs1FE7KX0kX2MIFQty4TQtbmI0yES29iVRKRVuplPKz7Urx7sozcyYx6/FMPZ7dRO48Cc/Bw/AKPAXHdpLNJEvV9gIxq43sc75zfH7tExWCl7rf/6t94a2333n3vYvvdy5tffDhR9uXP35a5pWKYS/ORa72I1aC4BL2NNcC9gsFLIsEPIuOv7f8ZyegSp7L3/SsgIOMjSVPecw0ko62/6CPHzzi44kmtMyYEKRDIxhzaZjgY/lV3aEThrwINCPDLwhlapxxeeQJ9MVsd98t6YujweGAfE2owLMTy1qhU9pB2Z39Q6rzguyvwB7eODS7YT3nzDoUZLI8+mi72+/13Uc2F+F80W3NvydHlzt/0iSPqwykjgUry1EYFvrAlIInUKIvVQkFi4/ZGEa4lCyD8sC4INbkOlISkuYK/6UmjroqYVhWlrMsQmTG9KRc51niebxRpdPbB4bLotIgY39QWgmic2IzQhKuINZihgsWK655TOIJUyzWmLfmKSdManSjBMyzHOsJ1TDVpzzRE3Ord5PLhosmynCfQEp1aiiiQHORgEnrOTnSE0yRoVFmqFsuGFhAojTUuhJF5pcF2efe0/PC2D2hSFnqmzpd08V+WpjpIS0Uz6DuXHekBJ4bWmo0UIHAFfpb6PmvngmgWnEmxwJqM1yocTIL+nMkSjiN8yxjWCr0uxOoR+GBoQJSjVVlMaQbEqpsUVPllTVl2FJmBT4HdhoRnFaSx3nizF8lCz3VijWj7RJfQNyk2lry1Os2bRnj0pLMfY699iuT5ZKB0pazc4+PuS6DR9igMrivAI5vvCb6AYgTzHXMHkMFu667fdFgfy8MKF9PciE26hBCHqIiHv9gu8R+Q9IU8+zAIu/mIlniNpGWvcStaD0P59kHTRN9Yq1hKcu4mC0us1cx0wFf3dY9gRl3Mq9i8Rn6XLvfNLT/B5tXe8ndPyoTqW0ns4h+N6zrDRQ+TU0UxeqbsAI8WoB2sCgV6VzrhopqXUOUlqA4lOceWG2euMA3jl5r+UpA9vKXwt1K+Ny6Hbb9GF8bVhNi1DiqTe920A96t9dQhaiyM8gtC7m1BhljC8olxgKCfjPaeLuibaMB3lzWQS+GPhjTHdT1BjbKpw7coTqOsNBMqtAN4u7ZobHvrdMw7IYBiXKt82xJuPLNzSuRQP/rwL2GbrAYLuMXIM6TPKhDY4wBKDuEGHtaPh2BnDAZQxJwy+H4rE54koAMkImxHe6GWYaOeCdOJ1zjle/dqJsJ8X5aZ5wjlZ11IsXUzJTH3N6ITbdP/zO3A+80qYoCbFX4uenM3DcOgRuCVtyeuwjyhKtc2uHGlM8rXk7sfFh3DLb2/HBHMNdoVAlsqGu17XpiyNnAYLD0IcPDscIKXftrY5VdMOW4A+R2NtkYvjXhdelVxDkK7DOd4eCAU4MJezexnptsFoFwAw2ybSOeo8FC/BFeHv/mMOMi54JgY3YPcAhU8CM+lj9hipjGMFMbIBqUVfQ7Dl00wCmMBvW/Yj24p3svAeGwWhv3u96B+Ez7QcNnx+4Vn9bd0Nm52LpBN1wfazcXTwe9sN8Lfx5079ydj7wXW5+3rrZ2WmHr29ad1oPWk9ZeK26321+2++3w0t9b21ufbn3moRfac5lPWo1v6+o/Ctholg==</latexit><latexit sha1_base64="el+v4cMXAHNtxVmtmnE54ehzJBM=">AAAMaHiczVbdjhs1FE7KX0kX2MIFQty4TQtbmI0yES29iVRKRVuplPKz7Urx7sozcyYx6/FMPZ7dRO48Cc/Bw/AKPAXHdpLNJEvV9gIxq43sc75zfH7tExWCl7rf/6t94a2333n3vYvvdy5tffDhR9uXP35a5pWKYS/ORa72I1aC4BL2NNcC9gsFLIsEPIuOv7f8ZyegSp7L3/SsgIOMjSVPecw0ko62/6CPHzzi44kmtMyYEKRDIxhzaZjgY/lV3aEThrwINCPDLwhlapxxeeQJ9MVsd98t6YujweGAfE2owLMTy1qhU9pB2Z39Q6rzguyvwB7eODS7YT3nzDoUZLI8+mi72+/13Uc2F+F80W3NvydHlzt/0iSPqwykjgUry1EYFvrAlIInUKIvVQkFi4/ZGEa4lCyD8sC4INbkOlISkuYK/6UmjroqYVhWlrMsQmTG9KRc51niebxRpdPbB4bLotIgY39QWgmic2IzQhKuINZihgsWK655TOIJUyzWmLfmKSdManSjBMyzHOsJ1TDVpzzRE3Ord5PLhosmynCfQEp1aiiiQHORgEnrOTnSE0yRoVFmqFsuGFhAojTUuhJF5pcF2efe0/PC2D2hSFnqmzpd08V+WpjpIS0Uz6DuXHekBJ4bWmo0UIHAFfpb6PmvngmgWnEmxwJqM1yocTIL+nMkSjiN8yxjWCr0uxOoR+GBoQJSjVVlMaQbEqpsUVPllTVl2FJmBT4HdhoRnFaSx3nizF8lCz3VijWj7RJfQNyk2lry1Os2bRnj0pLMfY699iuT5ZKB0pazc4+PuS6DR9igMrivAI5vvCb6AYgTzHXMHkMFu667fdFgfy8MKF9PciE26hBCHqIiHv9gu8R+Q9IU8+zAIu/mIlniNpGWvcStaD0P59kHTRN9Yq1hKcu4mC0us1cx0wFf3dY9gRl3Mq9i8Rn6XLvfNLT/B5tXe8ndPyoTqW0ns4h+N6zrDRQ+TU0UxeqbsAI8WoB2sCgV6VzrhopqXUOUlqA4lOceWG2euMA3jl5r+UpA9vKXwt1K+Ny6Hbb9GF8bVhNi1DiqTe920A96t9dQhaiyM8gtC7m1BhljC8olxgKCfjPaeLuibaMB3lzWQS+GPhjTHdT1BjbKpw7coTqOsNBMqtAN4u7ZobHvrdMw7IYBiXKt82xJuPLNzSuRQP/rwL2GbrAYLuMXIM6TPKhDY4wBKDuEGHtaPh2BnDAZQxJwy+H4rE54koAMkImxHe6GWYaOeCdOJ1zjle/dqJsJ8X5aZ5wjlZ11IsXUzJTH3N6ITbdP/zO3A+80qYoCbFX4uenM3DcOgRuCVtyeuwjyhKtc2uHGlM8rXk7sfFh3DLb2/HBHMNdoVAlsqGu17XpiyNnAYLD0IcPDscIKXftrY5VdMOW4A+R2NtkYvjXhdelVxDkK7DOd4eCAU4MJezexnptsFoFwAw2ybSOeo8FC/BFeHv/mMOMi54JgY3YPcAhU8CM+lj9hipjGMFMbIBqUVfQ7Dl00wCmMBvW/Yj24p3svAeGwWhv3u96B+Ez7QcNnx+4Vn9bd0Nm52LpBN1wfazcXTwe9sN8Lfx5079ydj7wXW5+3rrZ2WmHr29ad1oPWk9ZeK26321+2++3w0t9b21ufbn3moRfac5lPWo1v6+o/Ctholg==</latexit><latexit sha1_base64="el+v4cMXAHNtxVmtmnE54ehzJBM=">AAAMaHiczVbdjhs1FE7KX0kX2MIFQty4TQtbmI0yES29iVRKRVuplPKz7Urx7sozcyYx6/FMPZ7dRO48Cc/Bw/AKPAXHdpLNJEvV9gIxq43sc75zfH7tExWCl7rf/6t94a2333n3vYvvdy5tffDhR9uXP35a5pWKYS/ORa72I1aC4BL2NNcC9gsFLIsEPIuOv7f8ZyegSp7L3/SsgIOMjSVPecw0ko62/6CPHzzi44kmtMyYEKRDIxhzaZjgY/lV3aEThrwINCPDLwhlapxxeeQJ9MVsd98t6YujweGAfE2owLMTy1qhU9pB2Z39Q6rzguyvwB7eODS7YT3nzDoUZLI8+mi72+/13Uc2F+F80W3NvydHlzt/0iSPqwykjgUry1EYFvrAlIInUKIvVQkFi4/ZGEa4lCyD8sC4INbkOlISkuYK/6UmjroqYVhWlrMsQmTG9KRc51niebxRpdPbB4bLotIgY39QWgmic2IzQhKuINZihgsWK655TOIJUyzWmLfmKSdManSjBMyzHOsJ1TDVpzzRE3Ord5PLhosmynCfQEp1aiiiQHORgEnrOTnSE0yRoVFmqFsuGFhAojTUuhJF5pcF2efe0/PC2D2hSFnqmzpd08V+WpjpIS0Uz6DuXHekBJ4bWmo0UIHAFfpb6PmvngmgWnEmxwJqM1yocTIL+nMkSjiN8yxjWCr0uxOoR+GBoQJSjVVlMaQbEqpsUVPllTVl2FJmBT4HdhoRnFaSx3nizF8lCz3VijWj7RJfQNyk2lry1Os2bRnj0pLMfY699iuT5ZKB0pazc4+PuS6DR9igMrivAI5vvCb6AYgTzHXMHkMFu667fdFgfy8MKF9PciE26hBCHqIiHv9gu8R+Q9IU8+zAIu/mIlniNpGWvcStaD0P59kHTRN9Yq1hKcu4mC0us1cx0wFf3dY9gRl3Mq9i8Rn6XLvfNLT/B5tXe8ndPyoTqW0ns4h+N6zrDRQ+TU0UxeqbsAI8WoB2sCgV6VzrhopqXUOUlqA4lOceWG2euMA3jl5r+UpA9vKXwt1K+Ny6Hbb9GF8bVhNi1DiqTe920A96t9dQhaiyM8gtC7m1BhljC8olxgKCfjPaeLuibaMB3lzWQS+GPhjTHdT1BjbKpw7coTqOsNBMqtAN4u7ZobHvrdMw7IYBiXKt82xJuPLNzSuRQP/rwL2GbrAYLuMXIM6TPKhDY4wBKDuEGHtaPh2BnDAZQxJwy+H4rE54koAMkImxHe6GWYaOeCdOJ1zjle/dqJsJ8X5aZ5wjlZ11IsXUzJTH3N6ITbdP/zO3A+80qYoCbFX4uenM3DcOgRuCVtyeuwjyhKtc2uHGlM8rXk7sfFh3DLb2/HBHMNdoVAlsqGu17XpiyNnAYLD0IcPDscIKXftrY5VdMOW4A+R2NtkYvjXhdelVxDkK7DOd4eCAU4MJezexnptsFoFwAw2ybSOeo8FC/BFeHv/mMOMi54JgY3YPcAhU8CM+lj9hipjGMFMbIBqUVfQ7Dl00wCmMBvW/Yj24p3svAeGwWhv3u96B+Ez7QcNnx+4Vn9bd0Nm52LpBN1wfazcXTwe9sN8Lfx5079ydj7wXW5+3rrZ2WmHr29ad1oPWk9ZeK26321+2++3w0t9b21ufbn3moRfac5lPWo1v6+o/Ctholg==</latexit>
ˆβ = (X X) 1
X y<latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit><latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit><latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit><latexit sha1_base64="GkA464yXxIwjpje3nR8S3U0AT5c=">AAAMB3iczVZLbxs3EN6kr1Tuw2mPvTCVAzjAWtAadRoUEJCmQZMCaZo+nBgQbYO7Oyux5nI3JNeWQOwP6L/oP+ipRa+99Npe+2865EqyVnKDJIcia1ggZ74ZzoPDmbgUXJt+/59Ll197/Y0337rydmfjnXffe3/z6gePdVGpBPaTQhTqIGYaBJewb7gRcFAqYHks4El88oXjPzkFpXkhfzDTEg5zNpI84wkzSDre/Iw+vP+Aj8aGUJ0zIUhna4uOGW5jMIwMyPbBETVFSQ5uHNmdqJ7tpltbnePNbr/X9x9ZX0SzRTeYfY+Or3Z+pWmRVDlIkwim9TCKSnNoteAp6LpDKw0lS07YCIa4lCwHfWi9jzW5jpSUZIXCf2mIpy5LWJZrPc1jRObMjPUqzxEv4g0rk906tFyWlQGZNAdllSCmIC5gJOUKEiOmuGCJ4oYnJBkzxRKDYW2fcsqkQTc0YBrkyIypgYk546kZ25u9PS5bLto4x30KGTWZpYgCw0UKNqtn5NiMMQWWxrmlfjlnYH6FttS5Esf2uzmZqVHOZUMvSuv2hCJloW/idU3m+0lpJ0e0VDyHunPdk1J4aqk2aKACgSv0tzSzXzMVQI3iTI4E1HYwV+Nl5vSnSJRwlhR5zmRq6eenUA+jQ0sFZIYKjyHdiFDl7hxVjbK2DFvILMFnwE4rgpNK8qRIvfnLZGEmRrF2tH3iS0jaVHeXGup1l7accelI9h7HUvieSb1goLTjbN/lI250+ADrR4b3FMDJjRdE3wdxirlO2EOoYMcXX3NpsPzmBugXk5yLDTuEkK9QEU++dFXivgFpizXs0CHvFCJd4NaRjr3ALWm9CNewD9smNol1hmUs52I6f2uex0wPfH5b9wVm3Ms8j8Xn6AvtftnQvgo2L9eSf39ULjJXTnYe/W5U12so7BxtFMXbN2YlNGgBxsPiTGQzrWsqqlUNcaZBcdAXHlitnzjHt45eKflKQP7sTuFfJeyGfodlP8Juw2pCrBrFte3dCvth79YKqhRVfg656SA3VyAjLEG5wDhA2G9HG19XtG24iy+Xc7ARQx+s7e7W9Ro2LiYe3KEmifGi2UyhG8S/swPrmqzXMOhGIYkLY4p8Qbj2yd61WKD/dei7oe/7g0X8QsQ1pAbUoQnGAJSbEaw7rZgMQY6ZTCANueNwbKtjnqYgQ2RibAc7UZ6jI40TZ2Nu8Mlv3KjbCWn8dM54Ryo3isSKqanVJ9y9iG23z/43t8PGaVKVJbhbEcMIu+O5uS8dAgroyJLbMxdBnnJVSDfcWP204nrsxre6Y7G0Z4d7gt2icSWwoLZqV/XEkvOBweLVhxwPxxtWmrp5NpbZJVOeu4vczjobw7civCq9jLhAgWvTOQ4OODXYqLeH97nNZjEIP9Ag2xXiBRocpDmikce/Gcz6yPkguJjdBRwCFXyNzfIbTBEzGGbqAkRDXcU/4tBFQ5zCaFj/J7YB90zvGSA+Ql/872oFYptuBo0mO26v+KTuRt7O+bZ2g260OtauLx7v9qJ+L/p2t3v7zmzkvRJ8FHwcbAdR8GlwO7gfPAr2gyT4Ofgz+Cv4e+OnjV82ftv4vYFevjST+TBofRt//AuwLUrN</latexit>
Wildly different solutions
Tikhonov regularization
6. Geometric models of images
Total variation
Noisy
Patches
Denoised
Patches
Patch
Denoising
Combine to
estimate
denoised
pixel
Patch subspaces and manifolds
(Wavelet) sparsity
7. Regularization in inverse problems
y ˆβ<latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit><latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit><latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit><latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit>
ˆβ = arg min
β
y Xβ 2
2 + r(β)
<latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit><latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit><latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit><latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit>
8. Regularization in inverse problems
y ˆβ<latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit><latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit><latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit><latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit>
ˆβ = arg min
β
y Xβ 2
2 + r(β)
<latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit><latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit><latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit><latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit>
Classical: r(β) is a pre-defined
smoothness-promoting regularizer
(e.g. Tikhinov or ridge estimation)
Geometric: r(β) reflects image geometry
(e.g. sparsity, patch redundancy, total variation)
Learned: use training data to learn r(β)
9. Examples in recent literature
• Deep CNN’s for signal recovery
• Compressed sensing with GANs
• Unrolled algorithms for solving inverse problems
• Deep proximal gradient descent nets
• Deep ADMM nets
• Deep half-quadratic splitting
• Deep primal-dual nets
Chen, Yu, Pock, 2015
Sun, Li, Xu, 2016
Chang, Li, Poczos, Kumar, Sankaranarayanan, 2017
Adler and Öktem, 2018
Bora, Jalal, Price, Dimakis, 2017
Mousavi and Baraniuk, 2017
Dong, Loy, He, Tang, 2014
Mardani et al, 2018
Jin, McCann, Froustey, Unser, 2017
Zhang, Zuo, Gu, Zhang, 2017
Ye, Han, Cha, 2018
10. Classes of methods
Model Agnostic
(Ignore X)
Decoupled
(First learn, then reconstruct)
Unrolled Optimization
Neumann Networks
(this talk!)
11. X-1~
raw data
(low resolution image)
approximate
high-resolution
image
Train deep CNN
to remove artifacts
reconstruction
blurry/blocky
artifacts due to
re-scaling
bicubic
interpolation
Pictures from: http://webdav.tuebingen.mpg.de/pixel/enhancenet/
Super-resolution with CNNs
Model Agnostic
(Ignore X)
12. Classes of methods
Model Agnostic
(Ignore X)
Decoupled
(First learn, then reconstruct)
Unrolled Optimization
Neumann Networks
(this talk!)
13. GANs for inverse problems
y ˆβ<latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit><latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit><latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit><latexit sha1_base64="3qRIWc7pIXZR0YKMqzYlnbgFQBU=">AAAQE3iczVfdjttEFHbLXwnQbeGCC25cspWK5EZxuummFyuVpaJFKqUUtq2UWaqxPU6GHY+943E20ciPwRNwC0/AHeKWB+CeB+HM2E78E6q2UhGObNnnfOfM+Z+JlzCayuHwr3Pn33jzrbffufBu7733P7i4c+nyh4/TOBM+OfJjFounHk4Jo5wcSSoZeZoIgiOPkSfeyRea/2RBREpj/r1cJeQ4wjNOQ+pjCaRnlz5GD+7dp7O57O3uojmWNvKIxLu7zy71h4Ohuezui1u+9K3yevjs8sW/URD7WUS49BlO06nrJvJYpYwGJM17KEtJgv0TPCNTeOU4IumxMg7k9lWgBHYYC7i5tA21LqFwlKaryANkhOU8bfM0cRtvmslwcqwoTzJJuF8sFGbMlrGto2EHVBBfshW8YF9QSX3bn2OBfQkxa66ywFyCGymBGPOZnCNJlvKMBnKubg7GlDdcVF4E3wEJkQwVAhSRlAVEhXlJ9uQc4qyQFylkXisGJI+lCmlXPE89qshYzCLKC3qcKP1tI6Cs9S2NrmX1vUzU8geUCBqRvHfVkAJyqlAqwUBBGLyBv4ksn3LFCJKCYj5jJFcHlRojU9FPgcjJmR9HEeaBQp8vSD51jxViJJSIGYzdd20kdEEhUShryuC1TA1eAnuNCC4zTv04MObXyUwupcDNaJvEJ8RvUnUtFdSrOm0RplyT1F3KmP0d5umaAdKac+0OnVGZOvehObhzVxBy8tlLou8RtoBc+/gBych101lF0WDGKgPSl5OsxKY927a/AkXU/1J3ib4O7KZYwXY08jBmwRrXRWr2GlfTug1XsI+bJhaJ1YaFOKJsVQ2SFzHTAF/c1iMGGTcyL2LxBr3V7lcN7f/B5novmfkjIhbqdlJV9PtunndQsC00UQiqb44TUqAZkQbmhSwstXZUZG0NXpgSQUm6dcGsu2KFbyzdavmMkej5O4WZSrDVmS9o+xnsNji3bSVmXq4GE2foDCYtVMKyaAO5qSE3W5AZtCBfYzTAGTajDdMVbJuOYHJpBwsx8EGp/ijPO1gvXhpwD0nfg0JToQA3bDNnD5SMk2KPO+i7ju3FUsbRmnBlb3zFY+B/7pjd0GzqB+v4OYArSAWoh3yIARH6AKD0avFySvgcc58EDtUcCtvqnAYB4Q4wIbYH190oAkcKJ87mVMLIL9zImwkp/NTOGEcyfc7wBBYrlZ5QPRGbbp/9Z247hdN2liREV4VHZrA7bsx95RAgAo7U3C5dJHxBRcz14UalpxlN5/pslvcUtHa5uCGoXeRlDBpqN9ddbyt7c2BQUPokgsWhwhKZF2Ojzk6wMNwRcHtdNoSvJdyWriO2KNDbdAQHBzg1KHcwhnpusrFHmDnQAFs34hYNGlIsUcjDr4QpEzkTBB2zOwQOgYJ8DZvlN5AiLCHMSAcIOWnm/QiHLuTAKQw5+b9iC/BADp4DojPwxTzbHQjbdHHQKLKjvwVd5n3X2Fl9arH6IGCQIo/B5lZOgsEtxzYPN+8CBQlKnFui4NHCBVicNBQONco89rdANyqHg8lYo4zqUQs6h5HKzJZQX9/gx69rRsb6mFb5AVoGe3pGNiAb6wf7oKLNLqZsc8Y2APVAaRPaEfJjDzMofvXo7iE4PXTc0dAZt7VQHtBZnK9BNwC33/YGhkkdNroxcsy9146fxLxacKRXnMC911lyEYvVGjYeO+buwKD54nijbjjeEqQyzHVd7riD4nixXk+Hyh21MwpdSk3WDebWnnNjAsa3A7rCtYSMnddXPZ0dtutT5bm9boBtNQYHJV1lRXT2xk6ZuQ6qLLZNdgvsy1VsuyAnOfwvdtv/grsvj0cDF1r321H/9mH5D/mC9Yn1qXXNcq1967Z1z3poHVm+lVs/W79Yv+78tPPbzu87fxTQ8+dKmY+sxrXz5z9tHLea</latexit>
ˆβ = arg min
β
y Xβ 2
2 + r(β)
<latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit><latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit><latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit><latexit sha1_base64="/+r4aBrVfxqpWL/MVPFAYn0tVFs=">AAAQWXiczVfdr9M2FA/si3UblPG4F7NeJNBC1YSWWx4qMYYGkxhjbBeQ6nLlJG7rXecDx+ltZfLX7a+Y9rqXvW7/wI6d9CNJhwCJaa0S2ef8fOzz6RMv4SyVvd5vZ86+9/4HH3507uPWJ59+dv5C++LnT9I4Ez498mMei2ceSSlnET2STHL6LBGUhB6nT72TbzT/6YKKlMXRz3KV0ElIZhGbMp9IIB23JwobIePH9+5MlDvo2+WTK/zw/gM2m8vWwQGeE4k9KgkaIUzELGTRcTHHL1fXn5khfnnsPnfRV0hcNfNrBwf5cbvT6/bMDzUHTjnoWOXv0fHF83/gIPazkEbS5yRNx46TyIlKOQtomrdwltKE+CdkRscwjEhI04kyGuToClACNI0FPJFEhrq7QpEwTVehB8iQyHla52niPt44k9PhRLEoySSN/GKjacaRjJG2KQqYoL7kKxgQXzDJfOTPiSC+BMtXd1mQSIIaKQVPRTM5x5Iu5SkL5Fzd7A5YVFFReSHMAzrFcqowoKhkPKBqmpdkT87B0gp7ocJmuGZACPBUYa2K56nHa3LhuoIeJ0rPEQbKRt7SyFqu58tELZ/jRLCQ5q0rhhTQFwqnEg4oKIcR6JvI8i1XnGIpGIlmnOZqtBZj1qzpL4AY0VM/DkMSBQp/vaD52IE45HQqMTcY1HEQFjr4sCiEVdeQzZodeAlsVSy4zCLmx4E5/i6Zy6UUpGpt4/iE+lWqjqWCekW7LSQs0iR1j3GOfiJRumHAas25epfNmEztB5BikX1PUHpy7Q3R9ylfgK998pBm9LrJwiJoCOfrA6RvtnK9bNxCCH0Hgpj/rc4S/Ruh6rKCbWvknZgHG1wTqdkb3I7UfbiCPakesXCsPtiUhIyv1kXndY5pgK9/1iMOHjdrXufEW/Tec7+taf8PZ97NJVN/RMinOp02Jb/j5HkDBZdLFYUh+uYkoQWaU2lg3pRPS6kNEVldgjdNqWA03bth1txxja9sXUv5jNPw1TeFqUpwYZoZpP0MbhuSI6TEzMtVd2j37O6whkp4Fm4hNzXkZg0ygxSMNhgNsHtVa0N1hbONXahcWsFiGeigVMfN8wbWi5cG3MLS9yDQ1FSAGsjU2ZGScVLccaOOYyMvljION4TL/cFlj4P+uW1uQ9MajDb2swFXkApQC/tgAyp0G6H0bvFyTKM5iXwa2ExzGFyrcxYENLKBCbYdXXfCcN0+qNM5k1DyCzXyqkMKPbUyRpFMdyueIGKl0hOmK2JV7dP/TG27UBplSUJ1VHh0Brfj9rhvbQJMQZEdtUsVabRgIo50c6PSFxlL57rDy1sKUrvc3BDUAfYyDgl1kOusRwptGwYFoU9D2BwiLJF5UTZ22QkRhusCt9Vkg/lqi+urdxF7BOhrOoTGAboG5XQHEM9VNvEoNw0NsHUi7pGgIcUWxXr4lzBlLGeMoG12l0ITKOj3cFn+AC4iEsyMtYGwnWbeL9B0YRu6MGzn/4otwF3ZfQWIzUAX865nIFzTRaNReEfPBVvmHceccz3Vy3YLAQcXeRwut7ISdG/ZyLycvAkUNChxTomCVw0XEHFSEdjTKPM63APdiux1hwONMqLdGnQOJZWbK2F3f4MfvKsaGes2ba0HSOn2dY2sQLan7x6CiDq7qLLVGlsB7BpKH6FuIT/2CIfgV/DJA0r3bMft2YO6FBYFbBbnG9ANwB3WtYFisgtzb7i2efp1+0kSrTd09Y5DePqNLRexWG1gg4FtngYMki+Ot+J6gz1GKs28K8sZNFARWWz206Zy3LpHIUuZ8brB3OrbN4Zw+LpBV2THIQP73UVP44Zt6rTWHG0SYF+MQaOko6ywTn9gl55roMpg23q3wL5ZxNYDcqi/i536V3Bz8MTtOpC6P7qd23fKL+Rz1hfWl9ZVy7EOrdvWfeuRdWT51q/Wn9Zf1t8Xfm+faZ9rtwro2TPlmktW5de+9A+mU82X</latexit>
r(β) =
0, β on image manifold
, otherwise<latexit sha1_base64="TZlHvKtOKIKwvBb/Te4jyN8zrRk=">AAAMWHiczVbdjtQ2FM7SHyBsW6AXveiN6UAFVRhNVoVyMxKlqFCJUvqzgDTeIic5mXHXcYLt7M7IyhP0efow7Vv0DXrsZGYnM1sEXFTNakf2Od85Pr/2SSrBtRmN/tw58867771/9tz58MLuBx9+dPHS5ae6rFUK+2kpSvU8YRoEl7BvuBHwvFLAikTAs+TwG8d/dgRK81L+YhYVHBRsKnnOU2aQ9OLi7/Txw0d8OjOE6oIJEdIEplxaJvhUftGE6joSDLtBxqTjpHiabsJRRD4nnkeogbmxpJSEo3YgBcMTSpE1lIaUy9wsHLY0M1DHXENIQWZLNX7dHfbi4mA0HPmPbC/ibjEIuu/Ji0vhHzQr07oAaVLBtJ7EcWUOrBY888prDRVLD9GoCS4lK0AfWB+0hlxDSkbyUuG/NMRT1yUsK7ReFAkiC2ZmepPniKfxJrXJ7xxYLqvagEzbg/JaEFMSlwGScQWpEQtcsFRxw1OSzphiqcE89U85YtKgGxowr3JqZj7SxzwzM3t7eIvLnos2KXCfQU5NbimiwHCRgc2bjpxgBgyzNCks9cslAwtGaEudK0lif1qSmZoWmHBPLyvr9oQiZaVv7nXNl/t5Zee/0krxAprwmidl8NJSbdBABQJX6G9lul+zEECN4kxOBTR2vFTjZZb0l0iUcJyWBVZVZunXR9BM4gNLBeSGCo8hg5hQ5YqYqlZZX4atZNbgHTDsRXBeS56WmTd/nSzM3CjWj7ZPfAVpn+pqqaVec2krGNY/kuwDLgT5mUm9YqC041y/z6fc6OgRNqSMHiiAwxtviH4I4ghznbLHUMNN381t0WA/Lw3Qbya5FJuEhJDvUBFPv3Vd4r4x6Yu17Mgh72HXr3DbSMde4da0noZr2Qd9E9vEOsNyVnCxWF5er2OmB76+rfsCM+5lXsfiE/Spdr9taP8PNq/3kr9/VCFy1052Gf1B3DRbKHyK+iiK1TdjFbRoAcbDklzkndYtFfWmhiTXoDjoUw+st09c4ntHb7R8LaB49UvhbyV8Xv3OuhdO4tVJiFXTpLHDO9EoGt7ZQFWiLk4gtx3k9gZkii0oVxgHiEb9aOPtirZN9vDmcg62YuiDtYO9ptnCJuXcg0Nq0gQLzeYK3SD+nh1bU1btGzcexBFJSmPKYkW48uWtK4lA/5vIv4Z+kBiv4hchriW1oJCmGANQbuiw7rRyPgE5YzKFLOKOw/FZnfEsAxkhE2M7vhkXBTrSOnE84wav/NaNpp+Q1k/njHekdrNNophaWH3I3Y3Yd/v4P3M7ap0mdVWBq4p2HDox961D4GegNbc7F0EecVVKN9xY/bLmeubmwSa02Nrd4Z5gr9KkFthQVxvX9cSSk4HBYulDgYdjhVWmaa+NdXbFlOfuITfcZmP4NoQ3pdcRpyhwz3SBgwNODTYe3sJ67rNZAsIPNMh2jXiKBgdpj2jl8a+DWR85HwQXs/uAQ6CC7/Gx/AFTxAyGmboA0UjXyW84dNEIpzAaNf+KbcFDM3wFCGfVxvrfzQ7EZ7odNNrsuL3i82YQezuX2ybEQTfeHGu3F0/3hvFoGP+4N7h7rxt5zwWfBp8F14M4+Cq4GzwMngT7QRr8vfPJDtm5cuGv3WD37O75Fnpmp5P5OOh9u5f/AaYQZhc=</latexit><latexit sha1_base64="TZlHvKtOKIKwvBb/Te4jyN8zrRk=">AAAMWHiczVbdjtQ2FM7SHyBsW6AXveiN6UAFVRhNVoVyMxKlqFCJUvqzgDTeIic5mXHXcYLt7M7IyhP0efow7Vv0DXrsZGYnM1sEXFTNakf2Od85Pr/2SSrBtRmN/tw58867771/9tz58MLuBx9+dPHS5ae6rFUK+2kpSvU8YRoEl7BvuBHwvFLAikTAs+TwG8d/dgRK81L+YhYVHBRsKnnOU2aQ9OLi7/Txw0d8OjOE6oIJEdIEplxaJvhUftGE6joSDLtBxqTjpHiabsJRRD4nnkeogbmxpJSEo3YgBcMTSpE1lIaUy9wsHLY0M1DHXENIQWZLNX7dHfbi4mA0HPmPbC/ibjEIuu/Ji0vhHzQr07oAaVLBtJ7EcWUOrBY888prDRVLD9GoCS4lK0AfWB+0hlxDSkbyUuG/NMRT1yUsK7ReFAkiC2ZmepPniKfxJrXJ7xxYLqvagEzbg/JaEFMSlwGScQWpEQtcsFRxw1OSzphiqcE89U85YtKgGxowr3JqZj7SxzwzM3t7eIvLnos2KXCfQU5NbimiwHCRgc2bjpxgBgyzNCks9cslAwtGaEudK0lif1qSmZoWmHBPLyvr9oQiZaVv7nXNl/t5Zee/0krxAprwmidl8NJSbdBABQJX6G9lul+zEECN4kxOBTR2vFTjZZb0l0iUcJyWBVZVZunXR9BM4gNLBeSGCo8hg5hQ5YqYqlZZX4atZNbgHTDsRXBeS56WmTd/nSzM3CjWj7ZPfAVpn+pqqaVec2krGNY/kuwDLgT5mUm9YqC041y/z6fc6OgRNqSMHiiAwxtviH4I4ghznbLHUMNN381t0WA/Lw3Qbya5FJuEhJDvUBFPv3Vd4r4x6Yu17Mgh72HXr3DbSMde4da0noZr2Qd9E9vEOsNyVnCxWF5er2OmB76+rfsCM+5lXsfiE/Spdr9taP8PNq/3kr9/VCFy1052Gf1B3DRbKHyK+iiK1TdjFbRoAcbDklzkndYtFfWmhiTXoDjoUw+st09c4ntHb7R8LaB49UvhbyV8Xv3OuhdO4tVJiFXTpLHDO9EoGt7ZQFWiLk4gtx3k9gZkii0oVxgHiEb9aOPtirZN9vDmcg62YuiDtYO9ptnCJuXcg0Nq0gQLzeYK3SD+nh1bU1btGzcexBFJSmPKYkW48uWtK4lA/5vIv4Z+kBiv4hchriW1oJCmGANQbuiw7rRyPgE5YzKFLOKOw/FZnfEsAxkhE2M7vhkXBTrSOnE84wav/NaNpp+Q1k/njHekdrNNophaWH3I3Y3Yd/v4P3M7ap0mdVWBq4p2HDox961D4GegNbc7F0EecVVKN9xY/bLmeubmwSa02Nrd4Z5gr9KkFthQVxvX9cSSk4HBYulDgYdjhVWmaa+NdXbFlOfuITfcZmP4NoQ3pdcRpyhwz3SBgwNODTYe3sJ67rNZAsIPNMh2jXiKBgdpj2jl8a+DWR85HwQXs/uAQ6CC7/Gx/AFTxAyGmboA0UjXyW84dNEIpzAaNf+KbcFDM3wFCGfVxvrfzQ7EZ7odNNrsuL3i82YQezuX2ybEQTfeHGu3F0/3hvFoGP+4N7h7rxt5zwWfBp8F14M4+Cq4GzwMngT7QRr8vfPJDtm5cuGv3WD37O75Fnpmp5P5OOh9u5f/AaYQZhc=</latexit><latexit sha1_base64="TZlHvKtOKIKwvBb/Te4jyN8zrRk=">AAAMWHiczVbdjtQ2FM7SHyBsW6AXveiN6UAFVRhNVoVyMxKlqFCJUvqzgDTeIic5mXHXcYLt7M7IyhP0efow7Vv0DXrsZGYnM1sEXFTNakf2Od85Pr/2SSrBtRmN/tw58867771/9tz58MLuBx9+dPHS5ae6rFUK+2kpSvU8YRoEl7BvuBHwvFLAikTAs+TwG8d/dgRK81L+YhYVHBRsKnnOU2aQ9OLi7/Txw0d8OjOE6oIJEdIEplxaJvhUftGE6joSDLtBxqTjpHiabsJRRD4nnkeogbmxpJSEo3YgBcMTSpE1lIaUy9wsHLY0M1DHXENIQWZLNX7dHfbi4mA0HPmPbC/ibjEIuu/Ji0vhHzQr07oAaVLBtJ7EcWUOrBY888prDRVLD9GoCS4lK0AfWB+0hlxDSkbyUuG/NMRT1yUsK7ReFAkiC2ZmepPniKfxJrXJ7xxYLqvagEzbg/JaEFMSlwGScQWpEQtcsFRxw1OSzphiqcE89U85YtKgGxowr3JqZj7SxzwzM3t7eIvLnos2KXCfQU5NbimiwHCRgc2bjpxgBgyzNCks9cslAwtGaEudK0lif1qSmZoWmHBPLyvr9oQiZaVv7nXNl/t5Zee/0krxAprwmidl8NJSbdBABQJX6G9lul+zEECN4kxOBTR2vFTjZZb0l0iUcJyWBVZVZunXR9BM4gNLBeSGCo8hg5hQ5YqYqlZZX4atZNbgHTDsRXBeS56WmTd/nSzM3CjWj7ZPfAVpn+pqqaVec2krGNY/kuwDLgT5mUm9YqC041y/z6fc6OgRNqSMHiiAwxtviH4I4ghznbLHUMNN381t0WA/Lw3Qbya5FJuEhJDvUBFPv3Vd4r4x6Yu17Mgh72HXr3DbSMde4da0noZr2Qd9E9vEOsNyVnCxWF5er2OmB76+rfsCM+5lXsfiE/Spdr9taP8PNq/3kr9/VCFy1052Gf1B3DRbKHyK+iiK1TdjFbRoAcbDklzkndYtFfWmhiTXoDjoUw+st09c4ntHb7R8LaB49UvhbyV8Xv3OuhdO4tVJiFXTpLHDO9EoGt7ZQFWiLk4gtx3k9gZkii0oVxgHiEb9aOPtirZN9vDmcg62YuiDtYO9ptnCJuXcg0Nq0gQLzeYK3SD+nh1bU1btGzcexBFJSmPKYkW48uWtK4lA/5vIv4Z+kBiv4hchriW1oJCmGANQbuiw7rRyPgE5YzKFLOKOw/FZnfEsAxkhE2M7vhkXBTrSOnE84wav/NaNpp+Q1k/njHekdrNNophaWH3I3Y3Yd/v4P3M7ap0mdVWBq4p2HDox961D4GegNbc7F0EecVVKN9xY/bLmeubmwSa02Nrd4Z5gr9KkFthQVxvX9cSSk4HBYulDgYdjhVWmaa+NdXbFlOfuITfcZmP4NoQ3pdcRpyhwz3SBgwNODTYe3sJ67rNZAsIPNMh2jXiKBgdpj2jl8a+DWR85HwQXs/uAQ6CC7/Gx/AFTxAyGmboA0UjXyW84dNEIpzAaNf+KbcFDM3wFCGfVxvrfzQ7EZ7odNNrsuL3i82YQezuX2ybEQTfeHGu3F0/3hvFoGP+4N7h7rxt5zwWfBp8F14M4+Cq4GzwMngT7QRr8vfPJDtm5cuGv3WD37O75Fnpmp5P5OOh9u5f/AaYQZhc=</latexit><latexit sha1_base64="TZlHvKtOKIKwvBb/Te4jyN8zrRk=">AAAMWHiczVbdjtQ2FM7SHyBsW6AXveiN6UAFVRhNVoVyMxKlqFCJUvqzgDTeIic5mXHXcYLt7M7IyhP0efow7Vv0DXrsZGYnM1sEXFTNakf2Od85Pr/2SSrBtRmN/tw58867771/9tz58MLuBx9+dPHS5ae6rFUK+2kpSvU8YRoEl7BvuBHwvFLAikTAs+TwG8d/dgRK81L+YhYVHBRsKnnOU2aQ9OLi7/Txw0d8OjOE6oIJEdIEplxaJvhUftGE6joSDLtBxqTjpHiabsJRRD4nnkeogbmxpJSEo3YgBcMTSpE1lIaUy9wsHLY0M1DHXENIQWZLNX7dHfbi4mA0HPmPbC/ibjEIuu/Ji0vhHzQr07oAaVLBtJ7EcWUOrBY888prDRVLD9GoCS4lK0AfWB+0hlxDSkbyUuG/NMRT1yUsK7ReFAkiC2ZmepPniKfxJrXJ7xxYLqvagEzbg/JaEFMSlwGScQWpEQtcsFRxw1OSzphiqcE89U85YtKgGxowr3JqZj7SxzwzM3t7eIvLnos2KXCfQU5NbimiwHCRgc2bjpxgBgyzNCks9cslAwtGaEudK0lif1qSmZoWmHBPLyvr9oQiZaVv7nXNl/t5Zee/0krxAprwmidl8NJSbdBABQJX6G9lul+zEECN4kxOBTR2vFTjZZb0l0iUcJyWBVZVZunXR9BM4gNLBeSGCo8hg5hQ5YqYqlZZX4atZNbgHTDsRXBeS56WmTd/nSzM3CjWj7ZPfAVpn+pqqaVec2krGNY/kuwDLgT5mUm9YqC041y/z6fc6OgRNqSMHiiAwxtviH4I4ghznbLHUMNN381t0WA/Lw3Qbya5FJuEhJDvUBFPv3Vd4r4x6Yu17Mgh72HXr3DbSMde4da0noZr2Qd9E9vEOsNyVnCxWF5er2OmB76+rfsCM+5lXsfiE/Spdr9taP8PNq/3kr9/VCFy1052Gf1B3DRbKHyK+iiK1TdjFbRoAcbDklzkndYtFfWmhiTXoDjoUw+st09c4ntHb7R8LaB49UvhbyV8Xv3OuhdO4tVJiFXTpLHDO9EoGt7ZQFWiLk4gtx3k9gZkii0oVxgHiEb9aOPtirZN9vDmcg62YuiDtYO9ptnCJuXcg0Nq0gQLzeYK3SD+nh1bU1btGzcexBFJSmPKYkW48uWtK4lA/5vIv4Z+kBiv4hchriW1oJCmGANQbuiw7rRyPgE5YzKFLOKOw/FZnfEsAxkhE2M7vhkXBTrSOnE84wav/NaNpp+Q1k/njHekdrNNophaWH3I3Y3Yd/v4P3M7ap0mdVWBq4p2HDox961D4GegNbc7F0EecVVKN9xY/bLmeubmwSa02Nrd4Z5gr9KkFthQVxvX9cSSk4HBYulDgYdjhVWmaa+NdXbFlOfuITfcZmP4NoQ3pdcRpyhwz3SBgwNODTYe3sJ67rNZAsIPNMh2jXiKBgdpj2jl8a+DWR85HwQXs/uAQ6CC7/Gx/AFTxAyGmboA0UjXyW84dNEIpzAaNf+KbcFDM3wFCGfVxvrfzQ7EZ7odNNrsuL3i82YQezuX2ybEQTfeHGu3F0/3hvFoGP+4N7h7rxt5zwWfBp8F14M4+Cq4GzwMngT7QRr8vfPJDtm5cuGv3WD37O75Fnpmp5P5OOh9u5f/AaYQZhc=</latexit>
“Good” image on manifold
“Bad” image off manifold
Decoupled
(First learn, then
reconstruct)