Meta-learning through the lenses of Statistical Learning Theory (Carlo Ciliberto)

Meta-learning through the lenses of
Statistical Learning Theory
Carlo Ciliberto
University College London
Machine Learning Data Science Meetup - Italian Association for Machine Learning
21/01/2021

The Machine Learning Revolution

The Machine Learning Revolution
Image from Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition 2016
But better performance comes with added complexity...

The Barrier: Model Selection
But... model selection
☐ Key to good performance,
☐ Delicate,
☐ Diﬃcult,
☐ Time-consuming,
☐ BOOORING...
Machine Learning
Experts (You!)
New users!
is _______ :

AutoML
Solution?
www.automl.org
Automate Model Selection!

Meta-Learners as Data Scientists
Imitate a Data scientist
Meta-learners imitate "human" data scientists:
Learn from past experience
(i.e. previous ML problems)
to
Choose which learning
algorithm is best suited
to the task at hand

Overview
Problem
Setting
Conditional
Meta-Learning
Theory
Summary
In this talk:
- What is Meta-learning?
- Latest research.
- A general mathematical
framework.
Meta
Learning

Overview
In this talk:
- What is Meta-learning?
- Latest research.
- A general mathematical
framework.
Problem
Setting
Conditional
Meta-Learning
Theory
Summary
Meta
Learning

Supervised Learning 101 - Notation
(Unknown) probability on
⇢
<latexit sha1_base64="8XvFjDgdc9nDAE7zh/fgjz6wpqQ=">AAAB63icdVBNSwMxEM3Wr1q/qh69BIvgacluK+2x4MVjFVsL7VKyabYbmmyWJCuUpX/BiwdFvPqHvPlvzLYVVPTBwOO9GWbmhSln2iD04ZTW1jc2t8rblZ3dvf2D6uFRT8tMEdolkkvVD7GmnCW0a5jhtJ8qikXI6V04vSz8u3uqNJPJrZmlNBB4krCIEWwKaahiOarWkOtfNBFC0JJmveG1LKkj1PB96LlogRpYoTOqvg/HkmSCJoZwrPXAQ6kJcqwMI5zOK8NM0xSTKZ7QgaUJFlQH+eLWOTyzyhhGUtlKDFyo3ydyLLSeidB2Cmxi/dsrxL+8QWaiVpCzJM0MTchyUZRxaCQsHodjpigxfGYJJorZWyGJscLE2HgqNoSvT+H/pOe7HnK960atfbOKowxOwCk4Bx5ogja4Ah3QBQTE4AE8gWdHOI/Oi/O6bC05q5lj8APO2yds+46J</latexit>
X ⇥ Y
<latexit sha1_base64="2nFdJ3EXqg0O4/dJhpyQWLVUHVQ=">AAACBXicdVDLSsNAFJ3UV62vqEtdDBbBVZiklXZZcOOyin1IE8pkOmmHTh7MTIQSunHjr7hxoYhb/8Gdf+OkraKiBy4czrmXe+/xE86kQujdKCwtr6yuFddLG5tb2zvm7l5bxqkgtEViHouujyXlLKItxRSn3URQHPqcdvzxWe53bqiQLI6u1CShXoiHEQsYwUpLffPQDbEaEcyz7tRVLKTyS7ie9s0yspzTGkIIalKrVO26JhWEqo4DbQvNUAYLNPvmmzuISRrSSBGOpezZKFFehoVihNNpyU0lTTAZ4yHtaRphvc/LZl9M4bFWBjCIha5IwZn6fSLDoZST0Ned+Ynyt5eLf3m9VAV1L2NRkioakfmiIOVQxTCPBA6YoETxiSaYCKZvhWSEBSZKB1fSIXx+Cv8nbceykWVfVMuNy0UcRXAAjsAJsEENNMA5aIIWIOAW3INH8GTcGQ/Gs/Eyby0Yi5l98APG6wf4Q5mQ</latexit>
` : Y ⇥ Y ! R
<latexit sha1_base64="XyrQNeauanCcELKxR9HgLPhYosk=">AAACF3icdVDLSgMxFM34rPVVdekmWARXQ2ZaaXFVcOOyFvuQTimZNNOGZiZDkhHK0L9w46+4caGIW935N6bTClb0QODknHu59x4/5kxphD6tldW19Y3N3FZ+e2d3b79wcNhSIpGENongQnZ8rChnEW1qpjntxJLi0Oe07Y8vZ377jkrFRHSjJzHthXgYsYARrI3UL9ge5fzCC7EeEczT26mnWUjVkiCyn++njWm/UES2e15BCEFDKqWyUzWkhFDZdaFjowxFsEC9X/jwBoIkIY004ViproNi3Uux1IxwOs17iaIxJmM8pF1DI2yG99Lsrik8NcoABkKaF2mYqT87UhwqNQl9UznbUP32ZuJfXjfRQbWXsihONI3IfFCQcKgFnIUEB0xSovnEEEwkM7tCMsISE22izJsQvi+F/5OWazvIdq7LxVpjEUcOHIMTcAYcUAE1cAXqoAkIuAeP4Bm8WA/Wk/Vqvc1LV6xFzxFYgvX+BXPzoWs=</latexit>
Loss function
Goal: minimize
Input set Output set
X
<latexit sha1_base64="54aBn9K2MDOaBG7N6l0F/RBnmUA=">AAAB8nicdVDNSgMxGMzWv1r/qh69BIvgacluK+2x4MVjFVsL26Vk02wbmk2WJCuUpY/hxYMiXn0ab76N2baCig4EhpnvI/NNlHKmDUIfTmltfWNzq7xd2dnd2z+oHh71tMwUoV0iuVT9CGvKmaBdwwyn/VRRnESc3kXTy8K/u6dKMyluzSylYYLHgsWMYGOlYJBgMyGY5/35sFpDrn/RRAhBS5r1hteypI5Qw/eh56IFamCFzrD6PhhJkiVUGMKx1oGHUhPmWBlGOJ1XBpmmKSZTPKaBpQInVIf5IvIcnlllBGOp7BMGLtTvGzlOtJ4lkZ0sIurfXiH+5QWZiVthzkSaGSrI8qM449BIWNwPR0xRYvjMEkwUs1khmWCFibEtVWwJX5fC/0nPdz3keteNWvtmVUcZnIBTcA480ARtcAU6oAsIkOABPIFnxziPzovzuhwtOaudY/ADztsn4JCRsA==</latexit>
Y
<latexit sha1_base64="DyDjBthmOGFZ1VCqgAPaj1NvJ/I=">AAAB8nicdVDLSgMxFM3UV62vqks3wSK4GjJtpV0W3LisYh8yHUomzbShmWRIMkIZ+hluXCji1q9x59+YaSuo6IHA4Zx7ybknTDjTBqEPp7C2vrG5Vdwu7ezu7R+UD4+6WqaK0A6RXKp+iDXlTNCOYYbTfqIojkNOe+H0Mvd791RpJsWtmSU0iPFYsIgRbKzkD2JsJgTz7G4+LFeQW71oIISgJY1a3WtaUkOoXq1Cz0ULVMAK7WH5fTCSJI2pMIRjrX0PJSbIsDKMcDovDVJNE0ymeEx9SwWOqQ6yReQ5PLPKCEZS2ScMXKjfNzIcaz2LQzuZR9S/vVz8y/NTEzWDjIkkNVSQ5UdRyqGRML8fjpiixPCZJZgoZrNCMsEKE2NbKtkSvi6F/5Nu1fWQ613XK62bVR1FcAJOwTnwQAO0wBVogw4gQIIH8ASeHeM8Oi/O63K04Kx2jsEPOG+f4hWRsQ==</latexit>
R(f, ⇢) = E⇢ `(f(x), y)
<latexit sha1_base64="TJpc3hvxGTL4ZOsTMtNO2ARN7G8=">AAAD/XicfVLdbtMwFPZWfkb46+CSG4tqUoemKumGthukIZg0LhBjolunuqoc56S16tiR7YxVWeAteAPuEHeIG27hHXgbnLYbbTWwFPno+/GJP58wFdxY3/+9tFy5dv3GzZVb3u07d+/dr64+ODIq0wxaTAml2yE1ILiEluVWQDvVQJNQwHE4fFHyx6egDVfynR2l0E1oX/KYM2od1Kv6hEsJ+pCbYT3eIHqg1vEzTPbOUmC2l5dA8YGAEPW4fra+MVrvVWt+o/l02/d97Irtza1gxxWbvr/VbOKg4Y9XDU3XQW91+RuJFMsSkJYJakwn8FPbzam2nAkoPJIZSCkb0j50XClpAqabj69W4DWHRDhW2n3S4jE668hpYswoCZ0yoXZgFrkSvIrrZDbe6eZcppkFySaN4kxgq3CZE464dgmIkSso09z9K2YDqimzLk3P89awsVRGVEeuyOLYIxLeM5UkDszJYZGTsmcY4sNinppE+5ffKzyPvAQXjYbXDnqTgqZW6Sc5obqfcFnk0/1/Mno2kbndHZdqdcojuOxpGBVFp9nNiYDYEkFlX0BeC4qNvNYsiOb9gSV6jF5ll+4lOsGF+7w0Tj3nxTiKlsmowEqCS2busu3pPV1/3F4I4mSGO1ngBoMZcn/SxLjcysmdV16O74xhMXLl3nhRs7egoaI/wz4vPDfoF9OM/10cNRuB3wjebtV2X01HfgU9Qo9RHQVoG+2ifXSAWoihT+gH+ol+VT5WPle+VL5OpMtLU89DNLcq3/8AL29WXg==</latexit>
f : X ! Y
<latexit sha1_base64="1pIST/tctajb8dQkixaYp6h0iJ8=">AAAD1HicfVNdb9MwFPUWPkb42uCRF4tqEkKoSrKiTjwNQaVJCDEq2rWqq8lxnNSqY0e2M1alkZAQr7zwCj+DH8O/wWk7aKPBlSJfnXOur318E2acaeN5v7a2nWvXb9zcueXevnP33v3dvQd9LXNFaI9ILtUgxJpyJmjPMMPpIFMUpyGnp+H0VcWfnlOlmRQfzCyj4xQngsWMYGOhYfwCDZCRaHi22/CawfO253nQJu2Dln9okwPPawUB9JveIhpgFSdne9s/USRJnlJhCMdaj3wvM+MCK8MIp6WLck0zTKY4oSObCpxSPS4WJy7hvkUiGEtlP2HgAl2vKHCq9SwNrTLFZqLrXAVexY1yEx+OCyay3FBBlo3inEMjYXV9GDFFieEzm2CimD0rJBOsMDHWJNd196E2WERYRTbJ49hFgn4kMk0tWKBuWaCqZxjCbrlJdS4yu/FfvlO6LnpNrTWKvrXQu4wqbKR6WiCskpSJslit/5Phi6XMrna7TMlzFtE/PTXBvBwF4wJxGhvEsUg4LRp++axoBCVSLJkYpBboVeXCvsTIv6yeV4Wrmnm5sKKnc8yhFNQ6s3HZweqetj8c1IwYrnHDGjeZrJHHyyba+lYN5KaSCUFVl+npWkHdcmnfuK7p1DSYJ2vsy9K1g345zfDfST9o+l7Tf99qHL35tBz5HfAIPAZPgA/a4AgcgxPQAwSk4Bv4Dn44fWfufHa+LKXbW6vf5CHYCOfrbyNPR14=</latexit>
(candidate) input-output predictor
Expected Risk

In practice, can be accessed only via finite samples...
Supervised Learning 101 - Learning Algorithms
⇢
Learning Algorithm:
D = (xi, yi)n
i=1
<latexit sha1_base64="gU3oeb13aKBSNEGUsBTQUHhkQ+c=">AAAC6nicdVJda9swFFXcfXTeR9uNPe1FLBQ6GEFOM9KXQmErbG9tWNqUODOyLLcismQkeW0whv2GvpU9Dfa0/Yb9kP2bXicZS8J2QXA45+jocq/iXArrCPnd8Nbu3L13f/2B//DR4ycbm1tPT6wuDON9pqU2g5haLoXifSec5IPccJrFkp/G47e1fvqZGyu0+ugmOR9l9FyJVDDqgIo2n7/D+3jnKhKvJ5F4FZViP6g+Ad8krfabLiEEA+judoI9ALuEdNptHLTItJpoXkfRVuNXmGhWZFw5Jqm1w4DkblRS4wSTvPLDwvKcsjE950OAimbcjspp/xXeBibBqTZwlMNTdvFGSTNrJ1kMzoy6C7uq1eS/tGHh0r1RKVReOK7Y7KG0kNhpXA8DJ8Jw5uQEAGVGQK+YXVBDmYOR+b6/ja2jKqEmAVCkqR8qfsl0lgFZhr2qDOs34xj3qmXp8CqH4L/6YTVN69uCSqwVh/Al/2BuZSAPVrLOFrSzWY6F9HqJy0ahFDc9YccLF1Yb0zCJVQ80Bwv/s1X8f3DSbgWkFRx3mgcfvsxWv45eoJdoBwWoiw7Qe3SE+oihEn1DP9BPT3rX3o33dWb1GvPv8gwtlff9Frzb64Q=</latexit>
A : D 7! f
<latexit sha1_base64="rxRCUxecxlNxWmKVGR18sPoFLu4=">AAAC7HicdVLLbtNAFJ2YVzGvFBYs2IyIKrGKxmlQKlZFUAl2JSJtqjiKxuNxO8o8rJlxS2RZ4iPYIVZIrOAT+BD+hmsniCSCK1k6PufM8Z17neRSOE/Ir1Zw7fqNm7d2bod37t67/6C9+/DEmcIyPmJGGjtOqONSaD7ywks+zi2nKpH8NJm/qvXTS26dMPq9X+R8qui5Fplg1AM1az+OFfUXjMryZfXiNYa33HmDs1m7Q7q95wNCCAYw2O9HBwD2Cen3ejjqkqY6aFXHs93Wzzg1rFBceyapc5OI5H5aUusFk7wK48LxnLI5PecTgJoq7qZlc4MK7wGT4sxYeLTHDbt+oqTKuYVKwFn367a1mvyXNil8djAthc4LzzVbfigrJIY71uPAqbCcebkAQJkV0CtmF9RS5mFoYRjuYeepTqlNARRZFsaaXzGjFJBlPKzKZn5JgofVpnT0IYfgv/pR1aSNXEElNppD+IZ/vLLCKvB4K+tsTTtb5jhIr9e4aRRaczsUbr52YLsxA5PY9kBzsPA/W8X/Bye9bkS60bt+5/Dtx+Xqd9AT9BQ9QxEaoEP0Bh2jEWKoQl/Rd/Qj0MGn4HPwZWkNWqvf5RHaqODbb4kk7WI=</latexit>
Example: Empirical Risk Minimization (ERM)
A(D) = argmin
f2H
R(f, D)
<latexit sha1_base64="p35BsKgKcdJ67qga3z5AlsPM72s=">AAAD+XicfVPdbtMwGPVafkb4WQeX3FhUkzpUVUk3tN0gDbFJ4wIxKrp1qqvKcZ3WmuNEtjM2eeEheAPuELcIiVt4C96Gr222tdXAUpSjc77jLz7+EqZSGOv7f5ZK5Vu379xdvufdf/Dw0Upl9fGhSTLNeJslMtGdkBouheJtK6zknVRzGoeSH4Unr8f60SnXRiTqgz1PeS+mQyUiwagFql+pEyqHtd11/BITqoexUH0XEaHIaJR/wgAU1y1hTmpRfXe9X6n6jeaLLd/3MYCtjc1gG8CG7282mzho+JNVRcU66K+WfpBBwrKYK8skNaYb+KntOaqtYJLnHskMTyk7oUPeBahozE3PTY6V4zVgBjhKNDzK4gk763A0NuY8DqEypnZkFrUxeZPWzWy03XNCpZnlik0bRZnENsHjjPBAaM6sPAdAmRbwrZiNqKbMQpKe561hY6kaUD0AkEWRRxT/yJI4BtKRVu7IuGcY4lY+L+2dpbDxtb6Xex7Z5RCN5m+BepdyTW2in7viMnJXvP9XRs+mZfCG7VKdnIoBv+ppGJV5t9lzRPLIEknVUHJXDfK6qzZzosVwZImesDfZFdxEN7h0X4yNhecin0TRNhmVOFEckpk7bKc4J/THnYUgjme04wUNJu9a3J82MZDbeGrnK6+mc8awGHkCd7xYs7dQA7/AjPoq92DQL6cZ/xscNhuB3wjeb1Z33hQjv4yeomeohgK0hXbQPjpAbcTQZ/QT/UK/y678pfy1/G1aWloqPE/Q3Cp//wumM1SR</latexit>
R(f, D) =
1
n
n
X
i=1
`(f(xi), yi)
<latexit sha1_base64="SZNOZQTzuFdZpXta+u7naja8Qs4=">AAAEB3icfVLLbhMxFHUbHmV4tIUlG4uoUoqqaCYtaoVUqYhWKgtEqUibKg6Rx/EkVj2eke0pjVx/AH/BH7ACsWXDFrb8DTePliQqWJrx1Xn4js/cOJfC2DD8PTdfunHz1u2FO8Hde/cfLC4tPzwyWaEZr7NMZroRU8OlULxuhZW8kWtO01jy4/j05YA/PuPaiEy9s/2ct1LaVSIRjFqA2kvPiVCK60NhTivJ2u4q3sYk0ZS5yDvliSnSthPbkX+vMOFSVpLKeVusrvXh1V4qh9Xas80wDDEUm+sb0RYU62G4UavhqBoOVxmN10F7ef4L6WSsSLmyTFJjmlGY25aj2gomuQ9IYXhO2Snt8iaUiqbctNzwkh6vANLBSabhURYP0UmHo6kx/TQGZUptz8xyA/A6rlnYZKvlhMoLyxUbNUoKiW2GB4nhjtCcWdmHgjIt4Fsx61HIyEKuQRCsYGOp6lDdgaJIkoAo/oFlaQqgI4fekUHPOMaHfpraO8/h4L/8ng8CssshGs1fA/Qm55raTD91hOpuKpR34/1/Mno+ksEOx+U6OxMdftXTMCp9s9ZyRPLEEklVV3JXjvyaK9c80aLbs0QP0evsCv5EM7p0XwyMY8+FH0ZRNwWVOFMckpm6bGN8T+iPGzNBnExwJzNcrzdB7o+aGMhtMMPTyqtBnjDMRp7BP57V7M1oqOxOsC98AIN+Oc3438VRrRqF1ejtRnnn1XjkF9Bj9ARVUIQ20Q7aRweojhj6hH6gn+hX6WPpc+lr6dtIOj839jxCU6v0/Q+D0loE</latexit>
Choosing the Hypothesis
space is key!

Supervised Learning 101 - Prototypical Results
If is "nice enough":
⇢
Excess Risk Bound
↵ = ↵(A, ⇢) > 0
<latexit sha1_base64="nYpqvt1Tjd0yADnCjhhX9wX3DyA=">AAAC+nicdVJdi9NAFJ3GrzV+dfXRl8FSWEHKpFvpvigrsiD4sha726Up5WY62Q6dzISZya4lBnz3X/gmPgk+Kf4Q/42TpGJb9MLA4ZwzJzf3TpQKbiwhvxrelavXrt/Yuenfun3n7r3m7v0TozJN2ZAqofQoAsMEl2xouRVslGoGSSTYabR4WeqnF0wbruRbu0zZJIFzyWNOwTpq2myHINI54Ge4BnthAnZOQeQviiehnqvH+Dkm02aLdLpP+4QQ7EB/vxccOLBPSK/bxUGHVNVCqzqe7jZ+hjNFs4RJSwUYMw5Iaic5aMupYIUfZoalQBdwzsYOSkiYmeTV/xS47ZgZjpV2R1pcses3ckiMWSaRc5bdmm2tJP+ljTMbH0xyLtPMMknrD8WZwFbhcjh4xjWjViwdAKq56xXTOWig1o3Q9/02NhbkDPTMgSyO/VCyS6qSxJF5OCjyanpRhAfFpnT0LnXBf/WjokobmgwEVpK58A3/aGV1i8CjrayzNe2szjEuvVzqppFLyfSAm8Xahe3GlJvEtsc15xb+Z6v4/+Ck2wlIJ3jTax2+/lCvfgc9RI/QHgpQHx2iV+gYDRFFH9FX9B398N57n7zP3pfa6jVWz+UB2ijv22/k7/Ht</latexit>
is problem- & algorithm-dependent...
...Ideally, we would like to use the best algorithm for the learning problem
A
<latexit sha1_base64="kH9cZcTZazVKuq1dFXGDwMORLtg=">AAAC3nicdVJdixMxFE3Hr3X82tVHX4JlwaeS6Va6jyuyIPiyFrvbpVPWTObObmgmGZKMWoYBn3wTnwSf9Jf4Q/w33plWbIteCBzOOTm53JukUNJ5xn51gmvXb9y8tXM7vHP33v0Hu3sPT50prYCxMMrYScIdKKlh7KVXMCks8DxRcJbMXzT62TuwThr9xi8KmOX8UstMCu6RmsY591eCq+p5fbHbZb3+syFjjCIYHgyiQwQHjA36fRr1WFtdsqqTi73Ozzg1osxBe6G4c9OIFX5WceulUFCHcemg4GLOL2GKUPMc3Kxqe67pPjIpzYzFoz1t2fUbFc+dW+QJOpse3bbWkP/SpqXPDmeV1EXpQYvlQ1mpqDe0GQBNpQXh1QIBF1Zir1RcccuFxzGFYbhPnec65TZFUGZZGGt4L0yeI1nFo7pqZ5YkdFRvSscfCgz+qx/XbdrYlVxRowHDN/yTlRXHTydbWedr2vkyx2F6s7hNo9Qa7Ei6+dqF7cYMTmLbg83hwv9slf4fnPZ7EetFrwfdo1cfl6vfIY/JE/KURGRIjshLckLGRBBDvpLv5EfwNvgUfA6+LK1BZ/VdHpGNCr79BvHV6Kc=</latexit>
⇢
R A(D), ⇢) min
f:X!Y
R(f, ⇢)  O
✓
1
n↵
◆
<latexit sha1_base64="D867Q0u4r2BXrg/lN4rmZiGkFfo=">AAAEN3icfVNNbxMxEHUbPsry1cKRi0VUKUUQ7aZFrTgV0UrlgFoq0qaKQ+R1vFlTr3exvaWVu/1XHPgXXDlxQ1y5cIVJsm2TqGBp5dG89zzrN+Mwk8JY3/82M1u5dv3Gzblb3u07d+/dn194sGfSXDPeZKlMdSukhkuheNMKK3kr05wmoeT74eGrAb5/xLURqXpnTzLeSWhfiUgwaiHVnf9AhFJc7wpzSELRrxEq+7WNpadEx+kSfoZJIlTXRS9Ii9iUHBT4kl+LStYZkfzjGd6GLbI1EmnKXFA49R4Oy2JaEC36sV3qzlf9euP5qu/7GILV5ZVgDYJl319pNHBQ94erisq1012Y/Ux6KcsTriyT1Jh24Ge246i2gkleeCQ3PKPskPZ5G0JFE246bmhKgRch08NRquFTFg+z4wpHE2NOkhCYCbWxmcYGyauwdm6jtY4TKsstV2xUKMoltikeOIx7QnNm5QkElGkB/4pZTMEUC33wPG8RG0tVj+oeBHkUeUTxTyxNEkg6sls4MqgZhni3mIQ2jzM4+BLfLDyPbHCwRvM3kNrOuKY21U8coboPjStcuf+PRo9HNNjhuEynR6LHL2oaRmXRbnTcsLlEUtWX3FWD4qmrNsrOEj3MXiVX0Il2cK4+HQhLzWkxtKJpcipxqjg4M3HZVnlPqI9bU0YcjGEHU1gcj4FboyIGfBvM/CTzYpDHBNOWp9Djac7mFAdezBj6svBg0M+nGf872GvUA78evF2prr8uR34OPUKPUQ0FaBWtoy20g5qIoa/oN/ozgypfKt8rPyo/R9TZmVLzEE2syq+/S/9tHQ==</latexit>
the larger the better!

Ideally, we would like to find leading to the fastest rate
✓
<latexit sha1_base64="ZZkFWwYBe2kYrYg0RXABHQBzons=">AAADHnicdVJbi9NAFJ6NtzXedvXRl8GyID6USbfSfVzQhQUR12K7XZqyTKYnzdhkJsxM1BIC/gRf9clf45v4qv/Gk7ZiG/TAMB/fd+bc5kR5Kq1j7NeOd+Xqtes3dm/6t27fuXtvb//+0OrCCBgInWoziriFVCoYOOlSGOUGeBalcB7Nn9X6+TswVmr1xi1ymGR8pmQsBXdIDUOXgOOXey3W7jztMcYogt5hNzhCcMhYt9OhQZstrUXWdna573nhVIsiA+VEyq0dByx3k5IbJ0UKlR8WFnIu5nwGY4SKZ2An5bLcih4gM6WxNniUo0t280XJM2sXWYSeGXeJbWo1+S9tXLj4aFJKlRcOlFgliouUOk3r3ulUGhAuXSDgwkislYqEGy4cTsj3/QNqHVdTbqYIijj2QwXvhc4yJMuwX5VhnTOKaL/alk4+5Bj4r35S+X74HHA0Bl4i9SoHw502T8qQm1kmVVWu72XWgS14SrUCLGIr7mgdUqA8auS82NAuGlqSbIinqyQWS6w/fttTKgWmL+1840GzO43jbPpgh7g1f1aD/h8MO+2AtYPX3dbxi4+r/dklD8kj8pgEpEeOySk5IwMiyFvyiXwmX7yv3jfvu/dj5ertrHfuAdky7+dvWK4B1w==</latexit>
Enter the Hyper(or Meta)parameters
We can parametrize a family of Algorithms
can represent any
meta-parameter...
✓
↵ = ↵(A, ⇢) > 0
#Iteratio kernel
Drop-out
Regularization
Step-size
✓
Possible
A(D) = A(✓; D)
<latexit sha1_base64="Ws/MmLCeNmnkzdUVraCZMzqQ7Pk=">AAAD5XicfVJdb9MwFPVaPkb4WAcvSLxYVJM2hKqkK+okhDREK40HxKjo1qmuJse5Sa06TmQ7Y1UW/gFviFdeeIU3fgz/Bvdj0FYDS5GP7jnHNz6+fiq4Nq77a61Uvnb9xs31W87tO3fvbVQ27x/pJFMMuiwRier5VIPgErqGGwG9VAGNfQHH/ujVhD8+A6V5It+bcQqDmEaSh5xRY0unlYeEimi7tYNf4CkiZgiGPm/tnFaqbq3+rOm6Lragudvw9izYdd1GvY69mjtdVTRfh6ebpZ8kSFgWgzRMUK37npuaQU6V4UxA4ZBMQ0rZiEbQt1DSGPQgn16hwFu2EuAwUfaTBk+ri46cxlqPY98qY2qGepWbFK/i+pkJ9wY5l2lmQLJZozAT2CR4kgcOuAJmxNgCyhS3/4rZkCrKjE3NcZwtrA2VAVWBBVkYOkTCB5bEsS3mpFPkZNLT93GnWKba56k9+C/fLhyHtMBGo+CNLb1NQVGTqCc5oSqKuSzy+f4/GT2fyexuj0tVcsYD+NNTMyqKfn2QEwGhIYLKSEBe9YqnebVeEMWjoSFqWr3KLu1L9L1L98XEOPdcFNMoujqjAicSbDJLl+3N72n7495KECcL3MkKNxwukAezJtrmNpnQZSWXElSH69GCYTXyxL7xqqa9orFDvsC+LBw76JfTjP8Njuo1z6157xrV/dfzkV9Hj9BjtI081ET76AAdoi5i6CP6hr6jH+Wo/Kn8ufxlJi2tzT0P0NIqf/0NdoRLzw==</latexit>

Meta-learning through the lenses of Statistical Learning Theory (Carlo Ciliberto)

Meta-learning through the lenses of Statistical Learning Theory (Carlo Ciliberto)

Recommended

Recommended

More Related Content

Similar to Meta-learning through the lenses of Statistical Learning Theory (Carlo Ciliberto)

Similar to Meta-learning through the lenses of Statistical Learning Theory (Carlo Ciliberto) (20)

More from MeetupDataScienceRoma

More from MeetupDataScienceRoma (20)

Recently uploaded

Recently uploaded (20)

Meta-learning through the lenses of Statistical Learning Theory (Carlo Ciliberto)