Dmytro Fishman, profile picture
Uploaded byDmytro Fishman
PDF, PPTX869 views

Introduction to Gaussian Processes

The document provides an introduction to Gaussian processes. It explains that Gaussian processes allow modeling any function directly and estimating uncertainty for predictions. It demonstrates how two random variables can be jointly distributed as a multivariate Gaussian distribution, and how the conditional distribution of one variable given the other can be derived from the joint distribution. Gaussian processes use these properties to perform nonparametric machine learning by modeling relationships between variables without assuming a specific function form.

Data & Analyticsā—¦
Related topics:
Data Mining Insightsā€¢

Embed presentation

Download as PDF, PPTX
1 / 141
2 / 141
3 / 141
4 / 141
5 / 141
6 / 141
7 / 141
8 / 141
9 / 141
10 / 141
11 / 141
12 / 141
13 / 141
14 / 141
15 / 141
16 / 141
17 / 141
18 / 141
19 / 141
20 / 141
21 / 141
22 / 141
23 / 141
24 / 141
25 / 141
26 / 141
27 / 141
28 / 141
29 / 141
30 / 141
31 / 141
32 / 141
33 / 141
34 / 141
35 / 141
36 / 141
37 / 141
38 / 141
39 / 141
40 / 141
41 / 141
42 / 141
43 / 141
44 / 141
45 / 141
46 / 141
47 / 141
48 / 141
49 / 141
50 / 141
51 / 141
52 / 141
53 / 141
54 / 141
55 / 141
56 / 141
57 / 141
58 / 141
59 / 141
60 / 141
61 / 141
62 / 141
63 / 141
64 / 141
65 / 141
66 / 141
67 / 141
68 / 141
69 / 141
70 / 141
71 / 141
72 / 141
73 / 141
74 / 141
75 / 141
76 / 141
77 / 141
78 / 141
79 / 141
80 / 141
81 / 141
82 / 141
83 / 141
84 / 141
85 / 141
86 / 141
87 / 141
88 / 141
89 / 141
90 / 141
91 / 141
92 / 141
93 / 141
94 / 141
95 / 141
96 / 141
97 / 141
98 / 141
99 / 141
100 / 141
101 / 141
102 / 141
103 / 141
104 / 141
105 / 141
106 / 141
107 / 141
108 / 141
109 / 141
110 / 141
111 / 141
112 / 141
113 / 141
114 / 141
115 / 141
116 / 141
117 / 141
118 / 141
119 / 141
120 / 141
121 / 141
122 / 141
123 / 141
124 / 141
125 / 141
126 / 141
127 / 141
128 / 141
129 / 141
130 / 141
131 / 141
132 / 141
133 / 141
134 / 141
135 / 141
136 / 141
137 / 141
138 / 141
139 / 141
140 / 141
141 / 141
Dmytro Fishman (dmytro@ut.ee) Introduction to Gaussian Processes
x f(x) y Letā€™s take a look inside
x y= f(x) y
x Let be linear functiony= f(x) y= āœ“0 + āœ“1x y
x Let be linear functiony= f(x) y= āœ“0 + āœ“1x y
x Let be linear functiony= f(x) arg min āœ“ nX i=1 (yi Ė†yi)2 Ė†yi = āœ“0 + āœ“1xi yi = āœ“0 + āœ“1xi + āœi arg min āœ“ nX i=1 (yi Ė†yi)2 Ė†yi = āœ“0 + āœ“1xi yi = āœ“0 + āœ“1xi + āœi Find and by optimising error arg min āœ“ nX i=1 (yi Ė†yi)2 Ė†yi = āœ“0 + āœ“1xi y = āœ“ + āœ“ x + āœ arg min āœ“ nX i=1 (yi Ė†yi)2 Ė†yi = āœ“0 + āœ“1xi y = āœ“ + āœ“ x + āœ y= āœ“0 + āœ“1x y
x But if data is not linear? y
x But if data is not linear? y= āœ“0 + āœ“1x y
x y= āœ“0 + āœ“1x + āœ“2x2 But if data is not linear? y
x But if data is not linear? y= āœ“0 + āœ“1x + āœ“2x2 + āœ“3x3 y
x What if donā€™t want to assume a speciļ¬c form? y
x GPs let you model any function directly y
x y x y Parametric ML Nonparametric ML A learning model that summarizes data with a set of parameters of ļ¬xed size (independent of the number of training examples) is called a parametric model. Algorithms that do not make strong assumptions about the form of the mapping function are called nonparametric machine learning algorithms. y= āœ“0 + āœ“1x
x y x y Parametric ML Nonparametric ML A learning model that summarizes data with a set of parameters of ļ¬xed size (independent of the number of training examples) is called a parametric model. Algorithms that do not make strong assumptions about the form of the mapping function are called nonparametric machine learning algorithms. y= āœ“0 + āœ“1x Question: is K-nearest neighbour parametric or nonparametric algorithm according to these deļ¬nitions?
x GPs let you model any function directly y
x y GPs let you model any function directly estimates the uncertainty for each new prediction
x y If I ask you to predict for xi xiyi
x y If I ask you to predict for ? xiyi You better be very uncertain xi
How is it even possible?
We will need Normal distribution x xi y
Āµ 1p 2ā‡” e (x Āµ)2 2 2 With average coordinate and standard deviation from centre Āµ Many important processes follow normal distribution
Āµ 1p 2ā‡” e (x Āµ)2 2 2N(Āµ, 2 ) With average coordinate and standard deviation from centre Āµ Many important processes follow normal distribution
X1 ā‡  N(Āµ1, 2 1)1p 2ā‡” e (x Āµ)2 2 2 With average coordinate and standard deviation from centre Āµ Many important processes follow normal distribution Āµ1 1
1p 2ā‡” e (x Āµ)2 2 2 What If I draw another distribution? With average coordinate and standard deviation from centre Āµ Many important processes follow normal distribution X1 ā‡  N(Āµ1, 2 1) Āµ1 1
X1 ā‡  N(Āµ1, 2 1) X2 ā‡  N(Āµ2, 2 2) Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 X1 X20 0
X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1)
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1)
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1)
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1)
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) What if we would join them into one plot?
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) X2 X1
X2 X1 Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 X = ļ£æ x1 x2 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1)
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 X2 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X1 ā‡  N(0, 1) X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 X2 ā‡  N(0, 1) Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1 Covariance matrix or āŒƒ
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 Joint distribution of variables and 0 x1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 Joint distribution of variables andx1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 Joint distribution of variables andx1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1 Similarity
ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X10 0 X2 X10 0 X2 ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Positive value of does not tell much about X1 X2 Some similarity (correlation) Positive value of with good probability means positive X1 X2 No similarity (no correlation)
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 Joint distribution of variables andx1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1
Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 Joint distribution of variables and 0 x1 x2 X1 ā‡  N(0, 1) X2 ā‡  N(0, 1) X2 X1 ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—†
X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—†
X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—†
X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—†
X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 1|2 Āµ1|2 ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—†
X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—†
ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—† X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21
X2 X1 x2 1|2 Āµ1|2 ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—† Conditional distribution P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21 P(x1, x2) Joint distribution ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† X10 0 X2 N(Āµ, 2 ) Normal distribution or 1D Gaussian or 2D Gaussian
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—†
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† Sampling Samples from 2D Gaussian
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13) Sampling Samples from 2D Gaussian
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13) Sampling Samples from 2D Gaussian 0 1st 2nd 1 1
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13) Sampling Samples from 2D Gaussian 0 1st 2nd 1 1
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65)
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65)
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65)
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) There is little dependency
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† There is little dependency
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) There is little dependency Samples from 2D Gaussian
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) There is little dependency 0 1st 2nd 1 1 Samples from 2D Gaussian
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) There is little dependency 0 1st 2nd 1 1 Samples from 2D Gaussian
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) Samples from 2D Gaussian There is little dependency 0 1st 2nd 1 1 (-0.03,-0.24)
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) Samples from 2D Gaussian There is little dependency 0 1st 2nd 1 1 (-0.03,-0.24)
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) Samples from 2D Gaussian There is little dependency 0 1st 2nd 1 1 (-0.03,-0.24)
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) Samples from 2D Gaussian There is little dependency 0 1st 2nd 1 1 (-0.03,-0.24) More dependent values
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23, 1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) Samples from 2D Gaussian There is little dependency 0 1st 2nd 1 1 (-0.03,-0.24) More dependent values How would a sample from 20D Gaussian look like?
2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—†
20D Gaussian ?
20D Gaussian 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0 0 . . . 0 0 1 0 . . . 0 ... ... ... ... ... 0 0 0 . . . 1 3 7 7 7 5 1 C C C A Sampling (0.73, -0.12, 0.42, 1.2,ā€¦, 16 more) 0 1st 2nd 1 1 3rd 4th 5th 6th 7th
20D Gaussian Letā€™s add more dependency between points 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A
20D Gaussian Letā€™s add more dependency between points (0.73, 0.18, 0.68, -0.2,ā€¦, 16 more) 0 1st 2nd 1 1 3rd 4th 5th 6th 7th 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A
20D Gaussian Letā€™s add more dependency between points 0 1st 2nd 1 1 3rd 4th 5th 6th 7th We want some notion of smoothness between pointsā€¦ 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A
20D Gaussian Letā€™s add more dependency between points 0 1st 2nd 1 1 3rd 4th 5th 6th 7th We want some notion of smoothness between pointsā€¦ So that dependancy between 1st and 2nd points is larger than between 1st and the 3rd. 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A
20D Gaussian Letā€™s add more dependency between points We want some notion of smoothness between pointsā€¦ So that dependancy between 1st and 2nd points is larger than between 1st and the 3rd. 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A We might have just increased corresponding values in covariance matrix, right?
20D Gaussian Letā€™s add more dependency between points We want some notion of smoothness between pointsā€¦ So that dependancy between 1st and 2nd points is larger than between 1st and the 3rd. 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A We might have just increased corresponding values in covariance matrix, right? We need a way to generate a ā€œsmoothā€ covariance matrix automatically depending on the distance between points
We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 20D Gaussian
We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 20D Gaussian 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K120 K21 K22 K23 . . . K220 ... ... ... ... ... K201 K202 K203 . . . K2020 3 7 7 7 5 1 C C C A 0 1st 2nd 1 1 3rd 4th 5th 6th 7th
We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 Āµā‡¤ ā‡¤ 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 Āµā‡¤ ā‡¤ 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
f1 f2 f3 We are interested in modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3
f1 f2 f3 We are interested in modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Previously we were using: to generate correlated points, can we do it again here? ļ£æ f1 f2 ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—†
f1 f2 f3 We are interested in modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Wait! But now we have three points, we cannot use the same formula! ļ£æ f1 f2 ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Previously we were using: to generate correlated points, can we do it again here?
f1 f2 f3 We are interested in modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Okā€¦ What about now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.5 0.5 0.5 1 0.5 0.5 0.5 1 3 5 1 A
f1 f2 f3 We are interested in modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Okā€¦ What about now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.5 0.5 0.5 1 0.5 0.5 0.5 1 3 5 1 A Wait, did he just said that f2 should be more correlated to f1 than to f3?
f1 f2 f3 We are interested in modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Okā€¦ What about now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.5 0.5 0.5 1 0.5 0.5 0.5 1 3 5 1 A Wait, did he just said that f2 should be more correlated to f1 than to f3? Arrrrā€¦.
f1 f2 f3 We are interested in modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Better now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.7 0.2 0.7 1 0.5 0.2 0.5 1 3 5 1 A
f1 f2 f3 We are interested in modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Better now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.7 0.2 0.7 1 0.5 0.2 0.5 1 3 5 1 A Yes, but what if we want to obtain this matrix automatically based on how close points are by (Z)?
f1 f2 f3 We are interested in modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Better now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.7 0.2 0.7 1 0.5 0.2 0.5 1 3 5 1 A Yes, but what if we want to obtain this matrix automatically based on how close points are by (Z)? We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj
f1 f2 f3 We are interested in modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj So now, it will become: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A Which is the same as saying: f ā‡  N(0, K)
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A Which is the same as saying: f ā‡  N(0, K) But how do we model f*?
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A Which is the same as saying: f ā‡  N(0, K) But how do we model f*? Well, probably again some kinda normalā€¦
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A Which is the same as saying: f ā‡  N(0, K) But how do we model f*? Well, probably again some kinda normalā€¦ Maybe something like: fā‡¤ ā‡  N(0, ?)
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) Maybe something like: fā‡¤ ā‡  N(0, ?)
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) Maybe something like: fā‡¤ ā‡  N(0, ?) But what is this ā€œ?ā€ covariance matrix of z* with z*?
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) Maybe something like: fā‡¤ ā‡  N(0, ?) But what is this ā€œ?ā€ covariance matrix of z* with z*? fā‡¤ ā‡  N(0, Kā‡¤ā‡¤)
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) Maybe something like: fā‡¤ ā‡  N(0, ?) But what is this ā€œ?ā€ covariance matrix of z* with z*? fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) But isnā€™t K** is just 1? Kā‡¤ā‡¤ = e ||zā‡¤ zā‡¤||2 = 1
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤)
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: What else is left? f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤)
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: What else is left? f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: What else is left? f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: What else is left? f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Only one entity is left: K1ā‡¤ = K(z1, zā‡¤)
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: What else is left? f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Only one entity is left: K1ā‡¤ = K(z1, zā‡¤) I guess we know how to calculate this one! Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Yeah! We did it!
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Yeah! We did it! Waitā€¦ but what we do now?
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Yeah! We did it! Waitā€¦ but what we do now? Rememberā€¦.
ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—† X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21
ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—† X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21 What if we substitute x1 with f* and x2 with f?
ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—† X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21 Then we can compute mean and standard deviation of f*! What if we substitute x1 with f* and x2 with f?
ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—† X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21 Then we can compute mean and standard deviation of f*! Exactly! What if we substitute x1 with f* and x2 with f?
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf )
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf )
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ ā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ zā‡¤ zā‡¤ Āµā‡¤ Āµā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ zā‡¤ zā‡¤ Āµā‡¤ Āµā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ Āµā‡¤ Āµā‡¤ zā‡¤zā‡¤ zā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
What is ? f1 f2 f3 Zz1 z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤also given fā‡¤ Āµā‡¤ Āµā‡¤ Āµā‡¤ zā‡¤zā‡¤ zā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
Pros: 1. Can model almost any function directly 3. Provides uncertainty estimates 2. Can be made more ļ¬‚exible with different kernels Cons: 1. Cannot be interpreted 2. Loose efļ¬ciency in high dimensional spaces 3. Overļ¬tting
Cat or Dog? ā€œItā€™s always seemed obvious to me that itā€™s better to know that you donā€™t know, than to think you know and act on wrong information.ā€ Katherine Bailey
Teaching statistics Doing statistics
Resources: Katherine Baileyā€™s presentation: http://katbailey.github.io/gp_talk/ Gaussian_Processes.pdf Katherine Baileyā€™s blog post: from both sides now: the math of linear regression (http://katbailey.github.io/post/from-both-sides-now-the- math-of-linear-regression/) Katherine Baileyā€™s blog post: Gaussian processes for dummies (http:// katbailey.github.io/post/gaussian-processes-for-dummies/) Kevin P. Murphyā€™s book: Machine Learning - A Probabilistic Perspective, Chapter 15 (https://www.amazon.com/Machine-Learning- Probabilistic-Perspective-Computation/dp/0262018020) Alex Bridglandā€™s blog post: Introduction to Gaussian Processes - Part I (http://bridg.land/posts/gaussian-processes-1) Nando de Freitas, Machine Learning - Introduction to Gaussian Processes (https://youtu.be/4vGiHC35j9s)
in class Under the review

More Related Content

PPT
Ch05 6
byRendy Robert
Ā 
PDF
Transformation of random variables
byTarun Gehlot
Ā 
PDF
2-D formulation Plane theory of elasticity Att 6672
byShekh Muhsen Uddin Ahmed
Ā 
PDF
Solution homework2
byJairo Roberto
Ā 
PDF
Theory of elasticity and plasticity (Equations sheet part 01) Att 8676
byShekh Muhsen Uddin Ahmed
Ā 
PDF
6.7 airy
byLex2020rio
Ā 
PDF
Simple Comparison of Convergence of GeneralIterations and Effect of Variation...
byKomal Goyal
Ā 
PDF
Stability Analysis for Steady State Solutions of Huxley Equation
byMohammad Sabawi Lecturer at Mathematics Department/College of Educations for Women/Tikrit University
Ā 
Ch05 6
byRendy Robert
Ā 
Transformation of random variables
byTarun Gehlot
Ā 
2-D formulation Plane theory of elasticity Att 6672
byShekh Muhsen Uddin Ahmed
Ā 
Solution homework2
byJairo Roberto
Ā 
Theory of elasticity and plasticity (Equations sheet part 01) Att 8676
byShekh Muhsen Uddin Ahmed
Ā 
6.7 airy
byLex2020rio
Ā 
Simple Comparison of Convergence of GeneralIterations and Effect of Variation...
byKomal Goyal
Ā 
Stability Analysis for Steady State Solutions of Huxley Equation
byMohammad Sabawi Lecturer at Mathematics Department/College of Educations for Women/Tikrit University
Ā 

What's hot

PDF
Energy principles Att 6607
byShekh Muhsen Uddin Ahmed
Ā 
PDF
5. stress function
byYASWANTH BHAIRAVABHOTLA
Ā 
PDF
An Efficient Boundary Integral Method for Stiff Fluid Interface Problems
byAlex (Oleksiy) Varfolomiyev
Ā 
PPT
Krishna
byKrishna Godugunuru
Ā 
PDF
ć‚²ćƒ¼ćƒ ē†č«–BASIC ę¼”ēæ’4 -äŗ¤ęø‰é›†åˆć‚’걂悁悋-
byssusere0a682
Ā 
PDF
On Convergence of Jungck Type Iteration for Certain Contractive Conditions
byresearchinventy
Ā 
PPT
Dyadics
bySolo Hermelin
Ā 
PDF
INFLUENCE OF OVERLAYERS ON DEPTH OF IMPLANTED-HETEROJUNCTION RECTIFIERS
byZac Darcy
Ā 
PDF
Similarity Measure Using Interval Valued Vague Sets in Multiple Criteria Deci...
byiosrjce
Ā 
PDF
Flexural analysis of thick beams using single
byiaemedu
Ā 
PPT
Vcla - Inner Products
byPreetshah1212
Ā 
PPTX
Kites team l3
byaero103
Ā 
PDF
čæ‘ä¼¼ćƒ™ć‚¤ć‚ŗč؈ē®—ć«ć‚ˆć‚‹ćƒ™ć‚¤ć‚ŗęŽØ定
byKosei ABE
Ā 
PPT
1638 vector quantities
byDr Fereidoun Dejahang
Ā 
PPT
BS2506 tutorial 1
byChamp Pairach
Ā 
PPTX
Power Series - Legendre Polynomial - Bessel's Equation
byArijitDhali
Ā 
PPT
Linreg
byzet_set_go
Ā 
PDF
Weighted Analogue of Inverse Maxwell Distribution with Applications
byPremier Publishers
Ā 
PDF
8 ashish final paper--85-90
byAlexander Decker
Ā 
PPT
2 backlash simulation
bySolo Hermelin
Ā 
Energy principles Att 6607
byShekh Muhsen Uddin Ahmed
Ā 
5. stress function
byYASWANTH BHAIRAVABHOTLA
Ā 
An Efficient Boundary Integral Method for Stiff Fluid Interface Problems
byAlex (Oleksiy) Varfolomiyev
Ā 
Krishna
byKrishna Godugunuru
Ā 
ć‚²ćƒ¼ćƒ ē†č«–BASIC ę¼”ēæ’4 -äŗ¤ęø‰é›†åˆć‚’걂悁悋-
byssusere0a682
Ā 
On Convergence of Jungck Type Iteration for Certain Contractive Conditions
byresearchinventy
Ā 
Dyadics
bySolo Hermelin
Ā 
INFLUENCE OF OVERLAYERS ON DEPTH OF IMPLANTED-HETEROJUNCTION RECTIFIERS
byZac Darcy
Ā 
Similarity Measure Using Interval Valued Vague Sets in Multiple Criteria Deci...
byiosrjce
Ā 
Flexural analysis of thick beams using single
byiaemedu
Ā 
Vcla - Inner Products
byPreetshah1212
Ā 
Kites team l3
byaero103
Ā 
čæ‘ä¼¼ćƒ™ć‚¤ć‚ŗč؈ē®—ć«ć‚ˆć‚‹ćƒ™ć‚¤ć‚ŗęŽØ定
byKosei ABE
Ā 
1638 vector quantities
byDr Fereidoun Dejahang
Ā 
BS2506 tutorial 1
byChamp Pairach
Ā 
Power Series - Legendre Polynomial - Bessel's Equation
byArijitDhali
Ā 
Linreg
byzet_set_go
Ā 
Weighted Analogue of Inverse Maxwell Distribution with Applications
byPremier Publishers
Ā 
8 ashish final paper--85-90
byAlexander Decker
Ā 
2 backlash simulation
bySolo Hermelin
Ā 

Similar to Introduction to Gaussian Processes

PPT
lecture3 (1).ppt
byengcfc192
Ā 
PDF
Machine learning (2)
byNYversity
Ā 
PPT
1609 probability function p on subspace of s
byDr Fereidoun Dejahang
Ā 
PDF
X02 Supervised learning problem linear regression multiple features
byMarco Moldenhauer
Ā 
PPT
Machine Learning and Statistical Analysis
bybutest
Ā 
PPT
Machine Learning and Statistical Analysis
bybutest
Ā 
PPT
Machine Learning and Statistical Analysis
bybutest
Ā 
PPT
Machine Learning and Statistical Analysis
bybutest
Ā 
PPT
Machine Learning and Statistical Analysis
bybutest
Ā 
PPT
Machine Learning and Statistical Analysis
bybutest
Ā 
PPT
Machine Learning and Statistical Analysis
bybutest
Ā 
PPT
Symmetrical2
bySenthil Kumar
Ā 
PDF
Machine learning (10)
byNYversity
Ā 
PDF
Curve fitting
byJULIO GONZALEZ SANZ
Ā 
DOCX
stochastic notes
bycairo university
Ā 
PPTX
MODULE-2.pptx machine learning notes for vtu 6th sem cse
byVinuthaM32
Ā 
PPTX
MODULE-3edited.pptx machine learning modulk
byVinuthaM32
Ā 
PPTX
Condition Monitoring Of Unsteadily Operating Equipment
byJordan McBain
Ā 
PDF
Probability and Statistics Cookbook
byChairat Nuchnuanrat
Ā 
PPTX
Support Vector Machines Simply
byEmad Nabil
Ā 
lecture3 (1).ppt
byengcfc192
Ā 
Machine learning (2)
byNYversity
Ā 
1609 probability function p on subspace of s
byDr Fereidoun Dejahang
Ā 
X02 Supervised learning problem linear regression multiple features
byMarco Moldenhauer
Ā 
Machine Learning and Statistical Analysis
bybutest
Ā 
Machine Learning and Statistical Analysis
bybutest
Ā 
Machine Learning and Statistical Analysis
bybutest
Ā 
Machine Learning and Statistical Analysis
bybutest
Ā 
Machine Learning and Statistical Analysis
bybutest
Ā 
Machine Learning and Statistical Analysis
bybutest
Ā 
Machine Learning and Statistical Analysis
bybutest
Ā 
Symmetrical2
bySenthil Kumar
Ā 
Machine learning (10)
byNYversity
Ā 
Curve fitting
byJULIO GONZALEZ SANZ
Ā 
stochastic notes
bycairo university
Ā 
MODULE-2.pptx machine learning notes for vtu 6th sem cse
byVinuthaM32
Ā 
MODULE-3edited.pptx machine learning modulk
byVinuthaM32
Ā 
Condition Monitoring Of Unsteadily Operating Equipment
byJordan McBain
Ā 
Probability and Statistics Cookbook
byChairat Nuchnuanrat
Ā 
Support Vector Machines Simply
byEmad Nabil
Ā 

More from Dmytro Fishman

PDF
DOME: Recommendations for supervised machine learning validation in biology
byDmytro Fishman
Ā 
PDF
Tips for effective presentations
byDmytro Fishman
Ā 
PDF
Autonomous Driving Lab - Simultaneous Localization and Mapping WP
byDmytro Fishman
Ā 
PDF
Introduction to Deep Learning
byDmytro Fishman
Ā 
PDF
Introduction to Machine Learning for Taxify/Bolt
byDmytro Fishman
Ā 
PDF
Biit group 2018
byDmytro Fishman
Ā 
PDF
Detecting Nuclei from Microscopy Images with Deep Learning
byDmytro Fishman
Ā 
PDF
Deep Learning in Healthcare
byDmytro Fishman
Ā 
PDF
5 Introduction to neural networks
byDmytro Fishman
Ā 
PDF
4 Dimensionality reduction (PCA & t-SNE)
byDmytro Fishman
Ā 
PDF
3 Unsupervised learning
byDmytro Fishman
Ā 
PDF
1 Supervised learning
byDmytro Fishman
Ā 
PDF
What does it mean to be a bioinformatician?
byDmytro Fishman
Ā 
PDF
Machine Learning in Bioinformatics
byDmytro Fishman
Ā 
DOME: Recommendations for supervised machine learning validation in biology
byDmytro Fishman
Ā 
Tips for effective presentations
byDmytro Fishman
Ā 
Autonomous Driving Lab - Simultaneous Localization and Mapping WP
byDmytro Fishman
Ā 
Introduction to Deep Learning
byDmytro Fishman
Ā 
Introduction to Machine Learning for Taxify/Bolt
byDmytro Fishman
Ā 
Biit group 2018
byDmytro Fishman
Ā 
Detecting Nuclei from Microscopy Images with Deep Learning
byDmytro Fishman
Ā 
Deep Learning in Healthcare
byDmytro Fishman
Ā 
5 Introduction to neural networks
byDmytro Fishman
Ā 
4 Dimensionality reduction (PCA & t-SNE)
byDmytro Fishman
Ā 
3 Unsupervised learning
byDmytro Fishman
Ā 
1 Supervised learning
byDmytro Fishman
Ā 
What does it mean to be a bioinformatician?
byDmytro Fishman
Ā 
Machine Learning in Bioinformatics
byDmytro Fishman
Ā 

Recently uploaded

PDF
Understanding Data Analytics: Concepts, Types, and Use Cases
byThe IoT Academy
Ā 
PDF
Unlocking the Future of Logistics & Supply Chain
byKunal Suri
Ā 
PDF
US Digital Fan Engagement Index 2026: Benchmarking Online Demand for Sports T...
byHyperset Group Ltd
Ā 
PDF
WORLD-LITERATURE-2ND-SEMESTER2025-2026.pdf
byCHERIEMAYCAMO
Ā 
PPTX
Organizational Structure and Design Topic 4
bydavenieralayaay
Ā 
PDF
Trustworthy AI : Governance of AI through Ethics
byKarim BaĆÆna
Ā 
PPTX
Top Institute for Data Analyst Course in Navi Mumbai
byPatil sagar
Ā 
PPTX
End to End Supply Chain Management- Analytics Case Study.
byNandhuNandha1
Ā 
PPTX
AI IN CYBERSECURITY Best Practices for IT
byNIKHILGR3
Ā 
PDF
AI Data Analyst for Real Estate Reporting & Insights | Leni Analytics
byLeni Analyst
Ā 
PPTX
Big Data Anaytics ..................................(3).pptx
bypriyait4
Ā 
PDF
10 Best Sites To Buy iCloud Email Accounts.pdf
bymarketing
Ā 
PPTX
Blue and Yellow Education Technology Keynote Presentation.pptx
byjahid519241
Ā 
PDF
Columbus Data and Analytics Wednesday - People Analytics 101: Making Sense of...
byTim Wilson
Ā 
PPTX
How to Find Sources Wisely for Academic Writing.pptx
byNadiaputeriramadani
Ā 
PDF
Purdue CS592_ Databases for AI_ Vector Databases (Spring 2026)
byAlkin Tezuysal
Ā 
PDF
Analysis of Union Budget 2026-27: Major Announcements and their Impact
byDeeshiPavecha
Ā 
PDF
"Streamline Project Management with Trello + Butler Automation | CA Suvidha C...
byCA Suvidha Chaplot
Ā 
PDF
chapter one introduction to Probability theory.pdf
bySeid42
Ā 
DOCX
What Are Buy Verified Apple Pay Accounts and Why Do You Need Them_.docx
byhttps://pvaallshop.com/product/buy-verified-binance-accounts/
Ā 
Understanding Data Analytics: Concepts, Types, and Use Cases
byThe IoT Academy
Ā 
Unlocking the Future of Logistics & Supply Chain
byKunal Suri
Ā 
US Digital Fan Engagement Index 2026: Benchmarking Online Demand for Sports T...
byHyperset Group Ltd
Ā 
WORLD-LITERATURE-2ND-SEMESTER2025-2026.pdf
byCHERIEMAYCAMO
Ā 
Organizational Structure and Design Topic 4
bydavenieralayaay
Ā 
Trustworthy AI : Governance of AI through Ethics
byKarim BaĆÆna
Ā 
Top Institute for Data Analyst Course in Navi Mumbai
byPatil sagar
Ā 
End to End Supply Chain Management- Analytics Case Study.
byNandhuNandha1
Ā 
AI IN CYBERSECURITY Best Practices for IT
byNIKHILGR3
Ā 
AI Data Analyst for Real Estate Reporting & Insights | Leni Analytics
byLeni Analyst
Ā 
Big Data Anaytics ..................................(3).pptx
bypriyait4
Ā 
10 Best Sites To Buy iCloud Email Accounts.pdf
bymarketing
Ā 
Blue and Yellow Education Technology Keynote Presentation.pptx
byjahid519241
Ā 
Columbus Data and Analytics Wednesday - People Analytics 101: Making Sense of...
byTim Wilson
Ā 
How to Find Sources Wisely for Academic Writing.pptx
byNadiaputeriramadani
Ā 
Purdue CS592_ Databases for AI_ Vector Databases (Spring 2026)
byAlkin Tezuysal
Ā 
Analysis of Union Budget 2026-27: Major Announcements and their Impact
byDeeshiPavecha
Ā 
"Streamline Project Management with Trello + Butler Automation | CA Suvidha C...
byCA Suvidha Chaplot
Ā 
chapter one introduction to Probability theory.pdf
bySeid42
Ā 
What Are Buy Verified Apple Pay Accounts and Why Do You Need Them_.docx
byhttps://pvaallshop.com/product/buy-verified-binance-accounts/
Ā 

Introduction to Gaussian Processes

  • 1.
  • 3.
    x f(x) y Letā€™stake a look inside
  • 4.
  • 5.
    x Let be linearfunctiony= f(x) y= āœ“0 + āœ“1x y
  • 6.
    x Let be linearfunctiony= f(x) y= āœ“0 + āœ“1x y
  • 7.
    x Let be linearfunctiony= f(x) arg min āœ“ nX i=1 (yi Ė†yi)2 Ė†yi = āœ“0 + āœ“1xi yi = āœ“0 + āœ“1xi + āœi arg min āœ“ nX i=1 (yi Ė†yi)2 Ė†yi = āœ“0 + āœ“1xi yi = āœ“0 + āœ“1xi + āœi Find and by optimising error arg min āœ“ nX i=1 (yi Ė†yi)2 Ė†yi = āœ“0 + āœ“1xi y = āœ“ + āœ“ x + āœ arg min āœ“ nX i=1 (yi Ė†yi)2 Ė†yi = āœ“0 + āœ“1xi y = āœ“ + āœ“ x + āœ y= āœ“0 + āœ“1x y
  • 8.
    x But if datais not linear? y
  • 9.
    x But if datais not linear? y= āœ“0 + āœ“1x y
  • 10.
    x y= āœ“0 +āœ“1x + āœ“2x2 But if data is not linear? y
  • 11.
    x But if datais not linear? y= āœ“0 + āœ“1x + āœ“2x2 + āœ“3x3 y
  • 12.
    x What if donā€™twant to assume a speciļ¬c form? y
  • 13.
    x GPs let youmodel any function directly y
  • 14.
    x y x y Parametric ML NonparametricML A learning model that summarizes data with a set of parameters of ļ¬xed size (independent of the number of training examples) is called a parametric model. Algorithms that do not make strong assumptions about the form of the mapping function are called nonparametric machine learning algorithms. y= āœ“0 + āœ“1x
  • 15.
    x y x y Parametric ML NonparametricML A learning model that summarizes data with a set of parameters of ļ¬xed size (independent of the number of training examples) is called a parametric model. Algorithms that do not make strong assumptions about the form of the mapping function are called nonparametric machine learning algorithms. y= āœ“0 + āœ“1x Question: is K-nearest neighbour parametric or nonparametric algorithm according to these deļ¬nitions?
  • 16.
    x GPs let youmodel any function directly y
  • 17.
    x y GPs let youmodel any function directly estimates the uncertainty for each new prediction
  • 18.
    x y If I askyou to predict for xi xiyi
  • 19.
    x y If I askyou to predict for ? xiyi You better be very uncertain xi
  • 20.
    How is iteven possible?
  • 21.
    We will need Normaldistribution x xi y
  • 22.
    Āµ 1p 2ā‡” e (x Āµ)2 2 2 Withaverage coordinate and standard deviation from centre Āµ Many important processes follow normal distribution
  • 23.
    Āµ 1p 2ā‡” e (x Āµ)2 2 2N(Āµ,2 ) With average coordinate and standard deviation from centre Āµ Many important processes follow normal distribution
  • 24.
    X1 ā‡  N(Āµ1,2 1)1p 2ā‡” e (x Āµ)2 2 2 With average coordinate and standard deviation from centre Āµ Many important processes follow normal distribution Āµ1 1
  • 25.
    1p 2ā‡” e (x Āµ)2 2 2 WhatIf I draw another distribution? With average coordinate and standard deviation from centre Āµ Many important processes follow normal distribution X1 ā‡  N(Āµ1, 2 1) Āµ1 1
  • 26.
    X1 ā‡  N(Āµ1,2 1) X2 ā‡  N(Āµ2, 2 2) Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1 X1 X20 0
  • 27.
    X1 X20 0 X2ā‡  N(0, 1)X1 ā‡  N(0, 1) Āµ1 = 0 1 = 1 Āµ2 = 0 2 = 1
  • 28.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1)
  • 29.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1)
  • 30.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1)
  • 31.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1)
  • 32.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 X1 X20 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) What if we would join them into one plot?
  • 33.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) X2 X1
  • 34.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) X2 X1
  • 35.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) X2 X1
  • 36.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) X2 X1
  • 37.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) X2 X1
  • 38.
    X2 X1 Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 X = ļ£æ x1 x2 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1)
  • 39.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 X2 ā‡  N(0, 1)X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X2 X1
  • 40.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 X2 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X1 ā‡  N(0, 1) X2 X1
  • 41.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 X2 ā‡  N(0, 1) Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 X1
  • 42.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1
  • 43.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1
  • 44.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 0 M = ļ£æ Āµ1 Āµ2 Joint distribution of variables and X = ļ£æ x1 x2 x1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1 Covariance matrix or āŒƒ
  • 45.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 Joint distribution of variables and 0 x1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1
  • 46.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 Joint distribution of variables andx1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1
  • 47.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 Joint distribution of variables andx1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1 Similarity
  • 48.
    ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 01 ā—† X10 0 X2 X10 0 X2 ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Positive value of does not tell much about X1 X2 Some similarity (correlation) Positive value of with good probability means positive X1 X2 No similarity (no correlation)
  • 49.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 Joint distribution of variables andx1 x2 X1 ā‡  N(0, 1) ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† X2 ā‡  N(0, 1) X2 X1
  • 50.
    Āµ1 = 01 = 1 Āµ2 = 0 2 = 1 Joint distribution of variables and 0 x1 x2 X1 ā‡  N(0, 1) X2 ā‡  N(0, 1) X2 X1 ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—†
  • 51.
    X2 X1 Joint distribution ofvariables andx1 x2 P(x1, x2) ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—†
  • 52.
    X2 X1 Joint distribution ofvariables andx1 x2 P(x1, x2) x2 ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—†
  • 53.
    X2 X1 Joint distribution ofvariables andx1 x2 P(x1, x2) x2 ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—†
  • 54.
    X2 X1 Joint distribution ofvariables andx1 x2 P(x1, x2) x2 1|2 Āµ1|2 ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—†
  • 55.
    X2 X1 Joint distribution ofvariables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 11 12 21 22 ā—†
  • 56.
    ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 1112 21 22 ā—† X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21
  • 57.
    X2 X1 x2 1|2 Āµ1|2 ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 1112 21 22 ā—† Conditional distribution P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21 P(x1, x2) Joint distribution ļ£æ x1 x2 ā‡  N āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† X10 0 X2 N(Āµ, 2 ) Normal distribution or 1D Gaussian or 2D Gaussian
  • 58.
  • 59.
  • 60.
  • 61.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13) Sampling Samples from 2D Gaussian 0 1st 2nd 1 1
  • 62.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13) Sampling Samples from 2D Gaussian 0 1st 2nd 1 1
  • 63.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65)
  • 64.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65)
  • 65.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65)
  • 66.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) There is little dependency
  • 67.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† There is little dependency
  • 68.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) There is little dependency Samples from 2D Gaussian
  • 69.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) There is little dependency 0 1st 2nd 1 1 Samples from 2D Gaussian
  • 70.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) There is little dependency 0 1st 2nd 1 1 Samples from 2D Gaussian
  • 71.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) Samples from 2D Gaussian There is little dependency 0 1st 2nd 1 1 (-0.03,-0.24)
  • 72.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) Samples from 2D Gaussian There is little dependency 0 1st 2nd 1 1 (-0.03,-0.24)
  • 73.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) Samples from 2D Gaussian There is little dependency 0 1st 2nd 1 1 (-0.03,-0.24)
  • 74.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) Samples from 2D Gaussian There is little dependency 0 1st 2nd 1 1 (-0.03,-0.24) More dependent values
  • 75.
    2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0 0 1 ā—† (-0.23,1.13)Sampling Samples from 2D Gaussian 0 1st 2nd 1 1 (-1.14, 0.65) 2D Gaussian ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Sampling (0.13,0.52) Samples from 2D Gaussian There is little dependency 0 1st 2nd 1 1 (-0.03,-0.24) More dependent values How would a sample from 20D Gaussian look like?
  • 76.
  • 77.
  • 78.
    20D Gaussian 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0 0. . . 0 0 1 0 . . . 0 ... ... ... ... ... 0 0 0 . . . 1 3 7 7 7 5 1 C C C A Sampling (0.73, -0.12, 0.42, 1.2,ā€¦, 16 more) 0 1st 2nd 1 1 3rd 4th 5th 6th 7th
  • 79.
    20D Gaussian Letā€™s add moredependency between points 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A
  • 80.
    20D Gaussian Letā€™s add moredependency between points (0.73, 0.18, 0.68, -0.2,ā€¦, 16 more) 0 1st 2nd 1 1 3rd 4th 5th 6th 7th 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A
  • 81.
    20D Gaussian Letā€™s add moredependency between points 0 1st 2nd 1 1 3rd 4th 5th 6th 7th We want some notion of smoothness between pointsā€¦ 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A
  • 82.
    20D Gaussian Letā€™s add moredependency between points 0 1st 2nd 1 1 3rd 4th 5th 6th 7th We want some notion of smoothness between pointsā€¦ So that dependancy between 1st and 2nd points is larger than between 1st and the 3rd. 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A
  • 83.
    20D Gaussian Letā€™s add moredependency between points We want some notion of smoothness between pointsā€¦ So that dependancy between 1st and 2nd points is larger than between 1st and the 3rd. 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A We might have just increased corresponding values in covariance matrix, right?
  • 84.
    20D Gaussian Letā€™s add moredependency between points We want some notion of smoothness between pointsā€¦ So that dependancy between 1st and 2nd points is larger than between 1st and the 3rd. 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 1 0.5 0.5 . . . 0.5 0.5 1 0.5 . . . 0.5 ... ... ... ... ... 0.5 0.5 0.5 . . . 1 3 7 7 7 5 1 C C C A We might have just increased corresponding values in covariance matrix, right? We need a way to generate a ā€œsmoothā€ covariance matrix automatically depending on the distance between points
  • 85.
    We will usea similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 20D Gaussian
  • 86.
    We will usea similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 20D Gaussian 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K120 K21 K22 K23 . . . K220 ... ... ... ... ... K201 K202 K203 . . . K2020 3 7 7 7 5 1 C C C A 0 1st 2nd 1 1 3rd 4th 5th 6th 7th
  • 87.
    We will usea similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
  • 88.
    We will usea similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
  • 89.
    We will usea similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
  • 90.
    We will usea similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
  • 91.
    We will usea similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
  • 92.
    We will usea similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 Āµā‡¤ ā‡¤ 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
  • 93.
    We will usea similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj 200D Gaussian 0 1 1 Āµā‡¤ ā‡¤ 0 B B B @ 2 6 6 6 4 0 0 ... 0 3 7 7 7 5 2 6 6 6 4 K11 K12 K13 . . . K1200 K21 K22 K23 . . . K2200 ... ... ... ... ... K2001 K2002 K2003 . . . K200200 3 7 7 7 5 1 C C C A
  • 94.
    f1 f2 f3 We are interestedin modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3
  • 95.
    f1 f2 f3 We are interestedin modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Previously we were using: to generate correlated points, can we do it again here? ļ£æ f1 f2 ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—†
  • 96.
    f1 f2 f3 We are interestedin modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Wait! But now we have three points, we cannot use the same formula! ļ£æ f1 f2 ā‡  āœ“ļ£æ 0 0 ļ£æ 1 0.5 0.5 1 ā—† Previously we were using: to generate correlated points, can we do it again here?
  • 97.
    f1 f2 f3 We are interestedin modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Okā€¦ What about now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.5 0.5 0.5 1 0.5 0.5 0.5 1 3 5 1 A
  • 98.
    f1 f2 f3 We are interestedin modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Okā€¦ What about now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.5 0.5 0.5 1 0.5 0.5 0.5 1 3 5 1 A Wait, did he just said that f2 should be more correlated to f1 than to f3?
  • 99.
    f1 f2 f3 We are interestedin modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Okā€¦ What about now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.5 0.5 0.5 1 0.5 0.5 0.5 1 3 5 1 A Wait, did he just said that f2 should be more correlated to f1 than to f3? Arrrrā€¦.
  • 100.
    f1 f2 f3 We are interestedin modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Better now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.7 0.2 0.7 1 0.5 0.2 0.5 1 3 5 1 A
  • 101.
    f1 f2 f3 We are interestedin modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Better now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.7 0.2 0.7 1 0.5 0.2 0.5 1 3 5 1 A Yes, but what if we want to obtain this matrix automatically based on how close points are by (Z)?
  • 102.
    f1 f2 f3 We are interestedin modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 Better now? 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 2 4 1 0.7 0.2 0.7 1 0.5 0.2 0.5 1 3 5 1 A Yes, but what if we want to obtain this matrix automatically based on how close points are by (Z)? We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj
  • 103.
    f1 f2 f3 We are interestedin modelling for given Z Z So that is more correlated with than z1 z2 z3 F(z) F(z) f1f2 f3 We will use a similarity measure Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj So now, it will become: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A
  • 104.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤
  • 105.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤
  • 106.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤
  • 107.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A
  • 108.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A Which is the same as saying: f ā‡  N(0, K)
  • 109.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A Which is the same as saying: f ā‡  N(0, K) But how do we model f*?
  • 110.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A Which is the same as saying: f ā‡  N(0, K) But how do we model f*? Well, probably again some kinda normalā€¦
  • 111.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: 2 4 f1 f2 f3 3 5 ā‡  0 @ 2 4 0 0 0 3 5 , 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 1 A Which is the same as saying: f ā‡  N(0, K) But how do we model f*? Well, probably again some kinda normalā€¦ Maybe something like: fā‡¤ ā‡  N(0, ?)
  • 112.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) Maybe something like: fā‡¤ ā‡  N(0, ?)
  • 113.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) Maybe something like: fā‡¤ ā‡  N(0, ?) But what is this ā€œ?ā€ covariance matrix of z* with z*?
  • 114.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) Maybe something like: fā‡¤ ā‡  N(0, ?) But what is this ā€œ?ā€ covariance matrix of z* with z*? fā‡¤ ā‡  N(0, Kā‡¤ā‡¤)
  • 115.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) Maybe something like: fā‡¤ ā‡  N(0, ?) But what is this ā€œ?ā€ covariance matrix of z* with z*? fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) But isnā€™t K** is just 1? Kā‡¤ā‡¤ = e ||zā‡¤ zā‡¤||2 = 1
  • 116.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤)
  • 117.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: What else is left? f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤)
  • 118.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: What else is left? f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A
  • 119.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: What else is left? f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1
  • 120.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: What else is left? f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Only one entity is left: K1ā‡¤ = K(z1, zā‡¤)
  • 121.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: What else is left? f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Only one entity is left: K1ā‡¤ = K(z1, zā‡¤) I guess we know how to calculate this one! Kij = e ||zi zj ||2 = ( 0, ||zi zj|| ! 1 1, zi = zj
  • 122.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Yeah! We did it!
  • 123.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Yeah! We did it! Waitā€¦ but what we do now?
  • 124.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Yeah! We did it! Waitā€¦ but what we do now? Rememberā€¦.
  • 125.
    ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 1112 21 22 ā—† X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21
  • 126.
    ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 1112 21 22 ā—† X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21 What if we substitute x1 with f* and x2 with f?
  • 127.
    ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 1112 21 22 ā—† X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21 Then we can compute mean and standard deviation of f*! What if we substitute x1 with f* and x2 with f?
  • 128.
    ļ£æ x1 x2 i ā‡  N āœ“ļ£æ Āµ1 Āµ2 ļ£æ 1112 21 22 ā—† X2 X1 Joint distribution of variables andx1 x2 P(x1, x2) x2 Conditional distribution 1|2 Āµ1|2 P(x1|x2) = N(x1|Āµ1|2, 1|2) Āµ1|2 = Āµ1 + 12 + T 22(x2 Āµ2) 1|2 = 11 12 T 22 21 Then we can compute mean and standard deviation of f*! Exactly! What if we substitute x1 with f* and x2 with f?
  • 129.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf )
  • 130.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf )
  • 131.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ ā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
  • 132.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
  • 133.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ zā‡¤ zā‡¤ Āµā‡¤ Āµā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
  • 134.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤ zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ zā‡¤ zā‡¤ Āµā‡¤ Āµā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
  • 135.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤also given fā‡¤ Ok, so we have just modelled f: f ā‡  N(0, K) and fā‡¤ ā‡  N(0, Kā‡¤ā‡¤) ļ£æ f fā‡¤ ā‡  0 B B B B @ ļ£æ 0 0 2 6 6 6 6 4 2 4 K11 K12 K13 K21 K22 K23 K31 K32 K33 3 5 2 4 K1ā‡¤ K2ā‡¤ K3ā‡¤ 3 5 ā‡„ Kā‡¤1 Kā‡¤2 Kā‡¤3 ā‡¤ [Kā‡¤ā‡¤] 3 7 7 7 7 5 1 C C C C A K 1 Kiā‡¤ Kā‡¤i Āµā‡¤ Āµā‡¤ Āµā‡¤ zā‡¤zā‡¤ zā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
  • 136.
    What is ? f1 f2 f3 Zz1z2 z3 F(z) Given: {(f1, z1); (f2, z2); (f3, z3)} zā‡¤also given fā‡¤ Āµā‡¤ Āµā‡¤ Āµā‡¤ zā‡¤zā‡¤ zā‡¤ Āµā‡¤= Āµ(zā‡¤) + KT ā‡¤ K 1 (f Āµf ) ā‡¤ = Kā‡¤ā‡¤ KT ā‡¤ K 1 Kā‡¤
  • 137.
    Pros: 1. Can modelalmost any function directly 3. Provides uncertainty estimates 2. Can be made more ļ¬‚exible with different kernels Cons: 1. Cannot be interpreted 2. Loose efļ¬ciency in high dimensional spaces 3. Overļ¬tting
  • 138.
    Cat or Dog? ā€œItā€™salways seemed obvious to me that itā€™s better to know that you donā€™t know, than to think you know and act on wrong information.ā€ Katherine Bailey
  • 139.
  • 140.
    Resources: Katherine Baileyā€™s presentation:http://katbailey.github.io/gp_talk/ Gaussian_Processes.pdf Katherine Baileyā€™s blog post: from both sides now: the math of linear regression (http://katbailey.github.io/post/from-both-sides-now-the- math-of-linear-regression/) Katherine Baileyā€™s blog post: Gaussian processes for dummies (http:// katbailey.github.io/post/gaussian-processes-for-dummies/) Kevin P. Murphyā€™s book: Machine Learning - A Probabilistic Perspective, Chapter 15 (https://www.amazon.com/Machine-Learning- Probabilistic-Perspective-Computation/dp/0262018020) Alex Bridglandā€™s blog post: Introduction to Gaussian Processes - Part I (http://bridg.land/posts/gaussian-processes-1) Nando de Freitas, Machine Learning - Introduction to Gaussian Processes (https://youtu.be/4vGiHC35j9s)
  • 141.