Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PCA (Principal Component Analysis)

15 views

Published on

Accompanying slides for the PCA video:
https://www.youtube.com/watch?v=g-Hb26agBFg

Published in: Technology
  • Be the first to comment

  • Be the first to like this

PCA (Principal Component Analysis)

  1. 1. Principal Component Analysis (PCA)
  2. 2. Principal Component Analysis (PCA)
  3. 3. Principal Component Analysis (PCA) Luis Serrano
  4. 4. 1. Projections
  5. 5. Taking a picture
  6. 6. Taking a picture
  7. 7. Taking a picture
  8. 8. Taking a picture
  9. 9. Taking a picture
  10. 10. Taking a picture
  11. 11. Dimensionality Reduction
  12. 12. Dimensionality Reduction
  13. 13. Dimensionality Reduction
  14. 14. Dimensionality Reduction
  15. 15. Dimensionality Reduction
  16. 16. Dimensionality Reduction
  17. 17. Dimensionality Reduction
  18. 18. Dimensionality Reduction
  19. 19. Dimensionality Reduction
  20. 20. Dimensionality Reduction
  21. 21. Dimensionality Reduction
  22. 22. 2. Application to housing
  23. 23. Housing Data
  24. 24. Housing Data Size
  25. 25. Housing Data Size Number of rooms
  26. 26. Housing Data Size Number of rooms Number of bathrooms
  27. 27. Housing Data Size Number of rooms Number of bathrooms Schools around
  28. 28. Housing Data Size Number of rooms Number of bathrooms Schools around Crime rate
  29. 29. Housing Data Size Number of rooms Number of bathrooms Schools around Crime rate
  30. 30. Housing Data Size Number of rooms Number of bathrooms Size feature Schools around Crime rate
  31. 31. Housing Data Size Number of rooms Number of bathrooms Size feature Schools around Crime rate Location feature
  32. 32. 100 200 300 400 1 2 3 4 Number of Rooms Size 5
  33. 33. 100 200 300 400 1 2 3 4 Number of Rooms Size 5
  34. 34. 100 200 300 400 1 2 3 4 Number of Rooms Size 5
  35. 35. 100 200 300 400 1 2 3 4 Number of Rooms Size 5
  36. 36. Size feature
  37. 37. 2 dimensions
  38. 38. 2 dimensions 1 dimension
  39. 39. 2 dimensions 1 dimension size number of rooms
  40. 40. 2 dimensions 1 dimension size number of rooms size feature
  41. 41. Housing Data
  42. 42. Housing Data 5 dimensions
  43. 43. Housing Data 5 dimensions 2 dimensions
  44. 44. Housing Data Size Number of rooms Number of bathrooms Schools around Crime rate 5 dimensions 2 dimensions
  45. 45. Housing Data Size Number of rooms Number of bathrooms Size feature Schools around Crime rate 5 dimensions 2 dimensions
  46. 46. Housing Data Size Number of rooms Number of bathrooms Size feature Schools around Crime rate Location feature 5 dimensions 2 dimensions
  47. 47. 3. Mean, Variance, Covariance
  48. 48. Mean
  49. 49. Mean
  50. 50. Mean
  51. 51. Mean wall
  52. 52. Mean 1 2 3 4 5 6 wall
  53. 53. Mean 1 2 3 4 5 6 1+2+6 wall
  54. 54. Mean 1 2 3 4 5 6 1+2+6 wall Mean =
  55. 55. Mean 1 2 3 4 5 6 1+2+6 wall Mean =
  56. 56. Mean 1 2 3 4 5 6 1+2+6 3wall Mean =
  57. 57. Mean 1 2 3 4 5 6 1+2+6 3 = 3 wall Mean =
  58. 58. Mean 1 2 3 4 5 6 1+2+6 3 = 3 wall Mean =
  59. 59. Variance
  60. 60. Variance
  61. 61. Variance
  62. 62. Variance
  63. 63. Variance
  64. 64. Variance 1 1
  65. 65. Variance 1 1 5 5
  66. 66. Variance 1 1 5 5 Variance =
  67. 67. Variance 1 1 5 5 12 +02 +12 Variance =
  68. 68. Variance 1 1 5 5 12 +02 +12 Variance =
  69. 69. Variance 1 1 5 5 12 +02 +12 3 Variance =
  70. 70. Variance 1 1 5 5 12 +02 +12 3 = 2/3Variance =
  71. 71. Variance 1 1 5 5 12 +02 +12 3 = 2/3Variance = Variance =
  72. 72. Variance 1 1 5 5 12 +02 +12 3 = 2/3Variance = 52 +02 +52 Variance =
  73. 73. Variance 1 1 5 5 12 +02 +12 3 = 2/3Variance = 52 +02 +52 Variance =
  74. 74. Variance 1 1 5 5 12 +02 +12 3 = 2/3Variance = 52 +02 +52 3 Variance =
  75. 75. Variance 1 1 5 5 12 +02 +12 3 = 2/3Variance = 52 +02 +52 3 = 10/3Variance =
  76. 76. Mean 1 2 3 4 5 6
  77. 77. Mean 2 1 2 3 4 5 6
  78. 78. Mean 2 1 1 2 3 4 5 6
  79. 79. Mean 2 1 3 1 2 3 4 5 6
  80. 80. Mean 2 1 3 1 2 3 4 5 6 Variance =
  81. 81. Mean 2 1 3 1 2 3 4 5 6 22 +12 +32 Variance =
  82. 82. Mean 2 1 3 1 2 3 4 5 6 22 +12 +32 Variance =
  83. 83. Mean 2 1 3 1 2 3 4 5 6 22 +12 +32 3 Variance =
  84. 84. Mean 2 1 3 1 2 3 4 5 6 22 +12 +32 3 = 14/3Variance =
  85. 85. Variance?
  86. 86. Variance?
  87. 87. Variance?
  88. 88. Variance?
  89. 89. Variance?
  90. 90. Variance? x-variance
  91. 91. Variance? x-variance
  92. 92. Variance? x-variance y-variance
  93. 93. Variance?
  94. 94. Variance?
  95. 95. Variance?
  96. 96. Variance?
  97. 97. Variance? 22 22
  98. 98. Variance? x-variance = 22 +02 +22 3 = 8/3 22 22
  99. 99. Variance? x-variance = 22 +02 +22 3 = 8/3 22 22
  100. 100. Variance? x-variance = 22 +02 +22 3 = 8/3 22 22 1 1 1 1
  101. 101. Variance? x-variance = 22 +02 +22 3 = 8/3 y-variance = 12 +02 +12 3 = 2/3 22 22 1 1 1 1
  102. 102. Covariance (0,0) (2,1) (2,-1)(-2,-1) (-2,1)
  103. 103. Covariance (0,0) (2,1) (2,-1)(-2,-1) (-2,1) Product of coordinates
  104. 104. Covariance (0,0) (2,1) (2,-1)(-2,-1) (-2,1) 0 Product of coordinates
  105. 105. Covariance (0,0) (2,1) (2,-1)(-2,-1) (-2,1) +2 0 +2 Product of coordinates
  106. 106. Covariance (0,0) (2,1) (2,-1)(-2,-1) (-2,1) +2 -2 0 -2 +2 Product of coordinates
  107. 107. Covariance (0,0) (2,1) (2,-1)(-2,-1) (-2,1) +2 -2 0 -2 +2 Product of coordinates
  108. 108. Covariance (0,0) (2,1) (2,-1)(-2,-1) (-2,1) +2 -2 0 -2 +2 Product of coordinates
  109. 109. Covariance
  110. 110. Covariance (-2,1) (2,-1) -2 -2 (0,0) 0
  111. 111. Covariance (-2,1) (2,-1) -2 -2 (0,0) 0 (2,1) (-2,-1) 2 2(0,0) 0
  112. 112. Covariance covariance = (-2) + 0 + (-2) 3 = -4/3 (-2,1) (2,-1) -2 -2 (0,0) 0 (2,1) (-2,-1) 2 2(0,0) 0
  113. 113. Covariance covariance = (-2) + 0 + (-2) 3 = -4/3 covariance = 2 + 0 + 2 3 = 4/3 (-2,1) (2,-1) -2 -2 (0,0) 0 (2,1) (-2,-1) 2 2(0,0) 0
  114. 114. Covariance (0,0) (2,0) (2,1) (2,-1)(0,-1) (0,1)(-2,1) (-2,0) (-2,-1)
  115. 115. Covariance (0,0) (2,0) (2,1) (2,-1)(0,-1) (0,1)(-2,1) (-2,0) (-2,-1) -2 2 0 0 0 00 -22
  116. 116. Covariance covariance = (0,0) (2,0) (2,1) (2,-1)(0,-1) (0,1)(-2,1) (-2,0) (-2,-1) -2 2 0 0 0 00 -22
  117. 117. Covariance covariance = + + + + + + + + 9 (0,0) (2,0) (2,1) (2,-1)(0,-1) (0,1)(-2,1) (-2,0) (-2,-1) -2 2 00 000 -22
  118. 118. Covariance covariance = + + + + + + + + 9 = 0 (0,0) (2,0) (2,1) (2,-1)(0,-1) (0,1)(-2,1) (-2,0) (-2,-1) -2 2 00 000 -22
  119. 119. Covariance
  120. 120. Covariance
  121. 121. Covariance
  122. 122. Covariance
  123. 123. Covariance negative covariance
  124. 124. Covariance negative covariance covariance zero (or very small)
  125. 125. Covariance negative covariance covariance zero (or very small) positive covariance
  126. 126. 4. Eigenvectors and Eigenvalues
  127. 127. Covariance matrix
  128. 128. Covariance matrix
  129. 129. Covariance matrix Var(X)
  130. 130. Covariance matrix Var(X) Var(Y)
  131. 131. Covariance matrix Var(X) Var(Y) Cov(X,Y) Cov(X,Y)
  132. 132. Covariance matrix Var(X) Var(Y) Cov(X,Y) Cov(X,Y) =
  133. 133. Covariance matrix Var(X) Var(Y) Cov(X,Y) Cov(X,Y) = 9 3 4 4
  134. 134. Linear Transformations
  135. 135. Linear Transformations 9 3 4 4
  136. 136. Linear Transformations 9 3 4 4
  137. 137. Linear Transformations 9 3 4 4
  138. 138. Linear Transformations 9 3 4 4
  139. 139. Linear Transformations 9 3 4 4 (x, y)
  140. 140. Linear Transformations 9 3 4 4 (x, y)
  141. 141. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y)
  142. 142. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y)
  143. 143. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y)
  144. 144. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y)
  145. 145. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y)
  146. 146. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y)
  147. 147. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0)
  148. 148. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0)
  149. 149. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0)
  150. 150. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0)
  151. 151. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0)
  152. 152. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0)
  153. 153. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (9,4)
  154. 154. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (9,4)
  155. 155. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (9,4)
  156. 156. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (9,4)
  157. 157. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (9,4) (4,3)
  158. 158. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (9,4) (4,3)
  159. 159. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (-1,0) (9,4) (4,3)
  160. 160. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (-1,0) (9,4) (4,3)
  161. 161. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (-1,0) (9,4) (4,3) (-9,-4)
  162. 162. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (-1,0) (9,4) (4,3) (-9,-4)
  163. 163. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (-1,0) (0,-1) (9,4) (4,3) (-9,-4)
  164. 164. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (-1,0) (0,-1) (9,4) (4,3) (-9,-4)
  165. 165. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (-1,0) (0,-1) (9,4) (4,3) (-9,-4) (-4,-3)
  166. 166. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (-1,0) (0,-1) (9,4) (4,3) (-9,-4) (-4,-3)
  167. 167. Linear Transformations 9 3 4 4 (x, y) (9x+4y, 4x+3y) (0,0) (0,0) (1,0) (0,1) (-1,0) (0,-1) (9,4) (4,3) (-9,-4) (-4,-3)
  168. 168. Linear Transformations 9 3 4 4
  169. 169. Linear Transformations 9 3 4 4
  170. 170. Linear Transformations 9 3 4 4
  171. 171. Linear Transformations 9 3 4 4
  172. 172. Linear Transformations 9 3 4 4 11
  173. 173. Linear Transformations 9 3 4 4 11
  174. 174. Linear Transformations 9 3 4 4 11
  175. 175. Linear Transformations 9 3 4 4 11
  176. 176. Linear Transformations 9 3 4 4 11 1
  177. 177. Linear Transformations 9 3 4 4 2 1 11 1
  178. 178. Linear Transformations 9 3 4 4 2 1 -1 2 11 1
  179. 179. Linear Transformations 9 3 4 4 2 1 -1 2 Eigenvectors (direction) 11 1
  180. 180. Linear Transformations 9 3 4 4 2 1 -1 2 11 Eigenvectors (direction) 11 1
  181. 181. Linear Transformations 9 3 4 4 2 1 -1 2 11 1 Eigenvectors (direction) 11 1
  182. 182. Linear Transformations 9 3 4 4 2 1 -1 2 11 1 Eigenvectors (direction) Eigenvalues (magnitude) 11 1
  183. 183. Linear Transformations
  184. 184. Linear Transformations Eigenvectors
  185. 185. Linear Transformations Eigenvectors Eigenvalues
  186. 186. Linear Transformations 2 1 Eigenvectors Eigenvalues
  187. 187. Linear Transformations 2 1 11 Eigenvectors Eigenvalues
  188. 188. Linear Transformations 2 1 11 Eigenvectors Eigenvalues
  189. 189. Linear Transformations 2 1 -1 2 11 Eigenvectors Eigenvalues
  190. 190. Linear Transformations 2 1 -1 2 11 1 Eigenvectors Eigenvalues
  191. 191. Linear Transformations 2 1 -1 2 11 1 Eigenvectors Eigenvalues
  192. 192. Linear Transformations 2 1 -1 2 11 1 Eigenvectors Eigenvalues
  193. 193. Eigenstuff
  194. 194. Eigenstuff
  195. 195. Eigenstuff
  196. 196. Eigenstuff
  197. 197. Eigenstuff
  198. 198. Eigenstuff
  199. 199. Eigenstuff
  200. 200. Eigenstuff 9 3 4 4
  201. 201. Eigenstuff 9 3 4 4 v
  202. 202. Eigenstuff 9 3 4 4 v =
  203. 203. Eigenstuff 9 3 4 4 v =
  204. 204. Eigenstuff 9 3 4 4 v = v
  205. 205. Eigenstuff 9 3 4 4 v = v Eigenvector
  206. 206. Eigenstuff 9 3 4 4 v = v Eigenvector Eigenvalue
  207. 207. Eigenvalues 9 3 4 4
  208. 208. Eigenvalues Characteristic Polynomial 9 3 4 4
  209. 209. Eigenvalues Characteristic Polynomial x-9 x-3 -4 -4 9 3 4 4
  210. 210. Eigenvalues Characteristic Polynomial x-9 x-3 -4 -4 = (x-9)(x-3) - (-4)(-4) 9 3 4 4
  211. 211. Eigenvalues Characteristic Polynomial x-9 x-3 -4 -4 = (x-9)(x-3) - (-4)(-4) = x2 - 12x + 11 9 3 4 4
  212. 212. Eigenvalues Characteristic Polynomial x-9 x-3 -4 -4 = (x-9)(x-3) - (-4)(-4) = x2 - 12x + 11 (x-11)(x-1)= 9 3 4 4
  213. 213. Eigenvalues Characteristic Polynomial x-9 x-3 -4 -4 = (x-9)(x-3) - (-4)(-4) = x2 - 12x + 11 (x-11)(x-1)= Eigenvalues 9 3 4 4
  214. 214. Eigenvalues Characteristic Polynomial x-9 x-3 -4 -4 = (x-9)(x-3) - (-4)(-4) = x2 - 12x + 11 (x-11)(x-1)= Eigenvalues 11 9 3 4 4
  215. 215. Eigenvalues Characteristic Polynomial x-9 x-3 -4 -4 = (x-9)(x-3) - (-4)(-4) = x2 - 12x + 11 (x-11)(x-1)= Eigenvalues 11 and 9 3 4 4
  216. 216. Eigenvalues Characteristic Polynomial x-9 x-3 -4 -4 = (x-9)(x-3) - (-4)(-4) = x2 - 12x + 11 (x-11)(x-1)= Eigenvalues 11 1and 9 3 4 4
  217. 217. Eigenvectors
  218. 218. Eigenvectors 9 3 4 4
  219. 219. Eigenvectors 9 3 4 4 u v
  220. 220. Eigenvectors 9 3 4 4 u v =
  221. 221. Eigenvectors 9 3 4 4 u v = 11
  222. 222. Eigenvectors 9 3 4 4 u v = u v 11
  223. 223. Eigenvectors 9 3 4 4 u v = u v 11 9 3 4 4
  224. 224. Eigenvectors 9 3 4 4 u v = u v 11 9 3 4 4 u v
  225. 225. Eigenvectors 9 3 4 4 u v = u v 11 9 3 4 4 u v =
  226. 226. Eigenvectors 9 3 4 4 u v = u v 11 9 3 4 4 u v = 1
  227. 227. Eigenvectors 9 3 4 4 u v = u v 11 9 3 4 4 u v = u v 1
  228. 228. Eigenvectors 9 3 4 4 u v = u v 11 9 3 4 4 u v = u v 1 2 1 u v =
  229. 229. Eigenvectors 9 3 4 4 u v = u v 11 9 3 4 4 u v = u v 1 2 1 u v = -1 2 u v =
  230. 230. 3blue1brown
  231. 231. 4. PCA
  232. 232. Principal Component Analysis (PCA)
  233. 233. Principal Component Analysis (PCA)
  234. 234. Principal Component Analysis (PCA) =
  235. 235. Principal Component Analysis (PCA) 9 =
  236. 236. Principal Component Analysis (PCA) 9 3 =
  237. 237. Principal Component Analysis (PCA) 9 3 4 4 =
  238. 238. Principal Component Analysis (PCA) Eigenvectors 9 3 4 4 =
  239. 239. Principal Component Analysis (PCA) Eigenvectors 9 3 4 4 = (direction)
  240. 240. Principal Component Analysis (PCA) Eigenvectors 9 3 4 4 = Eigenvalues (direction)
  241. 241. Principal Component Analysis (PCA) Eigenvectors 9 3 4 4 = Eigenvalues (direction) (magnitude)
  242. 242. Principal Component Analysis (PCA) 2 1 Eigenvectors 9 3 4 4 = Eigenvalues (direction) (magnitude)
  243. 243. Principal Component Analysis (PCA) 2 1 Eigenvectors 9 3 4 4 = 11 Eigenvalues (direction) (magnitude)
  244. 244. Principal Component Analysis (PCA) 2 1 Eigenvectors 9 3 4 4 = 11 Eigenvalues (direction) (magnitude)
  245. 245. Principal Component Analysis (PCA) 2 1 -1 2 Eigenvectors 9 3 4 4 = 11 Eigenvalues (direction) (magnitude)
  246. 246. Principal Component Analysis (PCA) 2 1 -1 2 Eigenvectors 9 3 4 4 = 11 1 Eigenvalues (direction) (magnitude)
  247. 247. Principal Component Analysis (PCA) 2 1 -1 2 Eigenvectors 9 3 4 4 = 11 1 Eigenvalues (direction) (magnitude)
  248. 248. Principal Component Analysis (PCA) 2 1 -1 2 Eigenvectors 9 3 4 4 = 11 1 Eigenvalues (direction) (magnitude)
  249. 249. Principal Component Analysis (PCA) 2 1 -1 2 Eigenvectors 9 3 4 4 = 11 1 Eigenvalues (direction) (magnitude)
  250. 250. Principal Component Analysis (PCA) 2 1 Eigenvectors 9 3 4 4 = 11 Eigenvalues (direction) (magnitude)
  251. 251. Principal Component Analysis (PCA) 2 1 Eigenvectors 9 3 4 4 = 11 Eigenvalues (direction) (magnitude)
  252. 252. Principal Component Analysis (PCA) 2 1 Eigenvectors 9 3 4 4 = 11 Eigenvalues (direction) (magnitude)
  253. 253. Principal Component Analysis (PCA) 2 1 Eigenvectors 9 3 4 4 = 11 Eigenvalues (direction) (magnitude)
  254. 254. Principal Component Analysis (PCA) 2 1 Eigenvectors 9 3 4 4 = 11 Eigenvalues (direction) (magnitude)
  255. 255. Principal Component Analysis (PCA)
  256. 256. Principal Component Analysis (PCA)
  257. 257. Thank you!
  258. 258. Thank you! Subscribe, like, share, comment! youtube.com/c/LuisSerrano
  259. 259. Thank you! @luis_likes_math Subscribe, like, share, comment! youtube.com/c/LuisSerrano

×