Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Intro	
  To	
  Convolu,onal	
  Neural	
  
Networks	
  
Mark	
  Scully	
  
datapraxis.com	
  
Why	
  CNNs?	
  
h@ps://papers.nips.cc/paper/4824-­‐imagenet-­‐classifica,on-­‐with-­‐deep-­‐convolu,onal-­‐neural-­‐networks	
  
Image	
  C...
Object	
  Recogni,on	
  
h@ps://research.googleblog.com/2014/09/building-­‐deeper-­‐understanding-­‐of-­‐images.html	
  
h@p://cs.stanford.edu/people/karpathy/deepimagesent/	
  
Automa,c	
  Cap,oning	
  
h@ps://research.googleblog.com/2014/11/a-­‐picture-­‐is-­‐worth-­‐thousand-­‐coherent.html	
  
Facial	
  Recogni,on	
  	
  
Y.	
  Taigman,	
  M.	
  Yang,	
  M.	
  Ranzato,	
  L.	
  Wolf,	
  DeepFace:	
  Closing	
  the...
Terminator	
  Vision	
  
Colorize	
  Black	
  &	
  White	
  Images	
  
h@p://richzhang.github.io/coloriza,on/	
  
Style	
  Transfer	
  
h@p://genekogan.com/works/style-­‐transfer/	
  
Mona	
  Lisa	
  restyled	
  by	
  Picasso,	
  van	
 ...
Generate	
  An	
  Image	
  From	
  A	
  Sketch	
  
h@ps://affinelayer.com/pixsrv/	
  
ImageNet	
  Challenge	
  
Alexnet	
  
Li	
  Fei-­‐Fei:	
  ImageNet	
  Large	
  Scale	
  Visual	
  Recogni,on	
  Challenge,...
ImageNet	
  Challenge	
  
ILSVRC+
ImageNet Classification error throughout years and groups
Li	
  Fei-­‐Fei:	
  ImageNet	
 ...
Alexnet	
  Architecture	
  -­‐	
  2012	
  
Input	
  
Conv	
  
Relu	
  
Pool	
  
Conv	
  
Relu	
  
Pool	
  
Conv	
  
Relu	
...
Alexnet	
  Architecture	
  -­‐	
  2012	
  
ImageNet	
  Classifica,on	
  with	
  Deep	
  Convolu,onal	
  Neural	
  Networks	...
ImageNet	
  Challenge	
  
Alexnet	
  
Li	
  Fei-­‐Fei:	
  ImageNet	
  Large	
  Scale	
  Visual	
  Recogni,on	
  Challenge,...
Tradi,onal	
  Approach	
  To	
  Image	
  
Classifica,on	
  
Input	
  Image	
  
Hand	
  
Extracted	
  
Features	
  
Classifie...
Issues	
  
•  Who	
  makes	
  the	
  features?	
  
– Need	
  an	
  expert	
  for	
  each	
  problem	
  domain	
  
•  Which...
Are	
  these	
  pictures	
  of	
  the	
  same	
  thing?	
  
Features	
  Are	
  Hierarchical	
  
	
  
•  A	
  squirrel	
  is	
  a	
  combina,on	
  of	
  fur,	
  arms,	
  legs,	
  
&	
...
Image	
  Features	
  
•  A	
  feature	
  is	
  something	
  in	
  the	
  image	
  or	
  derived	
  
from	
  it	
  that’s	
...
Edges	
  
Ideally	
  We’d	
  Learn	
  Features	
  
Input	
  
Image	
  
Output	
  
Label	
  
Ideally	
  We’d	
  Learn	
  Features	
  
Input	
  
Image	
  
Output	
  
Label	
  
CNNs	
  
What	
  is	
  a	
  Neural	
  Network?	
  
•  Perceptron	
  is	
  biologically	
  inspired	
  
•  A	
  mental	
  model	
  f...
Perceptron	
  
1	
  
x1	
  
x2	
  
x3	
  
xm	
  
Σ	
   Output	
  
Ac,va,on	
  
Func,on	
  
Sum	
  
w0	
  
w1	
  
w2	
  
w3...
Perceptron	
  
1	
  
x1	
  
x2	
  
x3	
  
xm	
  
Σ	
   Output	
  
Ac,va,on	
  
Func,on	
  
Sum	
  
w0	
  
w1	
  
w2	
  
w3...
Ac,va,on	
  Func,ons	
  
Training:	
  Upda,ng	
  Weights	
  
1	
  
x1	
  
x2	
  
x3	
  
x4	
  
Σ	
   Output	
  
Ac,va,on	
  
Func,on	
  
Sum	
  
w0...
Perceptron	
  Decision	
  Boundary	
  
Deep	
  (Mul,-­‐Layer)	
  Neural	
  Network	
  
Backpropaga,on	
  
•  Error	
  propagates	
  backward	
  and	
  it	
  all	
  works	
  via	
  
(normally	
  stochas,c)	
  g...
Alexnet	
  Architecture	
  -­‐	
  2012	
  
ImageNet	
  Classifica,on	
  with	
  Deep	
  Convolu,onal	
  Neural	
  Networks	...
CNN	
  Layer	
  Architecture	
  
Input	
  
Convolu,on	
  
Nonlinearity	
  
Pooling	
  (op,onal)	
  
Dropout	
  (op,onal)	
...
CNN	
  Layer	
  Architecture	
  
Input	
  
Convolu,on	
  
Nonlinearity	
  
Pooling	
  (op,onal)	
  
Dropout	
  (op,onal)	
...
Input:	
  Pixels	
  Are	
  Just	
  Numbers	
  
h@ps://medium.com/@ageitgey/machine-­‐learning-­‐is-­‐fun-­‐part-­‐3-­‐deep...
CNN	
  Layer	
  Architecture	
  
Input	
  
Convolu,on	
  
Nonlinearity	
  
Pooling	
  (op,onal)	
  
Dropout	
  (op,onal)	
...
Goals	
  
•  Need	
  to	
  detect	
  the	
  same	
  feature	
  anywhere	
  in	
  
an	
  image	
  
•  Reuse	
  the	
  same	...
Neuron	
  =	
  Filter	
  
•  Act	
  as	
  detectors	
  for	
  some	
  specific	
  image	
  
feature	
  
•  Take	
  images	
...
Convolu,on	
  
•  Like	
  sliding	
  a	
  matrix	
  over	
  the	
  input	
  and	
  
performing	
  dot	
  products	
  
•  I...
Convolu,on	
  
Convolu,on	
  
Filters	
  (or	
  Kernels)	
  
Sharpen	
  
Filters	
  (or	
  Kernels)	
  
Box	
  Blur	
  
Filters	
  (or	
  Kernels)	
  
Edge	
  Detec,on	
  
Feature	
  Map	
  
Alexnet	
  Architecture	
  
Convolu,ons	
  
CNN	
  Layer	
  Architecture	
  
Input	
  
Convolu,on	
  
Nonlinearity	
  
Pooling	
  (op,onal)	
  
Dropout	
  (op,onal)	
...
Nonlinearity	
  
CNN	
  Layer	
  Architecture	
  
Input	
  
Convolu,on	
  
Nonlinearity	
  
Pooling	
  (op,onal)	
  
Dropout	
  (op,onal)	
...
Max	
  Pooling	
  Example	
  
Alexnet	
  Architecture	
  
3x3	
  stride	
  2	
  Max	
  Pooling	
  
Pooling	
  
•  Allows	
  us	
  to	
  look	
  at	
  more	
  of	
  the	
  image	
  
•  Max,	
  sum,	
  and	
  L2	
  pooling	...
CNN	
  Layer	
  Architecture	
  
Input	
  
Convolu,on	
  
Nonlinearity	
  
Pooling	
  (op,onal)	
  
Dropout	
  (op,onal)	
...
Alexnet	
  Architecture	
  -­‐	
  2012	
  
Input	
  
Conv	
  
Relu	
  
Pool	
  
Conv	
  
Relu	
  
Pool	
  
Conv	
  
Relu	
...
Dropout	
  
h@p://cs231n.github.io/neural-­‐networks-­‐2/	
  
•  Randomly	
  disable	
  some	
  neurons	
  on	
  the	
  
f...
Let’s	
  Predict	
  Something!	
  
•  We	
  have	
  all	
  these	
  features,	
  how	
  do	
  we	
  learn	
  
to	
  label	...
Alexnet	
  Architecture	
  -­‐	
  2012	
  
Input	
  
Conv	
  
Relu	
  
Pool	
  
Conv	
  
Relu	
  
Pool	
  
Conv	
  
Relu	
...
Fully	
  Connected	
  Layers	
  
•  Each	
  neuron	
  is	
  connected	
  to	
  all	
  inputs	
  
•  Standard	
  mul,layer	...
Alexnet	
  Architecture	
  
Alexnet	
  Architecture	
  -­‐	
  2012	
  
Input	
  
Conv	
  
Relu	
  
Pool	
  
Conv	
  
Relu	
  
Pool	
  
Conv	
  
Relu	
...
Which	
  Class	
  Is	
  It	
  Again?	
  
•  FC-­‐1000	
  gives	
  us	
  1000	
  numbers,	
  one	
  per	
  class,	
  
how	
...
Soqmax	
  
•  Mul,-­‐class	
  version	
  of	
  logis,c	
  func,on	
  
•  Outputs	
  normalized	
  class	
  “probabili,es”	...
h@ps://papers.nips.cc/paper/4824-­‐imagenet-­‐classifica,on-­‐with-­‐deep-­‐convolu,onal-­‐neural-­‐networks	
  
Image	
  C...
Alexnet	
  Architecture	
  -­‐	
  2012	
  
ImageNet	
  Classifica,on	
  with	
  Deep	
  Convolu,onal	
  Neural	
  Networks	...
Learned	
  Filters	
  –	
  Layer1	
  
Alexnet	
  Architecture	
  -­‐	
  2012	
  
ImageNet	
  Classifica,on	
  with	
  Deep	
  Convolu,onal	
  Neural	
  Networks	...
Learned	
  Filters	
  –	
  Layer2	
  
Visualizing	
  and	
  Understanding	
  Convolu,onal	
  Networks	
  -­‐	
  Zeiler	
  ...
Alexnet	
  Architecture	
  -­‐	
  2012	
  
ImageNet	
  Classifica,on	
  with	
  Deep	
  Convolu,onal	
  Neural	
  Networks	...
Learned	
  Filters	
  -­‐	
  Layer3	
  
Visualizing	
  and	
  Understanding	
  Convolu,onal	
  Networks	
  -­‐	
  Zeiler	
...
Alexnet	
  Architecture	
  -­‐	
  2012	
  
ImageNet	
  Classifica,on	
  with	
  Deep	
  Convolu,onal	
  Neural	
  Networks	...
Learned	
  Features	
  –	
  Layers	
  4	
  &	
  5	
  
Alexnet	
  Architecture	
  -­‐	
  2012	
  
ImageNet	
  Classifica,on	
  with	
  Deep	
  Convolu,onal	
  Neural	
  Networks	...
Alexnet	
  Architecture	
  -­‐	
  2012	
  
Input	
  
Conv	
  
Relu	
  
Pool	
  
Conv	
  
Relu	
  
Pool	
  
Conv	
  
Relu	
...
VGG16	
  
h@ps://blog.heuritech.com/2016/02/29/a-­‐brief-­‐report-­‐of-­‐the-­‐heuritech-­‐deep-­‐learning-­‐meetup-­‐5/	
...
Google’s	
  Incep,on	
  Module	
  
To	
  Learn	
  More	
  
•  h@p://colah.github.io/posts/2014-­‐07-­‐
Understanding-­‐Convolu,ons/	
  
•  h@ps://adeshpande3...
Ques,ons?	
  
Intro To Convolutional Neural Networks
Upcoming SlideShare
Loading in …5
×

Intro To Convolutional Neural Networks

1,048 views

Published on

Slides from Portland Machine Learning meetup, April 13th.
Abstract: You've heard all the cool tech companies are using them, but what are Convolutional Neural Networks (CNNs) good for and what is convolution anyway? For that matter, what is a Neural Network? This talk will include a look at some applications of CNNs, an explanation of how CNNs work, and what the different layers in a CNN do. There's no explicit background required so if you have no idea what a neural network is that's ok.

Published in: Data & Analytics
  • Be the first to comment

Intro To Convolutional Neural Networks

  1. 1. Intro  To  Convolu,onal  Neural   Networks   Mark  Scully   datapraxis.com  
  2. 2. Why  CNNs?  
  3. 3. h@ps://papers.nips.cc/paper/4824-­‐imagenet-­‐classifica,on-­‐with-­‐deep-­‐convolu,onal-­‐neural-­‐networks   Image  Classifica,on  
  4. 4. Object  Recogni,on   h@ps://research.googleblog.com/2014/09/building-­‐deeper-­‐understanding-­‐of-­‐images.html  
  5. 5. h@p://cs.stanford.edu/people/karpathy/deepimagesent/  
  6. 6. Automa,c  Cap,oning   h@ps://research.googleblog.com/2014/11/a-­‐picture-­‐is-­‐worth-­‐thousand-­‐coherent.html  
  7. 7. Facial  Recogni,on     Y.  Taigman,  M.  Yang,  M.  Ranzato,  L.  Wolf,  DeepFace:  Closing  the  Gap  to  Human-­‐Level  Performance  in  Face  Verifica,on,  CVPR   2014  
  8. 8. Terminator  Vision  
  9. 9. Colorize  Black  &  White  Images   h@p://richzhang.github.io/coloriza,on/  
  10. 10. Style  Transfer   h@p://genekogan.com/works/style-­‐transfer/   Mona  Lisa  restyled  by  Picasso,  van  Gough,  and  Monet  
  11. 11. Generate  An  Image  From  A  Sketch   h@ps://affinelayer.com/pixsrv/  
  12. 12. ImageNet  Challenge   Alexnet   Li  Fei-­‐Fei:  ImageNet  Large  Scale  Visual  Recogni,on  Challenge,  2014  
  13. 13. ImageNet  Challenge   ILSVRC+ ImageNet Classification error throughout years and groups Li  Fei-­‐Fei:  ImageNet  Large  Scale  Visual  Recogni,on  Challenge,  2014  
  14. 14. Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   FC  1000  
  15. 15. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012    
  16. 16. ImageNet  Challenge   Alexnet   Li  Fei-­‐Fei:  ImageNet  Large  Scale  Visual  Recogni,on  Challenge,  2014  
  17. 17. Tradi,onal  Approach  To  Image   Classifica,on   Input  Image   Hand   Extracted   Features   Classifier   Object  Label  
  18. 18. Issues   •  Who  makes  the  features?   – Need  an  expert  for  each  problem  domain   •  Which  features?   – Are  they  the  same  for  every  problem  type?   •  How  robust  are  these  features  to  real  images?   – Transla,on,  Rota,on,  contrast  changes,  etc.  
  19. 19. Are  these  pictures  of  the  same  thing?  
  20. 20. Features  Are  Hierarchical     •  A  squirrel  is  a  combina,on  of  fur,  arms,  legs,   &  a  tail  in  specific  propor,ons.   •  A  tail  is  made  of  texture,  color,  and  spa,al   rela,onships   •  A  texture  is  made  of  oriented  edges,   gradients,  and  colors  
  21. 21. Image  Features   •  A  feature  is  something  in  the  image  or  derived   from  it  that’s  relevant  to  the  task   •  Edges   •  Lines  at  different  angles,  curves,  etc.   •  Colors,  or  pa@erns  of  colors   •  SIFT,  SURF,  HOG,  GIST,  ORB,  etc  
  22. 22. Edges  
  23. 23. Ideally  We’d  Learn  Features   Input   Image   Output   Label  
  24. 24. Ideally  We’d  Learn  Features   Input   Image   Output   Label   CNNs  
  25. 25. What  is  a  Neural  Network?   •  Perceptron  is  biologically  inspired   •  A  mental  model  for  interpre,ng  the  math   h@p://cs231n.stanford.edu/index.html    
  26. 26. Perceptron   1   x1   x2   x3   xm   Σ   Output   Ac,va,on   Func,on   Sum   w0   w1   w2   w3   wm   Weights   Inputs  
  27. 27. Perceptron   1   x1   x2   x3   xm   Σ   Output   Ac,va,on   Func,on   Sum   w0   w1   w2   w3   wm   Weights   Inputs   wi xi i=0 m ∑ = w0 x0 + w1x1 + w2 x2 +...+ wm xm
  28. 28. Ac,va,on  Func,ons  
  29. 29. Training:  Upda,ng  Weights   1   x1   x2   x3   x4   Σ   Output   Ac,va,on   Func,on   Sum   w0   w1   w2   w3   w4   Weights   Inputs   Error  =  Output  -­‐  Target  
  30. 30. Perceptron  Decision  Boundary  
  31. 31. Deep  (Mul,-­‐Layer)  Neural  Network  
  32. 32. Backpropaga,on   •  Error  propagates  backward  and  it  all  works  via   (normally  stochas,c)  gradient  descent.   •  (wave  hands)  
  33. 33. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012    
  34. 34. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  35. 35. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  36. 36. Input:  Pixels  Are  Just  Numbers   h@ps://medium.com/@ageitgey/machine-­‐learning-­‐is-­‐fun-­‐part-­‐3-­‐deep-­‐learning-­‐and-­‐convolu,onal-­‐neural-­‐networks-­‐ f40359318721  
  37. 37. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  38. 38. Goals   •  Need  to  detect  the  same  feature  anywhere  in   an  image   •  Reuse  the  same  weights  over  and  over   •  What  we  really  want  is  one  neuron  that   detects  a  feature  that  we  slide  over  the  image  
  39. 39. Neuron  =  Filter   •  Act  as  detectors  for  some  specific  image   feature   •  Take  images  as  inputs  and  produce  image  like   feature  maps  as  outputs  
  40. 40. Convolu,on   •  Like  sliding  a  matrix  over  the  input  and   performing  dot  products   •  It’s  all  just  matrix  mul,plica,on  
  41. 41. Convolu,on  
  42. 42. Convolu,on  
  43. 43. Filters  (or  Kernels)   Sharpen  
  44. 44. Filters  (or  Kernels)   Box  Blur  
  45. 45. Filters  (or  Kernels)   Edge  Detec,on   Feature  Map  
  46. 46. Alexnet  Architecture   Convolu,ons  
  47. 47. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  48. 48. Nonlinearity  
  49. 49. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  50. 50. Max  Pooling  Example  
  51. 51. Alexnet  Architecture   3x3  stride  2  Max  Pooling  
  52. 52. Pooling   •  Allows  us  to  look  at  more  of  the  image   •  Max,  sum,  and  L2  pooling   •  A  type  of  downsampling  
  53. 53. CNN  Layer  Architecture   Input   Convolu,on   Nonlinearity   Pooling  (op,onal)   Dropout  (op,onal)  
  54. 54. Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   FC  1000   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   Dropout  
  55. 55. Dropout   h@p://cs231n.github.io/neural-­‐networks-­‐2/   •  Randomly  disable  some  neurons  on  the   forward  pass   •  Prevents  overfiong    
  56. 56. Let’s  Predict  Something!   •  We  have  all  these  features,  how  do  we  learn   to  label  something  based  on  them?  
  57. 57. Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   FC  1000   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   Fully  Connected  
  58. 58. Fully  Connected  Layers   •  Each  neuron  is  connected  to  all  inputs   •  Standard  mul,layer  neural  net   •  Learns  non-­‐linear  combina,ons  of  the  feature   maps  to  make  predic,ons  
  59. 59. Alexnet  Architecture  
  60. 60. Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   FC  1000  
  61. 61. Which  Class  Is  It  Again?   •  FC-­‐1000  gives  us  1000  numbers,  one  per  class,   how  do  we  compare  them?  
  62. 62. Soqmax   •  Mul,-­‐class  version  of  logis,c  func,on   •  Outputs  normalized  class  “probabili,es”   •  Takes  m  inputs  and  produces  m  outputs   between  zero  and  one,  that  sum  to  one   •  Cross-­‐entropy  loss   •  Differen,able  
  63. 63. h@ps://papers.nips.cc/paper/4824-­‐imagenet-­‐classifica,on-­‐with-­‐deep-­‐convolu,onal-­‐neural-­‐networks   Image  Classifica,on  
  64. 64. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  1  
  65. 65. Learned  Filters  –  Layer1  
  66. 66. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  2  
  67. 67. Learned  Filters  –  Layer2   Visualizing  and  Understanding  Convolu,onal  Networks  -­‐  Zeiler  &  Fergus,  ECCV  2014    
  68. 68. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  3  
  69. 69. Learned  Filters  -­‐  Layer3   Visualizing  and  Understanding  Convolu,onal  Networks  -­‐  Zeiler  &  Fergus,  ECCV  2014      
  70. 70. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012     Layer  4   Layer  5  
  71. 71. Learned  Features  –  Layers  4  &  5  
  72. 72. Alexnet  Architecture  -­‐  2012   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.  1097-­‐1105,   2012    
  73. 73. Alexnet  Architecture  -­‐  2012   Input   Conv   Relu   Pool   Conv   Relu   Pool   Conv   Relu   Conv   Relu   Conv   Relu   Pool   FC   Dropout   FC   Dropout   ImageNet  Classifica,on  with  Deep  Convolu,onal  Neural  Networks  Alex  Krizhevsky,  Ilya  Sutskever  and  Geoffrey  E.  Hinton   Advances  in  Neural  Informa,on  Processing  Systems  25  eds.F.  Pereira,  C.J.C.  Burges,  L.  Bo@ou  and  K.Q.  Weinberger  pp.   1097-­‐1105,  2012   FC  1000  
  74. 74. VGG16   h@ps://blog.heuritech.com/2016/02/29/a-­‐brief-­‐report-­‐of-­‐the-­‐heuritech-­‐deep-­‐learning-­‐meetup-­‐5/  
  75. 75. Google’s  Incep,on  Module  
  76. 76. To  Learn  More   •  h@p://colah.github.io/posts/2014-­‐07-­‐ Understanding-­‐Convolu,ons/   •  h@ps://adeshpande3.github.io/ adeshpande3.github.io/The-­‐9-­‐Deep-­‐Learning-­‐ Papers-­‐You-­‐Need-­‐To-­‐Know-­‐About.html   •  h@p://cs231n.github.io/   •  h@p://course.fast.ai/  
  77. 77. Ques,ons?  

×