Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Location:
QuantUniversity Meetup
December 21st 2016
Boston MA
Deep Learning : An introduction
2016 Copyright QuantUniversi...
2
Slides and Code will be available at:
http://www.analyticscertificate.com
- Analytics Advisory services
- Custom training programs
- Architecture assessments, advice and audits
• Founder of QuantUniversity LLC. and
www.analyticscertificate.com
• Advisory and Consultancy for Financial Analytics
• Pr...
5
Quantitative Analytics and Big Data Analytics Onboarding
• Trained more than 500 students in
Quantitative methods, Data ...
6
• January 2017
▫ 19th, Deep Learning Lecture Part II
• February 2017
▫ Deep Learning Workshop (Date TBD)
Events of Inter...
7
Dr. Victor Shnayder
Fellow
QuantUniversity
Prior Experience:
Product Manager EdX (March 2013-June 2016)
Harvard Universi...
8
Start with labeled pairs (Xi, Yi)
( ,“kitten”),( ,“puppy”)
…
9
Success: predict new examples
( ,?)
10
https://commons.wikimedia.org/wiki/Neural_network
“kitten”
“puppy”
“has fur?”
“pointy ears?”
“dangerously cute?”
11
12
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
13
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Weight...
14
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Non-li...
15
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Learni...
16
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Learni...
18
1. Our labeled datasets were thousands of times too small.
2. Our computers were millions of times too slow.
3. We init...
19
http://www.rsipvision.com/exploring-deep-learning/
20
http://www.asimovinstitute.org/neural-network-zoo/
21
22
https://research.googleblog.com/2014/09/building-deeper-understanding-of-images.html
23
https://research.googleblog.com/2014/09/building-deeper-understanding-of-images.html
24
Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary
Nod...
25
Towards End-to-End Speech Recognition with Recurrent Neural Networks
http://www.jmlr.org/proceedings/papers/v32/graves1...
26
https://www.technologyreview.com/s/544651/baidus-deep-learning-system-rivals-people-at-speech-recognition/
27
https://research.googleblog.com/2014/11/a-picture-is-worth-thousand-coherent.html
28
http://cs.umd.edu/~miyyer/data/deepqa.pdf
29
30
http://blog.ventureradar.com/2016/03/11/10-hot-startups-using-artificial-intelligence-in-cyber-security/
31
https://www.youtube.com/watch?v=H4V6NZLNu-c
32
https://www.engadget.com/2016/03/12/watch-alphago-vs-lee-sedol-round-3-live-right-now/
33
https://www.youtube.com/watch?v=kMMbW96nMW8
34
35
How is deep learning special?
Given (lots of) data, DNNs learn useful input
representations.
D. Erhan et al. ‘09
http:/...
36
37
Hardware
38
Data
http://www.theneweconomy.com/strategy/big-data-is-not-without-its-problems
39
New Approaches
http://deeplearning.net/reading-list/
40
41
• Theano is a Python library that allows you to define, optimize, and
evaluate mathematical expressions involving multi...
42
• GPU vs CPU
▫ Theano Test
▫ See Theano Test.ipyb
Demo
43
• Logistic Regression
Theano
See Theano-Logistic Regression.ipyb
44
MLP
45
Convolutional Neural Networks
Convolution
46
Convolutional Neural Networks
Sparse connectivity
Weight sharing-Max-pooling layer
See Theano-Conv-Net.ipynb
47
• Keras is a high-level neural networks library, written in Python and
capable of running on top of either TensorFlow o...
48
• Keras Examples
▫ Testing Keras: See KerasPython.ipynb
▫ Running Convolutional NN on Keras with a Theano Backend
 See...
49
• A case study for Convolutional Neural Networks
• Recurrent Neural Networks
• Auto Encoders
• Best Practices
Coming on...
50
Q&A
Thank you!
Members & Sponsors!
Sri Krishnamurthy, CFA, CAP
Founder and CEO
QuantUniversity LLC.
srikrishnamurthy
www.Quant...
Upcoming SlideShare
Loading in …5
×

Deep learning - Part I

8,332 views

Published on

Interest in Deep Learning has been growing in the past few years. With advances in software and hardware technologies, Neural Networks are making a resurgence. With interest in AI based applications growing, and companies like IBM, Google, Microsoft, NVidia investing heavily in computing and software applications, it is time to understand Deep Learning better!

In this lecture, we will discuss the basics of Neural Networks and discuss how Deep Learning Neural networks are different from conventional Neural Network architectures. We will review a bit of mathematics that goes into building neural networks and understand the role of GPUs in Deep Learning. We will also get an introduction to Autoencoders, Convolutional Neural Networks, Recurrent Neural Networks and understand the state-of-the-art in hardware and software architectures. Functional Demos will be presented in Keras, a popular Python package with a backend in Theano. This will be a preview of the QuantUniversity Deep Learning Workshop that will be offered in 2017.

Published in: Data & Analytics

Deep learning - Part I

  1. 1. Location: QuantUniversity Meetup December 21st 2016 Boston MA Deep Learning : An introduction 2016 Copyright QuantUniversity LLC. Presented By: Sri Krishnamurthy, CFA, CAP www.QuantUniversity.com sri@quantuniversity.com
  2. 2. 2 Slides and Code will be available at: http://www.analyticscertificate.com
  3. 3. - Analytics Advisory services - Custom training programs - Architecture assessments, advice and audits
  4. 4. • Founder of QuantUniversity LLC. and www.analyticscertificate.com • Advisory and Consultancy for Financial Analytics • Prior Experience at MathWorks, Citigroup and Endeca and 25+ financial services and energy customers. • Regular Columnist for the Wilmott Magazine • Author of forthcoming book “Financial Modeling: A case study approach” published by Wiley • Charted Financial Analyst and Certified Analytics Professional • Teaches Analytics in the Babson College MBA program and at Northeastern University, Boston Sri Krishnamurthy Founder and CEO 4
  5. 5. 5 Quantitative Analytics and Big Data Analytics Onboarding • Trained more than 500 students in Quantitative methods, Data Science and Big Data Technologies using MATLAB, Python and R • Launching the Analytics Certificate Program in September
  6. 6. 6 • January 2017 ▫ 19th, Deep Learning Lecture Part II • February 2017 ▫ Deep Learning Workshop (Date TBD) Events of Interest
  7. 7. 7 Dr. Victor Shnayder Fellow QuantUniversity Prior Experience: Product Manager EdX (March 2013-June 2016) Harvard University PhD, Computer Science
  8. 8. 8 Start with labeled pairs (Xi, Yi) ( ,“kitten”),( ,“puppy”) …
  9. 9. 9 Success: predict new examples ( ,?)
  10. 10. 10 https://commons.wikimedia.org/wiki/Neural_network “kitten” “puppy” “has fur?” “pointy ears?” “dangerously cute?”
  11. 11. 11
  12. 12. 12 http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
  13. 13. 13 http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double Weighted sum
  14. 14. 14 http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double Non-linear “activation” function
  15. 15. 15 http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double Learning = “find good weights”
  16. 16. 16 http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double Learning = “find good weights” How? Gradient descent!
  17. 17. 18 1. Our labeled datasets were thousands of times too small. 2. Our computers were millions of times too slow. 3. We initialized the weights in a stupid way. 4. We used the wrong type of non-linearity. - Geoff Hinton Neural nets were tried in the 1980s. What changed? https://youtu.be/IcOMKXAw5VA?t=21m29s
  18. 18. 19 http://www.rsipvision.com/exploring-deep-learning/
  19. 19. 20 http://www.asimovinstitute.org/neural-network-zoo/
  20. 20. 21
  21. 21. 22 https://research.googleblog.com/2014/09/building-deeper-understanding-of-images.html
  22. 22. 23 https://research.googleblog.com/2014/09/building-deeper-understanding-of-images.html
  23. 23. 24 Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans http://www.nature.com/articles/srep24454/figures/1
  24. 24. 25 Towards End-to-End Speech Recognition with Recurrent Neural Networks http://www.jmlr.org/proceedings/papers/v32/graves14.pdf
  25. 25. 26 https://www.technologyreview.com/s/544651/baidus-deep-learning-system-rivals-people-at-speech-recognition/
  26. 26. 27 https://research.googleblog.com/2014/11/a-picture-is-worth-thousand-coherent.html
  27. 27. 28 http://cs.umd.edu/~miyyer/data/deepqa.pdf
  28. 28. 29
  29. 29. 30 http://blog.ventureradar.com/2016/03/11/10-hot-startups-using-artificial-intelligence-in-cyber-security/
  30. 30. 31 https://www.youtube.com/watch?v=H4V6NZLNu-c
  31. 31. 32 https://www.engadget.com/2016/03/12/watch-alphago-vs-lee-sedol-round-3-live-right-now/
  32. 32. 33 https://www.youtube.com/watch?v=kMMbW96nMW8
  33. 33. 34
  34. 34. 35 How is deep learning special? Given (lots of) data, DNNs learn useful input representations. D. Erhan et al. ‘09 http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/247
  35. 35. 36
  36. 36. 37 Hardware
  37. 37. 38 Data http://www.theneweconomy.com/strategy/big-data-is-not-without-its-problems
  38. 38. 39 New Approaches http://deeplearning.net/reading-list/
  39. 39. 40
  40. 40. 41 • Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently • Performs efficient symbolic differentiation • Leverages NVIDIA GPU (Claim 140X faster than CPU) • Developed by University of Montreal researchers and is open-source • Works on Windows/Linux/Mac OS • See https://arxiv.org/abs/1605.02688 Theano
  41. 41. 42 • GPU vs CPU ▫ Theano Test ▫ See Theano Test.ipyb Demo
  42. 42. 43 • Logistic Regression Theano See Theano-Logistic Regression.ipyb
  43. 43. 44 MLP
  44. 44. 45 Convolutional Neural Networks Convolution
  45. 45. 46 Convolutional Neural Networks Sparse connectivity Weight sharing-Max-pooling layer See Theano-Conv-Net.ipynb
  46. 46. 47 • Keras is a high-level neural networks library, written in Python and capable of running on top of either TensorFlow or Theano. It was developed with a focus on enabling fast experimentation. • Allows for easy and fast prototyping (through total modularity, minimalism, and extensibility). • Supports both convolutional networks and recurrent networks, as well as combinations of the two. • Supports arbitrary connectivity schemes (including multi-input and multi-output training). • Runs seamlessly on CPU and GPU. Keras
  47. 47. 48 • Keras Examples ▫ Testing Keras: See KerasPython.ipynb ▫ Running Convolutional NN on Keras with a Theano Backend  See Keras-conv-example-mnist.ipynb Demo
  48. 48. 49 • A case study for Convolutional Neural Networks • Recurrent Neural Networks • Auto Encoders • Best Practices Coming on January 21st - Part II
  49. 49. 50 Q&A
  50. 50. Thank you! Members & Sponsors! Sri Krishnamurthy, CFA, CAP Founder and CEO QuantUniversity LLC. srikrishnamurthy www.QuantUniversity.com Contact Information, data and drawings embodied in this presentation are strictly a property of QuantUniversity LLC. and shall not be distributed or used in any other publication without the prior written consent of QuantUniversity LLC. 51

×