Deep learning - Part I
•
11 likes•9,384 views
Interest in Deep Learning has been growing in the past few years. With advances in software and hardware technologies, Neural Networks are making a resurgence. With interest in AI based applications growing, and companies like IBM, Google, Microsoft, NVidia investing heavily in computing and software applications, it is time to understand Deep Learning better! In this lecture, we will discuss the basics of Neural Networks and discuss how Deep Learning Neural networks are different from conventional Neural Network architectures. We will review a bit of mathematics that goes into building neural networks and understand the role of GPUs in Deep Learning. We will also get an introduction to Autoencoders, Convolutional Neural Networks, Recurrent Neural Networks and understand the state-of-the-art in hardware and software architectures. Functional Demos will be presented in Keras, a popular Python package with a backend in Theano. This will be a preview of the QuantUniversity Deep Learning Workshop that will be offered in 2017.
Recommended
More Related Content
What's hot
What's hot (10)
Similar to Deep learning - Part I
Similar to Deep learning - Part I (20)
More from QuantUniversity
More from QuantUniversity (20)
Recently uploaded
Recently uploaded (20)
Deep learning - Part I
- 1. Location: QuantUniversity Meetup December 21st 2016 Boston MA Deep Learning : An introduction 2016 Copyright QuantUniversity LLC. Presented By: Sri Krishnamurthy, CFA, CAP www.QuantUniversity.com sri@quantuniversity.com
- 2. 2 Slides and Code will be available at: http://www.analyticscertificate.com
- 3. - Analytics Advisory services - Custom training programs - Architecture assessments, advice and audits
- 4. • Founder of QuantUniversity LLC. and www.analyticscertificate.com • Advisory and Consultancy for Financial Analytics • Prior Experience at MathWorks, Citigroup and Endeca and 25+ financial services and energy customers. • Regular Columnist for the Wilmott Magazine • Author of forthcoming book “Financial Modeling: A case study approach” published by Wiley • Charted Financial Analyst and Certified Analytics Professional • Teaches Analytics in the Babson College MBA program and at Northeastern University, Boston Sri Krishnamurthy Founder and CEO 4
- 5. 5 Quantitative Analytics and Big Data Analytics Onboarding • Trained more than 500 students in Quantitative methods, Data Science and Big Data Technologies using MATLAB, Python and R • Launching the Analytics Certificate Program in September
- 6. 6 • January 2017 ▫ 19th, Deep Learning Lecture Part II • February 2017 ▫ Deep Learning Workshop (Date TBD) Events of Interest
- 7. 7 Dr. Victor Shnayder Fellow QuantUniversity Prior Experience: Product Manager EdX (March 2013-June 2016) Harvard University PhD, Computer Science
- 8. 8 Start with labeled pairs (Xi, Yi) ( ,“kitten”),( ,“puppy”) …
- 9. 9 Success: predict new examples ( ,?)
- 11. 11
- 15. 15 http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double Learning = “find good weights”
- 16. 16 http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double Learning = “find good weights” How? Gradient descent!
- 17. 18 1. Our labeled datasets were thousands of times too small. 2. Our computers were millions of times too slow. 3. We initialized the weights in a stupid way. 4. We used the wrong type of non-linearity. - Geoff Hinton Neural nets were tried in the 1980s. What changed? https://youtu.be/IcOMKXAw5VA?t=21m29s
- 20. 21
- 23. 24 Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans http://www.nature.com/articles/srep24454/figures/1
- 24. 25 Towards End-to-End Speech Recognition with Recurrent Neural Networks http://www.jmlr.org/proceedings/papers/v32/graves14.pdf
- 28. 29
- 33. 34
- 34. 35 How is deep learning special? Given (lots of) data, DNNs learn useful input representations. D. Erhan et al. ‘09 http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/247
- 35. 36
- 36. 37 Hardware
- 39. 40
- 40. 41 • Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently • Performs efficient symbolic differentiation • Leverages NVIDIA GPU (Claim 140X faster than CPU) • Developed by University of Montreal researchers and is open-source • Works on Windows/Linux/Mac OS • See https://arxiv.org/abs/1605.02688 Theano
- 41. 42 • GPU vs CPU ▫ Theano Test ▫ See Theano Test.ipyb Demo
- 42. 43 • Logistic Regression Theano See Theano-Logistic Regression.ipyb
- 43. 44 MLP
- 45. 46 Convolutional Neural Networks Sparse connectivity Weight sharing-Max-pooling layer See Theano-Conv-Net.ipynb
- 46. 47 • Keras is a high-level neural networks library, written in Python and capable of running on top of either TensorFlow or Theano. It was developed with a focus on enabling fast experimentation. • Allows for easy and fast prototyping (through total modularity, minimalism, and extensibility). • Supports both convolutional networks and recurrent networks, as well as combinations of the two. • Supports arbitrary connectivity schemes (including multi-input and multi-output training). • Runs seamlessly on CPU and GPU. Keras
- 47. 48 • Keras Examples ▫ Testing Keras: See KerasPython.ipynb ▫ Running Convolutional NN on Keras with a Theano Backend See Keras-conv-example-mnist.ipynb Demo
- 48. 49 • A case study for Convolutional Neural Networks • Recurrent Neural Networks • Auto Encoders • Best Practices Coming on January 21st - Part II
- 49. 50 Q&A
- 50. Thank you! Members & Sponsors! Sri Krishnamurthy, CFA, CAP Founder and CEO QuantUniversity LLC. srikrishnamurthy www.QuantUniversity.com Contact Information, data and drawings embodied in this presentation are strictly a property of QuantUniversity LLC. and shall not be distributed or used in any other publication without the prior written consent of QuantUniversity LLC. 51