Download as PDF, PPTX
Uploaded byMateusz Buśkiewicz
Deep Learning Lightning Talk
The document provides an overview of deep learning through a series of lightning talks. It introduces deep learning as a powerful machine learning technique, discusses how it can be used to extract meaning from large amounts of data through tasks like prediction, analytics, and recommendations. It then covers theoretical concepts like neural networks and how deep learning models work, and practical aspects like training models on GPUs and clusters. The document also discusses challenges with deep learning and hype in the industry.
More Related Content
Similar to Deep Learning Lightning Talk
Deep Learning Lightning Talk
- 1. Deep Learning LightningTalk 1 Deep Learning State-of-the-art, powerful way to do machine learning Keynote Template
- 2. Deep Learning LightningTalk 2 Our goal: Build awesome data products User activity, business data, images, text, audio… Big Data technologies: Hadoop, Spark, Hive, Pig, Storm, Impala… Extract meaning from data and incorporate it into product: predicion, analytics, recommendations ! Technology ! Data ! Machine Learning
- 3. Deep Learning LightningTalk 3 ! Machine Learning Machine Learning There are tons of different machine learning algorithms for different problems Unsupervised methods: Customer segmentation (clustering), dataset visualisation, dimensionality reduction ⊞ Supervised methods: Predictions, classifications. Example product: spam filtering ⊞ Recommendations, anomaly detection… ! ⊞
- 4. Deep Learning LightningTalk 4 Difficult problems Image Recognition " Speech Recognition ♫ Natural Language Processing $
- 5. Deep Learning LightningTalk 5 Breakthrough: Deep Learning And now it works ;)
- 6. Deep Learning LightningTalk 6 Examples ! Skype Translator ! Google+ Photo Tagging http://www.youtube.com/watch?v=eu9kMIeS0wQ + Voice recognition in Android 4.0+, Apple’s Siri, Baidu’s Image Search, and more…
- 7. Deep Learning LightningTalk 7 A bit of theory Deep Learning is a „bigger and badder” approach to neural networks, which are known since 80’ y= g(x ⊗ W)
- 8. Deep Learning LightningTalk 7 A bit of theory Deep Learning is a „bigger and badder” approach to neural networks, which are known since 80’ y= g(x ⊗ W)
- 9. Deep Learning LightningTalk 7 A bit of theory Deep Learning is a „bigger and badder” approach to neural networks, which are known since 80’ y= g(x ⊗ W)
- 10. Deep Learning LightningTalk 7 A bit of theory Deep Learning is a „bigger and badder” approach to neural networks, which are known since 80’ y= g(x ⊗ W) Now we have much more computing power to train large (and deep) networks ⊞ Now we know better regularization and optimization methods ⊞ Now we have much more labeled data⊞ Now we can also train models with unlabeled data ⊞
- 11. Deep Learning LightningTalk 8 Why it works? Let’s consider the problem of face recognition That’s how we see it 0.2 0.0 0.1 1.0 1.0 0.1 0.4 0.8 1.0 ... 0.1 That’s how „machine” sees it
- 12. Deep Learning LightningTalk 9 Why it works? It’s much easier to infer that something is a face based on that it has two eyes and nose, than it has some black pixels in lower left corner, and white area somewhere in the middle
- 13. Deep Learning LightningTalk 10 A bit of practice GPU Cluster
- 14. Deep Learning LightningTalk 11 A bit of practice GPU Numerical operations are very efficient, up to 100x faster than CPU ⊞ Single machine, no communication overhead⊞ Significant memory contraints, we can’t train larger models ⊟
- 15. Deep Learning LightningTalk 12 A bit of practice TASK one learning task, many workers different parameters for each worker PICK BEST MODEL Netflix style! GPU WORKER 1 WORKER 2 WORKER 3 WORKER 4
- 16. Deep Learning LightningTalk 13 A bit of practice Cluster
- 17. Deep Learning LightningTalk 13 A bit of practice Cluster WORKER2 WORKER1 WORKER3 WORKER4 + ASYNCHRONOUS PARAMETERS SERVER Google style!
- 18. Deep Learning LightningTalk 14 A bit of practice Cluster We can train much larger and more powerful models ⊞ Scalable⊞ Poor resource utlization, even if we restrict connectivity ⊟ Complicated⊟
- 19. Deep Learning LightningTalk 15 Hype NETFLIX MOVES INTO DEEP LEARNING RESEARCH TO IMPROVE PERSONALIZATION 10 BREAKTHROUGH TECHNOLOGIES 2013 GIGAOM GUIDE TO DEEP LEARNING: WHO’S DOING IT AND WHY IT MATTERS NYU „DEEP LEARNING” PROFESSOR LECUN WILL HEAD FACEBOOK’S NEW ARTIFICIAL INTELLIGENCE LAB Geoffrey Hinton Leading researcher in DL, his startup was acquired by Google Lookflow Deep Learning image startup, acquired by Yahoo DeepMind Deep Learning startup, acquired by Google for 400 mln USD Yan LeCun Leading researcher in DL, hired by Facebook to lead new AI lab.
- 20. Deep Learning LightningTalk 16 Geoffrey Hinton Leading researcher in DL, his startup was acquired by Google Lookflow Deep Learning image startup, acquired by Yahoo DeepMind Deep Learning startup, acquired by Google for 400 mln USD Yan LeCun Leading researcher in DL, hired by Facebook to lead new AI lab. Hype
- 21. Deep Learning LightningTalk 17 It’s not a silver bullet It’s difficult. Sometimes it’s better to use simpler method. " # Nevertheless, it’s a very powerful technique, has attention of biggest IT companies and brings us closer to real artificial intelligence It requires substantial computing power and memory. Sometimes it’s not feasible to use deep learning models, especially if we have to train them regularly ! It’s kind of `black-box` Sometimes we can’t draw conclusions from learned features
- 22. ! :) THANKS $ mateusz.buskiewicz@getbase.com RESOURCES MOOC: NeuralNetworks for Machine Learning & https://www.coursera.org/course/neuralnets DL Tutorials + sample code & http://deeplearning.net/ Google+ Deep Learning Community & https://plus.google.com/u/0/communities/112866381580457264725 Deep Learning Book by Yoshua Bengio (draft) & http://www.iro.umontreal.ca/~bengioy/dlbook/ Deep Learning Libraries & Software & http://deeplearning.net/software_links/