Nas 也可以揀土豆

826 views

Published on

TensorFlow 是由 Google 所公布的開源機器學習平台,根據 Github 的數據統計,TensorFlow 成為2016年最受關注的十大開源專案之一。此次分享將介紹,如何在 NAS 上整合TensorFlow 及相關 Open source project,以展示幾種相關的資料分析應用。

Published in: Data & Analytics
1 Comment
3 Likes
Statistics
Notes
  • 我記得目前所有用到GPU的Deep learning工具, 無論是Caffe, Tensorflow, Theano都沒有支援Intel HD Graphics. 只有Intel維護的Caffe版本有使用MKL加速和支援平行運算, 但也是以CPU為主, 請問哪裡可以找到以OpenCL開發的Deep learning套件呢?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
826
On SlideShare
0
From Embeds
0
Number of Embeds
444
Actions
Shares
0
Downloads
15
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide

Nas 也可以揀土豆

  1. 1. NAS 也可以揀土豆 Open source application
  2. 2. Hello! I am Cage Chung I am here because I like to share my experiences. You can find me at: QNAP 雲端應用部資深工程師 / http://kaichu.io
  3. 3. Andy
  4. 4. https://www.facebook.com/groups/GCPUG.TW/ https://plus.google.com/u/0/communities/116100913832589966421 Google Cloud Platform User Group Taiwan 我們是Google Cloud Platform Taiwan User Group。在Google雲端服務在台灣地區展露頭角之後, 有許多新的服務、新的知識、新的創意,歡迎大家一起分享,一起了解 Google雲端服務... GCPUG透過網際網路串聯喜好 Google Cloud的使用者,分享與交流使用 GCP的點滴鑑驗。如果您 是Google Cloud Platform的初學者,您應該來聽聽前輩們的使用經驗;如果您是 Google Cloud Platform的Expert,您應該來分享一下寶貴的經驗,並與更多高手互相交流;如果您還沒開始用 Google Cloud Platform,那麼您應該馬上來聽聽我們是怎麼使用 Google Cloud的!
  5. 5. “ Lessons learned building a classifier for NAS. Try to get a big picture, get some useful keywords I cannot explain everything, you cannot get every details
  6. 6. General user ○ Videos ○ Music ○ Movies ○ Photos ○ Files ○ Games Photographer ○ Photos (jpeg, RAW) Musicians ○ Music files (mp3) ○ Videos NAS | usage
  7. 7. Our scenario is easy Portrait landscape wildlife sports folk Retrain Inception classifier Photographer Product NAS
  8. 8. outline ◎ Machine learning ◎ Deep learning ○ Neural Network ○ Convolutional neural network ◎ Building a classifier for NAS ◎ Study information
  9. 9. 1. Machine Learning Let’s start with the first set of slides
  10. 10. Supervised Learning [image](http://www.safebee.com/family/5-healthy-hygiene-habits-your-child-needs-learn)
  11. 11. Supervised Learning workflow Raw Data Labes Feature Extraction Train the Model Eval Model Model Feature Extraction Predict New Data Model Labels Training Predicting
  12. 12. Supervised Learning workflow Raw Data Labes Feature Extraction Train the Model Eval Model Model Feature Extraction Predict New Data Model Labels Training Predicting Ripe Raw Color/Shape etc... Ripe Raw Ripe Raw
  13. 13. Unsupervised Learning [image](http://thoughtcatalog.com/nikolao-montaya/2014/06/how-to-be-cool-in-high-school/)
  14. 14. Ripe Ripe Raw Raw Raw Data Automated Clusters Learning Algorithm
  15. 15. [image](http://www.artbooms.com/blog/primo-toy-cubetto-robot-legno-programmazione-prescolare) Semi-Supervised Learning
  16. 16. Reinforcement Learning [image](https://www.shutterstock.com/search/horse%20carrot)
  17. 17. Reinforcement Learning (RL) [AlphaGo: Mastering the Game of Go with Deep Neural Networks and Tree …](http://www.slideshare.net/KarelHa1/alphago-mastering-the-game-of-go-with-deep-neural-networks-and-tree-search)
  18. 18. Mario - Machine Learning for Video Games 1 [NEATEvolve.lua - Pastebin.com](http://pastebin.com/ZZmSNaHX)
  19. 19. Mario - Machine Learning for Video Games 2 [NEATEvolve.lua - Pastebin.com](http://pastebin.com/ZZmSNaHX)
  20. 20. Smart programs can learn from examples [image](https://www.engadget.com/2016/07/12/machine-learning-ai/)
  21. 21. 2. Deep Learning Let’s start with the second set of slides
  22. 22. Deep learning [Deep Learning - Convolutional Neural Networks](http://www.slideshare.net/perone/deep-learning-convolutional-neural-networks)
  23. 23. Deep learning [自然言語処理のためのDeep Learning](http://www.slideshare.net/yutakikuchi927/deep-learning-26647407)
  24. 24. one architecture to rule them all [image](http://www.consultparagon.com/blog/what-is-leadership-digital-transformation)
  25. 25. 2.1. Neural Network
  26. 26. Neural Network Architecture [Neural networks and deep learning](http://neuralnetworksanddeeplearning.com/chap1.html)
  27. 27. First Example: MNIST handwritten digits
  28. 28. a few seconds 60,000 images MNIST resource Intel HD Graphics CPU build-in graphics Piece of cake ... 90% AccuracyFeeling good? But Google said it’s shameful ...
  29. 29. 2.2. Convolutional Neural Network The Google “Inception” deep neural network architecture. Source: Christian Szegedy et. al. Going deeper with convolutions. CVPR 2015.
  30. 30. WHAT IS CONVOLUTION? Source: http://deeplearning.stanford.edu/wiki/index.php/Feature_extraction_using_convolution Convolution with 3×3 Filter.
  31. 31. WHAT IS CONVOLUTION? Source: http://deeplearning.stanford.edu/wiki/index.php/Feature_extraction_using_convolution
  32. 32. Convolutional Neural Network, CNN [Understanding Convolutional Neural Networks for NLP – WildML](http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/)
  33. 33. Convolutional Neural Network, CNN
  34. 34. Convolutional Neural Network, CNN
  35. 35. becomes Peeking inside Convnets [Peeking inside Convnets | Audun M Øygard](https://auduno.github.io/2016/06/18/peeking-inside-convnets/)
  36. 36. a few hours 60,000 images MNIST resource Intel i5 2.5GhzCPU build-in graphics Not as easy as we think ... 99% AccuracyMy Goodness ...
  37. 37. Deep learning is a kind of neural network, and a neural network is a kind of machine learning. [image](https://www.hpcwire.com/2015/04/30/machine-learning-guru-sees-future-in-multi-gpu -clusters/)
  38. 38. 3. Building a classifier for NAS Let’s start with the third set of slides
  39. 39. Outline | Building a classifier for NAS ◎ How to train ? ○ Train from scratch ○ Re-train from inception modal ◎ Photo classifier Case study ○ Flickr Photos/ Google search ○ Fuji photography society monthly competition ○ Imagenet sources ◎ Image type classifier (photos、scan document、business card) ◎ Demo ◎ Next Steps ○ video post-processing? ○ Musicians? ◎ Search
  40. 40. 2 weeksSpend a lot of time 14,197,122 images ImageNet resource 8 NVIDIA Tesla K40s High-end professional graphics [Research Blog: Train your own image classifier with Inception in TensorFlow](https://research.googleblog.com/2016/03/train-your-own-image-classifier-with.html)
  41. 41. Retrain Inception's Final Layer for New Categories ◎ reuse Imagenet pre-trained model extract features to predict new tasks? ◎ Transfer learning ◎ General visual features DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
  42. 42. Retrain Inception's Final Layer for New Categories Cont. ◎ Installing and Running the TensorFlow Docker Image (gcr.io/tensorflow/tensorflow:latest-devel) ◎ Preparing target images ○ Quantity > 100 ○ representation ◎ Use Python to train your own image classifier ○ Distortions (--random_crop, --random_scale ects.) ○ Hyper-parameters (--learning_rate ects.) ◎ Classify images with your trained classifier [TensorFlow For Poets](https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/index.html?index=..%2F..%2Findex#0)
  43. 43. Case study: Types of Photography via Flickr Portrait landscape wildlife folk sport art
  44. 44. 68.8 %Final test accuracy 23,666 images Flickr photos & Google search Art x 4545 , Folk x 4782, Landscape x 3379 Portrait x 3706, Sport x 4967, Wildlife x 2887 5000 times Retain iterator
  45. 45. 89.0 %Final test accuracy 10,349 images Flickr photos Folk x 2083, Landscape x 3497 Portrait x 3215, Wildlife x 1554 4000 times Retain iterator
  46. 46. 97.0 %Final test accuracy 13,685 images Imagenet 11 categories Agaric/bolete/buckeye, horse chestnut, conker/coral fungus/ear, spike, capitulum/earthstar/gyromitra/hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa/peanut/stinkhorn, carrion fungus/toilet tissue, toilet paper, bathroom tissue/ 4000 times Retain iterator
  47. 47. Case study: Types of Photography via Fuji photography society (富士生活攝影協會月賽) Portrait landscape wildlife folk sport conceptual
  48. 48. Case study: Types of Photography via Fuji photography society (富士生活攝影協會月賽) competitor 銅牌 佳作 入選乙 入選甲 優選 金牌 銀牌 碩學會士/博學會士
  49. 49. 45.8 %Final test accuracy 424 images Fuji photography society monthly competition photos 佳作 x 32, 入選乙 x 137, 入選甲 x 255, 優選 x 2, 未入 選? 4000 times Retain iterator
  50. 50. Demo Custom Photo classifier
  51. 51. Image type classifier Invoices Business cards Scan documents
  52. 52. Next Steps | video [Large-scale Video Classification with Convolutional Neural Networks (CVPR 2014)](http://cs.stanford.edu/people/karpathy/deepvideo/)
  53. 53. Next Steps | music [Large-scale Video Classification with Convolutional Neural Networks (CVPR 2014)](http://cs.stanford.edu/people/karpathy/deepvideo/) [Deep Learning - Convolutional Neural Networks](http://www.slideshare.net/perone/deep-learning-convolutional-neural-networks)
  54. 54. Search
  55. 55. Open framework, models and worked examples for deep learning | Caffe Caffe offers the ○ model definitions ○ optimization settings ○ pre-trained weights so you can start right away. The BVLC models are licensed for unrestricted use. The community shares models in our Model Zoo. [DIY Deep Learning for Vision: a Hands-On Tutorial with Caffe - Google Slides](https://docs.google.com/presentation/d/1UeKXVgRvvxg9OUdh_UiC5G71UMscNPlvArsWER41PsU/edit#slide=id.gc2fcdcce7_216_48)
  56. 56. Brewing by the Numbers … | Caffe Speed with Krizhevsky's 2012 model: ○ 2 ms/image on K40 GPU ○ <1 ms inference with Caffe + cuDNN v4 on Titan X ○ 72 million images/day with batched IO ○ 8-core CPU: ~20 ms/image Intel optimization in progress 9k lines of C++ code (20k with tests) [DIY Deep Learning for Vision: a Hands-On Tutorial with Caffe - Google Slides](https://docs.google.com/presentation/d/1UeKXVgRvvxg9OUdh_UiC5G71UMscNPlvArsWER41PsU/edit#slide=id.gc2fcdcce7_216_48)
  57. 57. 4. Study information Let’s start with the fourth set of slides
  58. 58. ◎ [Deep Learning | Udacity](https://www.udacity.com/course/deep-learning--ud730) ◎ [Research Blog: Train your own image classifier with Inception in TensorFlow](https://research.googleblog.com/2016/03/train-your-own-image-cla ssifier-with.html) ◎ [jtoy/awesome-tensorflow: TensorFlow - A curated list of dedicated resources http://tensorflow.org](https://github.com/jtoy/awesome-tensorflow) ◎ [Deep Learning - Convolutional Neural Networks](http://www.slideshare.net/perone/deep-learning-convolutional-neura l-networks) ◎ [Neural networks and deep learning](http://neuralnetworksanddeeplearning.com/chap1.html) ◎ [Multiple Component Learning](http://valse.mmcheng.net/ftp/20150312/dsn.pdf) ◎ [Classifying Handwritten Digits with TF.Learn - Machine Learning Recipes #7 - YouTube](https://www.youtube.com/watch?v=Gj0iyo265bc) Study information
  59. 59. ◎ [DIGITS/GettingStarted.md at master · NVIDIA/DIGITS](https://github.com/NVIDIA/DIGITS/blob/master/docs/GettingStar ted.md) ◎ [How to Retrain Inception's Final Layer for New Categories](https://www.tensorflow.org/versions/r0.9/how_tos/image_retraining /index.html) ◎ [TensorFlow For Poets](https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/i ndex.html?index=..%2F..%2Findex#0) ◎ [DIY Deep Learning for Vision: a Hands-On Tutorial with Caffe - Google Slides](https://docs.google.com/presentation/d/1UeKXVgRvvxg9OUdh_UiC5G71 UMscNPlvArsWER41PsU/edit#slide=id.gc2fcdcce7_216_48) ◎ [Caffe | Deep Learning Framework](http://caffe.berkeleyvision.org/) ◎ [Understanding Convolutional Neural Networks for NLP – WildML](http://www.wildml.com/2015/11/understanding-convolutional-neural-n etworks-for-nlp/) Study information
  60. 60. Thanks! Any questions? You can find me at: http://kaichu.io cage.chung@gmail.com

×