Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
工業技術研究院機密資料 禁止複製、轉載、外流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE
Machine Learning, Deep
Learning and Data Analy...
Outline
2
• Overview of ML, DL and Data Analysis
• What is Machine Learning
▪ Take a Look At Linear Regression
▪ Other ML ...
My Experience for Machine Learning
3
• 學習過程走了一些冤望路!!
▪ Hope giving you an experience and guideline
• Take courses:
▪ Cours...
From AI to Deep Learning
4
• 推薦觀賞: 人工智能 极客公园 2017 年 大会演讲
▪ Google首席科学家李飛飛 https://www.youtube.com/watch?v=uZ-7DVzRCy8
http...
ML, DL and Data Analysis
5
• Visually Linking
• What we focus today
https://whatsthebigdata.com/2016/10/17/visually-linkin...
WHAT IS MACHINE
LEARNING
6
Machine Learning definition
7
• Arthur Samuel (1959). Machine Learning:
Field of study that gives computers the
ability to...
Machine Learning definition
8
• Suppose your email program watches
which emails you do or do not mark as
spam, and based o...
What is Machine Learning ?
• Without writing any
custom code
specific to the
problem
• Feed data to the
generic algorithm
...
Two styles of Machine Learning
• Supervised Learning 監督式學習
• Unsupervised Learning 非監督式學習
Use the logic to
predict the sal...
What are machine learning
algorithms?
• Regression Algorithms
▪ Linear Regression
▪ Logistic Regression
▪ LASSO
• Decision...
LET’S TAKE A LOOK AT
LINEAR REGRESSION
12
Linear Regression
13
• The Hypothesis Function
• Cost Function
• Gradient Descent for Multiple Variables
https://www.cours...
Gradient Descent
14
• How to choose learning α
https://www.coursera.org/learn/machine-learning/
Gradient Descent
15
• Convergence of gradient descent with an appropriate
learning rate α
Cost
Function
https://www.course...
Linear Regression
16
• Training data with linear regression fit
https://www.coursera.org/learn/machine-learning/
OTHER ML ALGORITHMS
AT A GLANCE
17
Logistic Regression
18
• Training data with decision boundary
linear decision boundary
no linear decision boundary
https:/...
Support Vector Machines
19
• The difference between the kernels in SVM
▪ Linear
▪ Polynomial
▪ Gaussian (RBF)
▪ Sigmoid
• ...
K-Means
20
• The original 128x128 image
with 24-bit color (three 8-bit )
• using K-means (K=16) to use
find the 16 colors ...
Principal Component Analysis
• An example to deal with image dimension
reduction and proximate recovery.
Faces Dataset
Rec...
WHAT IS NEURAL
NETWORK
(We will review the previous concepts a little bit)
22
ML -- write that program by
ourselves
• To estimate the price of a house
▪ If we could just figure out the perfect weights...
ML -- write that program by
ourselves
• Step 1
▪ Initialize weights to 1.0
• Step 2
▪ See the difference and how far off t...
ML -- What about that whole “try
every number” bit in Step 3?
θ is what represents your current weights. J(θ) means the ‘c...
Making Smarter Guesses
26
• We ended up with this simple estimation function
def estimate_house_sales_price(num_of_bedroom...
If there is more complicated situation?
▪ Different of weights for the different house sizes
What is a Neural Network
• Now we have four different price estimates.
• Let’s combine those four price estimates
into one...
What is a Neural Network
• Human Brains
http://www.slideshare.net/tw_dsconf/ss-62245351
What is a Neural Network
• Different
connections lead
to different
structured
network.
http://www.slideshare.net/tw_dsconf...
Fully Connected Feedforward
Network
http://www.slideshare.net/tw_dsconf/ss-62245351
Fully Connected Feedforward
Network
http://www.slideshare.net/tw_dsconf/ss-62245351
Fully connected feedforward network
• Matrix
Operation
http://www.slideshare.net/tw_dsconf/ss-62245351
Output Layer
• Softmax layer as the output layer
Neural Network Playground
• http://playground.tensorflow.org/
WHAT IS DEEP
LEARNING?
36
What is Deep Learning?
37
• Deep Learning is Large Neural Networks
• Deep Learning attracts lots of attention
http://stati...
Why Deep Learning?
• The more data, the more performance.
• Game Changer
▪ DL accuracy/performance is more than 99%
Deep Learning Models
39
• Convolutional Neural Network
▪ Inception-V3
• Recurrent Neural Network
▪ LSTM
• Auto-encoder
• R...
Deep Learning is not so simple
• Backpropagation
▪ an efficient way to compute Gradient Descent
• Overfitting
• Choosing L...
Backpropagation
41
• A common method of training artificial neural
networks and used in conjunction with an optimization
m...
Underfitting and Overfitting
• Bias-Variance Tradeoff
Convolutional Neural Network (CNN)
• Why CNN is for image?
▪ The first layer of fully connected network would be very
large
The solution is Convolution
The solution is Convolution
Convolutional Neural Network (CNN)
Adding Even More Steps
Convolutional Neural Network (CNN)
DEEP LEARNING USING
TENSORFLOW
48
Linear Regression in TensorFlow
49
X_data  array([ 0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8,
0.9, 1. , 1.1, 1.2, 1.3, ...
Linear Regression in TensorFlow
50
Linear Regression in TensorFlow
51
MNIST
52
• 手寫數字辨識
• MNIST dataset
▪ 55000 samples (50000 for training, 5000 for testing)
▪ For each sample, it has X, y pa...
MNIST
53
• X, y can be represented as follows
MNIST
54
• If you want to get the accuracy more than 99%, check it out:
• https://gotocon.com/dl/goto-london-2016/slides/M...
Image Recognition and Retraining
• Inception-v3 model is ready and made by Google
▪ it took Google researchers two weeks t...
Plate Number Recognition
• There is an example using UK’s Plate Number
and Character to train TensorFlow CNN model
• Take ...
Technical breakthrough for Deep-ANPR
http://matthewearl.github.io/2016/05/06/cnn-anpr/
Autoencoder
• Enccode the input data (MNIST data) and then decode it back
• It is similar to PCA
autoencoder
original
LSTM (RNN)
• It is a special kind of RNN, capable of learning
long-term dependencies
LSTM
Training with
Recurrent Neural Network
60
RNN Model
training
output
Play this map with Super Mario Maker
https://medium.com/@ageitgey/ma...
(BIG) DATA ANALYSIS
61
Data Analysis
62
• The steps to do data analysis
▪ Data Collection
a. From CSV files, database, … and so on.
▪ Data Prepro...
Analysis Tools and Libraries
63
• Open Sources(Python)
▪ Machine Learning
a. SciKit-Learn
b. NumPy
c. Matplotlib
d. Pandas...
Data Analysis
64
• In my experience with data analysis, I
belong to a “Rookie”…
• 製造資料科學:從預測性思維到處方性決策
▪ http://www.slidesh...
Reference
• Machine Learning is Fun! – Medium
• Machine Learning is Fun! Part 2 – Medium
• Machine Learning is Fun! Part 3...
Thank You
66
如果你了解了機器學習,看到下列這句話, 你會會心一笑。
Upcoming SlideShare
Loading in …5
×

Machine Learning, Deep Learning and Data Analysis Introduction

3,576 views

Published on

It gives a basic and overall introduction of Machine Learning, Deep Learning and Data Analysis.

Published in: Data & Analytics
  • 1 minute a day to keep your weight away!  https://tinyurl.com/1minweight4u
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Doctor's 2-Minute Ritual For Shocking Daily Belly Fat Loss! Watch This Video ♣♣♣ https://tinyurl.com/bkfitness4u
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Doctor's 2-Minute Ritual For Shocking Daily Belly Fat Loss! Watch This Video ◆◆◆ https://tinyurl.com/y6qaaou7
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • how to lose belly fat in 2 weeks without exercise-lose weight naturally ♣♣♣ https://tinyurl.com/bkfitness4u
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Machine Learning, Deep Learning and Data Analysis Introduction

  1. 1. 工業技術研究院機密資料 禁止複製、轉載、外流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE Machine Learning, Deep Learning and Data Analysis 簡介 劉得彥 teyen.liu@gmail.com
  2. 2. Outline 2 • Overview of ML, DL and Data Analysis • What is Machine Learning ▪ Take a Look At Linear Regression ▪ Other ML Algorithms at a Glance ▪ What is Neural Network? • What is Deep Learning? • Deep Learning using TensorFlow • Data Analysis ▪ Case 1, 2 and 3 ▪ Multivariate Analysis
  3. 3. My Experience for Machine Learning 3 • 學習過程走了一些冤望路!! ▪ Hope giving you an experience and guideline • Take courses: ▪ Coursera: Machine Learning ( Got Certificate ) ▪ Udemy: Data Science: Deep Learning in Python ( ongoing) • Study on-line resources: ▪ Youtube、ML/DL tutorials … and so on ▪ https://morvanzhou.github.io/ ▪ http://bangqu.com/gpu/blog ▪ http://www.jiqizhixin.com/insights • Get you hands dirty ▪ Python programming a. TensorFlow Deep Learning Library b. Scikit-Learn Library c. Numby, Pandas, matplotlib, …
  4. 4. From AI to Deep Learning 4 • 推薦觀賞: 人工智能 极客公园 2017 年 大会演讲 ▪ Google首席科学家李飛飛 https://www.youtube.com/watch?v=uZ-7DVzRCy8 https://blogs.nvidia.com.tw/2016/07/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/ CPU/ GPU Big Data Algorithms Breakthrough
  5. 5. ML, DL and Data Analysis 5 • Visually Linking • What we focus today https://whatsthebigdata.com/2016/10/17/visually-linking-ai-machine-learning-deep-learning-big-data-and-data-science/ 迷思?? Data Analysis
  6. 6. WHAT IS MACHINE LEARNING 6
  7. 7. Machine Learning definition 7 • Arthur Samuel (1959). Machine Learning: Field of study that gives computers the ability to learn without being explicitly programmed. • Tom Mitchell (1998) Well-posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.
  8. 8. Machine Learning definition 8 • Suppose your email program watches which emails you do or do not mark as spam, and based on that learns how to better filter spam. What is the task T in this setting? ▪ Classifying emails as spam or not spam. (T) ▪ Watching you label emails as spam or not spam. (E) ▪ The number (or fraction) of emails correctly classified as spam/not spam. (P) ▪ None of the above—this is not a machine learning problem
  9. 9. What is Machine Learning ? • Without writing any custom code specific to the problem • Feed data to the generic algorithm • It builds its own logic
  10. 10. Two styles of Machine Learning • Supervised Learning 監督式學習 • Unsupervised Learning 非監督式學習 Use the logic to predict the sales price figure out if there is a pattern or grouping or something Features Label
  11. 11. What are machine learning algorithms? • Regression Algorithms ▪ Linear Regression ▪ Logistic Regression ▪ LASSO • Decision Tree Algorithms ▪ Classification and Regression Tree (CART) ▪ Iterative Dichotomiser 3 (ID3) ▪ C4.5 and C5.0 (different versions of a powerful approach) • Bayesian Algorithms ▪ Naive Bayes • Clustering Algorithms (unsupervised) ▪ k-Means • Support Vector Machines • Principal Component Analysis • Anomaly Detection • Recommender Systems • Artificial Neural Network Algorithms
  12. 12. LET’S TAKE A LOOK AT LINEAR REGRESSION 12
  13. 13. Linear Regression 13 • The Hypothesis Function • Cost Function • Gradient Descent for Multiple Variables https://www.coursera.org/learn/machine-learning/
  14. 14. Gradient Descent 14 • How to choose learning α https://www.coursera.org/learn/machine-learning/
  15. 15. Gradient Descent 15 • Convergence of gradient descent with an appropriate learning rate α Cost Function https://www.coursera.org/learn/machine-learning/
  16. 16. Linear Regression 16 • Training data with linear regression fit https://www.coursera.org/learn/machine-learning/
  17. 17. OTHER ML ALGORITHMS AT A GLANCE 17
  18. 18. Logistic Regression 18 • Training data with decision boundary linear decision boundary no linear decision boundary https://www.coursera.org/learn/machine-learning/
  19. 19. Support Vector Machines 19 • The difference between the kernels in SVM ▪ Linear ▪ Polynomial ▪ Gaussian (RBF) ▪ Sigmoid • SVM (Gaussian Kernel) Decision Boundary ▪ Choose gamma ( auto ) Gaussian (RBF) Non-linear decision boundary https://www.coursera.org/learn/machine-learning/
  20. 20. K-Means 20 • The original 128x128 image with 24-bit color (three 8-bit ) • using K-means (K=16) to use find the 16 colors that best group (cluster) the pixels in the 3-dimensional RGB space. • K=3 and computing centroid means Iteratively https://www.coursera.org/learn/machine-learning/ 非監督式學習
  21. 21. Principal Component Analysis • An example to deal with image dimension reduction and proximate recovery. Faces Dataset Recovered faces Principal components 非監督式學習
  22. 22. WHAT IS NEURAL NETWORK (We will review the previous concepts a little bit) 22
  23. 23. ML -- write that program by ourselves • To estimate the price of a house ▪ If we could just figure out the perfect weights to use that work for every house, our function could predict house prices! ▪ How to do that with ML? def estimate_house_sales_price(num_of_bedrooms, sqft, neighborhood): price = 0 # a little pinch of this price += num_of_bedrooms * .841231951398213 price += sqft * 1231.1231231 price += neighborhood * 2.3242341421 price += 201.23432095 return price
  24. 24. ML -- write that program by ourselves • Step 1 ▪ Initialize weights to 1.0 • Step 2 ▪ See the difference and how far off the function is at guessing the correct price • Step 3 ▪ Repeat Step 2 over and over with every single possible combination of weights.
  25. 25. ML -- What about that whole “try every number” bit in Step 3? θ is what represents your current weights. J(θ) means the ‘cost for your current weights’. • Clever ways to quickly find good values for those weights without having to try very many. • If we graph this cost equation for all possible values of our weights for number_of_bedrooms and sqft Gradient Descent
  26. 26. Making Smarter Guesses 26 • We ended up with this simple estimation function def estimate_house_sales_price(num_of_bedrooms, sqft, neighborhood): price = 0 # a little pinch of this price += num_of_bedrooms * 0.123 price += sqft * 0.41 price += neighborhood * 0.57 return price a linear relationship with the input
  27. 27. If there is more complicated situation? ▪ Different of weights for the different house sizes
  28. 28. What is a Neural Network • Now we have four different price estimates. • Let’s combine those four price estimates into one final estimate. neurons This is a neural network
  29. 29. What is a Neural Network • Human Brains http://www.slideshare.net/tw_dsconf/ss-62245351
  30. 30. What is a Neural Network • Different connections lead to different structured network. http://www.slideshare.net/tw_dsconf/ss-62245351
  31. 31. Fully Connected Feedforward Network http://www.slideshare.net/tw_dsconf/ss-62245351
  32. 32. Fully Connected Feedforward Network http://www.slideshare.net/tw_dsconf/ss-62245351
  33. 33. Fully connected feedforward network • Matrix Operation http://www.slideshare.net/tw_dsconf/ss-62245351
  34. 34. Output Layer • Softmax layer as the output layer
  35. 35. Neural Network Playground • http://playground.tensorflow.org/
  36. 36. WHAT IS DEEP LEARNING? 36
  37. 37. What is Deep Learning? 37 • Deep Learning is Large Neural Networks • Deep Learning attracts lots of attention http://static.googleusercontent.com/media/research.google.com/en//people/jeff/BayLearn2015.pdf
  38. 38. Why Deep Learning? • The more data, the more performance. • Game Changer ▪ DL accuracy/performance is more than 99%
  39. 39. Deep Learning Models 39 • Convolutional Neural Network ▪ Inception-V3 • Recurrent Neural Network ▪ LSTM • Auto-encoder • Reinforcement Learning ▪ Q-Learning ▪ Policy Gradient • Wide and Deep Learning ▪ Recommender system
  40. 40. Deep Learning is not so simple • Backpropagation ▪ an efficient way to compute Gradient Descent • Overfitting • Choosing Loss function ▪ Square Error, Cross Entropy, and so on… • Mini-Batch • Too deep ( many hidden layers ) ▪ ReLU, MaxOut, … • Learning Rates • Momentum ▪ Adam ( optimizer ) • Weight Decay • Dropout
  41. 41. Backpropagation 41 • A common method of training artificial neural networks and used in conjunction with an optimization method such as gradient descent.
  42. 42. Underfitting and Overfitting • Bias-Variance Tradeoff
  43. 43. Convolutional Neural Network (CNN) • Why CNN is for image? ▪ The first layer of fully connected network would be very large
  44. 44. The solution is Convolution
  45. 45. The solution is Convolution
  46. 46. Convolutional Neural Network (CNN) Adding Even More Steps
  47. 47. Convolutional Neural Network (CNN)
  48. 48. DEEP LEARNING USING TENSORFLOW 48
  49. 49. Linear Regression in TensorFlow 49 X_data  array([ 0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2. , 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, … Y_data  array([ 0. , 0.29999667, 0.59997333, 0.89991 , 1.19978668, 1.49958339, 1.79928013, 2.09885695, 2.39829388, 2.69757098, 2.99666833, 3.29556602, …
  50. 50. Linear Regression in TensorFlow 50
  51. 51. Linear Regression in TensorFlow 51
  52. 52. MNIST 52 • 手寫數字辨識 • MNIST dataset ▪ 55000 samples (50000 for training, 5000 for testing) ▪ For each sample, it has X, y parts. ▪ X are the image with 28*28 pixels in 8 bit gray scale ▪ Y is the label answer: 0, 1, 2, …, 9
  53. 53. MNIST 53 • X, y can be represented as follows
  54. 54. MNIST 54 • If you want to get the accuracy more than 99%, check it out: • https://gotocon.com/dl/goto-london-2016/slides/MartinGorner_TensorflowAndDeepLearningWithoutAPhD.pdf 92%
  55. 55. Image Recognition and Retraining • Inception-v3 model is ready and made by Google ▪ it took Google researchers two weeks to build on a desktop with eight NVidia Tesla K40s. ▪ It can recognize > 1000 categories • Retraining ▪ To prepare the new images and categories ▪ Do training and testing
  56. 56. Plate Number Recognition • There is an example using UK’s Plate Number and Character to train TensorFlow CNN model • Take 3 days with GPU Card (GTX 750 TI) http://matthewearl.github.io/2016/05/06/cnn-anpr/
  57. 57. Technical breakthrough for Deep-ANPR http://matthewearl.github.io/2016/05/06/cnn-anpr/
  58. 58. Autoencoder • Enccode the input data (MNIST data) and then decode it back • It is similar to PCA autoencoder original
  59. 59. LSTM (RNN) • It is a special kind of RNN, capable of learning long-term dependencies LSTM Training with
  60. 60. Recurrent Neural Network 60 RNN Model training output Play this map with Super Mario Maker https://medium.com/@ageitgey/machine-learning-is-fun-part-2-a26a10b68df3
  61. 61. (BIG) DATA ANALYSIS 61
  62. 62. Data Analysis 62 • The steps to do data analysis ▪ Data Collection a. From CSV files, database, … and so on. ▪ Data Preprocessing ( very very very important ) a. Regularization, Normalization, … b. Table Join, ▪ Feature Extraction a. Reduce the dimensions …. ▪ Feature Selection a. Select the important features ▪ Machine Learning / Deep Learning a. To train the model b. To do the Prediction and Classification by the trained model c. Apply or implement to system • But, still needs: ▪ domain experts involved!! ▪ Studying related papers and researches
  63. 63. Analysis Tools and Libraries 63 • Open Sources(Python) ▪ Machine Learning a. SciKit-Learn b. NumPy c. Matplotlib d. Pandas ▪ Deep Learning a. TensorFlow b. Keras ▪ Hadoop & Spark • Commercial Software ( rare to use…) ▪ PolyAnalyst 6.5 ▪ SAS
  64. 64. Data Analysis 64 • In my experience with data analysis, I belong to a “Rookie”… • 製造資料科學:從預測性思維到處方性決策 ▪ http://www.slideshare.net/tw_dsconf/ss-71780267
  65. 65. Reference • Machine Learning is Fun! – Medium • Machine Learning is Fun! Part 2 – Medium • Machine Learning is Fun! Part 3: Deep Learning and ... - Medium • Deep Learning Tutorial • FIRST CONTACT WITH TENSORFLOW • https://ireneli.eu/2016/02/03/deep-learning-05-talk-about-convolutional-neural- network%EF%BC%88cnn%EF%BC%89/ • http://www.slideshare.net/tw_dsconf/ss-71780267 • https://morvanzhou.github.io/tutorials/python-basic/ • https://media.readthedocs.org/pdf/python-for-multivariate-analysis/latest/python-for- multivariate-analysis.pdf • http://blog.topspeedsnail.com/ • http://www.leiphone.com/news/201702/vJpJqREn7EyoAd09.html • Python 之機器學習套件 scikit-learn ▪ https://machine-learning-python.kspax.io/ ▪ version >= 0.17
  66. 66. Thank You 66 如果你了解了機器學習,看到下列這句話, 你會會心一笑。

×