Your SlideShare is downloading.
×

- 1. Machine Learning & Deep Learning Are Fun Toan Dang
- 2. Toan Dang Anh Technical Architect – NashTech Global • More than 15 years of experience in providing Database solutions for big corporations. • Rich experience using various solutions with real time processing, high availability and scale out. • Providing strategic solutions to various project about Database, Data Warehouse, BI, Big Data and Machine Learning & Deep Learning. • Email: datoan76@gmail.com • Skype: toan.dang
- 3. TABLE OF AGENDA Understanding Machine Learning Machine Learning Algorithms & Application Demo Machine Learning Why Deep Learning? Deep Learning Algorithms & Application Demo Deep Learning
- 4. 1. Understanding Machine Learning What is Machine Learning?
- 5. Understanding Machine Learning An application of artificial intelligence (AI) Learn and improve from experience Access data & use it learn for themselves Look for patterns in data and make better decisions
- 6. Artificial Intelligence History Dartmouth Assistant Professor John McCarthy 1956 1980 2010 2015 Machine Learning: offloading optimization Deep Learning Feature Learning Very Deep Learning Very Deep Networks with Skip Connections
- 7. Features Features are the variables found in the given problem set that can strongly/sufficiently help us build an accurate predictive model.
- 8. Features Weight(g) Wingspan(cm) Webbed feet? Back color Species 1 1000.1 125.0 No Brown Buteo jamaicensis 2 3000.7 200 No Gray Sagittarius 3 3300.0 220.3 No Gray Sagittarius 4 4100.0 136.0 Yes Black Gavia immer 5 3.0 11.0 No Green Calothorax lucifer 6 570.0 75.0 No Black Campephilus Principalis 1. Weight 2. Wingspan 3. Webbed feet 4. Back color Feature Name Feature Value 1. Numeric 2. Binary 3. Enumeration (color)
- 9. Training Set A training set is the set of training examples we’ll use to train our machine learning algorithms Weight(g) Wingspan(cm) Webbed feet? Back color Species 1 1000.1 125.0 No Brown Buteo jamaicensis 2 3000.7 200 No Gray Sagittarius 3 3300.0 220.3 No Gray Sagittarius 4 4100.0 136.0 Yes Black Gavia immer 5 3.0 11.0 No Green Calothorax lucifer 6 570.0 75.0 No Black Campephilus Principalis
- 10. Label - Target Variable o In classification the target variable takes on a nominal value o In the task of regression its value could be continuous o In a training set the target variable is known Weight(g) Wingspan(cm) Webbed feet? Back color Species 1 1000.1 125.0 No Brown Buteo jamaicensis 2 3000.7 200 No Gray Sagittarius 3 3300.0 220.3 No Gray Sagittarius 4 4100.0 136.0 Yes Black Gavia immer 5 3.0 11.0 No Green Calothorax lucifer 6 570.0 75.0 No Black Campephilus Principalis
- 11. Test Set Can split from Training set To test machine learning algorithms Must be separated with training set
- 12. Responsibility of MACHINE LEARNING In the classification problem the target variables are called classes Features Label
- 13. SIMPLE WORKFLOW Feature Extraction Label New/Test Data Training Model Model Label Raw Data (Train)
- 14. 2. Classification Algorithms & Application Some algorithms & Steps to build application
- 15. Unsupervised Learning Clustering Analysis K-Means Clustering Hierarchical Clustering Dimension Reduction Decision Tree K-Nearest Neighbors Supervised Learning Regression Linear Regression Logistic Regression Polynomial Regression Neural Networks Classification Decision Tree K-Nearest Neighbors Support vector machine Logistic Regression Naive Bayes Random Forests
- 16. Predict Nominal Value How to choose the right algorithm Predict or Groups Unsupervised Learning Groups what’s your target value Classification Continuous Value Regression Supervised Learning
- 17. Steps To Develop Machine Learning Applications
- 18. Steps To Develop Machine Learning Apps Define Object Data Collect Data Preparation Modeling Evaluation Deployment Monitor Operate
- 19. Process To Develop Application 1 What do we want to find out? Classify, predict or group Define Object 2 3 5 6 7 Evaluate how well the algorithm learned from its experience Evaluate Accuracy, lost error, validate errors Evaluate Model Data need to be gathered in an electronic format suitable for analysis. ETL, API, Web Scraping Data Collect Deploy model to API and use Deploy & Use Need features in a special format Need them to be integers Analyze data by Plotting Data Preparation Monitoring the model & the accuracy. Re- train model when need or data change. Improving model performance Monitor/Operate 4 Split Train & Test set. Feature Selection & Feature Engineering Training Model using right Algorithms. Integrate multi models, tuning parameters Model
- 20. Machine Learning Examples
- 21. Linear Regrestion
- 22. Logistic Regresion
- 23. Decision Tree Clinical Decision Tree - Weather Decision Tree Chatbot Decision Tree
- 24. SVM Face detect, Handwrite recognition Classification images
- 25. Association Rule Market Basket Analysis
- 26. 3. Demo Machine Learning Movie Recommendation
- 27. Collaborative Filtering Recommendation with User-Based & Item-Based Content-Based using TF-IDF (Terms-Frequency -Inverse Document Frequency)
- 28. Movie Recommendation Architecture 1.User access to website RDBMS 4.Build Model Collaborative filtering 2.Update DB 3.Send to Hadoop Recommendation (API) 8.Recommend Movies 118268, rating:5.2997 66389, rating:5.2092 173275, rating:5.1517 144202, rating:5.1516 117352, rating:5.1411 94101, rating:5.09185 ….. Ratings Model 10020 =>118268:5.2997 10020 => 66389:5.2092 10020 => 173275:5.15176.Deploy
- 29. Movie Recommendation. • User watch a movie on the website, which movie may be you like? System Recommendation
- 30. 4. DEEP LEARNING
- 31. Neural Network
- 32. Neural Network Face detection, Recognition, Translation, Object Detection Tracking moving object, Text classification, voice recognition … http://www.asimovinstitute.org/neural-network-zoo/
- 33. Recall Fully Connected Neural Network Total number of connections in this network?
- 34. Recall Fully Connected Neural Network Total number of connections in this network? = 50x50x3x2,000 + 2,000x2 15,004,000
- 35. Neural Network Problem 1. What exactly is deep learning ? 2. Why is it generally better than other methods on image, speech and certain other types of data? The short answers: ‘Deep Learning’ means using a neural network with several layers of nodes between input and output The series of layers between input & output do feature identification and processing in a series of stages, just as our brains seem to.
- 36. Difference Machine Learning & Deep Learning https://towardsdatascience.com/why-deep-learning-is-needed-over-traditional-machine-learning-1b6a99177063
- 37. AI vs ML vs DL https://goo.gl/6fKtGY
- 38. Deep Learning Comes In https://github.com/tavgreen/cnn-and-dnn
- 39. Deep Learning Was the first year that neural nets grew to prominence Alex Krizhevsky used them to win that year’s ImageNet Competition 2012 Dropping the classification error record from 26% to 15% 2012
- 40. Deep Learning Deep Learning Definition Methods based on learning data representations, as opposed to task-specific algorithms. Generally based on Artificial Neural Network
- 41. Deep Learning Feature learning - Representation learning In Machine Learning Set of techniques that allows a system to automatically discover the representations needed for feature detection or Classification from raw data This replaces manual feature engineering and allows a machine to both learn the features and use them to perform a specific task
- 42. Deep Learning Example Of Difference Representations https://towardsdatascience.com/deep-learning-d5fe55326e57
- 43. Deep Learning Learning data representations https://towardsdatascience.com/deep-learning-d5fe55326e57
- 44. 5. Deep Learning Algorithms & Application https://goo.gl/sD77JS
- 45. Convolutional Neural Network CNN - Almost Deep learning base on https://goo.gl/jHm3Hj
- 46. Convolutional Neural Network Convolution + ReLU Strides + Padding Pooling Layer Fully Connected Preserves the relationship between pixels by learning image features using small squares Calculated by input image matrix and a filter or kernel. Stride: number of pixels shifts over the input matrix Padding: filter does not fit perfectly fit the input image (drop or add zero) Reduce the number of parameters (Reduce dimension - Subsampling). Flattened our matrix into vector Apply activation function to classify output
- 47. Convolutional - Filters CNN - Almost Deep learning base on https://goo.gl/jHm3Hj
- 48. Convolutional Neural Network Example: ConvNetJS
- 49. Convolutional Neural Network Autoencoders & Decoders, R-CNN, Fast-RCNN, Faster-RCNN, Yolov1-3, SSD
- 50. Convolutional Neural Network Autoencoders (AE) & Decoder https://goo.gl/CBk86W
- 51. Convolutional Neural Network Object detection and segmentation R-CNN, Fast-RCNN, Faster-RCNN, Yolov1-3, SSD https://goo.gl/SuuQ46
- 52. Recurrent Neural Networks - RNN For sequential data and among others used by Apples Siri and Googles Voice Search
- 53. Recurrent Neural Networks Sequence Recurrent Memory Parameters Use of sequential information Perform the same task for every element of a sequence Limit to looking back only a few steps RNN shares the same parameters (U,V,W)
- 54. Two Problems Of RNN Exploding Gradients Assigned higher weights (gradients): The steeper the slope and the faster a model can learn Low accuracy Vanishing Gradients The gradient are too small: The model stops learning or takes way too long. Solved by LSTM
- 55. Recurrent Neural Networks - LSTM Improvement Of RNN
- 56. LSTMs reduce vanishing gradient problem - The darker the shade, the greater the sensitivity - The sensitivity decays exponentially over time as new inputs overwrite the activation of hidden unit and the network ‘forgets’ the first input Standard Recurrent Network LSTM Network
- 57. Reinforcement Learning How software agents ought to take actions in an environment so as to maximize some notion of cumulative reward
- 58. Reinforcement Learning
- 59. Reinforcement Learning Q-learning update Alpha (0<α≤1) is the extent to which our Q-values are being updated in every iteration Gamma: (0≤γ≤1): determines how much importance we want to give to future rewards. β: determines the sensitivity of the choice probabilities to difference in values to calculate Probability (Soft max)
- 60. Reinforcement Learning Value functions & policy gradients
- 61. Reinforcement Learning Deep Q-Network (DQN) vs Human
- 62. Reinforcement Learning Alphago: first release October 2015. At the 2017 Future of Go Summit, AlphaGo beat Ke Jie, the world No.1 ranked player at the time, in a three-game match
- 63. Frameworks
- 64. 6. Demo Deep Learning Image Captioning
- 65. Image Captioning Architecture
- 66. Image Captioning Architecture
- 67. Image Captioning Process Load dataset in batch and transform to PyTorch tensor 224x224x3. Batchsize=10 COCO dataset Process caption data and build vocabulary (words and count) Pre-processing Caption Load pretrained model Resnet50 Extract the features Connected to Decoder Layer CNN Encoder Do language modelling up to the word level Hidden state size 512, embedded size: 512 Output: model's predicted score RNN Decoder Using flask to expose API Develop Application to consume API Re-train model Expose API Load model & Vocabulary Transform input image Test sampler Generate prediction Prediction Apply BLEU to calculate model’s score Apply BEAM Search to calculate probability of words Generate sentence Validation Use pre-trained ResNet model (transfer learning) RNN with Embedding layer, a LSTM layer and a fully-connected layer Epochs: 20, CUDA, batchsize 32 Minimum word count: 5 Train Model
- 68. Demo: Image Captioning
- 69. THANK YOU www.b4uconference.com