Download to read offline
Uploaded byHoa Le
B4UConference_machine learning_deeplearning
The document discusses machine learning and deep learning, focusing on their definitions, algorithms, and applications. It covers essential concepts such as training sets, classification, feature extraction, and different types of machine learning algorithms, including supervised and unsupervised learning. Additionally, the document highlights real-world applications like movie recommendations and image captioning using deep learning methods.
More Related Content
Similar to B4UConference_machine learning_deeplearning
B4UConference_machine learning_deeplearning
- 1. Machine Learning &Deep Learning Are Fun Toan Dang
- 2. Toan Dang Anh TechnicalArchitect – NashTech Global • More than 15 years of experience in providing Database solutions for big corporations. • Rich experience using various solutions with real time processing, high availability and scale out. • Providing strategic solutions to various project about Database, Data Warehouse, BI, Big Data and Machine Learning & Deep Learning. • Email: datoan76@gmail.com • Skype: toan.dang
- 3. TABLE OF AGENDA UnderstandingMachine Learning Machine Learning Algorithms & Application Demo Machine Learning Why Deep Learning? Deep Learning Algorithms & Application Demo Deep Learning
- 4. 1. Understanding MachineLearning What is Machine Learning?
- 5. Understanding Machine Learning Anapplication of artificial intelligence (AI) Learn and improve from experience Access data & use it learn for themselves Look for patterns in data and make better decisions
- 6. Artificial Intelligence History DartmouthAssistant Professor John McCarthy 1956 1980 2010 2015 Machine Learning: offloading optimization Deep Learning Feature Learning Very Deep Learning Very Deep Networks with Skip Connections
- 7. Features Features are thevariables found in the given problem set that can strongly/sufficiently help us build an accurate predictive model.
- 8. Features Weight(g) Wingspan(cm) Webbedfeet? Back color Species 1 1000.1 125.0 No Brown Buteo jamaicensis 2 3000.7 200 No Gray Sagittarius 3 3300.0 220.3 No Gray Sagittarius 4 4100.0 136.0 Yes Black Gavia immer 5 3.0 11.0 No Green Calothorax lucifer 6 570.0 75.0 No Black Campephilus Principalis 1. Weight 2. Wingspan 3. Webbed feet 4. Back color Feature Name Feature Value 1. Numeric 2. Binary 3. Enumeration (color)
- 9. Training Set A trainingset is the set of training examples we’ll use to train our machine learning algorithms Weight(g) Wingspan(cm) Webbed feet? Back color Species 1 1000.1 125.0 No Brown Buteo jamaicensis 2 3000.7 200 No Gray Sagittarius 3 3300.0 220.3 No Gray Sagittarius 4 4100.0 136.0 Yes Black Gavia immer 5 3.0 11.0 No Green Calothorax lucifer 6 570.0 75.0 No Black Campephilus Principalis
- 10. Label - TargetVariable o In classification the target variable takes on a nominal value o In the task of regression its value could be continuous o In a training set the target variable is known Weight(g) Wingspan(cm) Webbed feet? Back color Species 1 1000.1 125.0 No Brown Buteo jamaicensis 2 3000.7 200 No Gray Sagittarius 3 3300.0 220.3 No Gray Sagittarius 4 4100.0 136.0 Yes Black Gavia immer 5 3.0 11.0 No Green Calothorax lucifer 6 570.0 75.0 No Black Campephilus Principalis
- 11. Test Set Can splitfrom Training set To test machine learning algorithms Must be separated with training set
- 12. Responsibility of MACHINELEARNING In the classification problem the target variables are called classes Features Label
- 13.
- 14. 2. Classification Algorithms& Application Some algorithms & Steps to build application
- 15. Unsupervised Learning Clustering Analysis K-MeansClustering Hierarchical Clustering Dimension Reduction Decision Tree K-Nearest Neighbors Supervised Learning Regression Linear Regression Logistic Regression Polynomial Regression Neural Networks Classification Decision Tree K-Nearest Neighbors Support vector machine Logistic Regression Naive Bayes Random Forests
- 16. Predict Nominal Value How tochoose the right algorithm Predict or Groups Unsupervised Learning Groups what’s your target value Classification Continuous Value Regression Supervised Learning
- 17. Steps To DevelopMachine Learning Applications
- 18. Steps To DevelopMachine Learning Apps Define Object Data Collect Data Preparation Modeling Evaluation Deployment Monitor Operate
- 19. Process To DevelopApplication 1 What do we want to find out? Classify, predict or group Define Object 2 3 5 6 7 Evaluate how well the algorithm learned from its experience Evaluate Accuracy, lost error, validate errors Evaluate Model Data need to be gathered in an electronic format suitable for analysis. ETL, API, Web Scraping Data Collect Deploy model to API and use Deploy & Use Need features in a special format Need them to be integers Analyze data by Plotting Data Preparation Monitoring the model & the accuracy. Re- train model when need or data change. Improving model performance Monitor/Operate 4 Split Train & Test set. Feature Selection & Feature Engineering Training Model using right Algorithms. Integrate multi models, tuning parameters Model
- 20.
- 21.
- 22.
- 23. Decision Tree Clinical DecisionTree - Weather Decision Tree Chatbot Decision Tree
- 24. SVM Face detect, Handwriterecognition Classification images
- 25. Association Rule Market BasketAnalysis
- 26. 3. Demo MachineLearning Movie Recommendation
- 27. Collaborative Filtering Recommendation withUser-Based & Item-Based Content-Based using TF-IDF (Terms-Frequency -Inverse Document Frequency)
- 28. Movie Recommendation Architecture 1.Useraccess to website RDBMS 4.Build Model Collaborative filtering 2.Update DB 3.Send to Hadoop Recommendation (API) 8.Recommend Movies 118268, rating:5.2997 66389, rating:5.2092 173275, rating:5.1517 144202, rating:5.1516 117352, rating:5.1411 94101, rating:5.09185 ….. Ratings Model 10020 =>118268:5.2997 10020 => 66389:5.2092 10020 => 173275:5.15176.Deploy
- 29. Movie Recommendation. • Userwatch a movie on the website, which movie may be you like? System Recommendation
- 30.
- 31.
- 32. Neural Network Face detection,Recognition, Translation, Object Detection Tracking moving object, Text classification, voice recognition … http://www.asimovinstitute.org/neural-network-zoo/
- 33. Recall Fully ConnectedNeural Network Total number of connections in this network?
- 34. Recall Fully ConnectedNeural Network Total number of connections in this network? = 50x50x3x2,000 + 2,000x2 15,004,000
- 35. Neural Network Problem 1.What exactly is deep learning ? 2. Why is it generally better than other methods on image, speech and certain other types of data? The short answers: ‘Deep Learning’ means using a neural network with several layers of nodes between input and output The series of layers between input & output do feature identification and processing in a series of stages, just as our brains seem to.
- 36. Difference Machine Learning& Deep Learning https://towardsdatascience.com/why-deep-learning-is-needed-over-traditional-machine-learning-1b6a99177063
- 37. AI vs MLvs DL https://goo.gl/6fKtGY
- 38. Deep Learning ComesIn https://github.com/tavgreen/cnn-and-dnn
- 39. Deep Learning Was thefirst year that neural nets grew to prominence Alex Krizhevsky used them to win that year’s ImageNet Competition 2012 Dropping the classification error record from 26% to 15% 2012
- 40. Deep Learning Deep LearningDefinition Methods based on learning data representations, as opposed to task-specific algorithms. Generally based on Artificial Neural Network
- 41. Deep Learning Feature learning- Representation learning In Machine Learning Set of techniques that allows a system to automatically discover the representations needed for feature detection or Classification from raw data This replaces manual feature engineering and allows a machine to both learn the features and use them to perform a specific task
- 42. Deep Learning Example OfDifference Representations https://towardsdatascience.com/deep-learning-d5fe55326e57
- 43. Deep Learning Learning datarepresentations https://towardsdatascience.com/deep-learning-d5fe55326e57
- 44. 5. Deep LearningAlgorithms & Application https://goo.gl/sD77JS
- 45. Convolutional Neural Network CNN- Almost Deep learning base on https://goo.gl/jHm3Hj
- 46. Convolutional Neural Network Convolution+ ReLU Strides + Padding Pooling Layer Fully Connected Preserves the relationship between pixels by learning image features using small squares Calculated by input image matrix and a filter or kernel. Stride: number of pixels shifts over the input matrix Padding: filter does not fit perfectly fit the input image (drop or add zero) Reduce the number of parameters (Reduce dimension - Subsampling). Flattened our matrix into vector Apply activation function to classify output
- 47. Convolutional - Filters CNN- Almost Deep learning base on https://goo.gl/jHm3Hj
- 48.
- 49. Convolutional Neural Network Autoencoders& Decoders, R-CNN, Fast-RCNN, Faster-RCNN, Yolov1-3, SSD
- 50. Convolutional Neural Network Autoencoders(AE) & Decoder https://goo.gl/CBk86W
- 51. Convolutional Neural Network Objectdetection and segmentation R-CNN, Fast-RCNN, Faster-RCNN, Yolov1-3, SSD https://goo.gl/SuuQ46
- 52. Recurrent Neural Networks- RNN For sequential data and among others used by Apples Siri and Googles Voice Search
- 53. Recurrent Neural Networks SequenceRecurrent Memory Parameters Use of sequential information Perform the same task for every element of a sequence Limit to looking back only a few steps RNN shares the same parameters (U,V,W)
- 54. Two Problems OfRNN Exploding Gradients Assigned higher weights (gradients): The steeper the slope and the faster a model can learn Low accuracy Vanishing Gradients The gradient are too small: The model stops learning or takes way too long. Solved by LSTM
- 55. Recurrent Neural Networks- LSTM Improvement Of RNN
- 56. LSTMs reduce vanishinggradient problem - The darker the shade, the greater the sensitivity - The sensitivity decays exponentially over time as new inputs overwrite the activation of hidden unit and the network ‘forgets’ the first input Standard Recurrent Network LSTM Network
- 57. Reinforcement Learning How softwareagents ought to take actions in an environment so as to maximize some notion of cumulative reward
- 58.
- 59. Reinforcement Learning Q-learning update Alpha(0<α≤1) is the extent to which our Q-values are being updated in every iteration Gamma: (0≤γ≤1): determines how much importance we want to give to future rewards. β: determines the sensitivity of the choice probabilities to difference in values to calculate Probability (Soft max)
- 60. Reinforcement Learning Value functions& policy gradients
- 61. Reinforcement Learning Deep Q-Network(DQN) vs Human
- 62. Reinforcement Learning Alphago: firstrelease October 2015. At the 2017 Future of Go Summit, AlphaGo beat Ke Jie, the world No.1 ranked player at the time, in a three-game match
- 63.
- 64. 6. Demo DeepLearning Image Captioning
- 65.
- 66.
- 67. Image Captioning Process Loaddataset in batch and transform to PyTorch tensor 224x224x3. Batchsize=10 COCO dataset Process caption data and build vocabulary (words and count) Pre-processing Caption Load pretrained model Resnet50 Extract the features Connected to Decoder Layer CNN Encoder Do language modelling up to the word level Hidden state size 512, embedded size: 512 Output: model's predicted score RNN Decoder Using flask to expose API Develop Application to consume API Re-train model Expose API Load model & Vocabulary Transform input image Test sampler Generate prediction Prediction Apply BLEU to calculate model’s score Apply BEAM Search to calculate probability of words Generate sentence Validation Use pre-trained ResNet model (transfer learning) RNN with Embedding layer, a LSTM layer and a fully-connected layer Epochs: 20, CUDA, batchsize 32 Minimum word count: 5 Train Model
- 68.
- 69.
Editor's Notes
- #4 The primary aim is to allow the computers learn automatically
- #5 AI: Human Intelligence Exhibited by Machines
- #6 The primary aim is to allow the computers learn automatically
- #7 Basic ‘AI’ has existed for decades, via rules-based programs that deliver rudimentary displays of ‘intelligence’ in specific contexts. Progress, however, has been limited — because algorithms to tackle many real-world problems are too complex for people to program by hand. Tower Hanoi, chess, 8 queens,… Father of AI: the beginning of AI research, Lisp program language for robotics, making intelligent machines. What if we could transfer the difficulty of making complex predictions — the data optimization and feature specification — from the programmer to the program? This is the promise of modern artificial intelligence. ML: algorithms for prediction engine. Deep Learning: feature learning (AI-> ML -> DL subset) 2012: best image classification by AlexNet (8 layers). 2015, improvement by VGG (16-19 layers), Inceptron (22 layers) with RNN & LSTM DEEPDREAM. Program of google, using CNN find & enhance image patern (generate image as 2010)
- #16 Reinforcement: Learning: learn from feedback or reward: play chess, car drive. Supervised & Unsuppervised
- #30 ~45.000 Movies ~26 million ratings ~300.000 movies
- #36 Multilayer neural network is not good to calculate for multi hidden layers and nodes. large matrix calculation capacity
- #43 On the right: can draw a line to separate two categories data easily On the left: impossible => easier to extract useful information when building classifiers or other predictors
- #44 each layer does some computation and stores its output in memory for the next layer to use. Lowest: color > edges -> …. => deep learning is an approach to find automatic solutions to problems that are intuitive to human beings
- #46 CL: Convolution involves the shift, multiply and sum operations. The main processing component of this layer is a filter or mask which is a matrix of weights Subsampling layer: reduce resolution Convolution ReLU images recognition, images classifications. Objects detections, recognition faces,…
- #47 FC: Apply softmax, sigmoid
- #48 CL: Convolution involves the shift, multiply and sum operations. The main processing component of this layer is a filter or mask which is a matrix of weights Subsampling layer: reduce resolution - Difference filters: edge detection, blur and sharpen
- #49 Some improvement of CNN: R-CNN, Fast-RCNN, Faster-RCNN, Yolov1-3, SSD
- #54 Xt: input, st: hidden state, st=f(U*xt+W*s(t-1)): tanh or ReLU, ot: output = softmax(Vst) Difference with NN – Input Independence, not sequence Traditional NN: difference parameters at each layer
- #59 Alpha (0<α≤1) is the extent to which our Q-values are being updated in every iteration Gamma: (0≤γ≤1): determines how much importance we want to give to future rewards. β: determines the sensitivity of the choice probabilities to difference in values to calculate Probability (Soft max)