Download as PDF, PPTX
Uploaded byEdge AI and Vision Alliance
“Getting Started with Vision AI Model Training,” a Presentation from NVIDIA
This document provides an introduction to model training for vision AI. It discusses how neural networks learn through gradient descent and backpropagation to minimize loss by adjusting weights and biases. Important factors that affect training like data preparation, hyperparameters, and preventing overfitting and underfitting are also covered. Finally, popular deep learning frameworks and community resources for learning more about machine learning are listed.
More Related Content
Similar to “Getting Started with Vision AI Model Training,” a Presentation from NVIDIA
“Getting Started with Vision AI Model Training,” a Presentation from NVIDIA
- 1. © 2021 NVIDIA GettingStarted with Vision AI Model Training Ekaterina Sirazitdinova NVIDIA
- 2. © 2021 NVIDIA Agenda •Introduction to deep learning training • How it works and how neural networks learn • What affects training • Important training parameters and problem mitigation • Data preparation • Essential steps for creating a deep learning data set • Where to start • Deep learning frameworks and community resources 2
- 3. © 2021 NVIDIA AI& Deep Learning are Changing the World 3 Robotics Manufacturing, construction, navigation Healthcare Cancer detection, drug discovery, genomics Internet Services Image classification, speech recognition, NLP Media & Entertainment Digital content creation Autonomous Vehicles Pedestrian & traffic sign detection, lane tracking Intelligent Video Analytics AI cities, urban safety
- 4. © 2021 NVIDIA HowDoes Deep Learning Work? 4 Trained model New capability Untrained neural network model Deep learning framework TRAINING Learning a new capability from existing data cat cat dog App or service Featuring capability INFERENCE Applying this capability to new data Trained model Optimized for performance cat ?
- 5. © 2021 NVIDIA ASimpler Case: Linear Model 5 ො 𝑦 𝑏𝑥 ො 𝑦 = 𝑏𝑥 + 𝑎 b = ? 𝑎 = ? Weights and biases are the learnable parameters of a machine learning model
- 6. © 2021 NVIDIA FromNeuron to Network 6 ො 𝑦 𝑤1 𝑤2 Input layer Hidden layer 1 Hidden layer 2 Output layer
- 7. © 2021 NVIDIA ActivationFunction Popular activation functions • ReLU • Tanh • Sigmoid • … 7 Input layer Hidden layer 1 Hidden layer 2 Output layer
- 8. How Does aNeural Network Learn? 8
- 9. © 2021 NVIDIA Loss 9 Inputlayer Hidden layer 1 Hidden layer 2 Output layer Ground truth 𝑙𝑜𝑠𝑠
- 10. © 2021 NVIDIA Backpropagation 10 Inputlayer Hidden layer 1 Hidden layer 2 Output layer Ground truth 𝑙𝑜𝑠𝑠
- 11. © 2021 NVIDIA GradientDescent • Gradient: which direction loss decreases the most • Learning rate: how far to travel • Step: an update to the weight parameters • Epoch: a model update with the full dataset • Batch: a sample of the full dataset • Momentum: accelerates the optimization process 11 b 𝑎
- 12. © 2021 NVIDIA Optimizers Somepopular optimizers • Adam • Adagrad • RMSprop • Stochastic gradient descent (SGD) • Madgrad 12 Global maxima Local minima Local maxima Global minima
- 13. What Can GoWrong and How to Make it Right? 13
- 14. © 2021 NVIDIA Underfitting– Just Right – Overfitting Regression Segmentation Deep learning loss curves loss epochs training validation loss epochs training validation loss epochs training validation 14
- 15. © 2021 NVIDIA PreventingOver- and Underfitting With the help of data • Add more examples • Check that your data set is balanced • Use a separate test set • Apply data augmentation 15 Using some techniques • Early stopping / increasing number of epochs • Changing network complexity • Regularization • Pruning • Dropout • Loss penalty (L1 and L2) • Ensembling • Transfer learning
- 16. It’s All AboutData 16
- 17. © 2021 NVIDIA PreparingDatasets for Deep Learning Training • Acquiring enough data • Ground truth labelling • Balancing classes • Splitting data into training, validation and testing sets Train Validate Training data Validation data Model after a training step 17
- 18.
- 19. © 2021 NVIDIA DeepLearning Frameworks: Your Starting Point 19
- 20. © 2021 NVIDIA Summary:In This Talk You Have Learned • What is a neural network and what does it mean to train it? • Important steps of the deep learning training process and parameters affecting it • What can go wrong and ways to make it right • How to prepare your data for training • What public resources you can use to master deep learning 20
- 21. © 2021 NVIDIA Resources 21 Communityresources Kaggle https://www.kaggle.com/ Papers with code https://paperswithcode.com/ MLPerf https://mlperf.org/ From the industry Google AI Education https://ai.google/education/ NVIDIA Devblog https://developer.nvidia.com/blog/ Machine learning course by Stanford (Coursera) https://www.coursera.org/learn/machi ne-learning AI Conferences NeurIPS https://nips.cc/ CVF conferences https://openaccess.thecvf.com/menu ICML https://icml.cc/
- 22.