Download as PDF, PPTX
Uploaded byYoss Cohen
Deep Learning - system view
The document provides an overview of deep learning, detailing its tools, types of models, and the processes involved in training and deploying these models. It highlights classifiers, box detectors, segmentors, and the importance of data gathering and optimization for effective deep learning applications. Additionally, it discusses various platforms for deployment and challenges in obtaining suitable datasets.
More Related Content
![Deep learning in python ommunic [CNN].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/deeplearninginpythoncnn-251207084743-a5c807e1-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Deep Learning - system view
Deep Learning - system view
- 1. 1 Deep Learning From blackmagic to structured programming
- 2. 2 Agenda ▪Introduction ▪Tools / ModelTypes ▪Classifiers to instance segmentors ▪Process ▪Program? Train? ▪Data ▪Where to get it? ▪Platforms ▪Optimization ▪Video Analytics
- 3. 3 Introduction What is DeepLearning?
- 4. 4 What is DeepLearning? ▪Deep Learning is not “Black Magic” ▪Deep Learning is using common statistics tools ▪Deep Learning is using iterative convergence process to find a local minimum point which minimize the loss (“best local solution”)
- 5. 5 What is DeepLearning ? Gradient Descent ! Neural Networks Convolutional Neural Networks Back Propagation!? X Impossible Math? X
- 6. 6 Deep Learning forsystem eng. & Architects ▪Set of Tools ▪Simple library ▪Development process ▪Non-standard programming
- 7.
- 8. 8 The tools ▪Most commontools: ▪Classifier ▪Detector (Box Detectors) ▪Segmentor ▪Instance Segmentor ▪Time series networks
- 9. 9 Classifiers - 2 ▪Answeron the existence of X (in image?) ▪Number classifier 0-9 (first CNN) ▪Common objects (Car, Chair, Cat…) ▪Cancer / No Cancer ▪Gun shot / broken glass ▪Most classifiers use some form of CNNs
- 10. 10 Classifiers ▪Single classifier couldclassify 100 objects ▪Most classifiers use COCO (~80 classes) as default ▪Example of La-net ▪First CNN for digits ▪Over 20 years old!!!
- 11. 11 BOX Detectors ▪Box detectorsdetects the location of the object ▪Used to detect: ▪People, face, corona mask ▪car, road signs ▪cats
- 12. 12 BOX Detectors ▪Box detectorscould detect more than one class ▪Box detectors could detect overlapping objects ▪Known box detectors: ▪YOLO3 (most used) YOLO5 (Latest common model) ▪Yolo2PP, YOLOR (Fast) ▪YOLACT-EDGE (fast and accurate) ▪SSD / SSD2 / Mobile-SSD
- 13. 13 Segmentor and Multi-InstanceSegmentor ▪Segmentor ▪defines class for each pixel ▪Pixel(X,Y) belongs to cat ▪FCN-8 basic implementation ▪Does not separate overlapping objects!!! ▪Common Segmentor U-Net
- 14. 14 Multi-Instance Segmentor ▪Multi-instance segmentor ▪Separatepixel to classes ▪Separate a class to instances ▪Pixel(X,Y) belongs to cat3
- 15. 15 Multi-Instance Segmentor -Implementation ▪Common / Simple implementation – MRCNN ▪Detectoron2 - Good implementation by Facebook
- 16. 16 (Time) series network ▪Runson sequential data ▪Number of visitors in site ▪ECG data ▪Solar spot data ▪Single variable Data ▪Multi Variable ▪Example: LSTM, RNN
- 17. 17 Landmarks extraction ▪Landmark extractionis done for: ▪Face recognition ▪Skeleton Pose estimation ▪Hand Gestures
- 18.
- 19.
- 20. 20 Data ▪Data gathering (Initial) ▪Cameras/ datasets ▪Data generation ▪Graphic engine (Unity / Omnviverse) ▪GAN ▪Data exploration ▪Anomaly detection ▪Class bias ▪Errored data ▪Data cleaning ▪Setting data pipes
- 21. 21 Training ▪Model selection Consideration ▪Performanceconstraints (run under < 1 TFLOPS) ▪Accuracy requirements ▪Model ▪Our own network ▪Existing network with/out weights ▪Training ▪No weights - from scratch ▪ Weights – Transfer learning ▪Hyper-param optimization ▪learning rate, decay, batch size, optimizer
- 22. 22 KPI – Measuringthe results ▪Classifier accuracy ▪Percentage detected out of total objects ▪False detection ▪Detector accuracy ▪Classifier accuracy + ▪Box Accuracy ▪Segmentor Accuracy ▪Class accuracy ▪Segmentation accuracy – IoU (Intersection of Union) ▪Instance segmentor accuracy…..
- 23. 23 Model Optimization ▪For whichplatform? CPU, GPU, TPU? ▪What type of GPU/TPU? ▪How many cores? ▪What type of cores (FP32, FP16, Int 16, Int8) ▪What batch to use?
- 24. 24 Deployment and Automation ▪Howto install the environment? ▪Python ▪Libraries ▪SDKs? ▪In Docker ? ▪How to automate the training and deployment platforms?
- 25.
- 26. 26 Data sources ▪Open-source datasets ▪Kaggle ▪Github ▪Buy tagged data ▪Tagging ourselves ▪Tagging service provider
- 27. 27 Dataset Challenges ▪Existence –does the data exists somewhere ▪Faces, numbers, clothing, animals, cars, fonts ▪Can we use the data to train freely? ▪Datasets Has license like code ▪Does it fit my needs? ▪Data is RGB we need IR ▪Data is from close range we need distant range ▪Good plants we need sick plants ▪Cancer non-cancer image ration is 1:4.3*10^5 ▪Data fine but too little
- 28. 28 Datasets ▪Know used datasets ▪MNIST– numbers ▪MNIST fashion – clothing ▪COCO – most used Dataset ▪Kaggle ▪Many dataset exists in Kaggle ▪Some closed for competition many open ▪Buy dataset ▪Create your own ▪Take images (10000? automate?) ▪Tag on your own / External service
- 29.
- 30. 30 Deployment and Automation ▪Forcode we have Git and Git lab for CI/CD ▪Can we use it for Deep Learning ▪Requirements ▪Manage data gathering ▪Store data versioning ▪Create and store model versioning ▪Automatic training + KPI Measurements ▪Automatic deployment
- 31. 31 Deep Learning Automation- 2 ▪MLOP Platforms ▪ClearML (Allegro) ▪Converge.IO (Intel) ▪Igazio ▪Neptune ▪SageMaker
- 32. 32 Optimization ▪CNN Models arerequire TOO much processing ▪Optimization process ▪Dev platform to single representation ▪Pruning ▪Convert type
- 33. 33 Accelerator software platforms ▪Modelporting and optimization ▪Nvidia TensorRT ▪Intel OpenVino ▪XILINX tools ▪Video processing and analytics platforms ▪NVIDIA DeepStream ▪Intel OpenVino ▪XILINX ▪All video analytics SDKs are based on GStreamer ▪Robotics ▪NVIDIA ISAAC (ROS / GEMS integration)
- 34.