SlideShare a Scribd company logo

“Optimization Techniques with Intel’s OpenVINO to Enhance Performance on Your Existing Hardware,” a Presentation from Intel

•
•
Edge AI and Vision Alliance
Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/06/optimization-techniques-with-intels-openvino-to-enhance-performance-on-your-existing-hardware-a-presentation-from-intel/ Nico Galoppo, Principal Engineer (substituting for Ansley Dunn, Product Marketing Manager), and Ryan Loney, Technical Product Manager, both of Intel, present the “Optimization Techniques with Intel’s OpenVINO to Enhance Performance on Your Existing Hardware” tutorial at the May 2022 Embedded Vision Summit. Whether you’re using TensorFlow, PyTorch or another framework, Galoppo and Loney show you optimization techniques to enhance performance on your existing hardware. With the OpenVINO Toolkit, built on the foundation of OneAPI, developers can utilize their own AI model or leverage one of the hundreds of pre-trained models available across vision and audio use cases. In this presentation, you’ll learn how the Neural Network Compression Framework provides optimal model training templates for performance boosts while preserving accuracy, and how the Model Optimizer reduces complexity and makes model conversion faster. Other areas explored by Galoppo and Loney include auto device discovery to enable automatic load balancing and how to optimize for latency or throughput based on your workload.

Technology
1 of 49
Download to read offline
1 Optimization Techniques with OpenVINO™ to Enhance Performance on Your Existing Hardware Nico Galoppo, Principal Engineer Ryan Loney, Technical Product Manager
The Challenge The Challenge © 2022 Intel 2
How do I Deploy My Great Neural Network? © 2022 Intel 3
GPU? © 2022 Intel 4
GPU? Cloud? © 2022 Intel 5
GPU? Cloud? © 2022 Intel 6
Maybe just CPU? © 2022 Intel 7
“But running on CPU is slow…” Almost Every Deep Learning Engineer Is it? © 2022 Intel 8
© 2022 Intel 9 The Solution
Visual Inference Optimization and Neural Network + NLP, Audio... © 2022 Intel Built on the foundation of
BUILD OPTIMIZE DEPLOY 3 2 1 Developer Journey © 2022 Intel 11
Optimized Performance CPU iGPU VPU © 2022 Intel 12
www.openvino.ai Installation Methods © 2022 Intel 13
Model Optimizer Get Your Model Run Model Optimizer IR For workloads and configurations visit www.intel.com/PerformanceIndex. Results may vary​.​ © 2022 Intel 14
For workloads and configurations please scan QR code. Results may vary​.​ 0 25 50 75 100 125 150 175 200 Intel® Core™ i3-8100 Intel® Core™ i5-8500 Intel® Core™ i7-8700T Intel® Core™ i7-1185G7 CPU-only Intel® Core™ i7-1185G7 iGPU-only Intel® Core™ i7-1185G7 CPU+iGPU Throughput [FPS] – FP32 resnet_50_TF [224x224] yolo-v3-tiny-tf [416x416] deeplabv3-TF [513x513] $ benchmark_app -m model_path -d device 15 Performance
For workloads and configurations visit www.intel.com/PerformanceIndex. Results may vary​.​ $ mo –-input_model model.onnx --data_type FP32 Get Your Model Run Model Optimizer IR FP32 FP16 © 2022 Intel 16 Model Optimizer
Neural Network (any format) Conv2D Add Relu6 Identity DepthwiseConv2DNative Mul Add Relu6 Identity © 2022 Intel 17
Intermediate Representation (IR) Convolution Add Clamp GroupConvolution Add Clamp © 2022 Intel 18
Post-Training Optimization Tool (POT) IR IR Dataset Post-Training Optimization Tool FP32 INT8 © 2022 Intel 19
© 2022 Intel 20 Neural Network Compression Framework (NNCF)
Quantization- Aware Training Mixed-Precision Quantization Filter Pruning © 2022 Intel 21 Neural Network Compression Framework (NNCF)
Neural Network Compression Framework (NNCF) Quantization- Aware Training Mixed-Precision Quantization Filter Pruning © 2022 Intel 22
OpenVINO Runtime from openvino.runtime import Core img = load_img() core = Core() model = core.read_model(model="model.xml", weights="model.bin") compiled_model = core.compile_model(model=model, device_name="CPU") output_layer = compiled_model.outputs[0] result = compiled_model([img])[output_layer] „dog” © 2022 Intel 23
compiled_model = core.compile_model(model=model, device_name="CPU") © 2022 Intel 24 Supported Devices
compiled_model = core.compile_model(model=model, device_name="CPU") CPU © 2022 Intel 25 Supported Devices
CPU GPU compiled_model = core.compile_model(model=model, device_name="CPU") © 2022 Intel 26 Supported Devices
CPU GPU MYRIAD compiled_model = core.compile_model(model=model, device_name="CPU") © 2022 Intel 27 Supported Devices
compiled_model = core.compile_model(model=model, device_name="CPU") CPU GPU MYRIAD HETERO CPU GPU Convolution Add Clamp GroupConvolution Add Clamp Convolution Add Convolution Add © 2022 Intel 28 Supported Devices
compiled_model = core.compile_model(model=model, device_name="CPU") CPU GPU MYRIAD HETERO MULTI CPU GPU Convolution Add Clamp GroupConvolution Add Clamp Convolution Add Convolution Add GPU Convolution Add Clamp GroupConvolution Add Clamp Convolution Add Convolution Add CPU Convolution Add Clamp GroupConvolution Add Clamp Convolution Add Convolution Add 29 Supported Devices
Supported Devices RUNTIME GPU CPU APPLICATION AUTO PLUGIN CPU PLUGIN GPU PLUGIN AUTO AUTO chooses the device AUTO sets device config based on hints AUTO handles the exec logic on multiple devices CPU GPU MYRIAD AUTO HETERO MULTI compiled_model = core.compile_model(model=model, device_name="CPU") 30
AUTO Device INFERENCE ON GPU INFERENCE ON CPU INFERENCE STOPS ON CPU COMPILE NETWORK FOR CPU CPU Workloads GPU Workloads 116.7 5,800 MS TIME APPLICATION LOGIC LOADING NN INFERENCE DEVICE SELECTION APPLICATION INFERENCE ON AUTO COMPILE / LOAD NETWORK FOR GPU © 2022 Intel 31
Input Data with Variable Shape? 32 “What is the weather going to be like today?”
How to handle variable input shapes? 33 Padding Multiple Precompiled Models Inefficient and Cumbersome!
Native Dynamic Shapes Support in OpenVINO™ 34 Dynamic in input image size
• Set-and-forget optimization knob – choose latency or throughput. • Completely portable between the devices • Throughput hint drives device-specific optimizations • Even works for AUTO! Portable Performance Hints 35
Automatic Batching 36 • How do you choose a good batch size? • Let the OpenVINO™ runtime decide for you! • No need to batch requests, runtime will do that too! • Batching can improve throughput with select devices and models.
300+ Pre-Trained + Optimized Models © 2022 Intel 37 Open Model Zoo
300+ Pre-Trained + Optimized Models 38 Open Model Zoo
300+ Pre-Trained + Optimized Models © 2022 Intel 39 Open Model Zoo
300+ Pre-Trained + Optimized Models © 2022 Intel 40 Open Model Zoo
300+ Pre-Trained + Optimized Models © 2022 Intel 41 Open Model Zoo
300+ Pre-Trained + Optimized Models © 2022 Intel 42 Open Model Zoo
300+ Pre-Trained + Optimized Models © 2022 Intel 43 Open Model Zoo
Jupyter Notebooks
Try it yourself! Learn more and download at openvino.ai Complete the Intel® Edge AI Certification! • Supercharge your career • Approx 20 hours of video + quizzes + coding
Deep Dive Session Thursday, May 19 Intel AI Developer Expo—Let's Build Something Wonderful Together Session: 3:00 – 5:30 pm Reception: 5:30 - 7:30 pm Location: Room 209/210 Afternoon snack and post-session Reception will be provided
Thank you!
Configuration Intel® Core™ i7-1185G7 Intel® Core™ i3-8100 Intel® Core™ i5-8500 Intel® Core™ i7-8700T Motherboard Intel Corporation internal/ Reference Validation Platform GIGABYTE* Z390 UD ASUS* PRIME Z370-A GIGABYTE* Z370M DS3H-CF CPU Intel® Core™ i7-1185G7 @ 3.00GHz Intel® Core™ i3-8100 CPU @ 3.60GHz Intel® Core™ i5-8500 CPU @ 3.00GHz Intel® Core™ i7-8700T CPU @ 2.40GHz Hyper Threading ON OFF OFF ON Turbo Setting ON OFF ON ON Memory 2 x 8 GB DDR4 3200MHz 4 x 8 GB DDR4 2400MHz 2 x 16 GB DDR4 2666MHz 4 x 16 GB DDR4 2400MHz Operating System Ubuntu* 18.04 LTS Ubuntu* 18.04 LTS Ubuntu* 18.04 LTS Ubuntu* 18.04 LTS Kernel Version 5.8.0-05-generic 5.3.0-24-generic 5.3.0-24-generic 5.3.0-24-generic BIOS Vendor Intel Corporation American Megatrends Inc.* American Megatrends Inc.* American Megatrends Inc.* BIOS Version TGLSFWI1.R00.3425. A00.2010162309 F8 2401 F14c BIOS Release 16-Oct-20 24-May-19 12-Jul-19 23-Mar-21 BIOS Settings Default Settings Select optimized default settings, set OS type to “other”, save and exit Select optimized default settings, save and exit Select optimized default settings, set OS type to “other”, save and exit Batch size 1 1 1 1 Precision FP32 FP32 FP32 FP32 Number of concurrent inference requests 4 4 3 4 Test Date 18-Jun-21 18-Jun-21 18-Jun-21 18-Jun-21 Rated maximum TDP/socket in Watt 28 65 65 35 48 © 2022 Intel Platform Configurations for Performance Benchmarks
Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex. Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details. No product or component can be absolutely secure. Intel technologies may require enabled hardware, software or service activation. Your costs and results may vary. Intel is committed to respecting human rights and avoiding complicity in human rights abuses. See Intel's Global Human Rights Principles. Intel's products and software are intended only to be used in applications that do not cause or contribute to a violation of an internationally recognized human right. © Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others. Notices and Disclaimers 49 © 2022 Intel

Recommended

superglue_slides.pdf
superglue_slides.pdfsuperglue_slides.pdf
superglue_slides.pdfXiaoguangHan3
 
Openvino ncs2
Openvino ncs2Openvino ncs2
Openvino ncs2艾鍗科技
 
Intermediate Intel® Distribution of OpenVINO™ toolkit for Computer Vision Ap...
Intermediate Intel® Distribution of OpenVINO™ toolkit for Computer Vision Ap... Intermediate Intel® Distribution of OpenVINO™ toolkit for Computer Vision Ap...
Intermediate Intel® Distribution of OpenVINO™ toolkit for Computer Vision Ap...kkaralek
 
Introduction to object detection
Introduction to object detectionIntroduction to object detection
Introduction to object detectionBrodmann17
 
Develop and optimize CV/DL applications with Intel OpenVINO toolkit
Develop and optimize CV/DL applications with Intel OpenVINO toolkitDevelop and optimize CV/DL applications with Intel OpenVINO toolkit
Develop and optimize CV/DL applications with Intel OpenVINO toolkitYury Gorbachev
 
FPGA Hardware Accelerator for Machine Learning
FPGA Hardware Accelerator for Machine Learning FPGA Hardware Accelerator for Machine Learning
FPGA Hardware Accelerator for Machine Learning Dr. Swaminathan Kathirvel
 
OpenVINO introduction
OpenVINO introductionOpenVINO introduction
OpenVINO introductionYury Gorbachev
 
Feature Engineering
Feature EngineeringFeature Engineering
Feature EngineeringSri Ambati
 

Recommended

superglue_slides.pdf
superglue_slides.pdfsuperglue_slides.pdf
superglue_slides.pdfXiaoguangHan3
 
Openvino ncs2
Openvino ncs2Openvino ncs2
Openvino ncs2艾鍗科技
 
Intermediate Intel® Distribution of OpenVINO™ toolkit for Computer Vision Ap...
Intermediate Intel® Distribution of OpenVINO™ toolkit for Computer Vision Ap... Intermediate Intel® Distribution of OpenVINO™ toolkit for Computer Vision Ap...
Intermediate Intel® Distribution of OpenVINO™ toolkit for Computer Vision Ap...kkaralek
 
Introduction to object detection
Introduction to object detectionIntroduction to object detection
Introduction to object detectionBrodmann17
 
Develop and optimize CV/DL applications with Intel OpenVINO toolkit
Develop and optimize CV/DL applications with Intel OpenVINO toolkitDevelop and optimize CV/DL applications with Intel OpenVINO toolkit
Develop and optimize CV/DL applications with Intel OpenVINO toolkitYury Gorbachev
 
FPGA Hardware Accelerator for Machine Learning
FPGA Hardware Accelerator for Machine Learning FPGA Hardware Accelerator for Machine Learning
FPGA Hardware Accelerator for Machine Learning Dr. Swaminathan Kathirvel
 
OpenVINO introduction
OpenVINO introductionOpenVINO introduction
OpenVINO introductionYury Gorbachev
 
Feature Engineering
Feature EngineeringFeature Engineering
Feature EngineeringSri Ambati
 
Deep learning
Deep learningDeep learning
Deep learningMohamed Loey
 
Deep Learning: a birds eye view
Deep Learning: a birds eye viewDeep Learning: a birds eye view
Deep Learning: a birds eye viewRoelof Pieters
 
Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...
Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...
Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...Universitat Politècnica de Catalunya
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Simplilearn
 
Quantum Computing
Quantum ComputingQuantum Computing
Quantum ComputingMinoarHossain
 
Xim $%
Xim $%Xim $%
Xim $%Azhar Ansari
 
One shot learning
One shot learningOne shot learning
One shot learningVuong Ho Ngoc
 
A brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsA brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsShunta Saito
 
History of deep learning
History of deep learningHistory of deep learning
History of deep learningayatan2
 
Introduction to Deep Learning (NVIDIA)
Introduction to Deep Learning (NVIDIA)Introduction to Deep Learning (NVIDIA)
Introduction to Deep Learning (NVIDIA)Rakuten Group, Inc.
 
Hardware Acceleration for Machine Learning
Hardware Acceleration for Machine LearningHardware Acceleration for Machine Learning
Hardware Acceleration for Machine LearningCastLabKAIST
 
An introduction to Machine Learning
An introduction to Machine LearningAn introduction to Machine Learning
An introduction to Machine Learningbutest
 
Attention Is All You Need
Attention Is All You NeedAttention Is All You Need
Attention Is All You NeedIllia Polosukhin
 
Transformers in Vision: From Zero to Hero
Transformers in Vision: From Zero to HeroTransformers in Vision: From Zero to Hero
Transformers in Vision: From Zero to HeroBill Liu
 
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...Simplilearn
 
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)Sergey Karayev
 
Deep Learning for Video: Action Recognition (UPC 2018)
Deep Learning for Video: Action Recognition (UPC 2018)Deep Learning for Video: Action Recognition (UPC 2018)
Deep Learning for Video: Action Recognition (UPC 2018)Universitat Politècnica de Catalunya
 
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object Detector
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object DetectorPR-270: PP-YOLO: An Effective and Efficient Implementation of Object Detector
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object DetectorJinwon Lee
 
Notes on attention mechanism
Notes on attention mechanismNotes on attention mechanism
Notes on attention mechanismKhang Pham
 
An AI accelerator ASIC architecture
An AI accelerator ASIC architectureAn AI accelerator ASIC architecture
An AI accelerator ASIC architectureKhanh Le
 
“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...
“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...
“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...Edge AI and Vision Alliance
 
Forts and Fights Scaling Performance on Unreal Engine*
Forts and Fights Scaling Performance on Unreal Engine*Forts and Fights Scaling Performance on Unreal Engine*
Forts and Fights Scaling Performance on Unreal Engine*Intel® Software
 

More Related Content

What's hot

Deep Learning: a birds eye view
Deep Learning: a birds eye viewDeep Learning: a birds eye view
Deep Learning: a birds eye viewRoelof Pieters
 
Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...
Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...
Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...Universitat Politècnica de Catalunya
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Simplilearn
 
A brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsA brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsShunta Saito
 
History of deep learning
History of deep learningHistory of deep learning
History of deep learningayatan2
 
Introduction to Deep Learning (NVIDIA)
Introduction to Deep Learning (NVIDIA)Introduction to Deep Learning (NVIDIA)
Introduction to Deep Learning (NVIDIA)Rakuten Group, Inc.
 
Hardware Acceleration for Machine Learning
Hardware Acceleration for Machine LearningHardware Acceleration for Machine Learning
Hardware Acceleration for Machine LearningCastLabKAIST
 
An introduction to Machine Learning
An introduction to Machine LearningAn introduction to Machine Learning
An introduction to Machine Learningbutest
 
Transformers in Vision: From Zero to Hero
Transformers in Vision: From Zero to HeroTransformers in Vision: From Zero to Hero
Transformers in Vision: From Zero to HeroBill Liu
 
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...Simplilearn
 
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)Sergey Karayev
 
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object Detector
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object DetectorPR-270: PP-YOLO: An Effective and Efficient Implementation of Object Detector
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object DetectorJinwon Lee
 
Notes on attention mechanism
Notes on attention mechanismNotes on attention mechanism
Notes on attention mechanismKhang Pham
 
An AI accelerator ASIC architecture
An AI accelerator ASIC architectureAn AI accelerator ASIC architecture
An AI accelerator ASIC architectureKhanh Le
 

What's hot (20)

Deep learning
Deep learningDeep learning
Deep learning
 
Deep Learning: a birds eye view
Deep Learning: a birds eye viewDeep Learning: a birds eye view
Deep Learning: a birds eye view
 
Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...
Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...
Object Detection with Deep Learning - Xavier Giro-i-Nieto - UPC School Barcel...
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
 
Quantum Computing
Quantum ComputingQuantum Computing
Quantum Computing
 
Xim $%
Xim $%Xim $%
Xim $%
 
One shot learning
One shot learningOne shot learning
One shot learning
 
A brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsA brief introduction to recent segmentation methods
A brief introduction to recent segmentation methods
 
History of deep learning
History of deep learningHistory of deep learning
History of deep learning
 
Introduction to Deep Learning (NVIDIA)
Introduction to Deep Learning (NVIDIA)Introduction to Deep Learning (NVIDIA)
Introduction to Deep Learning (NVIDIA)
 
Hardware Acceleration for Machine Learning
Hardware Acceleration for Machine LearningHardware Acceleration for Machine Learning
Hardware Acceleration for Machine Learning
 
An introduction to Machine Learning
An introduction to Machine LearningAn introduction to Machine Learning
An introduction to Machine Learning
 
Attention Is All You Need
Attention Is All You NeedAttention Is All You Need
Attention Is All You Need
 
Transformers in Vision: From Zero to Hero
Transformers in Vision: From Zero to HeroTransformers in Vision: From Zero to Hero
Transformers in Vision: From Zero to Hero
 
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...
 
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)
Lecture 4: Transformers (Full Stack Deep Learning - Spring 2021)
 
Deep Learning for Video: Action Recognition (UPC 2018)
Deep Learning for Video: Action Recognition (UPC 2018)Deep Learning for Video: Action Recognition (UPC 2018)
Deep Learning for Video: Action Recognition (UPC 2018)
 
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object Detector
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object DetectorPR-270: PP-YOLO: An Effective and Efficient Implementation of Object Detector
PR-270: PP-YOLO: An Effective and Efficient Implementation of Object Detector
 
Notes on attention mechanism
Notes on attention mechanismNotes on attention mechanism
Notes on attention mechanism
 
An AI accelerator ASIC architecture
An AI accelerator ASIC architectureAn AI accelerator ASIC architecture
An AI accelerator ASIC architecture
 

Similar to “Optimization Techniques with Intel’s OpenVINO to Enhance Performance on Your Existing Hardware,” a Presentation from Intel

“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...
“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...
“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...Edge AI and Vision Alliance
 
Forts and Fights Scaling Performance on Unreal Engine*
Forts and Fights Scaling Performance on Unreal Engine*Forts and Fights Scaling Performance on Unreal Engine*
Forts and Fights Scaling Performance on Unreal Engine*Intel® Software
 
“Acceleration of Deep Learning Using OpenVINO: 3D Seismic Case Study,” a Pres...
“Acceleration of Deep Learning Using OpenVINO: 3D Seismic Case Study,” a Pres...“Acceleration of Deep Learning Using OpenVINO: 3D Seismic Case Study,” a Pres...
“Acceleration of Deep Learning Using OpenVINO: 3D Seismic Case Study,” a Pres...Edge AI and Vision Alliance
 
Software Development Tools for Intel® IoT Platforms
Software Development Tools for Intel® IoT PlatformsSoftware Development Tools for Intel® IoT Platforms
Software Development Tools for Intel® IoT PlatformsIntel® Software
 
“Intel Video AI Box—Converging AI, Media and Computing in a Compact and Open ...
“Intel Video AI Box—Converging AI, Media and Computing in a Compact and Open ...“Intel Video AI Box—Converging AI, Media and Computing in a Compact and Open ...
“Intel Video AI Box—Converging AI, Media and Computing in a Compact and Open ...Edge AI and Vision Alliance
 
Tuning For Deep Learning Inference with Intel® Processor Graphics | SIGGRAPH ...
Tuning For Deep Learning Inference with Intel® Processor Graphics | SIGGRAPH ...Tuning For Deep Learning Inference with Intel® Processor Graphics | SIGGRAPH ...
Tuning For Deep Learning Inference with Intel® Processor Graphics | SIGGRAPH ...Intel® Software
 
Как выбрать оптимальную серверную архитектуру для создания высокоэффективных ЦОД
Как выбрать оптимальную серверную архитектуру для создания высокоэффективных ЦОДКак выбрать оптимальную серверную архитектуру для создания высокоэффективных ЦОД
Как выбрать оптимальную серверную архитектуру для создания высокоэффективных ЦОДNick Turunov
 
【視覺進化論】AI智慧視覺運算技術論壇_2_ChungYeh
【視覺進化論】AI智慧視覺運算技術論壇_2_ChungYeh【視覺進化論】AI智慧視覺運算技術論壇_2_ChungYeh
【視覺進化論】AI智慧視覺運算技術論壇_2_ChungYehMAKERPRO.cc
 
Технологии Intel для виртуализации сетей операторов связи
Технологии Intel для виртуализации сетей операторов связиТехнологии Intel для виртуализации сетей операторов связи
Технологии Intel для виртуализации сетей операторов связиCisco Russia
 
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Fin...
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Fin... Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Fin...
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Fin...Intel IT Center
 
Optimizing Direct X On Multi Core Architectures
Optimizing Direct X On Multi Core ArchitecturesOptimizing Direct X On Multi Core Architectures
Optimizing Direct X On Multi Core Architecturespsteinb
 
Intel® Select Solutions for the Network
Intel® Select Solutions for the NetworkIntel® Select Solutions for the Network
Intel® Select Solutions for the NetworkLiz Warner
 
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSci
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSciStreamline End-to-End AI Pipelines with Intel, Databricks, and OmniSci
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSciIntel® Software
 
Media and entertainment workload comparison: HP Z8 vs. Apple Mac Pro
Media and entertainment workload comparison: HP Z8 vs. Apple Mac ProMedia and entertainment workload comparison: HP Z8 vs. Apple Mac Pro
Media and entertainment workload comparison: HP Z8 vs. Apple Mac ProPrincipled Technologies
 
Achieve Business-Class Performance With Intel vPro, an Intel Evo Design
Achieve Business-Class Performance With Intel vPro, an Intel Evo DesignAchieve Business-Class Performance With Intel vPro, an Intel Evo Design
Achieve Business-Class Performance With Intel vPro, an Intel Evo DesignAdele Noble
 
Accelerate Your Game Development on Android*
Accelerate Your Game Development on Android*Accelerate Your Game Development on Android*
Accelerate Your Game Development on Android*Intel® Software
 
Intel Performance Workshop @ CES 2020
Intel Performance Workshop @ CES 2020Intel Performance Workshop @ CES 2020
Intel Performance Workshop @ CES 2020Low Hong Chuan
 
Open Source Firmware - FrOSCon 2019
Open Source Firmware - FrOSCon 2019Open Source Firmware - FrOSCon 2019
Open Source Firmware - FrOSCon 2019Daniel Maslowski
 
"How to Get the Best Deep Learning Performance with the OpenVINO Toolkit," a ...
"How to Get the Best Deep Learning Performance with the OpenVINO Toolkit," a ..."How to Get the Best Deep Learning Performance with the OpenVINO Toolkit," a ...
"How to Get the Best Deep Learning Performance with the OpenVINO Toolkit," a ...Edge AI and Vision Alliance
 

Similar to “Optimization Techniques with Intel’s OpenVINO to Enhance Performance on Your Existing Hardware,” a Presentation from Intel (20)

“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...
“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...
“Smarter Manufacturing with Intel’s Deep Learning-Based Machine Vision,” a Pr...
 
Forts and Fights Scaling Performance on Unreal Engine*
Forts and Fights Scaling Performance on Unreal Engine*Forts and Fights Scaling Performance on Unreal Engine*
Forts and Fights Scaling Performance on Unreal Engine*
 
“Acceleration of Deep Learning Using OpenVINO: 3D Seismic Case Study,” a Pres...
“Acceleration of Deep Learning Using OpenVINO: 3D Seismic Case Study,” a Pres...“Acceleration of Deep Learning Using OpenVINO: 3D Seismic Case Study,” a Pres...
“Acceleration of Deep Learning Using OpenVINO: 3D Seismic Case Study,” a Pres...
 
Software Development Tools for Intel® IoT Platforms
Software Development Tools for Intel® IoT PlatformsSoftware Development Tools for Intel® IoT Platforms
Software Development Tools for Intel® IoT Platforms
 
“Intel Video AI Box—Converging AI, Media and Computing in a Compact and Open ...
“Intel Video AI Box—Converging AI, Media and Computing in a Compact and Open ...“Intel Video AI Box—Converging AI, Media and Computing in a Compact and Open ...
“Intel Video AI Box—Converging AI, Media and Computing in a Compact and Open ...
 
Tuning For Deep Learning Inference with Intel® Processor Graphics | SIGGRAPH ...
Tuning For Deep Learning Inference with Intel® Processor Graphics | SIGGRAPH ...Tuning For Deep Learning Inference with Intel® Processor Graphics | SIGGRAPH ...
Tuning For Deep Learning Inference with Intel® Processor Graphics | SIGGRAPH ...
 
Как выбрать оптимальную серверную архитектуру для создания высокоэффективных ЦОД
Как выбрать оптимальную серверную архитектуру для создания высокоэффективных ЦОДКак выбрать оптимальную серверную архитектуру для создания высокоэффективных ЦОД
Как выбрать оптимальную серверную архитектуру для создания высокоэффективных ЦОД
 
【視覺進化論】AI智慧視覺運算技術論壇_2_ChungYeh
【視覺進化論】AI智慧視覺運算技術論壇_2_ChungYeh【視覺進化論】AI智慧視覺運算技術論壇_2_ChungYeh
【視覺進化論】AI智慧視覺運算技術論壇_2_ChungYeh
 
Технологии Intel для виртуализации сетей операторов связи
Технологии Intel для виртуализации сетей операторов связиТехнологии Intel для виртуализации сетей операторов связи
Технологии Intel для виртуализации сетей операторов связи
 
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Fin...
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Fin... Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Fin...
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Fin...
 
Optimizing Direct X On Multi Core Architectures
Optimizing Direct X On Multi Core ArchitecturesOptimizing Direct X On Multi Core Architectures
Optimizing Direct X On Multi Core Architectures
 
Intel® Select Solutions for the Network
Intel® Select Solutions for the NetworkIntel® Select Solutions for the Network
Intel® Select Solutions for the Network
 
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSci
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSciStreamline End-to-End AI Pipelines with Intel, Databricks, and OmniSci
Streamline End-to-End AI Pipelines with Intel, Databricks, and OmniSci
 
Media and entertainment workload comparison: HP Z8 vs. Apple Mac Pro
Media and entertainment workload comparison: HP Z8 vs. Apple Mac ProMedia and entertainment workload comparison: HP Z8 vs. Apple Mac Pro
Media and entertainment workload comparison: HP Z8 vs. Apple Mac Pro
 
Achieve Business-Class Performance With Intel vPro, an Intel Evo Design
Achieve Business-Class Performance With Intel vPro, an Intel Evo DesignAchieve Business-Class Performance With Intel vPro, an Intel Evo Design
Achieve Business-Class Performance With Intel vPro, an Intel Evo Design
 
Accelerate Your Game Development on Android*
Accelerate Your Game Development on Android*Accelerate Your Game Development on Android*
Accelerate Your Game Development on Android*
 
Intel Performance Workshop @ CES 2020
Intel Performance Workshop @ CES 2020Intel Performance Workshop @ CES 2020
Intel Performance Workshop @ CES 2020
 
Open Source Firmware - FrOSCon 2019
Open Source Firmware - FrOSCon 2019Open Source Firmware - FrOSCon 2019
Open Source Firmware - FrOSCon 2019
 
Intel 6th Gen vPro
Intel 6th Gen vProIntel 6th Gen vPro
Intel 6th Gen vPro
 
"How to Get the Best Deep Learning Performance with the OpenVINO Toolkit," a ...
"How to Get the Best Deep Learning Performance with the OpenVINO Toolkit," a ..."How to Get the Best Deep Learning Performance with the OpenVINO Toolkit," a ...
"How to Get the Best Deep Learning Performance with the OpenVINO Toolkit," a ...
 

More from Edge AI and Vision Alliance

“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...Edge AI and Vision Alliance
 
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...Edge AI and Vision Alliance
 
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...Edge AI and Vision Alliance
 
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...Edge AI and Vision Alliance
 
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...Edge AI and Vision Alliance
 
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...Edge AI and Vision Alliance
 
“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...Edge AI and Vision Alliance
 
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsightsEdge AI and Vision Alliance
 
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...Edge AI and Vision Alliance
 
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...Edge AI and Vision Alliance
 
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...Edge AI and Vision Alliance
 
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...Edge AI and Vision Alliance
 
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...Edge AI and Vision Alliance
 
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...Edge AI and Vision Alliance
 
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...Edge AI and Vision Alliance
 
“Updating the Edge ML Development Process,” a Presentation from Samsara
“Updating the Edge ML Development Process,” a Presentation from Samsara“Updating the Edge ML Development Process,” a Presentation from Samsara
“Updating the Edge ML Development Process,” a Presentation from SamsaraEdge AI and Vision Alliance
 
“Combating Bias in Production Computer Vision Systems,” a Presentation from R...
“Combating Bias in Production Computer Vision Systems,” a Presentation from R...“Combating Bias in Production Computer Vision Systems,” a Presentation from R...
“Combating Bias in Production Computer Vision Systems,” a Presentation from R...Edge AI and Vision Alliance
 
“Developing an Embedded Vision AI-powered Fitness System,” a Presentation fro...
“Developing an Embedded Vision AI-powered Fitness System,” a Presentation fro...“Developing an Embedded Vision AI-powered Fitness System,” a Presentation fro...
“Developing an Embedded Vision AI-powered Fitness System,” a Presentation fro...Edge AI and Vision Alliance
 
“Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups,” a ...
“Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups,” a ...“Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups,” a ...
“Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups,” a ...Edge AI and Vision Alliance
 
“Advanced Presence Sensing: What It Means for the Smart Home,” a Presentation...
“Advanced Presence Sensing: What It Means for the Smart Home,” a Presentation...“Advanced Presence Sensing: What It Means for the Smart Home,” a Presentation...
“Advanced Presence Sensing: What It Means for the Smart Home,” a Presentation...Edge AI and Vision Alliance
 

More from Edge AI and Vision Alliance (20)

“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
 
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
 
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
 
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
 
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
 
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
 
“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...
 
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
“ADAS and AV Sensors: What’s Winning and Why?,” a Presentation from TechInsights
 
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
“Computer Vision in Sports: Scalable Solutions for Downmarkets,” a Presentati...
 
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
“Detecting Data Drift in Image Classification Neural Networks,” a Presentatio...
 
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
“Deep Neural Network Training: Diagnosing Problems and Implementing Solutions...
 
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
“AI Start-ups: The Perils of Fishing for Whales (War Stories from the Entrepr...
 
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
“A Computer Vision System for Autonomous Satellite Maneuvering,” a Presentati...
 
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
“Bias in Computer Vision—It’s Bigger Than Facial Recognition!,” a Presentatio...
 
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
“Sensor Fusion Techniques for Accurate Perception of Objects in the Environme...
 
“Updating the Edge ML Development Process,” a Presentation from Samsara
“Updating the Edge ML Development Process,” a Presentation from Samsara“Updating the Edge ML Development Process,” a Presentation from Samsara
“Updating the Edge ML Development Process,” a Presentation from Samsara
 
“Combating Bias in Production Computer Vision Systems,” a Presentation from R...
“Combating Bias in Production Computer Vision Systems,” a Presentation from R...“Combating Bias in Production Computer Vision Systems,” a Presentation from R...
“Combating Bias in Production Computer Vision Systems,” a Presentation from R...
 
“Developing an Embedded Vision AI-powered Fitness System,” a Presentation fro...
“Developing an Embedded Vision AI-powered Fitness System,” a Presentation fro...“Developing an Embedded Vision AI-powered Fitness System,” a Presentation fro...
“Developing an Embedded Vision AI-powered Fitness System,” a Presentation fro...
 
“Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups,” a ...
“Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups,” a ...“Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups,” a ...
“Navigating the Evolving Venture Capital Landscape for Edge AI Start-ups,” a ...
 
“Advanced Presence Sensing: What It Means for the Smart Home,” a Presentation...
“Advanced Presence Sensing: What It Means for the Smart Home,” a Presentation...“Advanced Presence Sensing: What It Means for the Smart Home,” a Presentation...
“Advanced Presence Sensing: What It Means for the Smart Home,” a Presentation...
 

Recently uploaded

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsRoshan Dwivedi
 
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Paola De la Torre
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 

Recently uploaded (20)

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 

“Optimization Techniques with Intel’s OpenVINO to Enhance Performance on Your Existing Hardware,” a Presentation from Intel

  • 1. 1 Optimization Techniques with OpenVINO™ to Enhance Performance on Your Existing Hardware Nico Galoppo, Principal Engineer Ryan Loney, Technical Product Manager
  • 3. How do I Deploy My Great Neural Network? © 2022 Intel 3
  • 7. Maybe just CPU? © 2022 Intel 7
  • 8. “But running on CPU is slow…” Almost Every Deep Learning Engineer Is it? © 2022 Intel 8
  • 9. © 2022 Intel 9 The Solution
  • 11. BUILD OPTIMIZE DEPLOY 3 2 1 Developer Journey © 2022 Intel 11
  • 12. Optimized Performance CPU iGPU VPU © 2022 Intel 12
  • 14. Model Optimizer Get Your Model Run Model Optimizer IR For workloads and configurations visit www.intel.com/PerformanceIndex. Results may vary​.​ © 2022 Intel 14
  • 15. For workloads and configurations please scan QR code. Results may vary​.​ 0 25 50 75 100 125 150 175 200 Intel® Core™ i3-8100 Intel® Core™ i5-8500 Intel® Core™ i7-8700T Intel® Core™ i7-1185G7 CPU-only Intel® Core™ i7-1185G7 iGPU-only Intel® Core™ i7-1185G7 CPU+iGPU Throughput [FPS] – FP32 resnet_50_TF [224x224] yolo-v3-tiny-tf [416x416] deeplabv3-TF [513x513] $ benchmark_app -m model_path -d device 15 Performance
  • 16. For workloads and configurations visit www.intel.com/PerformanceIndex. Results may vary​.​ $ mo –-input_model model.onnx --data_type FP32 Get Your Model Run Model Optimizer IR FP32 FP16 © 2022 Intel 16 Model Optimizer
  • 17. Neural Network (any format) Conv2D Add Relu6 Identity DepthwiseConv2DNative Mul Add Relu6 Identity © 2022 Intel 17
  • 18. Intermediate Representation (IR) Convolution Add Clamp GroupConvolution Add Clamp © 2022 Intel 18
  • 19. Post-Training Optimization Tool (POT) IR IR Dataset Post-Training Optimization Tool FP32 INT8 © 2022 Intel 19
  • 20. © 2022 Intel 20 Neural Network Compression Framework (NNCF)
  • 21. Quantization- Aware Training Mixed-Precision Quantization Filter Pruning © 2022 Intel 21 Neural Network Compression Framework (NNCF)
  • 22. Neural Network Compression Framework (NNCF) Quantization- Aware Training Mixed-Precision Quantization Filter Pruning © 2022 Intel 22
  • 23. OpenVINO Runtime from openvino.runtime import Core img = load_img() core = Core() model = core.read_model(model="model.xml", weights="model.bin") compiled_model = core.compile_model(model=model, device_name="CPU") output_layer = compiled_model.outputs[0] result = compiled_model([img])[output_layer] „dog” © 2022 Intel 23
  • 24. compiled_model = core.compile_model(model=model, device_name="CPU") © 2022 Intel 24 Supported Devices
  • 25. compiled_model = core.compile_model(model=model, device_name="CPU") CPU © 2022 Intel 25 Supported Devices
  • 26. CPU GPU compiled_model = core.compile_model(model=model, device_name="CPU") © 2022 Intel 26 Supported Devices
  • 27. CPU GPU MYRIAD compiled_model = core.compile_model(model=model, device_name="CPU") © 2022 Intel 27 Supported Devices
  • 28. compiled_model = core.compile_model(model=model, device_name="CPU") CPU GPU MYRIAD HETERO CPU GPU Convolution Add Clamp GroupConvolution Add Clamp Convolution Add Convolution Add © 2022 Intel 28 Supported Devices
  • 29. compiled_model = core.compile_model(model=model, device_name="CPU") CPU GPU MYRIAD HETERO MULTI CPU GPU Convolution Add Clamp GroupConvolution Add Clamp Convolution Add Convolution Add GPU Convolution Add Clamp GroupConvolution Add Clamp Convolution Add Convolution Add CPU Convolution Add Clamp GroupConvolution Add Clamp Convolution Add Convolution Add 29 Supported Devices
  • 30. Supported Devices RUNTIME GPU CPU APPLICATION AUTO PLUGIN CPU PLUGIN GPU PLUGIN AUTO AUTO chooses the device AUTO sets device config based on hints AUTO handles the exec logic on multiple devices CPU GPU MYRIAD AUTO HETERO MULTI compiled_model = core.compile_model(model=model, device_name="CPU") 30
  • 31. AUTO Device INFERENCE ON GPU INFERENCE ON CPU INFERENCE STOPS ON CPU COMPILE NETWORK FOR CPU CPU Workloads GPU Workloads 116.7 5,800 MS TIME APPLICATION LOGIC LOADING NN INFERENCE DEVICE SELECTION APPLICATION INFERENCE ON AUTO COMPILE / LOAD NETWORK FOR GPU © 2022 Intel 31
  • 32. Input Data with Variable Shape? 32 “What is the weather going to be like today?”
  • 33. How to handle variable input shapes? 33 Padding Multiple Precompiled Models Inefficient and Cumbersome!
  • 34. Native Dynamic Shapes Support in OpenVINO™ 34 Dynamic in input image size
  • 35. • Set-and-forget optimization knob – choose latency or throughput. • Completely portable between the devices • Throughput hint drives device-specific optimizations • Even works for AUTO! Portable Performance Hints 35
  • 36. Automatic Batching 36 • How do you choose a good batch size? • Let the OpenVINO™ runtime decide for you! • No need to batch requests, runtime will do that too! • Batching can improve throughput with select devices and models.
  • 37. 300+ Pre-Trained + Optimized Models © 2022 Intel 37 Open Model Zoo
  • 38. 300+ Pre-Trained + Optimized Models 38 Open Model Zoo
  • 39. 300+ Pre-Trained + Optimized Models © 2022 Intel 39 Open Model Zoo
  • 40. 300+ Pre-Trained + Optimized Models © 2022 Intel 40 Open Model Zoo
  • 41. 300+ Pre-Trained + Optimized Models © 2022 Intel 41 Open Model Zoo
  • 42. 300+ Pre-Trained + Optimized Models © 2022 Intel 42 Open Model Zoo
  • 43. 300+ Pre-Trained + Optimized Models © 2022 Intel 43 Open Model Zoo
  • 45. Try it yourself! Learn more and download at openvino.ai Complete the Intel® Edge AI Certification! • Supercharge your career • Approx 20 hours of video + quizzes + coding
  • 46. Deep Dive Session Thursday, May 19 Intel AI Developer Expo—Let's Build Something Wonderful Together Session: 3:00 – 5:30 pm Reception: 5:30 - 7:30 pm Location: Room 209/210 Afternoon snack and post-session Reception will be provided
  • 48. Configuration Intel® Core™ i7-1185G7 Intel® Core™ i3-8100 Intel® Core™ i5-8500 Intel® Core™ i7-8700T Motherboard Intel Corporation internal/ Reference Validation Platform GIGABYTE* Z390 UD ASUS* PRIME Z370-A GIGABYTE* Z370M DS3H-CF CPU Intel® Core™ i7-1185G7 @ 3.00GHz Intel® Core™ i3-8100 CPU @ 3.60GHz Intel® Core™ i5-8500 CPU @ 3.00GHz Intel® Core™ i7-8700T CPU @ 2.40GHz Hyper Threading ON OFF OFF ON Turbo Setting ON OFF ON ON Memory 2 x 8 GB DDR4 3200MHz 4 x 8 GB DDR4 2400MHz 2 x 16 GB DDR4 2666MHz 4 x 16 GB DDR4 2400MHz Operating System Ubuntu* 18.04 LTS Ubuntu* 18.04 LTS Ubuntu* 18.04 LTS Ubuntu* 18.04 LTS Kernel Version 5.8.0-05-generic 5.3.0-24-generic 5.3.0-24-generic 5.3.0-24-generic BIOS Vendor Intel Corporation American Megatrends Inc.* American Megatrends Inc.* American Megatrends Inc.* BIOS Version TGLSFWI1.R00.3425. A00.2010162309 F8 2401 F14c BIOS Release 16-Oct-20 24-May-19 12-Jul-19 23-Mar-21 BIOS Settings Default Settings Select optimized default settings, set OS type to “other”, save and exit Select optimized default settings, save and exit Select optimized default settings, set OS type to “other”, save and exit Batch size 1 1 1 1 Precision FP32 FP32 FP32 FP32 Number of concurrent inference requests 4 4 3 4 Test Date 18-Jun-21 18-Jun-21 18-Jun-21 18-Jun-21 Rated maximum TDP/socket in Watt 28 65 65 35 48 © 2022 Intel Platform Configurations for Performance Benchmarks
  • 49. Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex. Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details. No product or component can be absolutely secure. Intel technologies may require enabled hardware, software or service activation. Your costs and results may vary. Intel is committed to respecting human rights and avoiding complicity in human rights abuses. See Intel's Global Human Rights Principles. Intel's products and software are intended only to be used in applications that do not cause or contribute to a violation of an internationally recognized human right. © Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others. Notices and Disclaimers 49 © 2022 Intel