SlideShare a Scribd company logo

Artificial intelligence at the edge

•
•
Jameson Toole
Jameson Toole

Jameson Toole, co-founder and CEO of Fritz talks about moving machine learning and AI model to edge devices.

Software
1 of 50
Download to read offline
Jameson Toole Artificial Intelligence at the Edge ODSC Meetup, 2018
Jameson Toole · Artificial Intelligence at the Edge · ODSC Before we get started... Demo + Code About me ● Community: heartbeat.fritz.ai ● App: bit.ly/tryheartbeat ● Code: bit.ly/heartbeatsource ● School: Michigan, MIT ● Background: Data science / ML ● Now: Cofounder / CEO, Fritz
Survey
Jameson Toole · Artificial Intelligence at the Edge · ODSC 4 Infer AI extracts relevance Act Based on intelligence Sense Everything, realtime Edge Intelligence with Centralized Learning Learn Source: The End of Cloud Computing by Peter Levine
Jameson Toole · Artificial Intelligence at the Edge · ODSC 60 Frame-per-second problems
Privacy meets big data
Jameson Toole · Artificial Intelligence at the Edge · ODSC Compute doesn’t grow on trees
Jameson Toole · Artificial Intelligence at the Edge · ODSC Edge devices will dominate AI inference
Jameson Toole · Artificial Intelligence at the Edge · ODSC Growth of AI Fueled by Edge Computing Now: Edge Intelligence Next: Mobile dominates AI inference Future: Training on Edge
Jameson Toole · Artificial Intelligence at the Edge · ODSC 10 1 Python in the cloud vs Swift on the edge. Stack Mismatch 2 Need engineers that know ML and mobile. Ninja Unicorns Wanted 3 Long chain from R&D to production. Failures are silent. Fragile Infrastructure4 Thousands of devices, processors, operating systems. Heterogeneous Hardware State of Edge Computing
Inspiration
Jameson Toole · Artificial Intelligence at the Edge · ODSC Magic Sudoku
Jameson Toole · Artificial Intelligence at the Edge · ODSC InstaSaber
Jameson Toole · Artificial Intelligence at the Edge · ODSC Weightlifting App (iOS)
Jameson Toole · Artificial Intelligence at the Edge · ODSC DensePose
Jameson Toole · Artificial Intelligence at the Edge · ODSC https://www.eff.org/ai/metrics
Lifecycle Management
Jameson Toole · Artificial Intelligence at the Edge · ODSC 18 Lifecycle Management Expectation management ● What’s different between cloud and edge? ● Lots of talk about deep learning (but traditional ML is fun too!) ● Tools and concepts
Jameson Toole · Artificial Intelligence at the Edge · ODSC 19 Collect Lifecycle Management: Overview Train Optimize Convert Monitor Deploy Protect The Life of a Mobile Model
Collect
Jameson Toole · Artificial Intelligence at the Edge · ODSC 21 Collect Open Data ● Image Recognition: ImageNet, CIFAR ● Object Detection: COCO, KITTI ● OCR: EMNIST, Tesseract ● Exhaustive List: deeplearning4j.org/opendata
Jameson Toole · Artificial Intelligence at the Edge · ODSC 22 Collect - Data Augmentation img = PIL.Image.open(img_data) rot_img = img.rotate(85) blur_img = img.filter(PIL.ImageFilter.BLUR) dark_img = PIL.Image.fromarray( (numpy.array(img) / 2).astype('uint8') ) Training matches production
Jameson Toole · Artificial Intelligence at the Edge · ODSC 23 Collect - Input Sizes ● Beware of large inputs ● Explicit dimensions TensorFlow / Keras Core ML [None, None, None, 3] -> [1, 1, 1, 3] ● Proportion and orientation ->
Jameson Toole · Artificial Intelligence at the Edge · ODSC 24 Collect - Input / Output Sampling ● Cache model inputs and outputs ○ Buffer size ○ Sampling rate ○ Device characteristics ● Send back to the cloud ○ Connectivity ○ Bandwidth limitations ● Privacy API DB Client
Train
Jameson Toole · Artificial Intelligence at the Edge · ODSC 26 Train ● Training environments should match production. ● Beware of pre- and post-processing. ● Keep an eye on distributed training.
Jameson Toole · Artificial Intelligence at the Edge · ODSC 27 Train - Mobile Friendly Tools ● Core ML ● TensorFlow Lite ● Windows ML ● Caffe2Go ● Turi Create ● IBM Watson Studio ● Azure Custom Vision
Optimize
Jameson Toole · Artificial Intelligence at the Edge · ODSC 29 Optimize ● Architecture ○ Depthwise-separable convolutions ○ Hardware-aware design ○ Architecture search ● Pruning ● Compression ○ Quantization ○ Serialization
Jameson Toole · Artificial Intelligence at the Edge · ODSC 30 Optimize - Architecture Iterative process ● Generate architecture ● Train network ● Benchmark performance ● Minimize Cost(speed, accuracy, size) alchemy.fritz.ai
Jameson Toole · Artificial Intelligence at the Edge · ODSC 31 Optimize - Architecture Depthwise-separable Convolution FLOPs = DK · DK · M · N · DF · DF FLOPs = DK ·DK · M · DF · DF + M · N · DF · DF Standard Convolution MobileNets: https://arxiv.org/abs/1704.04861 Kernel size Input channels Output channels Input size
Jameson Toole · Artificial Intelligence at the Edge · ODSC 32 Optimize - Pruning MobileNet width multiplier ● “The role of the width multiplier α is to thin a network uniformly at each layer.” ● The number of input channels M becomes αM ● The number of output channels N becomes αN
Jameson Toole · Artificial Intelligence at the Edge · ODSC 33 Optimize - Pruning Goal: Remove operations that don’t contribute to accuracy ● Step 1: Compute importance ● Step 2: Prune ● Step 3: Rewire ● Step 4: Fine-tune http://machinethink.net/blog/compressing-deep-neural-nets/ Original network size: 4,253,864 parameters Compressed network size: 3,210,232 parameters Compressed to: 75.5% of original size Top-1 accuracy over 50000 images = 67.2% Top-5 accuracy over 50000 images = 87.7%
Jameson Toole · Artificial Intelligence at the Edge · ODSC 34 Optimization - Quantization Fixed-point Quantization ● Smaller model size ● Faster runtime ● Less memory ● tf.contrib.quantize tf.contrib.quantize.create_training_graph(...) tf.contrib.quantize.create_eval_graph(...) ● mxnet.contrib.quantization mxnet.contrib.quantize_model( sym, arg_params, aux_params,...) ● 3-5x speed up, 2-10x compression, 0-5% less accurate
Convert
Jameson Toole · Artificial Intelligence at the Edge · ODSC 36 Convert Server Side TensorFlow Keras Caffe2 PyTorch MXNet Mobile Friendly Core ML TensorFlow Lite Windows ML Caffe2Go coremltools TOCO tf-coreml torch2coreml ONNX mxnet-to-coreml
Protect
Jameson Toole · Artificial Intelligence at the Edge · ODSC 38 Protect Once a model is on a device, assume anyone can access it. ● Encryption ○ Full model -> Can’t use mobile frameworks ○ Weight data -> Need to write custom layers ● Obfuscation ○ Proprietary pre-processing (i.e. rotate 90 degrees) ○ Scramble weights or layer order
Deploy
Jameson Toole · Artificial Intelligence at the Edge · ODSC 40 Deploy ● Port pre- and post-processing ● Pre-loaded vs fetch at runtime ● Models are features ○ Staged rollouts ○ A/B testing ○ Versioning ● Put the right model on the right device Integrate: bit.ly/heartbeatsource
Monitor
Jameson Toole · Artificial Intelligence at the Edge · ODSC 42 Monitor ● Metrics ○ Runtime ○ Memory ○ Battery drain ○ Usage ● Slice by ○ OS ○ Device ○ Chipset
Jameson Toole · Artificial Intelligence at the Edge · ODSC 43 Collect Lifecycle Management: Overview Train Optimize Convert Monitor Deploy Protect The Life of a Mobile Model
Jameson Toole · Artificial Intelligence at the Edge · ODSC SDKTrain your own model Use one of ours Cross-platform portability Analytics Monitoring Developer API Optimize Native ModelBuild Validation OTA Update Release Manage Monitoring iOS & Android Deploy ML/AI models on all your mobile devices Complete Platform for Edge Intelligence 44
Inspiration
Jameson Toole · Artificial Intelligence at the Edge · ODSC “Interactive Sketch-Based Normal Map Generation with Deep Neural Networks
Jameson Toole · Artificial Intelligence at the Edge · ODSC Skydio R1
https://machinelearning.apple.com/2017/10/01/hey-siri.html
Jameson Toole · Artificial Intelligence at the Edge · ODSC @ben_ferns
Want to be an edge expert? Questions? Jameson Toole, CEO jameson@fritz.ai Get started: www.fritz.ai Community: Heartbeat Tools: Alchemy

Recommended

Creating smaller, faster, production-ready mobile machine learning models.
Creating smaller, faster, production-ready mobile machine learning models.Creating smaller, faster, production-ready mobile machine learning models.
Creating smaller, faster, production-ready mobile machine learning models.
Jameson Toole
 
Meetup 18/10/2018 - Artificiële intelligentie en mobiliteit
Meetup 18/10/2018 - Artificiële intelligentie en mobiliteitMeetup 18/10/2018 - Artificiële intelligentie en mobiliteit
Meetup 18/10/2018 - Artificiële intelligentie en mobiliteit
Digipolis Antwerpen
 
NIPS - Deep learning @ Edge using Intel's NCS
NIPS - Deep learning @ Edge using Intel's NCSNIPS - Deep learning @ Edge using Intel's NCS
NIPS - Deep learning @ Edge using Intel's NCS
geetachauhan
 
Introducing google’s mobile nets
Introducing google’s mobile netsIntroducing google’s mobile nets
Introducing google’s mobile nets
Larry Guo
 
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
Numenta
 
Deep Learning on Everyday Devices
Deep Learning on Everyday DevicesDeep Learning on Everyday Devices
Deep Learning on Everyday Devices
Brodmann17
 
Learning where to look: focus and attention in deep vision
Learning where to look: focus and attention in deep visionLearning where to look: focus and attention in deep vision
Learning where to look: focus and attention in deep vision
Universitat Politècnica de Catalunya
 
Tiny intelligent computers and sensors - Open Hardware Event 2020
Tiny intelligent computers and sensors - Open Hardware Event 2020Tiny intelligent computers and sensors - Open Hardware Event 2020
Tiny intelligent computers and sensors - Open Hardware Event 2020
Jan Jongboom
 

Recommended

Creating smaller, faster, production-ready mobile machine learning models.
Creating smaller, faster, production-ready mobile machine learning models.Creating smaller, faster, production-ready mobile machine learning models.
Creating smaller, faster, production-ready mobile machine learning models.
Jameson Toole
 
Meetup 18/10/2018 - Artificiële intelligentie en mobiliteit
Meetup 18/10/2018 - Artificiële intelligentie en mobiliteitMeetup 18/10/2018 - Artificiële intelligentie en mobiliteit
Meetup 18/10/2018 - Artificiële intelligentie en mobiliteit
Digipolis Antwerpen
 
NIPS - Deep learning @ Edge using Intel's NCS
NIPS - Deep learning @ Edge using Intel's NCSNIPS - Deep learning @ Edge using Intel's NCS
NIPS - Deep learning @ Edge using Intel's NCS
geetachauhan
 
Introducing google’s mobile nets
Introducing google’s mobile netsIntroducing google’s mobile nets
Introducing google’s mobile nets
Larry Guo
 
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
FPGA Conference 2021: Breaking the TOPS ceiling with sparse neural networks -...
Numenta
 
Deep Learning on Everyday Devices
Deep Learning on Everyday DevicesDeep Learning on Everyday Devices
Deep Learning on Everyday Devices
Brodmann17
 
Learning where to look: focus and attention in deep vision
Learning where to look: focus and attention in deep visionLearning where to look: focus and attention in deep vision
Learning where to look: focus and attention in deep vision
Universitat Politècnica de Catalunya
 
Tiny intelligent computers and sensors - Open Hardware Event 2020
Tiny intelligent computers and sensors - Open Hardware Event 2020Tiny intelligent computers and sensors - Open Hardware Event 2020
Tiny intelligent computers and sensors - Open Hardware Event 2020
Jan Jongboom
 
deep learning from scratch chapter 3 neural network
deep learning from scratch chapter 3 neural networkdeep learning from scratch chapter 3 neural network
deep learning from scratch chapter 3 neural network
Jaey Jeong
 
Computer Vision with Deep Learning
Computer Vision with Deep LearningComputer Vision with Deep Learning
Computer Vision with Deep Learning
Capgemini
 
Image Classification Done Simply using Keras and TensorFlow
Image Classification Done Simply using Keras and TensorFlow Image Classification Done Simply using Keras and TensorFlow
Image Classification Done Simply using Keras and TensorFlow
Rajiv Shah
 
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st CenturyThe von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
Perry Lea
 
Soumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning World
Soumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning WorldSoumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning World
Soumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning World
AI Frontiers
 
Intelligent robot used in the field of practical
Intelligent robot used in the field of practicalIntelligent robot used in the field of practical
Intelligent robot used in the field of practicalUlaa Iman
 
Optical computer
Optical computer Optical computer
Optical computer
c_s_halabja
 
blue brain technology
blue brain technologyblue brain technology
blue brain technology
Manju Rajput
 
Optical computers
Optical computersOptical computers
Optical computers
Ajay AJ
 
TheThingsConference 2019 Slides of Alex Raimondi
TheThingsConference 2019 Slides of Alex RaimondiTheThingsConference 2019 Slides of Alex Raimondi
TheThingsConference 2019 Slides of Alex Raimondi
Schekeb Fateh
 
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario SpacagnaAnomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Data Science Milan
 
“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...
“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...
“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...
Edge AI and Vision Alliance
 
Machine learning on 1 square centimeter - Emerce Next 2019
Machine learning on 1 square centimeter - Emerce Next 2019Machine learning on 1 square centimeter - Emerce Next 2019
Machine learning on 1 square centimeter - Emerce Next 2019
Jan Jongboom
 
Optical Computer
Optical ComputerOptical Computer
Optical ComputerJirshad Keloth
 
“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung
“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung
“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung
Edge AI and Vision Alliance
 
AI & ML in Defence Systems - Sunil Chomal
AI & ML in Defence Systems - Sunil ChomalAI & ML in Defence Systems - Sunil Chomal
AI & ML in Defence Systems - Sunil Chomal
Sunil Chomal
 
Keras: Deep Learning Library for Python
Keras: Deep Learning Library for PythonKeras: Deep Learning Library for Python
Keras: Deep Learning Library for Python
Rafi Khan
 
Computational decision making
Computational decision makingComputational decision making
Computational decision making
Boris Adryan
 
Tensorflow IoT - 1 Wk coding challenge
Tensorflow IoT - 1 Wk coding challengeTensorflow IoT - 1 Wk coding challenge
Tensorflow IoT - 1 Wk coding challenge
geetachauhan
 
Transfer learning for IoT
Transfer learning for IoTTransfer learning for IoT
Transfer learning for IoT
geetachauhan
 
Internet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! nightInternet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! night
Andy Gelme
 
Edge Computing for the Industry
Edge Computing for the IndustryEdge Computing for the Industry
Edge Computing for the Industry
William Liang
 

More Related Content

What's hot

deep learning from scratch chapter 3 neural network
deep learning from scratch chapter 3 neural networkdeep learning from scratch chapter 3 neural network
deep learning from scratch chapter 3 neural network
Jaey Jeong
 
Computer Vision with Deep Learning
Computer Vision with Deep LearningComputer Vision with Deep Learning
Computer Vision with Deep Learning
Capgemini
 
Image Classification Done Simply using Keras and TensorFlow
Image Classification Done Simply using Keras and TensorFlow Image Classification Done Simply using Keras and TensorFlow
Image Classification Done Simply using Keras and TensorFlow
Rajiv Shah
 
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st CenturyThe von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
Perry Lea
 
Soumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning World
Soumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning WorldSoumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning World
Soumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning World
AI Frontiers
 
Intelligent robot used in the field of practical
Intelligent robot used in the field of practicalIntelligent robot used in the field of practical
Intelligent robot used in the field of practicalUlaa Iman
 
Optical computer
Optical computer Optical computer
Optical computer
c_s_halabja
 
blue brain technology
blue brain technologyblue brain technology
blue brain technology
Manju Rajput
 
Optical computers
Optical computersOptical computers
Optical computers
Ajay AJ
 
TheThingsConference 2019 Slides of Alex Raimondi
TheThingsConference 2019 Slides of Alex RaimondiTheThingsConference 2019 Slides of Alex Raimondi
TheThingsConference 2019 Slides of Alex Raimondi
Schekeb Fateh
 
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario SpacagnaAnomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Data Science Milan
 
“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...
“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...
“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...
Edge AI and Vision Alliance
 
Machine learning on 1 square centimeter - Emerce Next 2019
Machine learning on 1 square centimeter - Emerce Next 2019Machine learning on 1 square centimeter - Emerce Next 2019
Machine learning on 1 square centimeter - Emerce Next 2019
Jan Jongboom
 
“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung
“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung
“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung
Edge AI and Vision Alliance
 
AI & ML in Defence Systems - Sunil Chomal
AI & ML in Defence Systems - Sunil ChomalAI & ML in Defence Systems - Sunil Chomal
AI & ML in Defence Systems - Sunil Chomal
Sunil Chomal
 
Keras: Deep Learning Library for Python
Keras: Deep Learning Library for PythonKeras: Deep Learning Library for Python
Keras: Deep Learning Library for Python
Rafi Khan
 
Computational decision making
Computational decision makingComputational decision making
Computational decision making
Boris Adryan
 
Tensorflow IoT - 1 Wk coding challenge
Tensorflow IoT - 1 Wk coding challengeTensorflow IoT - 1 Wk coding challenge
Tensorflow IoT - 1 Wk coding challenge
geetachauhan
 
Transfer learning for IoT
Transfer learning for IoTTransfer learning for IoT
Transfer learning for IoT
geetachauhan
 

What's hot (20)

deep learning from scratch chapter 3 neural network
deep learning from scratch chapter 3 neural networkdeep learning from scratch chapter 3 neural network
deep learning from scratch chapter 3 neural network
 
Computer Vision with Deep Learning
Computer Vision with Deep LearningComputer Vision with Deep Learning
Computer Vision with Deep Learning
 
Image Classification Done Simply using Keras and TensorFlow
Image Classification Done Simply using Keras and TensorFlow Image Classification Done Simply using Keras and TensorFlow
Image Classification Done Simply using Keras and TensorFlow
 
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st CenturyThe von Neumann Memory Barrier and Computer Architectures for the 21st Century
The von Neumann Memory Barrier and Computer Architectures for the 21st Century
 
Soumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning World
Soumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning WorldSoumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning World
Soumith Chintala at AI Frontiers: A Dynamic View of the Deep Learning World
 
Intelligent robot used in the field of practical
Intelligent robot used in the field of practicalIntelligent robot used in the field of practical
Intelligent robot used in the field of practical
 
Optical computer
Optical computer Optical computer
Optical computer
 
blue brain technology
blue brain technologyblue brain technology
blue brain technology
 
Optical computers
Optical computersOptical computers
Optical computers
 
TheThingsConference 2019 Slides of Alex Raimondi
TheThingsConference 2019 Slides of Alex RaimondiTheThingsConference 2019 Slides of Alex Raimondi
TheThingsConference 2019 Slides of Alex Raimondi
 
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario SpacagnaAnomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
Anomaly Detection using Deep Auto-Encoders | Gianmario Spacagna
 
“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...
“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...
“An Introduction to Data Augmentation Techniques in ML Frameworks,” a Present...
 
Machine learning on 1 square centimeter - Emerce Next 2019
Machine learning on 1 square centimeter - Emerce Next 2019Machine learning on 1 square centimeter - Emerce Next 2019
Machine learning on 1 square centimeter - Emerce Next 2019
 
Optical Computer
Optical ComputerOptical Computer
Optical Computer
 
“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung
“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung
“A Highly Data-Efficient Deep Learning Approach,” a Presentation from Samsung
 
AI & ML in Defence Systems - Sunil Chomal
AI & ML in Defence Systems - Sunil ChomalAI & ML in Defence Systems - Sunil Chomal
AI & ML in Defence Systems - Sunil Chomal
 
Keras: Deep Learning Library for Python
Keras: Deep Learning Library for PythonKeras: Deep Learning Library for Python
Keras: Deep Learning Library for Python
 
Computational decision making
Computational decision makingComputational decision making
Computational decision making
 
Tensorflow IoT - 1 Wk coding challenge
Tensorflow IoT - 1 Wk coding challengeTensorflow IoT - 1 Wk coding challenge
Tensorflow IoT - 1 Wk coding challenge
 
Transfer learning for IoT
Transfer learning for IoTTransfer learning for IoT
Transfer learning for IoT
 

Similar to Artificial intelligence at the edge

Internet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! nightInternet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! night
Andy Gelme
 
Edge Computing for the Industry
Edge Computing for the IndustryEdge Computing for the Industry
Edge Computing for the Industry
William Liang
 
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Codemotion
 
HiPEAC-CSW 2022_Pedro Trancoso presentation
HiPEAC-CSW 2022_Pedro Trancoso presentationHiPEAC-CSW 2022_Pedro Trancoso presentation
HiPEAC-CSW 2022_Pedro Trancoso presentation
VEDLIoT Project
 
An In-Depth Look Into Microcontrollers
An In-Depth Look Into MicrocontrollersAn In-Depth Look Into Microcontrollers
An In-Depth Look Into Microcontrollers
ICS
 
Embedded Fest 2019. Dov Nimratz. Artificial Intelligence in Small Embedded Sy...
Embedded Fest 2019. Dov Nimratz. Artificial Intelligence in Small Embedded Sy...Embedded Fest 2019. Dov Nimratz. Artificial Intelligence in Small Embedded Sy...
Embedded Fest 2019. Dov Nimratz. Artificial Intelligence in Small Embedded Sy...
EmbeddedFest
 
Architecting IoT with Machine Learning
Architecting IoT with Machine LearningArchitecting IoT with Machine Learning
Architecting IoT with Machine Learning
Rudradeb Mitra
 
Data Summer Conf 2018, “Architecting IoT system with Machine Learning (ENG)” ...
Data Summer Conf 2018, “Architecting IoT system with Machine Learning (ENG)” ...Data Summer Conf 2018, “Architecting IoT system with Machine Learning (ENG)” ...
Data Summer Conf 2018, “Architecting IoT system with Machine Learning (ENG)” ...
Provectus
 
Sinfonier Storm Builder for Security Intelligence
Sinfonier Storm Builder for Security IntelligenceSinfonier Storm Builder for Security Intelligence
Sinfonier Storm Builder for Security Intelligence
Leonardo Amor
 
Intelligent Thumbnail Selection
Intelligent Thumbnail SelectionIntelligent Thumbnail Selection
Intelligent Thumbnail Selection
Kamil Sindi
 
Botnet detection in SDN by DL techniques
Botnet detection in SDN by DL techniquesBotnet detection in SDN by DL techniques
Botnet detection in SDN by DL techniques
Ivan Letteri
 
“Once-for-All DNNs: Simplifying Design of Efficient Models for Diverse Hardwa...
“Once-for-All DNNs: Simplifying Design of Efficient Models for Diverse Hardwa...“Once-for-All DNNs: Simplifying Design of Efficient Models for Diverse Hardwa...
“Once-for-All DNNs: Simplifying Design of Efficient Models for Diverse Hardwa...
Edge AI and Vision Alliance
 
AI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are DangerousAI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are Dangerous
Raffael Marty
 
LEGaTO: Use cases
LEGaTO: Use casesLEGaTO: Use cases
LEGaTO: Use cases
LEGATO project
 
AWS Big Data Demystified #1.2 | Big Data architecture lessons learned
AWS Big Data Demystified #1.2 | Big Data architecture lessons learned AWS Big Data Demystified #1.2 | Big Data architecture lessons learned
AWS Big Data Demystified #1.2 | Big Data architecture lessons learned
Omid Vahdaty
 
Coding the Continuum
Coding the ContinuumCoding the Continuum
Coding the Continuum
Ian Foster
 
AIoT: Intelligence on Microcontroller
AIoT: Intelligence on MicrocontrollerAIoT: Intelligence on Microcontroller
AIoT: Intelligence on Microcontroller
Andri Yadi
 
realtime_ai_systems_academia.pptx
realtime_ai_systems_academia.pptxrealtime_ai_systems_academia.pptx
realtime_ai_systems_academia.pptx
gopikahari7
 
AktaionPPTv5_JZedits
AktaionPPTv5_JZeditsAktaionPPTv5_JZedits
AktaionPPTv5_JZeditsRod Soto
 
“Enabling Ultra-low Power Edge Inference and On-device Learning with Akida,” ...
“Enabling Ultra-low Power Edge Inference and On-device Learning with Akida,” ...“Enabling Ultra-low Power Edge Inference and On-device Learning with Akida,” ...
“Enabling Ultra-low Power Edge Inference and On-device Learning with Akida,” ...
Edge AI and Vision Alliance
 

Similar to Artificial intelligence at the edge (20)

Internet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! nightInternet Of Things: Hands on: YOW! night
Internet Of Things: Hands on: YOW! night
 
Edge Computing for the Industry
Edge Computing for the IndustryEdge Computing for the Industry
Edge Computing for the Industry
 
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...
 
HiPEAC-CSW 2022_Pedro Trancoso presentation
HiPEAC-CSW 2022_Pedro Trancoso presentationHiPEAC-CSW 2022_Pedro Trancoso presentation
HiPEAC-CSW 2022_Pedro Trancoso presentation
 
An In-Depth Look Into Microcontrollers
An In-Depth Look Into MicrocontrollersAn In-Depth Look Into Microcontrollers
An In-Depth Look Into Microcontrollers
 
Embedded Fest 2019. Dov Nimratz. Artificial Intelligence in Small Embedded Sy...
Embedded Fest 2019. Dov Nimratz. Artificial Intelligence in Small Embedded Sy...Embedded Fest 2019. Dov Nimratz. Artificial Intelligence in Small Embedded Sy...
Embedded Fest 2019. Dov Nimratz. Artificial Intelligence in Small Embedded Sy...
 
Architecting IoT with Machine Learning
Architecting IoT with Machine LearningArchitecting IoT with Machine Learning
Architecting IoT with Machine Learning
 
Data Summer Conf 2018, “Architecting IoT system with Machine Learning (ENG)” ...
Data Summer Conf 2018, “Architecting IoT system with Machine Learning (ENG)” ...Data Summer Conf 2018, “Architecting IoT system with Machine Learning (ENG)” ...
Data Summer Conf 2018, “Architecting IoT system with Machine Learning (ENG)” ...
 
Sinfonier Storm Builder for Security Intelligence
Sinfonier Storm Builder for Security IntelligenceSinfonier Storm Builder for Security Intelligence
Sinfonier Storm Builder for Security Intelligence
 
Intelligent Thumbnail Selection
Intelligent Thumbnail SelectionIntelligent Thumbnail Selection
Intelligent Thumbnail Selection
 
Botnet detection in SDN by DL techniques
Botnet detection in SDN by DL techniquesBotnet detection in SDN by DL techniques
Botnet detection in SDN by DL techniques
 
“Once-for-All DNNs: Simplifying Design of Efficient Models for Diverse Hardwa...
“Once-for-All DNNs: Simplifying Design of Efficient Models for Diverse Hardwa...“Once-for-All DNNs: Simplifying Design of Efficient Models for Diverse Hardwa...
“Once-for-All DNNs: Simplifying Design of Efficient Models for Diverse Hardwa...
 
AI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are DangerousAI & ML in Cyber Security - Why Algorithms Are Dangerous
AI & ML in Cyber Security - Why Algorithms Are Dangerous
 
LEGaTO: Use cases
LEGaTO: Use casesLEGaTO: Use cases
LEGaTO: Use cases
 
AWS Big Data Demystified #1.2 | Big Data architecture lessons learned
AWS Big Data Demystified #1.2 | Big Data architecture lessons learned AWS Big Data Demystified #1.2 | Big Data architecture lessons learned
AWS Big Data Demystified #1.2 | Big Data architecture lessons learned
 
Coding the Continuum
Coding the ContinuumCoding the Continuum
Coding the Continuum
 
AIoT: Intelligence on Microcontroller
AIoT: Intelligence on MicrocontrollerAIoT: Intelligence on Microcontroller
AIoT: Intelligence on Microcontroller
 
realtime_ai_systems_academia.pptx
realtime_ai_systems_academia.pptxrealtime_ai_systems_academia.pptx
realtime_ai_systems_academia.pptx
 
AktaionPPTv5_JZedits
AktaionPPTv5_JZeditsAktaionPPTv5_JZedits
AktaionPPTv5_JZedits
 
“Enabling Ultra-low Power Edge Inference and On-device Learning with Akida,” ...
“Enabling Ultra-low Power Edge Inference and On-device Learning with Akida,” ...“Enabling Ultra-low Power Edge Inference and On-device Learning with Akida,” ...
“Enabling Ultra-low Power Edge Inference and On-device Learning with Akida,” ...
 

Recently uploaded

2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
Łukasz Chruściel
 
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Mind IT Systems
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
Paco van Beckhoven
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
Ayan Halder
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - DeugloEmpowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
Deuglo Infosystem Pvt Ltd
 
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
Donna Lenk
 
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppAI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
Google
 
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
Alina Yurenko
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Aftab Hussain
 
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket ManagementUtilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
timtebeek1
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
Adele Miller
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
GraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph TechnologyGraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph Technology
Neo4j
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
Rakesh Kumar R
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
Max Andersen
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
lorraineandreiamcidl
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 

Recently uploaded (20)

2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
 
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - DeugloEmpowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
 
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
 
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppAI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
 
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
 
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket ManagementUtilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
GraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph TechnologyGraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph Technology
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 

Artificial intelligence at the edge

  • 1. Jameson Toole Artificial Intelligence at the Edge ODSC Meetup, 2018
  • 2. Jameson Toole · Artificial Intelligence at the Edge · ODSC Before we get started... Demo + Code About me ● Community: heartbeat.fritz.ai ● App: bit.ly/tryheartbeat ● Code: bit.ly/heartbeatsource ● School: Michigan, MIT ● Background: Data science / ML ● Now: Cofounder / CEO, Fritz
  • 4. Jameson Toole · Artificial Intelligence at the Edge · ODSC 4 Infer AI extracts relevance Act Based on intelligence Sense Everything, realtime Edge Intelligence with Centralized Learning Learn Source: The End of Cloud Computing by Peter Levine
  • 5. Jameson Toole · Artificial Intelligence at the Edge · ODSC 60 Frame-per-second problems
  • 7. Jameson Toole · Artificial Intelligence at the Edge · ODSC Compute doesn’t grow on trees
  • 8. Jameson Toole · Artificial Intelligence at the Edge · ODSC Edge devices will dominate AI inference
  • 9. Jameson Toole · Artificial Intelligence at the Edge · ODSC Growth of AI Fueled by Edge Computing Now: Edge Intelligence Next: Mobile dominates AI inference Future: Training on Edge
  • 10. Jameson Toole · Artificial Intelligence at the Edge · ODSC 10 1 Python in the cloud vs Swift on the edge. Stack Mismatch 2 Need engineers that know ML and mobile. Ninja Unicorns Wanted 3 Long chain from R&D to production. Failures are silent. Fragile Infrastructure4 Thousands of devices, processors, operating systems. Heterogeneous Hardware State of Edge Computing
  • 12. Jameson Toole · Artificial Intelligence at the Edge · ODSC Magic Sudoku
  • 13. Jameson Toole · Artificial Intelligence at the Edge · ODSC InstaSaber
  • 14. Jameson Toole · Artificial Intelligence at the Edge · ODSC Weightlifting App (iOS)
  • 15. Jameson Toole · Artificial Intelligence at the Edge · ODSC DensePose
  • 16. Jameson Toole · Artificial Intelligence at the Edge · ODSC https://www.eff.org/ai/metrics
  • 18. Jameson Toole · Artificial Intelligence at the Edge · ODSC 18 Lifecycle Management Expectation management ● What’s different between cloud and edge? ● Lots of talk about deep learning (but traditional ML is fun too!) ● Tools and concepts
  • 19. Jameson Toole · Artificial Intelligence at the Edge · ODSC 19 Collect Lifecycle Management: Overview Train Optimize Convert Monitor Deploy Protect The Life of a Mobile Model
  • 21. Jameson Toole · Artificial Intelligence at the Edge · ODSC 21 Collect Open Data ● Image Recognition: ImageNet, CIFAR ● Object Detection: COCO, KITTI ● OCR: EMNIST, Tesseract ● Exhaustive List: deeplearning4j.org/opendata
  • 22. Jameson Toole · Artificial Intelligence at the Edge · ODSC 22 Collect - Data Augmentation img = PIL.Image.open(img_data) rot_img = img.rotate(85) blur_img = img.filter(PIL.ImageFilter.BLUR) dark_img = PIL.Image.fromarray( (numpy.array(img) / 2).astype('uint8') ) Training matches production
  • 23. Jameson Toole · Artificial Intelligence at the Edge · ODSC 23 Collect - Input Sizes ● Beware of large inputs ● Explicit dimensions TensorFlow / Keras Core ML [None, None, None, 3] -> [1, 1, 1, 3] ● Proportion and orientation ->
  • 24. Jameson Toole · Artificial Intelligence at the Edge · ODSC 24 Collect - Input / Output Sampling ● Cache model inputs and outputs ○ Buffer size ○ Sampling rate ○ Device characteristics ● Send back to the cloud ○ Connectivity ○ Bandwidth limitations ● Privacy API DB Client
  • 25. Train
  • 26. Jameson Toole · Artificial Intelligence at the Edge · ODSC 26 Train ● Training environments should match production. ● Beware of pre- and post-processing. ● Keep an eye on distributed training.
  • 27. Jameson Toole · Artificial Intelligence at the Edge · ODSC 27 Train - Mobile Friendly Tools ● Core ML ● TensorFlow Lite ● Windows ML ● Caffe2Go ● Turi Create ● IBM Watson Studio ● Azure Custom Vision
  • 29. Jameson Toole · Artificial Intelligence at the Edge · ODSC 29 Optimize ● Architecture ○ Depthwise-separable convolutions ○ Hardware-aware design ○ Architecture search ● Pruning ● Compression ○ Quantization ○ Serialization
  • 30. Jameson Toole · Artificial Intelligence at the Edge · ODSC 30 Optimize - Architecture Iterative process ● Generate architecture ● Train network ● Benchmark performance ● Minimize Cost(speed, accuracy, size) alchemy.fritz.ai
  • 31. Jameson Toole · Artificial Intelligence at the Edge · ODSC 31 Optimize - Architecture Depthwise-separable Convolution FLOPs = DK · DK · M · N · DF · DF FLOPs = DK ·DK · M · DF · DF + M · N · DF · DF Standard Convolution MobileNets: https://arxiv.org/abs/1704.04861 Kernel size Input channels Output channels Input size
  • 32. Jameson Toole · Artificial Intelligence at the Edge · ODSC 32 Optimize - Pruning MobileNet width multiplier ● “The role of the width multiplier α is to thin a network uniformly at each layer.” ● The number of input channels M becomes αM ● The number of output channels N becomes αN
  • 33. Jameson Toole · Artificial Intelligence at the Edge · ODSC 33 Optimize - Pruning Goal: Remove operations that don’t contribute to accuracy ● Step 1: Compute importance ● Step 2: Prune ● Step 3: Rewire ● Step 4: Fine-tune http://machinethink.net/blog/compressing-deep-neural-nets/ Original network size: 4,253,864 parameters Compressed network size: 3,210,232 parameters Compressed to: 75.5% of original size Top-1 accuracy over 50000 images = 67.2% Top-5 accuracy over 50000 images = 87.7%
  • 34. Jameson Toole · Artificial Intelligence at the Edge · ODSC 34 Optimization - Quantization Fixed-point Quantization ● Smaller model size ● Faster runtime ● Less memory ● tf.contrib.quantize tf.contrib.quantize.create_training_graph(...) tf.contrib.quantize.create_eval_graph(...) ● mxnet.contrib.quantization mxnet.contrib.quantize_model( sym, arg_params, aux_params,...) ● 3-5x speed up, 2-10x compression, 0-5% less accurate
  • 36. Jameson Toole · Artificial Intelligence at the Edge · ODSC 36 Convert Server Side TensorFlow Keras Caffe2 PyTorch MXNet Mobile Friendly Core ML TensorFlow Lite Windows ML Caffe2Go coremltools TOCO tf-coreml torch2coreml ONNX mxnet-to-coreml
  • 38. Jameson Toole · Artificial Intelligence at the Edge · ODSC 38 Protect Once a model is on a device, assume anyone can access it. ● Encryption ○ Full model -> Can’t use mobile frameworks ○ Weight data -> Need to write custom layers ● Obfuscation ○ Proprietary pre-processing (i.e. rotate 90 degrees) ○ Scramble weights or layer order
  • 40. Jameson Toole · Artificial Intelligence at the Edge · ODSC 40 Deploy ● Port pre- and post-processing ● Pre-loaded vs fetch at runtime ● Models are features ○ Staged rollouts ○ A/B testing ○ Versioning ● Put the right model on the right device Integrate: bit.ly/heartbeatsource
  • 42. Jameson Toole · Artificial Intelligence at the Edge · ODSC 42 Monitor ● Metrics ○ Runtime ○ Memory ○ Battery drain ○ Usage ● Slice by ○ OS ○ Device ○ Chipset
  • 43. Jameson Toole · Artificial Intelligence at the Edge · ODSC 43 Collect Lifecycle Management: Overview Train Optimize Convert Monitor Deploy Protect The Life of a Mobile Model
  • 44. Jameson Toole · Artificial Intelligence at the Edge · ODSC SDKTrain your own model Use one of ours Cross-platform portability Analytics Monitoring Developer API Optimize Native ModelBuild Validation OTA Update Release Manage Monitoring iOS & Android Deploy ML/AI models on all your mobile devices Complete Platform for Edge Intelligence 44
  • 46. Jameson Toole · Artificial Intelligence at the Edge · ODSC “Interactive Sketch-Based Normal Map Generation with Deep Neural Networks
  • 47. Jameson Toole · Artificial Intelligence at the Edge · ODSC Skydio R1
  • 49. Jameson Toole · Artificial Intelligence at the Edge · ODSC @ben_ferns
  • 50. Want to be an edge expert? Questions? Jameson Toole, CEO jameson@fritz.ai Get started: www.fritz.ai Community: Heartbeat Tools: Alchemy