AI On Your Phone: What's it Good For

•Download as PPTX, PDF•

2 likes•398 views

Talk given at Boston CodeCamp 4.7.2018 on applying machine learning models in mobile and small (IoT) devices using CoreML and TensorFlow Lite. Also dicusses common optimization techniques (quantization, pruning) and where the future might go with TensorFlow.js and JS/WebGL in general.

Software

AI On Your Phone
@ray_deck
github.com/rhdeck/boscc29

Artificial Intelligence
Deep Learning
Machine Learning
Statistical Computing

Supervised Learning
Unsupervised Learning
Reinforcement Learning

Ops and Numbers
Input Shape
“The Graph”
Conv2D, ReLU, MaxPool, etc…
Output Shape

Ops
Convolutions
Pooling
ReLU
TanH
Sigmoid
LSTM
GRU
Dense/Fully-Connected
More…

Old Stuff Works
https://arxiv.org/pdf/1803.01271.pdf

Models (Can Be) Files
Shapes of Input(s)
Operations
Hyperparameters
Weights
Sources & Drains
Shapes of Output(s)

What Do we Need?
1.Convert Image/Frame to Normalized Number
2. Run the model efficiently on local resources
3.Convert output number to what we seek:
•Dictionaries of labels
•Generated Image
•Prediction
•etc

ML Universe
Cloud
Local
Train Inference
Tensorflow,
Theano, CNTK…
Azure, Amazon,
Google,
MachineBox…
…CoreML, TF Lite,
TF Mobile, TF.JS
Caffe Lite
…plus MacGyver,
Algorithmia…

iOS Runtime and Bindings
File Format
Conversion Tooling

Runtime and Bindings
CVPixelBuffer in, Dictionary Out
Support for OTA/Runtime
Download and Compilation
Uses Metal Shaders for processing
Limited Operation Support

.mlmodel
1. Protobuf
2. Feature Normalization
•Value Range of Numbers (0-255 -> 0-1)
•Mean Value
3. Channel Ordering
•BGR vs RGB
4. Output Labeling/Denormalization

Conversion
Keras – coremlconverter
Tensorflow – tf-coreml
CNTK - mmdnn
MXNet – mxnet-to-coreml

Advantages
Ease of Application
Speed
Multiple analyses per second
Separation of Concerns
Compilation vs Inference
Source Agnostic

Lite
Multiple Runtimes
File Format
Converter

Lite
Cross-Platform
iOS, Android, Raspberry Pi
feed() and run()
No Compilation Step
Limited Operations
Number-to-Number

Lite
.tflite
FlatBuf
Quicker Memory Access
Alternative Precision: Bfloat16

Lite
Conversion
toco
Tensorflow Optimizing Converter

iOS Only Multiplatform
Input/Output Affordance Just the Numbers
Production Beta
Chunky Small

Planning
VGG16 550MB
InceptionV3 94MB
MobileNet 17MB

WebGL
Functional Programming
Cross-Platform Nirvana?

Similar to AI On Your Phone: What's it Good For

The deep learning tour - Q1 2017 Eran Shlomo

Designing Artificial IntelligenceDavid Chou

AI meets Big DataJan Wiegelmann

Dato KeynoteTuri, Inc.

Big Data & Artificial IntelligenceZavain Dar

GraphLab Conference 2014 Keynote - Carlos GuestrinTuri, Inc.

Spark Based Distributed Deep Learning Framework For Big Data Applications Humoyun Ahmedov

Angular and Deep LearningOswald Campesato

Big Data Analytics (ML, DL, AI) hands-onDony Riyanto

AWS re:Invent 2016: Deep Learning in Alexa (MAC202)Amazon Web Services

Advanced AI for People in a HurryScott Penberthy

Kaz Sato, Evangelist, Google at MLconf ATL 2016MLconf

OSCON 2014: Data Workflows for Machine LearningPaco Nathan

Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...Codemotion

Software Modeling and Artificial Intelligence: friends or foes?Jordi Cabot

Blockchains for AI [With New Applications]Trent McConaghy

Summit Australia 2019 - Supercharge PowerPlatform with AI - Dipankar Bhattach...Andrew Ly

AI for Good at MicrosoftMark Hamilton

Big Data LDN 2018: AI MEETS MAIL PROCESSINGMatt Stubbs

Building intelligent applications with Large Language ModelsSpeck&Tech

Similar to AI On Your Phone: What's it Good For (20)

The deep learning tour - Q1 2017

Designing Artificial Intelligence

AI meets Big Data

Dato Keynote

Big Data & Artificial Intelligence

GraphLab Conference 2014 Keynote - Carlos Guestrin

Spark Based Distributed Deep Learning Framework For Big Data Applications

Angular and Deep Learning

Big Data Analytics (ML, DL, AI) hands-on

AWS re:Invent 2016: Deep Learning in Alexa (MAC202)

Advanced AI for People in a Hurry

Kaz Sato, Evangelist, Google at MLconf ATL 2016

OSCON 2014: Data Workflows for Machine Learning

Artificial Intelligence in practice - Gerbert Kaandorp - Codemotion Amsterdam...

Software Modeling and Artificial Intelligence: friends or foes?

Blockchains for AI [With New Applications]

Summit Australia 2019 - Supercharge PowerPlatform with AI - Dipankar Bhattach...

AI for Good at Microsoft

Big Data LDN 2018: AI MEETS MAIL PROCESSING

Building intelligent applications with Large Language Models

Recently uploaded

XpertSolvers: Your Partner in Building Innovative Software SolutionsMehedi Hasan Shohan

Unit 1.1 Excite Part 1, class 9, cbse...aditisharan08

cybersecurity notes for mca students for learningVitsRangannavar

Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio, Inc.

ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...Christina Lin

What are the features of Vehicle Tracking System?Watsoo Telematics

Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝soniya singh

Professional Resume Template for Software DevelopersVinodh Ram

Cloud Management Software Platforms: OpenStackVICTOR MAESTRE RAMIREZ

(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...gurkirankumar98700

Asset Management Software - InfographicHr365.us smith

BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEOrtus Solutions, Corp

Salesforce Certified Field Service ConsultantAxelRicardoTrocheRiq

Der Spagat zwischen BIAS und FAIRNESS (2024)OPEN KNOWLEDGE GmbH

why an Opensea Clone Script might be your perfect match.pdfjoe51371421

The Evolution of Karaoke From Analog to App.pdfPower Karaoke

Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideChristina Lin

What is Binary Language? Computer Number SystemsJheuzeDellosa

Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...MyIntelliSource, Inc.

The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdfkalichargn70th171

Recently uploaded (20)

XpertSolvers: Your Partner in Building Innovative Software Solutions

Unit 1.1 Excite Part 1, class 9, cbse...

cybersecurity notes for mca students for learning

Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data

ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...

What are the features of Vehicle Tracking System?

Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝

Professional Resume Template for Software Developers

Cloud Management Software Platforms: OpenStack

(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...

Asset Management Software - Infographic

BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE

Salesforce Certified Field Service Consultant

Der Spagat zwischen BIAS und FAIRNESS (2024)

why an Opensea Clone Script might be your perfect match.pdf

The Evolution of Karaoke From Analog to App.pdf

Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide

What is Binary Language? Computer Number Systems

Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...

The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf

AI On Your Phone: What's it Good For

1. AI On Your Phone @ray_deck github.com/rhdeck/boscc29

2. What’s it Good For?

5. Voice

8. What is a Machine Learning Model?

9. Artificial Intelligence Deep Learning Machine Learning Statistical Computing

10. Supervised Learning Unsupervised Learning Reinforcement Learning

11. Ops and Numbers Input Shape “The Graph” Conv2D, ReLU, MaxPool, etc… Output Shape

12. y = mx + b

13. Ops Convolutions Pooling ReLU TanH Sigmoid LSTM GRU Dense/Fully-Connected More…

14. High-Dimensional Numbers

15. Images are High-Dimensional Numbers!

16. Old Stuff Works https://arxiv.org/pdf/1803.01271.pdf

17. Packaging

18. Models (Can Be) Files Shapes of Input(s) Operations Hyperparameters Weights Sources & Drains Shapes of Output(s)

19. What Do we Need? 1.Convert Image/Frame to Normalized Number 2. Run the model efficiently on local resources 3.Convert output number to what we seek: •Dictionaries of labels •Generated Image •Prediction •etc

20. Lite

21. ML Universe Cloud Local Train Inference Tensorflow, Theano, CNTK… Azure, Amazon, Google, MachineBox… …CoreML, TF Lite, TF Mobile, TF.JS Caffe Lite …plus MacGyver, Algorithmia…

22.

23. iOS Runtime and Bindings File Format Conversion Tooling

24. Runtime and Bindings CVPixelBuffer in, Dictionary Out Support for OTA/Runtime Download and Compilation Uses Metal Shaders for processing Limited Operation Support

25. .mlmodel 1. Protobuf 2. Feature Normalization •Value Range of Numbers (0-255 -> 0-1) •Mean Value 3. Channel Ordering •BGR vs RGB 4. Output Labeling/Denormalization

26. Conversion Keras – coremlconverter Tensorflow – tf-coreml CNTK - mmdnn MXNet – mxnet-to-coreml

27. Advantages Ease of Application Speed Multiple analyses per second Separation of Concerns Compilation vs Inference Source Agnostic

28.

29. Lite Multiple Runtimes File Format Converter

30. Lite Cross-Platform iOS, Android, Raspberry Pi feed() and run() No Compilation Step Limited Operations Number-to-Number

31. Lite .tflite FlatBuf Quicker Memory Access Alternative Precision: Bfloat16

32. Lite Conversion toco Tensorflow Optimizing Converter

33. iOS Only Multiplatform Input/Output Affordance Just the Numbers Production Beta Chunky Small

34. Cross-Platform Optimizations

35. Quantization

36. Pruning

37. Planning VGG16 550MB InceptionV3 94MB MobileNet 17MB