Top 4 Ways to Build Machine Learning Prediction on the Edge for Mobile and IoT Machine Learning at the edge is about making your apps smarter with real-time object and face recognition, offline predictions, and protecting user privacy. In this session, we'll explore and demo multiple ways to handle prediction, inference, and decision making in mobile apps. Demystify deep learning and easily build, train, and deploy ML models to mobile and IoT devices. Level: Intermediate Speaker: Dennis Hills - Developer Advocate, AWS Mobile Applications

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Top 4 Ways to Build Machine Learning
Prediction for Mobile and IoT
Dennis Hills
Mobile Developer Advocate, AWS Mobile
Pop-up Loft

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Smart Applications
Sense
Generate and receive rich data
about the environment
Infer
Extract relevance from huge
amounts of data in real time
Action
Take smart actions

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Benefits of ML on the Edge
Latency Bandwidth Availability Privacy

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deep Learning challenges at the Edge
• Resource-constrained devices
• CPU, memory, storage, power consumption.
• Network connectivity
• Availability, cost, bandwidth, latency.
• On-device prediction may be the only option.
• Deployment
• Updating code and models in mobile apps or a
fleet of devices is not easy.

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Prediction/Inference on Mobile

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Call Application Services 1
PLATFORM SERVICES
APPLICATION SERVICES
FRAMEWORKS & INTERFACES
Caffe2 CNTK
Apache
MXNet
PyTorch
TensorFlo
w
Torch Keras Gluon
AWS Deep Learning AMIs
Amazon SageMaker AWS DeepLens
Transcribe Translate Polly Comprehend LexRekognition

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Build/Train in Cloud – Deploy Model to Device 2
For iOS => Use Core ML to access a local on-device
trained model.
For Android => Use TensorFlow Lit to interact with the ML
Model on the device

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Build/Train in Cloud – Host Model Behind API 3
• Train a model in SageMaker (or bring your own).
• Deploy it to a prediction endpoint.
• Invoke the HTTP endpoint from your devices.
Best when
Devices are not powerful enough for local inference.
Models can’t be easily deployed to devices.
Additional cloud-based data is required for prediction.
Prediction activity must be centralized.
Requirements
Network is available and reliable.
Devices support HTTP.

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Utilize Platform APIs 4
The Vision framework from Apple performs face and face landmark detection, text
detection, barcode recognition, image registration, and general feature tracking. Vision
also allows the use of custom Core ML models for tasks like classification or object
detection.
Use Apple’s Natural Language framework to perform tasks like language and script
identification, tokenization, lemmatization, parts-of-speech tagging, and named entity
recognition.

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Recap
Call Managed Application Services
Build/Train in Cloud – Deploy Model to Device
Build/Train in Cloud – Host Model Behind API
Utilize Platform APIs
1
2
3
4

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Build/Train Custom Models

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
End-to-end
Machine Learning
Platform
Zero setup Flexible model
training
Pay by the second
Introducing Amazon SageMaker

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon SageMaker
1 2 3 4
I I I I
Notebook Instances Algorithms ML Training Service ML Hosting Service

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Real Use Cases of ML in Action
on Mobile & IoT

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you
Get Started:
aws.amazon.com/mobile
AWS Mobile Twitter:@AWSforMobile
Dennis Hills: @dmennis