The document discusses machine learning and computer vision at the edge using AWS DeepLens. It describes how most machine data cannot reach the cloud due to factors like being in extreme environments or on medical/industrial equipment. It then provides an overview of tools that can help with building datasets, training models in the cloud, and deploying inference at the edge using AWS DeepLens and AWS Greengrass. These tools include Amazon SageMaker for training models and Amazon SageMaker Ground Truth for efficiently annotating datasets.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Get Started with Machine Learning
and Computer Vision Using AWS DeepLens
Julien Simon
Global Evangelist, AI & Machine Learning
@julsimon

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Computing is increasingly available at the edge
Machine Learning predictions at the edge would make devices smarter.
Could we simply invoke cloud-based models?

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Most machine data can’t reach the Cloud
Medical equipment Industrial machinery Extreme environments

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Why this problem isn’t going away
Law of physics Law of economics Law of the land

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
To-do list
❑ Build a data set
❑ Experiment with state-of-the art algorithms for computer
vision
❑ Train in the Cloud at any scale
❑ Deploy inference code and model at the edge

6. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Annotating large datasets is time-consuming

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker Ground Truth
https://aws.amazon.com/blogs/aws/amazon-sagemaker-ground-truth-build-highly-accurate-datasets-and-reduce-labeling-costs-by-up-to-70
Easily integrate
human labelers
Get accurate
results
K E Y F E AT U R E S
Automatic labeling via
machine learning
Ready-made and custom
workflows for images and
text
Label
management
Quickly label
training data
Private and public human
workforce

9. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Do it yourself or fully-managed: you decide!
Amazon EC2
c5 p3
AWS Deep Learning AMIAmazon SageMaker
1
2
3

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker
Build, train,anddeployMLModels atanyscale
1
2
3
17 built-in algorithms,
including 3 for computer vision

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Deep Learning-based algorithms and pre-trained models
Classification,detection,segmentation
[electric_guitar],
with probability 0.671

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker
Build, train,anddeployMLModels atanyscale
1
2
3

14. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploying at the edge with AWS Greengrass
Install the
Greengrass
runtime
Deploy and run
Lambda functions
Manage from
AWS Console
Same programming
model as in the Cloud
Local
communication
and orchestration

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploying models with AWS Greengrass ML Inference
Define models as
Greengrass
resources and
transfer them to
your devices
Inference
takes place
on devices
Devices take
action quickly –
even when
disconnected

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Run inference and
local actions on
device
Send insights
to the Cloud
Generic
Deploy model
and Lambda function
Write inference code
Setup Greengrass
Architecture
Train model

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
To-do list
✓ Build a data set
✓ Experiment with state-of-the art algorithms
for computer vision
✓ Train in the Cloud at any scale
✓ Deploy inference code and model at the edge
❑ Put it all in practice with a fun device

19. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DeepLens
HD video camera
Custom-designed
deep learning
inference engine
Micro-SD
Mini-HDMI
USB
USB
Reset
Audio out
Power
HD video camera
with on-board
compute optimised
for deep learning
Tutorials, examples,
demos, and pre-built
models
From unboxing
to first
inference in <10
minutes
Integrates with
Amazon
SageMaker and
AWS Lambda
10
MIN

22. Get started in minutes with sample projects
Detect and recognise objects.
OBJECT DETECTION
Classify your food.
HOT DOG NOT HOT DOG
Detect a cat or dog.
CAT AND DOG
Transfer a style onto video.
ARTISTIC STYLE TRANSFER
Recognise common activities.
ACTIVITY RECOGNITION
Detect faces of people.
FACE DETECTION

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Use your own models with AWS DeepLens
• AWS DeepLens can run TensorFlow, Caffe and Apache MXNet models
• Inception
• MobileNet
• NasNet
• ResNet
• Etc.
• Train or fine-tune your model on Amazon SageMaker
• Deploy to AWS DeepLens with AWS Greengrass

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS DeepLens

25. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. Lambda function: load the model

27. Lambda function: optimize a custom model
• Custom models need to be optimized for the on-board GPU.
• The first call optimizes the model, further calls do nothing.

28. Lambda function: get a video frame and predict

29. Lambda function: annotate live stream

30. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
https://gitlab.com/juliensimon/dlnotebooks/sagemaker/

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
ml.aws
aws.amazon.com/deeplens
aws.training/machinelearning

32. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Julien Simon
Global Evangelist, AI and Machine Learning
@julsimon
https://medium.com/julsimon

33. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.