The document discusses machine learning at the edge for industrial applications. It describes an AIoT (Artificial Intelligence of Things) lifecycle that includes data transport and routing, data aggregation and processing, machine learning model generation in the cloud, and model inference at the edge. It provides examples of using AWS services like IoT, SageMaker, and Greengrass in an industrial IoT architecture. The presentation also covers topics like developing and deploying models on edge devices and integrating machine learning into mixed criticality systems.

1. S U M M I T
Taipei

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine Learning at the Edge
for Industrial Applications
Richard Elberger
Global Partner Solutions Architect, IoT
Amazon Web Services, Partner Network

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
Tenets
Industrial IoT architecture
AIoT lifecycle – a four-part story

4. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Effectively maintain the system over its
lifecycle

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Exploit system cost effectiveness via new
intelligence

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Better decisions through central dashboards
and monitoring

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Derive value through new capabilities

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS IoT
Greengrass
Amazon
FreeRTO
S
Amazon
FreeRTO
S
Amazon
FreeRTO
S
Amazon
FreeRTO
S
Amazon
FreeRTO
S

10. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Industrial
Control
Fieldbus

12. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground
Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
application
s

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground
Truth
XILINX
DNNDK
AWS
Snowmobile
AWS IoT
Greengrass
UltraScale
+
(DPU)
Amazon
FreeRTOS
Zynq7000
(DPU)

15. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground
Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
application
s

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
In the beginning

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Judgment on how to bring in data

19. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground
Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
application
s

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Ingesting and methodically curating

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Iteratively evaluating and refining
Where data science meets art
annotatio
n
cleansing
data types
re-process
raw data

23. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground
Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
application
s

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
FRAMEWORKS
ML Frameworks +
Infrastructure
ML Services
AI Services
INTERFACES INFRASTRUCTURE
Amazon
SageMaker
Amazon
Transcribe
Amazon
Polly
Amazon
Lex
CHATBOTS
Amazon
Rekognition
Image
Amazon
Rekognition
Video
VISION SPEECH
Amazon
Comprehend
Amazon
Translate
LANGUAGES
P3
P3d
n
C5 C5n Elastic inference Inferentia AWS Greengrass
Ground Truth Notebooks Algorithms + Marketplace RL Training Optimization Deployment Hosting
AWS Confidential - Do not Distribute

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Deep learning frameworks and toolchains

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Feeding the art: data sets

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Training job: compilation

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Staging

30. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground
Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
application
s

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Mixed criticality system – block diagram

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Mixed criticality system – software placement
PS PL
AWS IoT
Greengrass

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Mixed criticality system – pin wiring
PS PL

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Updating the model

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
InferenceHandle
r
m2
m
PS
PL
AWS IoT
Greengrass

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Device level dependencies
InferenceHandle
r
m2
m
PS
PL
/dev/uio0
/dev/uio1
/dev/i2c-0
/dev/i2c-1
/dev/mem
AWS IoT
Greengrass

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Reporting inference results
ImageUploadHandl
er
m2
m
PS
PL
AWS IoT
Greengrass

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Staging data for building up the data lake
ImageStageHandle
r
m2
m
PS
PL
AWS IoT
Greengrass

40. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Recap
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground
Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
application
s

42. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS DeepLens
The world’s first deep learning-
enabled video camera for
developers
• A new way to learn ML though sample projects, with practical,
hands-on examples
• Run deep learning models locally on the camera to recognize
or classify without streaming to the cloud
• Easy to customize and fully programmable using AWS Lambda
• Integrated with Amazon SageMaker for custom model
deployment
• Runs on any deep learning framework, including Apache
MXNet, TensorFlow, and Caffe.
Available now on amazon.com for $249

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
GET STARTED WITH SAMPLE PROJECTS
ARTISTIC STYLE TRANSFER
OBJECT DETECTION FACE DETECTIONHOT DOG / NOT HOT DOG
CAT VS. DOG
ACTIVITY DETECTION
Or build custom deep learning models in the cloud using
Amazon SageMaker
HEAD POSE DETECTION

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Intel Atom® Processor
• Gen9 graphics
• Ubuntu OS- 16.04 LTS
• 100 GFLOPS performance
• Dual band Wi-Fi
• 8 GB RAM
• 16 GB storage (eMMC)
• 32 GB SD card
• 4 MP camera with MJPEG
• H.264 encoding at 1080p resolution
• 2 USB ports
• Micro HDMI
• Audio out
• AWS Greengrass preconfigured
• clDNN-optimized for MXNet
• Key Differentiators/Technologies
• Intel cLDNN Library optimized for MXNet
• Intel Deep Learning Deployment Toolkit
AWS DeepLens Specifications

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS DEEPLENS ARCHITECTURE
Video out
Data out
I N F E R E N C E
D E P L O Y P R O J E C T S
Manage device
Security
Console Project
Management
AWS Cloud
Intel: Model Optimizer
cIDNN and Driver

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Richard Elberger
Global Partner Solutions Architect - IoT
https://github.com/rpcme/
https://www.linkedin.com/in/richardelberger/
https://twitter.com/richardelberger

48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.