Presented at AI NEXTCon Seattle 1/17-20, 2018 http://aisea18.xnextcon.com join our free online AI group with 50,000+ tech engineers to learn and practice AI technology, including: latest AI news, tech articles/blogs, tech talks, tutorial videos, and hands-on workshop/codelabs, on machine learning, deep learning, data science, etc..

1. AI NEXTCon Seattle ‘18
1/17-20th | Seattle
#ainextcon
http://aisea18.xnextcon.com

2. Machine
Learning in the
Physical World
Kip Larson
Principal Product Manager
AWS IoT Analytics

3. • Analytics in IoT
• Introducing AWS IoT Analytics
• AWS IoT Analytics Components
• ML Across IoT Fleets
• LSTM for IoT
What to expect from this session

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Devices
Sense & Act
Cloud
Storage & Compute
Intelligence
Insights & Logic → Action
Three pillars of IoT

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Why is AWS IoT Analytics important?
Industrial
automation
Improved product
design
Optimized business
processes
Improved user experience

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Analytics within AWS IoT
Devices Cloud
Greengrass
Rules Engine
AWS
Amazon
Redshift
a:FreeRTOS
Amazon Kinesis
Collect
Preprocess &
Enrich
Store
Analyze
Visualize
Corp Apps
Corp Data Center
Enterprise Applications
&
Non-IoT Data
Contextual
Data
Intelligence
Device shadow
Message Broker
Device Gateway
Device Registry
AWS IoT AnalyticsAWS IoT Core

7. Introducing
AWS IoT Analytics

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Analytics
IoT Analytics is a fully managed IoT analytics service
that collects, preprocesses, enriches, stores, analyzes,
and visualizes IoT device data at scale.
From raw
sensor data to
sophisticated
IoT analytics

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Analytics requirements
High volume of
data
Clean data Contextual
information

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Challenges with AWS IoT data and analytics
High volume Multiple sources Noisy and no standard
format
Incomplete and no
contextual information

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What customers are asking for
Business relevant
reporting
Preprocessing
unstructured, noisy data
Data collection from
multiple sources
Time series data storageAdvanced analytics
and machine learning

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
From AWS IoT data to sophisticated analytics
Thing Registry
Log InkLevel
Queue SQS
Ink Level
CloudWatch Log
Job History Tables
DynamoDB
deviceID/usage/
inklevels
Forward Ink
Levels Rule
Low Ink Test
Queue SQS
Log Ink Remaining
Lambda Worker
Alarm State
Worker Lambda
Send Mobile
App Alert
deviceID/events/
consumables Ink Replace Rule
Device Enricher
Worker Lambda
deviceID/events/jobs
Send Ink Alarm
Queue SQS Notifier Lambda
Low Ink Alarm
Queue SQS
Log Job
Queue SQS
Log Job Lambda
Worker
Job Ink Process
Queue SQS
Job Calculation
Lambda Worker
Job Alarm Threshold
Lambda Worker
Print Job
Raw Data
Storage
Lambda
Logging
2
2
21
3 3
4 4
Ink Level Remaining
CloudWatch Alarm
5 5 8
8
4
Sales Cloud
SFDC Customer,
Ink Order Table &
Promotions Table
6
7
7
5
6
Low Ink Test
Lambda Worker
Check Ink
Threshold
9
9
deviceID/inbound/
inkrefill
SFDC Ingest
Firehouse
10
11 11
9
12
19
14
14
18
Print Job Error
13 13
Device Enricher
Worker Lambda
Thing Registry
16
14
15 15
14
16
16
17
17 17
20
20
21
21
deviceID/*/*
Backup Rule
Job History &
Ink Level APIs
Enrich Job
Queue SQS
Enrich Job
Lambda Worker
Job Alarm
Queue SQS
Job Entry Completion
Lambda
19

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Analytics overview
Create IoT data Collect Preprocess &
enrich
Store Analyze & visualize

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Analytics components
DatasetsPipelines
Collect Preprocess
& enrich
Store Analyze & visualize,
machine learning
Channels Datastores Notebooks & Amazon QuickSight
QueryCreate IoT data
Operational and
business
applications

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IoT Analytics: Collect via channels
• Entry point to AWS IoT Analytics
• Native integration with AWS IoT Core
• Authoritative store of raw data from devices
• Built-in micro-batching partitioning by date
• Supports both binary and JSON data
MQTT
Amazon
Kinesis

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IoT Analytics: Preprocess & enrich
via pipelines
• AWS IoT specific device data preparation
pipeline
• Consumes raw data from a Channel and sends
processed data to a datastore
• Set of activities for filtering/transforming/
contextualizing messages
• Works best with JSON data

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Analytics: Pipeline activities
• Consume from a channel
• Send to a datastore
• Invoke AWS Lambda
• Add/remove/select attributes
• Apply regex
• Filter messages
• Validate messages
• Evaluate math expression
• Enrich from MQTT topic
• Enrich from AWS IoT device registry
• Enrich from AWS IoT device shadow

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Analytics: Store via datastores
• Store optimized for analytical & time series
queries against processed data
• Is not a database, but an abstraction on top of
several database technologies
• Works on semistructured JSON data
• Partitioned by time

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Analytics: Analyze via datasets
• Result of analysis against data store
• Run on schedule or ad hoc
• Conceptually similar to materialized view
• Accessible from the console/API/Jupyter
Notebooks/Amazon QuickSight

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Analytics: Powerful analytical
notebooks
• Jupyter-based machine learning notebooks (using
Amazon SageMaker)
• Integration with existing AWS IoT Analytics
datasets
• Built-in templates for predictive maintenance and
other IoT-specific use cases

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT Analytics: Visualize
via Amazon QuickSight
• Native integration with Amazon
QuickSight
• Makes it easy to build visualizations,
perform ad-hoc analysis, and get business
insights.

22. ML Across IoT Fleets

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IoT Fleet Problem Space
Fleets come in two classes: Consumer and Industrial
Consumer
Homogeneous,
non-interactive fleets
Industrial
Heterogeneous,
interacting fleets

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ML Mission Types in an IoT Context
• Missions have different levels of granularity across fleet types
• Business loss functions vary
• Some missions are fleet type-specific
Segmentation
“What do they have
in common?”
Survival Analysis
“How much longer till
it breaks?”
Classifiers
“Is it broken?”
Forecasting
“How much will
it produce?”
Optimization
“Can we improve
the way it works?”

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ML IoT Model Development
• Learn across the fleet, deploy to the device
• Model may be deployed in the cloud or on the device
• Opportunity to retrain may be limited, opportunity to
update covariates may be limited
• Almost always time series data
• Often multivariate

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Obvious Approaches Don’t Always Work
For business process forecasting (e.g. customer demand) we often deploy seasonalized time series
models, such as SARIMAX
𝑦"# = 𝜇 + 𝜑( 𝑦")( + 𝜑* 𝑦")* + ⋯ 𝜑, 𝑦"), + 𝜃( 𝜀")( + 𝜃* 𝜀")* + ⋯ 𝜃, 𝜀"), + Φ( 𝑦")0 + Θ( 𝜀")0 + 𝜉( 𝑥" + 𝜀"
IoT data often creates problems for these models:
Seasonal component
may exhibit epicycles
Data usually has
gaps
Seasonal component
may be evolving
Relevance of regressors may be
conditionally-relevant

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LSTM-based approaches
• Long Short-Term Memory networks show promise in
forecasting time series device telemetry
• Seasonal epicycles accommodated more easily than
SARIMA-style lag operations (needs data and hyper-
parameter optimization)
• Forget-gates useful for noise management
• Fewer cold-start problems
• Gap tolerant
• Areas of exploration: companion devices as inputs

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
After Model Training, now what?
Train in SageMaker
or Lambda
GreenGrass ML
on the edge
Automatically trigger
in IoT Analytics
Action in the cloud

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Get started with
AWS IoT Analytics
aws.amazon.com/iot-analytics

30. Thank you!
Interested in joining our team?
Message us at AWS-IoT-Analytics-Careers@amazon.com