AWS IoT & ML Recap - 20180423

AWS IoT & ML Recap
Taka Wang
2018.04.23

IoT Foundation
Digitization of the Physical World

Greenfield vs. Brownfield
Brownfield
Greenfield
VS
Courtesy of arabianbusiness & Colliers International

Clash of Two Worlds
Machine Camp vs. Internet Camp
(aka. OT vs. IT)
Machine Camp
‣ Brown field
‣ Strong company heritage, risk aversion
‣ Corporate career is the norm
‣ Domain: Physics, engineering
‣Think big
‣ Waterfall approach
‣ Standards like DIN/ISO
‣ Long QA & release cycles (defect free)
‣ Long lead time
Internet Camp
‣ Green field
‣ High-risk, VC-driven culture
‣ Entrepreneurial management and employees
‣ Domains: IT, services
‣ Focus on point solutions/MVP
‣ Agile approach
‣ Open source
‣ Perpetual beta (“Fast patches”)
Reference: Enterprise IoT, 2nd

IIoT vs. CIoT
Courtesy of National Instruments
Categories IIoT IoT
Impact Evolutionary Revolutionary
Focus Data Things
Connectivity Structured Ad Hoc
Criticality
Mission
Critical
Important but
not critical
Standards
Existing
Devices &
Standards
New Devices
& Standards
Reference: Industrial Internet of Things: PROFINET Intro
Further reading: Compare IoT and IIoT: Find 11 Differences

Why Edge? (or Cloud-Centric Assumption)
• There is sufficient bandwidth to push data to the Cloud
• Connectivity is not an issue. A device will (almost) always be connected
to the cloud
• The latency induced by cloud-centralized analytics and control is
compatible with the dynamic of the IoT system
• The connectivity cost is negligible
• Industrial companies are comfortable in exposing their data to the cloud
Reference: Fog Computing with Vortex

Fog Computing vs. Edge Computing
Courtesy of Forest Giant
Data
processing
Data
processing
Data
processing
Data
processing
Data processing

Which edge are you talking about?

Local Area Network
eNodeB
Mobile Core
MEC
ISP Backbone
ISP
Internet
eNodeB
Cloud, data centers,
app, services
Local Sensors/Actuators
4G, 5G
4G, 5G
Network Edge
Fog Node/IoT Edge Gateway
Customer Edge
MobileMulti-Access Edge Computing
Which edge are you talking about?
offload

intermittent connectivity
intermittent connectivity
Local connectivity
AWS IoT Architecture Recap
Things Edge/Gateway
Cloud
Things

AWS IoT Product Timeline
Things
Cloud
AWS IoT Core/SDK
15’ Oct
AWS IoT Device SDK
AWS IoT Core
AWS IoT Device SDK
17’ Jun
AWS Greengrass
Edge/Gateway
AWS
GreenGrass
AWS IoT Device
Mgmt.
AWS FreeRTOS
AWS FreeRTOS
17’ Nov
Amazon FreeRTOS
Amazon IoT Device Mgmt.

Amazon FreeRTOS
Amazon IoT Device
Mgmt.
AWS IoT Device SDK
AWS IoT Core AWS GreenGrass
AWS ML*

Amazon FreeRTOS
Device Software Stack
Reference: Amazon FreeRTOS Architecture
Embedded User Application
Wi-Fi Mgmt.
Library
Greengrass
Discovery
Shadow
Library
MQTT
Agent
Amazon FreeRTOS Internal Libraries
(TLS, OTA.., etc)
FreeRTOS
Kernel
Vendor Drivers
Hardware

Problem Solution Unique Value Proposition Unfair Advantage Customer
Segments
Key Metrics Channels Early Adopters
Cost Structure Revenue Streams
免費使⽤ FreeRTOS，透過 AWS IoT Device
Management (OTA) 與 IoT Core 等服務收取
Subscription fee
FreeRTOS project 的
Founder 在 Amazon
負責這項產品
- Chipset Vendor
- 既有的FreeRTOS
⽤⼾
預先整合(pre-integrated)
AWS IoT Device SDK 的
FreeRTOS 開發包
- MCU類的 Cloud/Edge
connectivity 開發包
- 預先整合好的 Security
Library
- 預先整合好的 secure
OTA update
- 整合 AWS 的 Device
Shadows 解決網路斷斷
續續問題
Greenfield
Machine/Device
Maker
- 導入的 chipset 數⽬
- Device Management 服務
註冊的 Device 數
Machine/Device
Maker 想要開發 MCU
類的 IoT 產品需要⾃
⾏處理
1. 安全性
2. 與Cloud串接
3. Wi-Fi 管理
4. 遠端裝置管理更新
等的開發問題
- Chipset Vendor
- FreeRTOS 社群
- Webinar, Workshop
- 開發⼈員⼈⼒成本
- Device Management Service 維運成本
- Enablement/Engagement的⼈⼒成本
Amazon FreeRTOS
Lean Canvas
Problem Solution
Cost Structure Revenue Streams
Unique Value Proposition Unfair Advantage Customer
Segments
Key Metrics Channels
Alternatives
Azure Sphere
Rev. 2

AWS IoT Device Mgmt.
Device Provisioning Fleet Indexing and Search Device Jobs
Reference: AWS IoT Device Management
Onboard Organize Update Output
User can onboard a large
number of devices at once
using device provisioning
Organize devices into groups which
can also be arranged into hierarchies
AWS IoT Device Management sends
a device job over-the-air (OTA), such
as a firmware update
Firmware updates can be sent to a
group or individual devices

AWS IoT Core/SDK
Things
Applications
AWS IoT Core AWS
MQTT over TLS
MQTT over WebSocket/TLS
HTTP
Reference: Smart Home Building Blocks with Intel Edison & AWS
Device SDK
Certificate
Allows devices to seamlessly
drop on/off connection
Manages
device identity
Allows integration with
other AWS services (route)
Communicates with
authenticates &
authorized devices
Device
Gateway
Rules Engine
Thing
Shadow
Thing
Registry
MQTT
HTTP (REST)
Amazon
CloudWatch
Amazon
Kinesis
Amazon
ES
Amazon
DynamoDB
Amazon
S3
Amazon
Lambda
Amazon
SQS
Amazon
SNS
Amazon
Machine Learning
Actions

AWS IoT Rules Engine
SELECT DATA FROM TOPIC WHERE FILTER ACTIONS
Reference: (MBL312) NEW! AWS IoT: Programming a Physical World w/ Shadows & Rules
{
"awsIotSqlVersion": "2016-03-23",
"sql": "SELECT * FROM 'iot/test'",
"ruleDisabled": false,
"actions": [
{
"s3": {
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_s3",
"bucketName": "my-bucket",
"key": "myS3Key"
}
}
]
}
Rule
Name
Description
SQL Statement
Array of Actions

AWS IoT Thing Shadows
Thing Shadow Mobile App
AWS IoT
Intermittent
Connection
REST APIs
Reported
Desired
Delta
• Report current state to one or multiple shadows
• Retrieve its desired state from shadow
Shadow reports delta, desired and reported states
along with metadata and version
• Set the desired state of a device
• Get the last reported state of the device
• Delete the shadow
Reference: The Lifecycle of an AWS IoT Thing

Thing Shadow - Step0
Thermostat
Thing Shadow
Permissions
Cognito Identity
Thing Shadow
68 Reported State
Desired State
State Delta
*nest is trademark owned by Google LLC
68
Reference: AWS Thing Shadow
Authenticated
X.509 Certificate
68
Report

Thermostat
Thing Shadow
68 Reported State
Desired State
State Delta
68
70
68
70
Update

Thermostat
Thing Shadow
68 Reported State
Desired State
State Delta
68
70
68
70
Calculate
+2
Pub
Sub

Thermostat
Thing Shadow
68 Reported State
Desired State
State Delta
70
68
70
70
70
1. React
2. Update

Thermostat
Thing Shadow
68 Reported State
Desired State
State Delta
70
68
70
70
70
Sub
Pub
Confirm

AWS IoT Device SDK
Embedded C
(Embedded OS)
C++11
Javascript
Node.js/Browser
Arduino Library
(Arduino Yún) Java 1.7+ Python
Android iOS
Reference: Getting Started with AWS IoT
Reference: AWS IoT Device SDK

Embedded C SDK - Layered Architecture
Reference: AWS IoT Embedded C Device SDK
AWS IoT
Embedded C SDK
Hardware / OS
Customer Application
Linux/LwIP TCP Sockets
OpenSSL/MbedTLS/… Linux/RTOS Timers
Network(TLS) Interface
network_interface.h
Timer Interface
timer_interface.h
AWS MQTT Library - derived from Paho Embedded C
MQTT Interface
aws_iot_mqtt_interface.h
Thing Shadow
aws_iot_shadow_interface.h
Customer Application

Types of Machine Learning
Machine
Learning
Supervised
Learning
Classification
Regression
Reinforcement
Learning
Unsupervised
Learning
Clustering
Dimensionality
Reduction
Reference: Machine Learning Algorithm - Backbone of emerging technologies

Machine Learning Workflow
Training Data
Test Data
Machine Learning
Algorithms
Production
Data
Machine
Learning
Model
Prediction
Iterations

Training vs. Inference
Large N
Forward
Backward
“car”
?
Error
TRANING
Smaller,
varied N
Forward
“car”
INFERENCE
Reference: Discover the Difference Between Deep Learning Training and Inference

AWS Machine Learning
binary classification
(binary attributes)
multiclass classification
(categorical attributes)
Regression
(numeric attributes)
AWS ML Process
Prepare Data
(Transform)
Create a Training
DataSource
Create a ML
model
Review model &
Set a score
threshold
Use model to
generate
predictions
Blackbox

AWS SageMaker
Jupyter
Notebook
Algorithms Training Hosting
‣ Interactive Environment
‣ build-in algorithms
‣ pre-built train/inference images
‣ your own images
‣ Scalable CPU/GPU Cluster
‣ Inference code image
‣ Trained model
First Launch: 07’ Nov.

Training code
Amazon ECR
Amazon SageMaker
Client Application
Model Training (on EC2)
Reference: Amazon Web Services
Inference code

Training code
Helper code
Training code
Amazon ECR
Amazon SageMaker
Client Application
Training Data
Inference code

Inference code
Training code
Helper code
Training code
Amazon ECR
Amazon SageMaker
Client Application
Model artifacts
Training Data

Inference code
Training code
Helper code
Inference code
Helper code
Training code
Amazon ECR
Amazon SageMaker
Client Application
Model Hosting (on EC2)
Model artifacts
Training Data

Inference code
Training code
Helper code
Inference code
Helper code
Training code
Amazon ECR
Amazon SageMaker
Client Application
Inference Endpoint
Inference request
Inference response
Model artifacts
Training Data

Inference code
Training code
Helper code
Inference code
Helper code
Training code
Amazon ECR
Amazon SageMaker
Client Application
Inference Endpoint
Inference request
Inference response
Model artifacts
Training Data
Ground
Truth

Edge Gateway
/Fog Node
Greengrass core
(Runtime)
Edge Devices
AWS
Greengrass core
(Runtime)
Device SDK
Device SDK
Factory Home

AWS Greengrass Features
Connectivity Latency Data
Local Lambda
function
Local
Actions
Local Messaging
Local
Triggers
Local Device
Shadow
Data and
state sync
ML Inference
Local
Inference
Concerns

Local Actions
Process
Process
Process
Process
Process
Arbitrary IPC
new
Engine
Running
Action
Running
Action
Running
Action
Trigger Trigger
Pod of actions
Python JS …
Event-Driven Programming Dataflow Programming
Process
Process
Process
Process
Process
new
Programming paradigms

Local Inference
Training
Data
Test Data
Machine Learning
Algorithms
Machine
Learning
Model
Cloud Infrastructure
Edge Infrastructure
Serverless
function
Production
Data
Predictive Analytics
Inference
Training
Data
Test Data
Machine Learning
Algorithms
Machine
Learning
Model
Cloud Infrastructure
Edge Infrastructure
Production
Data
Machine
Learning
Model
Predictive Analytics
Deployment
Inference
Reference: How Edge Computing And Serverless Deliver Scalable Machine Learning Services
Local Inference

Greengrass ML inference Workflow
ML model
Prediction
Data
AWS Greengrass
Core Device
Device performs inference
locally to action quickly
AWS
Greengrass service
AWS
SageMaker
Bring your own
ML model
Device data
Transfer trained
ML model
Edge/Gateway Cloud
Result from
inference processing
Trained ML models
Transfer ML models to
Greengrass devices
Build and train ML models
in the cloud
Reference: Perform Machine Learning Inference

Local Triggers
Core
Service
GG
Core
Device Device
MQTT Broker
MQTT Client MQTT Client
Lambda functions
Core Subscriptions
Devices
Group Definition
Source Destination
Deploy
Lambda funcs.
Subscriptions
REST Request
Group Name
IP address of Core
Group's root CA Cert.
Service Discovery

OPC-UA Bridge
PLC1
PLC2
OPCUA
Server
Processing Lambda
OPCUA
Adapter
/opcua/server/node/plc1
/opcua/server/node/plc2
Topic namespace for PLC2
Topic namespace for PLC1
Reference: AWS & Sample
Bridge

Summary - simplified model
AWS IoT
Device SDK
AWS
GreenGrass
AWS IoT Core
API Gateway
MQTT broker
clusters
AWS Ecosystem
MQTT Client
MQTT Broker
OTA agent
Daemon
State Mgmt.
Function
Watchdog
MQTT Client
TLS Library

Discussions of Greengrass
‣ Price (3 Free Core / 1 Year)
‣ Linux only proprietary implementation (sandbox: Namespace + Cgroup)
‣ Lack of local service discovery (Device Node should connect to cloud
once!)
‣ Complicated and cumbersome procedures
‣ Good for Greenfield applications

Friendly Reminders
‣ Internet connection is not stable (in the context of IoT)
‣ Security is not easy
‣ Device Mgmt. is not easy: Partial OTA, reboot
‣ OTA is not easy: (Partition based, Package based, Version, OS, Die-Hard)
‣ OPC-UA is one of the OT protocols (esp. Greenfield)
‣ Certs & Keys deployment in volume manufacturing is not easy
‣ Windows is still matter (in the context of IoT)
Further reading: Top 10 Reasons People Aren't Embracing the IoT

Takeaways
Courtesy of CNN Money & AZ Quotes

Takeaways
Customer Segments
IoT or IIoT
Greenfield or Brownfield
Where is your edge
分類思考法簡化問題

Readings
‣ Taking the pulse of enterprise IoT - Mckinsey & Company, July 2017
‣ Top 10 Reasons People Aren't Embracing the IoT - IoTI, April 2016
‣ Compare IoT and IIoT: Find 11 Differences - RAMAX Group, Feb 2018
‣ Enterprise IoT, 2nd - Strategies and Best Practices for Connected
Products and Services - Dirk Slama et al., Nov 2015
‣ The Lifecycle of an AWS IoT Thing - AWS, Oct 2016
Comprehensive Marketing survey
Know your customers
Narrow Your Focus
Fundamentals for every IoT Practitioners
Comprehensive AWS IoT Core Introduction

Differences between MQTT broker
& AWS IoT Device Gateway
• No retained message support
• No QOS2 support
• No message persistence
Reference: Message Broker for AWS IoT > Protocol

AWS IoT Core
Port/Protocol/Authentication combinations
Protocol Authentication TCP Port
MQTT Client Certificate 8883, 443*
HTTP Client Certificate 8443
HTTP AWS Signature Version 4 443
MQTT over Websockets AWS Signature Version 4 443
Reference: AWS IoT Core Now Supports MQTT Connections with Certificate Based Client Authentication On Port 443

Nvidia’s updates EULA to ban the use of gamer-oriented
GPUs in data centers
Reference: Nvidia’s updates EULA to ban the use of gamer-oriented GPUs in data centers
Reference: Nvidia: Using cheap GeForce, Titan GPUs in servers? Haha, nope!
Courtesy of License For Customer Use of NVIDIA GeForce Software

Where is your Edge?
Reference: Demystifying Edge Computing -- Device Edge vs. Cloud Edge
Sensors & Applications Cloud Infrastructure
Application Services
Compute Storage Network
Cloud
Edge
Cloud Edge
ex. MEC
Device
Edge
Sensors & Applications Cloud Infrastructure
Application Services
Compute Storage Network
Device Edge
ex. IoT gateway,
PACs (Controller)
AWS GreenGrass
Azure IoT Edge

AWS IoT & ML Recap - 20180423

Recommended

Recommended

More Related Content

Similar to AWS IoT & ML Recap - 20180423

Similar to AWS IoT & ML Recap - 20180423 (20)

More from Jamie (Taka) Wang

More from Jamie (Taka) Wang (20)

Recently uploaded

Recently uploaded (20)

AWS IoT & ML Recap - 20180423