Get the basics right before doing business with IoT. AWS IoT in conjunction with Apache NiFi is a good point of start.

1. www.immobilienscout24.de
Get the basics right
for your IoT business
IoT Innovation Lab @ Relayr – 2016-05-04 – Speaker: @KayLerch

2. Agenda
1. Know the tech stack
2. AWS can manage things for you
3. AWS is not the only truth
4. That’s Apache NiFi
5. AWS IoT basic concepts (Recap)
6. AWS IoT & Apache NiFi
7. Focus on business value
8. Focus on innovation
Seite 2
Get the basics right for your IoT business | Kay Lerch

3. Know the tech stack
before you do business
Seite 3
Get the basics right for your IoT business | Kay Lerch

4. A reference architecture for IoT
Seite 4
Source: WSO2 (IoT Whitepaper)
Value Value
Value
Value
Effort
Effort
Effort
Challenges:
• Set the basics right (EFFORT)
• Integrate (VALUE) in your current
business / IT landscape
• Be innovative!
Solution:
Minimize (EFFORT)
• Rely on managed services and tools
Maximize (VALUE)
• Have the basics right
• rely on established tools
• Think over your integration strategy
Drive innovation
• Think beyond borders
• disrupt business with combining
established things in a totally new fashion
Get the basics right for your IoT business | Kay Lerch

5. AWS can manage those basics
with a managed IoT solution
Seite 5
Get the basics right for your IoT business | Kay Lerch

6. What AWS (IoT) provides …
Seite 6
Device Registry
TLS Certificates with
associated IAM-like policies
Communication over HTTP and
MQTT
Message broker with topics
State persistence with shadows
Event processing and
forwarding with IoT Rules and
Rule actions
Device SDKs in Node.JS, C
and for Arduino Yún
Get the basics right for your IoT business | Kay Lerch

7. What AWS (IoT) provides …
Seite 7
Event processing and analytics
with Kinesis Eventstreams,
ML, EMR and Lambda
Data / event stores like S3 or
Dynamo DB
API Gateway for public
endpoint management
Have your IoT related key
metrics in CloudWatch
EC2 as a ground for your
(web) applications
communicating with devices
Get the basics right for your IoT business | Kay Lerch

8. AWS IoT core tents
Seite 8
Read this: “Core tenets of IoT” by AWS (as of April 2016)
Get the basics right for your IoT business | Kay Lerch

9. AWS is not the only truth
You surely have your lovely tools outside AWS
Seite 9
Get the basics right for your IoT business | Kay Lerch

10. A brief overview of data processing and analysis
That‘s quite a lot of tooling and technology …
Seite 10
Hidden complexities
Data
Producers
Data
Consumers
Event
Broker
(Big) data stores
Data analysis challenges
(Realtime) Data Processing & Analytics
Integration
challenges
Ingestionchallenges
Data security
challenges
Get the basics right for your IoT business | Kay Lerch

11. A brief overview of data processing and analysis
That‘s quite a lot of tooling and technology …
Seite 11
Hidden complexities
Data
Producers
Data
Consumers
Event
Broker
(Big) data stores
Data analysis challenges
(Realtime) Data Processing & Analytics
Integration
challenges
Ingestionchallenges
Data security
challenges
Get the basics right for your IoT business | Kay Lerch
If you want …
 a (realtime) big picture of your dataflows
 an option to overlook lineage of each data element
 have the flexibility to change things on the fly
 prioritize data
 overcome challenges of integrating the variety of
technologies with one overarching solution
 enforce security and compliance along dataflows
 rely on extensibility driven by OS community
 satisfy all those needs and keep your tools
 get rid of only those tools focused on moving data without
making concessions to overall performance
… then you might love:

12. Just to let you know …
Seite 12
Source: WSO2 (IoT Whitepaper)
Value Value
Value
Value
Effort
Effort
Effort
We are here
And a bit of here
Get the basics right for your IoT business | Kay Lerch

13. That’s Apache NiFi
Seite 13
Get the basics right for your IoT business | Kay Lerch

14. Seite 14
That‘s Apache NiFi
in one page
Apache nifi supports powerful and scalable directed graphs of data
routing, transformation, and system mediation logic.
Web-based user interface
Seamless experience between design, control, feedback, and monitoring
Highly configurable
Loss tolerant vs guaranteed delivery, Low latency vs high throughput, Dynamic prioritization, Flow can be modified at runtime, Back pressure
Data Provenance
Track dataflow from beginning to end
Designed for extension
Build your own processors and more, Enables rapid development and effective testing
Secure
SSL, SSH, HTTPS, encrypted content, etc..., Pluggable role-based authentication/authorization
Source: https://nifi.apache.org/ Get the basics right for your IoT business | Kay Lerch

15. Seite 15
That‘s Apache NiFi
in real and feel
Go to NiFi’s interface and understand:
Processors
Templates
Concept of back pressure
Concept of data prioritization
Provenence Graph
Get the basics right for your IoT business | Kay Lerch

16. NiFi Cluster
NiFi Cluster
Manager (NCM)
JVM
Node (Primary)
NiFi Clustered Architecture
JVM
Webserver
Provenance
Repository
Content
Repository
Flowfile
Repository
REST-APIAdminUI
Webserver REST-APIAdmin UI
Flow Controller
Cluster Manager
Processor 1
Processor 2
Isolated
Processor
Controller Service 1
Controller Service 2
Controller Service n
Heartbeat
Leader
election
Report
change
Embedded Apache Zookeeper
Node (Slave)
JVM
Webserver
Provenance
Repository
Content
Repository
Flowfile
Repository
REST-APIAdminUI
Flow Controller
Processor 1
Processor 2
Isolated
Processor
Controller Service 1
Controller Service 2
Controller Service n
Heartbeat
Report
change
Sync
State
Sync
State

17. AWS IoT
Basics (Recap)
Seite 17
Get the basics right for your IoT business | Kay Lerch

18. Seite 18
AWS IoT
The Shadow
AWS IoT
Thing
Thing
Shadow
Rule
Reports State Mirrors State
in Shadow
Gets reported
state or sets
desired state
Propagates
desired state
Receives
desired state
Fulfills
desiredstate
Subscribes to
particular messages
AWS Services
Some
AWS
Resource
Routes
message
TLS
1.2
TLS
1.2 Policy
Get the basics right for your IoT business | Kay Lerch

19. AWS IoT
MQTT topics
AWS IoT
Thing
Shadow
get
get/
accepted
get/
rejected
Request state
Get shadow state
Get error
update
update/
accepted
update/
rejected
update/
delta
Update state
Confirmation
Get error
Changed
state
1
2
1
2
3
Thing topics name pattern: $aws/things/thing_name/...
Get the basics right for your IoT business | Kay Lerch

20. Apache NiFi & AWS IoT
New processors
Seite 20
Get the basics right for your IoT business | Kay Lerch

21. Seite 21
Apache NiFi & AWS IoT
GetIOTMqtt – a MQTT client
AWS IoT
Thing
Shadow
update
Update state
Establish
Connection
Subscribe
Receive state
1
2
3
Flow
file
Get the basics right for your IoT business | Kay Lerch

22. Seite 22
Apache NiFi & AWS IoT
GetIOTMqtt – Configuration
Get the basics right for your IoT business | Kay Lerch

23. Seite 23
Apache NiFi & AWS IoT
GetIOTMqtt – Live demo
Get the basics right for your IoT business | Kay Lerch

24. Seite 24
Apache NiFi & AWS IoT
GetIOTMqtt – Live demo
Get the basics right for your IoT business | Kay Lerch

25. Seite 25
Apache NiFi & AWS IoT
PutIOTMqtt – instruct a „thing“ (but bypass the shadow)
AWS IoT
Thing
Shadow
update /
delta
Update state
Establish
Connection
Publish state
1
2
Flow
file
Flow
file
Get the basics right for your IoT business | Kay Lerch

26. Seite 26
Apache NiFi & AWS IoT
PutIOTMqtt – Configuration
Get the basics right for your IoT business | Kay Lerch

27. Seite 27
Apache NiFi & AWS IoT
PutIOTMqtt – Live demo
Get the basics right for your IoT business | Kay Lerch

28. Seite 28
Apache NiFi & AWS IoT
GetIOTShadow – constantly check last reported state
AWS IoT
Thing
Shadow
update
Report state
Request
Shadow
Flow
file
Flow
file
Get the basics right for your IoT business | Kay Lerch

29. Seite 29
Apache NiFi & AWS IoT
GetIOTShadow – Configuration
Get the basics right for your IoT business | Kay Lerch

30. Seite 30
Apache NiFi & AWS IoT
PutIOTShadow – instruct a „thing“ over its shadow
AWS IoT
Thing
Shadow
update /
delta
Desire state
Update
Shadow
Flow
file
Flow
file
Get the basics right for your IoT business | Kay Lerch

31. Seite 31
Apache NiFi & AWS IoT
PutIOTShadow – Configuration
Get the basics right for your IoT business | Kay Lerch

32. We got the basics
Let’s focus on business value
Seite 32
Get the basics right for your IoT business | Kay Lerch

33. Homework almost done
Seite 33
Source: WSO2 (IoT Whitepaper)
Value Value
Value
Value
Effort
Effort
Effort
Challenges:
• Set the basics right (EFFORT)
• Integrate (VALUE) in your current
business
• Be innovative!
Solution:
Minimize (EFFORT)
 Rely on managed services and tools
Maximize (VALUE)
 Have the basics right
 rely on established tools
• Think over your integration strategy
Drive innovation
• Think beyond borders
• disrupt business with combining
established things in a totally new fashion
Get the basics right for your IoT business | Kay Lerch

34. Agile integration strategy with AWS Simple Workflows at ImmobilienScout
Legacy / 3rd Party
AWS Simple Workflows (SWF)
AWSSNS
AWS
Lambda
AWSS3
AWSEC2
AWSECS
OnPremise
Broker
Start
Workflow
Put File
Bucket Topic
Publish
Message
Start
Workflow
Activity
Task
Func
tion
Func
tion
Trusted Applications / Mobile Apps
AWS Flow
Framework
Start
Workflow
Event
Trigger
Send Data
Invoke Service
Decider
Apps
AWS
Cloudwatch
AWS
CloudTrail
Logs
Metrics
Alarms
Log Measure
Activity
Task
Activity
Task
Decision
Task
Decision
Task
Workflow
History
AWS Flow
Framework
AWS***
Any
Activity
Task
Decision
Task
Start
Workflow
Trigger Trigger
Notify
API
Call
Discover
Decision
Task
Service
Apps
AWS Flow
Framework
Activity
Task
AWSIAM
Policies
Manage
Permissions
Func
tion
Lambda
Task
API
Call
IoT Device
AWSIoT
Topic
Publish
Message
Func
tion
Start
Workflow
Rule
Dynamo
Start
Workflow
Put File
Table
Func
tion
Trigger
Partner
AWSAPI
Gateway
Start
Workflow
Invoke
Func
tion
Route
Service
Lambda
Task
Decision
Task

35. Hybrid integration platform
aiming for Self-Service integration

36. We got the basics
Let’s focus on innovation
Seite 36
Get the basics right for your IoT business | Kay Lerch

37. Homework done!
Seite 37
Source: WSO2 (IoT Whitepaper)
Value Value
Value
Value
Effort
Effort
Effort
Challenges:
• Set the basics right (EFFORT)
• Integrate (VALUE) in your current
business
• Be innovative!
Solution:
Minimize (EFFORT)
 Rely on managed services and tools
Maximize (VALUE)
 Have the basics right
 rely on established tools
 Think over your integration strategy
Drive innovation
• Think beyond borders
• disrupt business with combining
established things in a totally new fashion
Get the basics right for your IoT business | Kay Lerch

38. Gain real business value
Seite 38
Source: WSO2 (IoT Whitepaper)
Value Value
Value
Value
Effort
Effort
Effort
1) Combine values of different worlds to
find a breathtaking use case
2) Rely on your basics
3) Build a product around your solution
4) Change the world
Value Value+
Effort Effort Effort+ +
+
=
+
Adopt new things…
Get the basics right for your IoT business | Kay Lerch

39. Gain value
Demo (Smart home)
Seite 39
+ +
Get the basics right for your IoT business | Kay Lerch
Pick some fancy frontend (VALUE) Pick some fancy “backend” (VALUE)
Rely on your
basics (EFFORT)
= Have
Smart Home

40. Gain value
Demo (VoiceOps)
Seite 40
+ +
Get the basics right for your IoT business | Kay Lerch
Pick some fancy frontend (VALUE) Pick some fancy “backend” (VALUE)
Rely on your
basics (EFFORT)
= Have
VoiceOps

41. Once you got the basics right, it’s up to you to …
Seite 41
Get the basics right for your IoT business | Kay Lerch

42. www.immobilienscout24.de
Thanks for you attention. Any questions?
Contact:
Immobilien Scout GmbH
Andreasstraße 10
10243 Berlin
Kay Lerch
Fon +49 30 24 301-1149
kay.lerch@immobilienscout24.de

43. Backup
Seite 43
Get the basics right for your IoT business | Kay Lerch

44. T-Cobbler
I need some “thing” …

45. Input & Output units on your
breadboard
I need some “thing” …

46. Control GPIO-Pins with sending
or receiving electric impulses
Control an output Control an input

47. Communicating with AWS IoT over MQTT topics

48. Alexa SkillAWS Lambda
AWS API Gateway
Lambda
Post
Message to
Slack
Lambda
Update
Shadow
Rpi 2
AWS IoT
Thing
Shadow
Python ScriptGPIO
Rule
Invoke
MQTT
Publish
MQTT
Receive
Update Shadow
on state change
Impulse
Sequence
SVC
Outgoing
Webhook
Post
Message
over Slack
Web API
433 Mhz
Signal
C++ D/A
Converter
Post
Message if
no state
changes

49. AWS Lambda
AWS EC2
Lambda
Start / Stop
EC2-
Instance
Rpi 2
AWS IoT
Thing
Shadow
Python Script
GPIO
Rule
Invoke
MQTT
Publish
Impulse
Sequence
Post Message over Slack Web API
433 Mhz
Signal
C++ A/D
Converter
Start / Stop over AWS SDK
AWS CW
Lambda
React on
EC2 state
change
Rule
Update Shadow
MQTT
Receive
Impulse
On state
change

50. Rpi 2
AWS IoT
Thing
Shadow
Python Script
GPIO
Rule
MQTT
Publish
Impulse
Sequence
Post Message over Slack Web API
AWS CW
Alarm
On Humidity > 30%
Metric Metric
Alexa Skill
Read out temperature
and humidity