IBM Internet-of-Things architecture and capabilities

© 2015 IBM Corporation
IBM
Analy*cs

IBM
Internet
of
Things
-‐
Architecture
and

Capabili*es

Kevin
Brown
–
Informix
CTO,
IoT
Solu7ons
and
Services

August
17,
2015

Agenda
•  IoT Architectural Components Overview
•  Informix for the Intelligent Gateway
•  Informix for the Cloud
•  IoT Partners and Platforms
2

3
IoT Architectural Components
Overview

4
Internet of Things Topography
Streams
Deep Analytics Zone
Device/Sensors
Smart Gateways
Sensor Analytics Zone
4

Simplified IoT Data Flow
Sensor Data History
Sensors
In-memory Analytics
Predictive
Analytics
Publish /
Subscribe
Cloud
Infrastructure
Real-time Analytics
Real-time Analytics Operational Analytics
Big Data
Analytics
(no gateway)
(Gateways)
HDFS / Hadoop
Big Data
Analytics
MessageSight
/ MQTT
Gateways for
local analytics
InfoSphere
Streams
Informix / Cloudant /
IOT Foundation
Service / TimeSeries
Service
Informix Warehouse
Accelerator / DashDB
PMQ / SPSS /
Cognos
Softlayer / BluMix
Watson / DashDB /
BigSQL
Informix /
Node-Red

•  Gateways can reduce the cost of the backend cloud
•  Reduces cloud storage by filtering/aggregating/analyzing data locally
•  Reduces cloud CPU requirements by precomputing values
•  Reduces latency since actions can be taken immediately
•  Intelligent gateways can detect and respond to local events as they
happen rather than waiting for transfer to the cloud
•  Some users are not comfortable putting all their data in the cloud
•  Gateways allow customers to capture and get value from their sensors
without sending data to the cloud
•  Protocol Consolidation
•  Cloud does not need to implement the 100’s of IoT protocols
Over time more and more of the processing will move from the cloud
to gateway devices
How Do Gateways Help IoT Solutions?
6

What are the Requirements for a Gateway Database?
•  The database management system must:
▪  Have a small install footprint, less than 100 MB
▪  Run with low memory requirements – less than 256 MB
▪  Use lossless compression or other techniques to minimize
storage space
▪  Have built-in support for common types of IoT data like time series
and spatial/GIS data
▪  Simple application development supporting both NoSQL and SQL
▪  Driverless, easy access to the data
▪  Require absolutely no administration
▪  Ideally should be able to network multiple gateways together to
create a single distributed database
7
The database must be powerful enough to ingest, process and
analyze data in real-time

IBM Informix: The Ideal Database for Gateways
Simple to use
▪  Hands-Free operation – No administration
▪  Supports popular interfaces such as REST and Mongo as well
as ODBC/JDBC
▪  Handles SQL and JSON data in the same database
Performance
▪  One of a kind support for Time Series and Spatial data
▪  Stream data continuously into the database
▪  Run analytics as data arrives
▪  Dynamically add and update analytics when needed
▪  Storage is typically 1/3 the size compared to other vendors
Invisible
Agile
8
Informix is the only database management system
perfectly suited to run in Gateways

Sensor Data is Time Series Data
•  What is a Time Series?
▪  A logically connected set of records ordered by time
•  What are the Key Strengths of Informix TimeSeries?
▪  Much less space required
• Typically about 1/3 the space required by other vendors
▪  Queries run orders of magnitude faster
• Unique optimized storage means codes paths are shorter and more
data fits in memory
▪  Purpose built streaming data loader for sensor data
• Automatically run analytic and/or aggregate functions on new data
▪  Can store structured (SQL) or unstructured (JSON) data for quick
application development
• REST/ODBC/JDBC/JSON interfaces available
▪  100’s of functions predefined
• Programming APIs available to create your own analytics
9

Traditional Table Approach
Informix TimeSeries Approach
Meter_ID Time KWH Voltage ColN
1 1-1-11 12:00 Value 1 Value 2 ……… Value N
… … … … ……… …
… … … … ……… …
Meter_ID Series
1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
…
Traditional Sensor data storage vs Informix TimeSeries
Storage
10

IoT Requirements for SpatioTemporal Data
•  Many IoT applications have a spatial
component to them
▪  Vehicles, cell phones, even pets…
tracking is common
•  In these cases both location and
time is important
▪  Show me the vehicles that have
passed by location X in the last
hour
▪  Where has my car been over the last
few hours?
•  Informix allows you to combine Time
series and Spatial data in the same
query
11

12
Both Structured and Unstructured Data is
Common in IoT Environments
JSON
Collection
SQL Driver
NoSQL Driver
SQL Data
Join Data
•  Informix can store SQL and JSON data in the same database
•  Write programs using SQL drivers or Mongo/NoSQL drivers
•  SQL data automatically transformed into JSON documents when needed
•  NoSQL data automatically transformed into SQL when needed
Embedded Device
or Database
server
Horizontal
Scale-out
with Shards

Informix Data Access Options
13
MongoDB
Client
REST Client
SQLI Client
DRDA Client
Informix
DBMS
Informix NoSQL
Listener
Informix
•  NoSQL ↔ SQL Translation
•  REST, MongoDB Protocol
Support
•  SQLI, DRDA Protocol Support
•  Relational, Collection, Time
Series, and Spatial Data
Support
Spatial
Time Series
JSON Collection
Relational Table
A
REST
client
is
any

program
capable
of

making
a
HTTP
request

Informix Data Access Options
14
MongoDB
Client
REST Client
SQLI Client
DRDA Client
Informix
DBMS
Informix NoSQL
Listener
Informix
•  NoSQL ↔ SQL Translation
•  REST, MongoDB Protocol
Support
•  SQLI, DRDA Protocol
Support
•  Relational, Collection, Time
Series, and Spatial Data
Support
Spatial
Time Series
JSON Collection
Relational Table
You
can
use
all
the
client

drivers
that
are
available

for
MongoDB
with
the

NoSQL
Listener

•  Rapid Development
•  Simple use with JSON
•  Simple REST
•  Simple MQTT and other adapters
•  Contributor Nodes – simple to use other services
15
Use Node-Red for Quick Gateway App. Development

IBM IoT Smart Gateway Kit
1.  Login as installing user.
▪  Ex: pi
2.  mkdir iot-gateway-kit-depend
3.  Run git clone https://github.com/ibm-iot/iot-gateway-kit.git
4.  cd iot-gateway-kit
5.  Run ./iot_install
17

IBM IoT Smart Gateway Kit
•  The iot-gateway-kit will install the following:
▪  Node.js
▪  Node-red
▪  Timeseries nodes
▪  Bluetooth node.js application sample
18

Smart Gateway Kit – TI Sensor Tag
19
1.  IoT gateway Kit
Designed using node-
red/node.js to work with
the TI Sensor Tag"
2.  Stores data in the
Timeseries database"
3.  Aggregate data and
push to the cloud"
4.  IoT Foundation or other."

IoT Developers - Get Started!
•  Gateway starter - https://ibm.biz/BdXr2W
•  Code samples - https://ibm.biz/BdX4QV
•  Github - https://github.com/IBM-IoT/
20

Informix on Docker Hub
https://registry.hub.docker.com/u/ibmcom/informix-innovator-c/
•  IBM Informix Innovator-C
•  12.10.FC4
21
Docker Hub
$docker pull ibmcom/informix-innovator-c!

22
Informix for the Cloud and
Operational Zone

23
What are the IoT Requirements for the Cloud?
•  Requirements - similar to gateways (but for different reasons):
•  Potentially 1000’s of servers means zero administration is a must
•  Data volume adds up very quickly so low storage overhead is required
•  Data flows into the cloud continuously and must be processed in real-time
•  Must be able to handle time series, spatial, and NoSQL data natively
•  Additional requirements
•  Must be able to scale-out
•  Must be available as a service
The database must be able to ingest, process and analyze
data in real-time

24
Why use Informix in the “Operational Zone”?
Simple to use
•  Hands-Free operation
•  Supports REST and Mongo APIs as well as ODBC/JDBC
•  Stores SQL and JSON database in the same database
Highly Available
•  Close to zero down time
•  Partition or Hash your data across servers in the cloud
•  Dynamically add/remove additional servers
Performance
•  Continuous High Performance Analytics
•  Specialized support for Time Series and Spatial data
Invisible
Agile
Resilient

25
Shards: Scale-out your Database across Servers or
Gateways
•  Distribute data among servers by
range or hash partitioning
•  Each shard can have an associated
secondary server for high
availability
•  Run queries across all shards or
a subset of the shards
•  Only shards that could qualify are
searched
•  Shards are searched in parallel
•  Ignores shards that are offline
Shards in a
Cloud

26
IoT Analytics - Operational and
Big Data Analytics
•  Operational Analytics
•  Needed when results are required in (near) real-time
•  Real-time monitoring, situational detection, correlation of events, e.g.
•  Dynamic advertising based on customer movement
•  Real-time equipment failure prediction
•  Operational analytics are required in gateways as well as the cloud
•  Gateways need to aggregate, filter, monitor for conditions
•  Analytics must run efficiently while new data is being loaded
•  Must be able to dynamically add and update analytics in the cloud
and gateways
•  Big Data analytics are required when you have the time to do
“Deeper/Wider” analysis
•  Latency between data arrival and results not an issue
•  Efficient ETL process from the operational repository is a must

27
TCP/IP
Bulk Loader
SQL Queries (from apps)
Informix Warehouse Accelerator
Compressed
DB partition
Query
Processor
Data Warehouse
Informix SQL
Query Router Results
Informix Warehouse Accelerator:
•  Connects to Informix via TCP/IP & DRDA
•  Analyzes, compresses, and loads to memory
•  Copy of (portion of) warehouse
•  Processes routed SQL query and
•  returns answer to Informix
Use Informix Warehouse Accelerator for
Mixed Operational/Analytic Workloads
Informix:
•  Routes SQL queries to accelerator
•  User need not change SQL or apps.
•  Can always run query in Informix
•  Too short an est. execution time

28
TimeSeries Service on BlueMix
Every IoT deployment will need to store time series data and possibly spatial data
•  Provides REST, JSON, ODBC/JDBC based services to:
•  Store, retrieve, and join time series, spatial, relational, and NoSQL data
•  Built-in time series analytics
•  Multi-tenancy support
•  Scales out automatically as needed
•  No Administration needed
IoT Foundation Service on BlueMix
BlueMix Cloud Services
•  Quickly and simply add new sensors
•  Interface for continuously loading sensor data
•  Adapters for MQTT and MessageSight

IBM Informix and Intel Deliver Leading Edge Solutions for IoT

▪  Informix
support
for
Intel’s

new
Quark
processor,
now

suppor*ng
Intel
family,
from

Quark
to
Xeon.

▪  Quark
port
in
the
free

Informix
Developer
Edi*on

▪  Informix
Developer
Edi*on

available
as
part
of
standard

Intel
Gateway
developer
SDK

Combining
IBM
and
Intel’s
strength
at
the
Edge,
Gateway
and
in
the
Cloud
provides

an
intelligent,
e2e
IoT
solu7on
for
rapid
7me
to
market.

Intel® Quark™ SoC / Intel®
Atom™ / Intel® Xeon™
Windriver
McAfee Security
Supports OSGI/TR69
Informix Database
30

Benchmark: Informix vs SQLite
31
Tests on Intel Quark Informix SQLite
Data loading – high-speed performance
(records per second)
950 / 1050
secs
(DK100 / DK50)1
700 secs
(Average)2
Storage space that is required for 1 day of data 275 MB 1200 MB
Aggregation query (seconds) 2 secs 4-25 secs
Moving average (seconds) 25 secs 2592003 secs
Missing interval search (seconds) 2 secs 14-30 secs
1.  The two figures for data loading with Informix reflect a slight difference in performance between the
DK100 and DK50. DK100 had more running components causing a drop in performance vs DK50
2.  Data loading with SQLite had significant variations in load performance as the database size
increased.
3.  The moving average result for SQLite is a projected figure that is based on a partial result after 10
minutes.

32
Metric Competitor Informix
Daily processing time
Maximum number of cores used
11 hours
62
5 hour 50 min
32
Maximum amount of memory used 192GB 192GB
Size of database per month of data 15TB 5TB
# Records processed each day 2.88 Billion 2.88 Billion
Billing determinants creation (1/21 of
the total meter population)
51,322 ~2 million reads
per second
TimeSeries Meter Data Management Benchmark
- 30 million smart meters sending data every 15 minutes
- 2.88 billion records inserted each day
- Workload: data Ingestion, data cleanup, and a daily billing cycle

Summary
•  IBM Informix - best fit for IoT architecture
•  IoT gateway
•  IoT cloud analytics
•  Supported on a wide array of platforms
•  Best in class embeddability
•  Native support for sensor data - TimeSeries & Spatial data
•  Native support for unstructured (JSON) data
•  Ease of application development - REST access
•  High availability and dynamic scaling
33

34
Kevin Brown – kbrown3@us.ibm.com

IBM Internet-of-Things architecture and capabilities

More Related Content

What's hot

Viewers also liked

Similar to IBM Internet-of-Things architecture and capabilities

More from IBM_Info_Management

Recently uploaded

IBM Internet-of-Things architecture and capabilities