Informix – The Ideal
Database for IoT
Pradeep Natarajan
@pradeepnatara
World of Watson 2016
Simply the Best
2
World of Watson 2016
Internet of Things Architecture – Analytics End-to-End
Deep Analytics Zone
Analytics Zone
Smart Gateways
Devices/Sensors
Uses
Gateway Direct to
Cloud
Insights from BigData
Message Broker
Flexible Hybrid
Data Management
3
World of Watson 2016
IoT applications have a common set of requirements
Opportunities for innovation
 Quickly and easily provision new sensors
 Create a real-time communication channel with the
sensor
 Capture data from the sensor and store it in a time
series database
 Provide Secure access to the collected data –
analytics at the Edge and Cloud, in real-time & on
historical data
 Trigger events based on specific data conditions
 Interact with the sensor from business/enterprise
applications and/or from mobile devices
 Monetize the service based on usage
4
World of Watson 2016
• Gateways can reduce the cost of the backend cloud
• Reduces cloud storage by filtering/aggregating/analyzing data locally
• Reduces cloud CPU requirements by precomputing values
• Reduces latency since actions can be taken immediately
• Intelligent gateways can detect and respond to local events as they happen rather
than waiting for transfer to the cloud
• Some users are not comfortable putting all their data in the cloud
• Gateways allow customers to capture and get value from their sensors without
sending data to the cloud
• Protocol Consolidation
• Cloud does not need to implement the 100’s of IoT protocols
Over time more and more of the processing will move from the cloud to gateway
devices
How Do Gateways Help IoT Solutions?
5
World of Watson 2016
What are the Requirements for a Gateway Database?
• The database management system must:
 Have a small install footprint, less than 100 MB
 Run with low memory requirements – less than 256 MB
 Use lossless compression techniques to minimize storage space
 Have built-in support for common types of IoT data like time series,
spatial, and JSON data
 Simple application development supporting both NoSQL and SQL
 Driverless, easy access to the data
 Require little or no administration
 Ideally should be able to network multiple gateways together to create a
single distributed database
6
The database must be powerful enough to ingest, process and
analyze data in real-time
World of Watson 2016
IBM Informix: The Ideal Database for Gateways
Simple to use
Hands-Free operation – No administration
Supports popular interfaces such as MQTT, REST, and Mongo as
well as ODBC/JDBC
Handles SQL and NoSQL data in the same database
Performance
One of a kind support for TimeSeries and Spatial data
Stream data continuously into the database
Run analytics as data arrives
Dynamically add and update analytics when needed
Storage is typically 1/3 the size compared to other vendors
Invisible
Agile
7
Informix is the only database management system perfectly suited
to run in Gateways
World of Watson 2016
Sensor Data is TimeSeries Data
• What is a Time Series?
 A logically connected set of records ordered by time
• What are the Key Strengths of Informix TimeSeries?
 Much less space required
• Typically about 1/3 the space required by other vendors
 Queries run orders of magnitude faster
• Unique optimized storage means codes paths are shorter and more data fits in
memory
 Purpose built streaming data loader for sensor data
• Automatically run analytic and/or aggregate functions on new data
 Can store structured (SQL) or unstructured (JSON) data for quick application
development
• REST/ODBC/JDBC/MongoDB/MQTT interfaces available
 100’s of functions predefined
• Programming APIs available to create your own analytics
8
World of Watson 2016
Traditional Table Approach
Informix TimeSeries Approach
Device_ID Time Sensor1 Sensor2 ColN
1 1-1-11 12:00 Value 1 Value 2 ……… Value N
2 1-1-11 12:00 Value 1 Value 2 ……… Value N
3 1-1-11 12:00 Value 1 Value 2 ……… Value N
… … … … ……… …
1 1-1-11 12:15 Value 1 Value 2 ……… Value N
2 1-1-11 12:15 Value 1 Value 2 ……… Value N
3 1-1-11 12:15 Value 1 Value 2 ……… Value N
… … … … ……… …
Device_ID Series
1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
2 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
3 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
4 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
…
Traditional Sensor data storage vs Informix TimeSeries Storage
9
World of Watson 2016
IoT Requirements for SpatioTemporal Data
• Many IoT applications have a spatial component
to them
Vehicles, cell phones, even pets… tracking is
common
• In these cases both location and time is
important
Show me the vehicles that have passed by location
X in the last hour
Where has my car been over the last few hours?
• Informix allows you to combine Time series and
Spatial data in the same query
10
World of Watson 2016
Ability to Recognize Patterns and Predict Events
an abnormal or a critical pattern
Similar patterns found
 Patterns are significant – heart rate, ECG, blood glucose, respiratory rate,
physical activity, …
• Now looking for anomalies and deviations, and symptomatic patterns
World of Watson 201612
Both Structured and Unstructured Data is Common in IoT
JSON
Collection
SQL Driver
NoSQL Driver
SQL Data
Join Data
• Informix can store SQL and JSON data in the same database
• Write programs using SQL drivers or Mongo/NoSQL drivers
• SQL data automatically transformed into JSON documents when needed
• NoSQL data automatically transformed into SQL when needed
Embedded
Device or
Database server
Horizontal
Scale-out
with
Shards
World of Watson 2016
Informix Data Access Options
13
MongoDB
Client
REST Client
SQLI Client
DRDA Client
Informix
DBMS
Informix NoSQL
Listener
Informix
• NoSQL ↔ SQL Translation
• MQTT, REST, MongoDB
Protocol Support
• SQLI, DRDA Protocol Support
• Relational, Collection, Time
Series, and Spatial Data
Support
Spatial
Time Series
JSON Collection
Relational Table
A REST client is any
program capable of
making a HTTP request
World of Watson 2016
Informix Data Access Options
14
MongoDB
Client
REST Client
SQLI Client
DRDA Client
Informix
DBMS
Informix NoSQL
Listener
Informix
• NoSQL ↔ SQL Translation
• MQTT, REST, MongoDB
Protocol Support
• SQLI, DRDA Protocol
Support
• Relational, Collection, Time
Series, and Spatial Data
Support
Spatial
Time Series
JSON Collection
Relational Table
You can use all the
client drivers that are
available for
MongoDB with the
NoSQL Listener
MQTT Client
World of Watson 2016
IBM IoT Smart Gateway Kit
• git clone https://github.com/ibm-iot/iot-gateway-kit.git
• The iot-gateway-kit will install the following:
▪ Node.js
▪ Node-red
▪ TimeSeries nodes
▪ Bluetooth node.js application sample
15
World of Watson 2016
IoT Developers - Get Started!
• Smart Gateway kit - https://ibm.biz/BdXr2W
• Code samples - https://ibm.biz/BdX4QV
• Github - https://github.com/IBM-IoT/
16
World of Watson 2016
Informix on Docker Hub
https://registry.hub.docker.com/u/ibmcom/informix-innovator-c/
• IBM Informix Innovator-C
• 12.10.FC7W1
https://registry.hub.docker.com/r/ibmcom/informix-rpi/
• IBM Informix Developer Edition for Raspberry Pi (32bit)
17
Docker Hub
$docker pull ibmcom/informix-innovator-c
World of Watson 2016 18
Informix for the Cloud and
Operational Zone
World of Watson 2016 19
What are the IoT Requirements for the Cloud?
• Requirements - similar to gateways (but for different reasons):
• Potentially 1000’s of servers means zero administration is a must
• Data volume adds up very quickly. Low storage overhead is required
• Data flows into the cloud continuously and must be processed in real-time
• Must be able to handle time series, spatial, and NoSQL data natively
• Additional requirements
• Must be able to scale-out
• Must be available as a service
The database must be able to ingest, process and analyze
data in real-time
World of Watson 2016 20
Why use Informix in the “Operational Zone”?
Simple to use
• Hands-Free operation
• Supports REST and Mongo APIs as well as ODBC/JDBC
• Stores SQL and JSON database in the same database
Highly Available
• Close to zero down time
• Partition or Hash your data across servers in the cloud
• Dynamically add/remove additional servers
Performance
• Continuous High Performance Analytics
• Specialized support for Time Series and Spatial data
Invisible
Agile
Resilient
World of Watson 2016 21
Shards: Scale-out your Database across Servers or Gateways
• Distribute data among servers by
range or hash partitioning
• Each shard can have an associated
secondary server for high availability
• Run queries across all shards or a
subset of the shards
• Only shards that could qualify are
searched
• Shards are searched in parallel
• Ignores shards that are offline
Shards in a
Cloud
World of Watson 2016 22
TCP/IP
Bulk Loader
SQL Queries (from apps)
Informix Warehouse Accelerator
Compressed
DB partition
Query
Processor
Data Warehouse
Informix
SQL
Query Router
Results
Informix Warehouse Accelerator:
• Connects to Informix via TCP/IP & DRDA
• Analyzes, compresses, and loads to memory
• Copy of (portion of) warehouse
• Processes routed SQL query and
• returns answer to Informix
Use Informix Warehouse Accelerator for
Mixed Operational/Analytic Workloads
Informix:
• Routes SQL queries to accelerator
• User need not change SQL or apps.
• Can always run query in Informix
• Too short an est. execution time
World of Watson 2016 23
Every IoT deployment will need to store time series data and
possibly spatial data and/or NoSQL data
Bluemix Cloud Service
Informix on Cloud – Hosted Service
• Quickly and simply deploy Informix
• Pre installed and pre configured instance
• Multiple size options (S, M, L, XL)
World of Watson 2016 24
Industry Use Cases
World of Watson 2016
Changing Business Model – Health care & Assisted Living
Informix Historian
Operational Analytics
Notification to Assisted
Living Central Monitoring
Station
Change patients medication, closer
monitoring, prevent stroke
1
2
3
Patient/Care giver
Hundreds of patients
Thousands of
devices
Locally Act Upon
Insights
Data
Consolidation
Gateway
Sensor Data
Input
Display Alerts and
Recommended Actions
4
5
Collection and analysis of data
for all devices across assisted
living facilities
Assisted Living Corporation
changes food sodium usage based
on trend of high blood pressure
Filter critical and life-saving
data
Blood pressure threshold
exceeded
• Embedded at
device/gateway
• Local decision making at
Facility
• Leverage all data:
NoSQL/SQL & Timeseries
data
 Automatic sensors to
monitor well being
 Pendants, shower &
bath buttons
 Activity sensors – rising
in the morning, taking
meds, using the fridge
 Bed & Chair sensors for
inactivity monitoring
 Outside alarms to alert
neighbors
World of Watson 2016 26
And Many More…
See for yourself and talk to us @
Analytics Demo Room – 530
DMT09, DMT13
World of Watson 2016
Summary
• IBM Informix - best fit for IoT architecture
• IoT gateway
• IoT cloud analytics
• Supported on a wide array of platforms
• Best in class embeddability
• Native support for sensor data - TimeSeries & Spatial data
• Native support for unstructured (JSON) data
• Ease of application development - REST access
• Support to receive IoT data via MQTT protocol
• High availability and dynamic scaling
27
Thank You

Informix - The Ideal Database for IoT

  • 1.
    Informix – TheIdeal Database for IoT Pradeep Natarajan @pradeepnatara
  • 2.
    World of Watson2016 Simply the Best 2
  • 3.
    World of Watson2016 Internet of Things Architecture – Analytics End-to-End Deep Analytics Zone Analytics Zone Smart Gateways Devices/Sensors Uses Gateway Direct to Cloud Insights from BigData Message Broker Flexible Hybrid Data Management 3
  • 4.
    World of Watson2016 IoT applications have a common set of requirements Opportunities for innovation  Quickly and easily provision new sensors  Create a real-time communication channel with the sensor  Capture data from the sensor and store it in a time series database  Provide Secure access to the collected data – analytics at the Edge and Cloud, in real-time & on historical data  Trigger events based on specific data conditions  Interact with the sensor from business/enterprise applications and/or from mobile devices  Monetize the service based on usage 4
  • 5.
    World of Watson2016 • Gateways can reduce the cost of the backend cloud • Reduces cloud storage by filtering/aggregating/analyzing data locally • Reduces cloud CPU requirements by precomputing values • Reduces latency since actions can be taken immediately • Intelligent gateways can detect and respond to local events as they happen rather than waiting for transfer to the cloud • Some users are not comfortable putting all their data in the cloud • Gateways allow customers to capture and get value from their sensors without sending data to the cloud • Protocol Consolidation • Cloud does not need to implement the 100’s of IoT protocols Over time more and more of the processing will move from the cloud to gateway devices How Do Gateways Help IoT Solutions? 5
  • 6.
    World of Watson2016 What are the Requirements for a Gateway Database? • The database management system must:  Have a small install footprint, less than 100 MB  Run with low memory requirements – less than 256 MB  Use lossless compression techniques to minimize storage space  Have built-in support for common types of IoT data like time series, spatial, and JSON data  Simple application development supporting both NoSQL and SQL  Driverless, easy access to the data  Require little or no administration  Ideally should be able to network multiple gateways together to create a single distributed database 6 The database must be powerful enough to ingest, process and analyze data in real-time
  • 7.
    World of Watson2016 IBM Informix: The Ideal Database for Gateways Simple to use Hands-Free operation – No administration Supports popular interfaces such as MQTT, REST, and Mongo as well as ODBC/JDBC Handles SQL and NoSQL data in the same database Performance One of a kind support for TimeSeries and Spatial data Stream data continuously into the database Run analytics as data arrives Dynamically add and update analytics when needed Storage is typically 1/3 the size compared to other vendors Invisible Agile 7 Informix is the only database management system perfectly suited to run in Gateways
  • 8.
    World of Watson2016 Sensor Data is TimeSeries Data • What is a Time Series?  A logically connected set of records ordered by time • What are the Key Strengths of Informix TimeSeries?  Much less space required • Typically about 1/3 the space required by other vendors  Queries run orders of magnitude faster • Unique optimized storage means codes paths are shorter and more data fits in memory  Purpose built streaming data loader for sensor data • Automatically run analytic and/or aggregate functions on new data  Can store structured (SQL) or unstructured (JSON) data for quick application development • REST/ODBC/JDBC/MongoDB/MQTT interfaces available  100’s of functions predefined • Programming APIs available to create your own analytics 8
  • 9.
    World of Watson2016 Traditional Table Approach Informix TimeSeries Approach Device_ID Time Sensor1 Sensor2 ColN 1 1-1-11 12:00 Value 1 Value 2 ……… Value N 2 1-1-11 12:00 Value 1 Value 2 ……… Value N 3 1-1-11 12:00 Value 1 Value 2 ……… Value N … … … … ……… … 1 1-1-11 12:15 Value 1 Value 2 ……… Value N 2 1-1-11 12:15 Value 1 Value 2 ……… Value N 3 1-1-11 12:15 Value 1 Value 2 ……… Value N … … … … ……… … Device_ID Series 1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …] 2 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …] 3 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …] 4 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …] … Traditional Sensor data storage vs Informix TimeSeries Storage 9
  • 10.
    World of Watson2016 IoT Requirements for SpatioTemporal Data • Many IoT applications have a spatial component to them Vehicles, cell phones, even pets… tracking is common • In these cases both location and time is important Show me the vehicles that have passed by location X in the last hour Where has my car been over the last few hours? • Informix allows you to combine Time series and Spatial data in the same query 10
  • 11.
    World of Watson2016 Ability to Recognize Patterns and Predict Events an abnormal or a critical pattern Similar patterns found  Patterns are significant – heart rate, ECG, blood glucose, respiratory rate, physical activity, … • Now looking for anomalies and deviations, and symptomatic patterns
  • 12.
    World of Watson201612 Both Structured and Unstructured Data is Common in IoT JSON Collection SQL Driver NoSQL Driver SQL Data Join Data • Informix can store SQL and JSON data in the same database • Write programs using SQL drivers or Mongo/NoSQL drivers • SQL data automatically transformed into JSON documents when needed • NoSQL data automatically transformed into SQL when needed Embedded Device or Database server Horizontal Scale-out with Shards
  • 13.
    World of Watson2016 Informix Data Access Options 13 MongoDB Client REST Client SQLI Client DRDA Client Informix DBMS Informix NoSQL Listener Informix • NoSQL ↔ SQL Translation • MQTT, REST, MongoDB Protocol Support • SQLI, DRDA Protocol Support • Relational, Collection, Time Series, and Spatial Data Support Spatial Time Series JSON Collection Relational Table A REST client is any program capable of making a HTTP request
  • 14.
    World of Watson2016 Informix Data Access Options 14 MongoDB Client REST Client SQLI Client DRDA Client Informix DBMS Informix NoSQL Listener Informix • NoSQL ↔ SQL Translation • MQTT, REST, MongoDB Protocol Support • SQLI, DRDA Protocol Support • Relational, Collection, Time Series, and Spatial Data Support Spatial Time Series JSON Collection Relational Table You can use all the client drivers that are available for MongoDB with the NoSQL Listener MQTT Client
  • 15.
    World of Watson2016 IBM IoT Smart Gateway Kit • git clone https://github.com/ibm-iot/iot-gateway-kit.git • The iot-gateway-kit will install the following: ▪ Node.js ▪ Node-red ▪ TimeSeries nodes ▪ Bluetooth node.js application sample 15
  • 16.
    World of Watson2016 IoT Developers - Get Started! • Smart Gateway kit - https://ibm.biz/BdXr2W • Code samples - https://ibm.biz/BdX4QV • Github - https://github.com/IBM-IoT/ 16
  • 17.
    World of Watson2016 Informix on Docker Hub https://registry.hub.docker.com/u/ibmcom/informix-innovator-c/ • IBM Informix Innovator-C • 12.10.FC7W1 https://registry.hub.docker.com/r/ibmcom/informix-rpi/ • IBM Informix Developer Edition for Raspberry Pi (32bit) 17 Docker Hub $docker pull ibmcom/informix-innovator-c
  • 18.
    World of Watson2016 18 Informix for the Cloud and Operational Zone
  • 19.
    World of Watson2016 19 What are the IoT Requirements for the Cloud? • Requirements - similar to gateways (but for different reasons): • Potentially 1000’s of servers means zero administration is a must • Data volume adds up very quickly. Low storage overhead is required • Data flows into the cloud continuously and must be processed in real-time • Must be able to handle time series, spatial, and NoSQL data natively • Additional requirements • Must be able to scale-out • Must be available as a service The database must be able to ingest, process and analyze data in real-time
  • 20.
    World of Watson2016 20 Why use Informix in the “Operational Zone”? Simple to use • Hands-Free operation • Supports REST and Mongo APIs as well as ODBC/JDBC • Stores SQL and JSON database in the same database Highly Available • Close to zero down time • Partition or Hash your data across servers in the cloud • Dynamically add/remove additional servers Performance • Continuous High Performance Analytics • Specialized support for Time Series and Spatial data Invisible Agile Resilient
  • 21.
    World of Watson2016 21 Shards: Scale-out your Database across Servers or Gateways • Distribute data among servers by range or hash partitioning • Each shard can have an associated secondary server for high availability • Run queries across all shards or a subset of the shards • Only shards that could qualify are searched • Shards are searched in parallel • Ignores shards that are offline Shards in a Cloud
  • 22.
    World of Watson2016 22 TCP/IP Bulk Loader SQL Queries (from apps) Informix Warehouse Accelerator Compressed DB partition Query Processor Data Warehouse Informix SQL Query Router Results Informix Warehouse Accelerator: • Connects to Informix via TCP/IP & DRDA • Analyzes, compresses, and loads to memory • Copy of (portion of) warehouse • Processes routed SQL query and • returns answer to Informix Use Informix Warehouse Accelerator for Mixed Operational/Analytic Workloads Informix: • Routes SQL queries to accelerator • User need not change SQL or apps. • Can always run query in Informix • Too short an est. execution time
  • 23.
    World of Watson2016 23 Every IoT deployment will need to store time series data and possibly spatial data and/or NoSQL data Bluemix Cloud Service Informix on Cloud – Hosted Service • Quickly and simply deploy Informix • Pre installed and pre configured instance • Multiple size options (S, M, L, XL)
  • 24.
    World of Watson2016 24 Industry Use Cases
  • 25.
    World of Watson2016 Changing Business Model – Health care & Assisted Living Informix Historian Operational Analytics Notification to Assisted Living Central Monitoring Station Change patients medication, closer monitoring, prevent stroke 1 2 3 Patient/Care giver Hundreds of patients Thousands of devices Locally Act Upon Insights Data Consolidation Gateway Sensor Data Input Display Alerts and Recommended Actions 4 5 Collection and analysis of data for all devices across assisted living facilities Assisted Living Corporation changes food sodium usage based on trend of high blood pressure Filter critical and life-saving data Blood pressure threshold exceeded • Embedded at device/gateway • Local decision making at Facility • Leverage all data: NoSQL/SQL & Timeseries data  Automatic sensors to monitor well being  Pendants, shower & bath buttons  Activity sensors – rising in the morning, taking meds, using the fridge  Bed & Chair sensors for inactivity monitoring  Outside alarms to alert neighbors
  • 26.
    World of Watson2016 26 And Many More… See for yourself and talk to us @ Analytics Demo Room – 530 DMT09, DMT13
  • 27.
    World of Watson2016 Summary • IBM Informix - best fit for IoT architecture • IoT gateway • IoT cloud analytics • Supported on a wide array of platforms • Best in class embeddability • Native support for sensor data - TimeSeries & Spatial data • Native support for unstructured (JSON) data • Ease of application development - REST access • Support to receive IoT data via MQTT protocol • High availability and dynamic scaling 27
  • 28.