Download as PDF, PPTX
![Traditional Table Approach
Informix TimeSeries Approach
Meter_ID Time KWH Voltage ColN
1 1-1-11 12:00 Value 1 Value 2 ……… Value N
2 1-1-11 12:00 Value 1 Value 2 ……… Value N
3 1-1-11 12:00 Value 1 Value 2 ……… Value N
… … … … ……… …
1 1-1-11 12:15 Value 1 Value 2 ……… Value N
2 1-1-11 12:15 Value 1 Value 2 ……… Value N
3 1-1-11 12:15 Value 1 Value 2 ……… Value N
… … … … ……… …
Meter_ID Series
1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
2 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
3 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
4 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
…
Traditional Sensor data storage vs Informix TimeSeries
Storage
9](https://image.slidesharecdn.com/ibmiotarchitectureandcapabilities-150514093320-lva1-app6892/85/Ibm_IoT_Architecture_and_Capabilities-9-320.jpg)
![IBM Analytics © 2015 IBM Corporation33
Legal Disclaimer
• © IBM Corporation 2015. All Rights Reserved.
• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in
this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject
to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained
in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the
applicable license agreement governing the use of IBM software.
• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or
capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to
future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you
will result in any specific sales, revenue growth or other results.
• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will
experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage
configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and
performance characteristics may vary by customer.
• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus®
Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to
http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentation. All product
names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the
IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the
United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.
• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.
• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:
Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.
• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:
Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and
other countries.
• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:
UNIX is a registered trademark of The Open Group in the United States and other countries.
• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.
• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta
Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes
only.](https://image.slidesharecdn.com/ibmiotarchitectureandcapabilities-150514093320-lva1-app6892/85/Ibm_IoT_Architecture_and_Capabilities-33-320.jpg)
More Related Content
Similar to Ibm_IoT_Architecture_and_Capabilities
Ibm_IoT_Architecture_and_Capabilities
- 1. © 2015 IBMCorporation IBM Analytics IBM Internet of Things - Architecture and Capabilities Pradeep Natarajan - Manager, IoT Solutions and Services April 30th, 2015
- 2. Agenda • IoT ArchitecturalComponents Overview • Informix for the Intelligent Gateway • Informix for the Cloud • IoT Partners and Platforms 2
- 3. 3 Informix for theIntelligent Gateway
- 4. 4 Internet of ThingsTopography Streams Deep Analytics Zone Device/Sensors Smart Gateways Sensor Analytics Zone 4
- 5. • Gateways canreduce the cost of the backend cloud • Reduces cloud storage by filtering/aggregating/analyzing data locally • Reduces cloud CPU requirements by precomputing values • Reduces latency since actions can be taken immediately • Intelligent gateways can detect and respond to local events as they happen rather than waiting for transfer to the cloud • Some users are not comfortable putting all their data in the cloud • Gateways allow customers to capture and get value from their sensors without sending data to the cloud • Protocol Consolidation • Cloud does not need to implement the 100’s of IoT protocols Over time more and more of the processing will move from the cloud to gateway devices How Do Gateways Help IoT Solutions? 5
- 6. What are theRequirements for a Gateway Database? • The database management system must: ▪ Have a small install footprint, less than 100 MB ▪ Run with low memory requirements – less than 256 MB ▪ Use lossless compression or other techniques to minimize storage space ▪ Have built-in support for common types of IoT data like time series and spatial/GIS data ▪ Simple application development supporting both NoSQL and SQL ▪ Driverless, easy access to the data ▪ Require absolutely no administration ▪ Ideally should be able to network multiple gateways together to create a single distributed database 6 The database must be powerful enough to ingest, process and analyze data in real-time
- 7. IBM Informix: TheIdeal Database for Gateways Simple to use ▪ Hands-Free operation – No administration ▪ Supports popular interfaces such as REST and Mongo as well as ODBC/JDBC ▪ Handles SQL and JSON data in the same database Performance ▪ One of a kind support for Time Series and Spatial data ▪ Stream data continuously into the database ▪ Run analytics as data arrives ▪ Dynamically add and update analytics when needed ▪ Storage is typically 1/3 the size compared to other vendors Invisible Agile 7 Informix is the only database management system perfectly suited to run in Gateways
- 8. Sensor Data isTime Series Data • What is a Time Series? ▪ A logically connected set of records ordered by time • What are the Key Strengths of Informix TimeSeries? ▪ Much less space required • Typically about 1/3 the space required by other vendors ▪ Queries run orders of magnitude faster • Unique optimized storage means codes paths are shorter and more data fits in memory ▪ Purpose built streaming data loader for sensor data • Automatically run analytic and/or aggregate functions on new data ▪ Can store structured (SQL) or unstructured (JSON) data for quick application development • REST/ODBC/JDBC/JSON interfaces available ▪ 100’s of functions predefined • Programming APIs available to create your own analytics 8
- 9. Traditional Table Approach InformixTimeSeries Approach Meter_ID Time KWH Voltage ColN 1 1-1-11 12:00 Value 1 Value 2 ……… Value N 2 1-1-11 12:00 Value 1 Value 2 ……… Value N 3 1-1-11 12:00 Value 1 Value 2 ……… Value N … … … … ……… … 1 1-1-11 12:15 Value 1 Value 2 ……… Value N 2 1-1-11 12:15 Value 1 Value 2 ……… Value N 3 1-1-11 12:15 Value 1 Value 2 ……… Value N … … … … ……… … Meter_ID Series 1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …] 2 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …] 3 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …] 4 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …] … Traditional Sensor data storage vs Informix TimeSeries Storage 9
- 10. IoT Requirements forSpatioTemporal Data • Many IoT applications have a spatial component to them ▪ Vehicles, cell phones, even pets… tracking is common • In these cases both location and time is important ▪ Show me the vehicles that have passed by location X in the last hour ▪ Where has my car been over the last few hours? • Informix allows you to combine Time series and Spatial data in the same query 10
- 11. Informix Handles Structuredand Unstructured Data • There is no common format for sensor data ▪ Vendor 1 sends just temperature, Vendor 2 sends temperature plus pressure • Informix is the only product to handle structured and unstructured data simultaneously in the same database ▪ Store meta data about your devices (e.g. manufacturer) as JSON ▪ Store device history data (e.g. temperature) as SQL time series • Informix translates between structured and unstructured on the fly ▪ Allows SQL apps to access the unstructured data and Mongo apps to access the structured data ▪ Structured and unstructured data can be combined in a single query • “Give me all the temperature data for sensors made by manufacturer X between Jan 1 and Feb 1” 11
- 12. Data Access Options 12 MongoDB Client RESTClient SQLI Client DRDA Client Informix DBMS Informix NoSQL Listener Informix • NoSQL ↔ SQL Translation • REST, MongoDB Protocol Support • SQLI, DRDA Protocol Support • Relational, Collection, Time Series, and Spatial Data Support Spatial Time Series JSON Collection Relational Table A REST client is any program capable of making a HTTP request
- 13. Data Access Options 13 MongoDB Client RESTClient SQLI Client DRDA Client Informix DBMS Informix NoSQL Listener Informix • NoSQL ↔ SQL Translation • REST, MongoDB Protocol Support • SQLI, DRDA Protocol Support • Relational, Collection, Time Series, and Spatial Data Support Spatial Time Series JSON Collection Relational Table You can use the MongoDB PHP drivers are available for MongoDB with the NoSQL Listener
- 14. What is Node-Red •nodered.org • A visual tool for wiring the Internet of Things • Node.js application. Uses building blocks of reusable code to control flow of data 14 • “Flows” are created to process data. • Example: Take data in, transpose, process and store in a database and/or publish to a message broker • Collaboration among node developers
- 15. Node-Red Benefits • RapidDevelopment • Simple use with JSON • Simple REST • Simple MQTT • Contributor Nodes – simple to use other services 15
- 16.
- 17. IoT Developers -Get Started! • Gateway starter - https://ibm.biz/BdXr2W • Code samples - https://ibm.biz/BdX4QV • Github - https://github.com/IBM-IoT/ 17
- 18. 18 Informix for theCloud and Operational Zone
- 19. 19 What are theIoT Requirements for the Cloud? • Requirements - similar to gateways (but for different reasons): • Potentially 1000’s of servers means zero administration is a must • Data volume adds up very quickly so low storage overhead is required • Data flows into the cloud continuously and must be processed in real- time • Must be able to handle time series, spatial, and NoSQL data natively • Additional requirements • Must be able to scale-out • Must be available as a service The database must be able to ingest, process and analyze data in real-time
- 20. 20 Why use Informixin the “Operational Zone”? Simple to use • Hands-Free operation • Supports REST and Mongo APIs as well as ODBC/JDBC • Stores SQL and JSON database in the same database Highly Available • Close to zero down time • Partition or Hash your data across servers in the cloud • Dynamically add/remove additional servers Performance • Continuous High Performance Analytics • Specialized support for Time Series and Spatial data Invisible Agile Resilient
- 21. 21 Shards: Scale-out yourDatabase across Servers or Gateways • Distribute data among servers by range or hash partitioning • Each shard can have an associated secondary server for high availability • Run queries across all shards or a subset of the shards • Only shards that could qualify are searched • Shards are searched in parallel • Ignores shards that are offline Shards in a
- 22. 22 IoT Analytics -Operational and Big Data Analytics • Operational Analytics • Needed when results are required in (near) real-time • Real-time monitoring, situational detection, correlation of events, e.g. • Dynamic advertising based on customer movement • Real-time equipment failure prediction • Operational analytics are required in gateways as well as the cloud • Gateways need to aggregate, filter, monitor for conditions • Analytics must run efficiently while new data is being loaded • Must be able to dynamically add and update analytics in the cloud and gateways • Big Data analytics are required when you have the time to do “Deeper/Wider” analysis • Latency between data arrival and results not an issue • Efficient ETL process from the operational repository is a must
- 23. 23 TCP/IP Bulk Loader SQL Queries(from apps) Informix Warehouse Accelerator Compressed DB partition Query Processor Data Warehouse Informix SQL Query Router Results Informix Warehouse Accelerator: • Connects to Informix via TCP/IP & DRDA • Analyzes, compresses, and loads to memory • Copy of (portion of) warehouse • Processes routed SQL query and • returns answer to Informix Informix Warehouse Accelerator (IWA) In-Memory Analytics Informix: • Routes SQL queries to accelerator • User need not change SQL or apps. • Can always run query in Informix • Too short an est. execution time
- 24. 24 TimeSeries Service onBlueMix Every IoT deployment will need to store time series data and possibly spatial data • Provides REST, JSON, ODBC/JDBC based services to: • Store, retrieve, and join time series, spatial, relational, and NoSQL data • Built-in time series analytics • Multi-tenancy support • Scales out automatically as needed • No Administration needed • Quickly and simply add new sensors • Interface for continuously loading sensor data • Adapters for MQTT and MessageSight IoT Foundation Service on BlueMix BlueMix Services
- 25. Docker Engine • Opensource project • Supported on every major Linux distro (MS Windows in 2015) • Client-server architecture with daemon deployed on physical or virtual host • Uses Linux kernel cgroups and namespaces for process resource management and isolation • Uses copy-on-write filesystem for git-like image change management 25 A portable, lightweight application runtime and packaging tool built on top of kernel container primitives Build Ship Run Docker Engine
- 26. Containers 26 App A’ Host OS Server Bins/Libs App A App B App B’ App B’ App B’ Container What isa Container • An isolated user space within a running Linux OS • Shared kernel across containers • Direct device access • All packages and data in an isolated run-time, saved as a filesystem. • Resource management implemented with cgroups • Resource isolation through namespaces Run Bins/Libs
- 27. Docker Hub • Enablesharing and collaboration of Docker Images • Private and public repositories of images • Certified base images by ISVs 27 Ship Docker Hub $ docker push $ docker pull
- 28. Informix on DockerHub https://registry.hub.docker.com/u/ibmcom/informix-innovator-c/ • IBM Informix Innovator-C • 12.10.FC4 28 Docker Hub $docker pull ibmcom/informix-innovator-c
- 29.
- 30. IBM Informix andIntel Deliver Leading Edge Solutions for IoT ▪ Informix support for Intel’s new Quark processor, now supporting Intel family, from Quark to Xeon. ▪ Quark port in the free Informix Developer Edition ▪ Informix Developer Edition available as part of standard Intel Gateway developer SDK Combining IBM and Intel’s strength at the Edge, Gateway and in the Cloud provides an intelligent, e2e IoT solution for rapid time to market. Intel® Quark™ SoC / Intel® Atom™ / Intel® Xeon™ Windriver McAfee Security Supports OSGI/TR69 Informix Database 30
- 31. Summary • IBM Informix- best fit for IoT architecture • IoT gateway • IoT cloud analytics • Supported on a wide array of platforms • Best in class embeddability • Native support for sensor data - TimeSeries & Spatial data • Native support for unstructured (JSON) data • Ease of application development - REST access • High availability and dynamic scaling 31
- 32. 32 Pradeep Natarajan –pnatara@us.ibm.com
- 33. IBM Analytics ©2015 IBM Corporation33 Legal Disclaimer • © IBM Corporation 2015. All Rights Reserved. • The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. • References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. • If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete: Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. • If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete: All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. • Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both. • If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete: Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. • If you reference Java™ in the text, please mark the first use and include the following; otherwise delete: Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. • If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete: Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. • If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete: Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. • If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete: UNIX is a registered trademark of The Open Group in the United States and other countries. • If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete: Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. • If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only.