Informix SQL & NoSQL: Putting it all together

2,293 views

Published on

Building IoT applications and handling IoT data with SQL, NoSQL, JSON, Spatial, timeseries, etc. Details of Informix SQL and NoSQL implementation and some things planned for 12.10.FC4.

Published in: Software, Technology
1 Comment
2 Likes
Statistics
Notes
  • thanks for sharing :-)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
2,293
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
101
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Informix SQL & NoSQL: Putting it all together

  1. 1. IBM INFORMIX putting it all together John Miller, Shawn Moe, Keshav Murthy IBM Informix Development
  2. 2. 2 Explosion of mobile devices – gaming and social apps Advertising: serving ads and real-time bidding Social networking, online communities E-commerce, social commerce Machine data and real-time operational decisions Smart Devices Internet of Things Internet of Things
  3. 3. 3 Explosion of mobile devices – gaming and social apps Advertising: serving ads and real-time bidding Social networking, online communities E-commerce, social commerce Machine data and real-time operational decisions Smart Devices Internet of Data, really Internet of Things SQLSQL SQL, {JSON}, SpatialSQL, {JSON}, Spatial {JSON}, TimeSeries {JSON}, TimeSeries SQL, {JSON}SQL, {JSON} Simple, {JSON}, Timeseries Simple, {JSON}, Timeseries SQL, {JSON}SQL, {JSON}
  4. 4. 4 IoT Applications – IBM Reference Architecture Gateway Operational Zone Warehouse/Mart Analytics Services and Contents Shared Operational Information Rule Engine ETL Real-Time Data Store Hadoop Video Analytics Big Data Explorer Analytic Tools Connected Device Analyzed Data MapReduce HDFS/GPFS Device Management : Predictive Maintenance Traffic Optimization Driving Behavior Incident Analysis Infotainment Service Raw Data Summarized Data Notification Analytic Report B2C/B2B Portal Admin Console Operator Console LocalIntelligence NetworkSupport Stream Processing ETL RDB DataMart SOE Data Video Management Asset Data Management Master Data Management Reference Data Hub Video Data .. Environment Data, etc. Other Data Local Database
  5. 5. © 2014 IBM Corporation5 IBM Cloud: Think it. Build it. Tap into it. IoT Solutions, an architecture. Collection of data for all sensors Data from other kinds of sensors Consumer / Business Sensors in the home Informix TimeSeries Service NoSQL, Relational, Timeseries & Spatial storage & analytics Informix Warehouse Accelerator SPSS/Cognos MessageSight / MQTT SoftLayer / BlueMix BigInsights Gateways for data consolidation Infosphere Streams (no gateway) = IBM products = IBM Informix Relational Database In-memory analytics Predictive analytics and dashboard Cloud infrastructure Hadoop Publish / Subscribe Real-time analytics
  6. 6. • Individual Car Recognition in the parking zone •Composite sensors to transmit license image •Cloud service to recognize the car plate number •Location tracking • Individual Car Recognition in the parking zone •Composite sensors to transmit license image •Cloud service to recognize the car plate number •Location tracking
  7. 7. IBM Informix developer edition. Download Now: http://www-03.ibm.com/software/products/en/infodeveedit IBM Informix developer edition. Download Now: http://www-03.ibm.com/software/products/en/infodeveedit
  8. 8. Handling Big Data
  9. 9. IBM Bluemix
  10. 10. 10 SQL {NoSQL:JSON} Define Schema first Write the program first Relational Key-value, Document, column family, graph and text Changing schema is hard Assumes dynamic schema Scale-up Scale-out ACID consistency BASE consistency Transactions No Transactions SQL Proprietary API; Sometimes has the “spirit” of SQL
  11. 11. 11 SQL Timeseries Define Schema first Create Timeseries Row Type Relational Timeseries Optimized with projection to relational Changing schema is hard Changing schema is hard; Change is easy with Timeseries({JSON}) Scale-up Scale-up & Scale-out ACID consistency ACID consistency SQL SQL extensions.
  12. 12. Data Management: devices to Cloud Enterprise replication + Flexible Grid App Server JDBC App Server Mongo Driver Listener Informix/1 Primary Informix/1 SDS/HDR Informix/1 RSS Informix/2 Primary Informix/2 SDS/HDR Informix/2 RSS Informix/3 Primary Informix/3 SDS/HDR Informix/3 RSS Informix/4 Primary Informix/4 SDS/HDR Informix/4 RSS Informix/5 Primary Informix/5 SDS/HDR Informix/5 RSS Informix/6 Primary Informix/6 SDS/HDR Informix/6 RSS Mongo APIMongo API Node.JS Express.JSExpress.JS AngularJSAngularJS REST APIsREST APIs NoSQL SQL Devices Devices Gateway Gateway CloudCloud Informix warehouse Accelerator
  13. 13. Informix: All Together Now! 13 SQL Tables JSON Collections TimeSeries MQ Series SQL APIs JDBC, ODBC SQL APIs JDBC, ODBC Informix IWA – BLU ACCELERATION GENBSON: SQL to {BSON} MongoDB Drivers MongoDB Drivers TEXT SEARCH SPATIAL TIME SERIES {BSON}
  14. 14. Data Model Should NOT restrict Data Access
  15. 15. Universal Schema Seamless Access
  16. 16. SQL APISQL API MongoDB API (NoSQL) MongoDB API (NoSQL) Relational TableRelational Table JSON CollectionsJSON Collections Standard ODBC, JDBC, .NET, OData, etc. Language SQL. Standard ODBC, JDBC, .NET, OData, etc. Language SQL. Mongo APIs for Java, Javascript, C++, C#,... Mongo APIs for Java, Javascript, C++, C#,... Direct SQL Access. Dynamic Views Row types Mongo APIs for Java, Javascript, C++, C#,... Mongo APIs for Java, Javascript, C++, C#,... Hybrid Access: SQL & JSON
  17. 17. SQL APISQL API MongoDB API (NoSQL) MongoDB API (NoSQL) Relational TableRelational Table JSON CollectionsJSON Collections Standard ODBC, JDBC, .NET, OData, etc. Language SQL. Standard ODBC, JDBC, .NET, OData, etc. Language SQL. Mongo APIs for Java, Javascript, C++, C#,... Mongo APIs for Java, Javascript, C++, C#,... Direct SQL Access. Dynamic Views Row types Direct SQL Access. Dynamic Views Row types Mongo APIs for Java, Javascript, C++, C#,... Mongo APIs for Java, Javascript, C++, C#,... JSON CollectionsJSON CollectionsJSON CollectionsJSON Collections Standard SQL/ext JDBC/ODBC JSON Support Standard SQL/ext JDBC/ODBC JSON Support Virtual Table JSON support Virtual Table JSON support TimeseriesTimeseriesJSON CollectionsJSON Collections TimeseriesTimeseriesRelational TableRelational Table JSON CollectionsJSON Collections TimeseriesTimeseries Hybrid Access: SQL, JSON & Timeseries Standard SQL/ext JDBC/ODBC JSON Support Standard SQL/ext JDBC/ODBC JSON Support TimeseriesTimeseries Standard SQL/ext JDBC/ODBC JSON Support Standard SQL/ext JDBC/ODBC JSON Support TimeseriesTimeseries Virtual Table JSON support Virtual Table JSON support Standard SQL/ext JDBC/ODBC JSON Support Standard SQL/ext JDBC/ODBC JSON Support Virtual Table JSON support Virtual Table JSON support TimeseriesTimeseries Standard SQL/ext JDBC/ODBC JSON Support Standard SQL/ext JDBC/ODBC JSON Support Virtual Table JSON support Virtual Table JSON support TimeseriesTimeseries Standard SQL/ext JDBC/ODBC JSON Support Standard SQL/ext JDBC/ODBC JSON Support Virtual Table JSON support Virtual Table JSON support TimeseriesTimeseries Standard SQL/ext JDBC/ODBC JSON Support Virtual Table JSON support Virtual Table JSON support
  18. 18. SQL APISQL API Mongo API (NoSQL) Mongo API (NoSQL) Relational TableRelational Table JSON CollectionsJSON Collections Standard ODBC, JDBC, .NET, OData, etc. Language SQL. Standard ODBC, JDBC, .NET, OData, etc. Language SQL. Mongo APIs for Java, Javascript, C++, C#,... Mongo APIs for Java, Javascript, C++, C#,... Direct SQL Access. Dynamic Views Row types Direct SQL Access. Dynamic Views Row types Mongo APIs for Java, Javascript, C++, C#,... Mongo APIs for Java, Javascript, C++, C#,... JSON CollectionsJSON CollectionsJSON CollectionsJSON Collections Standard SQL/ext JDBC/ODBC JSON Support Standard SQL/ext JDBC/ODBC JSON Support Virtual Table JSON support Virtual Table JSON support TimeseriesTimeseriesJSON CollectionsJSON Collections TimeseriesTimeseriesRelational TableRelational Table JSONJSON TimeseriesTimeseries Spatial Text Spatial Text Standard SQL JDBC/ODBC JSON Support JSON SupportJSON Support Hybrid Access: SQL, JSON, Timeseries & Spatial
  19. 19. Mapping A JSON To A SQL Table CREATE TABLE photos(data BSON); SELECT data.GPSLatitude::lvarchar as GPSLatitude, data.GPSLongitude::lvarchar as GPSLongitude, data.Make::varchar(64) as Make, data.oateTimeoriginal::datetime year to day as dt, data.exposuretime::int data.pixelxdimension::float, data.pixelydimension::float FROM photos;
  20. 20. 20 Timeseries on JSON CREATE ROW TYPE info( stime datetime year to fraction(5), jdata bson); CREATE TABLE iotdata(id int primary key, tsdata timeseries(info) ); INSERT INTO iotdata VALUES(472,'origin(2014-04-23 00:00:00.00000), …, regular,[({“temp":78, “wind":7.2, “loc":“Miami-1 "})]'); INSERT INTO iotdata values(384,'origin(2014-04-21 00:00:00.00000), …, regular,[({“sleep": 380, “steps":7423, “name":"Joe "})]'); SELECT GetFirstElem(tsdata,0)::row(timestamp datetime year to fraction(5), jdata json) FRONM tj; (expression) ROW('2014-04-21 00:00:00.00000','{“sleep":380,“steps":7423,“name":"Joe "}') …
  21. 21. 21 Timeseries on JSON CREATE TABLE iotvti(id INT PRIMARY KEY, stime DATETIME YEAR TO FRACTION(5)), jdata BSON); SELECT id, jdata.temp::int, jdata.loc::varchar(32) FROM iotvti WHERE jdata.temp > 75; db.iotvti.find({“jdata.temp”:{$gt:75}, {jdata:1}, {jdata:1}); {“temp":75, “wind":7.2, “loc":“Miami-1 "}
  22. 22. 22 Informix REST API REpresentational State Transfer http://<hostname>[:<port#>]/<db>/<collection> Integrated into Informix GET /demo/people?sort={age:-1}&fields={_id:0,lastName:0} RESPONSE: [{"firstName":"Anakin","age":49}, {"firstName":"Padme","age":47}, {"firstName":"Luke","age":31}, {"firstName":"Leia","age":31}] GET /stores_demo/ts_data_v?query={loc_esi_id:"4727354321046021"}
  23. 23. 23 A v a ila b le M e th o d s M e th o d P a th D e s c r ip tio n P O S T / C r e a te a n e w d a ta b a s e P O S T /d b C r e a te a n e w c o lle c tio n P O S T /d b /c o lle c tio n C r e a te s a n e w d o c u m e n t G E T / D a ta b a s e lis tin g G E T /d b C o lle c tio n lis tin g G E T /d b /c o lle c tio n Q u e r y th e c o lle c tio n D E L E T E / D r o p a ll d a ta b a s e s D E L E T E /d b D r o p a d a ta b a s e D E L E T E /d b /c o lle c tio n D r o p a c o lle c tio n D E L E T E /d b /c o lle c tio n ? q u e r y = { ...} D e le te d o c u m e n ts th a t s a tis fy th e q u e r y fr o m a c o lle c tio n P U T /d b /c o lle c tio n U p d a te a d o c u m e n t INFORMIX REST API
  24. 24. ODBC, JDBC connections Informix Dynamic Server Tables Tables Relational Tables and views Relational Tables and views JSON CollectionsJSON Collections {Customer}{Customer} partnerspartners SQL & BI Applications {Orders}{Orders} CRMCRM InventoryInventory Tables Timeseries TablesTimeseries Tables {mobile/devices}{mobile/devices} Analytics
  25. 25. Informix Warehouse Accelerator Informix Database Server Informix warehouse Accelerator BI Applications Data mart Tools Ready IBM Smart Analytics Studio
  26. 26. Informix Dynamic Server Tables Tables Relational Tables and views Relational Tables and views JSON CollectionsJSON Collections {Customer}{Customer} partnerspartners SQL & BI Applications {Orders}{Orders} CRMCRM InventoryInventory Tables Timeseries TablesTimeseries Tables {Orders}{Orders} Text index (BTS) spatial indices Text index (BTS) spatial indices Informix Warehouse Accelerator – In-Memory Query Engine ODBC, JDBC connections SQL Apps/Tools MongoDB Drivers NoSQL Apps/Tools Mongo clientMongo client Node.JS Express.JSExpress.JS AngularJSAngularJS
  27. 27. IWA: Complex Data Analysis Informix Database Server Informix Warehouse Accelerator BI Applications Informix Database Server Factdim1 Dim4 - View dim3 dim2 dim2 Informix IoT ApplicationsLoB Apps IoT Applications BI Applications Mobile Apps Informix
  28. 28. IWA: Complex Data Analysis Informix Database Server Informix Warehouse Accelerator Informix Database Server SQL Table SQL View {JSON} SQL Table SQL Table Informix LoB Apps IoT Applications BI Applications Mobile Apps Informix Timeseries {JSON} {JSON} Cognos SQL Table
  29. 29. ODBC, JDBC connections Informix Dynamic Server Tables Tables Relational Tables and views Relational Tables and views JSON CollectionsJSON Collections {Customer}{Customer} partnerspartners SQL & BI Applications {Orders}{Orders} CRMCRM InventoryInventory Tables Timeseries TablesTimeseries Tables {mobile/devices}{mobile/devices} Analytics Informix warehouse Accelerator
  30. 30. Hybrid Power for IoT Apps Right type for right data – SQL, JSON, Timeseris, Spatial, Text Variety of APIs: REST, JDBC, ODBC Platform & device options Right size for the right problem – Customize What’s good for embed is good good for the cloud Continuous availability Accelerate with Informix Warehouse Accelerator Mix it in with Bluemix Scale-up, Scale-out
  31. 31. IBM Informix developer edition. http://www-03.ibm.com/software/products/en/infodeveedit IBM Informix developer edition. http://www-03.ibm.com/software/products/en/infodeveedit Download Now! Download Now!
  32. 32. Thank you

×