Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Developing hybrid applications with informix


Published on

Learn about developing hybrid apps using IBM Informix

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Developing hybrid applications with informix

  1. 1. © 2015 IBM Corporation DMX-2904 : Developing Hybrid Applications With Informix: Your Data Model Should Not Restrict Data Access Erika Von Bargen (, Shawn Moe ( Thursday October 29, 2015 08:30 AM-09:30 AM
  2. 2. • IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. • Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. • The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. • The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. Please Note: 2
  3. 3. Agenda • The Informix Wire Listener • Unified Data Access • MongoDB API • REST API • Hybrid Operations • SQL passthrough • Joins • Application examples 2
  4. 4. The Informix Wire Listener • The wire listener is a mid-tier gateway server that enables communication between MongoDB and REST applications and the Informix database server. • It’s a software layer that translates MongoDB protocol and REST commands into Informix SQL for storage and processing of data • It’s a Java application packaged as an executed JAR file – jsonListener.jar – that communicates to the Informix server using the Informix JDBC driver. 3
  5. 5. NoSQL & Listener Architecture 4 REST Wire Listener Protocol conversion Mongo DB wire protocol REST Informix Relational, JSON, Timeseries Storage JDBC, SQL
  6. 6. The Informix Wire Listener: Unified Data Access • The Informix wire listener unified access to the following types of data: JSON data, stored in “collections” Relational data Time Series data Spatial data • The flexibility of JSON enables us to represent all of the above as a JSON document collection. • When you run an operation, the listener performs a lookup of the type of table you are querying to determine if it's a collection, relational, or time series table • A query processor will build the equivalent SQL statement for the table type you are querying 5
  7. 7. Unified Data Access • JSON Data / Collections JSON documents are transformed into a BSON representation and are stored in a BSON column of a relational table. Queries run against collections are rewritten by the listener to SQL queries using BSON manipulation functions Indexes can be created on fields of the JSON/BSON document using Informix functional indexes. Client access based on MongoDB API functions • Same capabilities available in MongoDB API are also available through the REST API 6
  8. 8. • Relational Data: The columns of a relational table are mapped to the fields of a JSON document. MongoDB API calls can be run on relational tables as though the tables were document collections. • For example, the same call to create an index on a field of a document would create an index on a column of the same name. All basic operations available with document collections (insert, update, delete, query, aggregations) can be used without modification against a relational table. • Story: We had a user, who was using a collection, create a relational table that had the same name as the collection and had columns with corresponding data types and names as the fields in the document. The application worked without modification. Informix row types are represented as sub-documents. Unified Data Access
  9. 9. • Time Series Data: Informix time series data is stored in a way that is similar to a column-store database. The present strategy is to use the Informix Virtual Table Interface (VTI) to make the TimeSeries data appear to be a relational table. This virtual table can then be accessed by the Mongo & REST APIs. • Spatial Data: Spatial data can be inserted into a document collection using the GeoJSON specification. Informix functional indexes enable users to efficiently query spatial data. Unified Data Access: Time Series & Spatial Data
  10. 10. The Mongo Wire Listener • The Mongo listener (listener.type=mongo) communicates with clients using the MongoDB wire protocol Allows MongoDB clients to communicate with Informix via the listener Mongo collections are mapped to special collection tables in Informix that contain a BSON column • Supports hybrid access Not only can Mongo client applications access BSON (collection) data, but Informix relational and timeseries data as well Same Mongo API functions (find, insert, update, delete, findAndModify, aggregate, count, explain, etc) can be used on all Informix data types 9
  11. 11. MongoDB Application Driver Compatibility • Use any of the MongoDB client drivers and frameworks against the Informix database server Little to no change required when running MongoDB programs Informix listens on the same default port as mongo, no need to change. • Leverage the different MongoDB drivers available • Other MongoDB Community Drivers are also available 10 C Perl C# PHP Java Python JavaScript Ruby Node.js Scala
  12. 12. Mongo Operations 11 Action Mongo Syntax Insert db.customer.insert( { name: “John", age: 21 } ) Query (select all) db.customer.find() Query with condition db.customer.find( {age: { $gt:21 } } ) Update db.customer.update( { age: { $gt: 18} }, { $set: { status: “Adult" } }, { multi: true } ) Delete db.customer.remove( { age: { $gt:21 } } ) Create Index db.customer.ensureIndex( { name : 1, age : -1 } ) Use the exact same syntax no matter if the underlying Informix data storage is JSON, relational, or time series.
  13. 13. The REST wire listener • The REST listener (listener.type=rest) communicates with clients using REST Allows driverless access to Informix via the listener • Eliminating middle tier of three-tier architecture in favor of a two-tier client database model simplifies development, debugging, deployment and speed time to market. • Eliminates reliance on vendor-supplied proprietary drivers • Supports hybrid access REST access to JSON, relational, and timeseries data Same REST methods (GET, POST, PUT, DELETE) work the same way no matter the underlying data type 12
  14. 14. RESTful Data Access • URL Structure http://<host>[:port]/<db>/<collection> where <collection> is the name of a JSON collection, relational table, or timeseries table/VTI • REST HTTP methods: GET = query a resource POST = create a resource (insert) PUT = update a resource DELETE = delete a resource 13
  15. 15. REST: HTTP method overview 14 Path Descriptio n Text Text Text
  16. 16. REST Examples • Query: GET /test/people Get all documents from the namespace test.people (db=test, collection=people) Example result [{"_id":{"$oid":"533501be27784337861702c5"},"name":"lance"}, {"_id":{"$oid":"533501c227784337861702c6"},"name":"brian"}, {"_id":{"$oid":"533501c627784337861702c7"},"name":"erika"}] The result is a JSON array of documents. • Query with condition: GET /test/users?query={age:{$gt:30}}&sort={age:1} &fields:{firstName:1, age:1, _id:0} Get all documents from the namespace test.users (db=test, collection=users) whose age > 30, sort the results by age and filter the fields in the response
  17. 17. REST Examples • Insert: POST to collection namespace, providing your JSON document to insert in the data of the HTTP request POST /test/users Data: {firstName:"John",lastName:"Doe",age:31} Response: {"n":1, "ok":true} • Delete: Send a DELETE request to a collection namespace, providing your query condition in the URL DELETE /test/users?query={name:“Larry”} Response: {"n":1, "ok":true}
  18. 18. REST Examples • Update: Send a PUT to a collection namespace, providing your query condition in the URL and the update operation in the data of the HTTP request PUT /test/users?query={firstName:“Larry”} Data: {"$set": {age : 25}} Response: {"n":1, "ok":true}
  19. 19. Hybrid Operations: SQL Passthrough • Allows users of Mongo and REST clients to execute pure SQL statements • Mixes the ease of use of JSON documents with the power of SQL processing • Special permission is needed to execute raw SQL statements • Listener properties file must have it enabled: security.sql.passthrough=true • If authentication is used, you must have the role sql or sqlAnyDatabase • Uses a pseudo collection “system.sql” to authenticate SQL queries and a new $sql query operator to instruct the listener to execute the JSON value that follows as a raw SQL statement • Results are packaged into JSON documents so clients can work with the results as they would a collection. 18
  20. 20. Hybrid Operations: SQL Passthrough • Example INSERT, UPDATE, and DELETE SQL will return a document containing the number of rows effected. db.getCollection("system.sql").findOne({ "$sql": "delete from cust_calls where (call_dtime + interval(5) year to year) < current" }) Result: { "n" : 7 } • Example QUERY db.getCollection("system.sql").find({ "$sql": "select c.customer_num,o.customer_num as order_cust,count(order_num) as order_count from customer c left outer join orders o on c.customer_num = o.customer_num group by 1, 2 order by 2" }) Result: { "customer_num" : 113, "order_cust" : null, "order_count" : 0 } { "customer_num" : 114, "order_cust" : null, "order_count" : 0 } { "customer_num" : 101, "order_cust" : 101, "order_count" : 1 } { "customer_num" : 104, "order_cust" : 104, "order_count" : 4 } { "customer_num" : 106, "order_cust" : 106, "order_count" : 2 } 19
  21. 21. Hybrid Operations: Joins • Hybrid applications may need to join and combine data from different data sources • The Informix wire listener enables the ability to run JOIN queries Collection-to-collection joins Relational-to-relational joins Collection-to-relational joins • Join support is available to both REST and MongoDB clients via an extension of the Mongo API • Join queries are done by querying the pseudo system.join collection and providing a query document that represents the join specification The listener automatically determines the types of objects being joined and generates the correct SQL syntax to join disparate data 20
  22. 22. Hybrid Operations: Joins • Syntax > db.getCollection(“system.join”).find( { $collections : { “tabName1” : { $project: {…}, $where: {…} }, “tabName2” : {$project: {…}, $where: {…} }, … }, “$condition” : { join_condition_specification } } ) 21
  23. 23. Hybrid Operations: Joins • Example Get the customers orders that totaled more than $100. Join the customers and orders collections/tables on the customer_num field/column where the order total is greater than 100. > db.getCollection(“system.join”).find( { “$collections” : { “customers” : { “$project”: { customer_num: 1, name: 1, phone: 1 } }, “orders” : { “$project”: { order_num: 1, nitems: 1, total: 1, _id: 0 }, “$where” : { total : { “$gt”: 100 } } } }, “$condition” : { “customers.customer_num” : “orders.customer_num” } } ) 22 Same query syntax regardless of where “customers” and “orders” are JSON collections, relational tables, or both.
  24. 24. Sample Applications • Informix sample applications are available for Java, JavaScript (Node.js), Python, and Ruby at Each programming language has both REST and Mongo sample applications • “HelloWorld” samples show basic database operations such as inserting, querying, updating. • “HelloGalaxy” samples go into more in depth topics such as transactions, commands, joins, and hybrid operations. • Samples applications can be run in a local environment or deployed to the Informix TimeSeries DB service in Bluemix 23
  25. 25. Python Example Application: Mongo 24
  26. 26. Python Example Application: REST 25
  27. 27. Conclusion • Informix and the wire listener offer Support for access to hybrid data in a single storage engine: • schema-less JSON document collections, structured relational data, and high performing timeseries data A unified data manipulation language: • A NoSQL query syntax that works on all underlying data storage types A RESTful API for driverless data access A MongoDB API that enables access from a large community of MongoDB language drivers • All of this enables Informix to provide a hybrid database solution that is uniquely capable of simplifying application development, reducing the time to market, and increasing the opportunity to create compelling applications. 26
  28. 28. We Value Your Feedback! Don’t forget to submit your Insight session and speaker feedback! Your feedback is very important to us – we use it to continually improve the conference. Access the Insight Conference Connect tool at to quickly submit your surveys from your smartphone, laptop or conference kiosk. 27
  29. 29. 28 Notices and Disclaimers Copyright © 2015 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided. Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.
  30. 30. 29 Notices and Disclaimers (con’t) Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. • IBM, the IBM logo,, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at:
  31. 31. © 2015 IBM Corporation Thank You