Db2 tech talk build next generation applications with db2 json capabilities


Published on

Db2 tech talk build next generation applications with db2 json capabilities

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Db2 tech talk build next generation applications with db2 json capabilities

  1. 1. © 2013 IBM Corporation 1 Build Next-Generation Applications with DB2 JSON Capabilities October 17, 2013 Presented by: Skyla Loomis, Senior Development Manager, DB2 JSON Support
  2. 2. © 2013 IBM Corporation 2 Need webcast troubleshooting help? Click attachments 1. The presentation for this Tech Talk: http://bit.ly/ttfileoct13 2. Next steps and troubleshooting guide: click “Attachments“ in this webcast window A few details …. Skyla Loomis Senior Development Manager, DB2 JSON Support Today’s technical presenters . . . DB2 Tech Talk series host and today‟s presenter: Rick Swagerman, Host and Today’s Presenter DB2 Language Architect
  3. 3. © 2013 IBM Corporation 3 Need webcast troubleshooting help? Click attachments Disclaimer The information contained in this presentation is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided “as is”, without warranty of any kind, express or implied. In addition, this information is based on IBM‟s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other documentation. Nothing contained in this presentation is intended to, or shall have the effect of: • Creating any warranty or representation from IBM (or its affiliates or its or their suppliers and/or licensors); or • Altering the terms and conditions of the applicable license agreement governing the use of IBM software. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
  4. 4. © 2013 IBM Corporation The Customer-Activated Enterprise  Three major themes emerged in 2013: – Open up to customer influence – Pioneer digital-physical innovation – Craft engaging customer experiences.  Role of mobile is key: – 84% CIOs expect to invest in mobility solutions over the next few years – 94% CMOs believe that mobile applications will play a big role in helping them realize their goals IBM Global C-Suite Studies http://www-935.ibm.com/services/us/en/c-suite/csuitestudy2013/ Need webcast troubleshooting help? Click attachments4
  5. 5. © 2013 IBM Corporation  Engaging  Mobile  Rapidly changing  Dynamic  Short-lived  Competitive  Gateway to enterprise systems Mobile Social Big Data Analytics Cloud A New Era of Applications Need webcast troubleshooting help? Click attachments5
  6. 6. © 2013 IBM Corporation Engaging Web and Mobile Applications  Modern User Interface and Form Factors – Implemented using JavaScript and HTML5 for browser presentation layer – Implemented using JavaScript in IBM Worklight for hybrid mobile application Need webcast troubleshooting help? Click attachments6
  7. 7. © 2013 IBM Corporation Client Challenge Key Capabilities Using standards-based technologies and tools and delivering an enterprise-grade services layer that meets the needs of mobile employees and customers Mobile optimized middleware • Open approach to 3rd-party integration • Cross platform, mix native and HTML • Strong authentication framework • Encrypted offline availability • Enterprise back-end connectivity • Unified push notifications • Data collection for analytics • Direct updates and remote disablement • Packaged runtime skins IBM Worklight goes beyond mobile app UI creation to deliver full mobile middleware. IBM Worklight: A mobile enterprise application platform Speeding the development, integration, and management of mobile applications Need webcast troubleshooting help? Click attachments7
  8. 8. © 2013 IBM Corporation Open standards-based development framework optimized for code re-use across device platforms. Mobile application server provides mobile-specific administrative, notifications, analytics and security services while leveraging existing investments in data, applications and infrastructure Client device layer enables client-side security, enforces app upgrades, secures local storage and allows access to device features. Essential development, runtime and client device services IBM Worklight Mobile Application Platform Overview SQL Adapter An adapter exposes a set of services. Mobile apps invoke procedures via Ajax requests. Information is retrieved from the back-end application. The back-end application then returns data in some format. Use IBM Worklight SQL adapter to execute parameterized SQL queries and stored procedures, in order to retrieve or update data in the database.Need webcast troubleshooting help? Click attachments8
  9. 9. © 2013 IBM Corporation New Era Application Requirements  Store data from web/mobile apps in it's native form – New web applications use JSON for storing and exchanging information – Very lightweight – write more efficient apps – It is also the preferred data format for mobile application back-ends  Move from development to production in no time! – Ability to create and deploy flexible JSON schema – Gives power to application developers by reducing dependency on IT; no need to pre-determine schemas and create/modify tables – Ideal for agile, rapid development and continuous integration DB2 Need webcast troubleshooting help? Click attachments9
  10. 10. © 2013 IBM Corporation JSON is the Language of the Web  JavaScript Object Notation – Lightweight data interchange format – Specified in IETF RFC 4627 – http://www.JSON.org  Designed to be minimal, portable, textual and a subset of JavaScript – Only 6 kinds of values! – Easy to implement and easy to use  Text format, so readable by humans and machines  Language independent, most languages have features that map easily to JSON  Used to exchange data between programs written in all modern programming languages { "firstName“ : "John", "lastName" : "Smith", "age" : “25”, "address" : { "streetAddress“: "21 2nd Street", "city" : "New York", "state" : "NY", "postalCode" : "10021" }, "phoneNumber“ : [ { "type" : "home", "number“: "212 555-1234" }, { "type" : "fax", "number“: "646 555-4567" } ] } Need webcast troubleshooting help? Click attachments10
  11. 11. © 2013 IBM Corporation Data Access Example Using Javascript and JSON  Relational representation  JSON representation JSON_string = „{“Lastname”:”Jones”,”Firstname”:”Billy”,”Street”:”123 Maple Drive”}‟;  Javascript data access var JSONelems = JSON.parse(JSON_string); l_name = JSONelems.Lastname; f_name = JSONelems.Firstname; l_street = JSONelems.Street; Lastname Firstname Street Jones Billy 123 Maple Drive Need webcast troubleshooting help? Click attachments11
  12. 12. © 2013 IBM Corporation 12 Insert a record, a blog post by Joe: db.posts.insert({author:"Joe", date:"2012-04-20", post:"..."}) Find all posts by Joe: db.posts.find({author:"Joe"}) Delete all posts of Joe: db.posts.remove({author:"Joe"}) Simple database API for JSON Need webcast troubleshooting help? Click attachments
  13. 13. © 2013 IBM Corporation Typical JSON Open Source Datastore Attributes  Logging is often turned off to improve performance  By default, no return code on insert (a.k.a. “fire and forget”) – App must verify update was performed  Data is sharded for scalability  Shards are replicated asynchronously for availability – Queries to replica nodes can return back-level data sometimes…  No concept of commit or rollback – Each JSON update is independent – Applications have to implement compensation logic to update multiple documents with ACID properties  JSON documents are stored in collections – But no “join” across collections  No document-level locking – App must manage a “revision” tag to detect document update conflicts  No document-level or tag-level security  No built-in temporal or geo-spatial query support Need webcast troubleshooting help? Click attachments13
  14. 14. © 2013 IBM Corporation Achieve Agility with a Trusted Foundation  Combine data from “systems of engagement” in same DB2 database with “systems of record” relational data – Best of both worlds – Simplicity and agility of JSON + enterprise strengths of DB2 – Simplify data access when a single application‟s data store has different access characteristics  Interoperate seamlessly with modern applications – Flexible schemas allow rapid delivery of applications  Preserve traditional DBMS capabilities, leverage existing skills and tools: – Multi-statement transactions – Management, operations – Security – Scale, performance, and high availability Need webcast troubleshooting help? Click attachments14
  15. 15. © 2013 IBM Corporation DB2 JSON Document Store Support Generally Available as of 10.5 Fix Pack 1  Java Driver for JSON API – Java Driver supporting JSON API – Transactions – Parametric SQL statements (Delete, Select) – Temporal tables  JSON Wire Listener – Leverage community drivers  CLP-Like Command Shell – Ad-hoc updates / queries – Administration commands  DB2 Data Server – JSON documents stored as BSON (Binary JSON) in BLOBs within DB2 – Scalar function and UDF extensions – Enhanced indexing on expression capabilities allows indexing of JSON members DB2 Database JSON API JSON CLP JDBC Driver DRDA Indexing of individual JSON members Java PHP NodeJS BSON Wire Protocol Applications NoSQL JSON Wire Listener JSON JSON JSON JSON JSON JSONJSON Need webcast troubleshooting help? Click attachments15
  16. 16. © 2013 IBM Corporation Coming Soon: JSON Document Store Support in DB2 V10 for z/OS  Available in Version 10 this December – Version 11 support to follow  Enable JSON support in DB2 V10 with: – Server-side built-in functionality for storing and indexing JSON documents (DB2 INFO APAR II14727) – Server-side UDFs for JSON document access (DB2 Accessories Suite for z/OS V3.1) – Client-side API and wire listener for use of community drivers (Latest DB2 JDBC Driver) Announcement Details: http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=ca&infotype=an&supplier=897&letternum=ENUS213-395 Need webcast troubleshooting help? Click attachments16
  17. 17. © 2013 IBM Corporation Java Driver for JSON API Translates API calls to SQL + function invocations  Supports transactions  Batches insertions  Fire-forget inserts (fast)  Indexing  Time travel query  Smart query re-write  Good performance with inline LOBS Need webcast troubleshooting help? Click attachments17
  18. 18. © 2013 IBM Corporation /*Set up Conn. and Database handle*/ Context ctx = new InitialContext(); DataSource ds = (DataSource)ctx.lookup(“jdbc/myDB2”); Connection conn = ds.getConnection(); Database db = new Database(conn); DBCollection shop = db.getCollection(“shop”); /*Create JSON objects and insert*/ BasicDBObject cart = new BasicDBObject(); BasicDBObject amtDue = new BasicDBObject(); cart.put(“sid”, “176”); cart.put(“customer”, "Bill“)‟; amtDue.put(“subtotal”, 50.07); amtDue.put(“tax”, 4.26); amtDue.put(“total”, 54.33) cart.put(“amtDue”, amtDue); shop.insert(cart); /* Use cursor to fetch back the JSON */ DBCursor cursor = shop.find(new BasicDBObject(“customer”, “Bill”)); try { while(cursor.hasNext()) { DBObject obj = cursor.next(); doSomething(obj); } } finally{ cursor.close(); //close the cursor no matter what. } JSON API from Java Need webcast troubleshooting help? Click attachments18
  19. 19. © 2013 IBM Corporation Partnership with IBM and MongoDB  MongoDB (then 10Gen) and IBM announced their partnership in June 2013  There are many common use cases of interest addressed by partnership: – Accessing JSON Data in DB2, Informix, MongoDB using JSON Query – Making JSON Data available for Interactive Analytics – Integrating JSON Data with MDM – Schema-less JSON Data for variety of applications – Securing JSON Data  IBM and MongoDB are collaborating in 3 areas: – Open Governance: Standards and Open Source – Technology areas of mutual interest – Products and solutions  The collaboration provides benefits to various members of the IT organization  DB2, Guardium, Data Stage and other IBM products have JSON support Need webcast troubleshooting help? Click attachments19
  20. 20. © 2013 IBM Corporation JSON Wire Listener  Built on JSON API  Leverages community (Node.js, Python, Perl, PHP, Ruby, etc)  Opportunity to contribute extensions to community drivers  Starting the listener – Runs as a java process – use wplistener.bat or .sh to start • Takes DB2 userid / password • Takes port number (defaults to Mongo default port # 27017) • Specify DB2 as backend  Redirecting traffic to the wire listener – Alter the IP address from MongoDB server to the location where the wire listener is running – Default port is same as MongoDB default https://github.com/acmeair/acmeair Need webcast troubleshooting help? Click attachments20
  21. 21. © 2013 IBM Corporation Node.js code sample var databaseUrl = “shop” var collections = [“cart”] var db = require(“mongojs”).connect(databaseUrl, collections); … Db.users.save({sid: “176”, customer: “shopper99@yahoo.com”, amtDue: {subtotal: 50.07, tax: 4.26, total: 54.33}}, function(err, saved) { if (err || !saved ) console.log(“cart not saved”); else console.log(“cart saved”); }); … Db.cart.find({customer: “shopper99@yahoo.com”}, function(err. carts) { if (err || !carts) console.log(“No carts found”); else carts.forEach( function(iCart) { console.log(iCart);});}); Need webcast troubleshooting help? Click attachments21
  22. 22. © 2013 IBM Corporation CLP-Like Command Shell Capabilities  Working with collections and documents – Store and query documents with implicitly created collections – Create collections explicitly – Import documents  Query JSON documents – Control options for result sets – Aggregate data  Update documents  Work with indexes  Administration – Work with collections – Get statistics – Remove data, indexes, and collections Need webcast troubleshooting help? Click attachments22
  23. 23. © 2013 IBM Corporation 23 CLP-Like Command Shell Details  Command Syntax – Most commands are parameterized – Ex: getCollectionNames()  help() or help – Will show list of acceptable commands – Further details about each available. – Example: db.collection.find.help()  Some basic command examples: – db.getCollectionNames() – db.createCollection(“mycollection”) – db.mycollection.insert({firstname:Bill, lastname: Smith})  Other useful options – debug(true) - shows (most) of the translated SQL – printSQL(true) - prnts the translated SQL but does not execute it » Useful for DDL options like createCollection – tracefile(filename) - more execution details sent to tracefile Need webcast troubleshooting help? Click attachments
  24. 24. © 2013 IBM Corporation  Simple Index db.collection.ensureIndex({sid:{1, "$int"}}); //create ascending integer index on „sid'. db.collection.ensureIndex({“customer":1}}); //create ascending varchar(50) (default type) index on „customer' field.  Composite index containing multiple fields db.collection.ensureIndex({customer:[1, "$string", 20], total:{-1, "$int"}}); //create compound index with two fields: customer ascending with type varchar(20), and total descending as integer.  Index on nested object db.collection.ensureIndex({amtDue.total:{1, "$int”});  How does JSON field indexing work? – Indexes are created on fields within the JSON document CREATE INDEX CUSTNDX ON JSON_VAL(JSONBLOB, “customer”, “:i”) – Subsequent queries searching on customer will use same functional expression in a predicate Indexes Need webcast troubleshooting help? Click attachments24
  25. 25. © 2013 IBM Corporation 25 Write safety levels  Write safety controlled by setting writeconcern property on the collection – Example: DBCollection webStats = _db.getCollection(“webPageStats"); webStats.setWriteConcern (WriteConcern.NONE);  Default writeConcern is “SAFE” – Wait for server write confirmation – Network or server errors are reported  WriteConcern.NONE means the application does not wait for write confirmation – a.k.a. “Fire and Forget” – Application essentially places inserts on a queue and continues – Useful for high ingestion rates, initial load of data – Pool of asynchronous threads / connections are used to perform the inserts in batches Need webcast troubleshooting help? Click attachments
  26. 26. © 2013 IBM Corporation  Flexible Schema is agile, liberating for application developers  But will we abandon years of expertise in data modeling / normalization theory? – How to maintain control in an enterprise, mission critical DBMS?  Identification of appropriate applications is critical  Application deployment procedures need to adapt – New controls to prevent schema chaos – Application Development Groups need to implement controls  When combining with application that uses relational schema – Identify portions that need to remain dynamic – Allocate / accommodate space for that as JSON – Future – combination of SQL and JSON will make this easier What is JSON‟s Role in the Enterprise? Need webcast troubleshooting help? Click attachments26
  27. 27. © 2013 IBM Corporation Blending Flexible Schemas In Traditional Defined Schema Environments  During prototyping, early development, validation –As system stabilizes, stable schema parts could be moved to relational columns  For cases where web message or document will be retrieved as-is –Yet retrieval on internal fields is needed  When parts of the schema will always be dynamic Need webcast troubleshooting help? Click attachments27
  28. 28. © 2013 IBM Corporation Relational Simple normalized schema (DB2 sample) with relational constraints: NoSQL JSON - Two approaches: embedded (de-normalized) Using references {dept: “A10”, deptname:”Shipping”, manager:”Samuel”, emp:[ {empno:”000999”, lastname:”Harrison”, edlevel:”16”}, {empno:”370001”, lastname:”Davis”, edlevel:”12”} ] proj:[ {projno:”397”, projname:”Site Renovation”, respemp:”370001” }, {projno:”397”, projname:”Site Renovation”, respemp:”370001”} … ] } If you need normalization and database-enforced constraints, JSON might not be best choice {_id dept … ) {_id emp dept ref … } {_id dept emp ref … ) Requires application- side join Chance for data redundancy Data Normalization - Choose the Right Solution Need webcast troubleshooting help? Click attachments28
  29. 29. © 2013 IBM Corporation  Consider DB2 JSON when: – Application and schema subject to frequent changes – Prototyping, early stages of application development – De-normalized data has advantages • Entity / document is in the form you want to save • Read efficiency – return in one fetch without sorting, grouping or ORM mapping – Systems of Engagement • Less stringent “CAP” requirements in favor of speed • Eventual consistency is good enough • Social media  Relational still best suited when these are critical – Data normalization to • Eliminate redundancy • Ensure master data consistency – Database-enforced constraints What data store format makes sense for your application? Need webcast troubleshooting help? Click attachments29
  30. 30. © 2013 IBM Corporation  Does NoSQL mean NoDBA? NoDB2? – Definitely Not - The relational database isn‟t going away anytime soon. – JSON becoming a complementary technology to relational  Transactional atomicity is essential for mission critical business transactions. – DB2 JSON document store solution brings commits, transaction scope  Future Direction – access JSON and relational data together more directly – Gain better balance between fixed schema and flexible schema JSON and DB2 – Complementary Technologies Need webcast troubleshooting help? Click attachments30
  31. 31. © 2013 IBM Corporation InformationOnDemand2013  Save the date November 3 - 7 Need webcast troubleshooting help? Click attachments31
  32. 32. © 2013 IBM Corporation Elective Sessions  Tuesday, Nov 5th, 10:00am – 11:00am – IPT-2909B: Embracing NoSQL: IBM and the Community – Cross Platform Data Management track  Tuesday, Nov 5th, 4:30pm – 5:45pm – IPT-2059A: JSON Document Support in IBM DB2 Schema Flexibility in a Trusted Database Environment – Cross Platform Data Management track  Wednesday, Nov 6th, 8:15am - 9:30am – IDZ-2599A: NoSQL and DB2 for z/OS: Receiving Agility from a Trusted Enterprise Database – DB2 for z/OS and Tools track  Wednesday, Nov 6th, 10:00am - 11:00am – IDB-2075A: Competitive Product Delivery with IBM DB2 JSON – DB2 for LUW and Tools track Hands-on Labs  Thursday, Nov 7th, 2pm – 5pm (also available as drop-in lab!) – IDB-1878A: Agile Product Development Using IBM DB2 with JSON – DB2 LUW; introductory; first time offered at a conference Expert Exchange  Tuesday, Nov 5th, 3:00pm – 4:00pm – IPT-3070A: Expert Exchange: Extending Information Management Solutions with NoSQL Capabilities – XML, RDF, and JSON – Cross Platform Data Management track DB2 JSON sessions at IOD 2013 Need webcast troubleshooting help? Click attachments32
  33. 33. © 2013 IBM Corporation The NoSQL, MongoDB, Big Data Meetup at IOD 2013 Wednesday, November 6th, 6-9:00 pm South Pacific A (North Convention Ctr) Register here: http://www.meetup.com/big-data- developers-in-las-vegas/members/ Dear Team Come meetup with us to interact with industry experts on NoSQL, MongoDB and Big Data. Learn how open technologies are changing the face of computing. Bring your clients, partners and colleagues. Tim Vincent, Angel Luis Diaz, and Jerry Cuomo Meetup will feature lightning talks and interactive panel on open technology & NoSQL, Big Data, Cloud and Mobile, highlighting value of open technologies in this vibrant and dynamic space. Free to all IOD registrants (with free food!) Hosts VP, Fellow and CTO, Tim Vincent VP, Standards & Cloud, Angel Diaz VP, Fellow and CTO, Gennaro Cuomo Questions: Contact Susan Malaika, malaika@us.ibm.comMore Info http://bit.ly/MeetupIOD Need webcast troubleshooting help? Click attachments33
  34. 34. © 2013 IBM Corporation 34 Need webcast troubleshooting help? Click attachments DB2 Tech Talk: Next Steps Roadmap - DB2 JSON Capabilities Read 4-part DB2 JSON Tech Article Series • Introduction to DB2 JSON ibm.co/15ImEke • Command line processor ibm.co/GYfi3e • Writing apps with Java API ibm.co/19RWv5Y • JSON Wire Listener ibm.co/16aLEmF Refer to the JSON API Java documentation and DB2 Info Center JSON API: ibm.co/GRf3q4 DB2 JSON: ibm.co/19RYNli Test drive the features • DB2 10.5 trial software: Ibm.co/db2download Learn about DB2 and MongoDB ibm.co/1gkGt81 Reference Call IBM to schedule a demo or learn more • 1 800 966-9875 (U.S) • 1-888-746-7426 (Canada) • 1800-425-3333 (India) • Or visit http://www.ibm.com/planetwide/ for contact information worldwide IBM DB2 10.5 product page Ibm.com/db2 IBM DB2 10.5 Product features ibm.co/12c1PJz IBM Data Studio product page Ibm.co/ibmdsinfo Tech forum on developerWorks bit.ly/db2forumluw SQL Tips Blog: www.sqltips4db2.com Step Three Step Two Step One Step Four
  35. 35. © 2013 IBM Corporation 35 Need webcast troubleshooting help? Click attachments Upcoming Tech Talks Don’t miss these in-depth DB2 10.1 feature talks! Dates and topics subject to change and modification. How to register : DB2 Tech Talks web site Next DB2 Tech Talk: Use DB2 on the Cloud • November 21, 2013 at 12:30 PM • Rav Ahuja, IBM Labs DB2 Evangelist • Registration: http://bit.ly/tt2013nvm IBM Information on Demand Conference November 3, 2013, Las Vegas, Nevada Big Data and Analytics Tutorials • November 3 – Sunil Soares on Data Governance • November 7 – Vlad Bacvanski on Hadoop November 7 – Peter Aiken on Information Architecture ibm.co/IODtutorials Some recommended replays from this year: • Introduction and Technical Tour of DB2 10.5 • Adopt BLU Acceleration the Faster Easier Way • pureScale + HADR for Always Available Transactions • IDUG Tech Talk: Tuning Queries with Data Studio • IDUG Tech Talk: Ten Principles of Highly Effective DBAs
  36. 36. © 2013 IBM Corporation DB2 Tech Talk 36 Need webcast troubleshooting help? Click attachments
  37. 37. © 2013 IBM Corporation 37 Need webcast troubleshooting help? Click attachments Questions Listening in replay? Questions: www.sqltips4db2.com Click submit a question.
  38. 38. © 2013 IBM Corporation 38 Need webcast troubleshooting help? Click attachments Thanks for attending! Please rate the session Presentation download: bit.ly/ttfileoct13 click Attachments in this webcast environment