0<br />Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery <br />Dr. Vladimir Bacvanski, Vice President, I...
What is this revolution about?<br />1<br />NO SLOW APPS!<br />NO BAD SQL!<br />GET CONTROL BACK !<br />
Show of Hands: What Data Access Technology Have You Used?<br />2<br />Hibernate<br />EJB Entity Beans<br />JDBC<br />JPA<b...
Java Data Access – Two Views of the World<br />Writing Java code is so easy with this eclipse environment.I wish it was th...
Meet in the<br />Middle<br />Data Mapping Approaches<br />Application-Centric<br />Top-Down<br />Start with Object Domain ...
EJB, JPA, and Hibernate  vs. The Database<br /><ul><li>DBA and SQL developer  chasm
Where is the SQL coming from?
What is it?
Where is it?
How do we tune it?
How de we manage it?
Performance Concerns:
Some App Server vendors claim (unsurprisingly) that Managed objects performs fine.
There are many user claims of bad Managed object performance is bad on the web.
As always, the truth is in the middle.
And will depend on your app server, application, database, etc ..</li></ul>“Our top story: Large Customer moves from COBOL...
Introducing pureQuery<br />A high-performance, data access platform to simplify developing, managing, securing, and optimi...
Add basic OR mapping and annotated-method style<br />pureQuery<br />pureQuery Balances Productivity and Control<br />Manag...
Design Phase pureQuery close-up<br />Jump Start Application Design<br /><ul><li>Generate SQL and Code from Database Objects
Setup basic DAO Pattern</li></ul>Existing JDBC to Static<br /><ul><li>Reroute Dynamic Queries to Static</li></ul>Code Deve...
Database aware, Java SQL Editor</li></ul>SQL Replacement<br /><ul><li>Replace Query w/o changing source</li></ul>SQL Perfo...
Track back to SQL and line of code in the application</li></ul>Static SQL<br /><ul><li>Lock in Access plans, Improve Secur...
Code Example: pureQuery<br />10<br />Employee myEmp= db.queryFirst(     &quot;SELECT NAME, ADDRESS, PHONE_NUM FROM EMP    ...
Why Should be the Data Specialists be interested in pureQuery?<br />11<br />
Motivations of the Data Specialists<br /><ul><li>SQL Performance Tuning
Ease of Tuning
Autonomy of Developers
Predictability of an Optimized Data Access Path
Reduction of Costs to satisfy SQL statements
Optimized Access Paths
Reduction of CPU intensive components of SQL Execution
Utilization of Specialty Processors
Capacity Planning
Hardware Utilization
Problem Determination capabilities</li></ul>12<br />
pureQuery Capabilities<br /><ul><li>Static SQL for Runtime with Dynamic SQL Execution in Development
pureQuery can utilize SELECT INTO from a Java application
Upcoming SlideShare
Loading in …5
×

Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2

2,723 views

Published on

Vladimir Bacvanski and Dan Galvin

Looking for a more flexible and efficient way for Java programs to access the database? Join us as we explore how you can
bridge the gap between Java and relational databases. Enhance your Java environment with access layer generation, data
access best practices, traceability between Java packages and SQL statements, improved impact analysis and more. And most importantly, see how new technology can improve not only new development, but existing applications as well. Be prepared to
see designs and code samples!

Published in: Technology
1 Comment
2 Likes
Statistics
Notes
  • A good introduction to #pureQuery for Java developers and DBAs.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
2,723
On SlideShare
0
From Embeds
0
Number of Embeds
107
Actions
Shares
0
Downloads
55
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide
  • What do the Data Specialists care about? Things like SQL tuning, CPU Costs, Capacity Planning and Hardware Resources. They want to easily identify the SQL statements in an application, obtain the information about the access path and make appropriate adjustments. So how can pureQuery help address these concerns? In several critical and impressive ways.
  • For SQL that is static in nature, it is advantageous to bind the SQL statically for many reasons. First, the bound package is easy to EXPLAIN. Second, the selected path is relatively assured and consistent. Third, the prepare process is eliminated. Each statement that is executed dynamically must be prepared on each execution. Statement caching in the container and on the DBMS can help reduce these costs, but still, the costs can be significant. The higher the volume of executions, the more pronounced the costs.
  • Here is an example of a method in a Data Access Object (DAO) that actually contains both a Heterogeneous and Homogeneous batch. Notice the data objects are instantiated and then the start batch is invoked. The methods in the pureQuery interfaces are invoked which actually contain the SQL and then the endBatch is executed. This causes the batch to be executed in one line turn.How does this help?
  • Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2

    1. 1. 0<br />Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery <br />Dr. Vladimir Bacvanski, Vice President, InferData, vladimir@inferdata.com<br />Daniel Galvin, Consultant, Galvin Consulting, dang@galvinconsulting.com<br />Session Number 2171<br />
    2. 2. What is this revolution about?<br />1<br />NO SLOW APPS!<br />NO BAD SQL!<br />GET CONTROL BACK !<br />
    3. 3. Show of Hands: What Data Access Technology Have You Used?<br />2<br />Hibernate<br />EJB Entity Beans<br />JDBC<br />JPA<br />iBatis<br />SQLJ<br />What’s most important to you?<br />Productivity<br />Performance<br />Security<br />Portability<br />
    4. 4. Java Data Access – Two Views of the World<br />Writing Java code is so easy with this eclipse environment.I wish it was that easy to get the SQL right.<br />JSP<br />QoS goals<br />Spring<br />Another runaway query! Where are these coming from? JDBC? Hmmm…<br />Runstats<br />XML<br />http<br />Partition strategy<br />Stored<br />Procedures<br />mashup<br />SQL<br />Response Time!<br />REORG<br />JSON<br />JDBC<br />This ORM doesn’t allow me to leverage all my database’s SQL.<br />Inconsistent response time? How long will it take me to find the offending application sending bad SQL this time? <br />JDBC<br />I can’t believe I got called out last week. I wish I could see how these queries will run in production.<br />SQLJ<br />JPA<br />These ad-hoc queries are dangerous. We need a library of tested SQL interfaces. <br />Application<br />Developer<br />Database Developer& Administrator<br />Spring<br />Why does this query take so long?<br />iBatis, . . .<br />Sometimes I need POJOs, sometime JSON, sometimes XML, what should I use?<br />Static SQL? Sounds like another delay to getting my program deployed<br />Another GRANT request? This security administration is out of control.<br />Can I examine the SQL “before” the application is deployed?<br />
    5. 5. Meet in the<br />Middle<br />Data Mapping Approaches<br />Application-Centric<br />Top-Down<br />Start with Object Domain Model<br />ORM Mapping<br />Well supported in dynamic languages and frameworks<br />Hybrid<br />Meet in the middle<br />Can be challenging w/o comprising<br />Data-Centric<br />Bottom-UP<br />Start with Relational Data Model<br />Not well supported in dynamic languages and frameworks<br />Top<br />Down<br />Persistence Layer<br />Bottom<br />Up<br />4<br />
    6. 6. EJB, JPA, and Hibernate vs. The Database<br /><ul><li>DBA and SQL developer chasm
    7. 7. Where is the SQL coming from?
    8. 8. What is it?
    9. 9. Where is it?
    10. 10. How do we tune it?
    11. 11. How de we manage it?
    12. 12. Performance Concerns:
    13. 13. Some App Server vendors claim (unsurprisingly) that Managed objects performs fine.
    14. 14. There are many user claims of bad Managed object performance is bad on the web.
    15. 15. As always, the truth is in the middle.
    16. 16. And will depend on your app server, application, database, etc ..</li></ul>“Our top story: Large Customer moves from COBOL to Java to become more agile. In other news, DBA develop amnesia.”<br />5<br />5<br />
    17. 17. Introducing pureQuery<br />A high-performance, data access platform to simplify developing, managing, securing, and optimizing data access.<br />pureQuery Components:<br />Simple and intuitive API<br />Enables SQL access to databases or in-memory Java objects<br />Facilitates best practices <br />Optim Development Studio (integrates with RAD/RSA)<br />Integrated development environment with Java and SQL support <br />Improve problem isolation and impact analysis<br />Optim pureQuery Runtime<br />Flexible static SQL deployment for DB2<br />
    18. 18. Add basic OR mapping and annotated-method style<br />pureQuery<br />pureQuery Balances Productivity and Control<br />Managed objects <br />Object-relational mapping<br />Full SQL control<br />Code all your SQL<br />JDBC / SQLJ<br />Use SQL templates, inline only<br />Spring <br />templates<br />iBATIS<br />Complex OR mapping and persistence management, but loss of controls<br />Hibernate<br />Adds container management option<br />OpenJPA <br />(EJB3)<br />
    19. 19. Design Phase pureQuery close-up<br />Jump Start Application Design<br /><ul><li>Generate SQL and Code from Database Objects
    20. 20. Setup basic DAO Pattern</li></ul>Existing JDBC to Static<br /><ul><li>Reroute Dynamic Queries to Static</li></ul>Code Development Productivity<br /><ul><li>Code Generation, Content Assist
    21. 21. Database aware, Java SQL Editor</li></ul>SQL Replacement<br /><ul><li>Replace Query w/o changing source</li></ul>SQL Performance Metrics<br /><ul><li>Find and sort query elapsed time from Java</li></ul>Oracle Support<br /><ul><li>Replace Query w/o changing source</li></ul>Java to SQL Integration<br /><ul><li>Categorize by Java, SQL, Database , Packages, track back to line of code</li></ul>Problem Determination<br /><ul><li>Monitor WebSphere Connection Pool, JDBC Driver, Network
    22. 22. Track back to SQL and line of code in the application</li></ul>Static SQL<br /><ul><li>Lock in Access plans, Improve Security, Consistent Performance</li></ul>SQL Injection Prevention<br /><ul><li>Lock down SQL for Dynamic</li></li></ul><li>Code Example: JDBC<br />java.sql.PreparedStatementps = con.prepareStatement(<br /> &quot;SELECT NAME, ADDRESS,PHONE_NUM FROM EMP <br /> WHERE NAME=?&quot;);<br />ps.setString(1, name);<br />java.sql.ResultSetrs= ps.executeQuery();<br />names.next();<br />Employee myEmp = new Employee();<br />myEmp.setName(rs.getString(1));<br />myEmp.setHomeAddress(rs.getString(2));<br />myEmp.setHomePhone(rs.getString(3));<br />names.close();<br />class Employee {<br />String name;<br />String homeAddress;<br />String homePhone;<br /> …<br />}<br />
    23. 23. Code Example: pureQuery<br />10<br />Employee myEmp= db.queryFirst( &quot;SELECT NAME, ADDRESS, PHONE_NUM FROM EMP WHERE NAME=?&quot;, Employee.class, name);<br />Even simpler, if we have a method getEmployee with a Java annotation or XML file with SQL for the query:<br />Employee myEmp= getEmployee(name);<br />
    24. 24. Why Should be the Data Specialists be interested in pureQuery?<br />11<br />
    25. 25. Motivations of the Data Specialists<br /><ul><li>SQL Performance Tuning
    26. 26. Ease of Tuning
    27. 27. Autonomy of Developers
    28. 28. Predictability of an Optimized Data Access Path
    29. 29. Reduction of Costs to satisfy SQL statements
    30. 30. Optimized Access Paths
    31. 31. Reduction of CPU intensive components of SQL Execution
    32. 32. Utilization of Specialty Processors
    33. 33. Capacity Planning
    34. 34. Hardware Utilization
    35. 35. Problem Determination capabilities</li></ul>12<br />
    36. 36. pureQuery Capabilities<br /><ul><li>Static SQL for Runtime with Dynamic SQL Execution in Development
    37. 37. pureQuery can utilize SELECT INTO from a Java application
    38. 38. With Client Optimization, Static SQL from existing JDBC with no changes to the Application
    39. 39. Homogeneous and Heterogeneous Batching of Statements
    40. 40. Statically bound packages are easy to EXPLAIN and monitor for changes in access path
    41. 41. pureQuery coupled with IBM Optim Performance Monitoring provides E2E Performance Monitoring and Problem Determination
    42. 42. Impact analysis is greatly improved by the static packages and the ability to tie each statement to a method in the application code</li></ul>13<br />
    43. 43. Static vs. Dynamic SQL Execution<br />14<br />
    44. 44. Static vs. Dynamic SQL<br />
    45. 45. Cost of Prepare<br />CPU cost of Short Prepare on DB2 9 for z/OS – between 400µs and 1ms<br />CPU cost of Full Prepare on DB2 9 for z/OS – approximately 30 to 50ms. Cost could be much higher and generally increase with complexity.<br />
    46. 46. Trial Results<br />Estimated CPU Savings<br />(In DB2 CPU)<br />
    47. 47. How well does it work? – Java applications<br />In-house testing shows significant performance improvements<br />IRWW – an OLTP workload, Type 4 driver<br />Cache hit ratio between 70 and 85%<br />23 % improvement in throughput using pureQuery over dynamic JDBC<br />15% - 25% reduction on CPU per transaction over dynamic JDBC <br />18<br />
    48. 48. How well does it work? - .Net applications<br />Throughput during static execution increased by 159% over dynamic SQL execution assuming a 79% statement cache hit ratio <br /><ul><li>IRWW – OLTP application
    49. 49. Application accesses DB2 for z/OS</li></ul>*Any performance data contained in this document were determined in various controlled laboratory environments and are for reference purposes only. Customers should not adapt these performance numbers to their own environments as system performance standards. The results that may be obtained in other operating environments may vary significantly. Users of this document should verify the applicable data for their specific environment.<br />19<br />
    50. 50. Batching SQL Statements<br />20<br />
    51. 51. Homogeneous & Heterogeneous Batch <br />Homogeneous Batch – all instances in the batch are the same statement and require only 1 line turn<br />Heterogeneous Batch – allows different SQL statements to be included in batch.<br />Both Utilize Multi-Row Insert<br />Heterogeneous Batches may contain 0 to many Homogeneous Batches<br />
    52. 52. Heterogeneous Batch Example<br />publicint[][] insertAgentAndPolicies(AgentTO agentTO, PolicyTO[] policyTO) {<br /> Data data = DataFactory.getData(ds, pdqProps);<br />try {<br /> AgentData agentData = DataFactory.getData(AgentData.class, data);<br /> PolicyData policyData = DataFactory.getData(PolicyData.class, data);data.startBatch(HeterogeneousBatchKind.heterogeneousModify__);<br /> agentData.insertAgent(agentTO);<br /> policyData.insertPolicies(policyTO);<br />returndata.endBatch();<br /> } finally {<br /> data.close();<br /> } <br />}<br />
    53. 53. Client Optimization <br />Allows you to bind static SQL packages from existing JDBC code<br />Avoids the cost of rewriting the application to code to the pureQuery API<br />Allows Heterogeneous batch with minor changes to the code<br />None of the productivity advantages are realized. Code is still maintained in JDBC.<br />End-to-End monitoring lacks some introspective capability into the coding<br />Creation of the static packages requires that you run the code.<br />Some overhead at runtime related to resolution of statements to static packages<br />
    54. 54. 24<br />Optimize Existing JDBC Applications<br />Improve performance for DB2 – without changing a line of code<br />Capture Configure Bind Execute<br />pureQuery client optimization enables static execution for JDBC applications (custom-developed, framework-based, or packaged)<br />Existing JDBC Application<br />Captured SQL- <br />related<br />metadata<br />JDBC Driver w/ pureQuery<br />Dynamic SQL execution <br />Static SQL execution <br />DB2 Data Servers<br />&quot;The ability to use static SQL with pureQuery is huge. Recently, I worked with a client who could reduce CPU usage by 7 percent thanks to this one feature.&quot; <br />— David Beulke, Pragmatic Solutions Inc.<br />
    55. 55. Design<br />Develop<br />Optimize<br />Govern<br />Models<br />Policies<br />Metadata<br />Deploy<br />Operate<br />IBM Optim pureQuery<br />Reduce costs<br />Increase system throughput <br />Improve developer productivity<br />Move workload to zIIP and zAAP<br />Improve quality of service for new and existing Java applications <br />Improve performance <br />Lock in access plans<br />Speed up problem resolution<br />Reduce development time for new Java applications<br />Bridge Java and data<br />Balance productivity and control<br />Enhance developer and DBA collaboration<br />Enhance security<br />Limit user access <br />Minimize SQL injection risk<br />Improve audit readiness<br />Developer<br />Develop<br />Code<br />Debug<br />Test<br />Tune, Package<br />Tester<br />
    56. 56. Why should DBAs care ?<br />DBAs have little to no visibility of application SQL before deployment, no opportunity for review and optimization <br />Problem isolation takes days with contemporary environments such as Java, PHP, .NET, etc due to inability to trace SQL to Java application and source code<br />Constantly increasing Java application workload taxes existing systems – need to fit more work into existing systems<br />SQL injection represents an increasing risk to data security<br />
    57. 57. Why should Developers care ?<br />Get data access right the first time !<br />Get it done faster - Improved productivity<br />Single environment that spans Java application and database development<br />Improved problem isolation and resolution<br />
    58. 58. 28<br />Control performance<br />Decide at deployment time how the SQL is executed<br />Understand and lock down the access plan for SQL<br />Replace suboptimal SQL without changing the application<br />Control security<br />Prevent SQL injection<br />Prevent execution of unauthorized SQL<br />Better manage database security <br />See inside applications that are driving your database<br />Understand where SQL comes from<br />Understand when frameworks and ORM’s are getting in the way<br />Simplify problem determination and troubleshooting<br />Correlate problem SQL with applications, ORM’s and frameworks<br />Optim pureQuery Runtime<br />
    59. 59. 29<br />How do I start with pureQuery?<br />Existing applications<br />Optimize existing JDBC (and .NET!) applications<br />No code changes needed<br />Have to go through the client optimization process to get to static SQL<br />New applications<br />Use the pureQuery API<br />Development codes using one API regardless of whether it is deployed dynamically or statically<br />DBA deploys statically<br />No need to go through client optimization process<br />Other<br />JPA, iBatis, Hibernate<br />
    60. 60. 30<br />pureQuery Facilitates Best Practices<br />Supports both inline SQL and Java annotations (method) <br />Intuitive interfaces for common data retrieval and manipulation scenarios hides JDBC complexity<br />Query First<br />Homogeneous Batch<br />Reduce network trips to the database <br />Query Over Java Collections<br />Heterogeneous Batch<br />Use custom result handlers to map results to POJO’s, XML, JSON, …<br />Write high performance Java data access applications, Part 3: <br />Data Studio pureQuery API best practices<br />-- VitorRodrigues<br />http://www.ibm.com/developerworks/db2/library/techarticle/dm-808rodrigues/?S_TACT=105AGX01&S_CMP=LP<br />
    61. 61. 31<br />A TypicalApplication Architecture with pureQuery<br />Presentation Layer<br />Implements the U/I or network protocols using the business services<br />Business Service Layer<br />Never use the pureQuery API directly. Gets data from the Data Access Layer<br />Data Access Layer<br />Using the pure-query API to access the database.<br />Provides a technology neutral API to the data used by the business services<br />pureQuery<br />pureQuery makes this layer easy, fast, consistent and traceable<br />Database<br />Sometimes additional layers are required<br />Workflow<br />Data federation<br />Workspace<br />
    62. 62. RAD or RSA / Optim Development Studio Data Centric Development Scenario<br />Write in JavaUsing RAD+WAS FP for Web 2.0<br />Write in Java with pureQueryUsing Optim Dev. Studio in RAD<br />Access generated Java data objects from code developed in RAD<br />WAS Feature Pack for Web 2.0<br />
    63. 63. Demo<br />33<br />Demo!<br />
    64. 64. 34<br />Conclusion: pureQuery Revolutionary Advantages<br />Excellent performance<br />Static and dynamic SQL is captured during test and optimized before deployment<br />Enables lock-in of access path<br />Great productivity<br />Excellent tool support through Optim Development Studio<br />Shell share with Rational tools<br />Mapping from SQL to Java captured and traceable<br />Facilitates collaboration between DBA’s and developers<br />Performance tuning, impact analysis<br />Better security<br />Limits SQL injection<br />Controlled database access<br />
    65. 65. Where to go Next? Resources and more…<br />Optim Development Studio <br />http://www.ibm.com/software/data/optim/development-studio/<br />IBM pureQuery<br />http://www.ibm.com/software/data/optim/purequery-platform/faq.html<br />pureQuery Custom Training <br />InferData, IBM Business Partner http://www.inferdata.com<br />Course: Developing Database Applications with Optim Development Studio and pureQuery<br />http://www.inferdata.com/training/data/optim_purequery_training.html<br />35<br />
    66. 66. Web, Blogs<br />Integrated Data Management (Optim and Data Studio)<br />http://www.ibm.com/developerworks/spaces/optim<br />Vladimir’s Blog: On Building Software<br />http://www.OnBuildingSoftware.com<br />Twitter:<br />http://twitter.com/OnSoftware<br />36<br />
    67. 67. 37<br />Thank You!Your Feedback is Important to Us<br />Please complete the survey for this session by:<br />Accessing the SmartSite on your smart phone or computer at: iodsmartsite.com <br />Surveys / My Session Evaluations<br />Visiting any onsite event kiosk<br />Surveys / My Session Evaluations<br />Each completed survey increases your chance to win an Apple iPod Touch with daily drawling sponsored by Alliance Tech<br />37<br />

    ×