© 2007 IBM Corporation IBM Information Management


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

© 2007 IBM Corporation IBM Information Management

  1. 1. Revolutionizing the Data Abstraction Layer and Solution Delivery for Data-Driven Applications Curt Cotner IBM Fellow Vice President and CTO for IBM Database Servers [email_address]
  2. 2. Disclaimer <ul><ul><li>© Copyright IBM Corporation 2009. All rights reserved. </li></ul></ul><ul><ul><li>U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. </li></ul></ul><ul><ul><li>THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.  WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.  IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE. </li></ul></ul><ul><li>IBM, the IBM logo, ibm.com, and DB2 are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml </li></ul><ul><li>Other company, product, or service names may be trademarks or service marks of others. </li></ul>
  3. 3. DB2 Connect Recommended Deployment Options Desktop PCs Application servers Web application servers Several options: Personal Edition or DB2 Connect Server or file server Co-locate DB2 Connect on the application server <ul><li>Recommendation: </li></ul><ul><li>Personal Edition is best for small numbers of end users </li></ul><ul><li>DB2 Connect server or file server deployment is best for lots of desktops </li></ul><ul><li>Rationale: </li></ul><ul><li>drivers now include the key DB2 Connect gateway features (sysplex workload balancing, connection concentrator, XA support, automatic reconnect, etc.) </li></ul><ul><li>fewer potential points of failure </li></ul><ul><li>less hardware cost </li></ul><ul><li>less system administration cost </li></ul><ul><li>fewer network hops (up to 40% better elapsed time) </li></ul><ul><li>simplified failover strategy </li></ul><ul><li>less complex problem determination and monitoring </li></ul>
  4. 4. pureQuery Runtime – every Java application benefits! <ul><li>JDBC – acceleration for any JDBC application </li></ul><ul><ul><li>Convert dynamic SQL to static SQL </li></ul></ul><ul><ul><li>Replace problem queries without changing the source </li></ul></ul><ul><ul><li>Remove literals from SQL to get better statement cache hit ratios </li></ul></ul><ul><li>Hibernate/OpenJPA/iBatis – acceleration for persistence layers </li></ul><ul><ul><li>Improved SQL “batch” peformance </li></ul></ul><ul><ul><li>Auto-tuning of Hibernate and OpenJPA peristence options </li></ul></ul><ul><li>SQL-friendly APIs for OO access to relational </li></ul><ul><ul><li>Object to relational mapping </li></ul></ul><ul><ul><li>APIs that can be tailored to return XML, JSON, arrays, etc. </li></ul></ul><ul><li>Improved management, monitoring, problem determination </li></ul><ul><ul><li>Tracks SQL back to the Java class file and line number </li></ul></ul><ul><ul><li>Enables performance monitors to report by application name </li></ul></ul><ul><li>Provides the foundation for improved developer tooling </li></ul><ul><ul><li>Syntax assist, code generation, performance reporting, etc. </li></ul></ul>
  5. 5. What’s so Great About DB2 Accounting for CICS Apps? z/OS LPAR CICS AOR1 Txn1 - Pgm1 - Pgm2 CICS AOR2 TxnA - PgmX - PgmY DB2PROD CICS AOR3 Txn1 - Pgm1 - Pgm2 App CPU PLAN Txn1 2.1 TN1PLN TxnA 8.3 TNAPLN <ul><li>DB2 Accounting for CICS apps allows you to study performance data from many perspectives: </li></ul><ul><li>By transaction (PLAN name) </li></ul><ul><li>By program (package level accounting) </li></ul><ul><li>By address space (AOR name) </li></ul><ul><li>By end user ID (CICS thread reuse) This flexibility makes it very easy to isolate performance problems, perform capacity planning exercises, analyze program changes for performance regression, compare one user’s resource usage to another’s, etc. </li></ul>
  6. 6. JDBC Performance Reporting and Problem Determination – Before pureQuery Application Server DB2 or IDS A1 A2 A5 A3 A6 A4 USER1 USER1 USER1 User CPU PACKAGE USER1 2.1 JDBC USER1 8.3 JDBC USER1 22.0 JDBC What is visible to the DBA? - IP address of WAS app server - Connection pooling userid for WAS - app is running JDBC or CLI What is not known by the DBA? - which app is running? - which developer wrote the app? - what other SQL does this app issue? - when was the app last changed? - how has CPU changed over time? - etc. Data Access Logic Persistence Layer DB2 Java Driver EJB Query Language
  7. 7. What’s so Great About Optim pureQuery Accounting for WebSphere Applications? z/OS LPAR CICS AOR2 TxnA (PLANA) - PgmX - PgmY App CPU TxnA 2.1 TxnB 8.3 <ul><li>Data Studio and pureQuery provide the same granularity for reporting WebSphere’s DB2 resources that we have with CICS: </li></ul><ul><li>By transaction (Set Client Application name ) </li></ul><ul><li>By class name (program - package level accounting) </li></ul><ul><li>By address space (IP address) </li></ul><ul><li>By end user ID (DB2 trusted context and DB2 Roles) This flexibility makes it very easy to isolate performance problems, perform capacity planning exercises, analyze program changes for performance regression, compare one user’s resource usage to another’s, etc. </li></ul>Unix or Windows WAS TxnA (Set Client App=TxnA) - ClassX - ClassY
  8. 8. Simplifying Problem Determination Scenario <ul><li>Application Developer </li></ul><ul><li>Available for each db access </li></ul><ul><ul><li>SQL text generated </li></ul></ul><ul><ul><li>Access path </li></ul></ul><ul><ul><li>Cost estimates </li></ul></ul><ul><ul><li>Estimated response time </li></ul></ul><ul><ul><li>Elapsed & CPU time </li></ul></ul><ul><ul><li>Data transfer (getpages) </li></ul></ul><ul><ul><li>Tuning advice </li></ul></ul><ul><li>Database Administrator </li></ul><ul><li>Available for each SQL </li></ul><ul><ul><li>Application name </li></ul></ul><ul><ul><li>Java class name </li></ul></ul><ul><ul><li>Java method name </li></ul></ul><ul><ul><li>Java object name </li></ul></ul><ul><ul><li>Source code line number </li></ul></ul><ul><ul><li>Source code context </li></ul></ul><ul><ul><li>J-LinQ transaction name </li></ul></ul><ul><ul><li>Last compile timestamp </li></ul></ul>Java Profiling pureQuery DRDA Extentions
  9. 9. Simpler Development Employee my_emp = db.queryFirst(&quot; SELECT Name, HomeAddress, HomePhone FROM Employee WHERE Name=? &quot;, Employee.class, my_emp); -or- Employee my_emp = getEmployee(name); #sql [con] { SELECT NAME, ADDRESS, PHONE_NUM INTO :name, :addr, :phone FROM EMP WHERE NAME=:name }; new Employee my_emp; my_emp.setName(name); my_emp.setHomeAddress(addr); my_emp.setHomePhone(phone); SQLJ: JDBC: java.sql.PreparedStatement ps = con.prepareStatement( &quot; SELECT NAME, ADDRESS, PHONE_NUM FROM EMP WHERE NAME=? &quot;); ps.setString(1, name); java.sql.ResultSet names = ps.executeQuery(); names.next(); new Employee my_emp; my_emp.setName(names.getString(1)); my_emp.setHomeAddress(names.getString(2)); my_emp.setHomePhone(names.getString(3)); names.close(); pureQuery API’s: XML file or Java annotation SELECT * FROM EMPLOYEE WHERE NAME=?1; class Employee { public String Name; public String HomeAddress; public String HomePhone; … } Table Column Type EMP NAME CHAR(64) EMP ADDRESS CHAR(128) EMP PHONE_NUM CHAR(10)
  10. 10. Data API: query Beans <ul><li>The class of the return Bean type is passed in as a parameter </li></ul><ul><ul><li>Person person = data.queryFirst(&quot; SELECT * FROM person </li></ul></ul><ul><ul><li>WHERE person.name=? &quot;, Person.class, &quot;Brian&quot;); </li></ul></ul><ul><ul><li>List<Person> people = data.queryList(&quot; SELECT * FROM person &quot;, </li></ul></ul><ul><ul><li>Person.class); </li></ul></ul><ul><ul><li>Person[] people = data.queryArray(&quot; SELECT * FROM person &quot;, </li></ul></ul><ul><ul><li>Person.class); </li></ul></ul><ul><ul><li>Iterator<Person> people = data.queryIterator(&quot; SELECT * FROM person &quot;, </li></ul></ul><ul><ul><li>Person.class); </li></ul></ul><ul><li>Beans, Maps, Arrays, Collections, Iterators, or your own data </li></ul>
  11. 11. Example – create XML from a query <ul><li>Create XML from a query: String xml = d.query(&quot; select * from Department &quot;, new XMLResultHandler()); </li></ul><ul><li>XMLResultHandler has one method: </li></ul><ul><li>String handle(ResultSet rs) </li></ul><ul><li>sb.append(&quot; <&quot;+ m.getTableName(x) +&quot;>&quot;); </li></ul><ul><li>for (int x=1; x<=cols; x++) { </li></ul><ul><li> sb.append(&quot;<&quot;+ m.getColumnName(x) +&quot;>&quot;); </li></ul><ul><li> sb.append( rs.getString(x) ); </li></ul><ul><li> sb.append(&quot;</&quot;+ m.getColumnName(x) +&quot;>&quot;); </li></ul><ul><li> } </li></ul><ul><li>sb.append(&quot; </&quot;+ m.getTableName(x) +&quot;>&quot;); </li></ul><ul><li>Output: </li></ul><ul><li><result> </li></ul><ul><li><DEPARTMENT><DEPTNO>A00</DEPTNO><DEPTNAME>SPIFFY </li></ul><ul><li>COMPUTER SERVICE DIV.</DEPTNAME> <MGRNO>000010</MGRNO> ... </DEPARTMENT> </li></ul><ul><li></result> </li></ul>
  12. 12. Java Data Access in 5 Simple Steps 1. Select table 2. Name bean & select styles 3. Generate test code 4. Map table to bean 5. Select template SQL CRUD
  13. 13. Developing with pureQuery Unleash SQL from Java SQL analysis SQL execution SQL validation SQL content assist
  14. 14. Developing with pureQuery Unleash SQL from Java SQL validation SQL content assist SQL analysis SQL execution
  15. 15. Developing with pureQuery Unleash SQL from Java <ul><li>Run SQL with parameters at design time in the Java program WITHOUT writing a test application </li></ul>SQL validation SQL content assist SQL analysis SQL execution
  16. 16. Developing with pureQuery Unleash SQL from Java <ul><ul><li>View explain plans for SQL inside you Java programs </li></ul></ul>SQL execution SQL validation SQL content assist SQL analysis
  17. 17. Using pureQuery to Foster Collaboration and Produce Enterprise-ready Apps Application Server Catalog data for SQL Application Meta data DB2 or IDS Prod A4 A1 A1 A6 A6 A2 A2 A3 A3 A4 A4 A5 A5 A1 A4 A5 Performance Data Warehouse Application Developer Database Administrator A1 A6 A2 A3 A4 A5 Quickly compare unit test perf results to production Use pureQuery app metadata as a way to communicate in terms familiar to both DBA and developer Application Meta data DB2 or IDS Dev System A1 A6 A2 A3 A4 A5 A1 A4 A5
  18. 18. pureQuery Outline View’s 3 categorizations <ul><li>How do you look at the relationship between SQL and Java? </li></ul>pureQuery Outline
  19. 19. pureQuery Tools SQL templates and customizations IBM Data Studio pureQuery For DBAs and Application Developers (v1.2) - Part 1 <ul><li>Create your own SQL templates </li></ul><ul><li>Use templates to write SQL that is frequently reused </li></ul><ul><li>Use tabs to change the variable names after inserting SQL statement from the template through SQL context assist </li></ul>Use newly created merge template in your java code using SQL assist
  20. 20. Optim Development Studio -- pureQuery Outline Speed up problem isolation for developers – even when using frameworks <ul><li>Capture application-SQL-data object correlation (with or without the source code) </li></ul><ul><li>Trace SQL statements to using code for faster problem isolation </li></ul><ul><li>Enhance impact analysis identifying application code impacted due to database changes </li></ul><ul><li>Answer “Where used” questions like “Where is this column used within the application?” </li></ul><ul><li>Use with modern Java frameworks e.g. Hibernate, Spring, iBatis, OpenJPA </li></ul>
  21. 21. How Optim Development Studio Helps
  22. 22. On-ramps to pureQuery Open Source Persistence Engine JPA Persistence Engine JDBC API Hibernate, iBATIS, EclipseLink,... JPA for WebSphere, Apache OpenJPA pureQuery API Data Studio pureQuery Plain JDBC Data Web Services, Project Zero, sMash JCC driver JDBC .Net Applications .Net applications pureQuery JPA API Open Source Persistence API Web API DB2, Informix, and Oracle now…more coming ADO .Net
  23. 23. Client Optimization Improve Java data access performance for DB2 – without changing a line of code <ul><li>Captures SQL for Java applications </li></ul><ul><ul><li>Custom-developed, framework-based, or packaged applications </li></ul></ul><ul><li>Bind the SQL for static execution without changing a line of code </li></ul><ul><ul><li>New bind tooling included </li></ul></ul><ul><li>Delivers static SQL execution value to existing DB2 applications </li></ul><ul><ul><li>Making response time predictable and stable by locking in the SQL access path pre-execution, rather than re-computing at access time </li></ul></ul><ul><ul><li>Limiting user access to tables by granting execute privileges on the query packages rather than access privileges on the table </li></ul></ul><ul><ul><li>Aiding forecasting accuracy and capacity planning by capturing additional workload information based on package statistics </li></ul></ul><ul><ul><li>Drive down CPU cycles to increase overall capability </li></ul></ul><ul><li>Choose between dynamic or static execution at deployment time, rather than development time </li></ul>
  24. 24. Optim pureQuery Runtime for z/OS <ul><li>In-house testing shows double-digit reduction in CPU costs over dynamic JDBC </li></ul><ul><li>IRWW – an OLTP workload, Type 4 driver </li></ul><ul><li>Cache hit ratio between 70 and 85% </li></ul><ul><li>15% - 25% reduction on CPU per txn over dynamic JDBC </li></ul>
  25. 25. Throughput Increase with .NET <ul><li>Same IRWW OLTP application used for the Java tests but in .NET </li></ul><ul><li>Application access DB2 for z/OS via Windows Application Server (IIS) </li></ul><ul><li>Throughput during static execution increased by 159% over dynamic SQL execution assuming a 79% statement cache hit ratio </li></ul>
  26. 26. Increased throughput with Optim pureQuery Runtime v2.2 as compared to JDBC with 90% Package Cache hit ratio <ul><li>In comparison with JDBC, pureQuery Static APIs doubled the throughput (ITR*) of the DB2 LUW database server. </li></ul>On identical hardware, pureQuery API Static (Method Style) more than doubled the transaction throughput. * See speaker notes
  27. 27. <ul><li>Compared to JDBC, pureQuery Client Optimization Static and pureQuery API Static (Method Style) increased the throughput (ITR*) of the DB2 LUW database server by 60 and 66% respectively </li></ul>Increased throughput with Optim pureQuery Runtime v2.2 as compared to JDBC with 95% Package Cache hit ratio On identical hardware, pureQuery Client Optimization Static improved transaction throughput by 60% * See speaker notes
  28. 28. Have You Heard of SQL Injection?
  29. 29. More Visibility, Productivity, and Control of Application SQL <ul><li>Capture </li></ul><ul><ul><li>Capture performance and application metadata </li></ul></ul><ul><li>Optimize </li></ul><ul><ul><li>Review and share SQL </li></ul></ul><ul><ul><ul><li>Visualize hotspots </li></ul></ul></ul><ul><ul><ul><li>Analyze impacts from schema changes </li></ul></ul></ul><ul><ul><ul><li>Trace SQL to originating source </li></ul></ul></ul><ul><ul><li>Optimize and replace SQL </li></ul></ul><ul><ul><li>Create approved SQL list </li></ul></ul><ul><li>Deploy </li></ul><ul><ul><li>Configure execution properties </li></ul></ul><ul><ul><li>Optionally bind for static execution </li></ul></ul>Capture Review Restrict Configure Bind Execute Monitor Optimize Java Persistence Architecture
  30. 30. Visualize execution metrics Execute, tune, share, trace, explore SQL Replace SQL without changing the application Position in Database Explorer Visualize application SQL
  31. 31. Optim Development Studio <ul><li>Problem determination and isolation </li></ul><ul><ul><li>with pureQuery outline go to the source of the problematic SQL </li></ul></ul><ul><li>Improve hibernate data access calls </li></ul><ul><ul><li>Re-write HQL </li></ul></ul><ul><ul><li>Use better performing native SQL </li></ul></ul><ul><li>With performance metrics Identify the hot spots in your hibernate application </li></ul>Source code correlation Database object correlation Identify and change the HQL
  32. 32. Analyze Use of Sensitive Data in Applications Optim Development Studio Icon identifies sensitive access See queries accessing sensitive data Filter to see use of sensitive data View only SQL accessing sensitive data Filter SQL by action
  33. 33. What Is Heterogeneous Batching? Data Server Heterogenous Batching – multiple operations across different tables all execute as one batch Table 1, operation 4 Table 2, operation 2 Table 1, operation 1 Table 1, operation 2 Table 1, operation 3 Table 2, operation 1 Table 3, operation 1
  34. 34. JDBC Batching v/s pureQuery Heterogeneous Batching <ul><li>JDBC batching used by Hibernate Batcher is currently limited </li></ul><ul><ul><li>Cannot batch entities that map to multiple tables </li></ul></ul><ul><ul><ul><li>Primary and Secondary tables. </li></ul></ul></ul><ul><ul><ul><li>Inheritance Join and Table per class strategies </li></ul></ul></ul><ul><ul><li>Cannot batch different operations against same table </li></ul></ul><ul><ul><ul><li>Field level updates </li></ul></ul></ul><ul><ul><ul><li>Insert, update </li></ul></ul></ul><ul><ul><li>Cannot batch different entities </li></ul></ul><ul><ul><li>Each batch is a message to the database </li></ul></ul><ul><li>pureQuery heterogeneous batching plug-in for Hibernate on the other hand </li></ul><ul><ul><li>Can batch entities that map to multiple tables </li></ul></ul><ul><ul><li>Can batch different operations against the same table </li></ul></ul><ul><ul><li>Can batch different entities into a single batch </li></ul></ul><ul><ul><li>Combines insert, deletes, updates into single batch </li></ul></ul>* Preliminary findings based on validation with a test designed to demonstrate heterogeneous batching differences. This is not intended to be a formal benchmark .
  35. 35. pureQuery – Stripping Literals from SQL JDBC app INSERT INTO T1 VALUES(‘ABC’,2,’DEF’) INSERT INTO T1 VALUES(:h1,:h2,:h3) pureQuery Runtime conversion <ul><li>pureQuery can identify statements that use no parameter markers, and strip the literals out at runtime </li></ul><ul><li>significant performance gains: </li></ul><ul><ul><li>less CPU cost at PREPARE </li></ul></ul><ul><ul><li>better use of dynamic statement cache </li></ul></ul>
  36. 36. Optim Performance Manager - Overview dashboard Enlarge ( Shows average and max values in the selected timeframe ) Open details ( shows timeseries in the selected timeframe ) <ul><li>Focus on specific database to see key performance indicators </li></ul>
  37. 37. Optim Performance Manager - Active SQL dashboard Show top running statements at a point in time according to selected metric, e.g. Elapsed time. “ Tune” launches Query Tuner “ Identify Workloads” refers to defined workloads in Extended Insight
  38. 38. Optim Performance Manager – Reporting <ul><li>Reporting </li></ul><ul><li>... gives you the flexibility you need to adapt OPM on your own environment </li></ul><ul><li>coming with sample reports for </li></ul><ul><li>- disk space usage </li></ul><ul><li>- system configuration </li></ul><ul><li>- top resource using SQL statements </li></ul><ul><li>- DB connections </li></ul><ul><li>easy to modify and adapt to your environment because report definition stored in XML document </li></ul><ul><li>can be saved and shared as PDF with your colleagues </li></ul><ul><li>a graphical report builder will also follow soon </li></ul>
  39. 39. Toughest issue for Web applications – Problem diagnosis and resolution Web Browser Users Web Server Application Server DB2 Server Business Logic Data Access Logic Persistence Layer DB2 Java Driver JDBC Package EJB Query Language
  40. 40. Customer Job Roles – A Barrier to a “Holistic View” Application Server DB Server Data Access Logic Persistence Layer DB Java Driver JDBC Package EJB Query Language WebSphere Connection Pool Business Logic 1 3 5 4 2 Application Developer System Programmer DBA Network Admin
  41. 41. How do we plan to help?
  42. 42. Extended Insight (EI) Overview dashboard
  43. 43. OPM and OPM EI – SQL Resolution Integration Points Query Tuner Integration provides tuning advice and tools pureQuery Integration provides source code identification
  44. 44. <ul><li>IBM Data Studio </li></ul><ul><ul><li>www.ibm.com/software/data/studio </li></ul></ul><ul><ul><ul><li>FAQs / Tutorials </li></ul></ul></ul><ul><ul><ul><li>Downloads </li></ul></ul></ul><ul><ul><ul><li>Forum / Blogs </li></ul></ul></ul><ul><ul><ul><li>Join the IBM Data Studio user community </li></ul></ul></ul><ul><li>Data Studio Book </li></ul><ul><ul><li>http:// bit.ly/dstudiobook </li></ul></ul>NEW!!!