• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2
 

Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2

on

  • 3,449 views

Vladimir Bacvanski and Dan Galvin ...

Vladimir Bacvanski and Dan Galvin

Looking for a more flexible and efficient way for Java programs to access the database? Join us as we explore how you can
bridge the gap between Java and relational databases. Enhance your Java environment with access layer generation, data
access best practices, traceability between Java packages and SQL statements, improved impact analysis and more. And most importantly, see how new technology can improve not only new development, but existing applications as well. Be prepared to
see designs and code samples!

Statistics

Views

Total Views
3,449
Views on SlideShare
3,379
Embed Views
70

Actions

Likes
1
Downloads
47
Comments
1

4 Embeds 70

http://www.onbuildingsoftware.com 45
http://www.slideshare.net 21
http://www.linkedin.com 3
http://www.blogger.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • A good introduction to #pureQuery for Java developers and DBAs.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • What do the Data Specialists care about? Things like SQL tuning, CPU Costs, Capacity Planning and Hardware Resources. They want to easily identify the SQL statements in an application, obtain the information about the access path and make appropriate adjustments. So how can pureQuery help address these concerns? In several critical and impressive ways.
  • For SQL that is static in nature, it is advantageous to bind the SQL statically for many reasons. First, the bound package is easy to EXPLAIN. Second, the selected path is relatively assured and consistent. Third, the prepare process is eliminated. Each statement that is executed dynamically must be prepared on each execution. Statement caching in the container and on the DBMS can help reduce these costs, but still, the costs can be significant. The higher the volume of executions, the more pronounced the costs.
  • Here is an example of a method in a Data Access Object (DAO) that actually contains both a Heterogeneous and Homogeneous batch. Notice the data objects are instantiated and then the start batch is invoked. The methods in the pureQuery interfaces are invoked which actually contain the SQL and then the endBatch is executed. This causes the batch to be executed in one line turn.How does this help?

Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2 Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2 Presentation Transcript

  • 0
    Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery
    Dr. Vladimir Bacvanski, Vice President, InferData, vladimir@inferdata.com
    Daniel Galvin, Consultant, Galvin Consulting, dang@galvinconsulting.com
    Session Number 2171
  • What is this revolution about?
    1
    NO SLOW APPS!
    NO BAD SQL!
    GET CONTROL BACK !
  • Show of Hands: What Data Access Technology Have You Used?
    2
    Hibernate
    EJB Entity Beans
    JDBC
    JPA
    iBatis
    SQLJ
    What’s most important to you?
    Productivity
    Performance
    Security
    Portability
  • Java Data Access – Two Views of the World
    Writing Java code is so easy with this eclipse environment.I wish it was that easy to get the SQL right.
    JSP
    QoS goals
    Spring
    Another runaway query! Where are these coming from? JDBC? Hmmm…
    Runstats
    XML
    http
    Partition strategy
    Stored
    Procedures
    mashup
    SQL
    Response Time!
    REORG
    JSON
    JDBC
    This ORM doesn’t allow me to leverage all my database’s SQL.
    Inconsistent response time? How long will it take me to find the offending application sending bad SQL this time?
    JDBC
    I can’t believe I got called out last week. I wish I could see how these queries will run in production.
    SQLJ
    JPA
    These ad-hoc queries are dangerous. We need a library of tested SQL interfaces.
    Application
    Developer
    Database Developer& Administrator
    Spring
    Why does this query take so long?
    iBatis, . . .
    Sometimes I need POJOs, sometime JSON, sometimes XML, what should I use?
    Static SQL? Sounds like another delay to getting my program deployed
    Another GRANT request? This security administration is out of control.
    Can I examine the SQL “before” the application is deployed?
  • Meet in the
    Middle
    Data Mapping Approaches
    Application-Centric
    Top-Down
    Start with Object Domain Model
    ORM Mapping
    Well supported in dynamic languages and frameworks
    Hybrid
    Meet in the middle
    Can be challenging w/o comprising
    Data-Centric
    Bottom-UP
    Start with Relational Data Model
    Not well supported in dynamic languages and frameworks
    Top
    Down
    Persistence Layer
    Bottom
    Up
    4
  • EJB, JPA, and Hibernate vs. The Database
    • DBA and SQL developer chasm
    • Where is the SQL coming from?
    • What is it?
    • Where is it?
    • How do we tune it?
    • How de we manage it?
    • Performance Concerns:
    • Some App Server vendors claim (unsurprisingly) that Managed objects performs fine.
    • There are many user claims of bad Managed object performance is bad on the web.
    • As always, the truth is in the middle.
    • And will depend on your app server, application, database, etc ..
    “Our top story: Large Customer moves from COBOL to Java to become more agile. In other news, DBA develop amnesia.”
    5
    5
  • Introducing pureQuery
    A high-performance, data access platform to simplify developing, managing, securing, and optimizing data access.
    pureQuery Components:
    Simple and intuitive API
    Enables SQL access to databases or in-memory Java objects
    Facilitates best practices
    Optim Development Studio (integrates with RAD/RSA)
    Integrated development environment with Java and SQL support
    Improve problem isolation and impact analysis
    Optim pureQuery Runtime
    Flexible static SQL deployment for DB2
  • Add basic OR mapping and annotated-method style
    pureQuery
    pureQuery Balances Productivity and Control
    Managed objects
    Object-relational mapping
    Full SQL control
    Code all your SQL
    JDBC / SQLJ
    Use SQL templates, inline only
    Spring
    templates
    iBATIS
    Complex OR mapping and persistence management, but loss of controls
    Hibernate
    Adds container management option
    OpenJPA
    (EJB3)
  • Design Phase pureQuery close-up
    Jump Start Application Design
    • Generate SQL and Code from Database Objects
    • Setup basic DAO Pattern
    Existing JDBC to Static
    • Reroute Dynamic Queries to Static
    Code Development Productivity
    • Code Generation, Content Assist
    • Database aware, Java SQL Editor
    SQL Replacement
    • Replace Query w/o changing source
    SQL Performance Metrics
    • Find and sort query elapsed time from Java
    Oracle Support
    • Replace Query w/o changing source
    Java to SQL Integration
    • Categorize by Java, SQL, Database , Packages, track back to line of code
    Problem Determination
    • Monitor WebSphere Connection Pool, JDBC Driver, Network
    • Track back to SQL and line of code in the application
    Static SQL
    • Lock in Access plans, Improve Security, Consistent Performance
    SQL Injection Prevention
    • Lock down SQL for Dynamic
  • Code Example: JDBC
    java.sql.PreparedStatementps = con.prepareStatement(
    "SELECT NAME, ADDRESS,PHONE_NUM FROM EMP
    WHERE NAME=?");
    ps.setString(1, name);
    java.sql.ResultSetrs= ps.executeQuery();
    names.next();
    Employee myEmp = new Employee();
    myEmp.setName(rs.getString(1));
    myEmp.setHomeAddress(rs.getString(2));
    myEmp.setHomePhone(rs.getString(3));
    names.close();
    class Employee {
    String name;
    String homeAddress;
    String homePhone;

    }
  • Code Example: pureQuery
    10
    Employee myEmp= db.queryFirst( "SELECT NAME, ADDRESS, PHONE_NUM FROM EMP WHERE NAME=?", Employee.class, name);
    Even simpler, if we have a method getEmployee with a Java annotation or XML file with SQL for the query:
    Employee myEmp= getEmployee(name);
  • Why Should be the Data Specialists be interested in pureQuery?
    11
  • Motivations of the Data Specialists
    • SQL Performance Tuning
    • Ease of Tuning
    • Autonomy of Developers
    • Predictability of an Optimized Data Access Path
    • Reduction of Costs to satisfy SQL statements
    • Optimized Access Paths
    • Reduction of CPU intensive components of SQL Execution
    • Utilization of Specialty Processors
    • Capacity Planning
    • Hardware Utilization
    • Problem Determination capabilities
    12
  • pureQuery Capabilities
    • Static SQL for Runtime with Dynamic SQL Execution in Development
    • pureQuery can utilize SELECT INTO from a Java application
    • With Client Optimization, Static SQL from existing JDBC with no changes to the Application
    • Homogeneous and Heterogeneous Batching of Statements
    • Statically bound packages are easy to EXPLAIN and monitor for changes in access path
    • pureQuery coupled with IBM Optim Performance Monitoring provides E2E Performance Monitoring and Problem Determination
    • Impact analysis is greatly improved by the static packages and the ability to tie each statement to a method in the application code
    13
  • Static vs. Dynamic SQL Execution
    14
  • Static vs. Dynamic SQL
  • Cost of Prepare
    CPU cost of Short Prepare on DB2 9 for z/OS – between 400µs and 1ms
    CPU cost of Full Prepare on DB2 9 for z/OS – approximately 30 to 50ms. Cost could be much higher and generally increase with complexity.
  • Trial Results
    Estimated CPU Savings
    (In DB2 CPU)
  • How well does it work? – Java applications
    In-house testing shows significant performance improvements
    IRWW – an OLTP workload, Type 4 driver
    Cache hit ratio between 70 and 85%
    23 % improvement in throughput using pureQuery over dynamic JDBC
    15% - 25% reduction on CPU per transaction over dynamic JDBC
    18
  • How well does it work? - .Net applications
    Throughput during static execution increased by 159% over dynamic SQL execution assuming a 79% statement cache hit ratio
    • IRWW – OLTP application
    • Application accesses DB2 for z/OS
    *Any performance data contained in this document were determined in various controlled laboratory environments and are for reference purposes only. Customers should not adapt these performance numbers to their own environments as system performance standards. The results that may be obtained in other operating environments may vary significantly. Users of this document should verify the applicable data for their specific environment.
    19
  • Batching SQL Statements
    20
  • Homogeneous & Heterogeneous Batch
    Homogeneous Batch – all instances in the batch are the same statement and require only 1 line turn
    Heterogeneous Batch – allows different SQL statements to be included in batch.
    Both Utilize Multi-Row Insert
    Heterogeneous Batches may contain 0 to many Homogeneous Batches
  • Heterogeneous Batch Example
    publicint[][] insertAgentAndPolicies(AgentTO agentTO, PolicyTO[] policyTO) {
    Data data = DataFactory.getData(ds, pdqProps);
    try {
    AgentData agentData = DataFactory.getData(AgentData.class, data);
    PolicyData policyData = DataFactory.getData(PolicyData.class, data);data.startBatch(HeterogeneousBatchKind.heterogeneousModify__);
    agentData.insertAgent(agentTO);
    policyData.insertPolicies(policyTO);
    returndata.endBatch();
    } finally {
    data.close();
    }
    }
  • Client Optimization
    Allows you to bind static SQL packages from existing JDBC code
    Avoids the cost of rewriting the application to code to the pureQuery API
    Allows Heterogeneous batch with minor changes to the code
    None of the productivity advantages are realized. Code is still maintained in JDBC.
    End-to-End monitoring lacks some introspective capability into the coding
    Creation of the static packages requires that you run the code.
    Some overhead at runtime related to resolution of statements to static packages
  • 24
    Optimize Existing JDBC Applications
    Improve performance for DB2 – without changing a line of code
    Capture Configure Bind Execute
    pureQuery client optimization enables static execution for JDBC applications (custom-developed, framework-based, or packaged)
    Existing JDBC Application
    Captured SQL-
    related
    metadata
    JDBC Driver w/ pureQuery
    Dynamic SQL execution
    Static SQL execution
    DB2 Data Servers
    "The ability to use static SQL with pureQuery is huge. Recently, I worked with a client who could reduce CPU usage by 7 percent thanks to this one feature."
    — David Beulke, Pragmatic Solutions Inc.
  • Design
    Develop
    Optimize
    Govern
    Models
    Policies
    Metadata
    Deploy
    Operate
    IBM Optim pureQuery
    Reduce costs
    Increase system throughput
    Improve developer productivity
    Move workload to zIIP and zAAP
    Improve quality of service for new and existing Java applications
    Improve performance
    Lock in access plans
    Speed up problem resolution
    Reduce development time for new Java applications
    Bridge Java and data
    Balance productivity and control
    Enhance developer and DBA collaboration
    Enhance security
    Limit user access
    Minimize SQL injection risk
    Improve audit readiness
    Developer
    Develop
    Code
    Debug
    Test
    Tune, Package
    Tester
  • Why should DBAs care ?
    DBAs have little to no visibility of application SQL before deployment, no opportunity for review and optimization
    Problem isolation takes days with contemporary environments such as Java, PHP, .NET, etc due to inability to trace SQL to Java application and source code
    Constantly increasing Java application workload taxes existing systems – need to fit more work into existing systems
    SQL injection represents an increasing risk to data security
  • Why should Developers care ?
    Get data access right the first time !
    Get it done faster - Improved productivity
    Single environment that spans Java application and database development
    Improved problem isolation and resolution
  • 28
    Control performance
    Decide at deployment time how the SQL is executed
    Understand and lock down the access plan for SQL
    Replace suboptimal SQL without changing the application
    Control security
    Prevent SQL injection
    Prevent execution of unauthorized SQL
    Better manage database security
    See inside applications that are driving your database
    Understand where SQL comes from
    Understand when frameworks and ORM’s are getting in the way
    Simplify problem determination and troubleshooting
    Correlate problem SQL with applications, ORM’s and frameworks
    Optim pureQuery Runtime
  • 29
    How do I start with pureQuery?
    Existing applications
    Optimize existing JDBC (and .NET!) applications
    No code changes needed
    Have to go through the client optimization process to get to static SQL
    New applications
    Use the pureQuery API
    Development codes using one API regardless of whether it is deployed dynamically or statically
    DBA deploys statically
    No need to go through client optimization process
    Other
    JPA, iBatis, Hibernate
  • 30
    pureQuery Facilitates Best Practices
    Supports both inline SQL and Java annotations (method)
    Intuitive interfaces for common data retrieval and manipulation scenarios hides JDBC complexity
    Query First
    Homogeneous Batch
    Reduce network trips to the database
    Query Over Java Collections
    Heterogeneous Batch
    Use custom result handlers to map results to POJO’s, XML, JSON, …
    Write high performance Java data access applications, Part 3:
    Data Studio pureQuery API best practices
    -- VitorRodrigues
    http://www.ibm.com/developerworks/db2/library/techarticle/dm-808rodrigues/?S_TACT=105AGX01&S_CMP=LP
  • 31
    A TypicalApplication Architecture with pureQuery
    Presentation Layer
    Implements the U/I or network protocols using the business services
    Business Service Layer
    Never use the pureQuery API directly. Gets data from the Data Access Layer
    Data Access Layer
    Using the pure-query API to access the database.
    Provides a technology neutral API to the data used by the business services
    pureQuery
    pureQuery makes this layer easy, fast, consistent and traceable
    Database
    Sometimes additional layers are required
    Workflow
    Data federation
    Workspace
  • RAD or RSA / Optim Development Studio Data Centric Development Scenario
    Write in JavaUsing RAD+WAS FP for Web 2.0
    Write in Java with pureQueryUsing Optim Dev. Studio in RAD
    Access generated Java data objects from code developed in RAD
    WAS Feature Pack for Web 2.0
  • Demo
    33
    Demo!
  • 34
    Conclusion: pureQuery Revolutionary Advantages
    Excellent performance
    Static and dynamic SQL is captured during test and optimized before deployment
    Enables lock-in of access path
    Great productivity
    Excellent tool support through Optim Development Studio
    Shell share with Rational tools
    Mapping from SQL to Java captured and traceable
    Facilitates collaboration between DBA’s and developers
    Performance tuning, impact analysis
    Better security
    Limits SQL injection
    Controlled database access
  • Where to go Next? Resources and more…
    Optim Development Studio
    http://www.ibm.com/software/data/optim/development-studio/
    IBM pureQuery
    http://www.ibm.com/software/data/optim/purequery-platform/faq.html
    pureQuery Custom Training
    InferData, IBM Business Partner http://www.inferdata.com
    Course: Developing Database Applications with Optim Development Studio and pureQuery
    http://www.inferdata.com/training/data/optim_purequery_training.html
    35
  • Web, Blogs
    Integrated Data Management (Optim and Data Studio)
    http://www.ibm.com/developerworks/spaces/optim
    Vladimir’s Blog: On Building Software
    http://www.OnBuildingSoftware.com
    Twitter:
    http://twitter.com/OnSoftware
    36
  • 37
    Thank You!Your Feedback is Important to Us
    Please complete the survey for this session by:
    Accessing the SmartSite on your smart phone or computer at: iodsmartsite.com
    Surveys / My Session Evaluations
    Visiting any onsite event kiosk
    Surveys / My Session Evaluations
    Each completed survey increases your chance to win an Apple iPod Touch with daily drawling sponsored by Alliance Tech
    37