Your SlideShare is downloading. ×
Dynamic Statement Cache
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Dynamic Statement Cache


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • We’ll begin this presentation with some basic training on SQL tuning fundamentals. Once we’ve differentiated between static and dynamic SQL we’ll drill into more detail about dynamic. This will lead to a discussion about DB2 dynamic statement caching. Finally, we’ll discuss how you can make use of the data that’s stored in the Global Dynamic Statement Cache.
  • Where should you spend most of your time tuning in DB2. Where’s the biggest bang for the buck that can justify the cost of your tools and time. Most experts agree that a large percentage of your problems will be SQL related. So, a good round number is that 70% of your tuning time will be focused in tuning SQL in your shop. Another 20% is related to object definition. And finally, 10% is related to subsystem or z/OS issues. These numbers are just guidelines and you’ll find the actual time you spend doing these task varies significantly. If your shop is doing 95% static SQL with stable applications then you need for tuning SQL will be less than this guideline.
  • We have two ways of looking at SQL. First, we can ponder the actual statement text to make sure the SQL meets our standards for best practices and we can make a prediction about performance. This should be a common practice in your shop especially if your applications are developed internally and are mainly static in nature. If you run a large dynamic workload or are using a purchased application you may little control over the SQL. Still it is important to understand your application SQL. A second important way to evaluate SQL performance is by looking at the actual workload metrics associated with an application. This can help understand the dynamics of the application. For example, when I run application components concurrently do locking problems develop. Also, this is the best place to look at what’s happening with dynamic SQL, which may very unpredictable.
  • Whether static or dynamic, the optimization process is fundamentally the same. The Optimizer takes a look at a number of inputs from external sources to determine the optimum way to satisfy the application request. The inputs include the statement text, DB2 statistics information about the objects, object definitions themselves, DB2 configuration (buffer pools for example) and even information about the z/OS environment. Things like number of engines available on your hardware platform can impact things like query parallelism. So the optimization process is pretty consistent. All this leads to an access path. There are other factors as well that are looked at during this process, whether during a static bind or during statement execution for dynamic. The other factors include authorizations, for example. The next chart looks at the characteristics of dynamic and static SQL.
  • This slide provides an initial comparison of dynamic and static SQL. We’ll build on this information as we continue through the presentation. With static SQL, the access path is determined at bind time in most cases. A typical development approach includes the precompile, compile, and bind steps. The access path to the data is hardwired into application structures. The access path typically won’t change on the bind process is complete unless a developer or DBA needs to rebind the plan or package for some reason. There are some exceptions to the rule about when access path selection is done for static SQL. The REOPT(ALWAYS) bind parameter, for example, causes static SQL statements with embedded host variables to go through access path selection at run time. This is definitely an exception to the rule. For dynamic SQL, access path selection occurs at statement execution. The operation is called a prepare. A prepare can be initiated by the PREPARE SQL statement. In other cases prepares happen as a result of another operation, an EXECUTE IMMEDIATE for example. So, each time the dynamic statement runs initially in a unit of work this prepare process occurs. DB2 has implemented some specific capabilities that can help reduce the cost of the prepare operation. One of these options is the KEEPDYNAMIC bind option. Ad second option is the Global Dynamic Statement Cache. These options can allow the results of the prepare operation to persist across units-of-work within a thread and across multiple threads. The other bullets on this chart highlight some of the other differences between static and dynamic, like authorization checking.
  • Initially, static SQL was by far the most common form of SQL. Dynamic was less than 10% of the SQL in most shops. This has changed over time. More and more dynamic SQL is now being developed. This change has occurred for a number of reasons. ERP vendors like SAP, PeopleSoft, and Siebel developed their applications using dynamic SQL, largely because dynamic is often more flexible in handling complex SQL. Execution platforms are often running on Windows with application servers on Unix, and backend database servers running on the mainframe. For a long time, the only support available for this environment was dynamic SQL. This has changed a bit now with SQLJ, an option for developing static SQL with Java.
  • This chart provides some basics on static SQL. In this example we’re defining a cursor for a statement doing a two table join. A host variable is embedded in the statement. The host variable will be populated at run time based on external input. The cursor is defined, then opened, next the individual rows in the result set are fetched and finally the cursor is closed. There’s a lot more going on in the application. This is just meant to provide a quick example for a discussion point.
  • Now we move on to discuss dynamic SQL. In this example, we have an SQL statement defined in the working storage of a COBOL program. With dynamic, the statement is build on the fly and the SQL PREPARE is executed. You can easily see the SQL text in the program. With dynamic SQL, host variables aren’t used; however, parameter markers provide the same functionality. In this example we also have a literal coded in the statement. Using literals is not a good idea because it limits the benefit of DB2 features meant to improve performance. More on that later.
  • So why would someone want to use dynamic SQL in an application. Here is a live working example from a major ERP application. In this case, the input panel allows for many different combinations of search options to be specified by the user. If you considered 10 different search criteria you could be talking about a large number of combinations of search arguments. To code this into static might require 100+ cursor definitions to accommodate the possible combination of key values. With dynamic you could construct a statement like you see above and could more easily accommodate the complexity. This is one of the classic uses for dynamic SQL.
  • There are additional considerations for dynamic SQL that you need to be aware of. Dynamic SQL is sensitive to things like DB2 statistics changes. Changes in statistics could cause an SQL statement to change access paths dramatically because DB2 looks at statistics each time the statement runs as part of access path selection. This can make application response times unpredictable Security is more complicated with dynamic. With static and end-user can be granted authorization at the plan or package level doesn’t require authorization to the individual tables being accessed. With dynamic that’s not the case. There are ways to mitigate this issue but it can still be a problem. Because access paths are determined at run time it’s possible that a dynamic SQL statement’s performance can fluctuate wildly, going from an efficient access path to an extremely expensive one. It might be necessary to make use of DB2’s governor capability to limit the potential for runaway threads. Finally, access path analysis is a big part of the tuning efforts. With dynamic we can’t evaluate access paths that haven’t been determined yet.
  • Just a little more on the prepare operation. This get more into the biggest issue with dynamic and that’s the cost of the repeated prepares that must be done. By default, a prepared statement is not persistent across UOWs. So, reducing or eliminating the cost of prepares was a key requirement from ERP vendor as well as mainstream DB2 customers developing new applications. IBM introduced Dynamic Statement Caching to address this issue.
  • The goal of the dynamic statement caching is too reduce or eliminate SQL prepare operations. There are 4 kinds of caching that can be considered. The first one is really not to do anything. Many customers may not be able to make use of dynamic statement caching in any event so this is their best option. The other 3 options provide varying degrees of relief from the high overhead of repeated prepare operations. With dynamic statement caching the prepared statement and/or the statement text are cached in the DBM1 address space and are available for reuse. Implementation of the caching is turned on by a combination of Bind options, DSNZPARMs and application coding techniques.
  • In this example we’re showing the do nothing option. In this example (functioning PL/1 program), the program issues a PREPARE statement. The prepare operation completes and the prepared statement is stored in local storage allocated for the thread in the DBM1 address space. Subsequently, the application issues two EXECUTES of that prepared statement. Finally, the application commits. At this time, the prepared statement (V1) is discarded. If the application must subsequently make the same call the PREPARE will need to be executed again. If the PREPARE is not done DB2 will return a -514 or -518 to the application.
  • Local statement caching by itself doesn’t have a lot of value. It does help reduce message tracking for distributed connections. Local statement caching is enabled by using the KEEPDYNAMIC(YES) bind parameter. The MAXKEEPD DSNZPARM defines the total number of prepared statements that can reside in local thread storage. With this scenario it is important to understand that DB2 keeps the prepared statement (access path information) as well as the SQL statement text itself. In this example, the RRS01 program issues the PREPARE (full prepare) and then two EXECUTEs, as before. When the program commits, the prepared statement is discarded; however, the statement text stays around. With local statement caching, the application can then do an EXECUTE immediately without first issuing a PREPARE. In this case, DB2 does the prepare on behalf of the SQL statement using the retained statement text. This is a called an IMPLICIT PREPARE but is still a full prepare so there’s no real performance benefit.
  • With Global Statement Caching only, things get much better. In this example RRS01 issues a PRERPARE for STMT2. The prepared statement ends up in RRS01’ s local thread storage; however, the prepared statement is also copied into the Global Statement Cache. Now RRS01 terminates and an new thread starts (Program RRS02). This program wants to execute the exact same statement. In this example, DB2 finds the prepared statement in the Global Statement Cache and it is copied directly into RRS02’s local thread storage. This is called a Short Prepare. The structured copied into the Global Statement Cache is called a Skeleton Dynamic Statement (SKDS) Use of the cache is driven by the CACHEDYN=YES DSNPARM.
  • When can a statement be reused? As show on the chart, the statement text must be 100% the same. This is the reason why literals are not a good idea and DB2 sees them as two unique statements is they use different literals. If they use parameter markers instead then there’s not problem. Other characteristics of the prepared statement must be the same or compatible as well. It’s important to evaluate the potential benefit of the statement cache. ERP vendors have gone to great lengths to make sure they can take full advantage of this capability. You need to make sure you can get the benefit with your dynamic SQL applications.
  • The final flavor of dynamic statement caching is called Full Caching. In the best case this eliminates the prepare (Prepare Avoidance) altogether. In the example, DB2 keeps the prepared statement in the local storage for the thread. The prepared statement is available across multiple UOWs with a single program execution. Full caching is enabled using the CACHEDYN=YES, MAXKEEPD>0, and KEEPDYNAMIC(YES) bind parameter.
  • This slide should give you an idea of the significance of avoiding the full prepare. So, if it cost 100 to do a full prepare, it will cost 1 if the prepared statement is in the global cache. If the prepared statement is still in the local cache then the prepare is completed avoided, and the cost is 0. The other boxes show some measurements about actual cost of a full prepare vs. a short prepare. You can see this is a major performance benefit.
  • Let’s talk about storage usage briefly. Since a lot of you are still on V7, this may still be of interest. In V7 DB2 DBDs, Skeleton Package (SKPT) and Cursor Tables (SKCT), Skeleton Dynamic Statements (SKDS), and Package Tables and Cursor Tables (PT/CT) in the DBM1 address space. Virtual storage constraint is a big problem in V7, so DB2 also allowed you to place the Dynamic Statement Cache in a dataspace.
  • With v8 and 64 bit addressing, things have changed dramatically. The EDMPOOL in now three different pools. The DBDPOOL and the Global Statement Cache are above the 2GB bar while the SKPT, SKCT, CT, and PT all reside below the Bar. There is no dataspace option in DB2 V8
  • With v8 and 64 bit addressing, things have changed dramatically. The EDMPOOL in now three different pools. The DBDPOOL and the Global Statement Cache are above the 2GB bar while the SKPT, SKCT, CT, and PT all reside below the Bar. There is no dataspace option in DB2 V8
  • This slide shows metrics that indicate the relative effectiveness of the Global and Local statement caches. Hopefully this rings a bell right now. We’re shooting for lots of the good things and none of the bad things. So we want high numbers for things like Global Cache hit Ratio and small numbers for things like Statement Pool Full. In this bottom section you see the Local Cache Effectiveness metrics. You can see the indicators for Avoided Prepares, Implicit Prepares, etc.
  • Now we begin to take a much deeper look at the Global Dynamic Statement Cache. The report you see on this screen shows a list of the statements that are currently in the cache or a least one screen’s worth. Dynamic statements are eligible for caching if CACHEDYN=YES. REOPT(ALWAYS) or (VARS) are not eligible. The graphic shows a list of the statements in the cache. Notice the Data and Time Cached columns. This can give you an idea about how long statements stay in the cache. Statements are removed from the cache when one of the items listed occurs. Of course, when you bring down DB2 the entire cache disappears.
  • This slide discusses technique for collecting data about the cache or from the cache. First, statistics about cache usage is available with most monitors in the market today. Slide 23 shows you some detail about how well your dynamic statement caching is working in your environment. IFCIDs 316 and 317 provide information about what’s going on in the cache. These IFCIDs are accessed by IFI applications using READS (Read Synchronous). IFCID 316 provide details on the contents of the cache for a specific statement. Only the first 60 bytes of the statement text is returned. IFC 317 will return the entire text of the statement. A key way to externalize data stored in the global statement cache is using the EXPLAIN STMTCACHE statement. This provides multiple options for getting statement data out of the cache and into DB2 tables for further examination.
  • This data is returned from the READS for a 316 IFCID. The It is formatted for viewing. As you can see we are getting the SQL statement text, BIND options, and statement statistics if they are being collected for the statement. This content tells you all there is to know about the statement except the access path currently in use for the statement. The only way to get this information is using the EXPLAIN STMTCACHE statement. That will be discussed later. The statistical information you see here is cumulative and will be discussed further in a subsequent slide.
  • This slide discusses using the EXPLAIN STMTCACHE ALL statement to externalize the contents of the dynamic statement cache. As mentioned DB2 puts a single row for each statement in the cache into the Dynamic Statement Cache Table. This table is very similar to what you see in the 316 records although it does have a few additional fields. Notice that fields like the COLLID are set to literal values like “DSNDYNAMICSQLCACHE”. This provides a good eye-catcher. Also, the STMT_ID is tied to the UNIQID in the dynamic statement cache. One thing to be aware of is the STMT_TEXT column, a 2M CLOB that could be hard to display. This variation of the EXPLAIN STMTCACHE statement only populates your Dynamic Statement Cache table. This makes sense since you could be talking about hundreds of statement. The lower bubble on this slide is meant to string together the different columns in the table. There is a lot of information to look at. Much of the data is statistics so if you aren’t collecting statistics you can ignore that.
  • If you want details about a specific statement in the cache, you can use the EXPLAIN STMTCACHE STMT_ID statement. This will return the contents of the global statement cache into three (or four) different tables. In addition to the DSN_STATEMENT_CACHE table, this also populates the PLAN, DSN_STATEMENT, and DSN_FUNCTION (if appropriate). Access path information is put into the PLAN_TABLE but this reflects that actual access path of the statement in the cache. This graphic shows the EXPLAIN command and the resulting rows generated. The join columns are highlighted in red. Be aware that this function could create duplicate entries in the PLAN_TABLE because the QUERYNO column could already exist in the table. If the statement requested doesn’t exist in the statement cache a -248 is returned to the application.
  • This variation uses the contents of the statement token to collect data from the cache. The graphic show the statement executed and shows the results returned. Note that this form of the EXPLAIN STMTCACHE returns rows to the PLAN_TABLE and the DSN_STATEMENT_TABLE as well. This variation would allow you to extract all statement for a specific application where you set the STMT_TOKEN to a value that uniquely identified your application.
  • This slide provides a bit more detail about the STMT_TOKEN. For some threads, it’s difficult to discern much information about the actual user, application, workstation, and such. In DB2 V8, IBM added special registers to help you identify CLIENT level information. These special registers are accessible from an application program but setting them is a bit of a challenge. This is also the case with the STMT_TOKEN. The documentation is a bit confused so you may need to play a bit to get this working for you. These values are set using either RRSAF (Resource Recovery Services Attachment Facility) functions or sqleseti for distributed threads. The graphic here shows an example of setting the values using RRSAF function. This is a working PL/1 program used to test setting these values. The first box shows using the SET_ID function to pass the STMT_TOKEN. The second box shows using the SET_CLIENT_ID function to pass the contents for the CLIENT special registers. We could probably spend a whole hour (or longer) talking about RRSAF so we’ll save that for another time.
  • We’ve seen these execution statistics before. Now let’s delve into a little more detail. DB2 allows you to collect statistics about the dynamic statements in the cache. This is done by turning on a monitor trace for IFCID 318. With this turned on DB2 begins accumulating statistics you see on the graphic in the lower right hand corner. Statistics are collected the length of time the trace is on. When the trace is turned off all the statistics are reset. There is an overhead of about 2-4% associated with this collection. So, a good approach might be: Turn on statistics collection ‘START TRACE(MON) IFCID(318)’ Let the trace run for a period time that meets your requirements Perform the EXPLAIN STMTCACHE variety that gives the required results. All the flavors externalize the statistics. Turn off statistics collection ‘STOP TRACE(MON) TNO(NN)’ where NN is the trace number assigned by DB2 in step 1 above. All statistics are reset at this point.
  • The topic of static and dynamic SQL along with dynamic statement caching is complex and there is information in many places. I found the two Redbooks mentioned above very helpful, although ‘Squeezing the Most out of Dynamic SQL’ has not been updated for DB2 V8. IBM manuals provide good technical information. And finally, a Google search on DB2 Dynamic Statement Cache will return numerous hits that can provide additional insight.
  • As previously mentioned, this is a complex topic. If you are considering more dynamic SQL in your shop it’s very important to understand how statement caching works so you can take full advantage of this exciting DB2 feature. Be aware that DB2 9 will have a number of changes in this area, especially as it applies to statistics collection. You’re sure to hear more about this during this conference.
  • Transcript

    • 1. Demystifying the DB2 Dynamic Statement Cache
    • 2. What Will We Talk About?
      • Some SQL Tuning Fundamentals
      • Dynamic SQL in More Detail
      • Introduction to DB2 Statement Caching
      • Mining for Gold in the Global Statement Cache
    • 3. DB2 Tuning - Where Should You Spend Your Time
      • What Can I tune in DB2
      • Where are the biggest problems
        • Purely an estimate and your experience may vary
        • Many tuning efforts combine multiple areas
          • Especially true of SQL and Object Analysis
      SQL 70% Object Analysis 20% Subsystem 10%
    • 4.
      • Focus on individual SQL statements
        • Do they meet “best practice” coding standards
        • Do they use expected/accepted DB2 access paths
        • Do they deliver desired result set in acceptable time with acceptable resource consumption
      • Developed and tested in controlled environment
      • More predictive in nature
      • Focus on workload dynamics
        • How does concurrent execution affect response time/resource consumption
        • Does this SQL statement/program collide with other transactions
        • Same application
          • Other applications in a shared subsystem
      • Real world un predictability comes into play
      • More focus on measuring the workload and rapidly reacting
      Solving the Problem SQL Analysis Across the Application Life Cycle SQL Workload Atomic SQL                                                                                                                                                                                    
    • 5. SQL Tuning Fundamentals DB2 Optimizer Determines SQL Performance SQL Statement Text OPTIMIZER DB2 Configuration Tablespace DB04.TS1 Index App1.Index1 Catalog Statistics Hardware Configuration Schema Definitions Access Path To the Data TABLE
    • 6. SQL Tuning Fundamentals Access Path Selection
      • Static SQL
      • Access path determined at bind time – better performance
        • Exceptions to the rule
          • REOPT (VARS) or (ALWAYS)
            • Access path determined at run time for those statements with host variables or parameter markers
          • PREPARE(DEFER)
            • Option useful in distributed environments for reducing message traffic
      • Authorization for execution at the plan/package level
      • Qualifiers passed via host variables
      • SQLJ provides for bound static SQL in Java applications
      • For Dynamic SQL
      • Access Path Selection determined at execution
        • That’s the PREPARE
        • Exceptions to the Rule
          • KEEPDYNAMIC bind option
            • Holds prepared statements across commits to avoid cost of re-preparing statement
          • Global Dynamic Statement Cache
            • Maintains Skeleton of prepared statements
      • Build and execute SQL on the fly
      • User requires authorization to all accessed objects
      • Parameter markers for passing variables
      SQL Statement Text OPTIMIZER DB2 Configuration Tablespace DB04.TS1 Index App1.Index1 Catalog Statistics TABLE Hardware Configuration Schema Definitions Access Path To the Data
    • 7. Trends in the Marketplace Static vs. Dynamic SQL
      • Dynamic SQL usage is on the increase
      • What’s driving it?
        • Dynamic SQL offers flexibility that can simplify developing complex applications
        • New applications being developed on distributed platforms using connections that only support dynamic SQL
          • DB2 CONNECT, etc.
        • ERP applications implemented with dynamic SQL
          • SAP, PeopleSoft, Siebel
        • New applications being developed on distributed platforms
          • New developers are much more familiar with GUI-based programming environments and don’t even sign on to the mainframe
            • More Java and C++
    • 8. SQL Fundamentals - Static SQL
      • Data access requirements well defined and predictable
      • Static SQL cursor constructs
        • Close the cursor
        • Open the cursor
        • Fetch the rows from the result set
      Host variable defined In working storage
        • Define the Cursor
    • 9. SQL Fundamentals - Dynamic SQL
      • Data access requirements are ad hoc in nature and identified on the fly
        • SELECT Operations
      Parameter marker provides placeholder for later substitution
      • Other operations
        • Cause the INSERT statement to be prepared and executed immediately
      Notice the literal
    • 10. SQL Fundamentals - Dynamic SQL In Practice
      • A Statement from a major ERP application
      • Built on the fly based on search criteria selected
      • A complex statement with unpredictable input
        • Default statement syntax includes minimal number of search criteria
        • More search criteria the statement expands to include those search arguments
        • If using static SQL could require over 100 cursor definitions in the program
    • 11. Dynamic SQL Operational Considerations
      • Sensitive to DB2 statistics
        • Dynamic SQL always uses current catalog statistics for access path selection
          • Changes in DB2 statistics can cause unpredictable changes in access paths
        • Some DB2 customers collect catalog statistics to drive maintenance processes
          • May cause SQL performance to fluctuate unexpectedly
      • Security is generally more complex with dynamic SQL
        • Application users generally require authorization to the objects being accessed
        • Auditing is also affected because statements are developed on the fly
      • Governor capability may be required
        • Performance characteristics can vary widely for dynamic
        • DB2 Resource Limit Facility may be required
      • Access path analysis difficult because access path is not available prior to execution
    • 12. Dynamic SQL Considerations PREPARE Yourself
      • Repeated PREPAREs drive up the cost of dynamic SQL
        • Prepared statements by default are not persistent across UOWs
        • Prepare costs vary widely but are significant
      • Key requirement from anyone developing dynamic SQL applications to reduce or eliminate the cost of preparing dynamic SQL statements
        • Driven initially by SAP and other ERP vendors
        • More in-house dynamic SQL applications drive this requirement
      • Enter Dynamic Statement Caching
    • 13. Introduction to Dynamic Statement Caching
      • Goal is to reduce or eliminate SQL Prepare operations required for dynamic SQL statements
      • Implementation
        • Four kinds of caching
          • No caching
          • Local Dynamic Statement Caching
          • Global Dynamic Statement Caching
          • Full Caching
        • Cache prepared SQL statement and statement text for dynamic SQL statements in DBM1address space
          • Local Statement Cache
          • Global Dynamic Statement Cache
        • Controlled by various parameters
          • Bind options
          • DSNZPARMs
          • Application constructs
    • 14. Dynamic Statement Caching No Statement Caching
      • Prepared statements do not persist across commits
        • Discarded at commit
        • Except for statements defined with CURSOR for HOLD
      • Default mode of operation
      Program RRS01 Thread Storage Prepared Statement STMT2(Version 1)
      • No prepare returns -514 or -518
      Full Prepare Used Discarded Used Prepared Statement STMT2(Version 2) Full Prepare
    • 15. Dynamic Statement Caching With Local Statement Caching Only
      • Eliminates need for application to do multiple prepares for same statement
        • Implicit prepares done by DB2
      • Enabling Local Statement Caching
        • KEEPDYNAMIC(YES) Bind Parameter
        • MAXKEEPD DSNZPARM controls maximum prepared statements
          • Does not affect statement text which is always kept
      • Differentiation between prepared statement and statement text
      • Minimal benefit if used alone
        • Some reduction in message traffic in a distributed environment is possible
      Statement Text Retained Program RRS01 Thread Storage Discarded Implicit Prepare Prepared Statement STMT2(Version 2) Used Full Prepare Prepared Statement STMT2(Version 1)
    • 16. Dynamic Statement Caching Global Statement Caching Only
      • Allows reuse of prepared statements across UOWs
        • Within and across program executions
        • Prepared statement (SKDS) cached in global dynamic statement cache
          • Copied into local storage when possible
          • Short Prepare
      • Enabling global statement caching
        • Storage allocation discussed later
      • Big benefit for applications with frequent reuse of dynamic SQL
        • Benefits with no coding changes required
      Global Statement Cache SKDS SKDS Program RRS01 RRS01 Local Thread Storage Program RRS01 RRS01 Local Thread Storage Prepared Statement STMT2(Version 1) Full Prepare SKDS Prepared Statement STMT2(Version 1) Short Prepare
    • 17. Dynamic Statement Caching Where Cached Statements can be Reused
      • Statement text must be 100% the same
        • Use parameter markers
        • Literals won’t work (usually)
      • Additional items must be 100% the same or compatible
        • Bind rules
        • Special registers
        • Authorizations
        • Others
      • You may not get any benefit out of the dynamic statement cache at all
        • Most likely to benefit if you using an ERP or some other application that uses dynamic SQL extensively
    • 18. Dynamic Statement Caching Full Caching – A Final Flavor
      • Combines benefits of local and global statement caching
        • Ability to completely avoid prepare operations
        • Prepared statement kept in local thread storage and not invalidated across commits
          • Prepare Avoidance
      • Enabling global statement caching
      • Maximum benefit within an application execution
        • Local thread storage is discarded at thread termination
      Program RRS01 RRS01 Local Thread Storage Prepared Statement STMT2(Version 1) Full Prepare Prepare Avoided
    • 19. Dynamic Statement Caching Cost Impacts
      • Full Prepare
        • Statement not in cache
        • Global statement caching not active
      • Short Prepare
        • Dynamic statement (SKDS) in the global cache
        • Global caching active
      • Avoided Prepare
        • Local and global caching active
      Relative Cost 0 1 100 DB2 Execution Metrics
    • 20. Dynamic Statement Caching Impacts on Storage EDM Pool in DB2 V7
      • Caches access path & internal structure definitions
      • This pool contains
        • DBDs – database descriptors
        • Skeleton Package and Cursor Tables (SKPT & SKCT)
        • Package and Cursor Tables – (PT/CT)
        • Authorization cache block for each plan (optional)
        • SKDS - Skeletons of dynamic SQL for CACHE DYNAMIC SQL (optional)
          • Optionally stored in a dataspace
        • Trigger Packages
    • 21. Dynamic Statement Caching Impacts on Storage EDM Pool In DB2 V8
      • EDMPOOL now in 3 separate pools
        • EDMDBDC – DBDs
          • Above the Bar
        • EDMSTMTC – Dynamic Statements
          • Above the Bar
        • EDMPOOL – Skeleton Package and Cursor Tables
          • Still below the bar and a potential source of VSC
      • No dataspace option for Dynamic Statement Cache
    • 22. Dynamic Statement Caching Impacts on Storage EDM Pool In DB2 V9
      • Portions of runtime Components moved above the bar
        • Plan and package skeletons above the bar
        • Bound/Prepared DML Statements
          • Statement Text
          • SQLDA DESCRIBE output
          • Portion of native SQL PL package
        • Portions of static SQL sections (CT/PT) are moved as well
      • Further reduces VSC in the DBM1 address space
    • 23. Dynamic SQL Statement Caching DB2 Cache Statistics Global Cache Hit Ratio Shoot for 80+% Statement Pool Full Failures Should be 0 Increase Statement Pool Size if not Statement Discarded Shoot for 0 Increase MAXKEEPD Local Cache Hit Ratio Specific for Applications bound with KEEPDYNAMIC(YES)
    • 24. The Global Dynamic Statement Cache What Goes In?
      • Dynamic Statements
        • If the Global Cache is active (CACHEDYN=YES) and not a REOPT(ALWAYS) application
        • Reside in the till they are thrown out
          • DROP or ALTER
          • Authorization Revoked
          • LRU
          • RUNSTATS
          • DB2 is recycled
    • 25. Retrieving Data From the Global Cache
      • As shown previously
        • Statement caching performance data in DB2 statistics records
        • Metrics show details about cache hit ratios and other useful data points that help you evaluate overall performance of your statement caches
      • For more detail on Global Statement Cache usage the following instrumentation is provided
        • IFCID 316 – Provides details on statements in the cache
          • First 60 bytes of SQL text
          • Includes execution statistics (0 if not being collected)
        • IFCID 317 can then be used to retrieve the entire SQL statement from the cache once you have identified the statement of interest
        • V8 feature that exports Dynamic Statement Cache information to the DSN_STATEMENT_CACHE_TABLE
        • Nearly identical to the detail in IFCID 316 & 317
        • Multiple options including ALL, stmt-id, and stmt-token
    • 26. Reviewing Global Statement Cache Information IFCID 316 Results
      • First 60 Bytes of SQL Text
        • IFCID 317 gives full text
      • Bind Options
      • Statement Statistics (more later)
    • 27. Mining the Dynamic Statement Cache EXPLAIN STMTCACHE ALL
      • Extracts all statements from the global cache
      • Inserts one row for each entry in the global DSC
        • Populates DSN_STATEMNT_CACHE_TABLE only
        • STMT_ID column matches the Unique ID in the global statement cache
        • Nearly exact match to the DSC with a few additional columns
        • STMT_TEXT is a 2M CLOB so be careful with that
    • 28. Mining the Dynamic Statement Cache EXPLAIN STMTCACHE STMT_ID
      • Extracts a single statement from the global DSC
        • Populates PLAN, DSN_DYNAMIC_STATEMNT, DSN_STATEMENT, and DSN_FUNCTION tables if they exist
        • Access path is current access path for statement in the cache
        • Numeric literal or host variable from program
        • -248 SQL Return Code back to program is STMT_ID not found
    • 29. Mining the Dynamic Statement Cache EXPLAIN STMTCACHE STMTTOKEN
      • Extracts a group of statements from the global DSC
        • Populates PLAN, DSN_DYNAMIC_STATEMNT, DSN_STATEMENT, and DSN_FUNCTION tables if they exist
        • Access path is current access path for statement in the cache
        • Based on STMT_TOKEN value in the cache
        • Alphanumeric literal or host variable in program
        • -248 SQL Return Code returned if no qualifying entries found in cache
    • 30. Mining the Dynamic Statement Cache More on the STMT_TOKEN in the Cache
      • Provides a method for grouping similar SQL statements
      • STMTTOKEN values set using RRSAF or sqleseti functions
      • Similar to Client special registers implemented in DB2 v8
      • PL/1 RRSAF Example
      • Set STMTTOKEN Value
      • Call to DSNRLI (RRSAF) with SET_ID function
      • Error handling follows
      • Set Client Special Registers
      • Call to DSNRLI (RRSAF) with SET_CLIENT_ID function
      • Error handling follows
    • 31. Reviewing Global Statement Cache Information IFCID 318
      • Execution statistics for dynamic SQL statements
      • Turn on collection with Monitor trace IFCID 318
        • Begins collecting statistics and accumulates them for the length of time the monitor trace is on
        • Stop Monitor trace resets all statistics
        • 2-4% overhead per dynamic SQL statement stored in the cache
      • Recommended approach
        • Run the trace only when actively monitoring the cache
      • Use EXPLAIN STMTCACHE to externalize data for evaluation
    • 32. Acknowledgements
      • There are numerous documents that discuss SQL in general and dynamic SQL in particular, including:
        • DB2 technical publications
        • Technical articles by numerous DB2 Subject Matter Experts
        • IDUG List Server Archives
      • IBM Redbooks on this topic were especially helpful in researching this presentation, including:
        • DB2 for z/OS and OS/390 : Squeezing the Most Out of Dynamic SQL
        • DB2 UDB for z/OS V8: Through the Looking Glass and What SAP Found There
    • 33. Summary
      • Dynamic SQL is growing in usage
        • ERP Vendors
        • Distributed applications
      • DB2 offers multiple options for reducing the overhead traditionally associated with dynamic SQL
      • These options include multiple types of statement caching
        • Local statement caching
        • Global statement caching
        • Full statement caching
      • DB2 9 will see big changes in the way the SQL statement execution statistics discussed in this session will be used captured and used