Your SlideShare is downloading. ×
  • Like
OUG Harmony 2012 -  Using SQL Plan Baselines for Performance Testing
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

OUG Harmony 2012 - Using SQL Plan Baselines for Performance Testing

  • 222 views
Published

My presentation (OUG Harmony 2012 in Tallin) on how to use SQL Plan baselines to assess the completeness of testing. I also have a white-paper on this topic, see the whitepapers section.

My presentation (OUG Harmony 2012 in Tallin) on how to use SQL Plan baselines to assess the completeness of testing. I also have a white-paper on this topic, see the whitepapers section.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
222
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
9
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Using SQL Baselines for Performance Testing Maris Elsins Oracle Applications DBA 30.05.2012
  • 2. © 2012 Pythian Few words about me... • 10y  Oracle • 3y – PL/SQL Developer • 7y – Oracle [Apps] DBA • Certificates: • 10g OCM, 9i/10g/11g OCP, • 11i Apps DBA OCP, 11i System Administrator OCE • Employed by Pythian since July 2011 • Speaker at conferences: 5* , 3* , 3* • How to find me? • Blog – http://www.pythian.com/news/author/elsins/ • Earlier blog posts – http://appsdbalife.wordpress.com/ • LinkedIn – http://lv.linkedin.com/in/mariselsins • Twitter – @MarisElsins • Email – elsins (at) pythian.com 2
  • 3. © 2012 Pythian Why Companies Trust Pythian Recognized Leader Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server Work with over 150 multinational companies such as Forbes.com, Fox Sports, Nordion and Western Union to help manage their complex IT deployments Expertise One of the world’s largest concentrations of dedicated, full-time DBA expertise. Employ 8 Oracle ACEs/ACE Directors. Hold 7 Specializations under Oracle Platinum Partner program, including Oracle Exadata, Oracle GoldenGate & Oracle RAC. Global Reach & Scalability 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response 3
  • 4. © 2012 Pythian Performance SQL Baselines Testing 4
  • 5. © 2012 Pythian PERFORMANCE 5
  • 6. © 2012 Pythian Performance • Performance is important because users talk a lot about it • Oracle is slow • Application hangs • The report was not ready by 8AM ... again • DBAs don’t usually hear much about «good» performance • How to know if the performance is «good»? • 0 complaints? May be users are tired of reporting problems? • 0 broken keyboards reported by IT Service staff? • DB Server Load Average < 2 ? What really is good or bad performance? Or should we ask if the performance is sufficient? 6
  • 7. © 2012 Pythian Performance • Response time of the application • Jackob Nielsen «Response Time Overview» • 0.1 second - Instant reaction of the system • 1 second - Limit where the flow of though stays uninterrupted • 10 seconds - Users gets the focus on something else • Users can define what response times would be acceptable for the application to consider it performing well • Response time allows to define «Oracle is slow» and «Application hangs» problems • Produced results per unit of time • Why «work / time» is not a good measure? • Results – amount of processed units (orders, rows, etc) • Users can define what amount of results need to be provided and in what time it has to be completed • Results/time allows to define the «The report was not ready by 8AM» problem in detail 7
  • 8. © 2012 Pythian The goal of the Presentation • Ensuring the sufficient performance is not easy • Applications change • Data grow • HW changes • Usage of applications change • It’s a big chellange to make sure the performance does not degrade • Performance testing is complicated • How to simulate production workload • How to make sure everything is tested? • Big issues are easily revealed by testing, what about smaller problems? • Let’s find out how SQL Baselines can help 8
  • 9. © 2012 Pythian SQL BASELINES 9
  • 10. © 2012 Pythian Introduction to «SQL Plan Management» • New feature introduced in 11gR1 • Ensures only verified execution plans are used • Prevents performance regressions caused by execution plan changes • Allows controled introduction of the new execution plans • «Stored outlines on steroids» • How it works? • SPM evaluates and records SQL execution plans over time • SPM creates SQL plan baselines – sets of plans known to perform well • Only approved plans in SQL plan baselines are used for execution • If SQL plan baseline does not exist execution plan provided by the optimizer is used 10
  • 11. © 2012 Pythian Introduction to «SQL Plan Management» • Typical use cases? • Database upgrades to 11g • Prevent SQL execution plan changes due to ongoing system and data changes • Transfer tested Execution plans from DEV DB to PROD • Forcing execution plan for particular SQL without changing the statement • Availability of SPM • Available in Enterprise Edition starting from 11gR1 • SPM does NOT require Diagnostic or Tuning pack licenses • Capturing the SQL Plan Baselines • Automatic - OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES parameter • Manual – use DBMS_SPM to load plans from SQL Tuning Set (STS) or Cursor Cache • STS has more options to load SQLs from, but requires Diag+Tuning licences 11
  • 12. © 2012 Pythian Introduction to «SQL Plan Management» • Evolving the SQL Plan Baselines • Evolving = process to verify the new execution plans • DBMS_SPM.EVOLVE_SQL_PLAN_BASELINE • Execute statement using the best accepted baseline and the questionable plan • Compare resource usage • Accept the new plan if it’s better then the current (threshold 1.43 - 1.76) • SQL Tuning Advisor task that runs during maintenenace window • Load new plans manually • Purging of SQL Plan Baselines • DBMS_SPM. DROP_SQL_PLAN_BASELINE • Automatic • plan_retention_weeks :=53 – deletes only unused execution plans • space_budget_percent :=10 (SMB size in SYSAUX, gives warning in alert log) 12
  • 13. © 2012 Pythian SQL Baselines Demo 1 13 OE@TEST:36> CREATE TABLE test2 AS SELECT ROWNUM n, Lpad('a', 100, 'a') v FROM dual CONNECT BY LEVEL <= 500000; Table created. OE@TEST:36> exec dbms_stats.gather_table_stats('OE', 'TEST2'); PL/SQL procedure successfully completed. OE@TEST:36> ALTER SESSION SET optimizer_capture_sql_plan_baselines = TRUE; Session altered. OE@TEST:36> SELECT * FROM test2 WHERE n = 3453; N V ---------- ---------------------------------------------------------------------------------------------------- 3453 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa OE@TEST:36> / N V ---------- ---------------------------------------------------------------------------------------------------- 3453 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa OE@TEST:36> ALTER SESSION SET optimizer_capture_sql_plan_baselines = FALSE; Session altered. SYS@TEST:1> SELECT sql_text, sql_handle, plan_name, accepted FROM dba_sql_plan_baselines; SQL_TEXT SQL_HANDLE PLAN_NAME ACC ---------------------------------------- ------------------------------ ------------------------------ --- select * from test2 where n=3453 SQL_4024b7fb6cadf7db SQL_PLAN_4095rzdqavxyv99963deb YES
  • 14. © 2012 Pythian SQL Baselines Demo 2 14 OE@TEST:36> SELECT * FROM TABLE(dbms_xplan.Display_sql_plan_baseline('SQL_4024b7fb6cadf7db', 'SQL_PLAN_4095rzdqavxyv99963deb')); PLAN_TABLE_OUTPUT -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- SQL handle: SQL_4024b7fb6cadf7db SQL text: select * from test2 where n=3453 -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- Plan name: SQL_PLAN_4095rzdqavxyv99963deb Plan id: 2576760299 Enabled: YES Fixed: NO Accepted: YES Origin: AUTO-CAPTURE -------------------------------------------------------------------------------- Plan hash value: 300966803 --------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 106 | 2126 (1)| 00:00:26 | |* 1 | TABLE ACCESS FULL| TEST2 | 1 | 106 | 2126 (1)| 00:00:26 | --------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - filter("N"=3453) 24 rows selected. OE@TEST:36> CREATE INDEX test2_ix ON test2(n); Index created.
  • 15. © 2012 Pythian SQL Baselines Demo 3 15 OE@TEST:36> SET AUTOTRACE ON EXPL OE@TEST:36> SELECT * FROM test2 WHERE n = 3453; N V ---------- ---------------------------------------------------------------------------------------------------- 3453 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Execution Plan ---------------------------------------------------------- Plan hash value: 300966803 --------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 106 | 2126 (1)| 00:00:26 | |* 1 | TABLE ACCESS FULL| TEST2 | 1 | 106 | 2126 (1)| 00:00:26 | --------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - filter("N"=3453) Note ----- - SQL plan baseline "SQL_PLAN_4095rzdqavxyv99963deb" used for this statement SYS@TEST:1> SELECT sql_text, sql_handle, plan_name, accepted FROM dba_sql_plan_baselines; SQL_TEXT SQL_HANDLE PLAN_NAME ACC ---------------------------------------- ------------------------------ ------------------------------ --- select * from test2 where n=3453 SQL_4024b7fb6cadf7db SQL_PLAN_4095rzdqavxyv99963deb YES select * from test2 where n=3453 SQL_4024b7fb6cadf7db SQL_PLAN_4095rzdqavxyvb4aa7de4 NO
  • 16. © 2012 Pythian SQL Baselines Demo 4 16 SYS@TEST:1> var rep clob SYS@TEST:1> exec :rep := dbms_spm.evolve_sql_plan_baseline(sql_handle => 'SQL_4024b7fb6cadf7db'); PL/SQL procedure successfully completed. SYS@TEST:1> print :rep Inputs: ------- SQL_HANDLE = SQL_4024b7fb6cadf7db PLAN_NAME = TIME_LIMIT = DBMS_SPM.AUTO_LIMIT VERIFY = YES COMMIT = YES Plan: SQL_PLAN_4095rzdqavxyvb4aa7de4 ------------------------------------ Plan was verified: Time used .49 seconds. Plan passed performance criterion: 1926.48 times better than baseline plan. Plan was changed to an accepted plan. Baseline Plan Test Plan Stats Ratio ------------- --------- ----------- Execution Status: COMPLETE COMPLETE Rows Processed: 1 1 Elapsed Time(ms): 29.928 .05 598.56 CPU Time(ms): 29.335 0 Buffer Gets: 7703 4 1925.75 Physical Read Requests: 0 0 Physical Write Requests: 0 0 Physical Read Bytes: 0 0 Physical Write Bytes: 0 0 Executions: 1 1 ------------------------------------------------------------------------------- ...
  • 17. © 2012 Pythian SQL Baselines Demo 5 17 OE@TEST:36> SELECT * FROM test2 WHERE n = 3453; N V ---------- ---------------------------------------------------------------------------------------------------- 3453 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Execution Plan ---------------------------------------------------------- Plan hash value: 430623757 ---------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ---------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 106 | 4 (0)| 00:00:01 | | 1 | TABLE ACCESS BY INDEX ROWID| TEST2 | 1 | 106 | 4 (0)| 00:00:01 | |* 2 | INDEX RANGE SCAN | TEST2_IX | 1 | | 3 (0)| 00:00:01 | ---------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("N"=3453) Note ----- - SQL plan baseline "SQL_PLAN_4095rzdqavxyvb4aa7de4" used for this statement
  • 18. © 2012 Pythian TESTING 18
  • 19. © 2012 Pythian What do we have? 19 SYS@TEST:1> desc DBA_SQL_PLAN_BASELINES –- 11.2 version, 11.1 does not have «REPRODUCED» column Name Null? Type ----------------------- -------- ---------------- SIGNATURE NOT NULL NUMBER -– SQL Baseline identifier SQL_HANDLE NOT NULL VARCHAR2(30) -– dec_to_hex(SIGNATURE) SQL_TEXT NOT NULL CLOB PLAN_NAME NOT NULL VARCHAR2(30) –- «plans» (hints) are stored in SQLOBJ$DATA CREATOR VARCHAR2(30) ORIGIN VARCHAR2(14) PARSING_SCHEMA_NAME VARCHAR2(30) DESCRIPTION VARCHAR2(500) VERSION VARCHAR2(64) CREATED NOT NULL TIMESTAMP(6) LAST_MODIFIED TIMESTAMP(6) LAST_EXECUTED TIMESTAMP(6) –- This is not exactly what we think it is (6.5 days) LAST_VERIFIED TIMESTAMP(6) –- Last time DBMS_SPM.EVOLVE_SQL_PLAN_BASELINE verified it ENABLED VARCHAR2(3) ACCEPTED VARCHAR2(3) -- only ACCEPTED baselines are used by the optimizer FIXED VARCHAR2(3) REPRODUCED VARCHAR2(3) -- New in 11gR2. NO = plan is not valid (e.g. indexes missing) AUTOPURGE VARCHAR2(3) OPTIMIZER_COST NUMBER -- potentially very misleading value, CBO does not use it MODULE VARCHAR2(64) -- it’s so much easier if the application is instrumented ACTION VARCHAR2(64) -- it’s so much easier if the application is instrumented EXECUTIONS NUMBER -- not available or not up to date ELAPSED_TIME NUMBER -- not available or not up to date CPU_TIME NUMBER -- not available or not up to date BUFFER_GETS NUMBER -- not available or not up to date DISK_READS NUMBER -- not available or not up to date DIRECT_WRITES NUMBER -- not available or not up to date ROWS_PROCESSED NUMBER -- not available or not up to date FETCHES NUMBER -- not available or not up to date END_OF_FETCH_COUNT NUMBER -- not available or not up to date
  • 20. © 2012 Pythian What do we have? • For each SQL baseline we have: • SQL Statements and execution plans • Creation and Last execution dates • Other properties like ACCEPTED, REPRODUCED, ENABLED • We don’t have: • Any statistics stored in baselines (elapsed/cpu time, Logical/Pysical IO counts, ...) • Any reliable information about cost of the SQL statement So how does it help to test the performance? • What is the typical involvement of a DBA in testing? • SPM allows DBAs to have higher control over testing • Assess the completeness of testing (are all modules tested?) • Assess the impact of schema changes (e.g. dropping/creating an index) • Impact of application change (identify new SQLs) 20
  • 21. © 2012 Pythian Main steps for performance testing using SQL baselines • Capture SQL Baselines in PROD DB • Prepare the TEST DB for performance testing • Import SQL Baselines from PROD into TEST DB • Calibrate the TEST DB • Make the planned changes in TEST DB • Run preformance tests in TEST • Compare states of SQL baselines before and after the testing to find: • SQLs not executed during the test – LAST_EXECUTED, SIGNATURE • new SQLs executed during the testing – CREATED, SIGNATURE • Non-reproducible execution plans (11gR2)– REPRODUCED, SIGNATURE, PLAN_NAME • new execution plans found – CREATE, SIGNATURE, PLAN_NAME 21
  • 22. © 2012 Pythian Capture SQL Baselines in PROD DB • Baselines have to be created for all queries for this to work • Which way of capturing SQL Baselines to choose? • optimizer_capture_sql_plan_baselines parameter • + Catches most of the execution plans • + Can be set at session level, consider a logon trigger • – Ongoing performance overhead • – Serious performance overhead if Bug 9910484 not fixed (before 11.2.0.3) • – can exhaust SYSAUX tablespace quickly • Frequent capture of execution plans from cursor cache • + Basic filtering available (schema, module, action, sql text) • + overhead can be «forecasted» (at least when it strikes) • + More control on space usage in SYSAUX • – Might not capture some queries • – Need to be done manually or need to be automated by yourself. 22
  • 23. © 2012 Pythian Capture SQL Baselines in PROD DB • Problems with the capture process • If you’re not using bind variables in your OLTP application • Do you care about the performance at all? • SQL Baselines are useless • SQL Baselines will clog your SYSAUX tablespace • will cause more performance overhead • 2 Executions !!! • Quite a few bugs in SPM • 11.1.0.6 – First release with SPM • 0 bugs fixed in 11.1.0.7 • 7 bugs fixed in 11.2.0.2 • 13 bugs fixed in 11.2.0.3 (including the 9910484) • Not usable for DWH applications as bind variables are typically not used 23
  • 24. © 2012 Pythian Prepare the TEST DB for performance testing • Quality of testing depends on how well your test environment represents the production • Test DBs differ from PROD DBs • hardware • software versions - RDBMS version and patches • statistics and optimizer parameters • Object definitions • changes to system settings • SQL profiles • data that is stored in tables • Most of the mentioned things can impact optimizer's calculations These differences can make your test results unreliable 24
  • 25. © 2012 Pythian Prepare the TEST DB for performance testing • Some problems arise because • testers and developers want to max performance on the DEV/TEST DBs • DBAs support them and «tune» the DBs making different from PROD • These differences make performance testing harder and gives room for surprises «after going live» • Things you can do: • clone TEST from PROD as late as possible • don't tune TEST DB unless same changes are planned in PROD • don't gather statistics in TEST, import them regularly from PROD • import baselines from PROD shortly before testing • Differences are inevitable 25
  • 26. © 2012 Pythian Import SQL Baselines from PROD DB into TEST DB • Simple process • Pack baselines into a staging table • Export from PROD, import into TEST • Unpack Baselines into TEST • «6.5 days» problem with LAST_EXECUTED column • Bug 8864741 : last_executed date in dba_sql_plan_baseline is not getting updated. All versions are affected. • LAST_EXECUTED date is updated once every ~6.5 days, • Oracle’s response, it would be too expensive to update it every time • We need to track usage of all plans, accurate data is important • Workaround: update LAST_EXECUTED to NULL before importing the baselines in TEST 26
  • 27. © 2012 Pythian DEMO – Importing the baselines 27 /* EXPORT FROM PROD */ /* Create staging table */ SQL> exec DBMS_SPM.CREATE_STGTAB_BASELINE(table_name=>'STAGE_BASELINES',table_owner=>'SYSTEM'); /* Pack baselines into the staging table*/ SQL> var n number SQL> exec :n :=DBMS_SPM.PACK_STGTAB_BASELINE(table_name=>'STAGE_BASELINES',table_owner=>'SYSTEM'); SQL> print n N ---------- 42 /* Export the staging table */ SQL> create directory BASELINES_EXPORT as '/tmp/'; SQL> !expdp tables=SYSTEM.STAGE_BASELINES directory=BASELINES_EXPORT dumpfile=STAGE_BASELINES.dmp logfile=expdpSTAGE_BASELINES.log SQL> drop directory BASELINES_EXPORT; /* IMPORT INTO TEST */ /* Import the staging table */ SQL> create directory BASELINES_IMPORT as '/tmp/'; SQL> !impdp tables=SYSTEM.STAGE_BASELINES directory=BASELINES_IMPORT dumpfile=STAGE_BASELINES.dmp logfile=impdpSTAGE_BASELINES.log SQL> drop directory BASELINES_IMPORT;
  • 28. © 2012 Pythian DEMO – Importing the baselines 28 /* Drop all current baselines*/ declare n number; cursor c1 is select distinct sql_handle from dba_sql_plan_baselines; begin for c in c1 loop n:=DBMS_SPM.DROP_SQL_PLAN_BASELINE(c.sql_handle); end loop; end; /* reset the LAST_EXECUTE date to NULL*/ SQL> update system.STAGE_BASELINES set last_executed=null; SQL> commit; /*Import the baselines*/ SQL> var n number SQL> exec :n :=DBMS_SPM.UNPACK_STGTAB_BASELINE(table_name=>'STAGE_BASELINES',table_owner=>'SYSTEM'); SQL> print n N ---------- 42 /*Check the baselines were imported*/ SQL> select count(*) from DBA_SQL_PLAN_BASELINES; COUNT(*) ---------- 42
  • 29. © 2012 Pythian Calibration • The purpose of Calibration • To assess the quality of the performance test (completeness) • To verify how differences between PROD and TEST affect the queries. • To distinguish changes introduced by configuration changes on TEST DB from changes introduced planned to test • The calibration process • The planned changes should not be installed yet • Save the state of the SQL Baselines before you start • Execute full performance tests • Collect data after testing and assess the results • Prepare SQL Baselines for final performance testing • Export into staging table • Update LAST_EXECUTED again to NULL • Import all baselines from staging table • Save current state of the SQL baselines 29
  • 30. © 2012 Pythian DEMO – Calibration 30 /* Capture the state of SQL Baselines before calibration test */ SQL> create table SPM_PRE_CALIB as select * from DBA_SQL_PLAN_BASELINES; SQL> create index SPM_PRE_CALIB_I1 on system.SPM_PRE_CALIB(signature); SQL> exec dbms_stats.gather_table_stats(user,'SPM_PRE_CALIB', cascade=>true); /* Make sure the baselines are captured and used */ SQL> alter system set optimizer_capture_sql_plan_baselines=true; SQL> alter system set optimizer_use_sql_plan_baselines=true; /* !!! RUN THE PERFORMANCE TESTS !!! */ /* No need to collect baselines while you’re checking the results */ SYS> alter system set optimizer_capture_sql_plan_baselines=false; /* !!! COLLECT THE DATA, I WILL SHOW SAMPLES LATER !!! */ /* Prepare SQL baselines (reset the LAST_EXECUTED) */ SQL> exec DBMS_SPM.CREATE_STGTAB_BASELINE(table_name=>'STAGE_BASE2',table_owner=>'SYSTEM'); SQL> exec :n:= DBMS_SPM.PACK_STGTAB_BASELINE(table_name=>'STAGE_BASE2',table_owner=>'SYSTEM'); SQL> update system.STAGE_BASE2 set last_executed=null; SQL> commit; SQL> exec :n:= DBMS_SPM.UNPACK_STGTAB_BASELINE(table_name=>'STAGE_BASE2',table_owner=>'SYSTEM'); /*Save the satae of the baselines again*/ SQL> create table SPM_POST_CALIB as select * from DBA_SQL_PLAN_BASELINES; SQL> create index SPM_POST_CALIB_I1 on system.SPM_POST_CALIB(signature); SQL> exec dbms_stats.gather_table_stats(user,'SPM_POST_CALIB', cascade=>true);
  • 31. © 2012 Pythian Run the Final performance tests • Implement the planned changes • DB side (change schema, parameters, statistics, configuration, ...) • Application side (new modules, fixes to code, ... ) • Make sure baselines are used and capture is on • Use any tools you have • Load testing tools • Automated testcases • Manual testing • ... • Considerations • Aim for full functionality testing • Remember! Run every test twice! • Creating a PROD-like workload is NOT required for this method to work 31
  • 32. © 2012 Pythian DEMO – Final performance testing 32 /* Implement the changes*/ SQL> DROP INDEX tpcc.orders_i1; SQL> CREATE INDEX tpcc.c_fn ON tpcc.customer(Upper(c_first)); SQL> ALTER TABLE orders ADD (ocn NUMBER); SQL> CREATE OR replace TRIGGER tpcc.cust_ord_counter BEFORE INSERT ON tpcc.orders FOR EACH ROW BEGIN SELECT Count(*) INTO :new.ocn FROM tpcc.orders WHERE o_c_id = :new.o_c_id; END; / /* Make sure the baselines are captured and used */ SQL> ALTER SYSTEM SET optimizer_capture_sql_plan_baselines=TRUE; SQL> ALTER SYSTEM SET optimizer_use_sql_plan_baselines=TRUE; /* !!! RUN THE PERFORMANCE TESTS !!! */ /* No need to collect baselines while you’re checking the results */ SYS> ALTER SYSTEM SET optimizer_capture_sql_plan_baselines=FALSE; /* !!! COLLECT THE DATA !!! */
  • 33. © 2012 Pythian RESULTS COLLECT THE DATA 33
  • 34. © 2012 Pythian SQLs not executed during the test 34 SELECT DISTINCT sql_handle ,dbms_lob.Substr(sql_text, 100, 1) sql_text --,module --,action FROM spm_post_calib c WHERE c.enabled = 'YES' AND c.parsing_schema_name = 'TPCC' AND NOT EXISTS (SELECT 1 FROM dba_sql_plan_baselines b WHERE b.signature = c.signature AND b.last_executed IS NOT NULL) AND NOT EXISTS (SELECT 1 FROM dba_sql_plan_baselines b WHERE C.signature = B.signature AND b.created > c.created); SQL_HANDLE SQL_TEXT ------------------------------ -------------------------------------------------------------- SQL_fc5efaa8ffabe508 SELECT I_PRICE, I_NAME, I_DATA FROM ITEM WHERE I_ID = :B1
  • 35. © 2012 Pythian new SQLs executed during the testing 35 SELECT DISTINCT sql_handle ,dbms_lob.Substr(sql_text, 100, 1) sql_text --,module --,action FROM dba_sql_plan_baselines b WHERE enabled = 'YES' AND parsing_schema_name = 'TPCC' AND NOT EXISTS (SELECT 1 FROM SYSTEM.spm_post_calib C WHERE C.signature = B.signature AND C.enabled = 'YES'); SQL_HANDLE SQL_TEXT ------------------------------ --------------------------------------------------------------- SQL_141e8b480198d134 SELECT COUNT(*) FROM TPCC.ORDERS WHERE O_C_ID=:B1
  • 36. © 2012 Pythian Non-reproducible execution plans 36 SELECT sql_handle, plan_name, dbms_lob.Substr(sql_text, 100, 1) sql_text FROM dba_sql_plan_baselines b WHERE enabled = 'YES' AND reproduced = 'NO' AND parsing_schema_name = 'TPCC' INTERSECT SELECT sql_handle,plan_name, dbms_lob.Substr(sql_text, 100, 1) sql_text FROM spm_post_calib c WHERE enabled = 'YES' AND reproduced = 'YES' AND parsing_schema_name = 'TPCC'; SQL_HANDLE PLAN_NAME SQL_TEXT ------------------------------ ------------------------------ ---------------------------------- SQL_1d3eb12408a63da1 SQL_PLAN_1ugpj4h4acgd12e067175 UPDATE ORDERS SET O_CARRIER_ID = :B4 WHERE O_ID = :B3 AND O_D_ID = :B2 AND O_W_ID = :B1 SQL_f7db40080b18fe6a SQL_PLAN_ggqu0105jjzma6d5a2ea5 SELECT O_C_ID FROM ORDERS WHERE O_ID = :B3 AND O_D_ID = :B2 AND O_W_ID = :B1
  • 37. © 2012 Pythian New execution plans found – candidates for evolving 37 SELECT sql_handle, plan_name, dbms_lob.Substr(sql_text, 100, 1) sql_text FROM dba_sql_plan_baselines b WHERE enabled = 'YES‘ AND parsing_schema_name = 'TPCC' AND EXISTS (SELECT 1 FROM SYSTEM.spm_post_calib C WHERE C.signature = B.signature AND C.enabled = 'YES') AND NOT EXISTS (SELECT 1 FROM SYSTEM.spm_post_calib C WHERE C.signature = B.signature AND C.plan_name = B.plan_name AND C.enabled = 'YES'); SQL_HANDLE PLAN_NAME SQL_TEXT ------------------------------ ------------------------------ ---------------------------------- SQL_1d3eb12408a63da1 SQL_PLAN_1ugpj4h4acgd12ce83b16 UPDATE ORDERS SET O_CARRIER_ID = :B4 WHERE O_ID = :B3 AND O_D_ID = :B2 AND O_W_ID = :B1 SQL_8659e8b608cffc3c SQL_PLAN_8cqg8qs4czz1wb1714af0 SELECT COUNT(*) FROM TPCC.CUSTOMER WHERE UPPER(C_FIRST) LIKE :B1 ||'%' SQL_f7db40080b18fe6a SQL_PLAN_ggqu0105jjzma4bfb13fc SELECT O_C_ID FROM ORDERS WHERE O_ID = :B3 AND O_D_ID = :B2 AND O_W_ID = :B1
  • 38. © 2012 Pythian RESULTS ASSESS THE RESULTS 38
  • 39. © 2012 Pythian Completeness of the performance test • Assessing the completeness of testing is usually not easy • You have • SQLs coming from PROD stored in SQL Baselines • You know which of them are not executed during testing • You can • Identify module/action which the untested SQLs belong to • Provide information to developers / testers about it • Probably it was a planned application change • Probably the functionality was not tested • Take a justified decision on the next steps 39
  • 40. © 2012 Pythian The Impact of schema changes • Find the queries which have new new execution plans recorded in the SQL baselines • Use DBMS_XPLAN.DISPLAY_SQL_PLAN_BASELINE to check the new execution plan • Evolve the baseline to activate it if it provides better performance • Import the new baselines in production with the implementation of the change so the best execution plan can be used immediately • Find the baselines which are not reproducible after the change • Be careful with DBMS_XPLAN.DISPLAY_SQL_PLAN_BASELINE • It does not show the plan for non-reproducible SQL baselines • It shows best available execution plan instead • You might find the original execution plan in AWR tables • You will find the original execution plan in Production DB 40
  • 41. © 2012 Pythian The Impact of application changes • Identify new SQL statements • Easy to find and view all new execution plans • Much harder to find real execution statistics • V$SQL.EXACT_MATCHING_SIGNATURE = DBA_SQL_PLAN_BASELINES.SIGNATURE • sqlobj$.PLAN_ID = PLAN_HASH_VALUE, look up AWR or STATSPACK tables for PLAN_HASH_VALUE 41
  • 42. © 2012 Pythian Summary • DBAs are rarely actively involved in performance testing • DBAs are usually ones who get called first when there are performance issues • Using SQL Baselines allows to involve DBAs in performance testing more actively • DBAs can take part in testing • DBAs can see the scope and completeness of testing • DBAs can partially assess the impact of the changes before they are implemented in production • DBAs can alert developers about issues before they happen If this does not sound good enough use the SQL Plan Management at least for what it was designed It’s absolutely worth it! 42
  • 43. © 2012 Pythian ? Tweet about the event: @MarisElsins and #ough2012 Give your feedback! Ask more questions: elsins@pythian.com, @MarisElsins Blog posts on this topic will follow: http://www.pythian.com/news/author/elsins/ Follow Pythian on twitter @pythian and LinkedIn http://linkd.in/pythian 43