Mining the AWR repository forCapacity Planning, Visualization,    & other real world stuff
Who am I?•   Karl Arao, Oracle ACE, OCP-DBA, RHCE•   Currently @ Enkitec - Senior Technical Consultant•   Formerly @ SQL*W...
Who am I?•   Karl Arao, Oracle ACE, OCP-DBA, RHCE•   Currently @ Enkitec - Senior Technical Consultant•   Formerly @ SQL*W...
Who am I?•   Karl Arao, Oracle ACE, OCP-DBA, RHCE•   Currently @ Enkitec - Senior Technical Consultant•   Formerly @ SQL*W...
What will I talk about?
Overwhelming
AWR HELL
DBA_HIST_* views
My first close encounter
gc block lost – sudden slow downhttp://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost
gc block lost – sudden slow downhttp://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost
gc block lost – sudden slow downhttp://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost
gc block lost – sudden slow downhttp://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost
After gc block lost – normal workloadhttp://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost
Utilization = Requirement / Capacity
Double Y Axis Graph
t1 -------------------------------------> t0    335 – 336 – 337 – 338 – 339
delta issue
AWR Scripts
Visualization
Can’t go back in time?
AAS – Average Active SessionsKyle Hailey: http://www.perfvision.com/ftp/class/02_AAS.ppt     Max CPU   Max CPU
AAS – the Golden MetricAAS & CPU count as a yardstick for a possible performance problem: if AAS < 1   -- Database is not ...
AAS from V$ACTIVE_SESSION_HISTORY AAS = Sample Count / Elapsed Time   CPU count = 4     = 19410 / 600     = 32.35
AAS from DBA_HIST_ACTIVE_SESS_HISTORY AAS = (Sample Count * 10) / Elapsed Time   CPU count = 4     = (1950 * 10) / 600    ...
AAS from AWR ReportAAS = DB Time / Elapsed Time   CPU count = 4    = 291.81 / 9.10    = 32.07
AAS from AWR Top EventsAAS = DB Time / Elapsed Time      CPU count = 4      291.81 / 9.10 = 32.07AAS = Event Time / Elapse...
awr_topevents.sql
Textual trends
AAS throughout the AWR retention                     period!http://karlarao.wordpress.com/2010/07/25/graphing-the-aas-with...
Capacity Planning
Utilization is the ultimate metric!
awr_genwl.sql
http://karlarao.wordpress.com/2010/01/31/workload-characterization-using-dba_hist-tables-and-ksar
U=R/C
where aas > 1
Filter the data points•   AAS range           aas > 1•   Per SNAP_ID or range of SNAP_IDs           id in (336)           ...
core need = # of cores * utilization * 1.25                               Database Consolidation Best Practices    http://...
Total disk IOPS = (IOPS * Read Ratio) + (IOPS * Write Ratio * RAID penalty)Number of disk = Total disk IOPS / IOPS per disk
awr_iowl.sql
Average latency issue60 minutes interval    10 minutes interval
latency (ms) = (readtim / phy reads) * 10
Linear Regression
x data (CPU) = is the "independent value", used to predict the value of yy data (AAS) = is the "dependent value", variable...
r2toolkit            Uses the following            inbuilt Oracle functions:               •regr_count               •regr...
r2toolkit        The toolkit systematically        gets the statistic with        highest correlation        coefficient (...
Linear Regression – what’s the value?                 Lets you do forecast that can                 guide you with targete...
Linear Regression on 2 node RAC     http://karlarao.tiddlyspot.com/#r2project   racnode1                                ra...
Drilling down on the peak workload...            with AAS of 10
Drilling down on the peak workload...            with AAS of 10
Now on the low workload period...        with AAS of 2.2
Now on the low workload period...        with AAS of 2.2
Recap• Mine the beautiful data set• Visualization tell a story immediately• Statistics to make sense of data
Let the    AWR data setchange your mind set!
Thank you!
References and Tools•   http://karlarao.wordpress.com     – http://karlarao.tiddlyspot.com/#%5B%5BStorage%20IOPS%2Ccapacit...
Contact me through: karl.arao@enkitec.com
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff
Upcoming SlideShare
Loading in...5
×

Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff

4,113

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,113
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
200
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Hotsos 2011: Mining the AWR repository for Capacity Planning, Visualization, & other real world stuff

  1. 1. Mining the AWR repository forCapacity Planning, Visualization, & other real world stuff
  2. 2. Who am I?• Karl Arao, Oracle ACE, OCP-DBA, RHCE• Currently @ Enkitec - Senior Technical Consultant• Formerly @ SQL*Wizard - Solutions Architect• Blog: http://karlarao.wordpress.com• Wiki: http://karlarao.tiddlyspot.com
  3. 3. Who am I?• Karl Arao, Oracle ACE, OCP-DBA, RHCE• Currently @ Enkitec - Senior Technical Consultant• Formerly @ SQL*Wizard - Solutions Architect• Blog: http://karlarao.wordpress.com• Wiki: http://karlarao.tiddlyspot.com
  4. 4. Who am I?• Karl Arao, Oracle ACE, OCP-DBA, RHCE• Currently @ Enkitec - Senior Technical Consultant• Formerly @ SQL*Wizard - Solutions Architect• Blog: http://karlarao.wordpress.com• Wiki: http://karlarao.tiddlyspot.com
  5. 5. What will I talk about?
  6. 6. Overwhelming
  7. 7. AWR HELL
  8. 8. DBA_HIST_* views
  9. 9. My first close encounter
  10. 10. gc block lost – sudden slow downhttp://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost
  11. 11. gc block lost – sudden slow downhttp://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost
  12. 12. gc block lost – sudden slow downhttp://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost
  13. 13. gc block lost – sudden slow downhttp://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost
  14. 14. After gc block lost – normal workloadhttp://karlarao.wordpress.com/2009/06/07/diagnosing-and-resolving-gc-block-lost
  15. 15. Utilization = Requirement / Capacity
  16. 16. Double Y Axis Graph
  17. 17. t1 -------------------------------------> t0 335 – 336 – 337 – 338 – 339
  18. 18. delta issue
  19. 19. AWR Scripts
  20. 20. Visualization
  21. 21. Can’t go back in time?
  22. 22. AAS – Average Active SessionsKyle Hailey: http://www.perfvision.com/ftp/class/02_AAS.ppt Max CPU Max CPU
  23. 23. AAS – the Golden MetricAAS & CPU count as a yardstick for a possible performance problem: if AAS < 1 -- Database is not blocked AAS ~= 0 -- Database basically idle -- Problems are in the APP not DB AAS < # of CPUs -- CPU available -- Database is probably not blocked -- Are any single sessions 100% active? AAS > # of CPUs -- Could have performance problems AAS >> # of CPUS -- There is a bottleneck
  24. 24. AAS from V$ACTIVE_SESSION_HISTORY AAS = Sample Count / Elapsed Time CPU count = 4 = 19410 / 600 = 32.35
  25. 25. AAS from DBA_HIST_ACTIVE_SESS_HISTORY AAS = (Sample Count * 10) / Elapsed Time CPU count = 4 = (1950 * 10) / 600 = 32.5
  26. 26. AAS from AWR ReportAAS = DB Time / Elapsed Time CPU count = 4 = 291.81 / 9.10 = 32.07
  27. 27. AAS from AWR Top EventsAAS = DB Time / Elapsed Time CPU count = 4 291.81 / 9.10 = 32.07AAS = Event Time / Elapsed Time 17410 / 546 = 31.9
  28. 28. awr_topevents.sql
  29. 29. Textual trends
  30. 30. AAS throughout the AWR retention period!http://karlarao.wordpress.com/2010/07/25/graphing-the-aas-with-perfsheet-a-la-enterprise-manager
  31. 31. Capacity Planning
  32. 32. Utilization is the ultimate metric!
  33. 33. awr_genwl.sql
  34. 34. http://karlarao.wordpress.com/2010/01/31/workload-characterization-using-dba_hist-tables-and-ksar
  35. 35. U=R/C
  36. 36. where aas > 1
  37. 37. Filter the data points• AAS range aas > 1• Per SNAP_ID or range of SNAP_IDs id in (336) where id >= 336 and id <= 340• Oracle CPU Utilization oracpupct > 50• OS CPU Utilization oscpupct > 50• Workload periods AND TO_CHAR(s0.END_INTERVAL_TIME,D) >= 1 -- Day of week: 1=Sunday 7=Saturday AND TO_CHAR(s0.END_INTERVAL_TIME,D) <= 7 AND TO_CHAR(s0.END_INTERVAL_TIME,HH24MI) >= 0900 -- Hour AND TO_CHAR(s0.END_INTERVAL_TIME,HH24MI) <= 1800 AND s0.END_INTERVAL_TIME >= TO_DATE(2010-jan-17 00:00:00,yyyy-mon-dd hh24:mi:ss) -- Data range AND s0.END_INTERVAL_TIME <= TO_DATE(2010-aug-22 23:59:59,yyyy-mon-dd hh24:mi:ss‘)
  38. 38. core need = # of cores * utilization * 1.25 Database Consolidation Best Practices http://husnusensoy.files.wordpress.com/2010/05/database-consolidation-best-practices.pdf
  39. 39. Total disk IOPS = (IOPS * Read Ratio) + (IOPS * Write Ratio * RAID penalty)Number of disk = Total disk IOPS / IOPS per disk
  40. 40. awr_iowl.sql
  41. 41. Average latency issue60 minutes interval 10 minutes interval
  42. 42. latency (ms) = (readtim / phy reads) * 10
  43. 43. Linear Regression
  44. 44. x data (CPU) = is the "independent value", used to predict the value of yy data (AAS) = is the "dependent value", variable whose value is to be predicted
  45. 45. r2toolkit Uses the following inbuilt Oracle functions: •regr_count •regr_r2 •regr_intercept •regr_slope
  46. 46. r2toolkit The toolkit systematically gets the statistic with highest correlation coefficient (relationship) No guess work!
  47. 47. Linear Regression – what’s the value? Lets you do forecast that can guide you with targeted response time optimizations and workload reduction. • Drill down on SNAP_IDs (data samples) with high AAS • Know what’s causing the high AAS on those SNAP_IDs • Tune the bottleneck - results to huge savings on system resources!
  48. 48. Linear Regression on 2 node RAC http://karlarao.tiddlyspot.com/#r2project racnode1 racnode2
  49. 49. Drilling down on the peak workload... with AAS of 10
  50. 50. Drilling down on the peak workload... with AAS of 10
  51. 51. Now on the low workload period... with AAS of 2.2
  52. 52. Now on the low workload period... with AAS of 2.2
  53. 53. Recap• Mine the beautiful data set• Visualization tell a story immediately• Statistics to make sense of data
  54. 54. Let the AWR data setchange your mind set!
  55. 55. Thank you!
  56. 56. References and Tools• http://karlarao.wordpress.com – http://karlarao.tiddlyspot.com/#%5B%5BStorage%20IOPS%2Ccapacity%2Cperformance %2Ccost%5D%5D – http://karlarao.tiddlyspot.com/#Statistics – http://karlarao.tiddlyspot.com/#OraclePerformance• Tanel Poder @ http://blog.tanelpoder.com – http://www.tanelpoder.com/files/TPT_public.zip – http://www.tanelpoder.com/files/PerfSheet.zip – Neil Gunther & Tanel Poder - Multidimensional Visualization of Oracle Performance using Barry007 http://arxiv.org/pdf/0809.2532• Kyle Hailey @ http://ashmasters.com , http://www.perfvision.com• Craig Shallahamer @ orapub.com – Introduction To Oracle Server Consolidation http://resources.orapub.com/product_p/server_consolidation_ppt.htm• Husnu Sensoy @ husnusensoy.wordpress.com – Database Consolidation Best Practices http://husnusensoy.files.wordpress.com/2010/05/database-consolidation-best- practices.pdf• Andy Rivenes @ http://www.appsdba.com/pubs.htm• Neeraj Bhatia @ www.nioug.org/files/Linear_Regression.pdf
  57. 57. Contact me through: karl.arao@enkitec.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×