DB2 Workload Manager Histograms

DB2 9.5 Monitoring Performance Tuning and Problem Determination

Agenda
What I’m going to talk about
Monitoring features new to DB2 9.5
Measuring against SLAs so that the system can be tuned to achieve them
Using new monitoring features for problem determination
What I’m not going to talk about
Monitoring features that existed prior to DB2 9.5
Tuning individual queries, bufferpool hit ratios, etc.

New DB2 9.5 Monitoring Features
Types of Information
Statistics
Counters
High watermarks
Histograms
Activities
Types of Presentation
Table functions
Event monitors

The Bell Curve

The Bell Curve in Height

The Power Law Curve

If Height Were A Power Law
“ If the average height of two hundred men was five foot ten; the most frequent (or modal) height would be held by dozens of men who were each only a foot tall …”

“… the median height would be two feet tall (a hundred men shorter than two feet, and a hundred taller)…”

“ Most important, in such a distribution, the five tallest men would be 40, 50, 66, 100, and 200 feet tall respectively.”
– Clay Shirky, Here Comes Everybody

“ When we encounter a system like a database where there is no such thing as a representative query, the habits of mind that come from thinking about averages are not merely useless, they’re harmful.”

Drawing A Histogram – Step 1

Drawing A Histogram – Step 2

6.20 20 5.90 19 5.75 18 5.72 17 5.38 16 4.79 15 4.73 14 4.70 13 4.53 12 4.34 11 3.67 10 3.28 9 2.77 8 2.48 7 2.41 6 1.63 5 1.36 4 1.24 3 1.21 2 0.53 1 Response Time Query

( 6 - 7 ] ( 5 - 6 ] ( 4 - 5 ] ( 3 - 4 ] ( 2 - 3 ] ( 1 - 2 ] [ 0 - 1 ] Frequency Bin Range 6.20 20 5.90 19 5.75 18 5.72 17 5.38 16 4.79 15 4.73 14 4.70 13 4.53 12 4.34 11 3.67 10 3.28 9 2.77 8 2.48 7 2.41 6 1.63 5 1.36 4 1.24 3 1.21 2 0.53 1 Response Time Query

6.20 20 5.90 19 5.75 18 5.72 17 5.38 16 4.79 15 4.73 14 4.70 13 4.53 12 4.34 11 3.67 10 3.28 9 2.77 8 2.48 7 2.41 6 1.63 5 1.36 4 1.24 3 1.21 2 0.53 1 Response Time Query ( 6 - 7 ] ( 5 - 6 ] ( 4 - 5 ] ( 3 - 4 ] ( 2 - 3 ] ( 1 - 2 ] 1 [ 0 - 1 ] Frequency Bin Range

( 6 - 7 ] ( 5 - 6 ] ( 4 - 5 ] ( 3 - 4 ] ( 2 - 3 ] 4 ( 1 - 2 ] 1 [ 0 - 1 ] Frequency Bin Range 6.20 20 5.90 19 5.75 18 5.72 17 5.38 16 4.79 15 4.73 14 4.70 13 4.53 12 4.34 11 3.67 10 3.28 9 2.77 8 2.48 7 2.41 6 1.63 5 1.36 4 1.24 3 1.21 2 0.53 1 Response Time Query

( 6 - 7 ] ( 5 - 6 ] ( 4 - 5 ] ( 3 - 4 ] 3 ( 2 - 3 ] 4 ( 1 - 2 ] 1 [ 0 - 1 ] Frequency Bin Range 6.20 20 5.90 19 5.75 18 5.72 17 5.38 16 4.79 15 4.73 14 4.70 13 4.53 12 4.34 11 3.67 10 3.28 9 2.77 8 2.48 7 2.41 6 1.63 5 1.36 4 1.24 3 1.21 2 0.53 1 Response Time Query

1 ( 6 - 7 ] 4 ( 5 - 6 ] 5 ( 4 - 5 ] 2 ( 3 - 4 ] 3 ( 2 - 3 ] 4 ( 1 - 2 ] 1 [ 0 - 1 ] Frequency Bin Range 6.20 20 5.90 19 5.75 18 5.72 17 5.38 16 4.79 15 4.73 14 4.70 13 4.53 12 4.34 11 3.67 10 3.28 9 2.77 8 2.48 7 2.41 6 1.63 5 1.36 4 1.24 3 1.21 2 0.53 1 Response Time Query

1 ( 6 - 7 ] 4 ( 5 - 6 ] 5 ( 4 - 5 ] 2 ( 3 - 4 ] 3 ( 2 - 3 ] 4 ( 1 - 2 ] 1 [ 0 - 1 ] Frequency Bin Range

Why Are Histograms Useful?
Reveal hidden distribution details
Detect outliers
Can be aggregated

Reveal Hidden Distribution Details
Month 1 - all users are happy
Average response time: 19 seconds
Standard deviation: 6 seconds

Month 2 - some users are complaining

Month 1 - all users are happy

Month 2 - some users are complaining

Outliers

Detecting Outliers
Sometimes, outliers have a small, even negligible effect on the average
Adding the heaviest land animal on earth to a group of 1000 people changes the average weight by less than 10%, that is, little more than 10 pounds.

Detecting Outliers
Sometimes, outliers have a significant effect
Adding the richest man on earth to a group of 1000 people changes the average net worth by 40,000%, that is, tens of millions of dollars.

Net Worth Histogram

Aggregating Histograms
Aggregate short periods of time into longer periods
Daily response times for day-to-day decision-making
Monthly reports

Aggregating Averages
In Innumeracy , John Allen Paulos gives an example of a sports anomaly involving batting averages.

Batting Averages
“ Babe Ruth hits for a higher batting average for the first half of the season and hits for a higher batting average in the second half of the season as well, …”

Batting Averages
“… but Lou Gehrig ends up with a higher batting average for the season as a whole.”

Batting Averages 200 .390 Second 100 .290 First Lou Gehrig 100 .400 Second 200 .300 First Babe Ruth At Bats Average Half Player

Batting Averages 200 .390 Second 100 .290 First Lou Gehrig 100 .400 Second 60.0 200 .300 First Babe Ruth Average × At Bats At Bats Average Half Player

Batting Averages 200 .390 Second 100 .290 First Lou Gehrig 40.0 100 .400 Second 60.0 200 .300 First Babe Ruth Average × At Bats At Bats Average Half Player

Batting Averages 78.0 200 .390 Second 29.0 100 .290 First Lou Gehrig 40.0 100 .400 Second 60.0 200 .300 First Babe Ruth Average × At Bats At Bats Average Half Player

Batting Averages 78.0 200 .390 Second 29.0 100 .290 First Lou Gehrig 100.0 300 40.0 100 .400 Second 60.0 200 .300 First Babe Ruth Average × At Bats At Bats Average Half Player

Batting Averages 107.00 300 78.0 200 .390 Second 29.0 100 .290 First Lou Gehrig 100.0 300 40.0 100 .400 Second 60.0 200 .300 First Babe Ruth Average × At Bats At Bats Average Half Player

0 ( 20 - 21 ] 0 ( 19 - 20 ] 0 ( 18 - 19 ] 0 ( 17 - 18 ] 0 ( 16 - 17 ] 0 ( 15 - 16 ] 0 ( 14 - 15 ] 0 ( 13 - 14 ] 0 ( 12 - 13 ] 0 ( 11 - 12 ] 438 ( 10 - 11 ] 1350 ( 9 - 10 ] 3238 ( 8 - 9 ] 6049 ( 7 - 8 ] 8802 ( 6 - 7 ] 9974 ( 5 - 6 ] 8802 ( 4 - 5 ] 6049 ( 3 - 4 ] 3238 ( 2 - 3 ] 1350 ( 1 - 2 ] 438 [ 0 - 1 ] January Range (seconds)

613 613 0 ( 20 - 21 ] 1890 1890 0 ( 19 - 20 ] 4533 4533 0 ( 18 - 19 ] 8469 8469 0 ( 17 - 18 ] 12322 12322 0 ( 16 - 17 ] 13963 13963 0 ( 15 - 16 ] 12322 12322 0 ( 14 - 15 ] 8469 8469 0 ( 13 - 14 ] 4533 4533 0 ( 12 - 13 ] 1890 1890 0 ( 11 - 12 ] 1051 613 438 ( 10 - 11 ] 1350 0 1350 ( 9 - 10 ] 3238 0 3238 ( 8 - 9 ] 6049 0 6049 ( 7 - 8 ] 8802 0 8802 ( 6 - 7 ] 9974 0 9974 ( 5 - 6 ] 8802 0 8802 ( 4 - 5 ] 6049 0 6049 ( 3 - 4 ] 3238 0 3238 ( 2 - 3 ] 1350 0 1350 ( 1 - 2 ] 438 0 438 [ 0 - 1 ] Jan + Feb February January Range (seconds)

613 0 613 0 ( 20 - 21 ] 1890 0 1890 0 ( 19 - 20 ] 4533 0 4533 0 ( 18 - 19 ] 8469 0 8469 0 ( 17 - 18 ] 12322 0 12322 0 ( 16 - 17 ] 14489 526 13963 0 ( 15 - 16 ] 13942 1620 12322 0 ( 14 - 15 ] 12355 3886 8469 0 ( 13 - 14 ] 11792 7259 4533 0 ( 12 - 13 ] 12452 10562 1890 0 ( 11 - 12 ] 13019 11968 613 438 ( 10 - 11 ] 11912 10562 0 1350 ( 9 - 10 ] 10497 7259 0 3238 ( 8 - 9 ] 9935 3886 0 6049 ( 7 - 8 ] 10422 1620 0 8802 ( 6 - 7 ] 10500 526 0 9974 ( 5 - 6 ] 8802 0 0 8802 ( 4 - 5 ] 6049 0 0 6049 ( 3 - 4 ] 3238 0 0 3238 ( 2 - 3 ] 1350 0 0 1350 ( 1 - 2 ] 438 0 0 438 [ 0 - 1 ] First Quarter March February January Range (seconds)

DB2 WLM Histograms
Activity
Execution time
Queue time
Lifetime
Inter-arrival time
Estimated cost
Request
Execution time

Lifecycle Of An Activity

Inter-arrival Times

How Much Data Is Enough?

How To Collect DB2 Histograms
Activity execution time
Activity queue time
Activity lifetime
alter service class MANAGERS
under MARKETING
collect aggregate activity data BASE
alter work action set MAPLOADS
alter work action MAPLOADS
collect aggregate activity data BASE

Activity execution time
Activity queue time
Activity lifetime
Activity inter-arrival time
DML activity estimated cost
under MARKETING
collect aggregate activity data EXTENDED
alter work action set MAPLOADS
alter work action MAPLOADS
collect aggregate activity data EXTENDED

Request execution time
under MARKETING
collect aggregate request data BASE

An event monitor must be active to receive the data and write it to a table, file or pipe
create event monitor DB2STATISTICS
for statistics write to table
set event monitor DB2STATISTICS state 1

Triggering A Collection
WLM_COLLECT_INT database configuration parameter
To collect once every 24 hours (1440 minutes)
update db cfg using WLM_COLLECT_INT 1440
WLM_COLLECT_STATS stored procedure
To collect immediately
call wlm_collect_stats()

The HISTOGRAMBIN Table
DESCRIBE TABLE HISTOGRAMBIN_DB2STATISTICS
Data type Column
Column name schema Data type name Length Scale Nulls
------------------------------- --------- ------------------- ---------- ----- -----
BIN_ID SYSIBM INTEGER 4 0 No
BOTTOM SYSIBM BIGINT 8 0 No
HISTOGRAM_TYPE SYSIBM VARCHAR 64 0 No
NUMBER_IN_BIN SYSIBM BIGINT 8 0 No
SERVICE_CLASS_ID SYSIBM INTEGER 4 0 No
STATISTICS_TIMESTAMP SYSIBM TIMESTAMP 10 0 No
TOP SYSIBM BIGINT 8 0 No
WORK_ACTION_SET_ID SYSIBM INTEGER 4 0 No
WORK_CLASS_ID SYSIBM INTEGER 4 0 No
9 record(s) selected.

DESCRIBE TABLE HISTOGRAMBIN _DB2STATISTICS
Data type Column
------------------------------- --------- ------------------- ---------- ----- -----

DESCRIBE TABLE HISTOGRAMBIN_DB2STATISTICS
Data type Column
------------------------------- --------- ------------------- ---------- ----- -----

Visualizing A Histogram
SELECT TOP, NUMBER_IN_BIN
FROM HISTOGRAMBIN_DB2STATISTICS

WHERE SERVICE_CLASS_ID = 13

AND HISTOGRAM_TYPE = ‘CoordActLifetime’

AND STATISTICS_TIMESTAMP = ‘2009-03-12-14.30.00.000000’

ORDER BY TOP

ORDER BY TOP
TOP NUMBER_IN_BIN
----------- -------------
-1 0
1 1
2 6
3 21
5 179
8 298
12 141
19 47
29 5
44 2
68 0

WITH HISTOGRAMS AS
( SELECT TOP, NUMBER_IN_BIN
AND STATISTICS_TIMESTAMP = ‘2009-03-12-14.30.00.000000’ )
SELECT TOP,
NUMBER_IN_BIN
FROM HISTOGRAMS
ORDER BY TOP

WITH HISTOGRAMS AS
(SELECT TOP, NUMBER_IN_BIN
AND STATISTICS_TIMESTAMP = ‘2009-03-12-14.30.00.000000’)
SELECT TOP,
NUMBER_IN_BIN /
(SELECT MAX(NUMBER_IN_BIN) FROM HISTOGRAMS)
FROM HISTOGRAMS
ORDER BY TOP

WITH HISTOGRAMS AS
SELECT TOP,
CAST(60 * NUMBER_IN_BIN /
AS INTEGER)
FROM HISTOGRAMS
ORDER BY TOP

WITH HISTOGRAMS AS
SELECT TOP,
SUBSTR(REPEAT(‘#’,
AS INTEGER) ), 1, 60) AS GRAPH
FROM HISTOGRAMS
ORDER BY TOP

WITH HISTOGRAMS AS
SELECT TOP,
SUBSTR(REPEAT(‘#’,
AS INTEGER) ), 1, 60) AS GRAPH
FROM HISTOGRAMS
ORDER BY TOP
TOP GRAPH
----------- ------------------------------------------------------------
-1
1
2 #
3 ####
5 ####################################

TOP GRAPH
----------- ------------------------------------------------------------
-1
1
2 #
3 ####
5 ####################################
8 ############################################################
12 ############################
19 #########
29 #
44
68
103
158
241
369
562
858
1309
1997

Controlling The Range Of A Histogram
create histogram template LIFETIME_TEMPLATE high bin value 44
alter service class SYSDEFAULTSUBCLASS
under SYSDEFAULTUSERCLASS
activity lifetime histogram template LIFETIME_TEMPLATE
call wlm_collect_stats
Note: You must call wlm_collect_stats after changing a histogram template for the change to take effect.

Duplicates now occur because bin sizes grow exponentially and are rounded to an integer.
Modify the query to group by top and change number_in_bin to sum(number_in_bin) to eliminate these duplicates – however, number of bins is reduced

SELECT TOP, SUM(NUMBER_IN_BIN) NUMBER_IN_BIN
GROUP BY TOP
ORDER BY TOP
TOP NUMBER_IN_BIN
----------- -------------
-1 0
1 1
2 6
3 21
4 89
5 90
6 91
7 112
8 96
9 82

The Purpose Of The Infinite Bin
The infinite bin, or catch-all bin, is the bin whose TOP is -1
Alerts you when the template fails to cover the entire range of the data
To choose a better high bin value:
For lifetime, use COORD_ACT_LIFETIME_TOP high watermark
For estimated cost, use COST_ESTIMATE_TOP high watermark

Why Do Bins Grow Exponentially?

6 hours
40 bins
=
9 minutes/bin

Service Level Agreements

Measuring For An SLA
The activity lifetime histogram provides an easy way to measure against such an SLA. Here’s how:
Convert NUMBER_IN_BIN into a percentage
Convert the percentage into a cumulative percentage

WITH HISTOGRAMS AS
SELECT TOP,
NUMBER_IN_BIN
FROM HISTOGRAMS
ORDER BY TOP

WITH HISTOGRAMS AS
SELECT TOP,
(SELECT CAST(SUM(NUMBER_IN_BIN) AS DOUBLE)
FROM HISTOGRAMS)
AS DECIMAL(9,2)) PERCENTAGE_IN_BIN
FROM HISTOGRAMS
ORDER BY TOP

WITH HISTOGRAMS AS
SELECT TOP,
FROM HISTOGRAMS)
AS DECIMAL(9,2)) PERCENTAGE_IN_BIN
CAST((SELECT 100 * SUM(NUMBER_IN_BIN)
FROM HISTOGRAM_BIN
WHERE TOP <= OUTERHIST.TOP) /
FROM HISTOGRAMS)
AS DECIMAL(9,2)) CUMULATIVE_PERCENTAGE
FROM HISTOGRAMS AS OUTERHIST
WHERE TOP != -1
ORDER BY TOP

TOP PERCENTAGE_IN_BIN CUMULATIVE_PERCENTAGE
----------- ----------------- ---------------------
1 0.00 0.00
2 0.09 0.09
3 0.00 0.09
5 0.09 0.19
8 0.00 0.19
12 0.09 0.28
19 0.28 0.57
29 0.09 0.67
44 0.76 1.44
68 0.67 2.11
103 1.34 3.46
158 1.53 5.00
241 1.92 6.92
369 2.98 9.91
562 1.63 11.54
858 2.11 13.66
1309 0.86 14.53
1997 2.02 16.55
3046 2.02 18.57
4647 3.84 22.42
7089 5.67 28.10
10813 7.12 35.22
16493 6.35 41.57
25157 12.41 53.99
38373 15.39 69.39
58532 13.95 83.34
89280 10.49 93.84

For More Information
DB2 workload management histograms, Part 1: A gentle introduction to histograms
https://www6.software.ibm.com/developerworks/offers/kits/db2/dbakit/articles/dm-0810mcdonald/
DB2 workload management histograms, Part 2: Understanding the six histograms of DB2 workload management
https://www6.software.ibm.com/developerworks/offers/kits/db2/dbakit/articles/dm-0810mcdonald2/
DB2 workload management histograms, Part 3: Visualizing and deriving statistics from DB2 histograms using SQL
https://www6.software.ibm.com/developerworks/offers/kits/db2/dbakit/articles/dm-0810mcdonald3/

http://www.thekguy.com	238
http://www.slideshare.net	8
url_unknown	2
http://static.slidesharecdn.com	1
http://feeds.feedburner.com	1
http://edse442su11.wikispaces.com	1

DB2 Workload Manager Histograms

by Keith McDonald on May 29, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

6 Embeds 251

Statistics

DB2 Workload Manager Histograms — Presentation Transcript