Why managing DDF is important, and how to do it. Plus all the lovely insight you can gain into DDF from SMF 101 records.

1. Even More Fun With DDF
Martin Packer, IBM
martin_packer@uk.ibm.com
Twitter: martinpacker

2. Abstract
The idea of "alien" DB2 work coming into your system through DDF strikes
fear into even the most seasoned Performance Specialist...
How will I classify it?
What will stop it from taking over my machine?
This presentation describes how to use performance data to address both
of those questions, based on the author's recent experiences with
numerous customers.
It also enables you to understand what applications and machines are
issuing the DDF requests, improving your knowledge of the application
landscape.

3. Agenda
• What Is DDF?
• Why Do I Care About DDF?
• Tutorial
• Client 1 – Some Basic Statistics
• Client 2 - A CPU Spike
• Client 3 – Sloshing Or Not?
• Client 3 Again – Some Gratuitous Nosiness?
• Conclusion

4. What Is DDF?
• One of the major gateways into z/OS
• Specific to DB2 as the server
• Around since DB2 Version 2
• Refined many times since
• Broadly clients are:
• JDBC – 'JCC'
• z/OS – 'DSN'
• OS/400 – 'QSQ'
• Other – 'SQL'
• Higher risk of "feral SQL"

5. Why Do I Care About DDF?
• Many modern applications use DDF to get to DB2
• You'd like them to perform well
• You'd like them not to take over your machine
• Or have the “low value” stuff cost you MLC
• You'd like to know what they even are
• Don't you just hate it when work shows up and nobody
tells you?
• You'd like to ensure zIIP exploitation is optimal

6. Feral Versus Production DDF
• Essentially two styles of DDF application:
• Production:
• Regular arrival pattern and shape
• Examples: SAP, Websphere Application Server
• Managed by some IT people, somewhere
• Likely to be tunable
• Feral:
• Could come from anywhere, at any time
• Examples: Excel, z/OS Batch with Type 4 JDBC connections
• Not managed, other than by the user
• Whose desk will you show up at with your shillelagh? J
• Important to know which you’ve got
• And you might well have both
• Different management approaches likely to be needed

7. Tutorial

8. We're Interested In

9. Starting Right At
The Top

10. Drilling
Down Into A
Service Class

11. Roughly 60%

12. But What Is A DDF Transaction?
• Conversations between requestors and DB2 are of
variable complexity
• Some do few commits
• Some do many
• Each commit / abort ends a transaction
• Infrequent commits can lead to long transactions
• Frequent commits to short ones
• So a transaction is generally not a whole
conversation
• A DDF transaction is a WLM transaction
• And we've already seen those
• A single SMF 101 record is generally NOT a single
transaction

13. Multiperiod Transactions
• As with many transaction types WLM can work
with "period aging"
• If a transaction accumulates enough service it'll fall into
second and subsequent periods
• Generally we're talking about CPU

15. CPU And The DBM1 And DIST Address Spaces
Mostly Independent Enclave Roughly 60% of
Independent Enclave

16. SMF 30 Contains Transaction Rate
• SMF30ETC is Independent Enclave Transaction Rate Count
• Process for DIST address space
• Counts transactions for all DDF service classes served by the
subsystem
• Likewise doesn't distinguish between period endings
• Many customers have multiple DB2 subsystems per LPAR
• Unnecessary to have separate service classes for each
• SMF30ETC lets you see how busy, transactionwise, each subsystem
is
• Could usefully divide Enclave CPU by transaction rate
• Can't do this for CICS
• Since I first wrote these slides I wrote RENCCSV
• In our tools hunts for all Enclave-using address spaces

18. DDF And The DB2 Accounting Trace (SMF 101) Record
• Everything you'd expect from a 101 record
• e.g. Timings
• e.g. SQL Counts
• Plus more:
• DDF Identifiers (QMDA section)
• DDF Counters (QLAC section)
• WLM Service Class (QWACWLME)
• Records for requester and for server
• DB2 on z/OS to DB2 on z/OS appears twice
• One in each subsystem
• Records can be rolled up
• ACCUMACC=10 is default
• 10th Commit / Rollback
• Or some other obscure cases
• ACCUMACC=NO
• 1 per Commit / Rollback
• Can calculate ACCUMACC value from SMF 101

19. QMDA and QLAC Sections
• QMDA – identifiers
• Different for z/OS and other platforms
• Useful for designing DDF WLM classification rules
• Accounting Information
• IP Address / Net Name
• End User ID
• z/OS DB2 Correlation / Plan information
• …
• QLAC – counters
• Common across all platforms
• Sent & Received
• SQL Statements
• Bytes
• Rows
• Blocks
• Commits / Aborts
• …

20. QWHS and QWACWLME
• QWHS Common to SMF 101 records from different types of
connections.
• Standard Header in Product Section
• QWHSLWID – 24-byte Logical Unit Of Work ID
• QWHSNID – 8-byte Network Name
• QWHSLUNM – 8-byte LU Name
• QWHSLUUV – 6-byte Uniqueness Value
• QWHSLUCC – 2-byte Commit Count
• QWHSLWID (minus QWHSLUCC) allows you to tie a
conversation together.
• QWACWLME unique to DDF
• WLM Service Class
• Binary zeroes for other attachments

21. Batch And DDF – Strange Bedfellows?
• SAP Batch is not JES Batch
• A bunch of "transactions" ie SMF 101 records
• Unique correlator in QWHS section
• Plotting commit (or record) rate by minute times the job
• Gives better granularity than JES Batch
• Eg Varying effect of CPU queuing – Not Accounted For time
• Bulk (“Batch” if you like) processing can present a
problem:
• Frequent committers might not period age
• Always in Period 1, even after hours
• Two approaches:
• Use Resource Groups to control overall access to CPU
• Separate out Bulk into its own Service Class
• But how to distinguish?
• Give a less lofty goal
• These approaches not mutually incompatible

22. Batch And DDF – Strange Bedfellows? …
• Conversely, some JES Batch uses DDF
• Job J attaches to Subsystem A
• Subsystem A talks DDF to Subsystem B
• Field QTXAOTSE for A documents time spent in B and
communicating
• Collect SMF 101 from both subsystems
• Correlators in QWHS sections match
• Apart from Commit Count
• In practice 101 timestamps don't match
• Server DB2 some minutes later

24. Client 1
-
Some Basic Statistics

29. Client 2
-
A CPU Spike

38. Client 3
-
Sloshing Or Not?

39. Sloshing Or Not? You Decide

40. Sloshing
• My term
• But customers seem to like the emotional payload of the term
• Describes situation where work rushes from one subsystem
(or LPAR) to another
• Work rushes to a "good" place to run it
• Suddenly not so good anymore
• Matters more at 90%+ average utilisation than at 50%
• A byproduct of SERVERWLM
• Round robin wouldn't induce this
• You probably wouldn't see this in RMF or SMF 30
• But SMF 101 goes down to 100th of a second granularity

41. A Nice Response Time Distribution

42. Client 3 Again
-
Some Gratuitous Nosiness?

43. Some Simple Nodes – No “Gateway”
Two JCC Levels

44. Sometimes It’s Useful To Group IP Address Ranges

45. Some Linux, AIX, Solaris & z/OS DB2 Connectors

46. So What Have We
Learnt?

47. Maybe you learnt something like this
• DDF Is Important To Manage
• DDF Management needs to happen with WLM and
by application examination / tuning
• Accounting Trace SMF 101 is key instrumentation
• Martin has some super DFSORT code to summarise it
• Down to arbitrarily short time intervals
• You can have more fun with DDF (than you
might've supposed)
• And even more than I supposed J