SlideShare a Scribd company logo
1 of 73
©2010 SoftwareOnZ©2010 SoftwareOnZ
DB2 and Storage Management
An Uneasy Partnership
Craig S. Mullins
©2010 SoftwareOnZ3
Agenda
• DBA vs. Storage Administration
• DB2 for z/OS and Storage
– Where does DB2 use storage?
– DB2 and DFSMS
– Newer DB2 Storage Features
• Mainframe Disk Storage Options
• What’s Still Missing?
• Thoughts and Best Practices
©2010 SoftwareOnZ4
How Most Shops Manage
DB2 and Storage Administration
©2010 SoftwareOnZ5
DBA vs. Storage Administration
From a Storage Perspective
Database Administration
• Capacity planning for
database objects
• Database and data
management
• Database object interest
• Storage is sometimes an
afterthought
Storage Administration
• Capacity planning for
entire environment
• Disk and tape device
management
• Data set interest
• Storage is always top of
mind
©2010 SoftwareOnZ6
6
But Something is Missing…
• There is a gap between Database Administration,
Storage Administration (DASD) and Capacity Planning.
– Storage management can be an afterthought for the DBA…
but it shouldn’t be.
– According to Gartner, Inc. the cost of managing storage is
4-10 times the initial cost of storage acquisition.
– CAGR of disk storage was 37% between 1996 and 2007.
• Information is available to DBAs (DB2 RUNSTATS, DB2
STOSPACE, DB2 RTS, ICF Catalog, SMS Storage Groups), but it is
scattered and difficult to show the full picture.
• Any historical view has to be managed manually.
©2010 SoftwareOnZ7
Are You Prepared?
• Do all of my databases have sufficient allocation to satisfy
business requirements?
• Why is DB2 storage growing when our business is not?
• Am I wasting any storage?
• When will more storage be required?
• How much additional storage is needed?
• What needs to be done to align the additional storage with
the DBMS?
Can you answer all of the following questions
about your database storage requirements:
©2010 SoftwareOnZ8
DB2 and Storage
• DB2 for z/OS and Storage
– Where does DB2 use storage?
– DB2 and DFSMS
– Modern Storage Architecture
– Newer DB2 Storage Features
©2010 SoftwareOnZ9
• Tables do NOT consume storage (table spaces do)
– However, table definitions impact storage usage
– If an EDITPROC is used,
the maximum row length
is 10 bytes fewer
Tables Impact Page Size
©2010 SoftwareOnZ11
Types of Table Spaces
• Simple – max size 64 GB
– Contains one or more tables
– No new simple table spaces as of DB2 9
– Replace with universal table space partition-by-growth
• Segmented – max size 64 GB
– Contains one or more tables
– Segments between 4 and 64 (multiples of 4)
• Partitioned - up to 4,096 partitions of up to 64 GB each
– Contains one table
– Usually used for larger tables
– Replace with universal table space partition-by-range
• Universal - up to 128 TB
– Combine partitioned and segmented characteristics
– Can contain both a base and a clone table
– Two types: Partition-By-Growth and Partition-By-Range
• LOB – up to 16 TB
• XML
©2010 SoftwareOnZ12
DB2 Objects and Storage
• DB2 Storage Groups: list of disk
volumes
• Table Spaces: stored on disk as at
least one VSAM LDS data set
– Could be multiple: partitioned
• Indexes: stored on disk (in an index
space) as at least one VSAM LDS
data set
• System data sets
– Active Log: stored on disk
– Archive Logs: stored on disk or tape
– BSDS: stored on disk
• Image Copy Backups: stored on disk
or tape
• Other “stuff”
– DB2 library data sets
– Temporary data sets (used by
utilities)
©2010 SoftwareOnZ13
VSAM and Media Manager
• DB2 uses VSAM Media Manager for its I/O operations.
• For every I/O, VSAM Media Manager builds a channel program and sends a
request to the I/O supervisor.
DB2 uses VSAM (LDS) data sets “under the covers” to store data.
©2010 SoftwareOnZ14
Multiple Data Sets per Object?
• When does DB2 utilize multiple VSAM data
sets for a table space or index?
– When the object is partitioned. Each partition will
reside in a separate data set.
• And possibly more than one per partition.
– When a data set in a segmented or simple table
space reaches its maximum size of 2 GB, DB2
might automatically create a new data set.
– When the table space is cloned.
• Each clone has its own underlying data set(s).
©2010 SoftwareOnZ15
Data Set Names
catname.DSNDBx.dbname.psname.y0001.z001
Integrated catalog name
or alias (up to eight characters).
C (for VSAM clusters) or
D (for VSAM data components).
DB2 database name
Page set name: either the table space name
or the index space name. Must be unique within
the database.
Instance qualifier for
the data set. y = I or J
(J is used when running online
REORG and CHECK utilities)1
Data set number: z = A, B, C, D, or E
For partitioned TS, the number is the
part number; A-E used for parts in excess
of 999.
For simple or segmented TS, start with
001 and increment by one as the TS
grows past max size of 2GB.
1
For cloned table spaces, there will be I0001 and I0002 – only one of which is “active”
©2010 SoftwareOnZ16
How Well Do You Track Your
DB2 VSAM Data Sets?
Screen shot source: zSPM
©2010 SoftwareOnZ17
Where Can You Find DB2-Related
Storage Information?
Online DB2 catalog Tool  http://www.z-dbs.com/zcat/zcat.php?version=9
• DB2 Catalog and Directory: stores metadata and control structures
• What about storage information?
– When did you last run STOSPACE and RUNSTATS?
©2010 SoftwareOnZ18
Storage Information in
The DB2 Catalog
• SYSTABLESPACESTATS
– NACTIVE
– NPAGES
– EXTENTS
– SPACE
– TOTALROWS
– DATASIZE
– UNCOMPRESSEDDATASIZE
• SYSTABLESPACE
– NACTIVEF
– AVGROWLEN
– SPACEF
• SYSTABLEPART
– PQTY
– SQTY (or SECQTYI)
– STORTYPE
– STORNAME
– FREEPAGE
– PCTFREE
– SPACEF
– EXTENTS
• SYSINDEXSPACESTATS
– NLEVELS
– NPAGES
– NLEAF
– NACTIVE
– SPACE
– EXTENTS
• SYSINDEXES
– NACTIVEF
– AVGROWLEN
– SPACEF
• SYSINDEXPART
– PQTY
– SQTY (or SECQTYI)
– STORTYPE
– STORNAME
– PSEUDO_DEL_ENTRIES
– FREEPAGE
– PCTFREE
– SPACEF
– EXTENTS
©2010 SoftwareOnZ19
DB2 and Storage
• DB2 for z/OS and Storage
– Where does DB2 use storage?
– DB2 and DFSMS
– Modern Storage Architecture
– Newer DB2 Storage Features
©2010 SoftwareOnZ20
DB2 Storage Options
• Let DB2 manage the data sets using Storage
Groups (STOGROUPs).
– Less work for DBAs… less control over data sets.
• Let SMS manage some or all of the data sets.
– Available when you use DB2 storage groups or
when you use data sets you have defined yourself.
• Define and manage your own data sets using
VSAM Access Method Services.
– More work for DBAs… more control over data
sets.
©2010 SoftwareOnZ21
Storage Groups
• DB2 Storage Group
– DB2 uses STOGROUPs for
disk allocation of the table
and index spaces.
– It is basically just a list of
storage volumes
• Or ‘*’ for SMS
– A STOGROUP can be
assigned to:
• Databases
• Table spaces
• Indexes
• SMS Storage Group
– An SMS Storage Group is a
pool of disk volumes upon
which SMS-managed data
sets are placed.
– A data set is placed on an
appropriate volume within
a Storage Group depending
upon the Storage Class,
Volume, Storage Group
status, and available free
space.
©2010 SoftwareOnZ22
Why Use DB2 Storage Groups?
• When a table space is created, DB2 defines the necessary VSAM data sets
using VSAM Access Method Services.
• When a table space is dropped, DB2 automatically deletes the associated
data sets.
• When a data set in a segmented or simple table space reaches its
maximum size of 2 GB, DB2 can automatically create a new data set. The
primary data set allocation is obtained for each new data set.
• DB2 can extend individual data sets as needed.
• When you create or reorganize a table space that has associated data
sets, DB2 deletes and then redefines them, reclaiming fragmented space.
– If you run REORG with the REUSE option and SHRLEVEL NONE, REORG resets
and reuses DB2-managed data sets without deleting and redefining them. If
the size of your table space is not changing, using the REUSE parameter could
be more efficient.
• When you want to move data sets to a new volume, you can alter the
volumes list in your storage group. DB2 automatically relocates your data
sets during the utility operations that build or rebuild a data set.
– LOAD REPLACE, REORG, REBUILD, and RECOVER.
©2010 SoftwareOnZ23
DFSMS
• What is DFSMS?
– Data Facility Storage Management System
– Data management, backup and HSM software
from IBM mainframes.
– It combines separate backup, copy, HSM and
device driver routines into a single package.
©2010 SoftwareOnZ24
DFSMS Components
• DFSMSdfp: Data Facility Product - provides the logical and physical input
and output for z/OS storage, it keeps track of all data and programs
managed within z/OS, and it provides data access both for native z/OS
applications and other platforms.
• DFSMSdss is a priced optional feature. It is a DASD data and space
management tool for moving and copying data.
• DFSMShsm: Hierarchical Storage Manager - a priced optional feature for
managing low-activity and inactive data. It provides backup, recovery,
migration, and space management functions.
• DFSMSrmm: Removable Media Manager - a priced optional feature for
managing removable media resources (e.g. IBM's Virtual Tape Server).
• DFSMStvs: Transactional VSAM Services – is another priced optional
feature that enables batch jobs and CICS online transactions to update
shared VSAM data sets concurrently.
DFSMS is actually multiple products; it is a suite of data
and storage management offerings.
©2010 SoftwareOnZ25
DFSMS and DB2
• Using DFSMS, a DB2 DBA can simplify the
interaction of DB2 database creation and
storage specification. It can deliver:
– Simplified data allocation
– Improved allocation control
– Improved performance management
– Automated disk space management
– Improved data availability management
– Simplified data movement
©2010 SoftwareOnZ26
Using SMS to Manage
DB2 Data Sets
Managing DB2 Data Sets with DFSMS
• The DB2 administrator can use DFSMS to achieve all the
objectives for data set placement and design.
• DFSMS has the necessary flexibility to support everything the
DB2 administrator may want.
• There is no reason to not take advantage of DFSMS for DB2
data sets.
• To achieve a successful implementation, an agreement
between the storage administrator and the DB2
administrator is required so that they can together establish
an environment that satisfies both their objectives
©2010 SoftwareOnZ27
DB2 and SMS
• Defining a DB2 STOGROUP with VOLUMES (’*’) indicates
usage of SMS-managed storage.
– With SMS, the system determines data set placement.
• Minimizes DBA work.
– Requires setup work in SMS using ACS
• For example, ACS can be used to differentiate between table spaces and
index data sets and place them on different devices.
– Consider using SMS to handle the majority of DB2 data set
placement
• Perhaps using non-SMS techniques for high volume or special data sets
©2010 SoftwareOnZ28
DB2 10, The Catalog and SMS
• SMS is a requirement for DB2 10 for z/OS
• The DB2 Catalog must be SMS managed as of V10
– The SMS environment that you use for DB2 catalog and
directory data sets must be established before you begin
migration to Version 10.
– DB2 10 requires a valid data class, management class, and storage
class
– No need to convert existing data sets
• They will be converted to SMS management when the related table
space is reorganized.
©2010 SoftwareOnZ29
SMS Terms
• Data Classes
– This is optional, although it is usually recommended that Data Classes be assigned. Even
though it is not saved for non SMS managed data sets, the allocation attributes in the
Data Class are used to allocate the data set.
• Storage Classes
– Data sets use a Storage Class to qualify for SMS management. Here GUARANTEED SPACE
is specified, along with availability, performance, and accessibility characteristics.
• Management Classes
– This is used for migration to level 1 and level 2 with or without backup, and indicates if
there should be no HSM management (backup or migration). It also includes expiration
of data sets and space release/compaction.
• Storage Groups
– The Storage Group contains volumes that satisfy the service requirements of the data
sets allocated to them. They can handle more than one type of data. Separate Storage
Groups should be defined for production table spaces, active logs, other production
data, and non-production data.
• ACS (Automatic Class Selection)
– ACS routines assign data sets to SMS storage classes. For example, indexes can be
assigned to one SMS storage class and table spaces to a different SMS storage class.
©2010 SoftwareOnZ30
DBA “Historical” Worries
• Extreme data placement
– VCAT-defined data sets for table spaces & indexes
• Managing DB2 storage groups
– Carefully associating volumes to STOGROUPs
• Extensive separation of data sets
– Examples
• careful placement to avoid contention
• always putting indexes on separate devices from data
©2010 SoftwareOnZ32
DB2 STOGROUPs
• Most shops that have not migrated to SMS
use DB2 STOGROUPs for DB/TS/IX creation.
• STOGROUP issue:
– Up thru V9: the list of volumes in the STOGROUP
is used “in that order” when allocating space
• VOLUMES(DV06, DV01, DV07)
– V10: the order “goes away” because there are no
more links in the DB2 Catalog
©2010 SoftwareOnZ33
Reducing Extents
• Modify PRIQTY / SECQTY and REORG
• Storage Administrators have other methods of
reducing extents that (perhaps) can be quicker
and/or easier:
– DFSMShsm MIGRATE and RECALL functions
– DFSMSdss COPY or DUMP and RESTORE functions
– DEFRAG with the CONSOLIDATE keyword
– Other products: e.g. Real Time Defrag
©2010 SoftwareOnZ34
Data Set Placement?
• The new disk architectures, with concepts like
log structured files and with cache in the
gigabyte sizes, have a noticeable impact on
database physical design considerations.
• Conventional database design rules based on
data set placement are becoming less
important and can be ignored in most cases.
©2010 SoftwareOnZ35
DB2 and Storage
• DB2 for z/OS and Storage
– Where does DB2 use storage?
– DB2 and DFSMS
– Modern Storage Architecture
– Newer DB2 Storage Features
©2010 SoftwareOnZ36
Modern Storage Architecture
Disk… Tape… Array…
IBM DS8700 Disk
TS3500 Tape Library
TS3400 Tape Library
EMC Symmetrix Disk
©2010 SoftwareOnZ37
A Disk is But a Disk?
• Mainframe disk, or DASD, is usually equated to a 3380 or
3390. In other words, physical hardware devices with a one-
to-one relationship between a disk drive and a volume. The
logical view is broken down as:
– Track size, or the number of bytes per track.
• 47476 for 3380
• 56664 for 3390
– Capacity, or the size of the device, in terms of number of tracks or
gigabytes.
– Device address, sometimes called device number, which is a thread
onto which I/O operations are serialized by the operating system
• Today these physical devices are replaced by disk arrays.
– An array is the combination of two or more physical disk storage
devices in a single logical device or multiple logical devices.
©2010 SoftwareOnZ38
From RAMAC (RVA) to ESS to DS
• The RAMAC Virtual Array (RVA) came first for the mainframe and it was based on
virtual disks that emulated 3380s and 3390s.
– There is no fixed physical to logical mapping. The RVA dynamically maps functional
volumes to physical drives. This mapping structure is contained in a series of tables stored
in the RVA control unit.
– RVA was OEM'ed from Storage Technology Corp (now part of Oracle).
• The ESS (Shark) followed when the STK OEM agreement expired.
– IBM technology; scalable from 420GB to 55.9 TB
– It offered improved performance (especially for prefetch and DSS)…
– But it lacked the virtual capability, at first.
• The DS8000 employs virtualized disk. Adds additional functionality for:
– Storage Pool Striping - a method of improving performance by allocating capacity to a
logical volume such that the capacity is spread over the set of ranks in the logical volume's
extent pool.
– Thin Provisioning – a method of configuring one or more logical volumes such that
capacity for data stored on the logical volume is not allocated until the data is written.
– Quick Initialization - dynamically initializes logical volumes when they are created or
expanded allowing logical volumes to be configured and placed online more quickly
– Capacity scales linearly from 1.1 TB up to 192 TB (up to 320 TB with turbo models).
©2010 SoftwareOnZ39
Disk Arrays: The Bottom Line
• They use RAID and
therefore are virtual.
• Data is spread across
multiple physical disks in
the array.
• Each I/O operation takes
place to, or from, several
physical disks.
• They emulate devices and
map logical disks to physical
locations.
3390 Model Cylinders Capacity
1 1,113 946 MB
3 3,339 2.83 GB
9 10,107 8.51 GB
27 32,760 27.84 GB
54 65,520 55.68 GB
Although the RVA, ESS, and DS8000 all work with logical disks that
map to physical locations, they are three different architectures and
are not the same type of hardware. But there are similarities:
Today’s disk arrays still emulate
the 3390 track architecture.
©2010 SoftwareOnZ40
What is RAID?
• Redundant Array of Independent Disks
• The basic idea of RAID is this:
– Multiple disks configured together into an array.
– The array is perceived by the system to be a single disk
device.
– Hot-swappable drives improve availability
– A drive can be replaced while the array is up & running.
• There are many levels of RAID technology
– Each delivers different levels of fault-tolerance and
performance.
– Examples on next slide…
©2010 SoftwareOnZ41
RAID Levels
Source: Advanced Computer & Network Corporation - http://www.acnc.com/04_00.html
(DS8000 and some ESS use RAID10)
©2010 SoftwareOnZ42
DS8000: RAID10
– RAID10 is implemented as a striped
array whose segments are RAID 1
arrays.
– RAID 10 has the same fault tolerance
as RAID level 1.
– RAID 10 has the same overhead for
fault-tolerance as mirroring alone.
– High I/O rates are achieved by striping
RAID 1 segments.
– Under certain circumstances,
RAID 10 array can sustain
multiple simultaneous drive
failures.
– Excellent solution for sites
that would have otherwise
gone with RAID 1 but need
some additional performance
boost.
• The IBM DS800 disk array is a RAID 10 implementation.
©2010 SoftwareOnZ43
Popular Modern Disk Arrays
• EMC Symmetrix
• EMC V-MAX
• EMC DMX
• Hewlett Packard XP24000/XP20000 Disk Array
• Hitachi Data Systems Lightning
• Hitachi Data Systems Universal Storage Platform
• IBM System Storage ESS (Shark)
• IBM System Storage DS6000
• IBM System Storage DS8000
• Sun StorageTek V2X4f
©2010 SoftwareOnZ44
What About Disk Cache?
• Data is written from DB2 buffer pools to “disk” based
on your buffer pool and checkpoint settings.
– Deferred Write Threshold (DWQT) is reached when this
pct. of the buffer pool is allocated to unavailable pages
(updated or in use). DB2 schedules write I/Os to
externalize data pages to disk.
– Vertical Deferred Write Threshold (VDWQT) is reached
when this pct. of the buffer pool is allocated to unavailable
pages of a single one data set.
– Checkpoint Frequency (CHKFREQ DSNZPARM) sets how
often a system checkpoint is taken (in minutes or log
records); data is “written” to disk at this point (among
other things)
©2010 SoftwareOnZ45
So What Happens When
DB2 Writes to Disk?
• When DB2 writes to disk…
– DB2 destages updated data to the disk cache.
– DB2 treats this as a disk write, even though the write is to
cache.
– The data may or may not remain in the buffer pool
depending on why the data was destaged (DWQT,
VDWQT, CHKFREQ).
– Data is also written to the NVS (Non Volatile Storage) part
of the disk controller that is battery backed.
• Even if the disk crashes, no data is lost.
– Based on disk thresholds, cache destages data back down
to disk.
©2010 SoftwareOnZ46
Some z Storage Terminology
• ESCON (Enterprise Systems Connection) is a data
connection created by IBM. It was used to connect
mainframe computers to peripheral devices such
as disk storage and tape drives.
• FICON (Fibre Connectivity) is the IBM proprietary name for
the ANSI FC-SB-3 Single-Byte Command Code Sets-3 Mapping
Protocol for Fibre Channel (FC) protocol. It is a FC layer 4
protocol used to map both IBM’s antecedent (either ESCON
or parallel) channel-to-control-unit cabling infrastructure and
protocol onto standard FC services and infrastructure.
– FICON has replaced ESCON in current deployments because of
FICON's technical superiority
• Higher performance and lower cost.
• MIDAW (Modified Indirect Address Word) is a technique for
virtual I/O addressing
©2010 SoftwareOnZ47
MIDAW
• MIDAW name stands for “modified IDAW.”
– An IDAW (indirect address word) is used to specify data
addresses for I/O operations in a virtual environment.
– This improvement is available as of the System z9™.
• MIDAW is a hardware channel enhancement that
improves sequential access speed substantially for 4
KB page sizes.
• MIDAW improves FICON performance, especially
when accessing DB2 databases, by reducing channel
utilization and increasing throughput for parallel
access streams.
©2010 SoftwareOnZ48
MIDAW and Prefetch
Source: IBM RedPaper (redp4201)
How does the MIDAW Facility Improve the Performance of FICON Channels Using DB2 and other workloads?
©2010 SoftwareOnZ49
MIDAW and the Log
Source: IBM RedPaper (redp4201)
How does the MIDAW Facility Improve the Performance of FICON Channels Using DB2 and other workloads?
©2010 SoftwareOnZ50
MIDAW & DB2: More Details
Source: IBM RedPaper (redp4201)
How does the MIDAW Facility Improve the Performance of FICON Channels Using DB2 and other workloads?
©2010 SoftwareOnZ51
DB2 and Storage
• DB2 for z/OS and Storage
– Where does DB2 use storage?
– DB2 and DFSMS
– Modern Storage Architecture
– Newer DB2 Storage Features
©2010 SoftwareOnZ52
Newer DB2 Storage Features
• Extents
– Sliding Scale Extent Allocation
– Extent Consolidation
• Variable CI Size
– Page size = CI size
• CLONE Tables
• As of V9, DB2 STOGROUP can define/alter SMS
constructs
– MGMTCLAS
– DATACLASS
– STORCLAS
• DSNZPARMs
©2010 SoftwareOnZ53
What About Extents?
• For SMS-managed data set you can have up to
123 extents on each of 59 volumes.
– As of z/OS 1.7.
• So the limit is 7,257 extents for a data set;
• Instead of the 255 we’ve been used to for some time.
– To enable this, modify the DFSMS Data Class to
set the Extent Constraint Removal to YES.
©2010 SoftwareOnZ54
OK, But Do Extents Still Matter?
Some folks think:
• “With RAID/modern storage devices and new DB2 & z/OS
features, extents are no longer anything to worry about.”
This is not exactly true:
• The latest extent management features only work with SMS-
managed data sets, so if you are user-managed old rules apply!
• For SMS-controlled data sets extents can still impact
performance.
– Each extent on a disk file has different control blocks
controlling access
– Elapsed time can increase if there is heavy insert activity
– Example:
• A program inserting 1m rows & using sliding extents with a very low
initial size doubled elapsed time compared to 1 extent
• May be caused by the allocation of each new dataset
©2010 SoftwareOnZ55
Sliding Scale Extent Allocation
• Enabled by setting MGEXTSZ to YES
(DSNZPARM)
– Default is NO for V8
– Changed to YES automatically when you
upgrade to DB2 9
• Extent sizes allocated gradually increase
• Maximum Extent size is based on DSSIZE
– Up to 16 GB the largest secondary extent size
is 127 Cylinders
– 32 GB & 64 GB the largest secondary size is
559 Cylinders
As of V8, DB2 can allocate secondary extents automatically
©2010 SoftwareOnZ56
Sliding Scale Extent Allocation
• DB2 uses a sliding scale for secondary extent allocations of
table spaces and indexes when:
– You do not specify a value for the SECQTY option of a CREATE
TABLESPACE or CREATE INDEX statement
– You specify a value of -1 for the SECQTY option of an ALTER
TABLESPACE or ALTER INDEX statement.
• Otherwise, DB2 uses the SECQTY value for secondary
extent allocations, if one is explicitly specified (and the
SECQTY value is larger than the value that is derived from the
sliding scale algorithm).
• If the table space or index space has a SECQTY greater than
0, the primary space allocation of each subsequent data set
is the larger of the SECQTY setting and the value that is
derived from a sliding scale algorithm.
©2010 SoftwareOnZ57
The Sliding Scale Algorithm
• If you do not provide a value for the secondary space allocation quantity,
DB2 calculates a secondary space allocation value equal to 10% of the
primary space allocation value and subject to the following conditions:
– The value cannot be less than 127 cylinders for data sets that range in initial
size from less than 1 GB to 16 GB, and cannot be less than 559 cylinders for 32
GB and 64 GB data sets.
– The value cannot be more than the value that is derived from the sliding scale
algorithm.
• The calculation that DB2 uses for the secondary space allocation value is:
Actual secondary extent size = max ( 0.1 × PRIQTY, min ( ss_extent, MaxAlloc ) )
• In this calculation, ss_extent represents the value that is derived from the
sliding scale algorithm, and MaxAlloc is either 127 or 559 cylinders,
depending on the maximum potential data set size. Secondary space
allocation quantities do not exceed DSSIZE or PIECESIZE clause values.
©2010 SoftwareOnZ58
Extent Consolidation
• Extent Consolidation
– Requires SMS-managed STOGROUPs
– If new extent is adjacent to old, they will be
merged together
– Some extents may end up being larger than the
PRIQTY or SECQTY specification(s)
– Introduced in z/OS 1.5
©2010 SoftwareOnZ59
Variable CI Size
• Beginning in Version 8, DB2 can define data sets with variable VSAM control intervals.
– Prior to Version 8, DB2 defined all data sets with 4K CI size.
• One of the biggest benefits of this change is an improvement in query processing
performance.
• The VARY DS CONTROL INTERVAL parameter on installation panel DSNTIP7 (DSVCI
DSNZPARM) allows you to control whether DB2–managed data sets have variable VSAM
control intervals:
– YES - indicates that a DB2–managed data set is created with a VSAM CI that corresponds to the
size of the buffer pool that is used for the table space. This is the default value.
– NO - indicates that a DB2–managed data set is created with a fixed VSAM CI of 4 KB, regardless
of the size of the buffer pool that is used for the table space.
©2010 SoftwareOnZ60
CLONE Tables
• This is not the place for a comprehensive discussion of cloning but…
– Basically, cloning creates a table with the exact same attributes as a table that
already exists, except that it has no data.
– It is created using the ALTER TABLE SQL statement with the ADD CLONE
parameter.
– The clone table is created in the same table space as the existing table.
• But in a different VSAM data set.
– After creating the clone table you can do whatever you want to do with it.
LOAD it, INSERT rows to it, etc.
– When the clone is ready to become the base table it is exchanged with the
base table. This is done using the EXCHANGE SQL statement.
– After running an EXCHANGE the clone becomes the
“real” table and the previously “real” table becomes
the clone - - and you can repeat the process.
• Bottom Line
– You’ll need more storage to manage clones.
©2010 SoftwareOnZ61
DB2 V9 STOGROUPs
• If the data set that is associated with the storage group is not managed by
Storage Management Subsystem (SMS), VOLUMES must be specified.
• Asterisks are recognized only by SMS.
• Having DB2 select the volume requires non-SMS usage or assigning an
SMS Storage Class with guaranteed space.
– However, because guaranteed space reduces the benefits of SMS allocation, it
is not recommended.
• If one or more of the DATACLAS, MGMTCLAS, or STORCLAS clauses are
specified, VOLUMES can be omitted. If the VOLUMES clause is omitted,
the volume selection is controlled by SMS.
• Verifying the existence of volumes and classes: When processing the
VOLUMES, DATACLAS, MGMTCLAS, or STORCLAS clauses, DB2 does not
check the existence of the volumes or classes or determine the types of
devices that are identified or if SMS is active. Later, when the storage
group allocates data sets, the list of volumes is passed in the specified
order to Data Facilities (DFSMSdfp).
©2010 SoftwareOnZ62
DSNZPARMs
• SEQCACH BYPASS | SEQ
– Original meaning (for 3390) was whether DB2 I/O should
bypass the disk cache, but the meaning is different now
– BYPASS - the disk will perform Sequential Detection
– SEQ - creates an explicit Prefetch request
• Recommendation: set to SEQ for improved performance
• SEQPRES YES | NO
– Similar to SEQCACH, but for DB2 LOAD and REORG utilities
– If set to YES the Cache is more likely to retain pages for
subsequent update, particularly when processing NPIs
– Recommendation: set to YES
©2010 SoftwareOnZ63
DSNZPARMs
• MGEXTSZ YES | NO
– Whether or not DB2 will manage Sliding Extents?
– Note: the default changed to YES in DB2 9 (it was NO in DB2 V8)
• TSQTY 0 and IXQTY 0
– Sets default object sizes if USING clause omitted
– Default values use 1 cylinder except for LOBs which use 10 cyls
• DSVCI YES | NO
– Whether to use variable CI sizes
– Note this is turned on by default when you install V8
• SVOLARC YES | NO
– YES means allocate a single volume for disk based archives
• Saves space if you use SMS guaranteed space option
©2010 SoftwareOnZ64
DSNZPARMs
SMS Related DSNZPARMs
• SMSDCFL
– Specifies a DFSMShsm data class for table spaces.
– If you assign a value to SMSDCFL, DB2 specifies that value
when it uses Access Method Services to define a data set
for a table space.
• SMSDCIX
– Specifies a DFSMShsm data class for indexes.
– If you assign a value to SMSDCIX, DB2 specifies that value
when it uses Access Method Services to define a data set
for an index.
©2010 SoftwareOnZ65
Wrapping It All Up
• Questions to ask yourself
• Thoughts and Ideas
©2010 SoftwareOnZ66
Storage Problems
• Out of space
• Allocation issues
• Growing very rapidly
• Too many extents
• Approaching thresholds
• Orphaned image copies
• My index keeps growing!
©2010 SoftwareOnZ67
Can You Answer These Questions?
• My volumes are SMS managed; how can I tell
what my volume names are, how much space
I have left, and how much space I have used?
• Can you rely on the information in the RTS?
– The EXTENTS column in the RTS will be updated
only when an update or applicable utility is run for
the object. A simple start after extent reduction or
a read based on SELECT will not update the
EXTENTS column (same issue as the catalog).
©2010 SoftwareOnZ68
Examples of DB2 Storage “Things”
You Should Be Monitoring
1. Space display and monitoring of a DB2 system.
2. Space display and monitoring of Databases.
3. Space display and monitoring of the Tablespaces (and Indexes).
4. Display and monitoring of the Storage Groups and the associated volumes of a
DB2 system. (Data, Workfile, Image Copies, Logs, Archives, Sort/Work etc.)
5. VSAM display of all Tablespaces and Indexes (Used, Allocated, Primary and
Secondary Quantity, Volumes) and monitoring of Extents (physical and logical).
6. Display of the Linear Page Sets of Tablespaces and Indexes, that reach their
maximum size (part. objects, DBSIZE) resp. the maximum number of datasets (non-
part. objects, PIECESIZE).
7. Intelligent HSM Migration of Image Copy Backup Data Sets, that are not used for
a local recovery (to CURRENT), as well as datasets older than the last Full Image
Copy (Dual and Remote Backups)
8. Delete of Image Copy Backup datasets of a DB2 system that are no longer
needed because of DROP, DROP/CREATE or MODIFY TABLESPACE (‘orphaned‘, not
listed in SYSIBM.SYSCOPY).
©2010 SoftwareOnZ69
Allocated Space
Screen shot source: zSPM
©2010 SoftwareOnZ70
70
Growth of DB2 & DB2 Objects
Screen(s) shot source: zSPM
©2010 SoftwareOnZ71
My Index Keeps Growing… Help!
• For an index, deleted keys are marked as pseudo deleted.
– Actual cleaning up occurs only during certain processes (for example,
before a page split).
– This can cause indexes to “grow” even when the data technically is not.
– It can also result in high CPU cost for index scans.
• Whenever an index scan is invoked, DB2 will scan all entries in the index
including pseudo-deleted entries that have yet to be removed.
• Calculate the percentage of pseudo-deleted RIDs using
information in SYSINDEXPART:
– (PSEUDO_DEL_ENTRIES/CARDF)*100
• Reorg the index when pct of pseudo-deleted entries is:
– Greater than 10% for non Data Sharing
– Greater than 5% for Data Sharing.
Problem: My index keeps on growing and tripping extents, even
after deletes. What gives? How can I control the extent growth?
©2010 SoftwareOnZ72
What About Compression?
• CPU vs. I/O
• DSN1COMP
– Anything less than 40%, is probably not worth
doing.
• Compression dictionary is above the 2GB bar
as of V8
– Allows for more table spaces that can be
compressed
©2010 SoftwareOnZ73
Best Practices
• Keep up-to-date on DB2/storage functionality.
• Perform regular and proactive monitoring.
• Build alerts to inform you of problems,
shortages, and potential errors.
• When possible, take automated action in case
of shortages, potential errors, superfluous
data sets, etc.
©2010 SoftwareOnZ74
References
• Advanced Computer & Network Corporation, RAID.edu, http://www.acnc.com/04_00.html
• IBM Manual, DFSMS Introduction (SC26-7397)
• IBM Manual, DFSMS Storage Administration Reference (for DFSMSdfp, DFSMSdss, DFSMShsm) (SC26-7402)
• IBM Manual, DFSMS Implementing System-Managed Storage (SC26-7407)
• IBM RedBook, Storage Management with DB2 for OS/390 (SG24-5462)
• IBM RedBook, Maintaining Your SMS Environment, (SG24-5484
• IBM RedPaper, Disk Storage Access with DB2 for z/OS (redp4187))
• IBM RedPaper, DB2 for OS/390 Performance on IBM Enterprise Storage Server (redp0026)
• IBM RedPaper, How does the MIDAW Facility Improve the Performance of FICON Channels Using DB2 and
other workloads? (redp4201)
• IBM System z Storage Management Strategy
ftp://ftp.software.ibm.com/software/systemz/pdf/whitepaper/IBM_System_z_Storage_Management_Strategy_v1.1.pdf
• IBM DS8000 Storage Virtualization Overview
ftp://public.dhe.ibm.com/common/ssi/sa/wh/n/diw03020usen/DIW03020USEN.PDF
• Iczkovits, John, DB2 and Storage Management: a Guide to Surviving a Perfect Marriage, SHARE February
2008
• Mullins, Craig S., DB2 Developer’s Guide, 5th
edition, Indianapolis, IN: SAMS Publishing: ISBN 0-672-32613-4
• Smith, Ellin, DB2 UDB on z/OS Using EMC Symmetric Storage Systems Solution Guide, EMC Corporation,
ISBN 9781604613407
• Thomas, Steve, Do Extent Numbers Really Matter Any More?, IDUG Europe 2008, Session A14
• Toigo, Jon William, The Holy Grail of Data Storage Management, Upper Saddle River, NJ: Prentice-Hall:
ISBN 0-13-013055-9
©2010 SoftwareOnZ75
Questions?
©2010 SoftwareOnZ76
SoftwareOnZ
• SoftwareOnZ specializes in optimizing your usage of System z.
Our software solutions will help reduce the amount of time,
effort, and human error involved in implementing and
maintaining a cost-effective mainframe computing
environment. Whether you are a DBA, a capacity manager, a
systems programmer, an application developer or a
development manager, SoftwareOnZ has a solution for
improving your workload and reducing your computing costs.
• Our solutions can help you to more effectively manage vWLC,
improve the administration and performance of your DB2
database systems, enhance the visibility of the impact of
application changes, as well as improve your overall system
and network operations.
• At SoftwareOnZ we really do offer “a better way on z…”
http://www.softwareonz.com

More Related Content

What's hot

Presentation db2 best practices for optimal performance
Presentation   db2 best practices for optimal performancePresentation   db2 best practices for optimal performance
Presentation db2 best practices for optimal performancesolarisyougood
 
Sistemas operacionais raid
Sistemas operacionais   raidSistemas operacionais   raid
Sistemas operacionais raidCarlos Melo
 
Storage Devices And Backup Media
Storage Devices And Backup MediaStorage Devices And Backup Media
Storage Devices And Backup MediaTyrone Turner
 
Raid data recovery Tips
Raid data recovery TipsRaid data recovery Tips
Raid data recovery TipsHone Software
 
Performance evolution of raid
Performance evolution of raidPerformance evolution of raid
Performance evolution of raidZubair Sami
 
In-memory Database and MySQL Cluster
In-memory Database and MySQL ClusterIn-memory Database and MySQL Cluster
In-memory Database and MySQL Clustergrandis_au
 
Storage Technology Overview
Storage Technology OverviewStorage Technology Overview
Storage Technology Overviewnomathjobs
 
Maaz Anjum - IOUG Collaborate 2013 - An Insight into Space Realization on ODA...
Maaz Anjum - IOUG Collaborate 2013 - An Insight into Space Realization on ODA...Maaz Anjum - IOUG Collaborate 2013 - An Insight into Space Realization on ODA...
Maaz Anjum - IOUG Collaborate 2013 - An Insight into Space Realization on ODA...Maaz Anjum
 

What's hot (20)

RAID
RAIDRAID
RAID
 
Raid
RaidRaid
Raid
 
Raid Technology
Raid TechnologyRaid Technology
Raid Technology
 
Raid
RaidRaid
Raid
 
Raid
RaidRaid
Raid
 
RAID LEVELS
RAID LEVELSRAID LEVELS
RAID LEVELS
 
Raid level
Raid levelRaid level
Raid level
 
raid technology
raid technologyraid technology
raid technology
 
Presentation db2 best practices for optimal performance
Presentation   db2 best practices for optimal performancePresentation   db2 best practices for optimal performance
Presentation db2 best practices for optimal performance
 
Raid level 4
Raid level 4Raid level 4
Raid level 4
 
Sistemas operacionais raid
Sistemas operacionais   raidSistemas operacionais   raid
Sistemas operacionais raid
 
Storage Devices And Backup Media
Storage Devices And Backup MediaStorage Devices And Backup Media
Storage Devices And Backup Media
 
Raid data recovery Tips
Raid data recovery TipsRaid data recovery Tips
Raid data recovery Tips
 
Performance evolution of raid
Performance evolution of raidPerformance evolution of raid
Performance evolution of raid
 
In-memory Database and MySQL Cluster
In-memory Database and MySQL ClusterIn-memory Database and MySQL Cluster
In-memory Database and MySQL Cluster
 
Raid_intro.ppt
Raid_intro.pptRaid_intro.ppt
Raid_intro.ppt
 
Raid
RaidRaid
Raid
 
Storage Technology Overview
Storage Technology OverviewStorage Technology Overview
Storage Technology Overview
 
SEMINAR
SEMINARSEMINAR
SEMINAR
 
Maaz Anjum - IOUG Collaborate 2013 - An Insight into Space Realization on ODA...
Maaz Anjum - IOUG Collaborate 2013 - An Insight into Space Realization on ODA...Maaz Anjum - IOUG Collaborate 2013 - An Insight into Space Realization on ODA...
Maaz Anjum - IOUG Collaborate 2013 - An Insight into Space Realization on ODA...
 

Similar to DB2 and Storage Management

DB2 for z/OS Real Storage Monitoring, Control and Planning
DB2 for z/OS Real Storage Monitoring, Control and PlanningDB2 for z/OS Real Storage Monitoring, Control and Planning
DB2 for z/OS Real Storage Monitoring, Control and PlanningJohn Campbell
 
Presentation db2 best practices for optimal performance
Presentation   db2 best practices for optimal performancePresentation   db2 best practices for optimal performance
Presentation db2 best practices for optimal performancexKinAnx
 
Best Practices for Deploying Hadoop (BigInsights) in the Cloud
Best Practices for Deploying Hadoop (BigInsights) in the CloudBest Practices for Deploying Hadoop (BigInsights) in the Cloud
Best Practices for Deploying Hadoop (BigInsights) in the CloudLeons Petražickis
 
Best practices for DB2 for z/OS log based recovery
Best practices for DB2 for z/OS log based recoveryBest practices for DB2 for z/OS log based recovery
Best practices for DB2 for z/OS log based recoveryFlorence Dubois
 
Optimizing RocksDB for Open-Channel SSDs
Optimizing RocksDB for Open-Channel SSDsOptimizing RocksDB for Open-Channel SSDs
Optimizing RocksDB for Open-Channel SSDsJavier González
 
DB2 for z/OS Bufferpool Tuning win by Divide and Conquer or Lose by Multiply ...
DB2 for z/OS Bufferpool Tuning win by Divide and Conquer or Lose by Multiply ...DB2 for z/OS Bufferpool Tuning win by Divide and Conquer or Lose by Multiply ...
DB2 for z/OS Bufferpool Tuning win by Divide and Conquer or Lose by Multiply ...John Campbell
 
Database Administration & Management - 01
Database Administration & Management - 01Database Administration & Management - 01
Database Administration & Management - 01FaisalMashood
 
DBAM-01.pdf
DBAM-01.pdfDBAM-01.pdf
DBAM-01.pdfhania80
 
DB2 Design for High Availability and Scalability
DB2 Design for High Availability and ScalabilityDB2 Design for High Availability and Scalability
DB2 Design for High Availability and ScalabilitySurekha Parekh
 
Tuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris EnvironmentTuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris EnvironmentJignesh Shah
 
DbB 10 Webcast #3 The Secrets Of Scalability
DbB 10 Webcast #3   The Secrets Of ScalabilityDbB 10 Webcast #3   The Secrets Of Scalability
DbB 10 Webcast #3 The Secrets Of ScalabilityLaura Hood
 
DBA Basics guide
DBA Basics guideDBA Basics guide
DBA Basics guideazoznasser1
 
Sap On Udb Layout
Sap On Udb LayoutSap On Udb Layout
Sap On Udb Layoutbpmfhu
 
Introduzione alla nuova famiglia di NAS SnapServer
Introduzione alla nuova famiglia di NAS SnapServerIntroduzione alla nuova famiglia di NAS SnapServer
Introduzione alla nuova famiglia di NAS SnapServerPaolo Rossi
 
Storage talk
Storage talkStorage talk
Storage talkchristkv
 
Using oracle12c pluggable databases to archive
Using oracle12c pluggable databases to archiveUsing oracle12c pluggable databases to archive
Using oracle12c pluggable databases to archiveSecure-24
 
Database as a Service on the Oracle Database Appliance Platform
Database as a Service on the Oracle Database Appliance PlatformDatabase as a Service on the Oracle Database Appliance Platform
Database as a Service on the Oracle Database Appliance PlatformMaris Elsins
 
IOD 2013 - Crunch Big Data in the Cloud with IBM BigInsights and Hadoop
IOD 2013 - Crunch Big Data in the Cloud with IBM BigInsights and HadoopIOD 2013 - Crunch Big Data in the Cloud with IBM BigInsights and Hadoop
IOD 2013 - Crunch Big Data in the Cloud with IBM BigInsights and HadoopLeons Petražickis
 

Similar to DB2 and Storage Management (20)

DB2 for z/OS Real Storage Monitoring, Control and Planning
DB2 for z/OS Real Storage Monitoring, Control and PlanningDB2 for z/OS Real Storage Monitoring, Control and Planning
DB2 for z/OS Real Storage Monitoring, Control and Planning
 
Presentation db2 best practices for optimal performance
Presentation   db2 best practices for optimal performancePresentation   db2 best practices for optimal performance
Presentation db2 best practices for optimal performance
 
Best Practices for Deploying Hadoop (BigInsights) in the Cloud
Best Practices for Deploying Hadoop (BigInsights) in the CloudBest Practices for Deploying Hadoop (BigInsights) in the Cloud
Best Practices for Deploying Hadoop (BigInsights) in the Cloud
 
Best practices for DB2 for z/OS log based recovery
Best practices for DB2 for z/OS log based recoveryBest practices for DB2 for z/OS log based recovery
Best practices for DB2 for z/OS log based recovery
 
DAS RAID NAS SAN
DAS RAID NAS SANDAS RAID NAS SAN
DAS RAID NAS SAN
 
Optimizing RocksDB for Open-Channel SSDs
Optimizing RocksDB for Open-Channel SSDsOptimizing RocksDB for Open-Channel SSDs
Optimizing RocksDB for Open-Channel SSDs
 
DB2 for z/OS Bufferpool Tuning win by Divide and Conquer or Lose by Multiply ...
DB2 for z/OS Bufferpool Tuning win by Divide and Conquer or Lose by Multiply ...DB2 for z/OS Bufferpool Tuning win by Divide and Conquer or Lose by Multiply ...
DB2 for z/OS Bufferpool Tuning win by Divide and Conquer or Lose by Multiply ...
 
Database Administration & Management - 01
Database Administration & Management - 01Database Administration & Management - 01
Database Administration & Management - 01
 
DBAM-01.pdf
DBAM-01.pdfDBAM-01.pdf
DBAM-01.pdf
 
DB2 Design for High Availability and Scalability
DB2 Design for High Availability and ScalabilityDB2 Design for High Availability and Scalability
DB2 Design for High Availability and Scalability
 
Tuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris EnvironmentTuning DB2 in a Solaris Environment
Tuning DB2 in a Solaris Environment
 
DbB 10 Webcast #3 The Secrets Of Scalability
DbB 10 Webcast #3   The Secrets Of ScalabilityDbB 10 Webcast #3   The Secrets Of Scalability
DbB 10 Webcast #3 The Secrets Of Scalability
 
DBA Basics guide
DBA Basics guideDBA Basics guide
DBA Basics guide
 
Sap On Udb Layout
Sap On Udb LayoutSap On Udb Layout
Sap On Udb Layout
 
Introduzione alla nuova famiglia di NAS SnapServer
Introduzione alla nuova famiglia di NAS SnapServerIntroduzione alla nuova famiglia di NAS SnapServer
Introduzione alla nuova famiglia di NAS SnapServer
 
Storage talk
Storage talkStorage talk
Storage talk
 
Using oracle12c pluggable databases to archive
Using oracle12c pluggable databases to archiveUsing oracle12c pluggable databases to archive
Using oracle12c pluggable databases to archive
 
Database as a Service on the Oracle Database Appliance Platform
Database as a Service on the Oracle Database Appliance PlatformDatabase as a Service on the Oracle Database Appliance Platform
Database as a Service on the Oracle Database Appliance Platform
 
IOD 2013 - Crunch Big Data in the Cloud with IBM BigInsights and Hadoop
IOD 2013 - Crunch Big Data in the Cloud with IBM BigInsights and HadoopIOD 2013 - Crunch Big Data in the Cloud with IBM BigInsights and Hadoop
IOD 2013 - Crunch Big Data in the Cloud with IBM BigInsights and Hadoop
 
Azure DBA with IaaS
Azure DBA with IaaSAzure DBA with IaaS
Azure DBA with IaaS
 

DB2 and Storage Management

  • 1. ©2010 SoftwareOnZ©2010 SoftwareOnZ DB2 and Storage Management An Uneasy Partnership Craig S. Mullins
  • 2. ©2010 SoftwareOnZ3 Agenda • DBA vs. Storage Administration • DB2 for z/OS and Storage – Where does DB2 use storage? – DB2 and DFSMS – Newer DB2 Storage Features • Mainframe Disk Storage Options • What’s Still Missing? • Thoughts and Best Practices
  • 3. ©2010 SoftwareOnZ4 How Most Shops Manage DB2 and Storage Administration
  • 4. ©2010 SoftwareOnZ5 DBA vs. Storage Administration From a Storage Perspective Database Administration • Capacity planning for database objects • Database and data management • Database object interest • Storage is sometimes an afterthought Storage Administration • Capacity planning for entire environment • Disk and tape device management • Data set interest • Storage is always top of mind
  • 5. ©2010 SoftwareOnZ6 6 But Something is Missing… • There is a gap between Database Administration, Storage Administration (DASD) and Capacity Planning. – Storage management can be an afterthought for the DBA… but it shouldn’t be. – According to Gartner, Inc. the cost of managing storage is 4-10 times the initial cost of storage acquisition. – CAGR of disk storage was 37% between 1996 and 2007. • Information is available to DBAs (DB2 RUNSTATS, DB2 STOSPACE, DB2 RTS, ICF Catalog, SMS Storage Groups), but it is scattered and difficult to show the full picture. • Any historical view has to be managed manually.
  • 6. ©2010 SoftwareOnZ7 Are You Prepared? • Do all of my databases have sufficient allocation to satisfy business requirements? • Why is DB2 storage growing when our business is not? • Am I wasting any storage? • When will more storage be required? • How much additional storage is needed? • What needs to be done to align the additional storage with the DBMS? Can you answer all of the following questions about your database storage requirements:
  • 7. ©2010 SoftwareOnZ8 DB2 and Storage • DB2 for z/OS and Storage – Where does DB2 use storage? – DB2 and DFSMS – Modern Storage Architecture – Newer DB2 Storage Features
  • 8. ©2010 SoftwareOnZ9 • Tables do NOT consume storage (table spaces do) – However, table definitions impact storage usage – If an EDITPROC is used, the maximum row length is 10 bytes fewer Tables Impact Page Size
  • 9. ©2010 SoftwareOnZ11 Types of Table Spaces • Simple – max size 64 GB – Contains one or more tables – No new simple table spaces as of DB2 9 – Replace with universal table space partition-by-growth • Segmented – max size 64 GB – Contains one or more tables – Segments between 4 and 64 (multiples of 4) • Partitioned - up to 4,096 partitions of up to 64 GB each – Contains one table – Usually used for larger tables – Replace with universal table space partition-by-range • Universal - up to 128 TB – Combine partitioned and segmented characteristics – Can contain both a base and a clone table – Two types: Partition-By-Growth and Partition-By-Range • LOB – up to 16 TB • XML
  • 10. ©2010 SoftwareOnZ12 DB2 Objects and Storage • DB2 Storage Groups: list of disk volumes • Table Spaces: stored on disk as at least one VSAM LDS data set – Could be multiple: partitioned • Indexes: stored on disk (in an index space) as at least one VSAM LDS data set • System data sets – Active Log: stored on disk – Archive Logs: stored on disk or tape – BSDS: stored on disk • Image Copy Backups: stored on disk or tape • Other “stuff” – DB2 library data sets – Temporary data sets (used by utilities)
  • 11. ©2010 SoftwareOnZ13 VSAM and Media Manager • DB2 uses VSAM Media Manager for its I/O operations. • For every I/O, VSAM Media Manager builds a channel program and sends a request to the I/O supervisor. DB2 uses VSAM (LDS) data sets “under the covers” to store data.
  • 12. ©2010 SoftwareOnZ14 Multiple Data Sets per Object? • When does DB2 utilize multiple VSAM data sets for a table space or index? – When the object is partitioned. Each partition will reside in a separate data set. • And possibly more than one per partition. – When a data set in a segmented or simple table space reaches its maximum size of 2 GB, DB2 might automatically create a new data set. – When the table space is cloned. • Each clone has its own underlying data set(s).
  • 13. ©2010 SoftwareOnZ15 Data Set Names catname.DSNDBx.dbname.psname.y0001.z001 Integrated catalog name or alias (up to eight characters). C (for VSAM clusters) or D (for VSAM data components). DB2 database name Page set name: either the table space name or the index space name. Must be unique within the database. Instance qualifier for the data set. y = I or J (J is used when running online REORG and CHECK utilities)1 Data set number: z = A, B, C, D, or E For partitioned TS, the number is the part number; A-E used for parts in excess of 999. For simple or segmented TS, start with 001 and increment by one as the TS grows past max size of 2GB. 1 For cloned table spaces, there will be I0001 and I0002 – only one of which is “active”
  • 14. ©2010 SoftwareOnZ16 How Well Do You Track Your DB2 VSAM Data Sets? Screen shot source: zSPM
  • 15. ©2010 SoftwareOnZ17 Where Can You Find DB2-Related Storage Information? Online DB2 catalog Tool  http://www.z-dbs.com/zcat/zcat.php?version=9 • DB2 Catalog and Directory: stores metadata and control structures • What about storage information? – When did you last run STOSPACE and RUNSTATS?
  • 16. ©2010 SoftwareOnZ18 Storage Information in The DB2 Catalog • SYSTABLESPACESTATS – NACTIVE – NPAGES – EXTENTS – SPACE – TOTALROWS – DATASIZE – UNCOMPRESSEDDATASIZE • SYSTABLESPACE – NACTIVEF – AVGROWLEN – SPACEF • SYSTABLEPART – PQTY – SQTY (or SECQTYI) – STORTYPE – STORNAME – FREEPAGE – PCTFREE – SPACEF – EXTENTS • SYSINDEXSPACESTATS – NLEVELS – NPAGES – NLEAF – NACTIVE – SPACE – EXTENTS • SYSINDEXES – NACTIVEF – AVGROWLEN – SPACEF • SYSINDEXPART – PQTY – SQTY (or SECQTYI) – STORTYPE – STORNAME – PSEUDO_DEL_ENTRIES – FREEPAGE – PCTFREE – SPACEF – EXTENTS
  • 17. ©2010 SoftwareOnZ19 DB2 and Storage • DB2 for z/OS and Storage – Where does DB2 use storage? – DB2 and DFSMS – Modern Storage Architecture – Newer DB2 Storage Features
  • 18. ©2010 SoftwareOnZ20 DB2 Storage Options • Let DB2 manage the data sets using Storage Groups (STOGROUPs). – Less work for DBAs… less control over data sets. • Let SMS manage some or all of the data sets. – Available when you use DB2 storage groups or when you use data sets you have defined yourself. • Define and manage your own data sets using VSAM Access Method Services. – More work for DBAs… more control over data sets.
  • 19. ©2010 SoftwareOnZ21 Storage Groups • DB2 Storage Group – DB2 uses STOGROUPs for disk allocation of the table and index spaces. – It is basically just a list of storage volumes • Or ‘*’ for SMS – A STOGROUP can be assigned to: • Databases • Table spaces • Indexes • SMS Storage Group – An SMS Storage Group is a pool of disk volumes upon which SMS-managed data sets are placed. – A data set is placed on an appropriate volume within a Storage Group depending upon the Storage Class, Volume, Storage Group status, and available free space.
  • 20. ©2010 SoftwareOnZ22 Why Use DB2 Storage Groups? • When a table space is created, DB2 defines the necessary VSAM data sets using VSAM Access Method Services. • When a table space is dropped, DB2 automatically deletes the associated data sets. • When a data set in a segmented or simple table space reaches its maximum size of 2 GB, DB2 can automatically create a new data set. The primary data set allocation is obtained for each new data set. • DB2 can extend individual data sets as needed. • When you create or reorganize a table space that has associated data sets, DB2 deletes and then redefines them, reclaiming fragmented space. – If you run REORG with the REUSE option and SHRLEVEL NONE, REORG resets and reuses DB2-managed data sets without deleting and redefining them. If the size of your table space is not changing, using the REUSE parameter could be more efficient. • When you want to move data sets to a new volume, you can alter the volumes list in your storage group. DB2 automatically relocates your data sets during the utility operations that build or rebuild a data set. – LOAD REPLACE, REORG, REBUILD, and RECOVER.
  • 21. ©2010 SoftwareOnZ23 DFSMS • What is DFSMS? – Data Facility Storage Management System – Data management, backup and HSM software from IBM mainframes. – It combines separate backup, copy, HSM and device driver routines into a single package.
  • 22. ©2010 SoftwareOnZ24 DFSMS Components • DFSMSdfp: Data Facility Product - provides the logical and physical input and output for z/OS storage, it keeps track of all data and programs managed within z/OS, and it provides data access both for native z/OS applications and other platforms. • DFSMSdss is a priced optional feature. It is a DASD data and space management tool for moving and copying data. • DFSMShsm: Hierarchical Storage Manager - a priced optional feature for managing low-activity and inactive data. It provides backup, recovery, migration, and space management functions. • DFSMSrmm: Removable Media Manager - a priced optional feature for managing removable media resources (e.g. IBM's Virtual Tape Server). • DFSMStvs: Transactional VSAM Services – is another priced optional feature that enables batch jobs and CICS online transactions to update shared VSAM data sets concurrently. DFSMS is actually multiple products; it is a suite of data and storage management offerings.
  • 23. ©2010 SoftwareOnZ25 DFSMS and DB2 • Using DFSMS, a DB2 DBA can simplify the interaction of DB2 database creation and storage specification. It can deliver: – Simplified data allocation – Improved allocation control – Improved performance management – Automated disk space management – Improved data availability management – Simplified data movement
  • 24. ©2010 SoftwareOnZ26 Using SMS to Manage DB2 Data Sets Managing DB2 Data Sets with DFSMS • The DB2 administrator can use DFSMS to achieve all the objectives for data set placement and design. • DFSMS has the necessary flexibility to support everything the DB2 administrator may want. • There is no reason to not take advantage of DFSMS for DB2 data sets. • To achieve a successful implementation, an agreement between the storage administrator and the DB2 administrator is required so that they can together establish an environment that satisfies both their objectives
  • 25. ©2010 SoftwareOnZ27 DB2 and SMS • Defining a DB2 STOGROUP with VOLUMES (’*’) indicates usage of SMS-managed storage. – With SMS, the system determines data set placement. • Minimizes DBA work. – Requires setup work in SMS using ACS • For example, ACS can be used to differentiate between table spaces and index data sets and place them on different devices. – Consider using SMS to handle the majority of DB2 data set placement • Perhaps using non-SMS techniques for high volume or special data sets
  • 26. ©2010 SoftwareOnZ28 DB2 10, The Catalog and SMS • SMS is a requirement for DB2 10 for z/OS • The DB2 Catalog must be SMS managed as of V10 – The SMS environment that you use for DB2 catalog and directory data sets must be established before you begin migration to Version 10. – DB2 10 requires a valid data class, management class, and storage class – No need to convert existing data sets • They will be converted to SMS management when the related table space is reorganized.
  • 27. ©2010 SoftwareOnZ29 SMS Terms • Data Classes – This is optional, although it is usually recommended that Data Classes be assigned. Even though it is not saved for non SMS managed data sets, the allocation attributes in the Data Class are used to allocate the data set. • Storage Classes – Data sets use a Storage Class to qualify for SMS management. Here GUARANTEED SPACE is specified, along with availability, performance, and accessibility characteristics. • Management Classes – This is used for migration to level 1 and level 2 with or without backup, and indicates if there should be no HSM management (backup or migration). It also includes expiration of data sets and space release/compaction. • Storage Groups – The Storage Group contains volumes that satisfy the service requirements of the data sets allocated to them. They can handle more than one type of data. Separate Storage Groups should be defined for production table spaces, active logs, other production data, and non-production data. • ACS (Automatic Class Selection) – ACS routines assign data sets to SMS storage classes. For example, indexes can be assigned to one SMS storage class and table spaces to a different SMS storage class.
  • 28. ©2010 SoftwareOnZ30 DBA “Historical” Worries • Extreme data placement – VCAT-defined data sets for table spaces & indexes • Managing DB2 storage groups – Carefully associating volumes to STOGROUPs • Extensive separation of data sets – Examples • careful placement to avoid contention • always putting indexes on separate devices from data
  • 29. ©2010 SoftwareOnZ32 DB2 STOGROUPs • Most shops that have not migrated to SMS use DB2 STOGROUPs for DB/TS/IX creation. • STOGROUP issue: – Up thru V9: the list of volumes in the STOGROUP is used “in that order” when allocating space • VOLUMES(DV06, DV01, DV07) – V10: the order “goes away” because there are no more links in the DB2 Catalog
  • 30. ©2010 SoftwareOnZ33 Reducing Extents • Modify PRIQTY / SECQTY and REORG • Storage Administrators have other methods of reducing extents that (perhaps) can be quicker and/or easier: – DFSMShsm MIGRATE and RECALL functions – DFSMSdss COPY or DUMP and RESTORE functions – DEFRAG with the CONSOLIDATE keyword – Other products: e.g. Real Time Defrag
  • 31. ©2010 SoftwareOnZ34 Data Set Placement? • The new disk architectures, with concepts like log structured files and with cache in the gigabyte sizes, have a noticeable impact on database physical design considerations. • Conventional database design rules based on data set placement are becoming less important and can be ignored in most cases.
  • 32. ©2010 SoftwareOnZ35 DB2 and Storage • DB2 for z/OS and Storage – Where does DB2 use storage? – DB2 and DFSMS – Modern Storage Architecture – Newer DB2 Storage Features
  • 33. ©2010 SoftwareOnZ36 Modern Storage Architecture Disk… Tape… Array… IBM DS8700 Disk TS3500 Tape Library TS3400 Tape Library EMC Symmetrix Disk
  • 34. ©2010 SoftwareOnZ37 A Disk is But a Disk? • Mainframe disk, or DASD, is usually equated to a 3380 or 3390. In other words, physical hardware devices with a one- to-one relationship between a disk drive and a volume. The logical view is broken down as: – Track size, or the number of bytes per track. • 47476 for 3380 • 56664 for 3390 – Capacity, or the size of the device, in terms of number of tracks or gigabytes. – Device address, sometimes called device number, which is a thread onto which I/O operations are serialized by the operating system • Today these physical devices are replaced by disk arrays. – An array is the combination of two or more physical disk storage devices in a single logical device or multiple logical devices.
  • 35. ©2010 SoftwareOnZ38 From RAMAC (RVA) to ESS to DS • The RAMAC Virtual Array (RVA) came first for the mainframe and it was based on virtual disks that emulated 3380s and 3390s. – There is no fixed physical to logical mapping. The RVA dynamically maps functional volumes to physical drives. This mapping structure is contained in a series of tables stored in the RVA control unit. – RVA was OEM'ed from Storage Technology Corp (now part of Oracle). • The ESS (Shark) followed when the STK OEM agreement expired. – IBM technology; scalable from 420GB to 55.9 TB – It offered improved performance (especially for prefetch and DSS)… – But it lacked the virtual capability, at first. • The DS8000 employs virtualized disk. Adds additional functionality for: – Storage Pool Striping - a method of improving performance by allocating capacity to a logical volume such that the capacity is spread over the set of ranks in the logical volume's extent pool. – Thin Provisioning – a method of configuring one or more logical volumes such that capacity for data stored on the logical volume is not allocated until the data is written. – Quick Initialization - dynamically initializes logical volumes when they are created or expanded allowing logical volumes to be configured and placed online more quickly – Capacity scales linearly from 1.1 TB up to 192 TB (up to 320 TB with turbo models).
  • 36. ©2010 SoftwareOnZ39 Disk Arrays: The Bottom Line • They use RAID and therefore are virtual. • Data is spread across multiple physical disks in the array. • Each I/O operation takes place to, or from, several physical disks. • They emulate devices and map logical disks to physical locations. 3390 Model Cylinders Capacity 1 1,113 946 MB 3 3,339 2.83 GB 9 10,107 8.51 GB 27 32,760 27.84 GB 54 65,520 55.68 GB Although the RVA, ESS, and DS8000 all work with logical disks that map to physical locations, they are three different architectures and are not the same type of hardware. But there are similarities: Today’s disk arrays still emulate the 3390 track architecture.
  • 37. ©2010 SoftwareOnZ40 What is RAID? • Redundant Array of Independent Disks • The basic idea of RAID is this: – Multiple disks configured together into an array. – The array is perceived by the system to be a single disk device. – Hot-swappable drives improve availability – A drive can be replaced while the array is up & running. • There are many levels of RAID technology – Each delivers different levels of fault-tolerance and performance. – Examples on next slide…
  • 38. ©2010 SoftwareOnZ41 RAID Levels Source: Advanced Computer & Network Corporation - http://www.acnc.com/04_00.html (DS8000 and some ESS use RAID10)
  • 39. ©2010 SoftwareOnZ42 DS8000: RAID10 – RAID10 is implemented as a striped array whose segments are RAID 1 arrays. – RAID 10 has the same fault tolerance as RAID level 1. – RAID 10 has the same overhead for fault-tolerance as mirroring alone. – High I/O rates are achieved by striping RAID 1 segments. – Under certain circumstances, RAID 10 array can sustain multiple simultaneous drive failures. – Excellent solution for sites that would have otherwise gone with RAID 1 but need some additional performance boost. • The IBM DS800 disk array is a RAID 10 implementation.
  • 40. ©2010 SoftwareOnZ43 Popular Modern Disk Arrays • EMC Symmetrix • EMC V-MAX • EMC DMX • Hewlett Packard XP24000/XP20000 Disk Array • Hitachi Data Systems Lightning • Hitachi Data Systems Universal Storage Platform • IBM System Storage ESS (Shark) • IBM System Storage DS6000 • IBM System Storage DS8000 • Sun StorageTek V2X4f
  • 41. ©2010 SoftwareOnZ44 What About Disk Cache? • Data is written from DB2 buffer pools to “disk” based on your buffer pool and checkpoint settings. – Deferred Write Threshold (DWQT) is reached when this pct. of the buffer pool is allocated to unavailable pages (updated or in use). DB2 schedules write I/Os to externalize data pages to disk. – Vertical Deferred Write Threshold (VDWQT) is reached when this pct. of the buffer pool is allocated to unavailable pages of a single one data set. – Checkpoint Frequency (CHKFREQ DSNZPARM) sets how often a system checkpoint is taken (in minutes or log records); data is “written” to disk at this point (among other things)
  • 42. ©2010 SoftwareOnZ45 So What Happens When DB2 Writes to Disk? • When DB2 writes to disk… – DB2 destages updated data to the disk cache. – DB2 treats this as a disk write, even though the write is to cache. – The data may or may not remain in the buffer pool depending on why the data was destaged (DWQT, VDWQT, CHKFREQ). – Data is also written to the NVS (Non Volatile Storage) part of the disk controller that is battery backed. • Even if the disk crashes, no data is lost. – Based on disk thresholds, cache destages data back down to disk.
  • 43. ©2010 SoftwareOnZ46 Some z Storage Terminology • ESCON (Enterprise Systems Connection) is a data connection created by IBM. It was used to connect mainframe computers to peripheral devices such as disk storage and tape drives. • FICON (Fibre Connectivity) is the IBM proprietary name for the ANSI FC-SB-3 Single-Byte Command Code Sets-3 Mapping Protocol for Fibre Channel (FC) protocol. It is a FC layer 4 protocol used to map both IBM’s antecedent (either ESCON or parallel) channel-to-control-unit cabling infrastructure and protocol onto standard FC services and infrastructure. – FICON has replaced ESCON in current deployments because of FICON's technical superiority • Higher performance and lower cost. • MIDAW (Modified Indirect Address Word) is a technique for virtual I/O addressing
  • 44. ©2010 SoftwareOnZ47 MIDAW • MIDAW name stands for “modified IDAW.” – An IDAW (indirect address word) is used to specify data addresses for I/O operations in a virtual environment. – This improvement is available as of the System z9™. • MIDAW is a hardware channel enhancement that improves sequential access speed substantially for 4 KB page sizes. • MIDAW improves FICON performance, especially when accessing DB2 databases, by reducing channel utilization and increasing throughput for parallel access streams.
  • 45. ©2010 SoftwareOnZ48 MIDAW and Prefetch Source: IBM RedPaper (redp4201) How does the MIDAW Facility Improve the Performance of FICON Channels Using DB2 and other workloads?
  • 46. ©2010 SoftwareOnZ49 MIDAW and the Log Source: IBM RedPaper (redp4201) How does the MIDAW Facility Improve the Performance of FICON Channels Using DB2 and other workloads?
  • 47. ©2010 SoftwareOnZ50 MIDAW & DB2: More Details Source: IBM RedPaper (redp4201) How does the MIDAW Facility Improve the Performance of FICON Channels Using DB2 and other workloads?
  • 48. ©2010 SoftwareOnZ51 DB2 and Storage • DB2 for z/OS and Storage – Where does DB2 use storage? – DB2 and DFSMS – Modern Storage Architecture – Newer DB2 Storage Features
  • 49. ©2010 SoftwareOnZ52 Newer DB2 Storage Features • Extents – Sliding Scale Extent Allocation – Extent Consolidation • Variable CI Size – Page size = CI size • CLONE Tables • As of V9, DB2 STOGROUP can define/alter SMS constructs – MGMTCLAS – DATACLASS – STORCLAS • DSNZPARMs
  • 50. ©2010 SoftwareOnZ53 What About Extents? • For SMS-managed data set you can have up to 123 extents on each of 59 volumes. – As of z/OS 1.7. • So the limit is 7,257 extents for a data set; • Instead of the 255 we’ve been used to for some time. – To enable this, modify the DFSMS Data Class to set the Extent Constraint Removal to YES.
  • 51. ©2010 SoftwareOnZ54 OK, But Do Extents Still Matter? Some folks think: • “With RAID/modern storage devices and new DB2 & z/OS features, extents are no longer anything to worry about.” This is not exactly true: • The latest extent management features only work with SMS- managed data sets, so if you are user-managed old rules apply! • For SMS-controlled data sets extents can still impact performance. – Each extent on a disk file has different control blocks controlling access – Elapsed time can increase if there is heavy insert activity – Example: • A program inserting 1m rows & using sliding extents with a very low initial size doubled elapsed time compared to 1 extent • May be caused by the allocation of each new dataset
  • 52. ©2010 SoftwareOnZ55 Sliding Scale Extent Allocation • Enabled by setting MGEXTSZ to YES (DSNZPARM) – Default is NO for V8 – Changed to YES automatically when you upgrade to DB2 9 • Extent sizes allocated gradually increase • Maximum Extent size is based on DSSIZE – Up to 16 GB the largest secondary extent size is 127 Cylinders – 32 GB & 64 GB the largest secondary size is 559 Cylinders As of V8, DB2 can allocate secondary extents automatically
  • 53. ©2010 SoftwareOnZ56 Sliding Scale Extent Allocation • DB2 uses a sliding scale for secondary extent allocations of table spaces and indexes when: – You do not specify a value for the SECQTY option of a CREATE TABLESPACE or CREATE INDEX statement – You specify a value of -1 for the SECQTY option of an ALTER TABLESPACE or ALTER INDEX statement. • Otherwise, DB2 uses the SECQTY value for secondary extent allocations, if one is explicitly specified (and the SECQTY value is larger than the value that is derived from the sliding scale algorithm). • If the table space or index space has a SECQTY greater than 0, the primary space allocation of each subsequent data set is the larger of the SECQTY setting and the value that is derived from a sliding scale algorithm.
  • 54. ©2010 SoftwareOnZ57 The Sliding Scale Algorithm • If you do not provide a value for the secondary space allocation quantity, DB2 calculates a secondary space allocation value equal to 10% of the primary space allocation value and subject to the following conditions: – The value cannot be less than 127 cylinders for data sets that range in initial size from less than 1 GB to 16 GB, and cannot be less than 559 cylinders for 32 GB and 64 GB data sets. – The value cannot be more than the value that is derived from the sliding scale algorithm. • The calculation that DB2 uses for the secondary space allocation value is: Actual secondary extent size = max ( 0.1 × PRIQTY, min ( ss_extent, MaxAlloc ) ) • In this calculation, ss_extent represents the value that is derived from the sliding scale algorithm, and MaxAlloc is either 127 or 559 cylinders, depending on the maximum potential data set size. Secondary space allocation quantities do not exceed DSSIZE or PIECESIZE clause values.
  • 55. ©2010 SoftwareOnZ58 Extent Consolidation • Extent Consolidation – Requires SMS-managed STOGROUPs – If new extent is adjacent to old, they will be merged together – Some extents may end up being larger than the PRIQTY or SECQTY specification(s) – Introduced in z/OS 1.5
  • 56. ©2010 SoftwareOnZ59 Variable CI Size • Beginning in Version 8, DB2 can define data sets with variable VSAM control intervals. – Prior to Version 8, DB2 defined all data sets with 4K CI size. • One of the biggest benefits of this change is an improvement in query processing performance. • The VARY DS CONTROL INTERVAL parameter on installation panel DSNTIP7 (DSVCI DSNZPARM) allows you to control whether DB2–managed data sets have variable VSAM control intervals: – YES - indicates that a DB2–managed data set is created with a VSAM CI that corresponds to the size of the buffer pool that is used for the table space. This is the default value. – NO - indicates that a DB2–managed data set is created with a fixed VSAM CI of 4 KB, regardless of the size of the buffer pool that is used for the table space.
  • 57. ©2010 SoftwareOnZ60 CLONE Tables • This is not the place for a comprehensive discussion of cloning but… – Basically, cloning creates a table with the exact same attributes as a table that already exists, except that it has no data. – It is created using the ALTER TABLE SQL statement with the ADD CLONE parameter. – The clone table is created in the same table space as the existing table. • But in a different VSAM data set. – After creating the clone table you can do whatever you want to do with it. LOAD it, INSERT rows to it, etc. – When the clone is ready to become the base table it is exchanged with the base table. This is done using the EXCHANGE SQL statement. – After running an EXCHANGE the clone becomes the “real” table and the previously “real” table becomes the clone - - and you can repeat the process. • Bottom Line – You’ll need more storage to manage clones.
  • 58. ©2010 SoftwareOnZ61 DB2 V9 STOGROUPs • If the data set that is associated with the storage group is not managed by Storage Management Subsystem (SMS), VOLUMES must be specified. • Asterisks are recognized only by SMS. • Having DB2 select the volume requires non-SMS usage or assigning an SMS Storage Class with guaranteed space. – However, because guaranteed space reduces the benefits of SMS allocation, it is not recommended. • If one or more of the DATACLAS, MGMTCLAS, or STORCLAS clauses are specified, VOLUMES can be omitted. If the VOLUMES clause is omitted, the volume selection is controlled by SMS. • Verifying the existence of volumes and classes: When processing the VOLUMES, DATACLAS, MGMTCLAS, or STORCLAS clauses, DB2 does not check the existence of the volumes or classes or determine the types of devices that are identified or if SMS is active. Later, when the storage group allocates data sets, the list of volumes is passed in the specified order to Data Facilities (DFSMSdfp).
  • 59. ©2010 SoftwareOnZ62 DSNZPARMs • SEQCACH BYPASS | SEQ – Original meaning (for 3390) was whether DB2 I/O should bypass the disk cache, but the meaning is different now – BYPASS - the disk will perform Sequential Detection – SEQ - creates an explicit Prefetch request • Recommendation: set to SEQ for improved performance • SEQPRES YES | NO – Similar to SEQCACH, but for DB2 LOAD and REORG utilities – If set to YES the Cache is more likely to retain pages for subsequent update, particularly when processing NPIs – Recommendation: set to YES
  • 60. ©2010 SoftwareOnZ63 DSNZPARMs • MGEXTSZ YES | NO – Whether or not DB2 will manage Sliding Extents? – Note: the default changed to YES in DB2 9 (it was NO in DB2 V8) • TSQTY 0 and IXQTY 0 – Sets default object sizes if USING clause omitted – Default values use 1 cylinder except for LOBs which use 10 cyls • DSVCI YES | NO – Whether to use variable CI sizes – Note this is turned on by default when you install V8 • SVOLARC YES | NO – YES means allocate a single volume for disk based archives • Saves space if you use SMS guaranteed space option
  • 61. ©2010 SoftwareOnZ64 DSNZPARMs SMS Related DSNZPARMs • SMSDCFL – Specifies a DFSMShsm data class for table spaces. – If you assign a value to SMSDCFL, DB2 specifies that value when it uses Access Method Services to define a data set for a table space. • SMSDCIX – Specifies a DFSMShsm data class for indexes. – If you assign a value to SMSDCIX, DB2 specifies that value when it uses Access Method Services to define a data set for an index.
  • 62. ©2010 SoftwareOnZ65 Wrapping It All Up • Questions to ask yourself • Thoughts and Ideas
  • 63. ©2010 SoftwareOnZ66 Storage Problems • Out of space • Allocation issues • Growing very rapidly • Too many extents • Approaching thresholds • Orphaned image copies • My index keeps growing!
  • 64. ©2010 SoftwareOnZ67 Can You Answer These Questions? • My volumes are SMS managed; how can I tell what my volume names are, how much space I have left, and how much space I have used? • Can you rely on the information in the RTS? – The EXTENTS column in the RTS will be updated only when an update or applicable utility is run for the object. A simple start after extent reduction or a read based on SELECT will not update the EXTENTS column (same issue as the catalog).
  • 65. ©2010 SoftwareOnZ68 Examples of DB2 Storage “Things” You Should Be Monitoring 1. Space display and monitoring of a DB2 system. 2. Space display and monitoring of Databases. 3. Space display and monitoring of the Tablespaces (and Indexes). 4. Display and monitoring of the Storage Groups and the associated volumes of a DB2 system. (Data, Workfile, Image Copies, Logs, Archives, Sort/Work etc.) 5. VSAM display of all Tablespaces and Indexes (Used, Allocated, Primary and Secondary Quantity, Volumes) and monitoring of Extents (physical and logical). 6. Display of the Linear Page Sets of Tablespaces and Indexes, that reach their maximum size (part. objects, DBSIZE) resp. the maximum number of datasets (non- part. objects, PIECESIZE). 7. Intelligent HSM Migration of Image Copy Backup Data Sets, that are not used for a local recovery (to CURRENT), as well as datasets older than the last Full Image Copy (Dual and Remote Backups) 8. Delete of Image Copy Backup datasets of a DB2 system that are no longer needed because of DROP, DROP/CREATE or MODIFY TABLESPACE (‘orphaned‘, not listed in SYSIBM.SYSCOPY).
  • 67. ©2010 SoftwareOnZ70 70 Growth of DB2 & DB2 Objects Screen(s) shot source: zSPM
  • 68. ©2010 SoftwareOnZ71 My Index Keeps Growing… Help! • For an index, deleted keys are marked as pseudo deleted. – Actual cleaning up occurs only during certain processes (for example, before a page split). – This can cause indexes to “grow” even when the data technically is not. – It can also result in high CPU cost for index scans. • Whenever an index scan is invoked, DB2 will scan all entries in the index including pseudo-deleted entries that have yet to be removed. • Calculate the percentage of pseudo-deleted RIDs using information in SYSINDEXPART: – (PSEUDO_DEL_ENTRIES/CARDF)*100 • Reorg the index when pct of pseudo-deleted entries is: – Greater than 10% for non Data Sharing – Greater than 5% for Data Sharing. Problem: My index keeps on growing and tripping extents, even after deletes. What gives? How can I control the extent growth?
  • 69. ©2010 SoftwareOnZ72 What About Compression? • CPU vs. I/O • DSN1COMP – Anything less than 40%, is probably not worth doing. • Compression dictionary is above the 2GB bar as of V8 – Allows for more table spaces that can be compressed
  • 70. ©2010 SoftwareOnZ73 Best Practices • Keep up-to-date on DB2/storage functionality. • Perform regular and proactive monitoring. • Build alerts to inform you of problems, shortages, and potential errors. • When possible, take automated action in case of shortages, potential errors, superfluous data sets, etc.
  • 71. ©2010 SoftwareOnZ74 References • Advanced Computer & Network Corporation, RAID.edu, http://www.acnc.com/04_00.html • IBM Manual, DFSMS Introduction (SC26-7397) • IBM Manual, DFSMS Storage Administration Reference (for DFSMSdfp, DFSMSdss, DFSMShsm) (SC26-7402) • IBM Manual, DFSMS Implementing System-Managed Storage (SC26-7407) • IBM RedBook, Storage Management with DB2 for OS/390 (SG24-5462) • IBM RedBook, Maintaining Your SMS Environment, (SG24-5484 • IBM RedPaper, Disk Storage Access with DB2 for z/OS (redp4187)) • IBM RedPaper, DB2 for OS/390 Performance on IBM Enterprise Storage Server (redp0026) • IBM RedPaper, How does the MIDAW Facility Improve the Performance of FICON Channels Using DB2 and other workloads? (redp4201) • IBM System z Storage Management Strategy ftp://ftp.software.ibm.com/software/systemz/pdf/whitepaper/IBM_System_z_Storage_Management_Strategy_v1.1.pdf • IBM DS8000 Storage Virtualization Overview ftp://public.dhe.ibm.com/common/ssi/sa/wh/n/diw03020usen/DIW03020USEN.PDF • Iczkovits, John, DB2 and Storage Management: a Guide to Surviving a Perfect Marriage, SHARE February 2008 • Mullins, Craig S., DB2 Developer’s Guide, 5th edition, Indianapolis, IN: SAMS Publishing: ISBN 0-672-32613-4 • Smith, Ellin, DB2 UDB on z/OS Using EMC Symmetric Storage Systems Solution Guide, EMC Corporation, ISBN 9781604613407 • Thomas, Steve, Do Extent Numbers Really Matter Any More?, IDUG Europe 2008, Session A14 • Toigo, Jon William, The Holy Grail of Data Storage Management, Upper Saddle River, NJ: Prentice-Hall: ISBN 0-13-013055-9
  • 73. ©2010 SoftwareOnZ76 SoftwareOnZ • SoftwareOnZ specializes in optimizing your usage of System z. Our software solutions will help reduce the amount of time, effort, and human error involved in implementing and maintaining a cost-effective mainframe computing environment. Whether you are a DBA, a capacity manager, a systems programmer, an application developer or a development manager, SoftwareOnZ has a solution for improving your workload and reducing your computing costs. • Our solutions can help you to more effectively manage vWLC, improve the administration and performance of your DB2 database systems, enhance the visibility of the impact of application changes, as well as improve your overall system and network operations. • At SoftwareOnZ we really do offer “a better way on z…” http://www.softwareonz.com

Editor's Notes

  1. Source for the CAGR was from an IBM presentation citing “IBM and Industry Studies” as the source
  2. DB2 uses the bootstrap data set (BSDS) to manage recovery and other DB2 subsystemwide information. The BSDS contains information needed to restart and to recover DB2 from any abnormal circumstance. For example, all log data sets (active and archive) are automatically recorded within the BSDS. While DB2 is active, the BSDS is open and is updated.
  3. The I/O supervisor enqueues this request on a device number for the channel subsystem. The channel program consists of standard commands, described in the ECKD disk architecture, that specify I/O demand to the control unit. The control unit executes these commands, propagates them and controls their requests to logical volumes and physical devices. It also manages data delivery to the channel subsystem. The channel subsystem manages the transfer of channel commands and of data through links to the control unit. This linking can be complex and involves ESCON Directors, channel extenders, and even telecommunication devices for remote I/Os. There are two views of a disk control unit. Physically it is a storage server to which disk drives are attached and the channel links from hosts are connected. The storage server contains all the facilities to perform the I/O operations. Logically the disk control unit is an aggregate of subunits known as logical control units (LCUs) or control unit images, doing the I/O operations.
  4. A base table exists with the data set name *I0001.*. The table is cloned and the clone’s data set is initially named *.I0002.*. After an exchange, the base objects are named *.I0002.* and the clones are named *I0001.*. Each time that an exchange happens, the instance numbers that represent the base and the clone objects change, which immediately changes the data contained in the base and clone tables and indexes.
  5. OK… so where can you find DB2—related storage information? How about the DB2 Catalog? STOSPACE RUNSTATS Real-time Statistics
  6. Installations that are SMS managed can define STOGROUP with VOLUMES(*). This specification implies that SMS assigns a volume to the table and index spaces in that STOGROUP. In order to do this, SMS uses ACS routines to assign a Storage Class, a Management Class and a Storage Group to the table or index space. SMS Storage Groups: • Cannot share a volume. • Cannot share data sets. • Must contain whole volumes. • Must contain volumes of the same device geometry. • Can contain multi-volume data sets. • Must contain a VTOC and a VVDS.
  7. Volume separation is easy when you have hundreds of volumes available. But this separation is good only if your volumes have separate access paths. Path separation is important to achieve high parallel data transfer rates. Without DFSMS, the user is responsible for distributing DB2 data sets among disks. This process needs to be reviewed periodically, either when the workload changes, or when the storage server configuration changes. With DFSMS, the user can distribute the DFSMS Storage Groups among storage servers with the purpose of optimizing access parallelism. Another purpose could be managing availability for disaster recovery planning. This can be combined with the previous purpose by letting DFSMS automatically fill in these Storage Groups with data sets, by applying policies defined in the automatic class selection routines.
  8. You are not required to convert existing DB2 data sets to an SMS environment before migrating to Version 10. These data sets can indefinitely remain non-SMS-managed, but they will be converted to SMS management when the related table space is reorganized. The SMS environment that you use for DB2 catalog and directory data sets must be established before you begin migration to Version 10. The SMS environment must include a data class for allocating the data sets in extended format and using extended addressability. To define a data class with this attribute, specify EXT in the DATA SET NAME TYPE field of the DATA SET CLASS DEFINE panel of ISMF. Then, ensure that the automatic class selection (ACS) routine associates the DB2 catalog and directory data sets with this data class.
  9. Many customers do not assign any Data Class for their DB2 data sets. Storage Administrators can set up many different Data Classes based on user requirements. Some of the most common reasons to use a Data Class in a DB2 environment include -- Enabling EF and/or EA; Bypassing the 255 extent rule for data sets; Bypassing the 5 extent rule for allocations of data sets; Reducing space requirements when no volume meets the space requirement; VSAM and/or sequential striping (Striping requires the Storage Class use of SDR as well); Allocating data sets with common DCB and/or space characteristics; Specifying additional volumes for DB2 or utility data sets; Specifying different data sets types, such as PDSE, large format, etc. In the past Storage Administrators would have to spend a significant amount of time planning out Storage Class requirements. The requirements involved such things as the speed of storage devices, amount of cache available, if the disk box allowed for concurrent copy or FlashCopy, as well as other issues relating to the disk box itself. Generally, in today’s technology most of the Storage Class information is no longer required. The three most common options for a DB2 environment are: Enabling Guaranteed Space; Set striping (Striping also requires an associated Data Class with EF enabled); Enabling the use of multi-tiered Storage Groups. Multi-tiered Storage Groups are specified in the Storage Group, however you must still enable them in the Storage Class. Management Class can be used for a variety of functions: Expire data sets; Specify if a user or Data Class can specify retention period; Partial release of data sets (For DB2 LDSes, partial release is for data sets with EF enabled, but without Guaranteed Space); Migrate to Level 1 for data sets not used for a specific time; Migrate to Level 2 for data sets not used for a specific time; Migrate from Level 1 to Level 2 for data sets not used for a specific time; Specify if migration should be automatic, by command, or both; Migrate based on the number if GDG objects; Determine action for rolled off GDG objects. Storage Groups contain: Volsers - a volume can only belong to one Storage Group; Determining if DFSMShsm migration, dump, or incremental backup should be enabled; Allow a Storage Group to be an overflow volume; Allow a Storage Group to be an extend Storage Group (An overflow Storage Group can also be an extended Storage Group); Specify a Copy Pool backup Storage Group name; HIGH and LOW values for the Storage Group; Specify a Break Point Value (BPV) for EAV devices.
  10. The DS8000 is the most popular disk system for IBM System z mainframe platforms. It is built for activities like Online Transaction Processing (OLTP) and large databases, supporting ESCON and FICON attachment to high-speed 15K RPM FC drives.
  11. All the latest Disk Storage arrays still emulate 3390 track architectures as that’s what z/OS understands. You may well have heard of your site using 3390 Model 3 disks. The Model number refers to the old triple capacity 3390 models, which could hold just under 3Gb of data. In fact modern arrays can emulate almost any volume configuration, up to an architectural limit of 65,520 cylinders. Recently z/OS 1.10 added support for Extended Address Volumes which will be supported by IBM DS8000 arrays and by other vendors. The EAV architecture supports a huge maximum theoretical volume size, although in the initial implementation the limit is 262,668 cylinders per volume or approximately 223Gb.
  12. that combines multiple disk devices into an array that is perceived by the system as a single disk drive. There are many levels of RAID technology that deliver different degrees of fault-tolerance and performance.
  13. RAID 0 - data striping without parity RAID 1 - dual copy RAID 2 - synchronized access with separate error correction disks RAID 3 - synchronized access with fixed parity disk RAID 4 - independent access with fixed parity disk RAID 5 - independent access with floating parity RAID 6 - dual redundancy with floating parity RAID 10 (DS8000 and some ESS) - RAID 0 + RAID 1, no parity Parity is additional data, “internal” to the RAID subsystem, that enables a RAID device to regenerate complete data when a portion of the data is missing. Parity works on the principle that you can sum the individual bits that make up a data block or byte across separate disk drives to arrive at an odd or even sum.
  14. From a conceptual point, the function of cache and buffer pools are similar. • From a storage perspective, DB2 data is typically considered “unfriendly” because of the relatively low reuse of data in cache. • DB2 will use the data residing in the buffer pool when available. It may not require the data in disk cache at all. Reading from cache is exponentially faster than from disk. • No need to go to disk, find the data, and bring it back through cache. • Just because your buffer pool casts out data, it does not mean that it is no longer retained in cache. • Newer disk controllers have very large cache sizes and can retain data forlonger periods.
  15. FICON – FIBRE CONNECTIVITY MIDAW is essentially a technical improvement to the Channel Instruction. It allows Media Manager to fully exploit the Track Level command operations of z/Architecture • Reduces number of Control Words required for an I/O • For an EF dataset reduces from 24 Control Words to 1 • Using MIDAW reduces EF dataset performance penalty • Requires z9 processor or above • Requires z/OS 1.7 or (retrofitted to 1.6 with APAR)
  16. Prior to MIDAWs, the maximum log throughput using the DS8000 and FICON Express 2 was 84 MBps, and striping the log increased the bandwidth only slightly. Nobody should ever allocate their logs as EF with a single stripe, because there is no advantage to doing so. (DB2 does not support Extended Addressability for log data sets.) Nevertheless, Figure 12 on page 13 shows how a single-stripe EF log would perform, so that we can compare two stripes to one stripe. Given a log with two stripes, Figure 12 on page 13 shows that MIDAWs increased the log bandwidth by 31%, reaching 116 MBps with two stripes.
  17. Figure 6 shows the I/O response times of DB2 prefetch I/Os with 4 KB pages, with and without MIDAWs, for both EF and non-EF data sets. Prior to MIDAWs the response time for EF was 1.9 ms versus 1.2 ms for non-EF data sets. MIDAWs did not improve the response time for non-EF data sets, but MIDAWs did lower the response time of EF data sets such that EF and non-EF data sets performed identical to each other. The response time of non-EF data sets remained at 1.2 ms. MIDAWs reduce the number of CCWs per track for EF data sets from 24 to 1. It would be easy to presume that reducing the number of CCWs is the only contributing factor to improving performance. However, MIDAWs also reduce the number of CCWs per track for non-EF data sets from 12 to 1, yet non-EF response time does not change. On the other hand, the channel utilization for non-EF data sets was reduced by half, from 51.5% to 26.2%, as shown in Figure 7.
  18. ROT – less than 10 extents per data set
  19. the table space or index space has a SECQTY setting of greater than zero, the primary space allocation of each subsequent data set is the larger of the SECQTY setting and the value that is derived from a sliding scale algorithm.
  20. All this sliding scale information is in Chapter 3 of the DBA Guide
  21. Beware REORG with manual allocations and multiple extents! Better results if the volume is not fragmented
  22. the clone is created in the same table space as the base table, but in a different VSAM dataset. A page set normally has the format catname.DSNDBx.dbname.spname. I000y.A001 It's the I000y portion of the page set name that we are interested in. The base table will have I0001 in the VSAM name while the clone will have I0002. The SQL EXCHANGE statement flips the VSAM datasets.
  23. Define the data classes for your table space data sets and index data sets. Code the SMS automatic class selection (ACS) routines to assign indexes to one SMS storage class and to assign table spaces to a different SMS storage class.
  24. Note: The EXTENTS column in the RTS will be updated only when an update or applicable utility is run for the object. A simple start after extent reduction or a read based on SELECT will not update the EXTENTS column (same issue as the catalog). 48 Disk storage access with DB2 for z/OS
  25. DFSMS Storage Administration Reference (for DFSMSdfp, DFSMSdss, DFSMShsm) SC26-7402