Paralellism with DB2 for z/OS (2010)
 

Paralellism with DB2 for z/OS (2010)

on

  • 2,098 views

Parallelism was first introduced to DB2 way back in 1993 with DB2 Version 3. With every release of DB2 parallelism has been enhanced. In applications like data warehousing and business intelligence, ...

Parallelism was first introduced to DB2 way back in 1993 with DB2 Version 3. With every release of DB2 parallelism has been enhanced. In applications like data warehousing and business intelligence, it's almost a necessity. Yet, a surprisingly large number of customers continue to avoid parallelism when it could offer significant elapsed time improvements. With this presentation, we'll try to debunk the myths that surround using DB2 parallelism. We will discuss a little bit of parallelism's history, how parallelism works, the parameters in DB2 that control parallelism and the affects of the value chosen for those parameters, and how to get the greatest benefits when parallelism is put to use; the DOS & DON'TS necessary for you to get the most out of DB2's parallelism. We will also discuss the latest enhancements to parallelism and how parallelism can take advantage of zIIP specialty engines.

Statistics

Views

Total Views
2,098
Views on SlideShare
2,092
Embed Views
6

Actions

Likes
1
Downloads
51
Comments
0

2 Embeds 6

http://www.williefavero.com 5
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Paralellism with DB2 for z/OS (2010) Paralellism with DB2 for z/OS (2010) Presentation Transcript

  • Willie FaveroDynamic Warehouse on System z Swat TeamIBM Silicon Valley Laboratory Myth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation All rights reserved
  • Willie FaveroDynamic Warehouse on System z Swat TeamIBM Silicon Valley LaboratoryTaking a Peak at DB2’s Parallelism Copyright © 2010 IBM Corporation All rights reserved
  • Disclaimer The information contained in this presentation has not been submitted to any formal IBM review and is distributed on an "As Is" basis without any warranty either expressed or implied. The use of this information is a customer responsibility. The materials in this presentation are also subject to enhancements at some future date, a new release of DB2, or a Programming Temporary Fix (PTF) IBM MAY HAVE PATENTS OR PENDING PATENT APPLICATIONS COVERING SUBJECT MATTER IN THIS DOCUMENT. THE FURNISHING OF THIS DOCUMENT DOES NOT IMPLY GIVING LICENSE TO THESE PATENTS. TRADEMARKS: THE FOLLOWING TERMS ARE TRADEMARKS OR ® REGISTERED TRADEMARKS OF THE IBM CORPORATION IN THE UNITED STATES AND/OR OTHER COUNTRIES: AIX, AS/400, DATABASE 2, DB2, e-business logo, Enterprise Storage Server, ESCON, FICON, OS/390, OS/400, ES/9000, MVS/ESA, Netfinity, RISC, RISC SYSTEM/6000, iSeries, pSeries, xSeries, SYSTEM/390, IBM, Lotus, NOTES, WebSphere, z/Architecture, z/OS, zSeries, System z. THE FOLLOWING TERMS ARE TRADEMARKS OR REGISTERED TRADEMARKS OF THE MICROSOFT CORPORATION IN THE UNITED STATES AND/OR OTHER COUNTRIES: MICROSOFT, WINDOWS, WINDOWS NT, ODBC and WINDOWS 95. For additional information visit the URL http://www.ibm.com/legal/copytrade.phtml for “Copyright and trademark information”Myth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 3 of 38 All rights reserved
  • The History of DB2’s Parallelism (on the mainframe of course)Myth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 4 of 38 All rights reserved
  • Query Without Parallelism Partition Table Space Response Time SQL ResultsMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 5 of 38 All rights reserved
  • Query With Parallelism (potentially) Partition Table Space Response Time SQL ResultsMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 6 of 38 All rights reserved
  • Batch Program Parallelism Part A = 1 -10 Batch Job Part A Part B = 11- 20 Part C = 21- 30 Part D = 31- 40 Part E = 40 - ? Batch Job Part B Batch Job Part C Batch Job Part D Batch Job Part E 1/5 the time, Partition Table ≈1/5 the contention …in theory SpaceMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 7 of 38 All rights reserved
  • Parallelism modes In priority order Sysplex Eligible for zIIP redirect EXPLAIN’s PARALLELISM_MODE=‘X CPU Most Common zIIP eligible EXPLAIN’s PARALLELISM_MODE=‘C I/O Not eligible for zIIP redirect EXPLAIN’s PARALLELISM_MODE=‘IMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 8 of 38 All rights reserved
  • How is parallelism plan determined? Parallelism is considered for SELECT only In V8 and prior Optimizer chooses the lowest cost sequential plan And then determines what operations of the winning plan can be executed in parallel How to parallelize Optimizer this plan? One, lowest Parallelism cost (sequential) plan survivesMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 9 of 38 All rights reserved
  • How is parallelism plan determined? In DB2 9 Lowest cost is AFTER parallelism Only a subset of plans are considered for parallelism Parallelism How to parallelize Optimizer these plans? One Lowest cost plan survivesMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 10 of 38 All rights reserved
  • I/O Parallelism SELECT * FROM TAB1 A, TAB2 B Introduced WHERE A.COLA=B.COLZ ORDER BY A.COLM in DB2 V3 S DB2B DB2A DB2C y s p l e x 1 2 Manages concurrent I/O requests for a singleNot zIIP eligible query, fetching pages into the buffer pool in parallel. This processing can significantly 3 improve the performance of I/O-bound queries. Myth Busting DB2’s Parallelism Myth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 11 of 38 All rights reserved
  • CP Parallelism SELECT * FROM TAB1 A, TAB2 B Introduced WHERE A.COLA=B.COLZ ORDER BY A.COLM in DB2 V4.1 S DB2B DB2A DB2C y s p l e x 1 2 Enables true multitasking within a query. A large query can be broken into multiple smaller queries. These smaller queries run simultaneously on multiple processors accessing data in parallel reducing thezIIP eligible elapsed time for each query. Starting with DB2 V8, the 3 parallel queries exploit zIIPs when they are available on the system thus reducing the costs. Myth Busting DB2’s Parallelism Myth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 12 of 38 All rights reserved
  • Parallelism Modes - CPU CPU parallelism is not considered if: Only one CPU CPU parallelism disabled by RLF RLFFUNC=‘4‘ in RLF table (for plan/package/authid) Cursor declared WITH HOLD and RR or RS isolation No MVS enclave service VPPSEQT=0 Ambiguous cursor with CURRENTDATA(YES) and ISOLATION(CS) CURRENTDATA (NO) default once again as of DB2 9Myth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 13 of 38 All rights reserved
  • Data Sharing A really high level view! Member Member DB2 A CF DB2 B BSDS BSDS Logs Logs User Data Catalog DirectoryMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 14 of 38 All rights reserved
  • Sysplex Query ParallelismDB2 can split a large query across SELECT * FROM TAB1 A, TAB2 Bdifferent DB2 members in a data Introducedsharing group, known as Sysplex WHERE A.COLA=B.COLZquery parallelism. ORDER BY A.COLM in DB2 V5 S 3-Way DB2B DB2A DB2C y Data Assistant Coordinator Assistant s Sharing Group p l e x 1 2 4 5 7 8 zIIP eligible? 3 6 9 Myth Busting DB2’s Parallelism Myth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 15 of 38 All rights reserved
  • Sysplex Parallelism Considerations Sysplex parallelism is not considered if: Data sharing not supported Sysplex parallelism disabled by RLF RLFFUNC=5‘ in RLF table (for plan/package/authid) RR or RS isolation specified Sysplex parallelism degrades to CPU parallelism if: Star join query RID access or IN-list parallelism Sparse index usedMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 16 of 38 All rights reserved
  • Star Schema Dimension Table Dimension Dimension Table Table Fact Table Dimension Dimension Table Table Star schema = Snowflake schemaMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 17 of 38 All rights reserved
  • Star Schema Store Product Date Sale Three or more tables can be joined at one time DSNZPARM SJTABLES and STARJOIN must be set zIIP eligible Promotion Associate Star schema = Snowflake schemaMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 18 of 38 All rights reserved
  • Pause for QuestionsMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 19 of 38 All rights reserved
  • Myths? Parallelism doesn’t work correctly. IBM recommends “x”. Corollary 1: PARAMDEG should be… Corollary 2: Never exceed x degrees of parallelism Corollary 3: Always use degree of 0 (zero) No documentation exist for parallelism. Parallelism is difficult to control?Myth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 20 of 38 All rights reserved
  • Pause simply for effectMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 21 of 38 All rights reserved
  • Enabling Parallelism – Part 1 BIND - DEGREE ANY (not 1, the default) CDSSRDEF in macro DSN6SPRM (CURRENT DEGREE on install panel) 1 is default, Valid values are 1 or ANY, Can update online Consider taking the defaultMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 22 of 38 All rights reserved
  • Enabling Parallelism – Part 2 Buffer Pool Affect on Parallelism VPSEQT - sequential steal threshold Percentage of the total buffer pool size (VPSIZE) % Default 80% VPPSEQT - parallel sequential threshold Percentage of the VPSEQT Default 50% VPPSEQT shuts down parallelism VPXPSEQT - assisting parallel sequential threshold Percentage of the VPPSEQT Default 0%Myth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 23 of 38 All rights reserved
  • Enabling Parallelism – Part 3 Set Parallelism’s Max. Degree Macro DSN6SPRM, keyword PARAMDEG MAX DEGREE field on install DSNTIP8 (DB2 9) Default 0 Valid range 0 to 254 Online Updatable (YES) Set the maximum degree between the # of CPs and the # of partitions CPU intensive queries - closer to the # of CPs I/O intensive queries - closer to the # of partitions Data skew can reduce # of degreesMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 24 of 38 All rights reserved
  • Disabling Parallelism Disabling parallelism: For dynamic SQL: SET CURRENT DEGREE = 1 Install value CURRENT DEGREE sets special register For DSNZPARM, CDSSRDEF in macro DSN6SPRM For static SQL: DEGREE(1) at BIND PACKAGE or BIND PLAN ALTER BUFFERPOOL(BPx) VPPSEQT(0) Resource Limit Facility (RLST) Add row for plan, package, or authid with RLFFUNC 3 disables I/O parallelism 4 disables CP parallelism 5 disables Sysplex query parallelism Need row for each type to disable all typesMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 25 of 38 All rights reserved
  • EXPLAIN PARALLELISM_MODE=I, C, or X I - parallel I/O operations C - parallel CP operations X - Sysplex query parallelismMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 26 of 38 All rights reserved
  • Parallelism and DSNZPARM PTASKROL in macro DSN6SYSP YES is default, Valid values are YES or NO, Can update online Accounting trace rollup Performance verses granularityMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 27 of 38 All rights reserved
  • DSNZPARM These ZPARMs are usually used under IBM’s guidance ZPARM PARAPAR1 More aggressive parallel exploitation for IN list predicates V7 & V8 APAR PQ87352 (April 2004) Removed in DB2 9Myth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 28 of 38 All rights reserved
  • Lower Bound To avoid parallelism for short running queries DSNZPARM SPRMPTH on DSN6SPRC macro Query block with a cost lower than SPRMPTH will not choose parallelism Default = 120 msec Parallelism 120 SPRMPTH No Parallelism APAR PQ25135 (V6), 0 APAR PQ45820Myth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 29 of 38 All rights reserved
  • Cost-Based Parallel Sort DB2 V8 introduces cost-based consideration to determine if parallel sort for single or multiple (composite) tables should be disabled Sort data size < 2 MB (500 pages) Sort data per parallel degree < 100 KB (25 pages) Hidden DSNZPARM OPTOPSE control - default is ON ON - cost-based parallel sort considerations for both single table and multi-table OFF - not cost-based - sort parallelism behavior same as V7, i.e. Single (composite) table : parallel sort Multiple tables : sequential sort only Considerations: Elapsed time improvement More usage of workfiles and virtual storageMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 30 of 38 All rights reserved
  • DB2 will use a higher degree of parallelism for read-only queries, batch jobs, and utilities when batch jobs are run in parallel and each job goes after different partitions. Parallel processing is most efficient when you spread the partitions over different disk volumes and allow each I/O stream to operate on a separate channelMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 31 of 38 All rights reserved
  • Parallelism Helps Fully Utilize zIIP SQL could be tuned to increase parallelism Parallel child task will run on zIIP Threshold exist that control when child goes to zIIP Parallel child tasks get higher % redirect than DRDA Applies to local or distributed Local non-parallel obtains 0% redirect Distributed non-parallel obtains x% redirect Parallel obtains x++% redirect Except first “y” milliseconds If you want to reduce TCO Exploit your zIIP as much as possible Key is to increase parallelism for your queriesMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 32 of 38 All rights reserved
  • Tips to Optimize Parallelism Partition size does matter; the closer in size the partitions, the better The more CPs (and zIIPs) available, the better the degree of parallelism that might be achieved Separating partitions across as many I/O devices & I/O paths as possible can improve parallelisms performance For CPU intensive queries, try to make the number of partitions and number of CPs as close as possible Place the partitions (table spaces & indexes) on separate disk volumes and separate control units if possible to minimize contention Execute RUNSTATS on a regular basis to maintain accurate partition level stats Monitor buffer pool sizes and thresholds to make sure they are adequate to maximize parallelism’s performanceMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 33 of 38 All rights reserved
  • Final Disclaimer Customers can design to encourage parallelism However, there is no guarantee that parallelism will be achieved Some restrictions on parallelism still exist DB2 Optimizer Development are working to reduce restrictions Substantial improvements in DB2 9 More improvements in DB2 10 And the future?????Myth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 34 of 38 All rights reserved
  • QuestionsMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 35 of 38 All rights reserved
  • More References My Blog: “Getting the Most out of DB2 for z/OS and System z” http://blogs.ittoolbox.com/database/db2zosMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 36 of 38 All rights reserved
  • Myth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 37 of 38 All rights reserved
  • Willie Favero Senior Certified Consulting IT Software Specialist Dynamic Warehousing on System z Swat Team IBM Silicon Valley Laboratory IBM Academic Initiative Ambassador for System z IBM Certified Database Administrator - DB2 Universal Database V8.1 for z/OS IBM Certified Database Administrator – DB2 9 for z/OS IBM zChampion wfavero@attglobal.netMyth Busting DB2’s ParallelismMyth Busting DB2’s Parallelism Copyright © 2010 IBM Corporation Slide 38 of 38 All rights reserved