Your SlideShare is downloading. ×
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
zHE Roadshow SWG Presentation
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

zHE Roadshow SWG Presentation

513

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
513
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. DB2 for z/OS PureXML Overview and Performance Practices Akiko Hoshikawa, DB2 for z/OS Performance Silicon Valley Lab
  • 2. Best Performance Practices Overview of pureXML support in DB2 9 for z/OS ► Quick view of what is XML? ► Understand the physical structure of XML support Best Practices for XML Performance ► Best Practices for Inserting XMLs ► Best Practices for XML Queris ► Best Practice for Cost Reduction ‒ XML workload and zAAP redirection ‒ XML workload and zIIP redirection 2
  • 3. What is pureXML? XML native storage Query Capability Indexing Inside of XML Documents Publishing Relational Data as XML DB2 Engine XML Schema Validation Capability Database Utilities and Monitoring XML Application Textual XML Relational XSR 3
  • 4. The XQuery Data Model <book> <authors> <author id=“47”>Donald Duck</author> <author id=“58”>Peter Pan</author> XML Parsing </authors> <title>Database systems</title> <price>29.95</price> <keywords> book <keyword>SQL</keyword> <keyword>relational</keyword> </keywords> authors title price keywords </book> Database author author 29.95 keyword keyword Systems Serialization id=47 Donald Duck id=58 Peter Pan SQL relational 4
  • 5. XMLQUERY, XMLEXISTS, XMLTABLE with XPath XMLQUERY is best to find a specific part of a XMLTABLE is best to find common parts of specific document. potentially many documents or to iterate through the result. XMLEXISTS is used for filtering documents last first SELECT XMLQUERY(‘/book/price’ PASSING book) Donald Duck FROM books WHERE XMLEXISTS(‘/book[title=“Database systems”]’ Peter Pan PASSING book); SELECT XT.* FROM book B, XMLTABLE(‘/book[title=“Database systems”]/author’ PASSING B.book COLUMNS last VARCHAR(50) PATH ‘last’, <price>29.95</price> first VARCHAR(50) PATH ‘first’ ) XT ORDER BY XT.last; 5
  • 6. XML Storage on Mature Infrastructure DocID index XML index (user) NodeID index B+tree B+tree B+tree B+tree B+tree B+tree Base Table Regular Internal XML Table Table space DocID … XMLCol DOCID MIN_NODEID XMLData (DB2_GENERATED_DOCID_FOR_XML) 1 1 02 2 2 02 3 2 0208 3 02 A table with an XML column has a DocID column, used to link from Each XMLData column is a VARBINARY, the base table to the XML table. containing a subtree or a sequence of A DocID index is used for getting subtrees, with context path. Rows in XML to base table rows from XML table are freely movable, linked with a indexes. NodeID index. 6
  • 7. Create XML column CREATE TABLE PROD(PID INT, PNAME VARCHAR(20), XMLCOL XML); CREATE INDEX My_index ON PROD (PID); DOCID_IX Index : My_index Node_ID_IX Segmented TS Universal TS Table : Prod Partition By Growth Table :XProd Column : PID, PNAME, DB2_GENERATED_DOCID_FOR_XML 7
  • 8. DBA’s Questions on XML table What is XML table space? How does it grow? What is difference from LOB? ► Base : Partitioned or Range Partitioned UTS ‒ XML : Range Partition UTS ► Base : Segmented or Partitioned By Growth UTS ‒ XML : PBG UTS ► Page size 16K 8
  • 9. A few Things that are New related to XML 1. Implicitly created objects – DBA cannot create explicitly 2. XML indexes: XPath and keys 3. XML Schema registration 4. XMLDATA contains StringIDs in a catalog table SYSIBM.SYSXMLSTRINGS (dictionary) • UNLOAD FROMCOPY restricted, DSN1COPY 5. A new XML lock type, ID '35'x in traces • IFCID 20, 21, 107, 150, 172, and 196. 6. XML keyword in some utilities 9
  • 10. LOB or XML ? LOB XML LOB space Universal Table Space ► No Compression ► Compression Flow optimization ► Append ► Plus zIIP DRDA Flow optimization + streaming Low CPU Cost in Insert/Update ► Plus zIIP DRDA Low CPU in select whole document High cost in Insert/Update LOB Lock Avoidance ► zIIP/zAAP eligible for parser No XML Query Validation No Index XML Index XML Query using XPath, SQL/XML XML Table 10
  • 11. LOB and XML BASE table LOB Aux table (4K,8K,16K,32K) row 1 LOB 1 LOB 2 LOB3 row 2 row 3 BASE table XML table(16K) row 1 XML 1 XML 3 row 2 XML 2 row 3 11
  • 12. Best Practice 1 : Choose “right” XML size Pick your XML size with your unit of UPDATE ► Liner cost on Insert/Serialization cost ► Update is an entire document replacement ► Concurrency ‒ New XML lock for document level concurrency Elapsed CPU 300 250 Time (Sec) 200 150 100 50 0 1K x 1000000 10K x 100000 100K x 10000 1M x 1000 10M x 100 Doc Size x Number 12
  • 13. XML Concurrency Document level concurrency by XML lock – X’35’ lock ► Before PK55966 (Dec/07) There were additional s-page locks on XML table ► Before PK72604/PK68265(plan 4Q/08) There are additional x-page locks on XML table DML Base Table Lock Base Lock XML Lock size XML Lock XML table Lock Size X’35’ INSERT PAGE/ROW X Document X (commit) Page latch UPDATE/ PAGE/ROW S->X Document X (commit) Page latch DELETE U->X X SELECT with PAGE/ROW S (release at Document S (next fetch) None CS, CD(NO) next fetch) or lock avoidance 13
  • 14. Best Practice 2 : Improve Insert Performance Insert performance considerations ► Logging - hidden log cost ► Spread your insert to multiple partitions if possible ► Define only the necessary indexes /a/b/c/@d rather than //@d 160% 138% 140% 125% 111% 116% 120% 100% 100% 100% XML 80% w/1 index 60% w/ 2index 40% 20% 0% Elapsed CPU 14
  • 15. Best Practice 3: Use Compression for Preserving White Space 180 160 140 120 Size in KB 100 70% 80 saving 60 80% 40 saving 20 0 Original XML XML Original XML XML Docs Strip Prerv Docs Strip Prerv WS WS WS WS Unifi-messages JPM Non compressed Compressed 15
  • 16. Best Practice 4: Be Aware of Cost of Validation Insert using UDF (SYSFUN.DSN_XMLVALIDATE function) ► Register XML schema in DB2 XSR ► Current limitation doc size 50MB Insert into Prod (pid, pname,xmlcol) values ( 12345, “Minnie Mouse”, XMLPARSE(DOCUMENT SYSFUN.DSN_XMLVALIDATE (CAST ? AS CLOB), 'SYSXSR.PRODCT'))) Typically 1.8- 3x CPU increase from non-validation case ► Future plan ‒ Remove 50MB limitation ‒ Support of Validation via Specialty Engines (zIIP/zAAP) 16
  • 17. UNIFI Validation 50K insert Validation: UNIFI documents Non_Val EL Non-Val CPU Val-EL Val-CPU 140.00 120.00 Time in second 100.00 80.00 60.00 40.00 20.00 0.00 3 4 2 7 8 8 7 2 1 00 00 00 00 00 00 00 00 00 in in in in cs cs cs cs cs pa pa pa pa pa pa pa pa pa Note : UNIFI (International Standard ISO 20022 – UNIversal Financial Industry message scheme ) UNIFI samples 17
  • 18. Insert and Validation : Multi threads TPoX Insert 20 threads 6000 5000 4000 throughput ETR 3000 ITR 2000 1000 0 Order Order_Validate CustAcc CustAcc_Validate 18
  • 19. Insert or Shred? 4K doc single thd, 3000x Insert Insert with Validation CPU Elapsed Decomposition 0 2 4 6 8 10 12 Time in Second 19
  • 20. Quick Shredding – XMLTABLE <product> <item item_id = 777> <name>Magic Pen</name> <category> Stationary </category> <manufacture > <name> “ABC” </name> </manufacture> <price> 5.00 </price> SELECT T.* <comment> erasable pen </comment> FROM Prod X, </item> </product > XMLTable (‘//item’, PASSING X.XMLCOL COLUMNS “Item_ID” INTEGER PATH ‘@item_id’, “Product Name” CHAR(20) PATH ‘name’, “Manufacture” CHAR(20) PATH ‘manufacture/name’, “US Price” DEC(9,2) PATH ‘price’, “Comment” CHAR(80) PATH ‘comment’ WITH ORDINALITY "Seqno") AS T WHERE X.PID = ? ; 20
  • 21. Best Practice 5: Use XML Publishing (V8 NFM) 30 25 20 15 10 5 0 1M doc 2M doc 3M doc 4M doc 5.8M doc COMP Elapsed SQL/XML Elasped 21
  • 22. Best Practice 6: Speed up with XML Index Assuming there are 100,000 rows (100,000 XML doc) in the table <product> <item item_id = 777> <name>Magic Pen</name> <category> Stationary </category> SELECT .. WHERE XMLEXISTS <manufacture > (‘/product/item[@item_id = 777]’…); <name> “ABC” </name> </manufacture> <price> 5 </price> <comment> erasable pen </comment> </item> </product > INDEX on '/product/item/@item_id' Or ‘//@item_id’ AS SQL DECFLOAT ; QNO A_TYPE A_NAME MC PREFETCH 1 DX IX1 1 L 22
  • 23. Speed up with XML Index <product> <item item_id = 777> XMLEXISTS <name>Magic Pen</name> <category> Stationary </category> (‘/product/item[@item_id = 777]’…) <manufacture > AND XMLEXISTS <name> “ABC” </name> (‘//manufacture[name=“ABC”] ‘…) ; </manufacture> <price> 5 </price> <comment> erasable pen </comment> </item> </product > INDEX on '/product/item/@item_id' AS DECFLOAT; INDEX on ‘//manufacture/name’ AS SQL VARCHAR(26) ; QNO A_TYPE A_NAME MC INDX_ONLY PREFETCH 1 M 0 N L 1 DX IX1 1 Y 1 DX IX2 1 Y S 1 DI 0 N 23
  • 24. Speed up with XML index Having a good XML index is critical for query performance No Index XML index DocScan 90 80 Doc DX Scan 70 (DocID 60 list 50 access DX/DI 40 DocID 30 Anding 20 10 Base IX access0 TPQ0101 TPQ0103 TPQ0104 TPQ0201 TPQ0400 TPQ0403 TPQ0405 24
  • 25. Best Practice 7 : Code Specific XML path and define “just right” index <product> Filtered by <item item_id = 777> <name>Magic Pen</name> /product/item/manufacture/name <category> Stationary </category> <manufacture > <name> “ABC” </name> </manufacture> <price> 5 </price> Indexed by <comment> erasable pen </comment> /product/item/manufacture/name </item> </product > Indexed by //name 25
  • 26. XML Index Create or Rebuild Create Elapsed Create CPU Rebuild Elapsed Rebuild CPU 140 120 Time in second 100 80 60 40 20 0 //e' /a/b/@c' /a/b/f/g' //e /a/b/@c /a/b/f/g 26
  • 27. Best Practice 8 : RUNSTATS on XML Table and Indexes RUNSTATS against XML table is “required” ► LISTDEF LISTDEF LIST01 INCLUDE TABLESPACES DATABASE XMLDB RUNSTATS TABLESPACE LIST LIST01 TABLE(ALL) INDEX(ALL) ► RUNSTATS does not scan XML documents. ‒ Collect basic information (number of pages, card) 27
  • 28. Summary for XML Access and Index Usage Indexes are only for XMLEXISTS predicate Use fully specified paths than // or * in your queries Index type and Namespaces in Index and Query needs to match Index and query containment ► //d index can be used for XMLEXISTS /a/b/c/d or //d ► /a/b/c/d index can be used for /a/b/c/d but not for //d Insert overhead ► //d index may be more expensive to maintain than /a/b/c/d Do not forget RUNSTATS 28
  • 29. Best Practice 9 : UNLOAD to HFS LOAD/UNLOAD using File Reference Variables ► XML > 32KBytes TEMPLATE TCLOBF DSN /u/akiko/xml3/M25/EBCDIC DIR(10) DSNTYPE(HFS) UNLOAD TABLESPACE XMLDB.TS25M FROM TABLE USRT014.TB25M (XML POSITION(*) VARCHAR CLOBF TCLOBF ) EBCDIC Avoid zFS ► zFS has known performance issue when single directory contains millions of files. 29
  • 30. Best Practice 10 : Monitor XML Performance Traditional Performance monitoring tools can be used for XML performance monitoring. ► DB2 Accounting, Statistics trace and RMF ► Access path check – OSC or plan tables Monitor XML storage consumption periodically. ► Accounting/Statistics will have XML storage usage section ► Zparm XMLVALA, XMLVALS to control XML storage 30
  • 31. Best Practice 11 : Stay Current on XML Maintenance XML APARs ► XMLTABLE function by PK51571, PK51572, PK51573 ‒ XMLTABLE and XMLCAST support ‒ XML Query various performance and storage optimization ► XML LOAD Performance improvement by PK47594 ► Additional BIF by PK55585/PK55831 ► XML query performance improvement by PK58914, PK56337 ► XML access path by PK57158 ► XML storage zparm by PK25747, XML thread storage monitor by PK54837 ► XML concurrency improvement by PK52885, PK55966, PK59770, PK68265 ► XML insert with index performance improvement by PK66218 Universal table space – PBG APAR ► PGB concurrency improvement by PK72414 PK72415 PK72417 PK72419 PK72420 XML System Service - up to 30% CPU reduction in parsing by z/OS 1.9 or above 31
  • 32. Best Practice to Reduce Cost of XML : DRDA and XML System Service Redirection DB2 zIIP support will apply with XML process ► DRDA zIIP ► Utility index zIIP (Only base table indexes) Non Validation Parser zAAP support ► 100% of XML System Service under TCB mode is eligible ► z/OS 1.9 above and 1.8/1.7 with APARs ► PK50575 for zAAP accounting support Non Validatoin Parser zIIP support ► 100% of XML System Service under SRB mode is eligible ► z/OS 1.10 and 1.9/1.8 with APARs Validation Parser zIIP support - planned in V9 maintenance stream 32
  • 33. What is Eligible on Which IBM Specialty Engines? DB2 9 XML workload invoked DRDA® via TCPIP Execution zAAP zIIP from . . . Mode eligible eligible CICS® No TCB Yes No IMS™ No TCB Yes No TSO No TCB Yes No WebSphere® for z/OS (JCC T2) No TCB Yes No Stored Procedure/UDF/Trigger No TCB Yes No Native SQL Stored Procedure No TCB Yes No Call Attach (CAF) No TCB Yes No RRS Attach No TCB Yes No Local LOAD utility (data) No TCB Yes No Index Build phase of LOAD * No SRB No Yes WebSphere for z/OS (JCC T4) Yes SRB No Yes WebSphere distributed (JCC T4) Yes SRB No Yes Distributed transactions using DB2 Yes SRB No Yes Connect™ Stored Procedure/UDF/Trigger Yes TCB Yes No Native SQL Stored Procedure Yes SRB No Yes 33
  • 34. zIIP/zAAP Monitoring RMF WLM Activity Report DB2 Accounting : Combined zIIP/zAAP counter (PK50575 5/08) AVERAGE APPL(CL.1) DB2 (CL.2) ------------ ---------- ---------- CP CPU TIME 0.006263 0.005252 AGENT 0.006263 0.005252 NONNESTED 0.001553 0.000560 STORED PRC 0.004660 0.004660 UDF 0.000049 0.000032 Eligible for zIIP but did TRIGGER 0.000000 0.000000 not run on zIIP PAR.TASKS 0.000000 0.000000 (same as IICP) SECP CPU 0.001739 N/A SE CPU TIME 0.003546 0.002318 Time spent in Specialty NONNESTED 0.001580 0.000351 Engines (zIIP + zAAP ) STORED PROC 0.001967 0.001967 34
  • 35. XML System Service Specialty Engine Support XML System Service in Insert and Load zAAP eligble CP 16 14 e s r sr C Utim - m fo in e 12 - s cfo lo d r a 10 8 14% 48% e 6 32% 4 P 2 0 Order_insert (1- CustAcc_insert (4- LOAD (2.5MB) 2K) 20KB) 35
  • 36. XML System Service Specialty Engine Support XML System Service zIIP usage- Insert thru DRDA Eligible 62.5% Actual 100% 56.4% 80% non zIIP eligible Eligible 60% 54.3% zIIP eligible 40% Actual %of Acutal Offload 49.8% 20% 0% DRDA - only DRDA + XML 36
  • 37. LOB vs. XML with zIIP 35 30 25 e(sec) LOB CPU 20 XML CPU Tim XML Charged CPU 15 LOB Charged CPU 10 5 0 25MB 250MB 372MB 500MB 744MB DocSize 37
  • 38. zIIP Tuning Practice Avoid zIIP Constraint situation ► Add more zIIP processors ► ZIIPAWMT (ZAAPAWMT for zAAP) in IEAOPTxx ‒ Default 12000 (12 ms) ‒ Impact when the specialty engine will ask for help from the general CPs. ‒ If response time is growing and you want the general CPs to help with more work, lower to 1000 or 500 (z10) ‒ If response time is fine, use default 12000 38
  • 39. Best Practices…. 1. Choose right XML size 2. Insert Performance 3. Consider Compression 4. Be Aware of Cost of Validation 5. Use XML Publishing Function for composition 6. Speed Up with XML Indexes 7. Code Specific XML path and “right” level path for index 8. RUNSTATS on XML Tables and Indexes 9. Unload to HFS 10. Monitor XML Performance 11. Stay Current on XML maintenance 12. Utilize IBM Specialty Engines for XML applications 39
  • 40. Resources DB2 9 and z/OS XML System Services Synergy Update ► http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101227 DB2 9 for z/OS Performance Redbook ► http://www.redbooks.ibm.com/redbooks/pdfs/sg247473.pdf). DB2 Version 9.1 for z/OS XML Guide, SC18-9858 XML topics in DB2developerworks ► http://www-128.ibm.com/developerworks/wikis/display/db2xml/Home 40
  • 41. Disclaimer © Copyright IBM Corporation [current year]. All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE. IBM, the IBM logo, ibm.com, DB2, PureXML are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml 41
  • 42. Thank you! akiko@us.ibm.com XML Questions ? Send them to db2zxml@us.ibm.com 42

×