Your SlideShare is downloading. ×
0
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
HALDB and OLR - IMS UG May 2013 Stockholm
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

HALDB and OLR - IMS UG May 2013 Stockholm

94

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
94
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. HALDB Online Reorganization·Click to add text5.4© Copyright IBM Corporation 2013HALDB Online ReorganizationJouko JänttiSenior IT SpecialistIMS Worldwide Technical Specialist Teamjouko.jantti@fi.ibm.com
  • 2. Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in allcountries in which IBM operates.Acknowledgements and DisclaimersThe workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. Theyare provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or otherguidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the informationcontained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsiblefor any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained inthis presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliersor licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.All customer examples described are presented as illustrations of how those customers have used IBM products and the results·Click to add text5.4© Copyright IBM Corporation 2013. All rights reserved.• U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP ScheduleContract with IBM Corp.IBM, the IBM logo, ibm.com, IMS, DB2, CICS and WebSphere MQ are trademarks or registered trademarks of InternationalBusiness Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms aremarked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered orcommon law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered orcommon law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademarkinformation” at www.ibm.com/legal/copytrade.shtmlOther company, product, or service names may be trademarks or service marks of others.All customer examples described are presented as illustrations of how those customers have used IBM products and the resultsthey may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothingcontained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by youwill result in any specific sales, revenue growth or other results.
  • 3. HALDB Online Reorganization• HALDB Overview• HALDB Online Reorganization – OLR© Copyright IBM Corporation 2013• HALDB OLR vs IMS ORF– IMS ORF refers to IMS Online Reorganization Facility for z/OSproduct in IMS Tools portfolio
  • 4. HALDB Highlights• Hierarchic structure is maintained– A database record resides in one partition• Minimal (or no) application changes required:– Cannot initially load logical child segments• New status code for load programs– Some secondary index segment formats change© Copyright IBM Corporation 2013– Some secondary index segment formats change• Might require I/O area changes when secondary index is processedas a database• HALDB database types:– PHDAM: Partitioned HDAM– PHIDAM: Partitioned HIDAM• Index is partitioned– PSINDEX: Partitioned secondary index
  • 5. High Availability Large Database - HALDB• Large Database– Databases are partitioned:• Up to 1001 partitions per database• Partitions have up to 10 data set groups• High Availability Database:© Copyright IBM Corporation 2013• High Availability Database:– Partition independence• Allocation, authorization, reorganization, and recovery are by partition– Self healing pointers• Reorganization of partition does not require changes to secondaryindexes or logically related databases
  • 6. Highlights• OSAM and VSAM (ESDS and KSDS) are supported• Partition selection is done by key range or user exit routine• Logical relationships and secondary indexes are supported– Secondary indexes may be partitioned© Copyright IBM Corporation 2013– Secondary indexes may be partitioned• DBRC is required– Databases must be registered> (IMS Catalog is a HALDB, and an exception to this)• Dynamic allocation uses DBRC information– DFSMDA is not used
  • 7. High Availability Large Database• Benefits:– Greater database capacity• Without application changes– Increased database availability:• Partitions, not databases, are removed from system© Copyright IBM Corporation 2013• Shortened reorganization process• Batch window is shortened with concurrent processing– Improved manageability• Data sets might be smaller
  • 8. HALDB SummaryHALDB Database Types are defined by using a single DBDthat creates the same structure for each partition. Thepartitions are defined for access in the DBRC Recons.Each HALDB Partition is treated as a single dataset andmay be allocated and reorganized individually, in sequential© Copyright IBM Corporation 2013groups or all together as a single database. HALDB allowsfor a maximum of 1001 partitions per database.A reorganization of the partitions does not require rebuildingof a secondary index or of logical relations. These can beself healed during regular processing.
  • 9. HALDB SummaryA single DBD defines the structure to IMS. A DBRCdefinition describes the required partitioning. Thiscombination results in a single database wheremultiple partitions remain available duringa reorganization of individual partitions.© Copyright IBM Corporation 2013Up to 1001 datasets ofup to 4 Gigabytes as partitions meansup to 4 Terabytes of data in a singledatabase structure.
  • 10. HALDB Online Reorganization• HALDB Online Reorganization (OLR) is a standard part ofIMS DB– Not a feature, product, tool, and so on• Benefits:– PHDAM and PHIDAM databases are reorganized© Copyright IBM Corporation 2013– PHDAM and PHIDAM databases are reorganized– 100% availability of database during reorganization:• Zero outages• Applications are unaffected– They never get data unavailable conditions– Full integrity and recoverability are maintained– Eliminates database outages for reorganizations
  • 11. HALDB Online Reorganization Overview© Copyright IBM Corporation 2013
  • 12. Online Reorganization Overview• Environments:– Runs in TM/DB or DBCTL system• Executes in DLISAS address space– Concurrent online and data sharing updates are allowed– XRF and RSR are supported© Copyright IBM Corporation 2013• Recoverability:– System, IMS, or media failures– DBRC support, standard recovery utilities, and DRF• Performance– External parameter for pacing
  • 13. Online Reorganization Overview• HALDB PHDAM and PHIDAM only– Reorganize by partition:• PHDAM data component• PHIDAM data component and primary index• Secondary indexes and logical relationships:– Database with secondary indexes can be reorganized© Copyright IBM Corporation 2013– Database with secondary indexes can be reorganized• But secondary index (PSINDEX) CANNOT be reorganized– Database with logical relationships can be reorganized– ILDS (ILEs) updated with new target RBAs• Restrictions– No DBD changes• (DBDS space allocation changes are OK)• (IMS 13 HALDB Alter is based on OLR)
  • 14. Each partition has an ID number– Assigned by IMS when partition is definedEach partition has an ‘data set name prefix’– Assigned by user when partition is defined– 1-37 characterData set name is formed from ‘data set name prefix’, dataPartition IDs and Data Set Names14Data set name is formed from ‘data set name prefix’, dataset letter, and partition ID– Data set names are registered in the RECON– If data set name prefix: IMSTESTS.PARTSDB– If partition ID: 00007• PHIDAM index: IMSTESTS.PARTSDB.X00007• Primary data set group: IMSTESTS.PARTSDB.A00007• ILDS: IMSTESTS.PARTSDB.L00007DSN prefix Partition IDData set letter
  • 15. Online Reorganization Technique• Online reorganization (OLR) is into new partner data sets:– A-J and X data sets alternate with M-V and Y data sets– Only one ILDS (L) per partition• Both sets of data sets are used during OLR© Copyright IBM Corporation 2013• At end of OLR, old data sets may be discarded• 100% availability of database during the reorganization:– No outages– No data set renames
  • 16. Online Reorganization Technique© Copyright IBM Corporation 2013
  • 17. ……ILDSIndexPHIDAM onlyLXLX…HALDB Naming Conventions• DDNAMEs– Partition name and data set letter:• Partition name: DJXK21• DDNAMEs:– DJXK21L, DJXK21X, DJXK21A,DJXK21B,…• Data set names© Copyright IBM Corporation 20131 to 1001 partitions…Datasetgroups…ABJ…ABJ……– Data set name prefix, data setletter, and partition ID:• DSN prefix: IMSP.DB.DJXAB• Partition ID: 00001• Data set names:– IMSP.DB.DJXAB.L00001– IMSP.DB.DJXAB.X00001– IMSP.DB.DJXAB.A00001– IMSP.DB.DJXAB.B00001– …
  • 18. ……ILDSIndexPHIDAM onlyLXYLXY…Partner Data Sets• Index and data set group datasets have partners:– Each X data set has a Y partner– Each A data set has an M partner– Each B data set has an N partner– …• DDNAMEs differ by the letter– For example:© Copyright IBM Corporation 20131 to 1001 partitions…Datasetgroups…ABJMNV…ABJMNV……… …– For example:• DJXK21A• DJXK21M• Data set names differ by theletter– For example:• IMSP.DB.DJXAB.A00001• IMSP.DB.DJXAB.M00001
  • 19. Terminology• Before or after reorganization:– Active data sets (either A-J, X or M-V, Y))• Data sets being accessed by applications– Inactive data sets• Data sets not being accessed by applications• During reorganization:© Copyright IBM Corporation 2013• During reorganization:– All data sets (A-J, X and M-V, Y) are active data sets– Input data set: Contains unreorganized data• Includes both active and inactive data– Output data set: Contains reorganized data– Cursor• Dividing line between active data and inactive data• Only used while reorganization in progress or suspended
  • 20. Reorganization• Reorganize by copying segments:– Read segments from one set of HALDB data sets (for example, A-J,X)– Write (insert) segments to another set (for example, M-V, Y)• Update ILDS for secondary index and logical relationship targets– Use locking protocols to provide concurrent access integrity– Log inserts for recoverability© Copyright IBM Corporation 2013A-JInputData SetsOutputData SetsYCopyCopyXLILDSM-V– Log inserts for recoverability– Use cursor to identify which set to use to access a database record• Database records before cursor, use output data sets• Database records after cursor, use input data sets
  • 21. 987654321987654321CopyUnit ofReorg.A MCursorCopying Records During Reorganization• Unit of Reorganization (UOR) isa set of database records:– Records are copied from input tooutput data sets– Records in UOR are locked whilebeing copied– At end of copy for UOR, the locks© Copyright IBM Corporation 20132019181716151413121110920191817161514131211109Already copiedBeing copiedActive dataNot yet used– At end of copy for UOR, the locksare released– Number of records in UOR isdynamically adjusted• Algorithm limits time taken,bytes copied, and locks heldduring copy
  • 22. 987654321987654321CopyUnit ofReorg.A MCursorApplication Access During Reorganization• Cursor points to last committedreorganized record:– PHDAM RAP RBA– PHIDAM root key• Data set used is based oncursor value:– Cursor on record 6© Copyright IBM Corporation 20132019181716151413121110920191817161514131211109Already copiedBeing copiedActive dataNot yet used– Cursor on record 6– Access Record 5:• Access from M data set– Access Record 14:• Access from A data set– Access Record 9:• Wait for lock,– then access from M data set– Access includes gets andupdates
  • 23. Completion of Reorganization• When OLR completes:–A-J,X becomes the inactive set - may be deleted–M-V,Y becomes the active set• Cursor reset to inactive• ILDS (ILEs) updated during reorganization© Copyright IBM Corporation 2013A-JInputData SetsOutputData SetsYXLILDSM-V• ILDS (ILEs) updated during reorganization
  • 24. Next Reorganization• Next reorganization:– Reorganize from M-V,Y to A-J,X– A-J, X data sets might be reusedOr– A-J, X data sets might be reallocated© Copyright IBM Corporation 2013A-JOutputData SetsInputData SetsYCopyCopyXLILDSM-V
  • 25. Setting Up Online Reorganizations• DBRC is used to set online reorganization capability for adatabase:– INIT.DB DBD(HALDB_master) OLRCAP|OLRNOCAP– CHANGE.DB DBD(HALDB_master) OLRCAP|OLRNOCAP– OLRCAP allows online reorganization for partitions of the© Copyright IBM Corporation 2013– OLRCAP allows online reorganization for partitions of thedatabase
  • 26. Output Data Set Creation (1 of 2)• Output data set allocation options:– Preallocation by user– Automatic allocation by OLR• Invoked for each data set which is not cataloged– Invoked on data set by data set basis© Copyright IBM Corporation 2013• Why preallocate?– Want to allocate on specific volume– Change space allocation• Blocks/CIs– Primary and secondary allocations• For PHIDAM Primary Index– Free space percentage
  • 27. Output Data Set Creation (2 of 2)• Automatic output data set creation:– Space is equivalent to existing input data set• Requested as a number of OSAM blocks or VSAM records– SMS-managed:• Same storage class as input data set• Same number of volumes as input data set© Copyright IBM Corporation 2013• Same number of volumes as input data set• With guaranteed space attribute, primary space allocation is takenon all volumes– Non-SMS, OSAM:• UNIT=SYSALLDA is used (storage or public volume)• If input is multivolume data set, output data set is not created– Non-SMS, VSAM• Data set is allocated on the same volume(s) as input data set
  • 28. Starting Online Reorganization• Command to initiate OLR:– Type-2 command:INIT OLREORG NAME(partname1, partname2,...)– Type-1 command:/INIT OLREORG NAME(partname1)– Command parameters:© Copyright IBM Corporation 2013– Command parameters:• Delete input data sets at completion of reorganizationOPTION(DEL|NODEL)• Set rate of executionSET(RATE(100|nn)
  • 29. Rate Parameter• RATE parameter on INIT:– RATE parameter determines how fast the reorganization runs:• RATE(100) - Runs at maximum speed• RATE(nn) - Online reorganization waits after each commit so thataverage speed of reorganization is nn% of maximum speed– Examples:© Copyright IBM Corporation 2013– Examples:• If RATE(50), after each commit, reorganization waits for the time thatthe last interval took– Possibly, run 1 second, wait 1 second, run 1 second, wait 1 second,...• If RATE(25), after each commit, reorganization waits for 3 times aslong as the last interval took– Possibly, run 1 second, wait 3 seconds, run 1 second, wait 3seconds,...
  • 30. Commands to Show Status of Reorganization• QRY command (type-2) example:QRY OLREORG NAME(*) SHOW(ALL)– Response:Partition MbrName CC LclStat Rate Bytes-Moved Segs-MovedPOHIDKA IMS1 0 RUNNING 100 72315678 244597PVHDJ5A IMS1 0 RUNNING 100 8454630 30029Roots-Moved Option Resumed StartTime22511 NODEL Y 2009.196 10:20:21.61© Copyright IBM Corporation 201322511 NODEL Y 2009.196 10:20:21.613775 DEL 2009.196 10:20:21.84• /DIS command (type-1) example:/DIS DB OLR– Response:DATABASE PART RATE BYTES SEGS ROOTS STARTTIMESTATUSDBHDOJ01 PDHDOJA 10 53330 217 31 09195/143354WAITRATE, OPTDEL*09195/143362*
  • 31. Logging By Online Reorganization (1 of 2)• Log records written:– Scheduling (x’08’)– Termination (x’07’)– UOR sync point (x’3730’)• For each UOR– UOR statistics (x’2950’)© Copyright IBM Corporation 2013• For each UOR– Database change (x’50’)• For all output data in the partition– This will be voluminous!
  • 32. Logging By Online Reorganization (2 of 2)• UOR statistics log record (x’2950’):– Written for each UOR– Data:• Total segments moved before this UOR• Total bytes moved before this UOR• Roots moved in UOR• Segments moved in UOR© Copyright IBM Corporation 2013• Segments moved in UOR• Bytes moved in UOR• Locks held by UOR• Start time of UOR• Execution time (elapsed time) of UOR• Time interval waited before this UOR (due to RATE parameter)
  • 33. Suspending and Restarting Online Reorg• Reorganization might be suspended:– Commands:• TERM command (type-2) example:TERM OLREORG NAME(PVHDJ5A)• /TERM command (type-1) example:/TERM OLREORG NAME(PVHDJ5A)– Input and output data sets remain active© Copyright IBM Corporation 2013– Input and output data sets remain active• Cursor remains active• Suspended reorganization might be restarted:– INIT and /INIT command will restart the reorganization• Restarts from the point of the cursor– Restart might be on the same IMS system or another IMS system
  • 34. Performance Considerations (1 of 3)• OSAM sequential buffering can be used– Recommended• Logging might affect performance:– All data is logged when moved– A few additional log records© Copyright IBM Corporation 2013• Buffer pool contention– Partner data sets use the same buffer pool:• Appropriate for times when reorganization is not running• Could cause buffer contention during reorganization
  • 35. Performance Considerations (2 of 3)• Lock contention:– Should be minimal• OLR has dynamic algorithm to limit the time that locks are held– OLR rarely causes a deadlock:• Asks for database record locks conditionally– If lock is not available, the UOR is shortened• OLR is always the victim in its deadlocks:© Copyright IBM Corporation 2013• OLR is always the victim in its deadlocks:– Application continues– OLR is dynamically backed out> Only the current UOR is backed out– OLR is automatically restarted at the current cursor position
  • 36. Performance Considerations (3 of 3)• Online reorganization runs in DL/I address space– Each reorganization uses one of 10 database TCBs• Same TCBs that are used for allocation and open/close/EOVprocessing• Online reorganization can run on any data sharing IMS system– Some installations may choose to dedicate an IMS to OLR:© Copyright IBM Corporation 2013– Some installations may choose to dedicate an IMS to OLR:• Buffer pool definitions can be tuned for OLR• Avoids buffer contention• Avoids logging contention• Limits the number of data sets with updates on the log– Logs are not required for change accum or recovery of other data sets
  • 37. IMS 11 OLR Performance Enhancements• One log record written for updates to a block• Sequential access for VSAM KSDS get processing• GNP calls eliminated for root-only databases• Reduced use of data set busy (ZID) lock© Copyright IBM Corporation 2013– PHIDAM index inserts are batched at end of unit of reorganization• Block locks eliminated for ILDS• IRLM lock look-aside– Avoids requesting locks already held
  • 38. HALDB Online Reorganization Summary• HALDB Online Reorganization is included in IMS DB– Not a feature, product, tool, and so on• Benefits:– Fast and efficient reorganizations© Copyright IBM Corporation 2013– Full integrity and recoverability are maintained– Eliminates database outages for reorganizations
  • 39. OLR vs ORF• OLR• HALDB Online Reorganization• ORF© Copyright IBM Corporation 2013• ORF• IMS Online Reorganization Facility for z/OS
  • 40. Data Base SupportedOLRPHDAMPHIDAMInternal Logical relationshipsORFHDAMHIDAMSHISAMPHDAM© Copyright IBM Corporation 2013PHDAMPHIDAMSecondary IndexesInternal Logical Relationships
  • 41. RequirementsOLR– DBRC– IMS DLI address space LSO=SORF– DBRC– IMS Tools Online SystemInterface© Copyright IBM Corporation 2013Interface– IMS Tools Common Services
  • 42. MethodologyOLR– Single partition per task– Invokes Get and InsertprocessingORF– Single partition per job– Invokes IMS utilities– Invokes IMS High Performance© Copyright IBM Corporation 2013processing– Uses Partner datasets that maybe deleted or reused– Invokes IMS High PerformanceTools– Uses Shadow data bases thatmay be deleted or reused
  • 43. FeaturesOLR– Online– DBD changes accepted• Free Space, dataset sizeORF– Online or Offline– DBD changes accepted• Free Space, dataset size© Copyright IBM Corporation 2013– DBD changes not accepted• Segment expansion, restructure– Uses Partner datasets that maybe deleted or reused– Reorganization is recoverable– DBD changes not accepted• Segment expansion, restructure– Uses Shadow data bases thatmay be deleted or reused– Reorganization is restartable
  • 44. PerformanceOLR– Runs under the DLI region– Uses up to 10 DLI TCBs – 10OLR tasks– Increase in IMS Log trafficORF– Runs as an IMS BMP– As many BMPS as resourcesallow– Relies on current IMS logging© Copyright IBM Corporation 2013– Increase in IMS Log traffic– Parameter to meter reorg– No data base accessinterruption– Relies on current IMS logging– Momentary data base accessinterruption (twice)
  • 45. OLR vs ORFOLR– IMS Base FeatureORF– IMS Tool– May require additional tooling© Copyright IBM Corporation 2013

×