Continued Innovation in IBM z/System Sort Optimization with Syncsort MFX
•Download as PPTX, PDF•
0 likes•73 views
Mainframe departments in larger enterprises are challenged to keep up with rising data volumes from ever-changing applications running on the mainframe and even in hybrid cloud environments. There is continued pressure to squeeze more performance out of their existing mainframes to reduce costs and defer major upgrades. Obvious measures to get relief from these pressures revolve around the mundane processing tasks (sort, copy, merge, compression, report generation) that take up a large and disproportionate share of CPU processing time and related mainframe resources That’s where Precisely’s Syncsort MFX has always delivered key benefits like optimized sorting, reduced CPU usage and faster job completion. Precisely has a 50 year history of innovation in this area and we continue to innovate our Syncsort MFX solution. We have several important enhancements coming for our customers. In addition, to highlighting this new functionality we want to share some powerful tips and tricks we have seen our customers use to improve their IBM z/System sorting. Watch this on-demand webinar and learn: • How Syncsort MFX can improve your sort performance and efficiency • What enhancements are coming for Syncsort MFX customers • Some tips and tricks for getting the most out of Syncsort MFX
Recommended
Recommended
More Related Content
Similar to Continued Innovation in IBM z/System Sort Optimization with Syncsort MFX
Similar to Continued Innovation in IBM z/System Sort Optimization with Syncsort MFX (20)
More from Precisely
More from Precisely (20)
Recently uploaded
Recently uploaded (20)
Continued Innovation in IBM z/System Sort Optimization with Syncsort MFX
- 1. Continued Innovation in IBM z/System Sort Optimization with Syncsort MFX Denise Tabor | Senior Product Manager Director Alissa Margulies | Principal Sales Engineer
- 2. Today’s Agenda • Syncsort MFX overview • Tips for Syncsort MFX optimization • Important enhancements for Syncsort MFX customers 2
- 3. Syncsort MFX
- 4. IBM® z Integrated Information Processor (zIIP): • Helps improve GP utilization • Allows customers to purchase additional processing power • Removes IBM software charges on zIIP capacity Bottom line: • Pay once, use often, without any additional cost • Eligible workloads moved to zIIP reduce license costs • HOWEVER, MOST workloads are not enabled to run on zIIP 4 IBM zIIP Engines
- 5. Syncsort MFX • Exploits z/OS hardware and software AND zIIP • Reduces CPU utilization for sort operations • Optimizes I/O activity Delivers • Improved processor and DASD capacity • Reduced cost 5
- 6. Syncsort MFX Reduces CPU time and I/O activity Syncsort ZPSaver zIIP-enabled sort operations Syncsort MFX PipeSort Reduces elapsed time 6
- 7. Syncsort MFX Almost 50 years of continual development and enhancements Performance The high-performance sort/copy/join solution that delivers better performance and saves money Proven Solution Improves sort performance while optimizing overall system efficiency zIIP Offload Sort workloads can be directed to the zIIP, thereby lowering the CPU time and costs Encryption Enhanced security and compliance with regulations such as GDPR 7
- 9. Data Manipulation Application & Environment Options 9
- 10. Tuning Objectives for Syncsort MFX 10 • Tuning sort requires evaluation of your current applications and defining your objectives • Consider what you are willing to trade in order to get better performance. • What is the primary outcome/goal? • Reduced CPU? • Reduced Elapsed time? • Avoiding contention with other workloads? • Reducing DASD contention? • Are you willing to: • Trade CPU for elapsed time improvements and visa versa? • Reschedule your sort jobs? • Move data sets to isolate DASD to be used by the sort? • Pass run-time parameters to the sort? • Change priorities of applications?
- 11. Tuning Options and Recommendation 11 Control Statements • Ensure you are using the optimal control statements for an application. • Evaluate errors in control statements that can adversely affect performance Optimization Mode • Select the proper mode for the desired outcome • Mode selection may affect other areas of performance. • Performance outcome between modes may need some experimentation. Virtual Storage • Most critical resource in determining how well the sort will run. • One of the most dangerous • Using too much can lead to system storage shortages and system outages.
- 12. Tuning Options and Recommendation 12 Rescheduling Work • Evaluate your overall concurrent workload • Is your CPU capacity, real storage and I/O resources impacted? • Determine if your sort work needs to be rescheduled to a quieter period. PARASORT • PARASORT improves the elapsed time performance for sorts whose input is a multi-volume tape data set and/or concatenated tape data set. • Uses parallel processing of the SORTIN input volumes. • Results in up to 33% reduction in elapsed time FILESIZE Estimates • Use of the FILSZ parameter provides the sort with an estimate of the amount of data to be sorted • Can significantly improve the optimization and performance of very large sorts.
- 13. Syncsort MFX Data Manipulation Reformatting Records • INREC, OUTREC and OUTFIL OUTREC control statements. • Used when information in the input record is not required by the applications, or the data needs to be a different format. • Allows you to: • Delete or repeat segments of record • Insert new field • Convert data • Perform arithmetic operations with numeric fields and/or constants • Perform MIN/MAX functions on numeric data • Change RECFM of output data set from fixed to variable or the reverse 13 Record Selection • INPUT PHASE: Selection of records in the order in which they appear in the input data set. • OUTPUT PHASE: Selection of records seen in sorted sequence. • These selections can be specified: • Skip the first “n” number of records • Stop after processing “n” number of records • Include/omit records based on comparisons of the contents of one or more fields within the record • Include a sample of “m” records after an interval of every “n” records • Distribute the records in rotation among all of the files in an OUTFIL group • Create a file that contains only those records that were not included in any other OUTFIL
- 14. Syncsort MFX Data Manipulation 14 Summation • Special processing done on records with equal sort keys. • Detailed records are replaced with a summary record, containing sum, average, maximum or minimum values • Detail records can be written to a separate data set Report Writer • Generate ad-hoc or scheduled reports. • Easy to use functionality with powerful record selection and formatting Join Records • Records created by joining 2 files that contain a common join key. • Join processing produces 3 types of records: • Paired records, • Unpaired from the first file • Unpaired from the second file
- 15. Additional Syncsort MFX product options Syncsort PipeSort Can simultaneously execute up to eight differently sequenced sorts from a single pass of the input data Syncsort PROCSort High performance, transparent replacement for the SAS®-provided PROC SORT. Presentation name 15 Syncsort ZPSaver A set of enhanced technologies to offload copy, SMS compression, and sort processing to zIIP processors
- 17. IBM Z Sort Accelerator Support (delivered) 17 • IBM’s Integrated Accelerator for Z Sort • New coprocessor designed for the z15 • Accelerates internal sorts • Precisely partnership w/IBM • Worked with HW architects • Developed new algorithms in Syncsort MFX • Results • Sort performance improvements
- 18. IBM Z Pervasive Encryption Support (delivered) 18 • The IBM Z Pervasive Encryption enables • Powerful encryption of data in-flight and at-rest • Highly secure ways to help deal with today’s compliance and regulatory requirements • Requires additional resources • Consumes processor cycles • Forces Syncsort MFX to use less performant I/O methods (BSAM) • Solution • For Basic/Large datasets, we can continue to use our low-level I/O
- 19. MFX Operational Visibility 19 • Customers need more insights into their sort workloads • Unexpected delays • Diagnostic information is frequently needed • Rerun a job and just to gather “debug” information • Provide more visibility • Expedite/streamline/improve troubleshooting for large/important jobs. • Examine both real-time and historical data • Port the data to an analytical tool for analysis
- 20. Questions?
Editor's Notes
- IBM® z Integrated Information Processor (zIIP): A purpose-built processor designed to operate asynchronously with the general processors in the mainframe to help improve utilization of computing capacity and control costs. zIIPs allow customers to purchase additional processing power without affecting the total million service units (MSU) rating IBM does not impose IBM software charges on zIIP capacity, but charges apply when additional general purpose engines capacity is used Bottom line: Once zIIPs are purchased they are used without any additional cost Moving eligible workloads to zIIP reduces general purpose engine utilization and can save SW license costs The big challenge is that most workloads are not enabled to run on zIIP
- Offers a high-performance sort, copy, and join utility designed to exploit the advanced facilities of the z/OS operating system and IBM Z, including zIIP engines. Can significantly reduce CPU utilization for sort operations and optimize I/O activity to reduce contention to control software costs and delay CPU upgrades Frees up general purpose MIPS for handling increased data volumes and new workloads Drive cost reduction strategies by delaying CPU upgrades and reducing software charges.
- Sort operations are resource intensive Syncsort MFX uses less CPU time than DFSORT Syncsort MFX optimizes I/O better than DFSORT Syncsort MFX encrypts sort work for required security and compliance Faster execution means more work can execute in the same amount of time === Significantly reduces general processor operation Makes use of under-utilized zIIP engines Helps control MLC costs and delay/prevent CPU upgrades Allows sort work encryption to be completed on zIIP, reducing CPU utilization dramatically and reducing costs === Simultaneously executes up to eight differently sequenced sorts from a single pass of the input data Uses advanced parallel sorting technology Cuts total elapsed time by more than 50% compared to running separate sorts
- In certain circumstances, sort may be unable to get a good filesize estimate, which may be an issue when the amount of data to be sorted is over 1 gigabyte. PARASORT requires additional tape drives and will automatically manage the tape drives and optimize their usage.
- Notes on Record Selection: Once again it isimportant to understand the flow of the sort and where the particular feature fits into this flow. The resulting output from the sort can be dramatically different if the record selection is performed in the output phase instead of the input phase. Notes on Join Records: The disposition of records in each category can be controlled independently. The output from join processing can contain any combination of these record types. This feature is similar to the join processing found in relational data bases.
- JOIN RECORDS: If there are “m” number of records from the first file and “n” records in the second file that contain the same value in the join key, m*n records will be created. SUMMATION: This processing will produce a data set that has only 1 record per sort key value unless an overflow occurs If summation or averaging is requested and the resulting calculation will cause an overflow condition the summation or averaging is not done Notes on Record Selection: Once again it isimportant to understand the flow of the sort and where the particular feature fits into this flow. The resulting output from the sort can be dramatically different if the record selection is performed in the output phase instead of the input phase. Notes on Join Records: The disposition of records in each category can be controlled independently. The output from join processing can contain any combination of these record types. This feature is similar to the join processing found in relational data bases. USE CASES for REPORTS: An example of a report that provides three separate invoice status reports. These reports, titled “Unpaid Invoices,” “Partially Paid Invoices,” and “Fully Paid Invoices,” represent three output files generated with a single pass of the sort. For this example, you can print these reports or write them to disk or tape.
- Syncsort ZPSaver is a set of enhanced technologies for MFX to offload copy, SMS compression, and sort sort processing to zIIP processors, effectively reducing the workload on the main CPU. ZPSaver can reduce TCB CPU time up to 95% in eligible Syncsort MFX applications
- IBM’s Integrated Accelerator for Z Sort New coprocessor designed for the z15 that reduces CPU usage and improves elapsed time Speeds up sorting, shortens batch windows, and improves select database functions. Precisely worked closely with IBM Worked with HW architects in z/OS Poughkeepsie team to develop support for the sort accelerator. Developed new algorithms in Syncsort MFX to take advantage of the coprocessor Customers can expect to see dramatic improvements to batch sort job performance. In our lab, we are seeing CPU and elapsed time improvements of up to 35% depending on the key length, record length, file size and some other factors.
- The IBM Z Pervasive Encryption enables Powerful encryption of data in-flight and at-rest Highly secure ways to help deal with today’s compliance and regulatory requirements Customers love the pervasive encryption approach, no one likes the additional resource consumption The challenge to implementing data encryption is it consumes processor cycles, so it doesn’t come without penalty. For Syncsort MFX users, the input or output data set is encrypted, BSAM must be used instead of our high performant low level IO access methods, and there is an extra cost from that perspective. Once again working closely with IBM, we have identified a way to continue to use our low-level IO for encrypted data sets and improve encryption performance. We have seen great performance improvement from our benchmark testing Although mainframe customers love the pervasive encryption approach, no one likes the additional resource consumption. . Syncsort MFX alone, we have seen up to 45% CPU and 40% elapsed savings, Combing Syncsort MFX with the Syncsort ZPSaver capability we have seen up to 80% CPU and 40% elapsed time savings.
- Are you faced with the challenge of providing the right diagnostic information when reporting a failure? Would you like a way to expedite the troubleshooting process? Are you constrained by where you can store job information for further analysis? What types of repositories or tools do you currently use for collecting job information for analysis? Would you like more visibility into your sorting workloads? What kind of analytical tools are you utilizing in other areas of the business to do analysis? Splunk? Elastic? What type of information would you like to see when evaluating your workloads? Would you benefit from a solution that: Identifies the key pieces of information to submit to support for review? Limits or specifies the jobs or groups of jobs to analyze? Provides examples of ways to view the information, or define your own reports/dashboards? What other types of details would you like us to know?