See what's new in IBM mainframe technology through August 2018. This is the newest of the new software mainframe technology. This presentation is a teaser for additional topics presented at SHARE in St. Louis.
Azure Monitor & Application Insight to monitor Infrastructure & Application
2018 08-13-ib ms-latest-buzz-share-final
1. IBM’s Latest Buzz
A Discussion of the Newest of the New
Technologies
Liya Ma – maliya@us.ibm.com
David Morlitz – dmorlitz@us.ibm.com
Bill Seubert – wfseube@us.ibm.com
Monday, August 13th - 10:00 – 11:00 AM
St. Louis Convention Center – Room 241
1
2. Session Overview and Goals
• In this session, we will:
• Provide a brief overview of IBM announcements since the previous SHARE
• Direct you to sessions later this week that can provide deeper technical information
• Only discuss publicly available information
• Skip many slides in the presentation, which are included for reference only
• After the session please feel free to ask us to connect you with IBM
technical experts on topics that you would like to learn more about, have a
deeper technical discussion, or arrange to bring back to your company.
• Assistance finding IBM technical resource after SHARE ends is also available
– please E-Mail the presenters as listed on the title chart
2
4. 4
IBM z14
Designed for trusted digital experiences
Pervasive encryption making data your
new perimeter
New flexible software pricing offering
for modern digital workloads
Machine learning applied directly to your
most valuable data
Open and connected with new economics
in the cloud
Blockchain on IBM Z® where trust meets transparency
4
8. What’s new in ML for z/OS v1.2
• Single-click model deployment
• PMML Model Support
(Predictive Modeling Markup Language)
• Multi-version support across
test/production environments
• Optimized High-Performance Online
Scoring Engines
• New version of Db2 IT Operational
Analytics Solution Template
9. Continuous Delivery Model Provides New Functionality at Regular Intervals
Ingest Data Preparation Train & Evaluate Models Deploy Go Live: Predict and MonitorExplore & Visualize
• Data connector+
• Data from HDFS**
• Db2 Analytics
Accelerator
(“IDAA”) as the
data source of
MLz^^
• Auto Data Preparation+
• Visualization libraries
• Collaboration
enhancements^
• Embedded SPSS Modeler^^
• Auto modelling (CADS/HPO)
• Jupyter Notebook
• Visual model builder (2.0 +)
• Python/scikit learn/Spark
2.1.1 (IzODA)++
• Model Training on Linux++
• Model training with built-in or
remote Spark**
• PySpark support^
• Interoperability between MLz
and DSX local^^
• Db2 ITOA Solution
Template^^
• Spark/MLEAP online
scoring*
• PMML model online
scoring*
• Runtime performance
monitoring
• HA Scoring Service++
• Evaluation/Feedback for
PMML models++
• Online scoring service
integrated in CICS region**
• Performance enhancement
for online scoring service^
• Lifecycle management for
PMML models trained in
SPSS modeler/Rstudio^^
• Model governance - audit
trace^^
• Feedback data ingestion
• Continuous model performance monitoring
• On-demand retraining+
• Single click
deployment
• Multi-model version
support+
• Batch deployment++
• One model, different
versions in different
scoring clusters**
• z/OSMF managed
scoring cluster^• Admin dashboard*
• Collaboration in
project+
• Role based user
access control+
Release intervals: June’17* - July’17+ - Sept’17++ - Nov’17** - Feb’18^ - May’18^^
9
10. Data Scientist
Application
Developer
ML and
Production
Engineers
Leverage existing ML assets
• Support for standard PMML models
• Model trained off platform (e.g., on open
systems) can be exported to standard PMML
and imported into MLz for model deployment if
data originates in IBM Z
• Real-time insights can be embedded with
transactions on the world’s most secure
platform
• Model accuracy can be monitored and
feedback data made available for model re-
training
Preserve Existing ML Assets and Benefit from
IBM Z Qualities of Service
R/RStudio
XGBoost
SAS
Other Models
Types…
Standard
PMML
Model
SPSS
11. Multi-version support for Models and Model Management
• A model can have
multiple versions
• Different versions of
the same model can
be deployed to
different online scoring
services
• Model version upgrade
is transparent to
applications
12. Optimized code path for requests from CICS
applications
Parallel execution for large models to meet
Service Level Agreements (SLAs) for in-
transaction scoring
• Optimized to achieve 1-2 milliseconds results for
end-to-end scoring service
In-memory model evaluation
• New scoring engine for Spark models called MLEAP
that performed 10-30x faster: All qualifying Spark
models are scored with MLEAP
Optimized High-Performance Online Scoring Engines
13. New version of Db2 IT Operational Analytics
Solution Template
New version of solution template
allows users to plug in real-time IT
operational data to monitor current Db2
for z/OS health status
• Historical operational data can plug into
the solution template
• The sample application can build dynamic
base lines for Db2 for z/OS key
performance indicators (KPI)
14. Updates on Db2 Analytics Accelerator for z/OS Version 7.1
Db2 Analytics Accelerator for z/OS Version 7.1 was available for IAS and IBM Z on March 28th
Highlights of the release:
Incremental Update is included as non-confidential Technology Preview
Full Db2 for z/OS V12 support
Support for Db2 Analytics Accelerator Loader (without AOT support)
We are on a continuous delivery schedule with bi-monthly shipments. It is our intention that every release
includes bug fixes, improvements in scalability, performance and robustness as well as new features.
Furthermore, we work continuously with each release to prepare upcoming features like HTAP or MLz.
The next release is scheduled for mid-Q2
15. Db2 Analytics Accelerator and IBM z14 Model ZR1
IBM z14 ZR1 is a single frame z14 and is the successor of IBM z13s
GA Q2 2018
19” single frame
Feature based sizing with options of 4, 12, 24, or 30 IFLs
Up to 8 TB Memory
4.5 GHz processor
As per the IBM z14 Model ZR1 announcement, Db2 Analytics Accelerator Version
7.1 will support deployment on IBM z14 ZR1. Deployment of the Db2 Analytics
Accelerator on IBM Z allows for deeper integration with the IBM Z infrastructure,
leveraging IBM Z qualities of service, such as resiliency, high availability and security, as
well as resource sharing
The PVU rating of an IFL is 100 PVU instead of 120 PVU (compared to z14 )
All available IFLs up to the maximum of 30 may be used for Db2 Analytics Accelerator
16. Extends capability of the Accelerator
beyond Db2 data
Direct Load
IBM Db2 Analytics Accelerator Loader
What data sources do you need for analytics?
Direct Load
Direct Load
Direct Load
SMF
Data
VSAM Files
Remote
Db2 for z/OS
Db2 LUW Oracle ADABAS
z/OS Syslog IMS
Other
Relational non
z
MQ Queues Datacom *CSV Files
* POT support available for Datacom
17. Query Management Facility (QMF) Version 12.2
17
QMF 12.2 for z/OS, announce May 15, Available June 15
• QMF Vision enhancements:
• Auto-dashboarding driven with machine learning technology
• Linux support
• JSON, Multi-tenancy for cloud, embedding of dashboards in web pages, ships with MongoDB
included
• YouTube Playlist for enhancements: IBMQMF YouTube Playlist for QMF Vision
• QMF Server and QMF Z client
• Enable all QMF workloads to be zIIP eligible – saving costs, increasing performance
• Make QMF objects available from all clients, regardless of where it was created
• Bring the analytics to the data with multiple QMF servers
• Support JavaScript with all QMF clients and objects
• Improve the user experience for TSO users with a new interface/client
• YouTube Playlist for QMF Server: https://www.youtube.com/playlist?list=PLJ12kP036u2-KgkClDjhrzUATpw7m9rMe
18. QMF 12.2: Enable QMF Workloads to be zIIP eligible
18
Before
• ~60% zIIP eligibility
• QMF for Workstation
• QMF for WebSphere
• QMF Vision
• Not zIIP eligible
• Workloads from QMF Classic
• TSO workloads
• Batch workloads
After
• ~60% zIIP eligibility
• QMF for Workstation
• QMF for WebSphere
• QMF Vision
• >60% eligibility
• Workloads from QMF EE
or QMF for z/OS
• Batch workloads
• TSO workloads
Existing TSO and Batch workloads must be changed to be submitted
via the new QMF z client to be zIIP eligible
19. DVM, MQ Queues and Messages
• Eliminate the need to
extract data from MQ
• Perform SQL-based
analytics directly against
MQ data
• Join any DVM data
sources with MQ
messages
Queue Manager
IBM Data Virtualization Manager for z/OS
Non-relational Data
Adabas IDMS IMS VSAM
MQGET MQPUT
SMF
SQL JSON Services
z/OS
IBM® MQ is messaging middleware that integrates diverse applications and
business data across multiple platforms. It uses message queues to facilitate the
exchanges of information.
20. 2
0
IBM ADVANCED ANALYTICS FOR z/OS
At-a-glance
A tightly integrated advanced
analytics solution that assembles
the best of IBM z Analytics
capabilities within a single
product offering to deliver
analytics and machine learning
on an enterprise scale
• Dramatically accelerates complex queries
• Enables enterprise-grade machine learning
• Integrate traditional and self-service business intelligence and
collaboration
• Simplifies access and analytics on IBM Z and non-Z data
sources
• Enables advanced analytics on IBM Z with a single pricing
metric, unlimited usage and an unlimited number of users
21. IBM ADVANCED ANALYTICS FOR z/OS
solution components
Machine Learning for z/OS
Db2AnalyticsAccelerator for z/OS, deployment on IBM Z
Db2AnalyticsAccelerator Loader for z/OS
DataVirtualization Manager for z/OS (DVM)
Query Management Facility for z/OS (QMF)
Open DataAnalytics for z/OS (IzODA)
SIMPLIFIED PRICING AND PACKAGING
• single pricing metric
• unlimited usage
• unlimited number of users
2
23. Enterprise grade. Open by design. Secured by IBM Z
Rapid
Innovation
Differentiated
Integration
Investment
Leverage
Management and
Compliance
• Modernization and Digital
Transformation Speed
• Highest levels of Security
• Unmatched level of scalability,
availability, and performance
• Application Modernization with IBM
Middleware and Container content.
• Digital Transformation with Z/OS*
• Cloud Native Services with Hyper
Protect Containers and Runtimes
Benefits on Z Adoption Patterns
IBM and ISV Middleware, Data, Analytics and
Developer Services
Core Operational Services
(security, logging, monitoring etc.)
Kubernetes Platform Industry
Standard Containers
IBM Cloud Private
*
IBM Z and LinuxONE
CAM
VM Based
Workloads
Introducing IBM Cloud Private on Z
23
24. IBM Cloud Private on LinuxONE / Linux on Z
• Kubernetes based CaaS model Docker CE and Kubernetes
• Includes Service Catalog with a library of IBM middleware and Open Source services (Db2, WAS, DSX, Node, etc.)
• Can be managed from Intel or Power to Z: managed-to-Z with worker nodes on z
• Create a Build, Test, Development & Production Environment for z
* Secure Service Container in ICP on roadmap - All statements regarding IBM’s future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
Kubernetes
based container
platform
Middleware, Data &
Analytics Services
Common
Services
with Linux on z
IBM Cloud
Automation
Manager
26. OptimizeEvolveExpose
Automate & TransformAPI Enable APPS & Data Predict & Respond
Enterprise Architect
Govern APIs to maximize security and
reuse
Transform monolithic applications into
granular services
Define an API channel for business
critical assets
Application Developer
Automate delivery pipeline Optimize development with advanced
analytics
Empower developers with an open and
modern development environment
Operations
Monitor APIs Have performance visibility from device
to backend
Predict and automatically respond to
service interruption
Be EfficientRun & Maintain
Have a current view of resources and leverage the latest compilers and subsystems to reduce costs and maximize performance
Discover and create APIs with zero to
minimal changes to assets
Embrace early test integration (shift left)
Embrace an integrated and intelligent IT
Operations experience
Leverage best fit language for
transformation
IBM Z Digital Transformation
ADDI, ADFz, zCEE
API Connect
Java, Node.js, Swift
ADDI, ADFz, zCEE
SMSZ, Specialty Tools Z APM Connect CDP, IOAz, Splunk
ADDIID&T, XATester, RTW, RTVS, ADDI
RTC EE, UCD, DBB, Open Source,
ADDI, ZD&TADFz, ADDI, zOS Explorer
ADDI, ADFz, zCEE
OMEGAMON, Compilers, ABO, CICS, IMS, TPF, Db2 with Tools
Z APM Connect
27
27. IBM Toolkit for
Swift on z/OS
Community Edition
https://developer.ibm.com/mainframe/prod
ucts/ibm-toolkit-swift-z-os/
28
Key features in Swift 4.0
• Swift compiler
• Standard Library
• Core Libraries
• Package Manager
• Sample Swift application based on Kitura
• Interoperability with C, PL/I, assembly,
VSAM, and DB2.
• Free of charge
28. IBM Toolkit Swift on z/OS – July 2018 Update
New update for IBM Toolkit Swift on z/OS. Please visit the IBM Toolkit for Swift on z/OS Mainframe
Products Site to download the latest update. It includes the following updates/changes:
• Runtime performance improvements (benchmarks comparable or better than X86)
• EBCDIC codepage enhancements (EBCDIC 037 support added)
• Swift Package Manager improvements (Informative diagnostic messages)
• SwiftyJSON added to shipped packages (JSON Parser/Generator)
• Example added: IBM Kitura RESTful web service with backend calling z/OS Connect. A related
article is available here
• SSL CA now packaged with Swift on z/OS, enabling remote https swift packages (eg.
https://github.com)
• DB2 package functional improvements
• -g (DWARF Debug generation) support added
• .loc annotations added in assembly output (use -g and -emit-assembly)
• Foundation library functionality improvements: RunLoop timer improvements
29
30. IBM Cloud HyperSecure Containers
Sample Scenario:
Swift on the IBM HyperSecure
Cloud
31
IBM HyperSecure Cloud
Simplified, fast deployment and
management of packaged
solutions
Based on Secure Service
Containers (SSC)
Only the data owner (client)
controls access to data (no
sysadmin access)
IBM Z QoS reputation
(reliability/resilience) combined
with security & scaling
IBM Cloud HyperSecure Platform
Services (Apple Starter Kit)
Z
31. Sample Scenario:
Extend COBOL business logic
using Swift
• High performance language for your core
business applications.
• Statically compiled language that can easily
fit the current DevOps process for COBOL.
• Conforms to tight audit controls required by
many financial institutions.
CICS
IMS
WAS
MQ
DB2
COBOL
JSON
Current
business logic
New/additional
business logic
32
33. What is Git?
Open Source source code management (SCM)
capability initially built to manage Linux development
One of the major Distributed Source Code
Management tools and growing fast
GitHub is a web-based hosting service for version control using git.
– It offers all of the distributed version control and source code management functionality of Git as well as adding its own features.
– It provides access control and several collaboration features such as bug tracking, feature requests, task management, and wikis for
every project.
Who is GitHub?
What is Dependency Based Build (DBB)?
Intelligent Build framework for z/OS applications stored in Git
– Consists of a Liberty application server that hosts metadata
(dependency data, build results) and a build toolkit (Java API,
Groovy Installation).
– Groovy driven build environment for compiling, linking and
processing z/OS programs and applications
Why is DBB and our Partnership with GitHub important?
The Open Source movement is growing and is a strategic initiative for many of our customers. DBB allows those customers to extend
their enterprise standard of using Git to mainframe assets thus opening up the platform as part of their Digital Transformation strategy.
GitHub is the premier partner for most organizations using Git as an enterprise tool.
Our release of DBB and partnership with GutHub are critical steps to helping position Z Systems as a modern platform for supporting
Digital Transformation and allowing the platform to be included with other enterprise initiatives including DevOps.
IBM Dependency Based Build (DBB)
for Git and GitHub
Included in IBM Developer
for z Systems Enterprise
Edition
34. “Z Development & Test” v12:
Product Rebranding & Offering Restructuring
Name changes as a result of family name rebranding:
– IBM z Systems Development and Test Environment (zD&T) to IBM Z Development and Test
Environment (ZD&T).
– IBM z Systems Development and Test Environment Personal Edition becomes IBM Z Development
and Test Environment Personal Edition
– IBM z Systems Development and Test Environment Tools becomes IBM Z Development and Test
Environment Enterprise Edition
– IBM z Systems Development and Test Environment with Parallel Sysplex becomes IBM Z
Development and Test Environment with Parallel Sysplex
– Software License Manager becomes ZD&T Software-based License Server
– Hardware License Server becomes ZD&T Hardware-based License Server
35. Z Trial – Hands-on experience with
IBM Z solutions
Try out the latest products/features - https://www.ibm.com/it-infrastructure/z/resources/trial
All it takes is an IBM ID and registration for the particular trial you’re interested in, and a
Remote Desktop app to access the trials
A sampling – just for DevOps & Cloud:
39. IBM Common Data Provider for z Systems: Continuous Delivery
4Q2017 1Q2017 3Q2017 4Q2017 1Q2018
Additional target consumers:
•Stream data to Splunk
•Stream data to ELK
•Load data directly into the IDAA
Additional Data Sources
•OPERLOG
•Open streaming API
Filtering enhancements
•Time
•Reg-ex filtering
Additional Data Sources
•SMF 110 (CICS) KPI
Performance improvements
Splunk CDPz Sample
Dashboards:
•z/OS System Dashboards
•CICS Dashboards
•DB2 Dashboards
•MQ dashboards
Additional Data Sources
•CA Top Secret SMF records
•ACF2 SMF records
SDE performance improvements
(5X)
V 1.1
ELK CDPz Sample Dashboards:
•z/OS System Dashboards
•CICS Dashboards
•DB2 Dashboards
•MQ dashboards
Additional Data Sources
•IMS log support
SYS1.MANx customer
enablement for near real-time
SMF gathering
40
40. Splunk CDPz Sample Dashboards (Dec 2017):
– Dashboards on Splunk base: ibm.biz/CDPzSamples
– Detailed procedure to properly configure CDPz to stream data into Splunk and leverage provided
Sample dashboards can be found ibm.biz/CDPzSplunkBlog
Elastic Search CDPz Sample Dashboards (Feb 2018):
– Detailed procedure to properly configure CDPz to stream data into ELK and leverage the provided
Sample Dashboards as well as link to the ELK dashboards can be found ibm.biz/CDPzElasticBlog
White Paper on integrating CDPz with Splunk
https://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102713
White Paper on Integrating CDPz with ELK
https://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102722
Sample dashboards available for Splunk and Elastic Stack
41
41. CDPz – What’s coming soon: Advanced filtering and extensibility
Advanced filtering and extensibility of CDPz (through the SDE language)
• Enables support for defining custom SMF data stream types to CDPz (these could be all new data stream types or derivative types)
• Enables logical filtering (at the record level and field level) to decrease the volume of data sent to the subscribers (such as Splunk which
will reduce ingestion costs)
• Enables the development of complicated data streams with calculations
Correlator
SMF70LEN
SMF70SEG
SMF70FLG
SMF70RTY
SMF70TME
SMF70DTE
SMF70SID
SMF70SSI
SMF70STY
......
SMF70WLA
SMF70LAC
......
SMF70DTE
SMF70TME
SMF70SID
SMF70LAC
SMF70WLA
Default layout of update definition
layout of template
definition
SM1LEN SM101
SGD
... QWHCAID ... QWHCEUTX ... QPKGPK
G
QPKGPKNF QPKGPKNL
901 0 U@MUPJ2 muecps.bi
n
4 1 4
901 0 U#MGXP0
1
MG005 6 1 6
901 0 U@FCPJ1 TF008 25 21 25
901 0 U@PYPJ4 ib85pr1d 8 10 20
901 0 U@MUPJ2 MG121 10 1 10
SMF records read from
Exit, Log stream, or
In-memory resource
Record
collected?
Record Level Filtering:
Field Level Filtering:
42
42. IDSz – IBM Z Decision Support (formerly TDSz)
Back in December 2017, IBM entered an IP Partnership with 21st Century
Software Inc to develop IDSz - IBM still owns the product and sales.
21st Century Software has made a significant development investment in the
new release and introduced new function that addresses some of the key
requirements for the older TDSz product
IDSz v1.9 – GA July 27th, 2018
New
Capacity Planning
component coming in
4Q (new PID)
43. • Save time and reduce configuration errors
• Quickly find, create and edit hundreds of IMS parameters
• Across multiple files
• Across tens or hundreds of IMS systems
• Review change history
• Learn more about each parameter using context-sensitive in-product assistance
• Avoid making costly mistakes by using our parameter templates and built-in syntax checker
• Automatically discover and reveal IMS topology across enterprise
• Collect and present all configuration data in one place
• Compared to multiple files across multiple systems
• Start analysis of potentially hundreds of systems within minutes of initial set up
• Analyze, manage and compare IMS configurations across enterprise
• Intelligent comparison of parameter members in all discovered systems via GUI
• Eliminate searching for and manually comparing parameters
• Reveal anomalies in parameter configurations
IMS Configuration Manager for z/OS V2.3 – Business Value
Important for
new IMS sysprogs
(millennials)