What’s New in Syncsort Ironstream 2.1

Syncsort Customer Education Webinar
What’s New in Syncsort Ironstream® Version 2.1
March 28, 2018

Today’s Presenters
Ed Wrazen Director, Mainframe Product Management
is responsible for the product strategy & roadmap for
Syncsort’s Mainframe products and solutions. With a career
in Enterprise IT spanning 35 years, Ed has held roles in
software development, database administration, product
management, consulting and marketing in global businesses
and enterprise technology companies. Ed has experience in
Enterprise systems architectures, performance management,
database and data management technologies and is a regular
speaker at industry events worldwide.
2Syncsort Confidential and Proprietary - do not copy or distribute
Ian Hartley, Principal Engineer, develops solutions for both
Splunk and Elastic. He has a close working relationship with
both organizations and uses his IT knowledge to build great
results. Ian has worked in the software industry since the late
80’s and has held several development and architecture roles.
He has worked with many different technology stacks and
environments for software companies. Ian also speaks at
events and writes articles.

Agenda
Introduction to Syncsort Ironstream®
Recent New Features
– SMF Versioning
– Enclave support for RMF III data
– Multiple Application Programming Interface
What’s new in Ironstream 2.1
– Support for Elastic Logstash
– IMS Log Forwarding
– Enhanced SMF data collection
Demo
Q&A
Syncsort Confidential and Proprietary - do not copy or distribute 3

Utilizing Machine Data for Log Analytics
Syncsort Confidential and Proprietary - do not copy or distribute
Log Analytics Platform
Cloud or On-Premise
Machine Data: Any Location, Type, Volume
Online
Services
Web
Services
Servers
Security GPS
Location
Storage
Desktops
Networks
Packaged
Applications
Custom
Apps
Messaging
Telecoms
Online
Shopping
Cart
Web
Clickstreams
Databases
Energy
Meters
Call Detail
Records
Smartphones
and Devices
RFID
On-
Premises
Private
Cloud
Public
Cloud
Platform Support (Apps / API / SDKs)
Enterprise Scalability
Universal Indexing
Answer Any Question
Developer
Platform
Report &
analyze
Custom
dashboards
Monitor
& alert
Ad hoc
search
Mainframe
4

Critical Mainframe Data 
Normalized and Streamed with Ironstream®

Log4jFile
Load
SYSLOG
SYSLOGD
logs
security
SMF
67+
types
RMF
Up to 50,000
values
DB2SYSOUT
Live/Stored
SPOOL Data
Alerts
Network
Components
Ironstream
API
Application Data
Assembler
C
COBOL
REXX
USS

Value of an End-to-End View, Inclusive of Mainframe
Extend What your Log Analytics Solution Does Already, to include
critical z/OS systems:
– 360ᵒ Degree View: Make the View of the Enterprise Complete via Including
Mainframe Data
– Same Dashboards, Bigger, More Complete Data Sets; Free Ironstream Elastic
and Splunk Apps

Mainframe Data
and Splunk Apps
Security and Compliance/SIEM
- Ensure Audits Passed

Mainframe Data
and Splunk Apps
IT Operational Analytics/ITOA
-Ensure Ops SLAs Met

Mainframe Data
and Splunk Apps
IT Operational Analytics/ITOA
-Ensure Ops SLAs Met
IT Service Intelligence/ITSI
-Ensure Services Health

Recent Enhancements
to Ironstream

SMF Versioning

z/OS SMF Changes
SMF record structures can and do change from one release of z/OS to
another
Significant No. of SMF changes with z/OS 2.3 (avail Sep 29, 2017)
– Up to 2048 record types supported
– Up to 65k subtypes per record type
– DSECT changes - Fields Added or Moved
– Extended Headers now available
Changes with subsystems
– Different versions of CICS or Db2
– Different versions running on same LPAR

SMF Processing requirement for different z/OS levels
Technologies that process SMF data from z/OS need to be mindful
of changes between releases
An SMF Processing Solution compiled on one z/OS may need to
process SMF data from another z/OS
Ability to process SMF data on a z/OS level that did not
generate the original data
Ability to detect subsystem version that generated SMF
– CICS 4.2, CICS 5.1, CICS 5.2 CICS 5.3, CICS 5.4
– Db2 v10, Db2 v11, Db2 v12

New Ironstream Support for SMF Versioning
Dynamically detects the z/OS level
Determines the structure of z/OS SMF records
Determines the subsystem SMF record structure
Supports multiple versions of z/OS SMF data in a single instance
Ironstream now includes a framework for supporting
multiple versions of z/OS SMF data
• Positioned for future z/OS releases
• Correctly formats the SMF record for Log analytics
• Multiple versions/formats supported

Enclave support for RMF III data

What are Enclave Attributes?
An enclave is a transaction that can span multiple dispatchable units
(SRBs and tasks) in one or more address spaces and is reported on and
managed as a unit.
The enclave is managed separately from the address spaces it runs in.
A program can create an enclave, schedule SRBs into it, or join tasks to it.
A multisystem work manager can process a transaction on multiple
systems by using a multisystem enclave.
An enclave represents a “business unit of work”
– Used by a variety of workloads – Db2, Db2 DDF, Websphere, MQ, LDAP, TCP/IP
– Enclave created and used for work across multiple address spaces and systems

Enclave support for RMF III data
Enclave is used to keep track of
address space independent
transactions
– Can consist of multiple TCBs
or SRBs executing across
multiple address spaces
Enclave used by Db2
– Distributed Data Facility (DDF)
– Db2 Stored Procedures
– Db2 Sysplex query parallelism
– Db2 sequential prefetch
Other users
– MQSeries
– Websphere
– TCP/IP
– LDAP
Displaying Enclave activity using RMF

Configuring Ironstream to collect Enclave Attributes
1. Activate Ironstream Desktop
Browser
2. On Admin menu, select
Ironstream instance
3. Select RMF Filters
– Automatically built using RMF
DDS
4. Select Enclave attributes from
MVS Image Attributes

Ironstream API Enhancement

Ironstream API Enhancements
The Ironstream API enables Ironstream instances to programmatically
collect user-defined EBCDIC or ASCII data and forward it.
The API includes a configurable IRONSTREAM_API data source and a
SSDFAPI routine, which is a standalone load module that can be called
or link edited statically into the calling program
New capability allows an API instance to be INITiated and TERMinated
– Enables multiple records to be sent while the connection is active

Using the Multi-Send API
CALL SSDFPAPI,(NUMPARM,REQUEST,
CLASS,TYPE,SUBTYPE,TOKEN,
RETCODE, RSNCODE)
|
|
|
PROCESS DATA 1-N
CALL SSDFPAPI,(NUMPARM,
REQUEST, TOKEN, DATA,
LENGTH, RETCODE, RSNCODE)
|
END DATA |
|
|
CALL SSDFPAPI,(NUMPARM,REQUEST,
TOKEN,RETCODE, RSNCODE)
Requesting Program Ironstream
INIT Request
SEND Request…1
SEND Request…N
TERM Request
▪ Validate Request
▪ Perform RACF Check
▪ Build Control Block Chain
▪ Send a valid Token to Requestor
▪ Validate request
▪ Process record for each SEND
request
▪ Validate request
▪ Release Storage
1. Request: Request type can be
“INIT”, “SEND” or “TERM”
2. Class: Identifies the
Ironstream instance running
with the same class
3. Type: Identifies the Ironstream
instance running with same
class and type combination.
4. Subtype: Identifies the
Ironstream instance running
with Same class, type and
subtype combination
5. Token: Used by API
6. Data: Data Address
7. Length: Data Length
API Parameter Options

Advantages of the Multi-Send API
Good for multiple, sequential send requests
– E.g. Processing file input
Connection persists over multiple send requests
– RACF check performed only on the initiation of the connection
Eliminates the need for the API to allocate and release storage for
each operation
Up to 10x Performance Improvement over Single-Send API when
sending multiple records
For single or few requests, Single-Send is appropriate

Extending Ironstream Data Sources, Connectivity & Integration
SMF Versioning
– Ensuring All Software Levels are
Supported by Ironstream
– Enable Ironstream to work seamlessly
with z/OS software upgrades
RMF Enclave Attributes
– Extend coverage of key z/OS metrics
– Deliver insight into operational services
– Monitor and identify problems faster
API Enhancements
– Enhance performance for application
data solutions to Ironstream
– Provide flexibility in connectivity options
– Minimize resource consumption &
utilization

What’s New in Ironstream 2.1

Ironstream 2.1 Overview
Mainframe
Data Forwarder DCE IDT
Ironstream DesktopData Collection Extension
Data ForwarderData Forwarder
z/OS
SPLUNK
Enterprise
Security
SPLUNK
IT Service
Intelligence
DB2SYSOUT
Live/Stored
SPOOL Data
Alerts
Network
Components
Ironstream API
Application Data
Assembler
C
COBOL
REXX
USSLog4jFile
Load
SYSLOG
SYSLOGD
logs
security
SMF
67
types
RMF
Up to 50,000
values
IMS

Mainframe
z/OS
SPLUNK
Enterprise
Security
SPLUNK
IT Service
Intelligence
DB2SYSOUT
Live/Stored
SPOOL Data
Alerts
Network
Components
Ironstream API
Application Data
Assembler
C
COBOL
REXX
USSLog4jFile
Load
SYSLOG
SYSLOGD
logs
security
SMF
67
types
RMF
Up to 50,000
values
IMS

Log Forwarding to Elastic Logstash
Ironstream now offers the capability
to forward z/OS Mainframe log and
application data directly to Logstash
– Connection via TCPIP, HTTP
– All data sources supported by
Ironstream can be forwarded
to Elastic
– Sample Ironstream Kibana
dashboards available

Elastic Support

Elastic Examples

IMS Log Forwarding
For organization who are running IMS systems, Ironstream now offers IMS log
forwarding to Splunk and ELK
– Ironstream supports both Synchronous (real-time) and Asynchronous log collection
– Data is sent in JSON for ingestion into Splunk or Elastic Logstash
– Ironstream exploits zIIP for reduced Mainframe MSU and associated savings

IMS Log Forwarding
IMS Log Records Supported
Master Terminal x’01’, X’03’
System x’45’
Transaction x’01’, x’03’, x’06’, x’07’, x’08’,
x’0A’, x’31’, x’32’, x’33’, x’34’,
x’35’, x’38’, x’56’
BMC Mainview x’FA’ (coming soon)
Summary level information also available for
Transaction statistics

IMS Log Forwarding
Insight into IMS System & Transaction Statistics & performance for ITOA
– Monitor IMS resource utilization
– Report transaction response times & isolate issues
– Problem identification
– Monitor Critical Service Levels
IMS Log Records Supported
Master Terminal x’01’, X’03’
System x’45’
Transaction x’01’, x’03’, x’06’, x’07’, x’08’,
x’0A’, x’31’, x’32’, x’33’, x’34’,
x’35’, x’38’, x’56’
BMC Mainview x’FA’ (coming soon)
Summary level information also available for
Transaction statistics

Ironstream IMS Log Collection - Synchronous
TCPIP
HTTP/HTTPS
Synchronous Collection for online regions using the “log write” exit point
▪ IMS calls the “log write” exit after a block of data has been written to the IMS online log dataset.
▪ The exit gathers the entire block of data.
▪ Required log records are extracted from within the block and formatted into JSON for forwarding at no impact to the online service
IMS Control
Region
IMS Log
Exit
SSDFILSG
Data
Space
Record
Extract
&
Format
Ironstream
Forwarder
Cloud or
On-premise
z/OS

SLDS
Ironstream IMS Log Collection - Asynchronous
IMS
Control
Region TCPIP
HTTP/HTTPS
Asynchronous Collection for online regions using a Log Archive process step
SLDS
IMS
Archive Log
Ironstream
IMS Job
Log
Gatherer
▪ The Asynchronous log gatherer processes the IMS Archive file produced by a previous job step. In this case, the entire
Ironstream process runs as a step within the IMS archive job
▪ Required log records are extracted from within the block and formatted into JSON for forwarding.
Data
Space
Record
Extract
&
Format
Ironstream
Forwarder
IMS Archive Job
z/OS
Cloud or
On-premise

SMF INMEM Resources
SMF PROCESSING OPTIONS
Data set mode - SMF information is recorded in
SYS1. MANx data sets
Logstream mode - SMF information is recorded in
multiple logstreams and data can be read directly
from the logstreams
In-memory (real-time) mode - SMF information is
read directly from the system buffer. SMF
information is read in real-time
– IFAMCON – Connect to SMF in-memory
– IFAMGET – Obtain data
– IFAMDSC – Disconnect
– IFAMQRY – Query SMF INMEM

SMF INMEM Resources
IRONSTREAM PROCESSING
Enables collection of SMF data through the SMF
INMEM resources and API
– Low-latency
– Can run on a zIIP Engine
– Requires LOGSTREAM to be defined in
SMFPRMxx definitions
from the logstreams

SMF INMEM Resources
– Low-latency
Defining SMF INMEM to Ironstream
– Add “SMF_SOURCE”:”INMEM”
– Add “INMEM_RESOURCE~.”<logstream>” as required
from the logstreams

SMF INMEM Resources
– Low-latency
Defining SMF INMEM to Ironstream
– Add “SMF_SOURCE”:”INMEM”
– Add “INMEM_RESOURCE~.”<logstream>” as required
Logstream connections can be controlled by the
Operator Modify Command
from the logstreams

Summary: Value Today for Enterprises with a z/OS Mainframe
Syncsort Confidential and Proprietary - do not copy or distribute
Less Complexity
Collect mainframe data; correlate
with data from other platforms; no
mainframe expertise required
Clearer Security Information
Identify unauthorized mainframe
access, other security risks; prepares
and visualizes key data for
compliance audits
Healthier IT Operations
Real-time alerts identify problems in
all key environments View latency,
transactions per second, exceptions,
etc.
Effective Problem-Resolution
Management
Real-time views to identify real or
potential failures earlier; view related
'surrounding' information to support
triage repair or prevention
Higher Operational Efficiency
Enhanced event correlation across
systems; Staff resolves problems faster;
“do more with less”
Eliminate Your Mainframe
“Blind-Spot”
Splunk/Elastic + Ironstream = Your
360ᵒ Enterprise View
51

What Now?
VISIT:
HTTP://WWW.SYNCSORT.COM/EN/PRODUCTS/MAINFRAME/IRONSTREAM
CONTACT:
INFO@SYNCSORT.COM
http://www.syncsort.com/en/TestDrive/Ironstream-Starter-Edition
Get Ironstream® for SYSLOG for free

Thank You.
Questions?

What’s New in Syncsort Ironstream 2.1

Recommended

Recommended

More Related Content

What's hot

What's hot (7)

Similar to What’s New in Syncsort Ironstream 2.1

Similar to What’s New in Syncsort Ironstream 2.1 (20)

More from Precisely

More from Precisely (20)

Recently uploaded

Recently uploaded (20)

What’s New in Syncsort Ironstream 2.1