Experiences in Mainframe-to-Splunk Big Data Access

Experiences in Mainframe-to-
Splunk® Big Data Access:
Learn What Your Peers are Doing
October 2016

Housekeeping
Webcast Audio:
– Today’s webcast audio is streamed through your computer speakers.
– If you need technical assistance with the web interface or audio, please
reach out to us using the chat window.
Questions Welcome:
– Submit your questions at any time during the presentation using the
chat window.
– We will answer them during our Q&A session following the
presentations.
Recording and Slides:
– This webcast is being recorded. You will receive an email following the
webcast with a link to download both the recording and the slides.
2

Session Abstract and Speakers
The requirement to add mainframe data to the stream of machine-to-machine or “log” data for
operational and security/compliance purposes is real. This webinar details 4 organizations who faced
these requirements and tells their individual stories as to what requirement/mandate they faced, what
options they considered, and how they ultimately addressed it. There will be a live Q&A for participants
to ask follow-up questions as to their stories and how they’re doing today.
3
etary - do not copy or distribute
David Friedman,
Senior Systems Engineer
Steven Menges, Director,
Product Management
Justin Eastman,
Senior Engineer

Big Data is No Longer a “Future”
Syncsort Confidential and Proprietary - do not copy or distribute 4
DB2SYSOUT
Live/Stored
SPOOL Data
Alerts
Network
Components
Ironstream API
Application Data
Assembler
C
COBOL
REXX
USSLog4jFile
Load
SYSLOG
SYSLOGD
logs
security
SMF
50+
types
RMF
Up to 50,000
values

Mainframes Still Host the Most Critical Applications at Big Orgs
5
Syncsort Confidential and Proprietary - do not copy or distribute
71%
Fortune 500
2.5 BillionBus. Transactions / day / per MF
23of Top 25
US Retailers
of World’s
Top Insurers10Top World
Banks92
Source: IBM Organizations Overall2000+

Organization #1: Justin Eastman
6
Justin Eastman,
Senior Engineer
Reminder:
Type in your questions at any time
during the presentation using the chat
window.
We will answer them during our Q&A
session following the presentations or
afterward.

USE CASE: THE NEED/PROBLEM
Incidents occurring in the organization would result
in the need to turn on additional CPUs to recover
from system being overloaded.
There was a need to get visibility into the health of
multiple sub-systems across different systems in
order to monitor to the load to proactively react to
these situations.
No single tool or monitor gave visibility into all the
different subsystems and across the entire
organization.
7

USE CASE ALTERNATIVES: IN-HOUSE, OTHER
Continue with the human flare gun approach currently
used in which multiple groups get involved elongating
the mean time to resolution.
Use existing monitors that require multiple sessions
and SMEs to access and perform triage.
Expand capacity to ensure that systems are not
overloaded.
Continue to rely on the customer to indicate when the
services provided become less responsive.
Look for a new solution to address their issues.
8

USE CASE: SOLUTION AND RESULTS
BIBD Solution to access z/OS log data in Splunk® for real-time
monitoring of critical subsystem performance
Creating single view into the health of all the systems and their
corresponding subsystems
Creating drill down dashboards that provide the KPIs to identify
where the source of the issue resides to that the immediate
source
Resulted in a significant reduction of MTTR and a
improve ability to detect problems before the
customer does.
9

Big Data Poll
Q1.Which Big Data analytics platforms does your company use today?
o Hadoop
o Splunk
o Other Data Warehouse
o Don’t Know
(Check all that apply)

Organization #2: David Friedman
11
David Friedman,

12
Customer had an audit and compliance mandate with
approaching deadline.
Using another product to manually retrieve information on a
daily basis.
Unable to monitor user log-on attempts, password changes,
and access violations on their mainframe environment.
Unable to obtain information in real-time.

Home-grown solution option explored; determined
would not meet implementation deadline (and may
not have satisfied requirement).
POC bake-off (Syncsort Ironstream performed very well
in a POC against competitive product).
Validated the ability to replace the manual processes
they were using with Ironstream.
13

Monitoring security activity on their mainframe
applications to meet audit and compliance requirements
outlined in regulation, including:
– log-on attempts
– password changes
– user access violations
– other security events
Get the information in real-time (and eliminated manual
processes previously accomplished using zSecure)
Filtering enables selection of only the SMF records
needed to produce desired results
14
Security
Compliance

Organization #3: Justin Eastman
15
Justin Eastman,
Senior Engineer

Security threats on the mainframe due to lack of
visibility.
Highly sensitive PHI (Protected Health Information)
escaping as data was moved from the production to
test environment despite having fences and an
automated scrubbing process.
Security information and event management (SIEM)
solution required.
16

Do nothing and wait for an audit, or even worse, a
security exposure.
Attempt to perform post-exposure forensics.
Manually extract and process logs, SMF records, etc.
and produce audit reports to demonstrate compliance.
Do solution vendor search and utilize Gartner Magic
Quadrant, etc. for enterprise-class SIEM.
17

SIEM Solution (Gartner SIEM Leader Splunk®)
BIBD Solution to access z/OS log data in
Splunk® for real-time alerts (Splunk’s chosen mainframe partner
Ironstream)
Combined solution for mainframe logs
provides fast access to:
 Unusual data movements, amount of
movements, and protocols being used
 How much of the data movement is compliant,
non-compliant, or unknown
 Sources of inbound traffic relating to any
anomalies
18
Organizational confidence in ability to audit data access compliance!

Big Iron to Big Data Poll
Q2. Is Mainframe “log” data going into your big data platform/repository?
o Yes, it is being streamed into it today
o Yes, it goes into it via periodic batch/other input method
o No, but that data has been requested/is desired
o No
o Don’t Know

Organization #4: David Friedman
20
David Friedman,
Reminder:
Type in your questions at any time
during the presentation using the chat
window.
We will answer them during our Q&A
session following the presentations or
afterward.

21
Disbursed transaction information systems.
Current tools provide partial solutions.
Require comprehensive analytics across operation.
Enterprise IT Operational Analytics (ITOA) dashboard
desired.

Organization selected Splunk® Enterprise as their ITOA
solution for distributed computing environment.
Considered Syncsort to access mainframe logs and get
comparable data from mainframe systems.
Conducted thorough POC of Ironstream in conjunction
with Splunk®
After POC, they were able to quickly start deploying it
as a comprehensive monitoring solution.
22

Complete picture of overall system health.
Meaningful correlation of information from
disparate sources for faster triage and shorter
MTTR.
Company now able to monitor entire IT
infrastructure to detect potential issues before
they become critical.
23
Reduce MTTR

Big Iron, Big Data and Big Iron to Big Data: Additional Use Cases?
24
24
Security & Compliance (SIEM)
• Access Control
• Data Movement
• Real-time Intrusion Detection
• Others?
IT Operations (ITOA)
• Systems Performance and
Tuning
• Capacity Planning
• Others?
IT Service Intelligence?
Other Monitoring & Analytics?

Big Iron
MVPs: Always Important, Big Iron and Big Data Functions, Staff Now Critical
25Syncsort Confidential and Proprietary - do not copy or distribute
“BMC Annual Mainframe Research Results 2015”1
Big Iron to Big Data Big Data

Syncsort Solutions for New and “Old” Requirements
26Syncsort Confidential and Proprietary - do not copy or distribute
High-performance sort for z/OS®
Best Sort for z Systems
Offload Copy & SMS Compression
and Sort work to zIIP processors
Savings with zIIP
Database Optimization Suites for
IBM DB2® and CA IDMS™
Network Management
z/OS® network management &
security components
Big Data integration with market-
leading support for integration and
access of mainframe and legacy data
sources
Data Access for Big Data
Collect, transform and stream
mainframe app and system log data
in near real time to Splunk Enterprise
Log Data Access for Big Data
High-performance Big Data integration
software – Linux/Unix/Windows;
Hadoop & Spark; on premise and
in the cloud
Big Data Integration
The most advanced sort features for
Unix, Linux, and Windows platforms
Best Sort for Distributed Platforms
Faster application modernization
with less hardware
AppMod
Big Iron Big Iron to Big Data Big Data
Data Funnel
Populate enterprise data lake
at the push of a button
Transparently migrate IMS to DB2
IMS and VSAM  DB2 Migration
Powerful new tools for your databases

Questions and More Information
Additional Questions for David and Justin?
For More Information:
syncsort.com/ironstream
blog.syncsort.com/
Try Ironstream for Free:
syncsort.com/ironstreamstarteredition
Comments/Other:
Steven Menges: smenges@syncsort.com
27

Experiences in Mainframe-to-Splunk Big Data Access

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to Experiences in Mainframe-to-Splunk Big Data Access

Similar to Experiences in Mainframe-to-Splunk Big Data Access (20)

More from Precisely

More from Precisely (20)

Recently uploaded

Recently uploaded (20)

Experiences in Mainframe-to-Splunk Big Data Access