Transform Your
Mainframe and IBM i
Data for the Cloud
with Precisely and
Apache Kafka
Ashwin Ramachandran | Senior Director, Product Management
Agenda
• Beginning your cloud transformation
journey
• Connecting the mainframe to the cloud
• Ingredients for success of mainframe to the
cloud
• Customer story: Transforming IBM i with
Precisely, Apache Kafka, and the cloud
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
Companies typically waste
an average of 35% of their
cloud budget on
inefficiencies.
PwC, 2021
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
Legacy sources
cannot be
left behind
of executives say their customer-
facing applications are completely
or very reliant on mainframe and
IBM i processing
Forrester Consulting, 2019
55%
Your traditional systems
– including mainframes, IBM i
servers & data warehouses –
adapt and deliver increasing
value with each new technology
wave
72%
increase in transaction volume
on mainframe environments in
2019
BMC 2019
$1.65trillion
invested by enterprise IT
to support data warehouse &
analytics workloads over the past
decade
Wikibon “10-Year Worldwide Enterprise IT Spending 2008-2017”
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
What happens when legacy data is unlocked?
Enhanced BI and
analytics
Improved data
discovery
Data
democratization
with governance
Critical data
available for next-
gen projects – AI
and ML
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
The global leader in data integrity
Trust your data. Build your possibilities.
Our data integrity software and data enrichment products
deliver accuracy and consistency to power confident
business decisions.
Brands you trust, trust us
Data leaders partner with us
of the Fortune 100
90
Customers in more than
100
2,000
employees
customers
12,000
countries
Trusted data is critical for decision-making
Make decisions
about customer
strategy and
experience from a
unified view of
customer data
Apply AI to proven
business use cases to
automate and
enhance decision-
making
Analyze location to
power confident
decisions and create
competitive
advantage
Navigate complex
regulations with
quality data for
compliant decisions
that mitigate risk
Customer 360 AI Location Governance
Harnessing exploding data volumes to deliver integrated, accurate views of data, enriched with
context and location, drives better decisions and successful outcomes
Confidential: Prepared for Precisely Customers and Prospects
Hybrid Cloud
68%
of organizations said
disparate data
negatively impacted
their organization2
Streaming
92%
of firms agree they
need to increase
use of outside
data5
Location
47%
of newly created
data records have at
least one
critical error3
AI
54%
of enterprises
challenged by lack of
data location
intelligence4
of CEOs are concerned about the integrity
of the data they’re basing decisions on1
Sources: 1. Forbes, 2. Data Trends Survey 2019, 3. Harvard Business Review, 4. IDC, 5. Forrester
84%
Data integrity is the next business imperative
Confidential: Prepared for Precisely Customers and Prospects
Genomics
IoT
Social
tech
GPS
3G/4G
Smart
phones
Storage
& cloud
Search
Machine
learning
Advanced
analytics
Speech
-to-
speech
Fully
ubiquitous
internet
Ubiquitous
sensors
Wearables
VR/AR
Blockchain
Auto-
nomous
vehicles
5G
Bio-
metrics
LiFi
3D
printing
Bioprinting
Biochips
Micro data
centers
Natural
language
Smart
robots
Affective
computing Drones
2000s 2020
Data Integrity
Deliver accuracy,
consistency & context
Big Data
Include any kind of data
from anywhere
Cloud
Make data elastic,
easy & accessible
Data Warehousing
Centralize transactional
data for reporting
The journey to data-driven decisions
Confidential: Prepared for Precisely Customers and Prospects
Connect today’s
infrastructure with
tomorrow’s technology
to unlock the potential of
all your enterprise data
Integrate
Understand your data and
ensure it is accurate,
consistent and complete
for confident
business decisions
Verify
Analyze location data for
enhanced and
actionable business
insights that drive
superior outcomes
Locate
Power enhanced decision
making with expertly
curated, up-to-date
business, location, and
consumer data
Enrich
The four elements of data integrity
D A T A I N T E G R I T Y
Confidential: Prepared for Precisely Customers and Prospects
Beginning
your cloud
transformation
journey
Focus on
application
transformation first
Consider
containerization
for hybrid and
multi-cloud
deployments
Understand on-
prem data
pipelines
Determine what
on-prem solutions
need to open to
cloud
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
Connecting mainframe
to the cloud
Bring rich transaction data to
the cloud
Improve cloud analytics and
insights
Speed delivery of information
Scale with next-generation
initiatives
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
Ingredients for success of mainframe
to the cloud
Ingredient 1: Log-based data capture
Did you know?
• Connect CDC can leverage
published Log or Journal
standards to identify and
capture the change before
copying to the Share Queue.
• The Connect CDC Queue
ensures that data integrity is
maintained and zero data loss
occurs in the event of a dropped
connection during file
transmission.
1
1
2
3
4
Changed data
Source DBMS
Change selector
Log/Journal Queue Retrieve/Transform/Send
Apply
1. Use of transaction logs or triggers eliminates the
need for invasive actions on the DBMS
2. Selective extracts from the logs and a defined
queue space ensures data integrity
3. Transformation in many cases can be done off
box to reduce impact to production
4. The apply process returns acknowledgement to
queue to complete pseudo two-phase commit
Target DBMS
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
Ingredient 2: Real-time data
pipelines
• Stream real-time application data from relational database, such as
DB2 for IBM i to mission critical business applications and analytics
platforms
• Other systems examples: mainframes, EDWs
• Business application examples: fraud detection, hotel reservations,
mobile banking, etc.
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
Ingredient 3: Flexible replication options
One Way Two Way
Cascade
Bi-Directional
Distribute
Consolidate
Data integration
solutions should
enable topologies
that meet your
data sharing
needs
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
RDBMS
EDWs
Data Streams
Strategic Projects:
Real-time analytics, AI
and machine learning
Targets
Connect CDC is
cloud platform
enabled for
Ingest and Stream
Ingredient 4:
Precisely Connect
Other
Db2 (IBM i,
z, LUW)
Sybase
Oracle Informix PostgreSQL
MySQL
MS SQL Server
Flat Files
(delimited)
Mainframe
IMS Db2 for z/OS
VSAM
Precisely Connect
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
Connect is the best
solution for accessing and
integrating mainframe
and IBM i data with cloud
frameworks in real time.
Quickly and efficiently integrate
all enterprise data – including
mainframe and IBM i
Design-once, deploy anywhere
approach to different integration
architectures
Reduce costs and development
time – from weeks to days
Secure and governed +
unrivaled scalability and
performance
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
Scale to meet the needs of
your business
Self-service data integration through browser-based interface
• Design, deploy and monitor real-time change replication from a variety of traditional
systems (mainframe, IMS, RDBMS, EDW) to next-generation distributed streaming
platforms like Apache Kafka
• Enable the construction of real-time data pipelines
Resilient data delivery
• Fault tolerant – resilient to network/source/target/application server outages
• Protects against loss of data if a connection is temporarily lost
• Keeps track of exactly where data transfer left off and automatically restarts at that
exact point – no manual intervention
• No missing or duplicate data!
Maintain metadata integrity
• Integrates with Kafka schema registry
• On-demand metadata-driven pulls of data from a variety of database systems to next
generation data stores like the cloud and cluster
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
Customer Story
• Connect leverages transactional CICS events (committed changes)
within the mainframe, replicating the changes to Latitude’s
Confluent-based Kafka event bus
• Connect performs one-way, resilient replication to Confluent Kafka,
maintaining transactional integrity and ensuring proper data
delivery
• Simplified data transformation, COBOL copybook mapping,
REDEFINE handling, and more so that data is intelligible in Kafka
• Improved customer engagement and clearer insight into client
behavior
About
Australian financial services company with
headquarters in Melbourne, Victoria. Core business
is in consumer finance through a variety of services
including unsecured personal loans, credit cards,
car loans, personal insurance and interest free
retail finance. It is the biggest non-bank lender of
consumer credit in Australia.
Problem
Sought to modernize their transaction monitoring
capabilities in order to improve their client
engagement. This goal required gaining real-time
insight into client behavior, so that they could
provide alerting, notifications, offers, and reminders
as events happened. Unlocking this data from
mainframe VSAM was a critical step to achieving
success.
Solution
Precisely Connect
Confluent Kafka
Transform Your Mainframe and IBM i Data for the Cloud with
Precisely and Apache Kafka
Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka | Ashwin Ramachandran, Precisely

Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka | Ashwin Ramachandran, Precisely

  • 1.
    Transform Your Mainframe andIBM i Data for the Cloud with Precisely and Apache Kafka Ashwin Ramachandran | Senior Director, Product Management
  • 2.
    Agenda • Beginning yourcloud transformation journey • Connecting the mainframe to the cloud • Ingredients for success of mainframe to the cloud • Customer story: Transforming IBM i with Precisely, Apache Kafka, and the cloud Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 3.
    Companies typically waste anaverage of 35% of their cloud budget on inefficiencies. PwC, 2021 Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 4.
    Legacy sources cannot be leftbehind of executives say their customer- facing applications are completely or very reliant on mainframe and IBM i processing Forrester Consulting, 2019 55% Your traditional systems – including mainframes, IBM i servers & data warehouses – adapt and deliver increasing value with each new technology wave 72% increase in transaction volume on mainframe environments in 2019 BMC 2019 $1.65trillion invested by enterprise IT to support data warehouse & analytics workloads over the past decade Wikibon “10-Year Worldwide Enterprise IT Spending 2008-2017” Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 5.
    What happens whenlegacy data is unlocked? Enhanced BI and analytics Improved data discovery Data democratization with governance Critical data available for next- gen projects – AI and ML Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 6.
    The global leaderin data integrity Trust your data. Build your possibilities. Our data integrity software and data enrichment products deliver accuracy and consistency to power confident business decisions. Brands you trust, trust us Data leaders partner with us of the Fortune 100 90 Customers in more than 100 2,000 employees customers 12,000 countries
  • 7.
    Trusted data iscritical for decision-making Make decisions about customer strategy and experience from a unified view of customer data Apply AI to proven business use cases to automate and enhance decision- making Analyze location to power confident decisions and create competitive advantage Navigate complex regulations with quality data for compliant decisions that mitigate risk Customer 360 AI Location Governance Harnessing exploding data volumes to deliver integrated, accurate views of data, enriched with context and location, drives better decisions and successful outcomes Confidential: Prepared for Precisely Customers and Prospects
  • 8.
    Hybrid Cloud 68% of organizationssaid disparate data negatively impacted their organization2 Streaming 92% of firms agree they need to increase use of outside data5 Location 47% of newly created data records have at least one critical error3 AI 54% of enterprises challenged by lack of data location intelligence4 of CEOs are concerned about the integrity of the data they’re basing decisions on1 Sources: 1. Forbes, 2. Data Trends Survey 2019, 3. Harvard Business Review, 4. IDC, 5. Forrester 84% Data integrity is the next business imperative Confidential: Prepared for Precisely Customers and Prospects
  • 9.
    Genomics IoT Social tech GPS 3G/4G Smart phones Storage & cloud Search Machine learning Advanced analytics Speech -to- speech Fully ubiquitous internet Ubiquitous sensors Wearables VR/AR Blockchain Auto- nomous vehicles 5G Bio- metrics LiFi 3D printing Bioprinting Biochips Micro data centers Natural language Smart robots Affective computingDrones 2000s 2020 Data Integrity Deliver accuracy, consistency & context Big Data Include any kind of data from anywhere Cloud Make data elastic, easy & accessible Data Warehousing Centralize transactional data for reporting The journey to data-driven decisions Confidential: Prepared for Precisely Customers and Prospects
  • 10.
    Connect today’s infrastructure with tomorrow’stechnology to unlock the potential of all your enterprise data Integrate Understand your data and ensure it is accurate, consistent and complete for confident business decisions Verify Analyze location data for enhanced and actionable business insights that drive superior outcomes Locate Power enhanced decision making with expertly curated, up-to-date business, location, and consumer data Enrich The four elements of data integrity D A T A I N T E G R I T Y Confidential: Prepared for Precisely Customers and Prospects
  • 11.
    Beginning your cloud transformation journey Focus on application transformationfirst Consider containerization for hybrid and multi-cloud deployments Understand on- prem data pipelines Determine what on-prem solutions need to open to cloud Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 12.
    Connecting mainframe to thecloud Bring rich transaction data to the cloud Improve cloud analytics and insights Speed delivery of information Scale with next-generation initiatives Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 13.
    Ingredients for successof mainframe to the cloud
  • 14.
    Ingredient 1: Log-baseddata capture Did you know? • Connect CDC can leverage published Log or Journal standards to identify and capture the change before copying to the Share Queue. • The Connect CDC Queue ensures that data integrity is maintained and zero data loss occurs in the event of a dropped connection during file transmission. 1 1 2 3 4 Changed data Source DBMS Change selector Log/Journal Queue Retrieve/Transform/Send Apply 1. Use of transaction logs or triggers eliminates the need for invasive actions on the DBMS 2. Selective extracts from the logs and a defined queue space ensures data integrity 3. Transformation in many cases can be done off box to reduce impact to production 4. The apply process returns acknowledgement to queue to complete pseudo two-phase commit Target DBMS Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 15.
    Ingredient 2: Real-timedata pipelines • Stream real-time application data from relational database, such as DB2 for IBM i to mission critical business applications and analytics platforms • Other systems examples: mainframes, EDWs • Business application examples: fraud detection, hotel reservations, mobile banking, etc. Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 16.
    Ingredient 3: Flexiblereplication options One Way Two Way Cascade Bi-Directional Distribute Consolidate Data integration solutions should enable topologies that meet your data sharing needs Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 17.
    RDBMS EDWs Data Streams Strategic Projects: Real-timeanalytics, AI and machine learning Targets Connect CDC is cloud platform enabled for Ingest and Stream Ingredient 4: Precisely Connect Other Db2 (IBM i, z, LUW) Sybase Oracle Informix PostgreSQL MySQL MS SQL Server Flat Files (delimited) Mainframe IMS Db2 for z/OS VSAM Precisely Connect Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 18.
    Connect is thebest solution for accessing and integrating mainframe and IBM i data with cloud frameworks in real time. Quickly and efficiently integrate all enterprise data – including mainframe and IBM i Design-once, deploy anywhere approach to different integration architectures Reduce costs and development time – from weeks to days Secure and governed + unrivaled scalability and performance Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 19.
    Scale to meetthe needs of your business Self-service data integration through browser-based interface • Design, deploy and monitor real-time change replication from a variety of traditional systems (mainframe, IMS, RDBMS, EDW) to next-generation distributed streaming platforms like Apache Kafka • Enable the construction of real-time data pipelines Resilient data delivery • Fault tolerant – resilient to network/source/target/application server outages • Protects against loss of data if a connection is temporarily lost • Keeps track of exactly where data transfer left off and automatically restarts at that exact point – no manual intervention • No missing or duplicate data! Maintain metadata integrity • Integrates with Kafka schema registry • On-demand metadata-driven pulls of data from a variety of database systems to next generation data stores like the cloud and cluster Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka
  • 20.
    Customer Story • Connectleverages transactional CICS events (committed changes) within the mainframe, replicating the changes to Latitude’s Confluent-based Kafka event bus • Connect performs one-way, resilient replication to Confluent Kafka, maintaining transactional integrity and ensuring proper data delivery • Simplified data transformation, COBOL copybook mapping, REDEFINE handling, and more so that data is intelligible in Kafka • Improved customer engagement and clearer insight into client behavior About Australian financial services company with headquarters in Melbourne, Victoria. Core business is in consumer finance through a variety of services including unsecured personal loans, credit cards, car loans, personal insurance and interest free retail finance. It is the biggest non-bank lender of consumer credit in Australia. Problem Sought to modernize their transaction monitoring capabilities in order to improve their client engagement. This goal required gaining real-time insight into client behavior, so that they could provide alerting, notifications, offers, and reminders as events happened. Unlocking this data from mainframe VSAM was a critical step to achieving success. Solution Precisely Connect Confluent Kafka Transform Your Mainframe and IBM i Data for the Cloud with Precisely and Apache Kafka