The document discusses the importance of orchestration and operationalization in the DataOps journey, highlighting challenges such as data pipeline inefficiencies and the need for seamless data management across platforms. It emphasizes the growing significance of big data and analytics, projecting substantial market growth alongside the necessity for enhanced data engineering practices. BMC's Control-M platform is presented as a solution to improve operational efficiency and meet service level agreements through intelligent monitoring and automation.

1. Orchestration and
Operationalization:
The Next Step in your
DataOps Journey
BASIL FARUQUI | DIRECTOR, SOLUTIONS MARKETING | BMC SOFTWARE

2. BMC Connected Digital Ops
We operationalize innovation, connecting and amplifying hybrid IT with the most innovative portfolio of
infrastructure software, partners, and expertise toenable the Autonomous Digital Enterprise.
ServiceOps
AIOps DevOps DataOps AutonomousOps
We connect across your
entire technology landscape.
We turn systems of record
into systems of action.
We make “Ops” the catalyst
for competitive advantage.
THE VALUE
WE DELIVER
PORTFOLIO
OUTCOMES
SOLUTIONS

3. Agenda
✓ What’s happening in the world of Data
✓ DataOps and the role of Orchestration
✓ Control-M Platform Capabilities for DataOps

4. What’s Happening in the
Data World?

5. THE 2023 MAD (MACHINE LEARNING, ARTIFICIAL INTELLIGENCE & DATA) LANDSCAPE

6. “
Big data and analytics software and
cloud services market reached $90
billion in 2021, and it is expected to
more than double by 2026.
Companies Are Investing in Data
IDC: Worldwide Big Data and Analytics Software Forecast, 2022–2026

7. “
Through 2022, only 15% of use cases leveraging AI techniques (such as ML and
DNNs) and involving edge and IoT environments will be successful.
Future of Intelligence Survey, IDC, August 2021
80 percent of companies’ time in analytics projects is spent on repetitive tasks
such as preparing data, whereas the actual value-added work is limited.
Moreover, just 10 percent of companies believe they have this issue under
control.
McKinsey Survey, 2020
“
And Are Not Seeing Results

8. DataOps to
the Rescue

9. Gartner Market
Guide for DataOps
Tools
Source: Gartner® Market Guide for DataOps Tools, December 5, 2022
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S.
and internationally and is used herein with permission. All rights reserved. Gartner does not endorse
any vendor, product or service depicted in its research publications and does not advise technology
users to select only those vendors with the highest ratings or other designation. Gartner research
publications consist of the opinions of Gartner’s Research & Advisory organization and should not be
construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to
this research, including any warranties of merchantability or fitness for a particular purpose.

10. Enhance Data
Engineering
Practices
Source: 5 Ways to Enhance Your Data Engineering Practices, January 24, 2023

11. “
… today we are bringing data pipelines from 500 plus different systems and
there are so many platforms and so many events that are coming, data gets
delayed, the model is failing in production, or there’s an outage going on with
a platform, users are not able to consume data, … and when the business
wants to reach out to our teams, the CDO organization, to understand why,
the business gets ping-ponged, between multiple teams. The platform teams
say it’s the DataOps team (there are multiple DataOps teams), or sometimes
they go to the ModelOps teams as well…
Large NA Telco

12. What Is the Orchestration
Challenge?

13. Data-Driven Business:
The production at scale challenge
DELIVER
INSIGHTS
INGEST DATA FROM
MULTIPLE SOURCES
STORE
DATA
PROCESS
DATA
Databases Files
ERP/CRM/ Enterprise
Applications
Devices/Sensors Clickstream
Social Media
Azure
Databricks
Azure Data
Factory
Blob Storage
Azure Synapse Analytics
Amazon EMR
AWS Glue
Azure Synapse
© Copyright 2022 BMC Software, Inc.

14. How Data Engineers See Data Pipelines
Fraud Detection
Predictive Maintenance

15. Procure to Pay Financial Close
Financial
Reporting Order to Cash
Recommendation
Engines
Predictive
Maintenance
Fraud
Detection
Customer Churn
Predictions
Risk
Modeling
Claims Processing Billing
Core
Banking
Claims
System Brokerage
DELIVER INSIGHTS
INGEST DATA FROM MULTIPLE
SOURCES
STORE DATA
PROCESS
DATA
Interoperability is needed across enterprise and data pipeline orchestrators

16. Requirements for Production at Scale!
Support
heterogeneous
workflows
SLA management
for workflows
Error handling and
notification
Self-healing and
remediation
End-to-end
visibility
Appropriate UX for
multiple personas
Standards for
running in
production
Support DevOps
practices

17. Control-M
Platform Capabilities

18. Machine Learning
Amazon
SageMaker
Azure Machine
Learning
New integrations
every month!!!
App. Workflows
Azure
LogicApps
Airflow
AWS Step
Functions
Google Cloud
Composer
ERP
Oracle
PeopleSoft
Oracle
E-Business Suite
SAP R/3 SAP S/4
HANA
File Transfer
Amazon
S3
SFTP
FTP / FTPS
S3 Comp.
Storage
Azure Blob
Storage
Azure Data Lake
Storage Gen2
GCP
Storage
AS2
Databases
IBM
DB2
Microsoft
SQL Server
Oracle
Database
Sybase Any JDBC
compliant DB
PostgreSQL
Data Processing & Analytics
Amazon
EMR
Azure
HDInsight
Azure
Synapse
Azure
Databricks
Google
Dataproc
Google
Dataflow
Google
BigQuery
Databricks
Snowflake Hadoop
& Spark
Data Integration
Informatica
Cloud Services
AWS Glue
DataBrew
AWS
Glue
Alteryx
Trifacta
Azure
Data Factory
Boomi
Atomsphere
AWS Data
Pipeline
Informatica
PowerCenter
Talend Data
Management
SAP Business
Warehouse
IBM
DataStage
Microsoft
SSIS
dbt
BI & Analytics
Microsoft
Power BI
Qlik
Cloud
Amazon
QuickSight
IBM
Cognos
Miscellaneous
Micro
Focus
SAP Data
Archiving
Veritas
NetBackup
VMware
Communication
Suite
Azure
Backup
Cloud Computing
AWS
Lambda
AWS
Batch
Amazon
EC2
Azure
Functions
Azure
Batch
Azure
VM
Google
Batch
Google
VM
Amazon
ECS
RPA
Automation
Anywhere
UiPath
Control-M Integration Plug-ins

19. “How late are we going to be?”
“How much time do we have?”
Customer Care
Call Analysis Tracking
SLA Management and Impact Analysis

20. Access and interact with
Control-M from Python
Control-M
Python Client

21. Control-M
Workbench
https://hub.docker.com/r/controlm/workbench

22. Business Users:
Always up to date
Track progress
Order ad-hoc
workflows
“Self-Service has opened a whole new
world for me. The dashboard view is used
every day. I regularly check the status of
our customer billing from home in the
evening on my iPhone or iPad. It provides
me with reassurance that we are hitting
our SLAs.“
—Gillian Glass, Finance Manager, SKY TV
0

23. File
Transfer
BI
Transformation
Business
Owners
Developers
Data/File
Ops
Application
Ops
Cloud
Ops
Detailed Job monitoring
Detailed FTP monitoring
Predictive SLA monitoring
Workflow Insights
Operational
Views
End-to-End Flow
© Copyright 2022 BMC Software, Inc.

24. The Data Challenge at Railinc
Operational Excellence
Fleet management
simulation, system
anomaly detection for
improved SLAs and
business decision-making
Data Acquisition
and Management
Real-time Data Ingest
ELT Data Ingest
Metadata Management &
Business Glossary
Intelligent Data
Platform
Enterprise Data Warehouse
Hadoop
Machine Learning
and Analytics
Model Engineering &
Management
Streaming Analytics
Historical Data Analytics
Increased Industry Value
Preventative maintenance,
predictive ERA, improved
models for bad actor and
equipment failure analysis
Business
Outcomes
Daily Data Movement Stats
8M External & internal
web service calls 4.5M External EID
messages 360M Processed
messages

25. Control-M lets us monitor jobs without having people in front
of a console 24x7. If the solution detects potential delays or
failures, it alerts us immediately. Our customers rely on us to
meet those SLAs. Delayed reports on the Chicago Terminal,
for example, could affect rail operations throughout North
America.
Control-M gives us an intelligent, proactive approach to
keeping processes — and trains — running.
—Robert Redd, Release Engineer, Railinc
Railinc leverages big data and
automation to help keep 1.6M
railcars rolling across 140k miles of
track

26. Technical Content Resources
Blog: How to orchestrate a data
pipeline on Google Cloud with
Control-M
26
Blog: Orchestration and
Operationalization: The Next Step
In Your Data Journey
Blog: How to orchestrate a data
pipeline on Google Cloud with
Control-M
Control-M Demo: Orchestrating
Real Time Delivery with Artificial
Intelligence
https://bit.ly/3mGQ6HT
https://bit.ly/3HBs1so
https://go.aws/3UySM73
https://bit.ly/3zZFxTu

27. BMC, the BMC logo, and BMC’s other product names are the exclusive properties of BMC Software, Inc. or its affiliates, are registered or pending registration with the U.S. Patent and
Trademark Office, and may be registered or pending registration in other countries. All other trademarks or registered trademarks are the property of their respective owners. © Copyright
2023 BMC Software, Inc.
About BMC
BMC works with 86% of the Forbes Global 50 and customers and partners around the world to create their future. With our history of innovation, industry-leading
automation, operations, and service management solutions, combined with unmatched flexibility, we help organizations free up time and space to become an
Autonomous Digital Enterprise that conquers the opportunities ahead.
BMC—Run and Reinvent
www.bmc.com
THANK YOU