SlideShare a Scribd company logo
Sherif Rasmy - MDM
1
Enterprise Application Integration
The Master Data Manager – MDM
Sherif Rasmy - March 13, 2014
Introduction
KGX is a broker-dealer that recently made unpleasant appearances in the Wall Street Journal. It was
fined multiple times by the Security Exchange Commission (SEC) and reported major losses due to
trading errors. The SEC warned KGX of losing its trading license for the following reasons:
 Failure to accurately report ownership of public companies (SEC-Schedule-13G)
 Frequent trade errors and cancellation (Trade Busts).
 Failure to capture market manipulative activities (Front Running and Wash)
KGX’s systems use multiple versions of master data. In many cases, these are incomplete, inaccurate
and inconsistent. The bad data quality caused trading errors, failure to fulfill regulatory reporting and
inability to implement effective trade surveillance models.
A Master Data Management solution is required to provide operational systems with high quality
master data. Although master data issues spanned across customer, security and market data;
security data was the most problematic. For example, missing derivative’s conversion rate eliminated
many trades from being reported to the SEC. KGX decided to adopt a Master Data Management
solution to get a single version of the truth for all its master and reference data. KGX decided to start
its MDM experience with building Security Master.
The Problem
 KGX is comprised of multiple home grown systems and databases (Figure 1)
 There are multiple copies operational security data sourced from multiple security data vendors.
OLTP High Frequency
Trading Engine DB
OLTP Retail Brokerage
Trading Desk DB
SecutiySEFY Securities
Data VEndor
LTI Securities
Data Vendor
UI
UI
Read
Secutiy
Front Office Systems
Back Office Systems
NYSE-NASDAQ
Securities
UI
OLTP Back Office DBs
Secutiy
Front Office
Data warehouses
Secutiy
Back Office
Secutiy
Risk
Secutiy
Compliance
Secutiy
ETL
ETL
ETL
ETL
ETL
ETL
ETL
Read/Write
Read/Write
ETL
ETL
ETL
ETL
Figure 1 – Current Architecture
Sherif Rasmy - MDM
2
 The data quality of the data warehouses is not much better than that of the operational ones.
dependent on the quality of the sources
 Many ETL processes that are owned and maintained by different business and IT units
 Security data in many instances is incorrect, incomplete, inconsistent, duplicated and can hardly
be grouped and categorized (Figure 2)
OLTP Retail Brokerage
Trading Desk
CUSIP: 345900145
UNDR CUSIP: 80004584876
TYPE: OPT
CONVR:
TICKER: IBMOPX
UNDRL TICKER:
TYPE: 12
CONVR: 100
ISIN: 889774584876
CONVR: 1
OLTP High Frequency
Trading Engine OLTP Back Office
Figure 2 – Current Data Characteristics
 KGX’s growing number of software components increased operational risk & cost.
The Solution
Architecture Overview
 Security Master is a Transactional MDM Hub (Figure 3).
 Master security data is stored in the MDM database and operational databases. However, the MDM
data base serves as the single source of security data
 Security data lifecycle is centrally controlled through MDM services
OLTP High Frequency
Trading Engine DB
OLTP Retail Brokerage
Trading Desk DB
Secutiy
SEFY Securities
Data VEndor
LTI Securities
Data Vendor
UI
UI
Secutiy
Front Office Systems
Back Office Systems
NYSE-NASDAQ
SecuritiesUI
OLTP Back Office DBs
Secutiy
Front Office
Data warehouses
Secutiy
Back Office
Secutiy
Risk
Secutiy
Compliance
Secutiy
The Master Data Manager
UI
Read
Read
Read
Extractors
Cleanse Correct Complete Match
Master Database
Security Master
Batch
Load
Synchronous Real
Time Update
Asynchronous Near
Real Time Update
Services Interface
Data Synchronizer
Other Security
Data Sources
Life Cycle
Web Services
Clients
Read/Write
Figure 3 – Security Master Architecture
Sherif Rasmy - MDM
3
The Master Data Manager
 MDM is comprised of three key components:
o Services Interface: A central multiprotocol interface for security data maintenance. Data
will be maintained (created/updated/deleted) through: ETL batch processes. Web Services
and User Interfaces.
o Master Database: Stores a single consolidated view of security data
o Data Synchronizer: Responsible for synchronizing master security data with operational
security data with any changes. This done either in real-time, near real-time or batch.
 IBM and Oracle are the leading providers of MDM solutions. IBM InfoSphere MDM was selected
because it will blend with existing IBM InfoSphere solutions KGX uses including: Data Warehouse,
Analytics and Business Intelligence.
 Best efforts are made by the MDM engine to create a complete, correct and consistent version of
each security (Figure 4). Security master supports all attributes for all operational sources it feeds.
MDM matching algorithms determined that all options are the same. It assigned it an internal id
7789954326 that is imported to all operational sources.
SMID: 7789954326
NAME: IBMOPX 120914 C
CUSIP: 345900145
UNDR: 80004584876
TYPE: OPT
CONVR: 100
SMID: 7789954326
TICKER: IBMOPX
UNDRL TICKER: IBM
TYPE: 12
CONVR: 100
SMID: 7789954326
NAME: IBM OPT 12/09/14
ISIN: 889774584876
CONVR: 100
Security Master
SMID: 7789954326
NAME: IBMOPX 120914 CLS A
CUSIP: 345900145
ISIN: 889774584876
TICKER: IBMOPX
UNDR CUSIP: 80004584876
UNDR ISIN: 443274589823
UNDR TICKER: IBM
TYPE: OPT
CONVR: 100
Figure 4 – Security Master Data Characteristics
 All vendor data sources are moved from feeding operational databases feeding MDM.
 Although, security master data is stored in standardized canonical structure, the MDM is
responsible for transforming the data in the operational format during synchronization.
Return on Investment
 Reduced Operational Risk and Cost: Security Master reduced the number of ETL components
dramatically. The reduced number of software components reduced the cost associated with
maintenance, licensing and support
 Increased Agility: New security data sources to improve the data quality can be plugged in one
location. We can more rapidly integrate new acquisitions
Sherif Rasmy - MDM
4
Master Data Governance
Master Data Governance is about defining policies to ensure ownership and accountability associated
with master data metadata, life-cycle, quality, security and usage. KGX created a simple governance
organization from councils and stewards (Figure 5)
implements
Stewards
Procedures
Policies
establish develop
develop
outline- Business representative from Front Office, Back
Office and Information Technology
- Establishing the policy and procedures of the
organization
- Business representative from Front Office, Back
Office and Information Technology
- Establishing the policy and procedures of the
organization
- Business representative from Front Office, Back Office and
Information Technology
- Responsible for master data quality and work closely with the
corresponding Information Stewards to develop and implement
governance procedures in line with expectations set forth under
governance policies.
- Business representative from Front Office, Back Office and
Information Technology
- Responsible for master data quality and work closely with the
corresponding Information Stewards to develop and implement
governance procedures in line with expectations set forth under
governance policies.
Councils
Figure 5 – Security Master Data Governance
Project Planning
An MDM project is like any software project and should follow typical SDLC. Security Master went
through the following steps (Figure 6).
- SEFY Security Data Vendor
- LTI Securities Data Vendor
- NYSE-NASDAQ Securities
Identify Sources of Security Data1
Develop Master Security Data Model5
SMID
NAME
CUSIP
ISIN
UNDR TICKER
TYPE
CONVR
TICKER
ISIN
TICKER
UNDR CUSIP
UNDR ISIN
Choose an MDM Solution Provider6
- Front Office Systems and DB
- Back Office Systems and DB
- Data warehouses
Modify Producing & Consuming Systems7
- Security Master Services Interfaces
- Security Master Data Synchronizer
Implement Maintenance Processes8
- Name councils, stewards and custodian
- Establish policies
- Develop procedures
Implement a Data Governance Program4
- No Standard Security Identifier
- Underlying Securities not Available
- Inaccurate Conversion Rate
- Inconsistent Reference Data
Analyze Security Data Quality3
- Front Office Systems and DB
- Back Office Systems and DB
- Data warehouses
Identify Who Uses Security Data2
Figure 6 – Security Master Project Plan
Challenges
- Buy in and Commitment
- Data efforts Realignment
- Projects Prioritization
- Clear Mission Statement
- Adequate Training and Support
- Strong Leadership
Executive Challenges
- Behavior Change
- Adherence to New Policies & Procedures
- Implementation of new Procedures
- Resourcing
- Resistance to Change and Losing Control
Managerial Challenges
- Data Analysis
- Resistance to Change and Losing Control
- Choosing an MDM vendor
- Achieving proper QoS
Technical Challenges
Figure 7 – Security Master Implementation Challenges
Sherif Rasmy - MDM
5
Conclusion
Nowadays, enterprises can’t grow and survive without quick access to correct and complete
information. Data issues not only affect enterprise’s profitability but its existence. This paper covered
the reasons for adopting master data management, the process of developing a solution and several
options for technological implementation of the solution.

More Related Content

What's hot

Informatica Presents: 10 Best Practices for Successful MDM Implementations fr...
Informatica Presents: 10 Best Practices for Successful MDM Implementations fr...Informatica Presents: 10 Best Practices for Successful MDM Implementations fr...
Informatica Presents: 10 Best Practices for Successful MDM Implementations fr...
DATAVERSITY
 
Harmonize Your Enterprise Processes with Product Master Data Management Solut...
Harmonize Your Enterprise Processes with Product Master Data Management Solut...Harmonize Your Enterprise Processes with Product Master Data Management Solut...
Harmonize Your Enterprise Processes with Product Master Data Management Solut...
garry thomos
 
Tips & tricks to drive effective Master Data Management & ERP harmonization
Tips & tricks to drive effective Master Data Management & ERP harmonizationTips & tricks to drive effective Master Data Management & ERP harmonization
Tips & tricks to drive effective Master Data Management & ERP harmonization
Verdantis
 
Multidomain MDM at Amadeus
Multidomain MDM at AmadeusMultidomain MDM at Amadeus
Multidomain MDM at Amadeus
Orchestra Networks
 
5 Level of MDM Maturity
5 Level of MDM Maturity5 Level of MDM Maturity
5 Level of MDM Maturity
PanaEk Warawit
 
Master Data Management
Master Data ManagementMaster Data Management
Master Data Management
MoniqueO Opris
 
09 mdm tool comaprison
09 mdm tool comaprison09 mdm tool comaprison
09 mdm tool comaprison
Sneha Kulkarni
 
Master Data Management - Gartner Presentation
Master Data Management - Gartner PresentationMaster Data Management - Gartner Presentation
Master Data Management - Gartner Presentation
303Computing
 
Lean Master Data Management
Lean Master Data ManagementLean Master Data Management
Lean Master Data Management
nnorthrup
 
Credit Suisse, Reference Data Management on a Global Scale
Credit Suisse, Reference Data Management on a Global ScaleCredit Suisse, Reference Data Management on a Global Scale
Credit Suisse, Reference Data Management on a Global Scale
Orchestra Networks
 
Master Data Management methodology
Master Data Management methodologyMaster Data Management methodology
Master Data Management methodology
Database Architechs
 
Informatica MDM Presentation
Informatica MDM PresentationInformatica MDM Presentation
Informatica MDM Presentation
MaxHung
 
Master Data Management: Extracting Value from Your Most Important Intangible ...
Master Data Management: Extracting Value from Your Most Important Intangible ...Master Data Management: Extracting Value from Your Most Important Intangible ...
Master Data Management: Extracting Value from Your Most Important Intangible ...
FindWhitePapers
 
Infosys best practices_mdm_wp
Infosys best practices_mdm_wpInfosys best practices_mdm_wp
Infosys best practices_mdm_wp
wardell henley
 
A Reference Process Model for Master Data Management
A Reference Process Model for Master Data ManagementA Reference Process Model for Master Data Management
A Reference Process Model for Master Data Management
Boris Otto
 
Ebook - The Guide to Master Data Management
Ebook - The Guide to Master Data Management Ebook - The Guide to Master Data Management
Ebook - The Guide to Master Data Management
Hazelknight Media & Entertainment Pvt Ltd
 
IBM InfoSphere MDM v11 Overview - Aomar BARIZ
IBM InfoSphere MDM v11 Overview - Aomar BARIZIBM InfoSphere MDM v11 Overview - Aomar BARIZ
IBM InfoSphere MDM v11 Overview - Aomar BARIZ
IBMInfoSphereUGFR
 
5 Steps To Master Data Management
5 Steps To Master Data Management5 Steps To Master Data Management
5 Steps To Master Data Management
Embarcadero Technologies
 
Reference master data management
Reference master data managementReference master data management
Reference master data management
Dr. Hamdan Al-Sabri
 
Master Data Management
Master Data ManagementMaster Data Management
Master Data Management
Sreekanth Narendran
 

What's hot (20)

Informatica Presents: 10 Best Practices for Successful MDM Implementations fr...
Informatica Presents: 10 Best Practices for Successful MDM Implementations fr...Informatica Presents: 10 Best Practices for Successful MDM Implementations fr...
Informatica Presents: 10 Best Practices for Successful MDM Implementations fr...
 
Harmonize Your Enterprise Processes with Product Master Data Management Solut...
Harmonize Your Enterprise Processes with Product Master Data Management Solut...Harmonize Your Enterprise Processes with Product Master Data Management Solut...
Harmonize Your Enterprise Processes with Product Master Data Management Solut...
 
Tips & tricks to drive effective Master Data Management & ERP harmonization
Tips & tricks to drive effective Master Data Management & ERP harmonizationTips & tricks to drive effective Master Data Management & ERP harmonization
Tips & tricks to drive effective Master Data Management & ERP harmonization
 
Multidomain MDM at Amadeus
Multidomain MDM at AmadeusMultidomain MDM at Amadeus
Multidomain MDM at Amadeus
 
5 Level of MDM Maturity
5 Level of MDM Maturity5 Level of MDM Maturity
5 Level of MDM Maturity
 
Master Data Management
Master Data ManagementMaster Data Management
Master Data Management
 
09 mdm tool comaprison
09 mdm tool comaprison09 mdm tool comaprison
09 mdm tool comaprison
 
Master Data Management - Gartner Presentation
Master Data Management - Gartner PresentationMaster Data Management - Gartner Presentation
Master Data Management - Gartner Presentation
 
Lean Master Data Management
Lean Master Data ManagementLean Master Data Management
Lean Master Data Management
 
Credit Suisse, Reference Data Management on a Global Scale
Credit Suisse, Reference Data Management on a Global ScaleCredit Suisse, Reference Data Management on a Global Scale
Credit Suisse, Reference Data Management on a Global Scale
 
Master Data Management methodology
Master Data Management methodologyMaster Data Management methodology
Master Data Management methodology
 
Informatica MDM Presentation
Informatica MDM PresentationInformatica MDM Presentation
Informatica MDM Presentation
 
Master Data Management: Extracting Value from Your Most Important Intangible ...
Master Data Management: Extracting Value from Your Most Important Intangible ...Master Data Management: Extracting Value from Your Most Important Intangible ...
Master Data Management: Extracting Value from Your Most Important Intangible ...
 
Infosys best practices_mdm_wp
Infosys best practices_mdm_wpInfosys best practices_mdm_wp
Infosys best practices_mdm_wp
 
A Reference Process Model for Master Data Management
A Reference Process Model for Master Data ManagementA Reference Process Model for Master Data Management
A Reference Process Model for Master Data Management
 
Ebook - The Guide to Master Data Management
Ebook - The Guide to Master Data Management Ebook - The Guide to Master Data Management
Ebook - The Guide to Master Data Management
 
IBM InfoSphere MDM v11 Overview - Aomar BARIZ
IBM InfoSphere MDM v11 Overview - Aomar BARIZIBM InfoSphere MDM v11 Overview - Aomar BARIZ
IBM InfoSphere MDM v11 Overview - Aomar BARIZ
 
5 Steps To Master Data Management
5 Steps To Master Data Management5 Steps To Master Data Management
5 Steps To Master Data Management
 
Reference master data management
Reference master data managementReference master data management
Reference master data management
 
Master Data Management
Master Data ManagementMaster Data Management
Master Data Management
 

Similar to EAI - Master Data Management - MDM - Use Case

Kaizentric Presentation
Kaizentric PresentationKaizentric Presentation
Kaizentric Presentation
Azhagarasan Annadorai
 
NZS-4532 - Bringing Historical Data to Life with IBMs SMF Data Engine
NZS-4532 - Bringing Historical Data to Life with IBMs SMF Data EngineNZS-4532 - Bringing Historical Data to Life with IBMs SMF Data Engine
NZS-4532 - Bringing Historical Data to Life with IBMs SMF Data Engine
IBM z Systems Software - IT Service Management
 
Fbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_servicesFbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_services
Cindy Irby
 
Government Agencies Using Splunk: Is Your Critical Data Missing?
Government Agencies Using Splunk: Is Your Critical Data Missing?Government Agencies Using Splunk: Is Your Critical Data Missing?
Government Agencies Using Splunk: Is Your Critical Data Missing?
Precisely
 
Complexities of Separating Data in an ERP Environment
Complexities of Separating Data in an ERP EnvironmentComplexities of Separating Data in an ERP Environment
Complexities of Separating Data in an ERP Environment
eprentise
 
Klarna Tech Talk - Mind the Data!
Klarna Tech Talk - Mind the Data!Klarna Tech Talk - Mind the Data!
Klarna Tech Talk - Mind the Data!
Jeffrey T. Pollock
 
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
DATAVERSITY
 
Old Dogs, New Tricks: Big Data from and for Mainframe IT
Old Dogs, New Tricks: Big Data from and for Mainframe ITOld Dogs, New Tricks: Big Data from and for Mainframe IT
Old Dogs, New Tricks: Big Data from and for Mainframe IT
Precisely
 
Shield db data security
Shield db   data securityShield db   data security
Shield db data security
Tapan Biswas
 
Shield db data security
Shield db   data securityShield db   data security
Shield db data security
Mousumi Manna
 
Shield db data security
Shield db   data securityShield db   data security
Shield db data security
Mousumi Manna
 
Empowering you with Democratized Data Access, Data Science and Machine Learning
Empowering you with Democratized Data Access, Data Science and Machine LearningEmpowering you with Democratized Data Access, Data Science and Machine Learning
Empowering you with Democratized Data Access, Data Science and Machine Learning
DataWorks Summit
 
IBM Governance, Risk and Compliance for Data Movement and Exchange
IBM Governance, Risk and Compliance for Data Movement and ExchangeIBM Governance, Risk and Compliance for Data Movement and Exchange
IBM Governance, Risk and Compliance for Data Movement and Exchange
Lightwell
 
Enterprise Data Management “As-a-Service”
Enterprise Data Management “As-a-Service”Enterprise Data Management “As-a-Service”
Enterprise Data Management “As-a-Service”
LindaWatson19
 
Data Management Strategy
Data Management StrategyData Management Strategy
Data Management Strategy
Nandeep Nagarkar
 
KASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
KASHTECH AND DENODO: ROI and Economic Value of Data VirtualizationKASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
KASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
Denodo
 
DMsuite Static & Dynamic Data Masking Overview
DMsuite Static & Dynamic Data Masking OverviewDMsuite Static & Dynamic Data Masking Overview
DMsuite Static & Dynamic Data Masking Overview
Axis Technology, LLC
 
AtomicDBCoreTech_White Papaer
AtomicDBCoreTech_White PapaerAtomicDBCoreTech_White Papaer
AtomicDBCoreTech_White Papaer
JEAN-MICHEL LETENNIER
 
Building Elastic into security operations
Building Elastic into security operationsBuilding Elastic into security operations
Building Elastic into security operations
Elasticsearch
 
Intro to big data and applications -day 3
Intro to big data and applications -day 3Intro to big data and applications -day 3
Intro to big data and applications -day 3
Parviz Vakili
 

Similar to EAI - Master Data Management - MDM - Use Case (20)

Kaizentric Presentation
Kaizentric PresentationKaizentric Presentation
Kaizentric Presentation
 
NZS-4532 - Bringing Historical Data to Life with IBMs SMF Data Engine
NZS-4532 - Bringing Historical Data to Life with IBMs SMF Data EngineNZS-4532 - Bringing Historical Data to Life with IBMs SMF Data Engine
NZS-4532 - Bringing Historical Data to Life with IBMs SMF Data Engine
 
Fbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_servicesFbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_services
 
Government Agencies Using Splunk: Is Your Critical Data Missing?
Government Agencies Using Splunk: Is Your Critical Data Missing?Government Agencies Using Splunk: Is Your Critical Data Missing?
Government Agencies Using Splunk: Is Your Critical Data Missing?
 
Complexities of Separating Data in an ERP Environment
Complexities of Separating Data in an ERP EnvironmentComplexities of Separating Data in an ERP Environment
Complexities of Separating Data in an ERP Environment
 
Klarna Tech Talk - Mind the Data!
Klarna Tech Talk - Mind the Data!Klarna Tech Talk - Mind the Data!
Klarna Tech Talk - Mind the Data!
 
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
 
Old Dogs, New Tricks: Big Data from and for Mainframe IT
Old Dogs, New Tricks: Big Data from and for Mainframe ITOld Dogs, New Tricks: Big Data from and for Mainframe IT
Old Dogs, New Tricks: Big Data from and for Mainframe IT
 
Shield db data security
Shield db   data securityShield db   data security
Shield db data security
 
Shield db data security
Shield db   data securityShield db   data security
Shield db data security
 
Shield db data security
Shield db   data securityShield db   data security
Shield db data security
 
Empowering you with Democratized Data Access, Data Science and Machine Learning
Empowering you with Democratized Data Access, Data Science and Machine LearningEmpowering you with Democratized Data Access, Data Science and Machine Learning
Empowering you with Democratized Data Access, Data Science and Machine Learning
 
IBM Governance, Risk and Compliance for Data Movement and Exchange
IBM Governance, Risk and Compliance for Data Movement and ExchangeIBM Governance, Risk and Compliance for Data Movement and Exchange
IBM Governance, Risk and Compliance for Data Movement and Exchange
 
Enterprise Data Management “As-a-Service”
Enterprise Data Management “As-a-Service”Enterprise Data Management “As-a-Service”
Enterprise Data Management “As-a-Service”
 
Data Management Strategy
Data Management StrategyData Management Strategy
Data Management Strategy
 
KASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
KASHTECH AND DENODO: ROI and Economic Value of Data VirtualizationKASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
KASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
 
DMsuite Static & Dynamic Data Masking Overview
DMsuite Static & Dynamic Data Masking OverviewDMsuite Static & Dynamic Data Masking Overview
DMsuite Static & Dynamic Data Masking Overview
 
AtomicDBCoreTech_White Papaer
AtomicDBCoreTech_White PapaerAtomicDBCoreTech_White Papaer
AtomicDBCoreTech_White Papaer
 
Building Elastic into security operations
Building Elastic into security operationsBuilding Elastic into security operations
Building Elastic into security operations
 
Intro to big data and applications -day 3
Intro to big data and applications -day 3Intro to big data and applications -day 3
Intro to big data and applications -day 3
 

Recently uploaded

Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 

Recently uploaded (20)

Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 

EAI - Master Data Management - MDM - Use Case

  • 1. Sherif Rasmy - MDM 1 Enterprise Application Integration The Master Data Manager – MDM Sherif Rasmy - March 13, 2014 Introduction KGX is a broker-dealer that recently made unpleasant appearances in the Wall Street Journal. It was fined multiple times by the Security Exchange Commission (SEC) and reported major losses due to trading errors. The SEC warned KGX of losing its trading license for the following reasons:  Failure to accurately report ownership of public companies (SEC-Schedule-13G)  Frequent trade errors and cancellation (Trade Busts).  Failure to capture market manipulative activities (Front Running and Wash) KGX’s systems use multiple versions of master data. In many cases, these are incomplete, inaccurate and inconsistent. The bad data quality caused trading errors, failure to fulfill regulatory reporting and inability to implement effective trade surveillance models. A Master Data Management solution is required to provide operational systems with high quality master data. Although master data issues spanned across customer, security and market data; security data was the most problematic. For example, missing derivative’s conversion rate eliminated many trades from being reported to the SEC. KGX decided to adopt a Master Data Management solution to get a single version of the truth for all its master and reference data. KGX decided to start its MDM experience with building Security Master. The Problem  KGX is comprised of multiple home grown systems and databases (Figure 1)  There are multiple copies operational security data sourced from multiple security data vendors. OLTP High Frequency Trading Engine DB OLTP Retail Brokerage Trading Desk DB SecutiySEFY Securities Data VEndor LTI Securities Data Vendor UI UI Read Secutiy Front Office Systems Back Office Systems NYSE-NASDAQ Securities UI OLTP Back Office DBs Secutiy Front Office Data warehouses Secutiy Back Office Secutiy Risk Secutiy Compliance Secutiy ETL ETL ETL ETL ETL ETL ETL Read/Write Read/Write ETL ETL ETL ETL Figure 1 – Current Architecture
  • 2. Sherif Rasmy - MDM 2  The data quality of the data warehouses is not much better than that of the operational ones. dependent on the quality of the sources  Many ETL processes that are owned and maintained by different business and IT units  Security data in many instances is incorrect, incomplete, inconsistent, duplicated and can hardly be grouped and categorized (Figure 2) OLTP Retail Brokerage Trading Desk CUSIP: 345900145 UNDR CUSIP: 80004584876 TYPE: OPT CONVR: TICKER: IBMOPX UNDRL TICKER: TYPE: 12 CONVR: 100 ISIN: 889774584876 CONVR: 1 OLTP High Frequency Trading Engine OLTP Back Office Figure 2 – Current Data Characteristics  KGX’s growing number of software components increased operational risk & cost. The Solution Architecture Overview  Security Master is a Transactional MDM Hub (Figure 3).  Master security data is stored in the MDM database and operational databases. However, the MDM data base serves as the single source of security data  Security data lifecycle is centrally controlled through MDM services OLTP High Frequency Trading Engine DB OLTP Retail Brokerage Trading Desk DB Secutiy SEFY Securities Data VEndor LTI Securities Data Vendor UI UI Secutiy Front Office Systems Back Office Systems NYSE-NASDAQ SecuritiesUI OLTP Back Office DBs Secutiy Front Office Data warehouses Secutiy Back Office Secutiy Risk Secutiy Compliance Secutiy The Master Data Manager UI Read Read Read Extractors Cleanse Correct Complete Match Master Database Security Master Batch Load Synchronous Real Time Update Asynchronous Near Real Time Update Services Interface Data Synchronizer Other Security Data Sources Life Cycle Web Services Clients Read/Write Figure 3 – Security Master Architecture
  • 3. Sherif Rasmy - MDM 3 The Master Data Manager  MDM is comprised of three key components: o Services Interface: A central multiprotocol interface for security data maintenance. Data will be maintained (created/updated/deleted) through: ETL batch processes. Web Services and User Interfaces. o Master Database: Stores a single consolidated view of security data o Data Synchronizer: Responsible for synchronizing master security data with operational security data with any changes. This done either in real-time, near real-time or batch.  IBM and Oracle are the leading providers of MDM solutions. IBM InfoSphere MDM was selected because it will blend with existing IBM InfoSphere solutions KGX uses including: Data Warehouse, Analytics and Business Intelligence.  Best efforts are made by the MDM engine to create a complete, correct and consistent version of each security (Figure 4). Security master supports all attributes for all operational sources it feeds. MDM matching algorithms determined that all options are the same. It assigned it an internal id 7789954326 that is imported to all operational sources. SMID: 7789954326 NAME: IBMOPX 120914 C CUSIP: 345900145 UNDR: 80004584876 TYPE: OPT CONVR: 100 SMID: 7789954326 TICKER: IBMOPX UNDRL TICKER: IBM TYPE: 12 CONVR: 100 SMID: 7789954326 NAME: IBM OPT 12/09/14 ISIN: 889774584876 CONVR: 100 Security Master SMID: 7789954326 NAME: IBMOPX 120914 CLS A CUSIP: 345900145 ISIN: 889774584876 TICKER: IBMOPX UNDR CUSIP: 80004584876 UNDR ISIN: 443274589823 UNDR TICKER: IBM TYPE: OPT CONVR: 100 Figure 4 – Security Master Data Characteristics  All vendor data sources are moved from feeding operational databases feeding MDM.  Although, security master data is stored in standardized canonical structure, the MDM is responsible for transforming the data in the operational format during synchronization. Return on Investment  Reduced Operational Risk and Cost: Security Master reduced the number of ETL components dramatically. The reduced number of software components reduced the cost associated with maintenance, licensing and support  Increased Agility: New security data sources to improve the data quality can be plugged in one location. We can more rapidly integrate new acquisitions
  • 4. Sherif Rasmy - MDM 4 Master Data Governance Master Data Governance is about defining policies to ensure ownership and accountability associated with master data metadata, life-cycle, quality, security and usage. KGX created a simple governance organization from councils and stewards (Figure 5) implements Stewards Procedures Policies establish develop develop outline- Business representative from Front Office, Back Office and Information Technology - Establishing the policy and procedures of the organization - Business representative from Front Office, Back Office and Information Technology - Establishing the policy and procedures of the organization - Business representative from Front Office, Back Office and Information Technology - Responsible for master data quality and work closely with the corresponding Information Stewards to develop and implement governance procedures in line with expectations set forth under governance policies. - Business representative from Front Office, Back Office and Information Technology - Responsible for master data quality and work closely with the corresponding Information Stewards to develop and implement governance procedures in line with expectations set forth under governance policies. Councils Figure 5 – Security Master Data Governance Project Planning An MDM project is like any software project and should follow typical SDLC. Security Master went through the following steps (Figure 6). - SEFY Security Data Vendor - LTI Securities Data Vendor - NYSE-NASDAQ Securities Identify Sources of Security Data1 Develop Master Security Data Model5 SMID NAME CUSIP ISIN UNDR TICKER TYPE CONVR TICKER ISIN TICKER UNDR CUSIP UNDR ISIN Choose an MDM Solution Provider6 - Front Office Systems and DB - Back Office Systems and DB - Data warehouses Modify Producing & Consuming Systems7 - Security Master Services Interfaces - Security Master Data Synchronizer Implement Maintenance Processes8 - Name councils, stewards and custodian - Establish policies - Develop procedures Implement a Data Governance Program4 - No Standard Security Identifier - Underlying Securities not Available - Inaccurate Conversion Rate - Inconsistent Reference Data Analyze Security Data Quality3 - Front Office Systems and DB - Back Office Systems and DB - Data warehouses Identify Who Uses Security Data2 Figure 6 – Security Master Project Plan Challenges - Buy in and Commitment - Data efforts Realignment - Projects Prioritization - Clear Mission Statement - Adequate Training and Support - Strong Leadership Executive Challenges - Behavior Change - Adherence to New Policies & Procedures - Implementation of new Procedures - Resourcing - Resistance to Change and Losing Control Managerial Challenges - Data Analysis - Resistance to Change and Losing Control - Choosing an MDM vendor - Achieving proper QoS Technical Challenges Figure 7 – Security Master Implementation Challenges
  • 5. Sherif Rasmy - MDM 5 Conclusion Nowadays, enterprises can’t grow and survive without quick access to correct and complete information. Data issues not only affect enterprise’s profitability but its existence. This paper covered the reasons for adopting master data management, the process of developing a solution and several options for technological implementation of the solution.