Intelligent Data Management NDMO_Data Catalog and Metadata Domain Specifications coverage v1.0.pdf

Intelligent Data
Management Platform
Powered by _
AI/ML engine

2 © Informatica. Proprietary and Confidential.
KSA Data Management and Personal Data Protection Framework
Informatica Coverage Overview
Data Classification and Availability
Data Protection
Data Usage
Data Assetization
7- RMD - Reference and
Master Data Mgmt.
9- DSI - Data Sharing and
Interoperability
8- BIA - Business
Intelligence and Analytics
10- DVR - Data Value
Realization
11- OD - Open Data
5- DCM - Document and
Content Mgmt.
2- MCM - Data Catalog
and Metadata
4- DO - Data Operations
6- DAM - Data
Architecture and Modeling
3- DQ - Data Quality
13- DC - Data Classification
1- DG - Data Governance
15- DS - Data Security and
Protection (covered by NCA)
14- PDP - Personal Data
Protection
12- FOI - Freedom of
Information

Data Catalog and Metadata (MCM) Domain Focus - Informatica
Coverage Overview
Data Classification and Availability
Data Protection
Data Usage
Data Assetization
7- RMD - Reference and
Master Data Mgmt.
9- DSI - Data Sharing and
Interoperability
8- BIA - Business
Intelligence and Analytics
10- DVR - Data Value
Realization
11- OD - Open Data
5- DCM - Document and
Content Mgmt.
2- MCM - Data Catalog
and Metadata
4- DO - Data Operations
6- DAM - Data
Architecture and Modeling
3- DQ - Data Quality
13- DC - Data Classification
1- DG - Data Governance
15- DS - Data Security and
Protection (covered by NCA)
14- PDP - Personal Data
Protection
12- FOI - Freedom of
Information

Data Catalog and Metadata (MCM) Domain Focus
Introduction

Human Capital Assets
Financial Assets
Data Assets
How Organizations Manage their critical Assets?
Protect & optimize the
business value of the
organization's financial
assets
Protect & optimize the
business value of
human capital assets
Data Governance fill the same
sort of role as Finance and HR,
a coordinated enterprise effort
that protects and optimizes the
business value of the
organization data assets.

The Challenge of Enterprise Data
Finance
Chief Data Office
Legal
Business Operations
Strategy
IT
Marketing and Sales
Data Stewardship
Reporting and Analytics
Business
Transformation
Research and
Development
Risk & Compliance

Enable Data-driven Transformation
Discover Data Catalog
Understand Data Governance
Access Data Integration
Trust Data Quality
Single View Master Data
Consume Data Marketplace
Citizen
Integrator
Data
Scientist
Citizen
Analyst
LoB
Consumer
Data
Architect
Data
Engineer
Data
Steward Data
Community

Informatica Intelligent Data Management Platform

9
Data
Steward
How can I manage
metadata for key
enterprise data
assets?
How do I assess
and manage data
quality through the
lifecycle?
Data
Governance Office
How can we
validate and
enforce our data
governance
policies and
definitions?
How can I ensure
data managed
within application
and supporting
processes deliver
value to the
business?
Data
Owner
How can I
discover,
understand and
trust data required
for my analysis?
Data
Consumer
How can IT enable
business discover
data assets with
verified data
quality and
traceability?
Data
Architect
Technical
Business
The questions a data catalog solves

Informatica Enterprise Data Catalog (EDC) Brief
• Find Data (Data catalog) - automated metadata
extraction (scanners) for physical assets (dbs, tables,
columns, reports, csv files, …)
• Understand Data (Auto Data Discovery and
Profiling) - auto tagging through Data Domains
• Auto Lineage Discovery - Sources to Targets
transformation logic extraction (ETLs, Stored
procedures, …)
• Business Context (Auto Glossary assignment) –
Integration with Enterprise Data Governance (Axon)
• Accountability (Key stakeholders assignment) -
to technical assets (Data Owners and Data Stewards)
• Collaborative social capabilities - assist in data
curation (ratings, certify, Q&A, Reviews, follow,
change notifications)
• 360 Relationships view

Enterprise Data Governance (Axon) Brief
Provides Business Context for
the technical metadata
• User experience for the business
community
• Active documentation
• Glossary
• Policies and Regulations
• Process & Systems
• Data Quality impact analysis
• Change Requests and workflows

…Enable collaboration between Business and IT
Business Processes
e.g. KYC, Customer
onboarding
Reports and Applications
e.g. BI, Core banking
Business Concepts
e.g. CIF, EAD
CIF: Customer Information File
EAD: Exposure at Default
Regulations
and Policies
Data
Quality
Databases, Data
Lake, and Schemas
Data model and
Columns
ETLs, Stored
Procedures and SQL
Business World
IT World
Data Governance
Powered by CLAIRE
Change
Request
Automated
Data Discovery
Data Lineage &
Impact Analysis
Business Glossary
Assignment
Automation
Data Quality
Automation
Stakeholder
Assignment
CR
Workflows
Process
Maps
....

…Governance framework (predefined model) for holistic data governance.
Inventories of governance elements connected using relationships to create powerful
context and knowledge graph

Specification Coverage

Data Catalog and Metadata (MCM)
Domain Specifications
Coverage by Informatica EDC and Axon

Production ready metadata framework
(business and technical) … extendable and scalable
Business Metadata
Structure
Technical Metadata
Structure
… Custom Business Attributes
… Custom Technical Attributes
MCM.1.2 - Data Sources
Prioritization
• along with the definition of their
business and technical metadata
(or metadata structure).
MCM.1.3 – Metadata Architecture
• develop and document a target
metadata architecture to include
(Metadata sources, Metadata
repository, Metadata flows,
Metadata model)
MCM.4.3 - Metadata Structure
• develop the Metadata structure
• defines Business Metadata
attributes that are required to
be populated in the Data
Catalog.
• augment the mandated
Metadata structure with
additional attributes
MCM.5.1 - Data Catalog
Automated Tool
• implement the Data Catalog
automated tool acting as an
inventory of the Entity's data
assets and supporting
automation of Entity's Metadata
management.
Metadata Source Connection
and Discovery Configuration
Automation
schedule

AI-Powered Automated Data
Discovery…for Entities and Elements
Automatically relate business
terms & definitions to data
assets at scale
Discovers data domains
(name, phone, email…) and data
entities (purchase order,
health record…)
Automatically tags data by
learning from users tagging
fields and columns, etc.
Column similarity based
on data overlap and metadata
distance
Prioritization
• prioritize data sources to be
included in the Data Catalog,
along with the definition of their
(or metadata structure)..
MCM.4.2 – Metadata Population
• establish and follow a clear
process for registering and
populating the Metadata within
the Data Catalog.
Data
Domains
Assign Data
Domain to
Glossary

18
CDE Onboarding Process Flow
Axon
EDC
IDQ
Propose CDEs
Agree on Business
Glossary Definition
Assign
Stakeholders
CDE to Data Domain
Mapping
Auto onboarding
Curation
DQ Rule Creation
Auto Discovery
and Tagging
Curation/Meta
data
Enrichment
Dataset/Attribute
& lineage
onboarding
Profiling
Analysis
Data Steward
DQ Specialist
CDE data discovery and tagging
Certification and curating data sets
Reference data management
Report governance
Operational risk compliance
Sample Scenarios
CDE: Critical Data Elements
Prioritization
• prioritize data sources to be
included in the Data Catalog,
along with the definition of their
(or metadata structure)..
MCM.4.2 – Metadata Population
process for registering and
populating the Metadata within
the Data Catalog.

19
• Data Catalog Communication plans, Data
Catalog power users identification,
communication actions along with its
frequency and target audience can be
implemented using Axon Project concept
- Axon Project is a collection of activities that are
planned and organized in order to achieve a
particular set of objectives.
- Axon project can be linked to other data
governance elements like glossaries,
systems/data sets, change requests, people (Data
Catalog power users), ….
Axon Project to plan and communicate
data governance activities
MCM.3.2 - Data Catalog Adoption
and Usage
• Identification of Data Catalog
power users (Data Catalog
advanced users) as coaches for
other users
• Creation of a communication
plan announcing current Data
Catalog power users
(communication actions,
Frequency of communication
actions, Target audience)

20
Data Catalog Security Management
Users management
Group management
Resources security
management
MCM.2.2 - Metadata Access
Approval
• implement role-based access to
the Data Catalog, which includes
the creation of access groups (for
example read only, read and
update, and administrator) and
an assignment of the Data
Catalog's users to these groups.
• The Data Catalog access groups
shall be defined based on access
rights to the Metadata and scope
of the Metadata.

21
Data Governance and Catalog
Stakeholders and Automated reviews
MCM.4.1 - Metadata Stewardship
Coverage
• assign Business and IT Data
Stewards to all the Metadata
registered within the Data
Catalog.
• Data Stewards assigned to the
Metadata shall be continuously
updated and shall reflect the
most current Data Stewards'
assignments within the Entity.
Data Governance and Catalog Stakeholders Automated review schedules
Collaboration (Certification,
Rating, Reviews, Questions)

22
Automated update and review
schedules
Automated review schedules
Automated Metadata update schedule
MCM.4.4 - Metadata Update
process for updating Metadata
within its Data Catalog. The
process of metadata update shall
be implemented as a workflow
in the Data Catalog automated
tool.

23
Review and Curate Metadata.
DQ metrics
MCM.4.5 - Metadata Quality
process for identifying and
addressing quality issues with
the Metadata.
• The Metadata quality
management process shall
include reporting of identified
quality issues and development
of remediation actions within
defined SLAs.
• The process shall be
implemented as a workflow in
the Data Catalog automated tool
Discovery and Profiling results
Curate metadata: Accept, reject and modify data domain,
business term, …
Profiling results details (e.g. frequency
distribution, patterns, data types, …)
Overall and details quality metrics

24
Review, Curate metadata and certify assets.
Enable collaboration and automation
Curate and Add metadata and annotations
Enable
collaborations
Automated review schedules
Asset Certification
Management
MCM.4.6 - Metadata Annotation
& MCM.4.7 - Metadata
Certification
• establish a clear process for
reviewing on regular basis
Metadata annotations (tags,
comments) and trust certificates
added and assigned by users to
the Metadata within the Data
Catalog.
• The process shall be
implemented as a workflow in

25
Follow (monitor) data catalog assets
and receive notifications
MCM.5.2 - Metadata and Catalog
Notifications
• monitor changes to its Metadata
by setting up automated
notifications functionality within
the Data Catalog automated tool.

26
Metadata and Catalog audit trailing
MCM.5.3 - Metadata and Catalog
Audit Trail
• monitor activity of users within
by setting-up a tracking
functionality provided by the
tool.
• Monitoring shall include
information about users’ logins
to the Data Catalog and
operations they invoke. The
Entity shall store as artifacts the
Data Catalog's activity and
tracking logs.
Technical Metadata auditing
Business Metadata auditing

27
Tools Versioning
MCM.5.4 – Tools Versioning
• have the Data Catalog
automated tool updated to the
latest published Vendor release
or shall have a plan to update to
the latest release reflected in the
Data Catalog Development Plan.

28
Data Catalog KPIs
MCM.6.1 - Data Catalog KPIs
• establish key performance
indicators (KPIs) to gather
statistics on the usage and the
adoption of the Data Catalog by
users. KPIs shall include, at
minimum, the following
• Number of registered Data
Catalog users
• Number of active Data Catalog
users
• Number of logins to Data
Catalog
• Number of performed
metadata queries
• Number of annotations (tags,
comments) added to data
assets
• Number of ratings added to
data assets
• Number of assigned trust
certificates to metadata

29
Metadata Quality KPIs
MCM.6.2 - Metadata Quality KPIs
• The Entity shall establish key
performance indicators (KPIs) to
measure quality of its Metadata.
KPIs shall include, at minimum,
the following:
• Completeness (degree to which
business glossaries and data
dictionaries are completed
• Accuracy (degree to which
definitions and descriptions
align to business context
• Consistency (degree to which
definitions of Metadata are
consistent across Entity).

These graphics were published by Gartner, Inc. as part of larger research documents and should be evaluated in the context of the entire document. The Gartner documents are available
upon request from Informatica. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those
vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of
fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
A Leader in Five Gartner Magic Quadrants
Sep 2020
Enterprise
Integration Platform
as a Service
Data
Integration Tools
Aug 2020
Metadata
Management
Solutions
Nov 2020
Jan 2020
Master Data
Management
Solutions
Data Quality
Solutions
July 2020
Informatica is placed furthest in completeness of vision and highest in ability to execute
in ALL of these Magic Quadrants

31
More Capabilities

Scheduled metadata scanning
Scanning can be recurring daily,
weekly, and monthly schedules to
extract metadata at regular
intervals.
MCM4.4 - Metadata
Update

Search (discovered) enterprise data
Customizable
search tabs
Dynamic facets
Certified assets &
review ratings

Data Assets view
Business Title
(from Glossary)
Object Path /
Hierarchy
Tags
Description from
Glossary or custom
User/Group roles
Data Domains
found in object
MCM.4.1 - Metadata
Stewardship Coverage

Columns / Fields view with high level profiling
Datatypes
High level
profiling results
(unique, nulls)
Discovered Data
Domain(s)
(Suggested)
business terms
Technical name

Column / Field detailed view
… with Frequency distribution

Column / Field detailed view
… with similar columns and data patterns

Smart Domains and Column Similarity
Minimize need to define rules: just tag and let EDC do the work
Like photo tagging
CLAIRE for Columns
Column similarity based on
data overlap. Large overlap
of distinct values
Similar value frequencies for
overlapping columns
Clustering based on column
metadata and Jaccard
Coefficient & Bray Curtis
Similarity
Data Discovery | Wrap up

Lineage & Impact
Zoom controls
Click dotted line
to see flow details
Control # of
upstream levels
Control # of
downstream levels
Enable
transformation
logic details
Overlay business
titles
Open summary
table

Intelligent Business Term Associations
• CLAIRETM powered automatic
association of business terms
with physical data
• Built on top of auto-data domain
discovery and data similarity
capabilities
• Uses NLP techniques to relate
business terms to field and
column names
• Reduces a tedious manual step in
data governance
Name Asset Type Business Term Recommendation
Mil_ID Column Military ID Number
Med_Nmbr Column Practitioner Medicare number
C_ADJ_KEY Column Claim Adjustment Key
NH_Plan_ID Column National Health Plan Identifier
DSCH_STAT_CD Column Discharge Status Code
NatEmp_ID Column National Employee Identifier

Relationships
Holistic Relationships View
Get a holistic view of data in a knowledge graph that lets you quickly
search, discover, and understand enterprise data and meaningful
data relationships. Automatically discover related data sets,
technical, business, semantic and usage based relationships.

Advanced Relationship Discovery
• Detect primary
and foreign key
relationships in
data
• Assemble
composite
business entities
via ML-powered
inference of
relationships
between datasets

Collaboration and Social Curation
Certification of datasets by
SMEs/data owners
Q&A platform and change
notifications to foster
collaboration
Ratings and reviews by data
consumers

Change Notifications
• Follow data assets of interest, get
notified of changes
• Follow specific changes to objects
- source changes, enrichments,
collaboration updates
• Watch for changes at individual
asset level or for an entire resource
• Change notifications sent via in
app notification center, event email,
periodic digest email
Get notified when new
assets are introduced to
apply data standards and
associate business
context
Data Steward
Manage change impact
by tracking changes at
the data source level
DB Admin
Stay on top of changes
to important datasets
and reports
Data Consumer

Data Asset Analytics
Instant visibility to data asset
usage, inventory, enrichment,
collaboration and data value
Out-of-the box dashboards for
instant visibility with built-in
filters for quick analysis
Prepackaged and extensible
report types that are predefined
by event and audit data sets
MCM.5.3 - Metadata
and Catalog Audit
Trial

EDC Advanced Scanners
Extract detailed metadata and
lineage from complex and
legacy enterprise systems
Automatic lineage extraction
from code embedded in multi-
vendor ETL tools
Automatic lineage extraction
from both static and dynamic
code in SQL scripts and stored
procedures

50
EDC Advanced Scanners
Code and Scripting
• Oracle
• SQL Server
• Teradata
• Netezza
• Greenplum
• PostgreSQL
• MySQL
• IBM Informix
ETL Tools
• IBM Datastage
• Oracle Data Integrator
• SAP BODS
• Talend DI
• Microsoft SSIS
• Oracle Warehouse Builder
• Talend DI
Mainframes
• COBOL
• JCL
Statistical and BI Tools
• SAS
• Microsoft SSAS
• Microsoft SSRS
Enterprise
Data
Catalog
Auto extraction of data objects and relationships
Broadest and Most Complete Metadata Connectivity
Metadata-driven
Artificial Intelligence

Market Leading Active Metadata Driven Intelligence
and Automation
DATA
ANOMALY
DETECTION
ENTITY
MATCHING
MASS DATA
CORRECTION
COLUMN
SIMILARITY
SEARCH
RANKING
BUSINESS
RULES
TRANSLATION
SELF
HEALING
PROCESSING
DATASET
SIMILARITY
OPERATIONAL
ANOMALY
DETECTION
NATURAL
LANGUAGE
DESCRIPTION
OF CODE
DATA
DOMAIN
INFERENCE
SELF
TUNING
PROCESSING
DATA
RELATIONSHIP
INFERENCE
TRANSFORMATION
RECOMMENDATIONS
BUSINESS
RULE
ASSOCIATIONS
SCHEMA
MAPPING
COST OF
DATA
BREACH
ENTITY
EXTRACTION
ECONOMIC
VALUE
OF DATA
SMART DATA
VISUALIZATION
SCHEMA
INFERENCE
BUSINESS
TERM
ASSOCIATIONS
DATA
LINEAGE
INFERENCE
DATASET
RECOMMENDATIONS
PREDICTIVE
OPERATIONAL
ANALYTICS
SCHEDULE
OPTIMIZATION

What is Business
Glossary?
• Glossary Term
• Security Classification
• Assigned Data Attributes
• DQ Scorecards
• DQ Breakdown and Automation
• Stakeholders and Community
• Related Business Processes
• Related Policies
• Data Lineage across Systems (Overlays)
• DQ
• Data Sets
• Attributes
• Business Processes
• Policies
• CR Workflow
Business Glossary
Data Sets
Attributes
Processes
Systems
Policies
Data Quality
Lineage
Workflow

What is Business
Glossary?
Another example
EAD

54
Business
Process
PoV
Business
Glossary
Data
Sets
Attributes
Processes
Systems
Data
Quality
Lineage

Processes
Policies
Business
Process
PoV

Automated Data Quality Rule Application
How It Works
• Identify anomalies in data and isolate inconsistent data or
unexpected patterns
• Identify instances of business entities using data catalog
integration
• Execute Data Quality rules across all instances of the business
entity
• Data quality rules are applied consistently across the enterprise
per governance policies
Use Case Summary
Automatically apply data quality rules per governance policies
Benefits

NLP Data Quality Rule Generation
How It Works
• Translate natural language sentences to executable Data Quality
Rules: run pre-processer to generate primitives, use deep learning
to tag entities, build data quality rule
• Provides recommendation based on existing rule or generates a
new one if none found
• Business can quickly define and execute data quality rules per
governance policies automatically and consistently across the
enterprise
Use Case Summary
Automatically generate data quality rules based on natural language
and apply rules to all business entities
Benefits

58
Informatica Data Marketplace
A storefront, with order management, and governed provisioning capabilities
Fulfill & Track
Create & Publish Shop & Checkout

© Informatica. Proprietary and Confidential.
60
60
Consistent Data Quality Process and Methodology
Across All Products
Data Quality
& Governance
Process
Discover
Define
Apply
Measure
&
Monitor

61
Discover
Discovery and Profiling
Discover
Define
Apply
Measure
Profile data to examine its
structure and context using
out-of-the-box templates
Drill down to see details
and filter on results
Compare profile runs to
identify trends over time

62
Define
Business Rule Definition
Empower the business to
lead data quality initiatives
Reduce project cycles
Enable IT to focus on
strategic projects
Discover
Define
Apply
Measure

63
Apply
Centralized Re-usable Rules
Consistently apply data
quality rules across the
enterprise in support of
data governance
Reduce cost through re-use
of centrally managed data
quality rules
Streamline the resolution
of data quality issues
Discover
Define
Apply
Measure

64
Measure and Monitor
Provide Continuous Insight
Align data quality and data
governance efforts
Track data quality
improvements over time
Enable IT to focus on
strategic projects
Discover
Define
Apply
Measure

Scorecards
Provide Continuous Insight
Trends and
Weights
Counts and
Scores
Drill Down Historical
Trends

66
Data Quality Functions
james smith
1008 6th avenue suite 7
nyc new york 10018
First Name:james
Last Name: smith
AddressL1: 1008 6th avenue
AddressL2: suite 7
City: nyc
State:new york
Zip Code: 10018
First Name:james
Last Name: smith
AddressL1: 1008 Avenue of the Americas
AddressL2: Suite 7
City: New York
State:new york
Zip Code: 10018
First Name: James
Last Name: Smith
AddressL1:1008 Avenue of the Americas
AddressL2:Suite 7
City:New York
State:NY
Zip Code: 10018
First Name: Jim
Mid Name: J.
Last Name: Smyth
AddressL1: 1008 Avenue of the Americas
AddressL2: Suite 7
City: New York
State:NY
Zip Code: 10018-5402
Longitude: 40.7325525
Latitude: -74.004970
Phone: (212) 755-2551
Email: jsmyth@mywork.com
Category: Affluent Couples & Families
Group: Affluent Families
Profile
Parse
Correct
Standardize
Enrich
.
.
.
.

67
DQ Exception handling (Human tasks)

AI Powered Integrated Intelligent Platform
• What does the data mean?
• Where does it come from?
• What is the data quality?
• What are the privacy controls?
• What is the appropriate use?
• Can I use the data?
• Who can help me?
Data Governance | Axon
Scan | Identify | Tag | Classify |
Catalog Data | Curate |
Catalog
Enterprise Data Catalog
Measure Quality | Remediate
Issues | Monitoring |
Data Quality
Informatica Data Quality
Automation
Operationalize
Enabling organizations easily Define | Discover | Collaborate | Trust |
Companies that empower employees to consistently use data as a basis for their decision
making, are nearly twice as likely as others to report reaching their data and analytics objectives.
McKinsey: How Leaders in Data and Analytics Have Pulled Ahead

AI Powered Intelligent Platform
• What does the data mean?
• Where does it come from?
• What is the data quality?
• What are the privacy controls?
• What is the appropriate use?
• Can I use the data?
• Who can help me?
Data Governance | Axon
Scan | Identify | Tag | Classify |
Catalog Data | Curate |
Catalog
Enterprise Data Catalog
Measure Quality | Remediate
Issues | Monitoring |
Data Quality
Informatica Data Quality
Policy Enforcement | Risk
Reporting | Access Request |
Breach Analysis |
Data Privacy
Data Privacy Management
Automation
Operationalize
Enforcement
Enabling organizations easily Define | Discover | Collaborate | Trust |
Companies that empower employees to consistently use data as a basis for their decision
making, are nearly twice as likely as others to report reaching their data and analytics objectives.
McKinsey: How Leaders in Data and Analytics Have Pulled Ahead

Intelligent Data Management NDMO_Data Catalog and Metadata Domain Specifications coverage v1.0.pdf

More Related Content

What's hot

Similar to Intelligent Data Management NDMO_Data Catalog and Metadata Domain Specifications coverage v1.0.pdf

Recently uploaded

Intelligent Data Management NDMO_Data Catalog and Metadata Domain Specifications coverage v1.0.pdf