Petr Hájek November 25, 2020
Webinar:
Data Landscape Mapping
2
Too much data…
3
Typical responses to “problems with data”
Metadata
Governance
Data
Warehouse
Data
Stewardship
Data
Stewardship
Data
Governance
Officer
Data
Quality
Department
Master Data
Management
Information
Management
Competence
Data
Architecture
Operational
Data Store
Business
Glossary
Data
Dictionary
Data
Management
Program
4
Each good story book
begins with a map
5
How to achieve a “Data Transparency”
The goal is to prepare multi-dimensional or layered map in the form of
(semi-)structured metadata which will allow us to browse through the
enterprise data landscape like in any geographical digital map.
We call this process a “Data Landscape Mapping”
6
Metadata structure for Data Transparency Model
DATA
ELEMENT
Logical
Model
Entity
Business
Process
Mapping
Physical
Data
Storage
Data
Lineage
Data
Utilisation
Information
Security
& Privacy
Detected
Semantic
Data
Profile
Data
Quality
Ownership
3
2
1
4
7
Before you start
› Do not be ashamed for Excel
(Do not start with oversized data
management toolsets)
› Combine manual, automated and
semi-automated activities
› Allow for ‘Hic Sunt Leones’
places in your map
8
Step 1 – Logical Data Model:
What data?
› Identifications of entities
› Business definitions
of entities
› Structures of entities,
their attributes and
relationships
9
Step 2 – Physical Data Stores:
Where is the data?
› Where is the data physically?
› Are there any overlaps in the
data?
› Do we need to bother with data
consolidation?
› Shall we aspire for “golden
records”?
› What are the volumes of the
data?
› What are numbers of records?
› What are daily increments
of the data?
› How much data is changed
during the day/month/year?
Semantic Model Real World
Physical Data
Stores
10
Step 3 – Business Processes Context:
Who needs the data?
› How frequently do we need
to “touch” the data?
› How frequently to we need
to update/refresh the data?
› Are answers for these questions the
same equally for all business
processes?
› Or, are there different needs for the
data in terms of accessibility, level of
detail, data quality, frequency etc.?
› What is the quality of data?
› Are we able to define it and
measure it?
Credit:
https://medium.com/@sonicmsba/how-to-
build-an-effective-business-context-for-
data-analytical-problems-cb02906341cd
Business
Context
Modeling
Data
Garage
Storytelling
11
Step 4 – Organization dimension:
Who owns the data?
› Who is responsible owner of the data?
› Who understands the data?
› Who takes care of the data?
12
Metadata for Data Transparency Model
DATA
ELEMENT
Logical
Model
Entity
Business
Process
Mapping
Physical
Data
Storage
Data
Lineage
Data
Utilisation
Information
Security
& Privacy
Detected
Semantic
Data
Profile
Data
Quality
Ownership
13
Metadata Model – Reductio ad absurdum
DATA_OBJECT DATA_OBJECT_
INSTANCE
ATTRIBUTE ATTRIBUTE_
INSTANCE
DATA_ELEMENT DATA_ELEMENT_
INSTANCE
14
Present your maps
1 7 3,5 5 0,5
Business
Proces 1
Business
Proces 2
Business
Proces 3
Business
Proces 4
Business
Proces 5
1 System A 100% 14% 29% 20% 200%
15 System B 1500% 214% 429% 300% 3000%
3 System C 300% 43% 86% 60% 600%
0,5 System D 50% 7% 14% 10% 100%
1 System E 100% 14% 29% 20% 200%
4 System F 400% 57% 114% 80% 800%
5 System G 500% 71% 143% 100% 1000%
3 System H 300% 43% 86% 60% 600%
17 System I 1700% 243% 486% 340% 3400%
3 System J 300% 43% 86% 60% 600%
10 System K 1000% 143% 286% 200% 2000%
DataRetentionCapacity(yrs)
Data Retention Requirements (yrs)
15
Meta MartmDWH
Metadata
sources
What next? Build your “Metadata Warehouse”
Standard Business DWH solution
Stage / Data Lake DWH Core Data Mart
Integrated Metadata solution
Data Load Data Integration Data Usage
Ingest Metadata Organize Metadata Consume Metadata
16
Questions
& Answers
Profinit EU, s.r.o.
Tychonova 2, 160 00 Praha 6 | Telefon + 420 224 316 016
Web
www.profinit.eu
LinkedIn
linkedin.com/company/profinit
Twitter
twitter.com/Profinit_EU
Facebook
facebook.com/Profinit.EU
Youtube
Profinit EU
Thanks
Backup Slides

4 Steps Towards Data Transparency

  • 1.
    Petr Hájek November25, 2020 Webinar: Data Landscape Mapping
  • 2.
  • 3.
    3 Typical responses to“problems with data” Metadata Governance Data Warehouse Data Stewardship Data Stewardship Data Governance Officer Data Quality Department Master Data Management Information Management Competence Data Architecture Operational Data Store Business Glossary Data Dictionary Data Management Program
  • 4.
    4 Each good storybook begins with a map
  • 5.
    5 How to achievea “Data Transparency” The goal is to prepare multi-dimensional or layered map in the form of (semi-)structured metadata which will allow us to browse through the enterprise data landscape like in any geographical digital map. We call this process a “Data Landscape Mapping”
  • 6.
    6 Metadata structure forData Transparency Model DATA ELEMENT Logical Model Entity Business Process Mapping Physical Data Storage Data Lineage Data Utilisation Information Security & Privacy Detected Semantic Data Profile Data Quality Ownership 3 2 1 4
  • 7.
    7 Before you start ›Do not be ashamed for Excel (Do not start with oversized data management toolsets) › Combine manual, automated and semi-automated activities › Allow for ‘Hic Sunt Leones’ places in your map
  • 8.
    8 Step 1 –Logical Data Model: What data? › Identifications of entities › Business definitions of entities › Structures of entities, their attributes and relationships
  • 9.
    9 Step 2 –Physical Data Stores: Where is the data? › Where is the data physically? › Are there any overlaps in the data? › Do we need to bother with data consolidation? › Shall we aspire for “golden records”? › What are the volumes of the data? › What are numbers of records? › What are daily increments of the data? › How much data is changed during the day/month/year? Semantic Model Real World Physical Data Stores
  • 10.
    10 Step 3 –Business Processes Context: Who needs the data? › How frequently do we need to “touch” the data? › How frequently to we need to update/refresh the data? › Are answers for these questions the same equally for all business processes? › Or, are there different needs for the data in terms of accessibility, level of detail, data quality, frequency etc.? › What is the quality of data? › Are we able to define it and measure it? Credit: https://medium.com/@sonicmsba/how-to- build-an-effective-business-context-for- data-analytical-problems-cb02906341cd Business Context Modeling Data Garage Storytelling
  • 11.
    11 Step 4 –Organization dimension: Who owns the data? › Who is responsible owner of the data? › Who understands the data? › Who takes care of the data?
  • 12.
    12 Metadata for DataTransparency Model DATA ELEMENT Logical Model Entity Business Process Mapping Physical Data Storage Data Lineage Data Utilisation Information Security & Privacy Detected Semantic Data Profile Data Quality Ownership
  • 13.
    13 Metadata Model –Reductio ad absurdum DATA_OBJECT DATA_OBJECT_ INSTANCE ATTRIBUTE ATTRIBUTE_ INSTANCE DATA_ELEMENT DATA_ELEMENT_ INSTANCE
  • 14.
    14 Present your maps 17 3,5 5 0,5 Business Proces 1 Business Proces 2 Business Proces 3 Business Proces 4 Business Proces 5 1 System A 100% 14% 29% 20% 200% 15 System B 1500% 214% 429% 300% 3000% 3 System C 300% 43% 86% 60% 600% 0,5 System D 50% 7% 14% 10% 100% 1 System E 100% 14% 29% 20% 200% 4 System F 400% 57% 114% 80% 800% 5 System G 500% 71% 143% 100% 1000% 3 System H 300% 43% 86% 60% 600% 17 System I 1700% 243% 486% 340% 3400% 3 System J 300% 43% 86% 60% 600% 10 System K 1000% 143% 286% 200% 2000% DataRetentionCapacity(yrs) Data Retention Requirements (yrs)
  • 15.
    15 Meta MartmDWH Metadata sources What next?Build your “Metadata Warehouse” Standard Business DWH solution Stage / Data Lake DWH Core Data Mart Integrated Metadata solution Data Load Data Integration Data Usage Ingest Metadata Organize Metadata Consume Metadata
  • 16.
  • 17.
    Profinit EU, s.r.o. Tychonova2, 160 00 Praha 6 | Telefon + 420 224 316 016 Web www.profinit.eu LinkedIn linkedin.com/company/profinit Twitter twitter.com/Profinit_EU Facebook facebook.com/Profinit.EU Youtube Profinit EU Thanks
  • 18.