The document discusses the need for a metadata framework to improve data management, addressing issues such as data quality, manual report preparation, and efficiency. It emphasizes the importance of making data visible and optimizations measurable while providing a structure for automated processes. The framework aims to facilitate the rapid prototyping and analysis of data, enabling business users to map data effectively and streamline the creation of data warehouses.

1. Mark Nießen
(mark.niessen@mepos.org)
Copyright © 2020
All rights reserved
All you needneed is ...
MMeta
DData
FFrameworkramework

2. Do you know that?
●
Dissatisfied with Data-warehouse
●
Missing Data Quality (Partially no reliable
figures)
●
Huge Manuel Effort for Report preparation
●
Long Running Processes
●
From Requirement to Report, take ages..
●
Different Tools and Approaches in place
●
……...

3. What do you need ??What do you need ??
All Your Data
VisibleVisible and
Optimisations MeasurableOptimisations Measurable
MMeta DData
FFramework (MDFMDF)
CustomerCustomer
ProductProduct
Then you need a:

4. The value chainThe value chain
Customer
Product
Change
is a multidimensionalmultidimensional construct
with many actorsmany actors and
their own viewsown views
which must be made
visiblevisible and optimizationsoptimizations
measurablemeasurable.

5. Analysis and rapid prototypingAnalysis and rapid prototyping
1.Prototype
2.Review
3.Refine
CustomerCustomer
ProductProduct
functional reportingfunctional reporting
results in
that leads to make
context visiblecontext visible
and becomes
understandableunderstandable

6. Meta Data is...Meta Data is...
"data that provides information about other data".
In other words,
it is
““data about data”data about data”
Customer
Product

7. A Framework is...A Framework is...
A structure that forms
a support and a frame
for your datafor your data

8. The basis for actionThe basis for action
Making optimizationsMaking optimizations
measurablemeasurable
MMeta DData FFramework
(MDFMDF)
+
Multidimensional View on
Value ChainValue Chain
+
AnalyseAnalyse
and
Rapid PrototypingRapid Prototyping

9. Analyze:
●
Requirements
●
Data Profiling
●
Lineage
●
Meta Data
Create Change:
●
Add Sources
●
Mapping Source→DWH RawVault
●
(Hubs/Sats/Lnks)
●
Automatic creation
of Tables and Stored Procedures
Execute:
●
Job creation and
scheduling
●
Parallel processing
based on dependencies
and sources
●
TSQL, SSIS,
PowerShell, etc.
Monitoring:
●
Errors
●
When?
●
What?
●
How much?
●
Changes to Source
Systems
Design:
●
Templates
●
Raw and Business Vault
●
Information Layer with
Business User
●
Master Data
Management
Version Control:
●
Track Changes
●
Versioning
●
Documentation
Deploy:
●
Test Automation
●
Deployment
Pipelines
●
Continuous
Delivery
●
Dev Ops
MDFMDF
The full
cycle :

10. Done and Tested
●
The Metadata Framework enables the fully automated creation and parallel loading of
a DataVault 2.0 DWH from source to raw core.
●
All sources are described in their structures/characteristics (tables, columns, data
types, etc.) and assigned to a corresponding table/column in the Raw Vault.
●
This mapping enables a further description of metadata relationships in the form of a
source and target description.
●
The high degree of automation provided by the metadata framework enables the fast
capture of systems and their changes, as well as the parallel loading of the required
data in a standardized procedure that eliminates manual, time-consuming, repetitive
and error-prone work and is scalable at the same time.
●
It is written in TSQL
●
Enables non-technical business users to create a mapping of data, since the
RawVault and Information Layer is designed for business understandable

11. Next Step Implementation
●
Mapping to Point in Time Tables (PIT)
●
Automatic creation of Business Core, Information Layer and Tabular Models
●
Azure Polybase and CTAS Creation
●
Automatic creation of Azure Resources
●
Indexing and Partitioning
●
Web Interface
I´m looking for partners to do further
development