Your SlideShare is downloading. ×
0
© 2014 Genesee Academy, LLC
 Data Modeling  Data Vault Modeling  Big Data  Agile DW Ensemble Modeling Certification
...
© 2014 Genesee Academy, LLC 2
CDVDM ReConnect Event
© 2014 Genesee Academy, LLC 3
© 2014 Genesee Academy, LLC
Then & Now PresentationAgenda
• Looking Back & Progress
• Colors and Reverse Engineering
• Bus...
© 2014 Genesee Academy, LLC 5
Then and Now…
2007 *2008 *2009 *2010*2011*2012 *2013 *2014
© 2014 Genesee Academy, LLC
Genesee Academy Activities
6
Seminars
Advising
Online
Conferences
© 2014 Genesee Academy, LLC
Genesee Academy Activities
38%
29%
17%
14%
GA Activities
Seminars
Advising
Online
Conferences
...
© 2014 Genesee Academy, LLC
Unified Decomposition™
8
• With the EDW, we seek to break things out into component parts for
...
© 2014 Genesee Academy, LLC
Ensemble Modeling™
9
All the parts of a thing taken together, so that
each part is considered ...
© 2014 Genesee Academy, LLC
The Data Vault Ensemble
10
• The Data Vault Ensemble conforms to a single key – embodied in th...
© 2014 Genesee Academy, LLC
Data Vault means thinkingdifferently
11
Customer
Customer
• The minimal constructthen for an “...
© 2014 Genesee Academy, LLC
Data Vault means thinkingdifferently
12
Customer
Customer
© 2014 Genesee Academy, LLC
DV versus 3NF
Sat
Sat
SatSat
Sat
Sat
Sat
Sat
Sat
SatSatSat
13
EDWHistoryOperational
© 2014 Genesee Academy, LLC
The Data Vault modeling approach
• As the scope of the EDW is expanded and new data sources ad...
© 2014 Genesee Academy, LLC
Data Vault Modeling Process
• The Modeling Process for creating a Data Vault model includes
th...
© 2014 Genesee Academy, LLC 16
Anatomy of a Hub
© 2014 Genesee Academy, LLC 17
Anatomy of a Link
© 2014 Genesee Academy, LLC 18
Anatomy of a Satellite
© 2014 Genesee Academy, LLC
Sales DV Model - Backbone
19
SampleModel
© 2014 Genesee Academy, LLC
Sample: Sales Data Vault Model
20
© 2014 Genesee Academy, LLC
Identifying the Core Business Concepts
21
© 2014 Genesee Academy, LLC
Business Key?
• The Business Key that forms the basis of the Hub should be:
– Enterprise Wide ...
© 2014 Genesee Academy, LLC
Starting with Stars
• Begins to get complicated…
Star 1
Reach complexity and lack of agility l...
© 2014 Genesee Academy, LLC
Adapting & Expanding the EDW
• With Data Vault, scale easily – without re-engineering!
Star 1
...
© 2014 Genesee Academy, LLC
FundamentalArchitecture
Data Mart
Star
Schema
Other Marts
& Error
Marts
Enterprise DWBI
Soluti...
© 2014 Genesee Academy, LLC
Identifying relationships that are really Ensembles
• Rules and Guidelines
• Does the Link hav...
© 2014 Genesee Academy, LLC
Applying the Data Vault Ensemble
27
• Mixing “color types of data” is not Data Vaulting but
ra...
© 2014 Genesee Academy, LLC
Sourcing the Data Vault EDW
28
• Sourcing Data Vault requires more joins (Hub to Sats, 2 sides...
© 2014 Genesee Academy, LLC
Link:Link:Link
29
• What does a L:L:L mean?
• Can a relationship have relationships to other r...
© 2014 Genesee Academy, LLC 30
Benefits of Data Vault Modeling
Agility Auditability History Scalability Simplicity Loadabi...
© 2014 Genesee Academy, LLC
• Financial Institutions
• Telecommunications
• Retail
• Manufacturing
• Technology
• Energy &...
© 2014 Genesee Academy, LLC 32
© 2014 Genesee Academy, LLC
Links and Information
CDVDM Training & Certification
www.GeneseeAcademy.com
Hans@GeneseeAcadem...
Upcoming SlideShare
Loading in...5
×

2014 Data Vault ReConnect Event Then & Now DDVM

129

Published on

From the June 5 Dutch Data Vault Masters Event in Amsterdam. This CDVDM Reconnect / Recertification day included presentations from several certified data vault data modelers. This particular presentation was part of a discussion on "then and now" for data vault in the Netherlands.

Published in: Data & Analytics, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
129
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
12
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "2014 Data Vault ReConnect Event Then & Now DDVM"

  1. 1. © 2014 Genesee Academy, LLC  Data Modeling  Data Vault Modeling  Big Data  Agile DW Ensemble Modeling Certification CDVDM Recertification Event Data Vault: Then & Now © 2014 Genesee Academy, LLC USA +1 303 526 0340 Sweden 072 736 8700 Hans@GeneseeAcademy.com www.GeneseeAcademy.com CDVDM ReConnect 2014 gohansgo
  2. 2. © 2014 Genesee Academy, LLC 2 CDVDM ReConnect Event
  3. 3. © 2014 Genesee Academy, LLC 3
  4. 4. © 2014 Genesee Academy, LLC Then & Now PresentationAgenda • Looking Back & Progress • Colors and Reverse Engineering • Business Oriented Modeling • Effective Dates • Architecture Revisited • Link Unique Specific Natural • Thinking Differently • Modeling Address • Sourcing the Data Vault • The L:L:L constructs • Automation Mini-Topics for 5x5 Updates • Ensemble Modeling • Core Business Concepts • The Business Key • Unit of Work & Possessive • Raw versus Business • Link & Why its not an Event • Satellite & Why its not MV • Big Data & Unstructured • SuccessfulAgile DV DW • Industry Reference Models • Ensemble Forms 4 AGENDA ITEMS
  5. 5. © 2014 Genesee Academy, LLC 5 Then and Now… 2007 *2008 *2009 *2010*2011*2012 *2013 *2014
  6. 6. © 2014 Genesee Academy, LLC Genesee Academy Activities 6 Seminars Advising Online Conferences
  7. 7. © 2014 Genesee Academy, LLC Genesee Academy Activities 38% 29% 17% 14% GA Activities Seminars Advising Online Conferences 7 Genesee Academy, LLC – World Class Training • Seminars – 1-4 day, on-location& in-company courses. – Certifications issuedby GA. – Blended(hybrid) Pedagogy. • Advising – DWBI Programs, Modeling Patterns, Enterprise Architecture, Agility, etc. – Reviews:Programs, Models, Architectures, etc. • Online – Classroomstudio, online, on-demandvideolessons. – Multiple channels DVA andTrainOvation. • Conferences – Speaking, Presenting, andsometimescoordinating industry conferencesaroundthe globe.
  8. 8. © 2014 Genesee Academy, LLC Unified Decomposition™ 8 • With the EDW, we seek to break things out into component parts for flexibility, adaptability, agility, and generally to facilitate the capture of things that are either interpreted in different ways or changing independentlyof each other. • At the same time a core premise of data warehousing is integration and moving to a common standard view of unified concepts. So we also want to tie things together – to Unify.
  9. 9. © 2014 Genesee Academy, LLC Ensemble Modeling™ 9 All the parts of a thing taken together, so that each part is considered only in relation to the whole. • The constellation of component parts acts as a whole – an Ensemble. • With Ensemble Modeling the Core Business Concepts that we define and model are represented as a whole – an ensemble – including all of the component parts. An Ensemble is based on all things defining a Core Business Concept that can be uniquely and specifically said for one instance of that Concept.
  10. 10. © 2014 Genesee Academy, LLC The Data Vault Ensemble 10 • The Data Vault Ensemble conforms to a single key – embodied in the Hub construct. • The component parts for the Data Vault Ensemble include: – Hub The Natural Business Key – Link The Natural Business Relationships – Satellite All Context, Descriptive Data and History
  11. 11. © 2014 Genesee Academy, LLC Data Vault means thinkingdifferently 11 Customer Customer • The minimal constructthen for an “entity” such as “Customer” is now a Hub with a set of Satellites
  12. 12. © 2014 Genesee Academy, LLC Data Vault means thinkingdifferently 12 Customer Customer
  13. 13. © 2014 Genesee Academy, LLC DV versus 3NF Sat Sat SatSat Sat Sat Sat Sat Sat SatSatSat 13 EDWHistoryOperational
  14. 14. © 2014 Genesee Academy, LLC The Data Vault modeling approach • As the scope of the EDW is expanded and new data sources added, the Data Vault can adapt to these changes without impacting the existing model. This is what allows the EDW to be built incrementallyand to adapt to change without the need for re-engineering. New Area absorbed 14 H_Cust H_Sale H_Empl H_Store H_Car
  15. 15. © 2014 Genesee Academy, LLC Data Vault Modeling Process • The Modeling Process for creating a Data Vault model includes three primary steps: 1) Identify and Model your Core Business Concepts • Business Interviews is at the heart of this step What do you do? What are the main things you work with? • Also find best/target Natural Business Key 2) Identify and Model your Natural Business Relationships • Specific Unique Relationships • Be considerate of the Unit of Work and Grain 3) Analyze and Design your Context Satellites • Consider Rate of Change, Type of Data and also the Sources of your data during design process 15
  16. 16. © 2014 Genesee Academy, LLC 16 Anatomy of a Hub
  17. 17. © 2014 Genesee Academy, LLC 17 Anatomy of a Link
  18. 18. © 2014 Genesee Academy, LLC 18 Anatomy of a Satellite
  19. 19. © 2014 Genesee Academy, LLC Sales DV Model - Backbone 19 SampleModel
  20. 20. © 2014 Genesee Academy, LLC Sample: Sales Data Vault Model 20
  21. 21. © 2014 Genesee Academy, LLC Identifying the Core Business Concepts 21
  22. 22. © 2014 Genesee Academy, LLC Business Key? • The Business Key that forms the basis of the Hub should be: – Enterprise Wide Unique – Central Business View Aligned This means that: – It is not a “Technical Key” but rather a “Business Key” – It is not the source system primary key (id) – It is not driven by any one source system – Should be aligned with central business initiatives In a data warehouse this means: – Will have clashes – Will have duplicates 22
  23. 23. © 2014 Genesee Academy, LLC Starting with Stars • Begins to get complicated… Star 1 Reach complexity and lack of agility level… Star 2 Star 3 Star 4 Star 5 Star 6 Star 7 Star 8 Star 9 Star 10 Star 11 Star n… 23 Accounting Finance Logistics Sales
  24. 24. © 2014 Genesee Academy, LLC Adapting & Expanding the EDW • With Data Vault, scale easily – without re-engineering! Star 1 Easily adapts to changes… Star 2 Star 3 Star 4 Star 5 Star 6 Star 7 Star 8 Star 9 Star 10 Star 11 Star n… EDWDV EDW 24 Accounting Finance Logistics Sales
  25. 25. © 2014 Genesee Academy, LLC FundamentalArchitecture Data Mart Star Schema Other Marts & Error Marts Enterprise DWBI Solution Load Transform Calculate Convert Cleanse Profile Validate Extract Load D/TStamp Integrate Extract Staging EDW Transform Calculate Convert Cleanse Profile Validate Integrate Raw BDW * Integrate * Align * Reconcile Mart Specific Rules Common Business Rules 25 Data Mart Star Schema
  26. 26. © 2014 Genesee Academy, LLC Identifying relationships that are really Ensembles • Rules and Guidelines • Does the Link have its own Business Key? • Does the Link represent its own Core Business Concept? • Are there several Satellites on the Link? • Are there many attributes to describe the Link? • Are there relationships (Link to Link) with this Link? IF YES to any of these questions then the Link is Likely a Hub. When a Link becomes a Hub 26
  27. 27. © 2014 Genesee Academy, LLC Applying the Data Vault Ensemble 27 • Mixing “color types of data” is not Data Vaulting but rather unvaulting * A blended pattern has different dynamics Thinking Differently • Stay with the Ensemble Modeling Pattern. Continue practicing Unified Decomposition. Continue Vaulting. Be aware when you change patterns. Option 1 Option 2 Option 3
  28. 28. © 2014 Genesee Academy, LLC Sourcing the Data Vault EDW 28 • Sourcing Data Vault requires more joins (Hub to Sats, 2 sides of Links) • Sourcing Data Vault can be more efficient than sourcing other forms • Primary path to efficient sourcing is thinking differently… 1. ETL team needs to understand the DV model to be efficient 2. Automation and templates for repeatable patterns make this easier 3. Pulling context fromsubset of Satellites eases this join impact 4. Hubs and Links are thin and short tables with no redundancy (fast) 5. Data Marts should not be based on creating another copy of DW 6. Data Mart design should be agile,purpose-built, and business driven 7. Data Marts should pass the virtualizationtest 8. Tune with PITS, Bridges,other Mart Stage views (& materialized)
  29. 29. © 2014 Genesee Academy, LLC Link:Link:Link 29 • What does a L:L:L mean? • Can a relationship have relationships to other relationships? Whenever you see a Link:Link you should take a moment to find the Hub you are missing. Either there or not yet modeled. • Automation:
  30. 30. © 2014 Genesee Academy, LLC 30 Benefits of Data Vault Modeling Agility Auditability History Scalability Simplicity Loadability Responds Faster & Costs Less
  31. 31. © 2014 Genesee Academy, LLC • Financial Institutions • Telecommunications • Retail • Manufacturing • Technology • Energy & Utility • HealthCare • Consultancy • Transportation • Government • Gaming • Etc. 31 Applying Data Vault
  32. 32. © 2014 Genesee Academy, LLC 32
  33. 33. © 2014 Genesee Academy, LLC Links and Information CDVDM Training & Certification www.GeneseeAcademy.com Hans@GeneseeAcademy.com gohansgo Book DataVaultBook.blogspot.com HansHultgren.WordPress.com HansHultgren 33 Online video-lesson training DataVaultAcademy.com DataVaultAcademy
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×