Data Vault Agility Bi Podium November 2013


Published on

Achieving Agility with Data Vault Data Modeling for the Data Warehouse EDW. BI Podium Conference EU Netherlands 2013.

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Data Vault Agility Bi Podium November 2013

  1. 1.  Data Vault Modeling  DW2.0 & Unstructured Data  Big Data  Ensemble Modeling  Agile DW Agile BI with the Data Vault DW Knowledge is Power. © 2013 Genesee Academy, LLC USA +1 303 526 0340 Sweden 072 736 8700 gohansgo Hans Hultgren
  2. 2. Agile BI with the Data Vault DW How is it that Data Vault supports Agile BI so well? What about the other new approaches? In this session we break down the characteristics of the data modeling approaches that support data warehouse agility. Using a Modeling Pattern Characteristics analysis, Hans will in this session compare leading Ensemble modeling methods and their unique capabilities for supporting features of your EDW program. These include Data Vault, Anchor Modeling, Focal Point, and Hyper Agility modeling. What are the strengths and what are the weaknesses for each approach? How are they deployed and what do each mean for our overall DWBI architecture? Topics covered in this session also include Unified Decomposition, the definition and purpose of Raw versus BDW layers, the role of the Information Model related to our EDW, and a discussion on where and how to apply automation tooling and techniques. © 2013 Genesee Academy, LLC 2
  3. 3. Agile BI with the Data Vault DW Knowledge is Power. | Premise #1 • You won’t achieve agility with a modeling pattern alone. • You won’t achieve agility without a modeling pattern that supports it. © 2013 Genesee Academy, LLC 3
  4. 4. Agile BI with the Data Vault DW • A Saga of Data Warehousing: Once upon a time data warehousing was becoming more popular and everyone was eager to build their own. But whenever they tried they failed. They called upon their best to fix this but they just couldn’t solve the problem. They discovered that meeting the needs of the data warehouse meant that the tables got too big and too hard to work with. They just could not handle changes over time. If the smallest thing changed it always meant they had to change the entire table. When just a single attribute was updated they had to insert a record for all of the attributes. All seemed lost. But around the world there were rebels who questioned the conventional wisdom. And their voices were finally heard: Why not separate the things that change from the things that don’t change? © 2013 Genesee Academy, LLC 4
  5. 5. Modeling Pattern Awareness • Separating the things that change from the things that don’t change. • break things out into component parts for flexibility and to capture things that – are interpreted in different ways or – changing independently of each other – Unified Decomposition™ © 2013 Genesee Academy, LLC 5
  6. 6. Ensemble Modeling™ • The constellation of component parts acts as a whole – an Ensemble. All the parts of a thing taken together, so that each part is considered only in relation to the whole. • With Ensemble Modeling the Core Business Concepts that we define and model are represented as a whole – an ensemble – including all of the component parts. © 2013 Genesee Academy, LLC 6
  7. 7. What Forms are there? Data Mart ERP EDW Acctg Data Mart Sales 3NF Anchor Ensemble Focal Point Data Vault 2G Data Mart Dimensional Temporal, 6NF, Hyper Agillity + Matter, DV2.0 + © 2013 Genesee Academy, LLC 7
  8. 8. The Data Vault Ensemble • The Data Vault Ensemble conforms to a single key – embodied in the Hub construct. • The component parts for the Data Vault Ensemble include: – Hub The Natural Business Key – Link The Natural Business Relationships – Satellite All Context, Descriptive Data and History © 2013 Genesee Academy, LLC 8
  9. 9. Comparing the Big 3 Modeling Forms • • • • Both 3NF and Dimensional use highly encapsulated concepts. All forms are Attributed and stay rather close to the Business Concept. 3NF can sometimes move towards Abstracted Concepts. None of these forms are focused on Fully Abstracted or Generic Context methods. © 2013 Genesee Academy, LLC
  10. 10. Data Vault means thinking differently Customer Customer © 2013 Genesee Academy, LLC 10
  11. 11. Agile BI with the Data Vault DW Knowledge is Power. | Premise #2 • Modeling Awareness (understanding the pattern you are applying) is the key to successful modeling. • Not knowing when you are applying a modeling exception is dangerous. © 2013 Genesee Academy, LLC 11
  12. 12. Applying the Data Vault Ensemble • Mixing “color types of data” is not Data Vaulting but rather unvaulting • A blended pattern has different dynamics… ? Thinking Differently ! © 2013 Genesee Academy, LLC 12
  13. 13. Applying the Data Vault Ensemble • Stay with the Ensemble Modeling Pattern. Continue practicing Unified Decomposition. Continue Vaulting. Be aware when you change patterns. Option 1 Option 2 or © 2013 Genesee Academy, LLC 13
  14. 14. Agile BI with the Data Vault DW Knowledge is Power. | Premise #3 • Identify your own specific Modeling Pattern (including hybrid or blended patterns) to meet the needs of your EDW. • Document your pattern and apply it consistently to gain the full benefits (including agility, repeatability and automation). © 2013 Genesee Academy, LLC 14
  15. 15. Sample: Sales Data Vault Model © 2013 Genesee Academy, LLC 15
  16. 16. Agile BI with the Data Vault DW Knowledge is Power. | Premise #4 • There is no Integration without Semantic Integration. • Integrating Data without understanding the meaning of that data is meaningless. © 2013 Genesee Academy, LLC 16
  17. 17. Agile BI with the Data Vault DW Information Modeling • Logical Models, Conceptual Models, Industry Models, Semantic Models, Taxonomies and Ontologies are all forms of Information Modeling. • These seek to target the central meaning of the core business concepts we work with and also how they relate to each other. • These should be consistent with the concepts coming from your BICC, SOA, MDM, MDD, GCI, and other organizational information modeling, data governance or business glossary initiatives. Information Model © 2013 Genesee Academy, LLC 17
  18. 18. Staging © 2013 Genesee Academy, LLC Load Transform Calculate Convert Cleanse Profile Validate Extract Raw Transform Calculate Convert Cleanse Profile Validate Integrate Load D/T Stamp Integrate Extract Architecture: Raw and BDV Information Model BDW Data Mart Data Mart Data Mart EDW 18
  19. 19. Agile BI with the Data Vault DW Knowledge is Power. | Premise #5 • Automation must address the Central Meaning of our data in order for it to be useful. • Automating the modeling and loading of a source system can never result in multi-source data integration. © 2013 Genesee Academy, LLC 19
  20. 20. Automation Matrix Consider and compare the full set of capabilities for your automation solutions.
  21. 21. Process for modeling/deploying DV EDW • Business Driven Modeling. Since an EDW integrates data from several sources, departments, divisions, functions, and countries over time, the integration target needs to be based on the organizations central view and not on a handful of source systems that happen to be in scope at the time. t + 10 yrs Now / Today t - 10 yrs © 2013 Genesee Academy, LLC Information Model EDW
  22. 22. The Enterprise Data Warehouse Information Model Data © 2013 Genesee Academy, LLC Stage Data Warehouse Marts Info
  23. 23. Recap • Agile modeling for the DW comes from Unified Decomposition • Data Vault modeling leads the Ensemble Modeling family • Knowledge is Power: • • • • • Agility is more than modeling | Modeling must support it Know your modeling pattern | Know your architecture Customize your pattern, but | Apply it consistently Integration requires meaning | Create an Information Model Understand what you automate | Automate beyond sources © 2013 Genesee Academy, LLC 23
  24. 24. About Data Vault Ensemble Estimated 800 Data Vault based Data Warehouses around the world © 2013 Genesee Academy, LLC 24
  25. 25. Links and Information CDVDM Training & Certification gohansgo Book HansHultgren DataVaultAcademy Online video-lesson training © 2013 Genesee Academy, LLC 25