SlideShare a Scribd company logo
DEMYSTIFYING DATA VAULT
2.0
What is the current landscape for
Information Managers?
Security
Open Data
» Lockdown vs. Democratisation
» Redaction
» De-personalisation
Regulatory compliance
» GDPR
» SOX
» FMA/APRA
» Cloud / ISO27001
External factors
GDPR
› This is likely to be replicated in
New Zealand
› Security Data Breach Notification Law in
effect as at 25 May 2018
› Lack of compliance will result in
heavy penalties
› More data more often
› New data sets all the time
› Data Quality challenges
› Corporate memory
› Reliance on individuals
› Reconciliation and audit
Internal factors
What else?
Data Vault vs.
Kimball/Inmon/L
ake
What’s wrong with traditional Data Warehousing methods?
‘Shadow IT’
Big Data
E
T
L
M
A
R
T
S
3rd party
data
Source
System
Source
System
JSON/XML
Semi-
structured
Unstruct-
ured data
BI
Analytics
Data Science
Automation Framework
Data Acquisition
Real-Time
CDC
Messaging
Batch
ETL/ELT
Files
PDF, Docs,
Video, etc.
Staging
Raw
Data
Vault
Data Provisioning
Information Governance - Metadata Management, Lineage, Data Quality
Big Data/No SQL
Information
Marts
(Virtual/Physical)
Operational Data
Store
Business
Data
Vault
Enterprise Data
Vault
Data Vault: Scalable, Extensible, Agnostic
3rd party
data
Source
System
JSON/XML
Semi-
structured
Unstruct-
ured data
BI
Analytics
Data Science
"It is not the strongest of the species that survives, nor the most intelligent
that survives. It is the one that is the most adaptable to change." Charles
Darwin
Data Lake
› Data flows in ‘naturally’
› Some boundaries
› Content flows out with little constraint
Data Swamp
› Uncontrolled flow
› No borders and potentially bottomless
› Filled with flotsam and jetsam
Data Harbour
› Controlled flow
› Trust in the delivery and content of data
› Built to use and extract value from the data
What is Data Vault?
› Invented by Dan Linstedt in the late 1990s, Data Vault is
a System of Information Delivery containing the
necessary components needed to accomplish
enterprise vision in Data Warehousing and Business
Intelligence
› Data Vault includes a data modelling technique for data
repositories that has significant advantages over
traditional methodologies: auditable, extensible,
automated
› With the release of Data Vault 2.0 in 2013, it extended
from just the data model to a full methodology
› Is effectively vendor agnostic: works with multiple data
processing tools, relational databases, and file stores
DV2.0 System: Foundational Pillars
Methodology
Architecture
Model
• Consistent & Repeatable
• Pattern Based
• Automation
• Agile
• Multi-Tier
• Scalable
• Supports NoSQL
• Insert only architecture
• Flexible
• Scalable (Big Data)
• Hub & Spoke
Demystifying Data Vault
Automati
on
Integrati
on
Governa
nce

More Related Content

What's hot

Moving beyond Big Data, BAE Systems Detica
Moving beyond Big Data, BAE Systems Detica Moving beyond Big Data, BAE Systems Detica
Moving beyond Big Data, BAE Systems Detica
Internet World
 

What's hot (20)

Your Worst GDPR Nightmare - Unstructured Data
Your Worst GDPR Nightmare - Unstructured DataYour Worst GDPR Nightmare - Unstructured Data
Your Worst GDPR Nightmare - Unstructured Data
 
Load Balancing and Data Management in Cloud Computing
Load Balancing and Data Management in Cloud ComputingLoad Balancing and Data Management in Cloud Computing
Load Balancing and Data Management in Cloud Computing
 
Expanded top ten_big_data_security_and_privacy_challenges
Expanded top ten_big_data_security_and_privacy_challengesExpanded top ten_big_data_security_and_privacy_challenges
Expanded top ten_big_data_security_and_privacy_challenges
 
Information governance: Can Blockchain be the answer?
Information governance: Can Blockchain be the answer?Information governance: Can Blockchain be the answer?
Information governance: Can Blockchain be the answer?
 
Moving beyond Big Data, BAE Systems Detica
Moving beyond Big Data, BAE Systems Detica Moving beyond Big Data, BAE Systems Detica
Moving beyond Big Data, BAE Systems Detica
 
Where's My Data? Managing the Data Residency Challenge
Where's My Data? Managing the Data Residency ChallengeWhere's My Data? Managing the Data Residency Challenge
Where's My Data? Managing the Data Residency Challenge
 
Corporate & Regulatory Compliance Boot Camp - Data Privacy Compliance
Corporate & Regulatory Compliance Boot Camp - Data Privacy ComplianceCorporate & Regulatory Compliance Boot Camp - Data Privacy Compliance
Corporate & Regulatory Compliance Boot Camp - Data Privacy Compliance
 
De groote de man Ingrid de Poorter
De groote de man Ingrid de PoorterDe groote de man Ingrid de Poorter
De groote de man Ingrid de Poorter
 
Data Discovery Automation: How to Save Time & Protect Customer Data
Data Discovery Automation: How to Save Time & Protect Customer DataData Discovery Automation: How to Save Time & Protect Customer Data
Data Discovery Automation: How to Save Time & Protect Customer Data
 
Partner Keynote: How Logical Data Fabric Knits Together Data Visualization wi...
Partner Keynote: How Logical Data Fabric Knits Together Data Visualization wi...Partner Keynote: How Logical Data Fabric Knits Together Data Visualization wi...
Partner Keynote: How Logical Data Fabric Knits Together Data Visualization wi...
 
Modern content management technology
Modern content management technologyModern content management technology
Modern content management technology
 
DIMS- Digital Image Management System
DIMS- Digital Image Management SystemDIMS- Digital Image Management System
DIMS- Digital Image Management System
 
BigID, OneTrust, IAPP Webinar: Bridging the Privacy Office with IT
BigID, OneTrust, IAPP Webinar: Bridging the Privacy Office with ITBigID, OneTrust, IAPP Webinar: Bridging the Privacy Office with IT
BigID, OneTrust, IAPP Webinar: Bridging the Privacy Office with IT
 
Collibra Data Citizen '19 - Bridging Data Privacy with Data Governance
Collibra Data Citizen '19 - Bridging Data Privacy with Data Governance Collibra Data Citizen '19 - Bridging Data Privacy with Data Governance
Collibra Data Citizen '19 - Bridging Data Privacy with Data Governance
 
Realizing the Value of Social: Evolving from Social Media to Customer Experience
Realizing the Value of Social: Evolving from Social Media to Customer ExperienceRealizing the Value of Social: Evolving from Social Media to Customer Experience
Realizing the Value of Social: Evolving from Social Media to Customer Experience
 
National Archives Corp. Bulgaria
National Archives Corp. Bulgaria National Archives Corp. Bulgaria
National Archives Corp. Bulgaria
 
Dcg cba legal ethics and the cloud final 06.20.17
Dcg cba legal ethics and the cloud final 06.20.17Dcg cba legal ethics and the cloud final 06.20.17
Dcg cba legal ethics and the cloud final 06.20.17
 
Privacera Databricks CCPA Webinar Feb 2020
Privacera Databricks CCPA Webinar Feb 2020Privacera Databricks CCPA Webinar Feb 2020
Privacera Databricks CCPA Webinar Feb 2020
 
CCPA Compliance for Analytics and Data Science Use Cases with Databricks and ...
CCPA Compliance for Analytics and Data Science Use Cases with Databricks and ...CCPA Compliance for Analytics and Data Science Use Cases with Databricks and ...
CCPA Compliance for Analytics and Data Science Use Cases with Databricks and ...
 
Webinar: How to Design a Compliant and GDPR Ready Collaboration System
Webinar: How to Design a Compliant and GDPR Ready Collaboration SystemWebinar: How to Design a Compliant and GDPR Ready Collaboration System
Webinar: How to Design a Compliant and GDPR Ready Collaboration System
 

Similar to Dv decision makers presentation 310518[1]

Similar to Dv decision makers presentation 310518[1] (20)

A Key to Real-time Insights in a Post-COVID World (ASEAN)
A Key to Real-time Insights in a Post-COVID World (ASEAN)A Key to Real-time Insights in a Post-COVID World (ASEAN)
A Key to Real-time Insights in a Post-COVID World (ASEAN)
 
KASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
KASHTECH AND DENODO: ROI and Economic Value of Data VirtualizationKASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
KASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
 
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
 
Bridging the Last Mile: Getting Data to the People Who Need It
Bridging the Last Mile: Getting Data to the People Who Need ItBridging the Last Mile: Getting Data to the People Who Need It
Bridging the Last Mile: Getting Data to the People Who Need It
 
Data Governance, Compliance and Security in Hadoop with Cloudera
Data Governance, Compliance and Security in Hadoop with ClouderaData Governance, Compliance and Security in Hadoop with Cloudera
Data Governance, Compliance and Security in Hadoop with Cloudera
 
Why a Data Services Marketplace is Critical for a Successful Data-Driven Ente...
Why a Data Services Marketplace is Critical for a Successful Data-Driven Ente...Why a Data Services Marketplace is Critical for a Successful Data-Driven Ente...
Why a Data Services Marketplace is Critical for a Successful Data-Driven Ente...
 
Data Ninja Webinar Series: Realizing the Promise of Data Lakes
Data Ninja Webinar Series: Realizing the Promise of Data LakesData Ninja Webinar Series: Realizing the Promise of Data Lakes
Data Ninja Webinar Series: Realizing the Promise of Data Lakes
 
Data Democratization for Faster Decision-making and Business Agility (ASEAN)
Data Democratization for Faster Decision-making and Business Agility (ASEAN)Data Democratization for Faster Decision-making and Business Agility (ASEAN)
Data Democratization for Faster Decision-making and Business Agility (ASEAN)
 
Accelerate Cloud Migrations and Architecture with Data Virtualization
Accelerate Cloud Migrations and Architecture with Data VirtualizationAccelerate Cloud Migrations and Architecture with Data Virtualization
Accelerate Cloud Migrations and Architecture with Data Virtualization
 
Got data?… now what? An introduction to modern data platforms
Got data?… now what?  An introduction to modern data platformsGot data?… now what?  An introduction to modern data platforms
Got data?… now what? An introduction to modern data platforms
 
SureSkills GDPR - Discover the Smart Solution
SureSkills GDPR - Discover the Smart Solution SureSkills GDPR - Discover the Smart Solution
SureSkills GDPR - Discover the Smart Solution
 
Shield db data security
Shield db   data securityShield db   data security
Shield db data security
 
Shield db data security
Shield db   data securityShield db   data security
Shield db data security
 
Shield db data security
Shield db   data securityShield db   data security
Shield db data security
 
Microsoft Cloud GDPR Compliance Options (SUGUK)
Microsoft Cloud GDPR Compliance Options (SUGUK)Microsoft Cloud GDPR Compliance Options (SUGUK)
Microsoft Cloud GDPR Compliance Options (SUGUK)
 
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
 
Data Vault 2.0 Demystified: East Coast Tour
Data Vault 2.0 Demystified: East Coast TourData Vault 2.0 Demystified: East Coast Tour
Data Vault 2.0 Demystified: East Coast Tour
 
Data Leakage Prevention
Data Leakage PreventionData Leakage Prevention
Data Leakage Prevention
 
Data Mesh using Microsoft Fabric
Data Mesh using Microsoft FabricData Mesh using Microsoft Fabric
Data Mesh using Microsoft Fabric
 
A Logical Architecture is Always a Flexible Architecture (ASEAN)
A Logical Architecture is Always a Flexible Architecture (ASEAN)A Logical Architecture is Always a Flexible Architecture (ASEAN)
A Logical Architecture is Always a Flexible Architecture (ASEAN)
 

More from Certus Solutions

More from Certus Solutions (20)

A Design Approach To Drive Business Innovation Nov
A Design Approach To Drive Business Innovation NovA Design Approach To Drive Business Innovation Nov
A Design Approach To Drive Business Innovation Nov
 
Melbourne: Certus Data 2.0 Vault Meetup with Snowflake - Data Vault In The Cl...
Melbourne: Certus Data 2.0 Vault Meetup with Snowflake - Data Vault In The Cl...Melbourne: Certus Data 2.0 Vault Meetup with Snowflake - Data Vault In The Cl...
Melbourne: Certus Data 2.0 Vault Meetup with Snowflake - Data Vault In The Cl...
 
Sydney: Certus Data 2.0 Vault Meetup with Snowflake - Data Vault In The Cloud
Sydney: Certus Data 2.0 Vault Meetup with Snowflake - Data Vault In The Cloud Sydney: Certus Data 2.0 Vault Meetup with Snowflake - Data Vault In The Cloud
Sydney: Certus Data 2.0 Vault Meetup with Snowflake - Data Vault In The Cloud
 
Design thinking to drive innovation v1.0 handout
Design thinking to drive innovation v1.0 handoutDesign thinking to drive innovation v1.0 handout
Design thinking to drive innovation v1.0 handout
 
Accelerate Blockchain slideshare
Accelerate Blockchain slideshareAccelerate Blockchain slideshare
Accelerate Blockchain slideshare
 
Data Vault 2.0 - Getting Started | Certus Solutions
Data Vault 2.0 - Getting Started | Certus SolutionsData Vault 2.0 - Getting Started | Certus Solutions
Data Vault 2.0 - Getting Started | Certus Solutions
 
4th Industrial Revolution by Sam Williams
4th Industrial Revolution by Sam Williams4th Industrial Revolution by Sam Williams
4th Industrial Revolution by Sam Williams
 
Accelerate 2017_ Maarten van der Zeyden_Mining the Facts, Revealing the Truth
Accelerate 2017_ Maarten van der Zeyden_Mining the Facts, Revealing the Truth Accelerate 2017_ Maarten van der Zeyden_Mining the Facts, Revealing the Truth
Accelerate 2017_ Maarten van der Zeyden_Mining the Facts, Revealing the Truth
 
Accelerate 2017_Julien Redmond_Designing Systems to Mitigate Predictable Surp...
Accelerate 2017_Julien Redmond_Designing Systems to Mitigate Predictable Surp...Accelerate 2017_Julien Redmond_Designing Systems to Mitigate Predictable Surp...
Accelerate 2017_Julien Redmond_Designing Systems to Mitigate Predictable Surp...
 
Accelerate 2017_What LEGO + The New York Times have been learning about disru...
Accelerate 2017_What LEGO + The New York Times have been learning about disru...Accelerate 2017_What LEGO + The New York Times have been learning about disru...
Accelerate 2017_What LEGO + The New York Times have been learning about disru...
 
Accelerate 2017_Brand experience and context_Craig Parnham
Accelerate 2017_Brand experience and context_Craig ParnhamAccelerate 2017_Brand experience and context_Craig Parnham
Accelerate 2017_Brand experience and context_Craig Parnham
 
Accelerate 2017_Navigating Digital Disruption_James Slezak
Accelerate 2017_Navigating Digital Disruption_James SlezakAccelerate 2017_Navigating Digital Disruption_James Slezak
Accelerate 2017_Navigating Digital Disruption_James Slezak
 
Certus Accelerate - Why You Need to Invest in Your Data by Vincent McBurney
Certus Accelerate - Why You Need to Invest in Your Data by Vincent McBurneyCertus Accelerate - Why You Need to Invest in Your Data by Vincent McBurney
Certus Accelerate - Why You Need to Invest in Your Data by Vincent McBurney
 
Certus Accelerate - A Crystal Ball for Asset Intensive Industry by Scott Peters
Certus Accelerate - A Crystal Ball for Asset Intensive Industry by Scott PetersCertus Accelerate - A Crystal Ball for Asset Intensive Industry by Scott Peters
Certus Accelerate - A Crystal Ball for Asset Intensive Industry by Scott Peters
 
Certus Accelerate - Building the business case for why you need to invest in ...
Certus Accelerate - Building the business case for why you need to invest in ...Certus Accelerate - Building the business case for why you need to invest in ...
Certus Accelerate - Building the business case for why you need to invest in ...
 
Certus Accelerate - User Centred Everything by Sam Williams
Certus Accelerate - User Centred Everything by Sam WilliamsCertus Accelerate - User Centred Everything by Sam Williams
Certus Accelerate - User Centred Everything by Sam Williams
 
Certus Accelerate - Disruptive Thinking Disrupting Markets by David Mast
Certus Accelerate - Disruptive Thinking Disrupting Markets by David MastCertus Accelerate - Disruptive Thinking Disrupting Markets by David Mast
Certus Accelerate - Disruptive Thinking Disrupting Markets by David Mast
 
Certus Accelerate - Fourth Industrial Revolution by James Harwright
Certus Accelerate - Fourth Industrial Revolution by James HarwrightCertus Accelerate - Fourth Industrial Revolution by James Harwright
Certus Accelerate - Fourth Industrial Revolution by James Harwright
 
Innovation and Transformation in Financial Services
Innovation and Transformation in Financial ServicesInnovation and Transformation in Financial Services
Innovation and Transformation in Financial Services
 
Just ask Watson Seminar
Just ask Watson SeminarJust ask Watson Seminar
Just ask Watson Seminar
 

Recently uploaded

PETAVIT SIP-01.pdfAAAAAAAAAAAAAAAAAAAAAAAAAAA
PETAVIT SIP-01.pdfAAAAAAAAAAAAAAAAAAAAAAAAAAAPETAVIT SIP-01.pdfAAAAAAAAAAAAAAAAAAAAAAAAAAA
PETAVIT SIP-01.pdfAAAAAAAAAAAAAAAAAAAAAAAAAAA
lawrenceads01
 
chapter 10 - excise tax of transfer and business taxation
chapter 10 - excise tax of transfer and business taxationchapter 10 - excise tax of transfer and business taxation
chapter 10 - excise tax of transfer and business taxation
AUDIJEAngelo
 
Cree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBdCree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBd
creerey
 
anas about venice for grade 6f about venice
anas about venice for grade 6f about veniceanas about venice for grade 6f about venice
anas about venice for grade 6f about venice
anasabutalha2013
 
Sustainability: Balancing the Environment, Equity & Economy
Sustainability: Balancing the Environment, Equity & EconomySustainability: Balancing the Environment, Equity & Economy
Sustainability: Balancing the Environment, Equity & Economy
Operational Excellence Consulting
 

Recently uploaded (20)

PETAVIT SIP-01.pdfAAAAAAAAAAAAAAAAAAAAAAAAAAA
PETAVIT SIP-01.pdfAAAAAAAAAAAAAAAAAAAAAAAAAAAPETAVIT SIP-01.pdfAAAAAAAAAAAAAAAAAAAAAAAAAAA
PETAVIT SIP-01.pdfAAAAAAAAAAAAAAAAAAAAAAAAAAA
 
Using Generative AI for Content Marketing
Using Generative AI for Content MarketingUsing Generative AI for Content Marketing
Using Generative AI for Content Marketing
 
sales plan presentation by mckinsey alum
sales plan presentation by mckinsey alumsales plan presentation by mckinsey alum
sales plan presentation by mckinsey alum
 
Unveiling the Secrets How Does Generative AI Work.pdf
Unveiling the Secrets How Does Generative AI Work.pdfUnveiling the Secrets How Does Generative AI Work.pdf
Unveiling the Secrets How Does Generative AI Work.pdf
 
Taurus Zodiac Sign_ Personality Traits and Sign Dates.pptx
Taurus Zodiac Sign_ Personality Traits and Sign Dates.pptxTaurus Zodiac Sign_ Personality Traits and Sign Dates.pptx
Taurus Zodiac Sign_ Personality Traits and Sign Dates.pptx
 
Business Valuation Principles for Entrepreneurs
Business Valuation Principles for EntrepreneursBusiness Valuation Principles for Entrepreneurs
Business Valuation Principles for Entrepreneurs
 
chapter 10 - excise tax of transfer and business taxation
chapter 10 - excise tax of transfer and business taxationchapter 10 - excise tax of transfer and business taxation
chapter 10 - excise tax of transfer and business taxation
 
Cree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBdCree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBd
 
Lookback Analysis
Lookback AnalysisLookback Analysis
Lookback Analysis
 
Maksym Vyshnivetskyi: PMO Quality Management (UA)
Maksym Vyshnivetskyi: PMO Quality Management (UA)Maksym Vyshnivetskyi: PMO Quality Management (UA)
Maksym Vyshnivetskyi: PMO Quality Management (UA)
 
Lars Winkelbauer — Sustainable Development in the Era of Air Cargo Technology
Lars Winkelbauer — Sustainable Development in the Era of Air Cargo TechnologyLars Winkelbauer — Sustainable Development in the Era of Air Cargo Technology
Lars Winkelbauer — Sustainable Development in the Era of Air Cargo Technology
 
Filing Your Delaware Franchise Tax A Detailed Guide
Filing Your Delaware Franchise Tax A Detailed GuideFiling Your Delaware Franchise Tax A Detailed Guide
Filing Your Delaware Franchise Tax A Detailed Guide
 
anas about venice for grade 6f about venice
anas about venice for grade 6f about veniceanas about venice for grade 6f about venice
anas about venice for grade 6f about venice
 
Introduction to Amazon company 111111111111
Introduction to Amazon company 111111111111Introduction to Amazon company 111111111111
Introduction to Amazon company 111111111111
 
Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...
Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...
Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...
 
Global Interconnection Group Joint Venture[960] (1).pdf
Global Interconnection Group Joint Venture[960] (1).pdfGlobal Interconnection Group Joint Venture[960] (1).pdf
Global Interconnection Group Joint Venture[960] (1).pdf
 
Sustainability: Balancing the Environment, Equity & Economy
Sustainability: Balancing the Environment, Equity & EconomySustainability: Balancing the Environment, Equity & Economy
Sustainability: Balancing the Environment, Equity & Economy
 
Premium MEAN Stack Development Solutions for Modern Businesses
Premium MEAN Stack Development Solutions for Modern BusinessesPremium MEAN Stack Development Solutions for Modern Businesses
Premium MEAN Stack Development Solutions for Modern Businesses
 
Improving profitability for small business
Improving profitability for small businessImproving profitability for small business
Improving profitability for small business
 
The Parable of the Pipeline a book every new businessman or business student ...
The Parable of the Pipeline a book every new businessman or business student ...The Parable of the Pipeline a book every new businessman or business student ...
The Parable of the Pipeline a book every new businessman or business student ...
 

Dv decision makers presentation 310518[1]

  • 2. What is the current landscape for Information Managers?
  • 3. Security Open Data » Lockdown vs. Democratisation » Redaction » De-personalisation Regulatory compliance » GDPR » SOX » FMA/APRA » Cloud / ISO27001 External factors
  • 4. GDPR › This is likely to be replicated in New Zealand › Security Data Breach Notification Law in effect as at 25 May 2018 › Lack of compliance will result in heavy penalties
  • 5. › More data more often › New data sets all the time › Data Quality challenges › Corporate memory › Reliance on individuals › Reconciliation and audit Internal factors
  • 8. What’s wrong with traditional Data Warehousing methods? ‘Shadow IT’ Big Data E T L M A R T S 3rd party data Source System Source System JSON/XML Semi- structured Unstruct- ured data BI Analytics Data Science
  • 9. Automation Framework Data Acquisition Real-Time CDC Messaging Batch ETL/ELT Files PDF, Docs, Video, etc. Staging Raw Data Vault Data Provisioning Information Governance - Metadata Management, Lineage, Data Quality Big Data/No SQL Information Marts (Virtual/Physical) Operational Data Store Business Data Vault Enterprise Data Vault Data Vault: Scalable, Extensible, Agnostic 3rd party data Source System JSON/XML Semi- structured Unstruct- ured data BI Analytics Data Science "It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change." Charles Darwin
  • 10. Data Lake › Data flows in ‘naturally’ › Some boundaries › Content flows out with little constraint Data Swamp › Uncontrolled flow › No borders and potentially bottomless › Filled with flotsam and jetsam Data Harbour › Controlled flow › Trust in the delivery and content of data › Built to use and extract value from the data
  • 11. What is Data Vault? › Invented by Dan Linstedt in the late 1990s, Data Vault is a System of Information Delivery containing the necessary components needed to accomplish enterprise vision in Data Warehousing and Business Intelligence › Data Vault includes a data modelling technique for data repositories that has significant advantages over traditional methodologies: auditable, extensible, automated › With the release of Data Vault 2.0 in 2013, it extended from just the data model to a full methodology › Is effectively vendor agnostic: works with multiple data processing tools, relational databases, and file stores
  • 12. DV2.0 System: Foundational Pillars Methodology Architecture Model • Consistent & Repeatable • Pattern Based • Automation • Agile • Multi-Tier • Scalable • Supports NoSQL • Insert only architecture • Flexible • Scalable (Big Data) • Hub & Spoke

Editor's Notes

  1. Even Gartner is advising clients to beware of the data lake fallacy (http://www.gartner.com/newsroom/id/2809117). Excerpt …. "In broad terms, data lakes are marketed as enterprise-wide data management platforms for analyzing disparate sources of data in its native format," said Nick Heudecker, research director at Gartner. "The idea is simple: instead of placing data in a purpose-built data store, you move it into a data lake in its original format. This eliminates the upfront costs of data ingestion, like transformation. Once data is placed into the lake, it's available for analysis by everyone in the organization."  However, while the marketing hype suggests audiences throughout an enterprise will leverage data lakes, this positioning assumes that all those audiences are highly skilled at data manipulation and analysis, as data lakes lack semantic consistency and governed metadata.  This is why the data harbour is taking shape. It allows the skilled people to use the data under lesser control; but still a level of trust