Data vault: What's Next
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Data vault: What's Next

on

  • 2,040 views

This was a presentation about Data Warehousing, where it's going - covers operational Data Vault. I gave this presentation in 2009 at an Array Conference in the Netherlands. ...

This was a presentation about Data Warehousing, where it's going - covers operational Data Vault. I gave this presentation in 2009 at an Array Conference in the Netherlands.

IF you want to use these slides, then please let me know, and add: "(C) Dan Linstedt, all rights reserved, http://LearnDataVault.com"

Statistics

Views

Total Views
2,040
Views on SlideShare
1,825
Embed Views
215

Actions

Likes
2
Downloads
112
Comments
0

3 Embeds 215

http://danlinstedt.com 196
http://www.linkedin.com 12
https://www.linkedin.com 7

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Data vault: What's Next Presentation Transcript

  • 1. Data Vault ModelingWHAT’S NEXT???
    © Dan Linstedt 2009-2012
    This was a presentation I gave at an Array Conference In the Netherlands, in 2009.
  • 2. A bit about me…
    2
    Author, Inventor, Speaker – and part time photographer…
    25+ years in the IT industry
    Worked in DoD, US Gov’t, Fortune 50, and so on…
    Find out more about the Data Vault:
    http://www.youtube.com/LearnDataVault
    http://LearnDataVault.com
    Full profile on http://www.LinkedIn.com/dlinstedt
    10/5/2011
    LearnDataVault.com
  • 3. Who’s Using It?
    10/5/2011
    LearnDataVault.com
    3
  • 4. Featured Word from Our Customer
    10/5/2011
    LearnDataVault.com
    4
    Jonathan Rice, BSc
    ING Real Estate/Development International/Operational Support
    Location HP A.05.071
    PO Box 90463, 2509 LL The Hague, The Netherlands
    They are connecting MS SharePoint back-end to Data Vault Modeling EDW.
  • 5. What’s Changing the Game?
    Solid State Disk (SSD)
    Hosted Cloud Computing
    Column Based Databases
    Unstructured Information
    Ontologies / Taxonomies
    Mining Engines
    Broad Based Web Services
    Business Demand for Immediate Answers (pushing real-time)
    Compliance and Auditability (pushing volume)
    10/5/2011
    LearnDataVault.com
    5
    • Data Visualization
    • 6. Flash & Silverlight Front-Ends
    • 7. Analytic functions in Database Engines
    • 8. Business Rules Engines Melding with ETL and Web Services
    • 9. DW Appliances
  • Current EDW + DV Architecture
    10/5/2011
    LearnDataVault.com
    6
    SOA
    Enterprise BI Solution
    Star
    Schemas
    (real-time)
    Sales
    (batch)
    DV
    EDW
    (batch)
    Staging
    Error
    Marts
    Finance
    Contracts
    Report
    Collections
    Business Rules Downstream!
    (the Lens Filter)
  • 10. First Change: Business Data Vault
    10/5/2011
    LearnDataVault.com
    7
    SOA
    Enterprise BI Solution
    Star
    Schemas
    (real-time)
    Sales
    Batch
    DV
    EDW
    BDV
    EDW
    Staging
    Error
    Marts
    Finance
    Contracts
    Report
    Collections
    Business Rules Downstream!
    (the Lens Filter)
  • 11. Next Change: Staging Removal
    10/5/2011
    LearnDataVault.com
    8
    Enterprise BI Solution
    SOA
    Write Back
    (real-time)
    Star
    Schemas
    Sales
    DV
    EDW
    BDV
    EDW
    Error
    Marts
    Finance
    Contracts
    Report
    Collections
    Business Rules Downstream!
    (the Lens Filter)
  • 12. Next Change: Virtual Marts
    10/5/2011
    LearnDataVault.com
    9
    Enterprise BI Solution
    SOA
    Write Back
    Virtual
    Marts
    & Dynamic Cubes
    (real-time)
    Sales
    Finance
    DV
    EDW
    BDV
    EDW
    Contracts
    Business Rules Downstream!
    (the Lens Filter)
  • 13. Next Change: Unstructured Data
    10/5/2011
    LearnDataVault.com
    10
    Unstructured Data Sets
    Ontologies/Taxonomies
    Unstructured
    Processing Engine
    • Email
    • 14. Docs
    • 15. Images
    • 16. Movies
    • 17. Sound
    On-Demand
    Cubes
    Joins through LINK Structures
    Raw Data Vault EDW
  • 18. A Look at Ontologies
    10/5/2011
    LearnDataVault.com
    11
    Hierarchies of Data:
    Synonyms, Homonyms, Antonyms, Related Terms, Definitions, Categorizations, Organizations, Views of the data world
    Ontologies HOLD THE KEY to understanding/conceptual relevance
    Ontologies can PIVOT raw data in to many different results
  • 19. Plateau: Operational Data Warehouse
    10/5/2011
    LearnDataVault.com
    12
    Operational
    Metadata
    Management
    Operational
    Applications
    Master Data
    Strategic
    Reports
    & OLAP
    & W.SVCS
    Direct Edits
    Dynamic
    Cubes
    Web Interface (usually)
    Direct Edits
    Real-Time
    Collector
    SOR
    Real-Time Data
    Data Vault EDW
    • Stored
    • 20. Analyzed / Scored
    Operational
    Systems
    Unstructured
    Semi-Structured
    Staging
    Area
    Non-S.O.R.
    Historical Batch Data
    Non-SOR
    Batch Data
    Operational
    Alerts
    Operational
    Systems
    Strategic
    Reports
    & OLAP
    & W.SVCS
    Virtual
    Marts
    Real-Time
    Mining
    Engine
    • Flexible
    • 21. Accountable
    • 22. Compliant
    • 23. Scalable
    • 24. Normalized
    • 25. Dynamic
    • 26. Granular
    • 27. Historic
  • Data Vault
    (EDW)
    Operational Data Vault
    10/5/2011
    LearnDataVault.com
    13
    Operational
    Application
    Update/Unlock
    Read/Lock
    Common Data Access Layer
    Insert Changes
    Read Current
  • 28. Why go Operational Data Vault?
    Benefits Include
    Direct access to data
    Removal of 90% of batch streams, replaced by Transactions
    Better/faster alerting capabilities
    Direct control over changes to BI answer sets
    What are the risks?
    The Data Vault EDW becomes an ODV, brings with it ALL responsibilities of the Operational System
    What are the problems?
    Row Locking
    Consistent Data Access
    Data Version Control
    Security of Data through access points
    Has it been done before?
    Yes, Cendant Timeshare Resource Group did it in 2002!
    10/5/2011
    LearnDataVault.com
    14
  • 29. Plateau: Dynamic Data Warehouse
    10/5/2011
    LearnDataVault.com
    15
    Dynamically Constructed Links
    Dynamically Created Hubs
    Dynamically Altered Satellites
    Dynamically changed ETL/ELT
    Dynamically Altered Queries
    Automatically Updated Cubes
    = Self Morphing (guided) Data Vault
    H
    L
    DL
    H
    L
    Meta
    Mining
    Engine
    H
    L
    H
    H
    DL
    L
    H
  • 30. Impacts of Dynamic Data Vaults
    Business
    New found knowledge
    Evolving Data Warehouse models over time
    Faster Reporting
    Self-Maintaining Back-Ends
    Technical
    Guided Changes to Structures
    Auto-Adapting Load Routines
    Auto-Adapting Query Sets
    Auto-Suggested Star Schemas
    10/5/2011
    LearnDataVault.com
    16
  • 31. Why go Dynamic?
    Benefits include:
    Faster Turn Around Time
    Auto Configuration of “mundane tasks”
    Discovery of new relationships (could result in increased revenue)
    Self Healing Structures
    How Long before we See It?
    5 to 7 years out
    How do we get there?
    Meta-Mining
    Use of Ontologies inside the EDW
    What’s driving us there?
    Business Users want faster turn around time
    Business Users want ingestion of Unstructured Data Sets
    Business Users need more control over their systems
    10/5/2011
    LearnDataVault.com
    17
  • 32. Where To Learn More
    The Technical Modeling Book: http://LearnDataVault.com
    The Discussion Forums: & eventshttp://LinkedIn.com – Data Vault Discussions
    Contact me:http://DanLinstedt.com - web siteDanLinstedt@gmail.com - email
    World wide User Group (Free)http://dvusergroup.com
    18
    10/5/2011
    LearnDataVault.com