Using OBIEE and Data Vault to
Virtualize Your BI Environment:
An Agile Approach
Kent Graziano, Data Warrior LLC
Stewart Bryson, Rittman Mead
Kent Graziano
 Twitter: @KentGraziano
 Certified Data Vault Master
 Oracle ACE Director, Oracle BI&DW
 Data Architecture and Data Warehouse Specialist
● 30+ years in IT
● 20+ years of Oracle-related work
● 15+ years of data warehousing experience
 Co-Author of 2 Books
● The Business of Data Vault Modeling
● The Data Model Resource Book (1st Edition)
 Editor of “The” Data Vault Book
● Super Charge Your Data Warehouse
 Co-Chair BI/DW SIG for ODTUG
 Past-President of Oracle Development Tools User
Group and Rocky Mountain Oracle User Group
Stewart Bryson
• Twitter : @StewartBryson
• Oracle ACE in BI/DW
• Oracle BI/DW Architect and Delivery
Specialist
• Community Speaker and Enthusiast
• Writer for Rittman Mead Blog:
http://www.rittmanmead.com/blog
• US Conference Chair of the Rittman Mead
BI Forum
• Developer of Transcend Framework
• Email : stewart.bryson@rittmanmead.com
• Real Time BI with Kevin & Stewart
‣ iTunes: http://bit.ly/realtimebi
‣ YouTube:
http://www.youtube.com/user/realtimebi
About Rittman Mead
• Oracle BI and DW Partner
• World leader in solutions delivery
and innovation in Oracle BI
• Approximately 70 consultants
worldwide
• Offices in US (Atlanta), Europe,
Australia and India
• Skills in broad range of
supporting Oracle BI Tools
‣ OBIEE
‣ OBIA
‣ ODIEE
‣ Essbase, Oracle OLAP
‣ GoldenGate
‣ Exadata
‣ Endeca
Questions We Hope to Answer
• Data Vault
‣ What is Data Vault?
‣ Why would I choose Data Vault over
competing technologies?
• Oracle Information Management
Reference Architecture
‣ What are the core components of the
Reference Architecture?
‣ Is there possibly an acronym for that?
• Oracle Business Intelligence
‣ What is OBIEE?
‣ Why does OBIEE work so well with Data
Vault?
• What is Agile BI and how does this
help?
Oracle Information Management Reference Architecture
 Staging Layer
● Change tables for Oracle GoldenGate
● Reject tables for Data Quality
● External tables for file feeds
 Foundation Layer
● Transactional granularity maintained
● Process neutral: no user or business
requirements
● Just recording what happened
 Access and Performance Layer
● Dimensional model
● “Star Schemas”
● Process specific: targeting user and
business requirements
What is Data Vault Trying to Solve?
 What are our other Enterprise
Data Warehouse options?
● Third-Normal Form (3NF):
Complex primary keys (PK’s) with
cascading snapshot dates
● Star Schema (Dimensional):
Difficult to reengineer fact tables
for granularity changes
 Difficult to get it right the first
time
 Not adaptable to rapid
business change
 NOT AGILE!
Data Vault: Definition
 The Data Vault is a detail oriented,
historical tracking and uniquely linked set
of normalized tables that support one or
more functional areas
of business.
 It is a hybrid approach encompassing the
best of breed between 3rd normal form
(3NF) and star schema. The design is
flexible, scalable, consistent, and
adaptable to the needs of the enterprise. It
is a data model that is architected
specifically to meet the needs of today’s
enterprise data warehouses.
Dan Linstedt: Defining the Data Vault
TDAN.com Article
Data Vault Timeline
What is the Foundation Layer?
• Basis for long term enterprise
scale data warehouse
• Must be atomic level data
‣ A historical source of facts
‣ No user requirements applied
• Not based on any one data
source or system
• Single point of integration
• Flexible
• Extensible
• Provides data to the
access/reporting layer
‣ Based on targeted business
requirements
‣ Can be virtual
Standard Approach to Agile Business
Intelligence
• Design iterations around smaller chunks
‣ Iteration 1: Interviews and user requirements
‣ Iteration 2: Logical modeling
‣ Iteration 3: ETL Development
‣ Iteration 4: Front-end development
• Requires 4 iterations before we get any
usable content
Manifesto for Agile Software
Development
 “We are uncovering better ways of developing
software by doing it and helping others do it.
 Through this work we have come to value:
 Individuals and interactions over processes and
tools
 Working software over comprehensive
documentation
 Customer collaboration over contract negotiation
 Responding to change over following a plan
 That is, while there is value in the items on the right,
we value the items on the left more.”
 http://agilemanifesto.org/
Applying the Agile Manifesto to BI
Development
 User Stories instead of requirements
documents
● User asks for content or functionality through a
narrative
● Typically includes current version of the report
 Time-boxed iterations
● Iteration has a standard length
● Choose one or more user stories to fit in that
iteration
 Rework is part of the game
● There are no “missed requirements”... only those
that haven’t been delivered yet.
What is Our Approach?
 Model iteratively
● Use Data Vault data modeling technique
● Create basic components, then add over
time
 Virtualize the Access Layer
● Don’t waste time building facts and
dimensions up front
● ETL and testing takes too long
● “Project” objects using pattern-based DV
model with OBIEE BMM
 Users see real reports with real data
Data Vault: Three Simple Structures
1. Hub = Business Keys
Hubs = Unique Lists of Business Keys
Business Keys are used to
TRACK and IDENTIFY key information
2: Links = Associations
Links = Transactions and
Associations
They are used to hook
together multiple sets of
information
3. Satellites = Descriptors
Satellites provide context
for the Hubs and the Links
Flexibility (Agility) and Productivity
• Adding new components to the
EDW has NEAR ZERO impact
to:
‣ Existing Loading Processes
‣ Existing Data Model
‣ Existing Reporting & BI Functions
‣ Existing Source Systems
‣ Existing Star Schemas and Data Marts
• Standardized modeling rules
‣ Highly repeatable and learnable
modeling technique
‣ Allows automation of models, loads,
and extracts
‣ Can use a BI-meta layer to virtualize
the reporting structures
‣ OBIEE Business Model and Mapping
tool
What is OBIEE?
•Dashboards, Ad-hoc Reporting,
Alerts, Microsoft Office Integration
• High quality graphical, role/user based
views
• Multiple views of same data
•Point and click ease of use
•Common Enterprise Information
Model
• Unified semantic/logical view of data
from multiple sources
• Heterogeneous database access
• True enterprise deployment
•Alerts, scheduling and distribution
Where Does OBIEE Fit?
•OBIEE is the
Information Access
Layer
•BI Abstraction layer
allows us “bypass” the
creation of the Access
& Performance Layer
•We “virtualize” the
dependent data marts
Flow of Data Through the Three-Layer
Semantic Model
Simplification of the Data Model
Integration of Disparate Data Sources
Addition of Business Logic and Calculations
Addition of Aggregate Sources
OBIEE Physical Model
OBIEE Tips and Tricks (Discovered by
Stewart Bryson)
• Create folders in the Physical
Layer
‣ Separate Hubs, Links and Satellites
‣ Each has distinct uses
• Hubs
‣ Business Keys
‣ Used in defining Primary Keys and
Level Keys
• Links
‣ Used in Extending the Logical Table
Source (LTS)
‣ Never references in display columns or
measures
• Satellites
‣ Use these for Attributes and Measures
‣ Anything displayed to the user
Building a Simple Dimension: Mapping
the Primary Key
Building a Simple Dimension:
Renaming for Clarification
Building a Simple Dimension: Defining
the Primary Key
Building a Simple Dimension: Extending
the Logical Table Source (LTS)
Building a Simple Dimension: Adding
Descriptive Attributes
Building a Simple Fact: Mapping
Measures
Building a Simple Fact: Renaming for
Clarification
Building a Simple Fact: Mapping the
Primary Key
Building a Simple Fact: Extending the
Logical Table Source (LTS)
Simple Dimension and Fact: An Analysis
Building a Factless Fact: Adding the LTS
Building a Factless Fact: Adding the
“Fake” Count Measure
Links: Added to Logical Facts or
Logical Dimensions?
Logical Fact
Logical Dimension
Links: Added to Logical Facts or
Logical Dimensions?
“Link”-ing Levels Within a Hierarchy in
a Logical Dimension
“Link”-ing Levels Within a Hierarchy in
a Logical Dimension
Organizations Using Data Vault
• WebMD Health Services
• Anthem Blue-Cross Blue Shield
• Denver Public Schools
• Independent Purchasing
Cooperative (IPC, Miami)
• Owner of Subway
• Kaplan
• US Defense Department
• Colorado Springs Utilities
• State Court of Wyoming
• Federal Express
• US Dept. Of Agriculture
Summary
• Data Vault provides a data modeling
technique that allows:
‣ Model Agility
‣ Enabling rapid changes and additions
‣ Productivity
‣ Enabling low complexity systems with high value
output at a rapid pace
‣ Easy projections of dimensional models
• OBIEE provides
‣ Framework for Agile BI
‣ Rapid development of virtualized layer on a data
vault model
Super Charge Your Data Warehouse
Available on Amazon.com
Soft Cover or Kindle Format
Now also available in PDF at
LearnDataVault.com
Hint: Kent is the Technical
Editor
Kscope Special for LearnDataVault
Go to
http://learndatavault.com/kscope13
Discount coupons for:
Super Charge book
DV Implementation course
DV using Informatica course
Data Vault References
www.learndatavault.com
www.datavaultcertification.com
www.danlinstedt.com
On YouTube:
www.youtube.com/LearnDataVault
On Facebook:
www.facebook.com/learndatavault
Contact Information
Kent Graziano
The Oracle Data Warrior
Data Warrior LLC
Kent.graziano@att.net
Visit my blog at
http://kentgraziano.com
@KentGraziano
Stewart Bryson
US Managing Director
Rittman Mead
stewart.bryson@rittmanmead.com
www.rittmanmead.com
@stewartbryson
T : +44 (0) 8446 697 995

Using OBIEE and Data Vault to Virtualize Your BI Environment: An Agile Approach

  • 1.
    Using OBIEE andData Vault to Virtualize Your BI Environment: An Agile Approach Kent Graziano, Data Warrior LLC Stewart Bryson, Rittman Mead
  • 2.
    Kent Graziano  Twitter:@KentGraziano  Certified Data Vault Master  Oracle ACE Director, Oracle BI&DW  Data Architecture and Data Warehouse Specialist ● 30+ years in IT ● 20+ years of Oracle-related work ● 15+ years of data warehousing experience  Co-Author of 2 Books ● The Business of Data Vault Modeling ● The Data Model Resource Book (1st Edition)  Editor of “The” Data Vault Book ● Super Charge Your Data Warehouse  Co-Chair BI/DW SIG for ODTUG  Past-President of Oracle Development Tools User Group and Rocky Mountain Oracle User Group
  • 3.
    Stewart Bryson • Twitter: @StewartBryson • Oracle ACE in BI/DW • Oracle BI/DW Architect and Delivery Specialist • Community Speaker and Enthusiast • Writer for Rittman Mead Blog: http://www.rittmanmead.com/blog • US Conference Chair of the Rittman Mead BI Forum • Developer of Transcend Framework • Email : stewart.bryson@rittmanmead.com • Real Time BI with Kevin & Stewart ‣ iTunes: http://bit.ly/realtimebi ‣ YouTube: http://www.youtube.com/user/realtimebi
  • 4.
    About Rittman Mead •Oracle BI and DW Partner • World leader in solutions delivery and innovation in Oracle BI • Approximately 70 consultants worldwide • Offices in US (Atlanta), Europe, Australia and India • Skills in broad range of supporting Oracle BI Tools ‣ OBIEE ‣ OBIA ‣ ODIEE ‣ Essbase, Oracle OLAP ‣ GoldenGate ‣ Exadata ‣ Endeca
  • 5.
    Questions We Hopeto Answer • Data Vault ‣ What is Data Vault? ‣ Why would I choose Data Vault over competing technologies? • Oracle Information Management Reference Architecture ‣ What are the core components of the Reference Architecture? ‣ Is there possibly an acronym for that? • Oracle Business Intelligence ‣ What is OBIEE? ‣ Why does OBIEE work so well with Data Vault? • What is Agile BI and how does this help?
  • 6.
    Oracle Information ManagementReference Architecture  Staging Layer ● Change tables for Oracle GoldenGate ● Reject tables for Data Quality ● External tables for file feeds  Foundation Layer ● Transactional granularity maintained ● Process neutral: no user or business requirements ● Just recording what happened  Access and Performance Layer ● Dimensional model ● “Star Schemas” ● Process specific: targeting user and business requirements
  • 7.
    What is DataVault Trying to Solve?  What are our other Enterprise Data Warehouse options? ● Third-Normal Form (3NF): Complex primary keys (PK’s) with cascading snapshot dates ● Star Schema (Dimensional): Difficult to reengineer fact tables for granularity changes  Difficult to get it right the first time  Not adaptable to rapid business change  NOT AGILE!
  • 8.
    Data Vault: Definition The Data Vault is a detail oriented, historical tracking and uniquely linked set of normalized tables that support one or more functional areas of business.  It is a hybrid approach encompassing the best of breed between 3rd normal form (3NF) and star schema. The design is flexible, scalable, consistent, and adaptable to the needs of the enterprise. It is a data model that is architected specifically to meet the needs of today’s enterprise data warehouses. Dan Linstedt: Defining the Data Vault TDAN.com Article
  • 9.
  • 10.
    What is theFoundation Layer? • Basis for long term enterprise scale data warehouse • Must be atomic level data ‣ A historical source of facts ‣ No user requirements applied • Not based on any one data source or system • Single point of integration • Flexible • Extensible • Provides data to the access/reporting layer ‣ Based on targeted business requirements ‣ Can be virtual
  • 11.
    Standard Approach toAgile Business Intelligence • Design iterations around smaller chunks ‣ Iteration 1: Interviews and user requirements ‣ Iteration 2: Logical modeling ‣ Iteration 3: ETL Development ‣ Iteration 4: Front-end development • Requires 4 iterations before we get any usable content
  • 12.
    Manifesto for AgileSoftware Development  “We are uncovering better ways of developing software by doing it and helping others do it.  Through this work we have come to value:  Individuals and interactions over processes and tools  Working software over comprehensive documentation  Customer collaboration over contract negotiation  Responding to change over following a plan  That is, while there is value in the items on the right, we value the items on the left more.”  http://agilemanifesto.org/
  • 13.
    Applying the AgileManifesto to BI Development  User Stories instead of requirements documents ● User asks for content or functionality through a narrative ● Typically includes current version of the report  Time-boxed iterations ● Iteration has a standard length ● Choose one or more user stories to fit in that iteration  Rework is part of the game ● There are no “missed requirements”... only those that haven’t been delivered yet.
  • 14.
    What is OurApproach?  Model iteratively ● Use Data Vault data modeling technique ● Create basic components, then add over time  Virtualize the Access Layer ● Don’t waste time building facts and dimensions up front ● ETL and testing takes too long ● “Project” objects using pattern-based DV model with OBIEE BMM  Users see real reports with real data
  • 15.
    Data Vault: ThreeSimple Structures
  • 16.
    1. Hub =Business Keys Hubs = Unique Lists of Business Keys Business Keys are used to TRACK and IDENTIFY key information
  • 17.
    2: Links =Associations Links = Transactions and Associations They are used to hook together multiple sets of information
  • 18.
    3. Satellites =Descriptors Satellites provide context for the Hubs and the Links
  • 19.
    Flexibility (Agility) andProductivity • Adding new components to the EDW has NEAR ZERO impact to: ‣ Existing Loading Processes ‣ Existing Data Model ‣ Existing Reporting & BI Functions ‣ Existing Source Systems ‣ Existing Star Schemas and Data Marts • Standardized modeling rules ‣ Highly repeatable and learnable modeling technique ‣ Allows automation of models, loads, and extracts ‣ Can use a BI-meta layer to virtualize the reporting structures ‣ OBIEE Business Model and Mapping tool
  • 20.
    What is OBIEE? •Dashboards,Ad-hoc Reporting, Alerts, Microsoft Office Integration • High quality graphical, role/user based views • Multiple views of same data •Point and click ease of use •Common Enterprise Information Model • Unified semantic/logical view of data from multiple sources • Heterogeneous database access • True enterprise deployment •Alerts, scheduling and distribution
  • 21.
    Where Does OBIEEFit? •OBIEE is the Information Access Layer •BI Abstraction layer allows us “bypass” the creation of the Access & Performance Layer •We “virtualize” the dependent data marts
  • 22.
    Flow of DataThrough the Three-Layer Semantic Model Simplification of the Data Model Integration of Disparate Data Sources Addition of Business Logic and Calculations Addition of Aggregate Sources
  • 23.
  • 24.
    OBIEE Tips andTricks (Discovered by Stewart Bryson) • Create folders in the Physical Layer ‣ Separate Hubs, Links and Satellites ‣ Each has distinct uses • Hubs ‣ Business Keys ‣ Used in defining Primary Keys and Level Keys • Links ‣ Used in Extending the Logical Table Source (LTS) ‣ Never references in display columns or measures • Satellites ‣ Use these for Attributes and Measures ‣ Anything displayed to the user
  • 25.
    Building a SimpleDimension: Mapping the Primary Key
  • 26.
    Building a SimpleDimension: Renaming for Clarification
  • 27.
    Building a SimpleDimension: Defining the Primary Key
  • 28.
    Building a SimpleDimension: Extending the Logical Table Source (LTS)
  • 29.
    Building a SimpleDimension: Adding Descriptive Attributes
  • 30.
    Building a SimpleFact: Mapping Measures
  • 31.
    Building a SimpleFact: Renaming for Clarification
  • 32.
    Building a SimpleFact: Mapping the Primary Key
  • 33.
    Building a SimpleFact: Extending the Logical Table Source (LTS)
  • 34.
    Simple Dimension andFact: An Analysis
  • 35.
    Building a FactlessFact: Adding the LTS
  • 36.
    Building a FactlessFact: Adding the “Fake” Count Measure
  • 38.
    Links: Added toLogical Facts or Logical Dimensions? Logical Fact Logical Dimension
  • 39.
    Links: Added toLogical Facts or Logical Dimensions?
  • 40.
    “Link”-ing Levels Withina Hierarchy in a Logical Dimension
  • 41.
    “Link”-ing Levels Withina Hierarchy in a Logical Dimension
  • 43.
    Organizations Using DataVault • WebMD Health Services • Anthem Blue-Cross Blue Shield • Denver Public Schools • Independent Purchasing Cooperative (IPC, Miami) • Owner of Subway • Kaplan • US Defense Department • Colorado Springs Utilities • State Court of Wyoming • Federal Express • US Dept. Of Agriculture
  • 44.
    Summary • Data Vaultprovides a data modeling technique that allows: ‣ Model Agility ‣ Enabling rapid changes and additions ‣ Productivity ‣ Enabling low complexity systems with high value output at a rapid pace ‣ Easy projections of dimensional models • OBIEE provides ‣ Framework for Agile BI ‣ Rapid development of virtualized layer on a data vault model
  • 45.
    Super Charge YourData Warehouse Available on Amazon.com Soft Cover or Kindle Format Now also available in PDF at LearnDataVault.com Hint: Kent is the Technical Editor
  • 46.
    Kscope Special forLearnDataVault Go to http://learndatavault.com/kscope13 Discount coupons for: Super Charge book DV Implementation course DV using Informatica course
  • 47.
    Data Vault References www.learndatavault.com www.datavaultcertification.com www.danlinstedt.com OnYouTube: www.youtube.com/LearnDataVault On Facebook: www.facebook.com/learndatavault
  • 48.
    Contact Information Kent Graziano TheOracle Data Warrior Data Warrior LLC Kent.graziano@att.net Visit my blog at http://kentgraziano.com @KentGraziano Stewart Bryson US Managing Director Rittman Mead stewart.bryson@rittmanmead.com www.rittmanmead.com @stewartbryson T : +44 (0) 8446 697 995

Editor's Notes

  • #2 This is your opening slide.