Data Warehouse Concepts and Architecture

1,120 views

Published on

Introduction to Data Warehouse (DW or EDW) trends and concepts.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,120
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
46
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Data Warehouse Concepts and Architecture

  1. 1. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 1 PPPP II Data Warehouse Concepts & Architecture
  2. 2. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 2 PPPP II Topics To Be Discussed: • Why Do We Need A Data Warehouse ? • The Goal Of A Data Warehouse ? • What Exactly Is A Data Warehouse ? • Comparison Of A Data Warehouse And An Operational Data Store. • Data Warehouse Trends. Data Warehouse Concepts
  3. 3. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 3 PPPP II Why Do We Need A Data Warehouse ? We Can Only See - What We Can See ! Data Warehouse Concepts
  4. 4. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 4 PPPP II Why Do We Need A Data Warehouse ? BETTER ! FASTER ! FUNCTIONALLY COMPLETE ! CHEAPER ! Data Warehouse Concepts
  5. 5. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 5 PPPP II Data A/P O/P DSS EIS Data Driven Vs. Order Processing Data Function Driven Data Warehouse Development Perspective Data Warehouse Concepts
  6. 6. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 6 PPPP II What Do We Need To Do ? Use Operational Legacy Systems’ Data: To Build Operational Data Store, That Integrate Into Corporate Data Warehouse, That Spin-off Data Marts. Some May Tell You To Develop These In Reverse! Data Warehouse Concepts
  7. 7. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 7 PPPP II Our Goal for A Data Warehouse ? • Collect Data-Scrub, Integrate & Make It Accessible • Provide Information - For Our Businesses • Start Managing Knowledge • So Our Business Partners Will Gain Wisdom ! Data Warehouse Concepts
  8. 8. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 8 PPPP II Data Warehouse Concepts Data Warehouse Definition • Subject Oriented • Integrated • Time Variant • Non-volatile A Data Warehouse Is A Structured Repository of Historic Data. It Is Developed in an Evolutionary Process By Integrating Data From Non-integrated Legacy Systems. It Is Usually:
  9. 9. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 9 PPPP II Data Warehouse Concepts Subject Oriented Data is Integrated and Loaded by Subject D/W Data 2001 2002 2003 2002 A/R O/P Cust Prod
  10. 10. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 10 PPPP II Data Warehouse Concepts Time Variant • Designated Time Frame (3 - 10 Years) • One Snapshot Per Cycle • Key Includes Date Data Warehouse • View of The Business Today • Operational Time Frame • Key Need Not Have Date Operational System
  11. 11. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 11 PPPP II Operational Systems Order Processing Order ID = 10 D/W Accounts Receivable Order ID = 12 Order ID = 16 Product Management Order ID = 8 HR System Sex = M/F D/W Payroll Sex = 1/2 Sex = M/F Product Management Sex = 0/1 Data Warehouse Concepts Integrated
  12. 12. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 12 PPPP II Data Warehouse Concepts Non-Volatile • “CRUD” Actions Operational System Read Insert Update Replace Create Delete • No Data Update Data Warehouse Load Read Read Read Read
  13. 13. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 13 PPPP II Data Warehouse ConceptsData Warehouse Concepts Data Warehouse Environment Architecture Contains Integrated Data From Multiple Legacy Applications A/P O/P Pay Mktg Best System of Record Data Integration Criteria Load Read Insert Update Delete Replace ODS D/W Load D/W All Or Part Of System of Record Data Read Data Warehouse Load Criteria Data Mart Data Mart Data Mart LoadsA/R HR
  14. 14. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 14 PPPP II Data Warehouse Concepts Meta Data - Map of Integration The Data That Provides the “Card Catalogue” Of References For All Data Within The Data Warehouse Data Source Source Data Structure Allowable Domains System of Record D/W Structure Definition Aliases Data Relationships
  15. 15. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 15 PPPP II Data Warehouse Concepts ODS Vs. Data Warehouse Operational DataStore DataWarehouse Characteristics: Data FocusedIntegration FromTransactionProcessing FocusedSystems Subject Oriented Integrated Non-Volatile Time Variant Age Of The Data: Current, Near Term (Today, Last Week’s) Historic (Last Month, Qtrly, Five Years) Primary Use: Day-To-Day Decisions Tactical Reporting Current Operational Results Long-TermDecisions Strategic Reporting TrendDetection Frequency Of Load: Twice Daily , Daily, Weekly Weekly, Monthly, Quarterly
  16. 16. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 16 PPPP II • Define Project Scope • Define Business Reqmts • Define System of Record Data • Define Operational Data Store Reqmts • Map SOR to ODS • Acquire / Develop Extract Tools • Extract Data & Load ODS • Scope Definition • Logical Data Model • Physical Database Data Model • Operational Data Store Model • ODS Map • Extract Tools and Software • Populated ODS Building The Data Warehouse Tasks Deliverables Data Warehouse Concepts
  17. 17. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 17 PPPP II Building The Data Warehouse • Define D/W Data Reqmts • Map ODS to D/W • Document Missing Data • Develop D/W DB Design • Extract and Integrate D/W Data • Load Data Warehouse • Maintain Data Warehouse • Transition Data Model • D/W Data Integration Map • To Do Project List • D/W Database Design • Integrated D/W Data Extracts • Initial Data Load • On-going Data Access and Subsequent Loads Tasks Deliverables (Continued) Data Warehouse Concepts
  18. 18. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 18 PPPP II Relationship Among Data Warehouse Data Models Business Requirements Logical Model Current Database Physical Model Data Whse Requirements Transition Model Operational Data Store Physical Model Business Partner Knowledge & Wisdom Current Structure Data Load Tactical Business Reqmts & Structures Validation of Current Data Business Requirements Structured Requirements Data Warehouse Concepts Data Warehouse Physical Model Strategic Business Requirements
  19. 19. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 19 PPPP II Sources of Data Warehouse Data Archives (Historic Data) Current Systems of Record (Recent History) Operational Transactions (Future Data Source) Data Warehouse Concepts Enterprise Data Warehouse
  20. 20. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 20 PPPP II Appropriate Uses of Data Warehouse Data • Produce Reports For Long Term Trend Analysis • Produce Reports Aggregating Enterprise Data • Produce Reports of Multiple Dimensions (Earned revenue by month by product by branch) Data Warehouse Concepts
  21. 21. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 21 PPPP II Inappropriate Uses of Data Warehouse Data Data Warehouse Concepts • Replace Operational Systems • Replace Operational Systems’ Reports • Analyze Current Operational Results
  22. 22. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 22 PPPP II Data Warehouse Concepts Levels of Granularity of Data Warehouse Data •Atomic (Transaction) •Lightly Summarized •Highly Summarized
  23. 23. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 23 PPPP II Data Warehouse Concepts Options for Viewing Data • Text• • 1 s t Q t r 2 n d Q t r 3 r d Q t r 4 t h Q t r 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 s t Q t r 2 n d Q t r 3 r d Q t r 4 t h Q t r
  24. 24. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 24 PPPP II Data Warehouse Concepts Next Steps In Data Warehouse Evolution • Use It - Analyze Data Warehouse Data • Determine Additional Data Requirements • Define Sources For Additional Data • Add New Data (Subject Areas) to Data Warehouse
  25. 25. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 25 PPPP II Data Warehouse Concepts Future Trends In Data Warehouse • Increased Data Mining Exploration Prove Hypothesis • Increase Competitive Advantage (i.e., Identify Cross-selling Opportunities) • Integration into Supply Chain & e-Business
  26. 26. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 26 PPPP II • Subject Oriented • Integrated • Time Variant • Non-volatile Summary Data Warehouse Concepts A Data Warehouse Is A Structured Repository of Historic Data. It Is: It Contains: • Business Specified Data, To Answer Business Questions
  27. 27. © Principle Partners, Inc. Info@PrinciplePartners.Com Page 27 PPPP II Questions and Answers Data Warehouse Concepts

×