Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Agile data warehouse

4,367 views

Published on

Agile data warehouse

  1. 1. © 2013 KMS Technology
  2. 2. AGILE DATA WAREHOUSE DESIGN Dao Vo Confidential 2
  3. 3. AGENDA • Overview of data warehousing • Designing and implementing a data warehouse • Waterfall BI/WH development • Agile BI/WH development framework • Q&A Confidential 3
  4. 4. OVERVIEW OF DATA WAREHOUSING What is a data warehouse? Confidential 4
  5. 5. OVERVIEW OF DATA WAREHOUSING • The business problem • What is a data warehouse? • BI/WH Architectures Confidential 5
  6. 6. THE BUSINESS PROBLEM • Key business data is distributed across multiple systems
  7. 7. THE BUSINESS PROBLEM • Finding the information required for business decision making is time- consuming and error-prone
  8. 8. THE BUSINESS PROBLEM • Fundamental business questions are hard to answer
  9. 9. WHAT IS A DATA WAREHOUSE?
  10. 10. WHAT IS A DATA WAREHOUSE? • A centralized store of business data for reporting and analysis • Typically, a data warehouse: – Contains large volumes of historical data – Is optimized for querying data (as opposed to inserting or updating) – Is incrementally loaded with new business data at regular intervals – Provides the basis for enterprise business intelligence solutions
  11. 11. DESIGNING AND IMPLEMENTING A DATA WAREHOUSE How to design a data warehouse and BI solution? Confidential 11
  12. 12. DESIGN AND IMPLEMENT WH • Introduction to Dimensional Modeling • Star Schemas • Considerations for Dimension Tables • Considerations for Fact Tables • Snowflake Schemas Confidential 12
  13. 13. WAREHOUSE MODELING Confidential 13
  14. 14. INTRODUCTION TO DIMENSIONAL MODELING • Business questions focus on measures that are aggregated by business dimensions • Measures are facts about the business • Dimensions are ways in which the measures can be aggregated Product Line Sales person Product Time CustomerRegion Quantity Revenue Cost Profit
  15. 15. STAR SCHEMAS • Group related dimensions into dimension tables • Group related measures into fact tables • Relate fact tables to dimension tables by using foreign keys DimSalesPerson SalesPersonKey SalesPersonName StoreName StoreCity StoreRegion DimProduct ProductKey ProductName ProductLine SupplierName DimCustomer CustomerKey CustomerName City Region FactOrders CustomerKey SalesPersonKey ProductKey ShippingAgentKey TimeKey OrderNo LineItemNo Quantity Revenue Cost Profit DimDate DateKey Year Quarter Month Day DimShippingAgent ShippingAgentKey ShippingAgentName
  16. 16. SNOWFLAKE SCHEMAS DimSalesPerson SalesPersonKey SalesPersonName StoreKey DimProduct ProductKey ProductName ProductLineKey SupplierKey DimCustomer CustomerKey CustomerName GeographyKey FactOrders CustomerKey SalesPersonKey ProductKey ShippingAgentKey TimeKey OrderNo LineItemNo Quantity Revenue Cost Profit DimDate DateKey Year Quarter Month Day DimShippingAgent ShippingAgentKey ShippingAgentName DimProductLine ProductLineKey ProductLineName DimGeography GeographyKey City Region DimSupplier SupplierKey SupplierName DimStore StoreKey StoreName GeographyKey
  17. 17. WAREHOUSE MODELING Confidential 17
  18. 18. WATERFALL BI/WH DEVELOPMENT Traditional SDLC to develop a BI/WH product Confidential 18
  19. 19. WATERFALL BI/WH DEVELOPMENT • SDLC Overview Confidential 19
  20. 20. WATERFALL BI/WH DEVELOPMENT Confidential 20
  21. 21. SDLC OVERVIEW Confidential 21
  22. 22. AGILE BI/WH DEVELOPMENT FRAMEWORK Incremental development framework for BI/WH product Confidential 22
  23. 23. AGILE BI/WH DEVELOPMENT FRAMEWORK • Agile BI/WH life cycle • Agile DW design overview • Agile ETL Solution Confidential 23
  24. 24. AGILE BI/WH LIFE CYCLE Confidential 24
  25. 25. AGILE BI/WH LIFE CYCLE Confidential 25
  26. 26. AGILE DW DESIGN OVERVIEW How to design to answer business question? Confidential 26
  27. 27. AGILE DW DESIGN OVERVIEW • How do we ask question? • The 7Ws framework • Design using natural language • Straightforward methodology • Model storming • BEAM methodology Confidential 27
  28. 28. HOW DO WE ASK QUESTION? • Events/Transactions – A immutable "fact" that occurs in a time and place • Interrogatives: – Who, What, When, Where, Why – Descriptive context that fully describes the event – A set of “dimensions" that describe events Confidential 28
  29. 29. THE 7WS FRAMEWORK Confidential 29 WhyHow How Many
  30. 30. THE 7WS FRAMEWORK HOW – FACTs Much Many Often £$€ Who Customer Employee Seller Organization What Product Service Transactions Booking Event Why Causal Promotion Reason Weather Competition Where Location Geographic Store Ship to Hospital When Time Day Month Year
  31. 31. DESIGN USING NATURAL LANGUAGE • Verbs – Events – Relationships – Fact Tables • Nouns – Details – Entities – Dimensions • Main Clause – Subject-Verb-Object • Prepositions – connect additional details to the main clause • Interrogatives – The 7Ws – Dimension Types Confidential 31
  32. 32. STRAIGHTFORWARD METHODOLOGY Confidential Who What When Where How (many) Why How 1 3 1 1 1 1 1 4 5 2 6 7 8 Declare Event Type Subject-Verb-Object Quantities - Facts Sufficient Detail Fact Granularity Initial Data Examples 9
  33. 33. DESIGN USING NATURAL LANGUAGE • Verbs – Events – Relationships – Fact Tables • Nouns – Details – Entities – Dimensions • Main Clause – Subject-Verb-Object • Prepositions – connect additional details to the main clause • Interrogatives – The 7Ws – Dimension Types Confidential 33
  34. 34. BUSINESS EVENT ANALYSIS AND MODELING (BEAM ) An agile approach to dimensional modeling Confidential 34
  35. 35. MODEL STORMING Confidential 35 Quick Data Modeler BI Stakeholders Inclusive Interactive Fun
  36. 36. BEAM ✲ METHODOLOGY Confidential 36 Structured, non-technical, collaborative working conversation directly with BI Users • BI User’s Business Process, Organizational, Hierarchical, and Data Knowledge • Focused Data Profiling • Logical and Physical Dimensional Data Models • Example data • Detailed and Testable ETL Specification • DW Prototype BEAM✲ Data Modeler BI Stakeholders
  37. 37. Q&A 37
  38. 38. © 2013 KMS Technology THANK YOU.

×