Agile data warehouse
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Agile data warehouse

on

  • 577 views

 

Statistics

Views

Total Views
577
Views on SlideShare
505
Embed Views
72

Actions

Likes
2
Downloads
11
Comments
0

3 Embeds 72

http://votrongdao.blog.com 66
http://www.slideee.com 5
https://www.linkedin.com 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Audience?
  • Immutable: Do not change
  • Inclusive: Tinhtoanven

Agile data warehouse Presentation Transcript

  • 1. © 2013 KMS Technology
  • 2. AGILE DATA WAREHOUSE DESIGN Dao Vo Confidential 2
  • 3. AGENDA • Overview of data warehousing • Designing and implementing a data warehouse • Waterfall BI/WH development • Agile BI/WH development framework • Q&A Confidential 3
  • 4. OVERVIEW OF DATA WAREHOUSING What is a data warehouse? Confidential 4
  • 5. OVERVIEW OF DATA WAREHOUSING • The business problem • What is a data warehouse? • BI/WH Architectures Confidential 5
  • 6. THE BUSINESS PROBLEM • Key business data is distributed across multiple systems
  • 7. THE BUSINESS PROBLEM • Finding the information required for business decision making is time- consuming and error-prone
  • 8. THE BUSINESS PROBLEM • Fundamental business questions are hard to answer
  • 9. WHAT IS A DATA WAREHOUSE?
  • 10. WHAT IS A DATA WAREHOUSE? • A centralized store of business data for reporting and analysis • Typically, a data warehouse: – Contains large volumes of historical data – Is optimized for querying data (as opposed to inserting or updating) – Is incrementally loaded with new business data at regular intervals – Provides the basis for enterprise business intelligence solutions
  • 11. DESIGNING AND IMPLEMENTING A DATA WAREHOUSE How to design a data warehouse and BI solution? Confidential 11
  • 12. DESIGN AND IMPLEMENT WH • Introduction to Dimensional Modeling • Star Schemas • Considerations for Dimension Tables • Considerations for Fact Tables • Snowflake Schemas Confidential 12
  • 13. WAREHOUSE MODELING Confidential 13
  • 14. INTRODUCTION TO DIMENSIONAL MODELING • Business questions focus on measures that are aggregated by business dimensions • Measures are facts about the business • Dimensions are ways in which the measures can be aggregated Product Line Sales person Product Time CustomerRegion Quantity Revenue Cost Profit
  • 15. STAR SCHEMAS • Group related dimensions into dimension tables • Group related measures into fact tables • Relate fact tables to dimension tables by using foreign keys DimSalesPerson SalesPersonKey SalesPersonName StoreName StoreCity StoreRegion DimProduct ProductKey ProductName ProductLine SupplierName DimCustomer CustomerKey CustomerName City Region FactOrders CustomerKey SalesPersonKey ProductKey ShippingAgentKey TimeKey OrderNo LineItemNo Quantity Revenue Cost Profit DimDate DateKey Year Quarter Month Day DimShippingAgent ShippingAgentKey ShippingAgentName
  • 16. SNOWFLAKE SCHEMAS DimSalesPerson SalesPersonKey SalesPersonName StoreKey DimProduct ProductKey ProductName ProductLineKey SupplierKey DimCustomer CustomerKey CustomerName GeographyKey FactOrders CustomerKey SalesPersonKey ProductKey ShippingAgentKey TimeKey OrderNo LineItemNo Quantity Revenue Cost Profit DimDate DateKey Year Quarter Month Day DimShippingAgent ShippingAgentKey ShippingAgentName DimProductLine ProductLineKey ProductLineName DimGeography GeographyKey City Region DimSupplier SupplierKey SupplierName DimStore StoreKey StoreName GeographyKey
  • 17. WAREHOUSE MODELING Confidential 17
  • 18. WATERFALL BI/WH DEVELOPMENT Traditional SDLC to develop a BI/WH product Confidential 18
  • 19. WATERFALL BI/WH DEVELOPMENT • SDLC Overview Confidential 19
  • 20. WATERFALL BI/WH DEVELOPMENT Confidential 20
  • 21. SDLC OVERVIEW Confidential 21
  • 22. AGILE BI/WH DEVELOPMENT FRAMEWORK Incremental development framework for BI/WH product Confidential 22
  • 23. AGILE BI/WH DEVELOPMENT FRAMEWORK • Agile BI/WH life cycle • Agile DW design overview • Agile ETL Solution Confidential 23
  • 24. AGILE BI/WH LIFE CYCLE Confidential 24
  • 25. AGILE BI/WH LIFE CYCLE Confidential 25
  • 26. AGILE DW DESIGN OVERVIEW How to design to answer business question? Confidential 26
  • 27. AGILE DW DESIGN OVERVIEW • How do we ask question? • The 7Ws framework • Design using natural language • Straightforward methodology • Model storming • BEAM methodology Confidential 27
  • 28. HOW DO WE ASK QUESTION? • Events/Transactions – A immutable "fact" that occurs in a time and place • Interrogatives: – Who, What, When, Where, Why – Descriptive context that fully describes the event – A set of “dimensions" that describe events Confidential 28
  • 29. THE 7WS FRAMEWORK Confidential 29 WhyHow How Many
  • 30. THE 7WS FRAMEWORK HOW – FACTs Much Many Often £$€ Who Customer Employee Seller Organization What Product Service Transactions Booking Event Why Causal Promotion Reason Weather Competition Where Location Geographic Store Ship to Hospital When Time Day Month Year
  • 31. DESIGN USING NATURAL LANGUAGE • Verbs – Events – Relationships – Fact Tables • Nouns – Details – Entities – Dimensions • Main Clause – Subject-Verb-Object • Prepositions – connect additional details to the main clause • Interrogatives – The 7Ws – Dimension Types Confidential 31
  • 32. STRAIGHTFORWARD METHODOLOGY Confidential Who What When Where How (many) Why How 1 3 1 1 1 1 1 4 5 2 6 7 8 Declare Event Type Subject-Verb-Object Quantities - Facts Sufficient Detail Fact Granularity Initial Data Examples 9
  • 33. DESIGN USING NATURAL LANGUAGE • Verbs – Events – Relationships – Fact Tables • Nouns – Details – Entities – Dimensions • Main Clause – Subject-Verb-Object • Prepositions – connect additional details to the main clause • Interrogatives – The 7Ws – Dimension Types Confidential 33
  • 34. BUSINESS EVENT ANALYSIS AND MODELING (BEAM ) An agile approach to dimensional modeling Confidential 34
  • 35. MODEL STORMING Confidential 35 Quick Data Modeler BI Stakeholders Inclusive Interactive Fun
  • 36. BEAM ✲ METHODOLOGY Confidential 36 Structured, non-technical, collaborative working conversation directly with BI Users • BI User’s Business Process, Organizational, Hierarchical, and Data Knowledge • Focused Data Profiling • Logical and Physical Dimensional Data Models • Example data • Detailed and Testable ETL Specification • DW Prototype BEAM✲ Data Modeler BI Stakeholders
  • 37. Q&A 37
  • 38. © 2013 KMS Technology THANK YOU.