The document discusses the data warehouse lifecycle and key components. It covers topics like source systems, data staging, presentation area, business intelligence tools, dimensional modeling concepts, fact and dimension tables, star schemas, slowly changing dimensions, dates, hierarchies, and physical design considerations. Common pitfalls discussed include becoming overly focused on technology, tackling too large of projects, and neglecting user acceptance.

1. The Data Warehouse LifecycleBart LoweDecision Source Inc.

2. AgendaDiscuss high level concepts related to Data warehousing

3. The Data Warehouse Must…Make information easily accessiblePresent information consistentlyBe adaptive & resilient to changeBe SecureServe as the foundation for decision making

4. The Business Community Must…Accept and trust the data warehouse if it is to be successful

5. Data Warehouse Lifecycle

6. Data Warehouse Lifecycle

7. Data Warehouse Components

8. Data Warehouse Components

9. Source SystemsNot Optimized for Reporting

10. Data schemas optimized for transactions not queries

11. Difficult to share data

12. Typically do not maintain historical dataData Staging AreaConsists of Staging Storage & ETL Processes

13. This is typically the most difficult and labor intensive component

14. Data is cleansed & conformed

15. Typically a normalized data schema

16. No direct querying is allowed to this componentData Presentation AreaThis is the data that is made available to users & analytical applications

17. Consists of a series of conformed dimensional data marts

18. Each Data Mart represents a difference business process.

19. Dimensional modeling emphasizes simplicity & query performance.Business Intelligence ToolsThese are the tools that are used to query the data

20. In General only a small subset of users will need true ad-hoc query capability

21. 80-90% of users will used a parameterized analytic systemDimensional Modeling Vocabulary

22. Fact TableThis is the primary table in a dimensional modelThe measurements of the dimensional model are stored hereEach measurement is tracked at the intersection of several dimensionsThis is the “grain” of the modelMost useful facts are additive

23. Dimension TableDescriptors of each factTend to have many attributes but fewer rowsTend to be used as query constraints.The better the attribute descriptions the better the warehouseTypically highly denormalized

24. Star SchemaThis is a fact table joined to a set of dimensionsRelates data in a manner that is familiar to business users.Symmetrical nature allows for many answering many different business questionsOne dimensional model will exist for each business process. A single data warehouse can have dozens of these models.

25. Dimensional Modeling Key Concepts

26. Store the most atomic dataBy storing the most detail data possible you can ensure that users can drill to the level they need. Its OK to provide aggregate facts as well to improve performance.

27. Conformed DimensionsBy conforming your dimensions you can correlate performance across business processes.Can be very painful (but worth it) if combining data from disparate systems.

28. Always use an artificial key as the primary keySurrogate Key allow you to:Protect you model from changes in the source systemIntegrate data from multiple sourcesAdd rows that do not exist in the source system.Track changes to dimensions over time.Use Surrogate Keys

29. A key design consideration is what to do when dimension values change.A change may or may not have business meaning.Three ways to handle changesSlowly Changing Dimensions

30. Slowly Changing Dimension TypesType ISimply overwrite the old values.Simplest case, used when you don’t care about changes to data.Type IICreate a new dimension row for new valuesExisting facts still relate to old dimension valueUsed when you do care about the historical changes.Type IIIAdd a new column to table to store the new valueRarely used.

31. Dates are a fundamental Business concept and nearly every DW has a date dimensionThe date dimension is the classic role playing dimension.Allows rollups/filters on any date related attribute such as month/quarter/year Date dimension records still use a surrogate to handle unknown dates.Date Dimensions

32. Snowflaking is the process of hooking up lookup tables to a dimension.This is in a way re-normalizing the data.Snowflaking is in general discouraged since it adds complexity to the model.Snowflaking

33. Most relationships are one-to-many. This is the simplest case.Real world scenarios are often more complex.Many to Many between facts & dimensions are represented by creating a bridge table between the facts and the dimensionMany to Many Relationships

34. Hierarchies summarize or group the data within the dimension.Typically are de-normalized into the dimension tableHierarchies

35. There are three types of fact tablesTransactionTracks each transaction as it occurs.Periodic SnapshotCaptures cumulative performance over a specific period of timeOften used for periodic rollupsAccumulating SnapshotUpdated over timeTypes of Fact Tables