Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

IOUG93 - Technical Architecture for the Data Warehouse - Presentation


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

IOUG93 - Technical Architecture for the Data Warehouse - Presentation

  1. 1. David M Walker Consultant Data Management & Warehousing ATechnical Architecture For The Data Warehouse
  2. 2. Data Warehouse Implementation Strategy Project Management Business Analysis Database Schema Design Technical Architecture
  3. 3. Business Analysis•! End user driven•! Cross Functional Workshops•! Iterative design principle (80/20 rules)•! Determine the Key Performance Indicators (KPI)•! Determine constraints on KPI
  4. 4. Database Schema Design•! Identify sources of information•! Qualify external sources of information•! Translate KPI into facts•! Translate constraints into dimensions•! Choose required aggregations•! Build Meta Data and Security Model
  5. 5. Project Management•! Iterative Process•! Rapid Application Development (RAD) techniques•! Arbitration when 80/20 rule used•! Conflict of short and long term goals
  6. 6. The Data Warehouse Systems Logical Architecture Presentation Third Party Tools Third Party Tools Layer The Data Warehouse Middleware Middleware Security EIS EIS Meta Data Decision Decision Support Systems Support Systems Transaction Repository Data Acquisition Operational Systems OLTP Legacy External System System Data Sources
  7. 7. Data Acquisition Data Extraction Data Load •!Extraction •!Loading •!Transformation •!Exception Processing •!Collation •!Quality Assurance •!Migration •!Publication
  8. 8. Transaction Repository Dimension Dimension Dimension Dimension Fact Fact Fact Fact Dimension Dimension Fact Fact Fact Dimension Dimension Dimension Dimension
  9. 9. Data Aggregation Year Executive Information Systems Quarter Month Decision Support System Week Transaction Repository Day
  10. 10. The Cost Of AggregationA very simple schema:100 Stores 1095 Days 100000 Products 10 Regions 157 Weeks 1000 Categories 1 Company 36 Month 10 Groups 12 Quarters 1 Type 3 YearsRows: No aggregation, No sparsity: 10950000000 Aggregation, No sparsity: 14609523963 Growth 33% No aggregation,30% sparsity: 7665000000 Aggregation, Variable sparsity: 10574481741 Growth 38%If each row is 64 bytes long, a 10Billion row schema without indexesand other overheads would be 630Gb!
  11. 11. Data Mart Time Dimension Associated Another Dimension Day Facts Week Month Quarter Year Another Dimension Another Dimension
  12. 12. Meta Data Dictionary And Security Meta Data •!Master schema Security •!Star schema Control of •!Star schema description user access •!Table to the data •!Table description •!Table row count •!Column •!Column description •!Column derivation •!Column format
  13. 13. Middleware and Presentation•! Use a common middleware•! Group users based on their requirements•! Try a number of tools for each group•! Final solution will have more than one front end, but not an infinite number•! Add value with alert systems
  14. 14. ConclusionStrategy Technical Architeture •! Project Managment •! Source Systems •! Business Analysis •! Data Acquisition •! Schema Design •! Transaction Repository •! Technical Architecture •! Data Aggregation •! Data Mart •! Meta Data & Security •! Middleware & Presentation Help your users find it !
  15. 15. Contacts•! Data Management & Warehousing –! WWW –! Mail –! Telephone +44 1734 771291 –! Fax +44 1734 773058•! The Data Warehouse Institute –! WWW –! Mail•! The Data Warehouse Information Center –! WWW