IOUG93 - Technical Architecture for the Data Warehouse - Presentation

  • 908 views
Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
908
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
28
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. David M Walker Consultant Data Management & Warehousing ATechnical Architecture For The Data Warehouse
  • 2. Data Warehouse Implementation Strategy Project Management Business Analysis Database Schema Design Technical Architecture
  • 3. Business Analysis•! End user driven•! Cross Functional Workshops•! Iterative design principle (80/20 rules)•! Determine the Key Performance Indicators (KPI)•! Determine constraints on KPI
  • 4. Database Schema Design•! Identify sources of information•! Qualify external sources of information•! Translate KPI into facts•! Translate constraints into dimensions•! Choose required aggregations•! Build Meta Data and Security Model
  • 5. Project Management•! Iterative Process•! Rapid Application Development (RAD) techniques•! Arbitration when 80/20 rule used•! Conflict of short and long term goals
  • 6. The Data Warehouse Systems Logical Architecture Presentation Third Party Tools Third Party Tools Layer The Data Warehouse Middleware Middleware Security EIS EIS Meta Data Decision Decision Support Systems Support Systems Transaction Repository Data Acquisition Operational Systems OLTP Legacy External System System Data Sources
  • 7. Data Acquisition Data Extraction Data Load •!Extraction •!Loading •!Transformation •!Exception Processing •!Collation •!Quality Assurance •!Migration •!Publication
  • 8. Transaction Repository Dimension Dimension Dimension Dimension Fact Fact Fact Fact Dimension Dimension Fact Fact Fact Dimension Dimension Dimension Dimension
  • 9. Data Aggregation Year Executive Information Systems Quarter Month Decision Support System Week Transaction Repository Day
  • 10. The Cost Of AggregationA very simple schema:100 Stores 1095 Days 100000 Products 10 Regions 157 Weeks 1000 Categories 1 Company 36 Month 10 Groups 12 Quarters 1 Type 3 YearsRows: No aggregation, No sparsity: 10950000000 Aggregation, No sparsity: 14609523963 Growth 33% No aggregation,30% sparsity: 7665000000 Aggregation, Variable sparsity: 10574481741 Growth 38%If each row is 64 bytes long, a 10Billion row schema without indexesand other overheads would be 630Gb!
  • 11. Data Mart Time Dimension Associated Another Dimension Day Facts Week Month Quarter Year Another Dimension Another Dimension
  • 12. Meta Data Dictionary And Security Meta Data •!Master schema Security •!Star schema Control of •!Star schema description user access •!Table to the data •!Table description •!Table row count •!Column •!Column description •!Column derivation •!Column format
  • 13. Middleware and Presentation•! Use a common middleware•! Group users based on their requirements•! Try a number of tools for each group•! Final solution will have more than one front end, but not an infinite number•! Add value with alert systems
  • 14. ConclusionStrategy Technical Architeture •! Project Managment •! Source Systems •! Business Analysis •! Data Acquisition •! Schema Design •! Transaction Repository •! Technical Architecture •! Data Aggregation •! Data Mart •! Meta Data & Security •! Middleware & Presentation Help your users find it !
  • 15. Contacts•! Data Management & Warehousing –! WWW http://www.datamgmt.com –! Mail davidw@datamgmt.com –! Telephone +44 1734 771291 –! Fax +44 1734 773058•! The Data Warehouse Institute –! WWW http://www.tekptnr.com/tpi/tdwi –! Mail tdwi@aol.com•! The Data Warehouse Information Center –! WWW http://pwp.starnetinc.com/larryg/index.html