Phua Chiu Kiang
Microsoft MVP (SQL Server)
•
•
•
•
•
•
Microsoft Data Warehousing Vision
Make SQL Server the gold standard for data warehousing offering customers




Massive Sc...
Approximate data volume
    •                                                              managed by data warehouse
    •...
Data Warehouse Industry Trends
                           100%
       Broad Commitment




                               ...
6
•   Building a traditional DW
       •   Time consuming
       •   Expensive
       •   Performance varies
       •   Scal...
Software:
  • SQL Server 2008
     Enterprise
  • Windows Server 2008

Configuration guidelines:
     • Physical table str...
Reduces DBA effort; fewer indexes,
  much higher level of sequential I/O




        Dell, HP, Bull, EMC and IBM – more in...
SQL Server
                          Teradata                                 Comparison
                                 ...
•
    −

    −
    −

•
    −

    −
    −

•
    −

    −
    −
•
    −
    −
    −

•
    −
    −
    −

•
    −

    −
    −
•
    −

    −
    −
    −
    −

•
    −

    −
    −
    −
    −
•
                         −

                         −
                         −

                   •
                ...
Fast Track vNext
                                                            Fast Track Data Warehouse 2.0                ...
•
•

•
•
•
Parallel Data Warehouse compute node




      Database Server      Storage Node
Parallel Data Warehouse Appliance - Hardware
Architecture
                                                                ...
Parallel Data Warehouse demo at BI conference 2008
                                       • Query
                        ...
Existing                  Current            Madison
    Environment                Challenges         Highlights

Hardwar...
Parallel Data Warehouse

•
•
•
•
    −
    −
•
•
    −
    −
PDW vNext
                                                                                                       Focus on ...
Hub and Spoke – Flexible Business Alignment




  EDW provides “single version of truth” but makes it difficult to support...
Hub and Spoke – Flexible Business Alignment




   Departmental data marts enable mixed workloads, but make it difficult t...
Hub and Spoke – Flexible Business Alignment

    Parallel database copy                                  Support user grou...
GEO AREAS   METRICS
Analytic MDM
Faster time to solution
                          High scale: up to 48TB
   Fast Track             Low TCO with better pri...
• Fast Track Data Warehouse offers
    −
    −
    −
    −
    −
• Parallel Data Warehouse offers
    −
    −
    −
    −
...
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be...
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Sql2008 R2 Dw (Phua Chiu Kiang)
Upcoming SlideShare
Loading in …5
×

Sql2008 R2 Dw (Phua Chiu Kiang)

851 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
851
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
21
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Sql2008 R2 Dw (Phua Chiu Kiang)

  1. 1. Phua Chiu Kiang Microsoft MVP (SQL Server)
  2. 2. • • • • • •
  3. 3. Microsoft Data Warehousing Vision Make SQL Server the gold standard for data warehousing offering customers Massive Scalability at Low Hardware Choice Improved Business Agility Cost and Alignment
  4. 4. Approximate data volume • managed by data warehouse • Today In 3 Years • 21% Less than 500 GB 5% • • 500 GB – 1 TB 12% 20% 21% 1 – 3 TB 18% • 19% • 3 – 10 TB 25% 17% More than 10 TB 34% 2% Don’t Know 6% Source: TDWI Report – Next Generation DW Microsoft Confidential—Preliminary Information Subject to Change 4
  5. 5. Data Warehouse Industry Trends 100% Broad Commitment Advanced Centralized Analytics EDW Data Quality  75% Analytics within EDW HA for DW  64-bit  MDM  Analytics Web Services Outside EDW DBMS Built for DW  Plan to Use Real-time DW Blades in Security MPP Racks 50% DBMS Built for DW Appliance  Streaming Data  Transactions Mixed Workloads  SOA Server Virtualization Data Federation Low-Power SMP Hardware Columnar DBMS DW In-Memory DBMS Bundles 25% SaaS Narrow Commitment Open Source Open Source OS Reporting Software Open Source Appliance Data Integration Open Source DBMS Public Cloud 0% -50% -25% 0% 25% 50% 75% 100% Decreasing Usage Anticipated Growth in the next 3 Years Increasing Usage  Areas of strategic investment for Microsoft Source: TDWI
  6. 6. 6
  7. 7. • Building a traditional DW • Time consuming • Expensive • Performance varies • Scalability issues Potential bottlenecks in standard DW architecture • The DW appliance model • Tuned h/w + s/w Faster • Views entire stack holistically Lower TCO deployment • Known performance & scalability Benefits • Encapsulates best practices Better Minimised performance DBA time • Leverages Sequential I/O ©2009 Microsoft Corporation
  8. 8. Software: • SQL Server 2008 Enterprise • Windows Server 2008 Configuration guidelines: • Physical table structures • Indexes • Compression • SQL Server settings • Windows Server settings • Loading Hardware: • Tight specifications for servers, storage and networking • ‘Per core’ building block
  9. 9. Reduces DBA effort; fewer indexes, much higher level of sequential I/O Dell, HP, Bull, EMC and IBM – more in future Commodity Hardware and value pricing; Lower storage costs. New reference architectures scale up to 48TB (assuming 2.5x compression) Validated by Microsoft; better choice of hardware; application of Best Practice
  10. 10. SQL Server Teradata Comparison Fast Track DW Loading 5:10:21 total time 0:51:31 total time R Subject Area 1 6x faster Loading 4:36:08 total time 1:50.01 total time R Subject Area 2 2.5x faster Query times 3:03 avg query time (using 9 benchmark 0:15 avg query time (using 9 benchmark R Subject Area 1 queries) queries) 12x faster Query times 56:44 avg query time (using 4 benchmark 8:09 avg query time (using 4 benchmark R Subject Area 2 queries) queries) 7x faster ©2009 Microsoft Corporation
  11. 11. • − − − • − − − • − − −
  12. 12. • − − − • − − − • − − −
  13. 13. • − − − − − • − − − − −
  14. 14. • − − − • − − − • − − − − − Microsoft Confidential
  15. 15. Fast Track vNext Fast Track Data Warehouse 2.0 Future Partners to create new Enterprise ETL Services Validated Reference Star Join Query Optimizations New Reference Architectures from IBM Architectures with Test Harness Updated Configurations from HP, Dell and Bull EMC as a Service Partner for Fast Track 2008 2009 2010 Beyond Fast Track Data Warehouse New Test Harness for Partners DW Reference Architectures Microsoft to create new Test Predictable performance at low Harness for validation of new cost Fast Track configurations Faster time to solution NEC to validate new Reference Architectures Microsoft Confidential—Preliminary 16
  16. 16. • • • • •
  17. 17. Parallel Data Warehouse compute node Database Server Storage Node
  18. 18. Parallel Data Warehouse Appliance - Hardware Architecture Database Servers Storage Nodes Control Nodes SQL Active / Passive SQL SQL SQL SQL Management Servers Dual Fiber Channel SQL Dual Infiniband SQL Landing Zone SQL SQL SQL Backup Node SQL Spare Database Server Corporate Network Private Network
  19. 19. Parallel Data Warehouse demo at BI conference 2008 • Query ‐ Cache flushed ‐ Inner joins • Report ‐ Retailer: day-part analysis ‐ Sales, Time, Date, Prod type • Sample Results ‐ 625K rows returned in 11 seconds from 1 trillion row table ‐ Final product will be even faster
  20. 20. Existing Current Madison Environment Challenges Highlights Hardware Data Load Speeds Improved by 300% 16 CPU HP 8620 Itanium Hitachi Storage 27TB Raw SATA 21 LUNS Analytic Capacity 30TB/160 Cores Software Analytic Speed Query Speeds 70X Windows 2003 SP2 Improvement SQLServer 2008 SSIS/SSRS Mixed Workload Concurrency Data Warehouse Mixed Workload 18 Terabytes Star Schema Total Cost of TCO Lowered by 80 Fact Tables 500 + Dimensions Ownership 50%
  21. 21. Parallel Data Warehouse • • • • − − • • − −
  22. 22. PDW vNext Focus on continually lowering the Microsoft Announce Intention to MTP Program Launched costs of high end DW, while Acquire DATAllegro (July) Circa 10 Customers Provided with early increasing performance Acquisition Closes (Sept) Madison Benchmark Additional Hardware Partners 150TB demo of DATAllegro on SQL Madison Named as SQL Server 2008 R2 Closer functional alignment with SQL Server run at BI Conference (Oct) Parallel Data Warehouse Server List Price at $57.5K per proc Better integration with SQL and tools and technologies 2008 2009 2010 Beyond Project “Madison” MTP 2 Program to Launch (fully Compatibility with DATAllegro v3 functional, fully performant) MS BI integration TAP Program (on client site) RTM in H1 2010 ?
  23. 23. Hub and Spoke – Flexible Business Alignment EDW provides “single version of truth” but makes it difficult to support mixed workloads and multiple user groups, each requiring SLAs
  24. 24. Hub and Spoke – Flexible Business Alignment Departmental data marts enable mixed workloads, but make it difficult to consolidate information across the enterprise
  25. 25. Hub and Spoke – Flexible Business Alignment Parallel database copy Support user groups with technology enables rapid very different SLAs: data movement and Performance consistency between hub Capacity and spokes Loading Concurrency Create SQL Server 2008, Fast Track Data Warehouse, and SQL Server Analysis Services spokes A Hub and Spoke solution gives you the flexibility to add/change diverse workloads/user groups, while maintaining data consistency across the enterprise
  26. 26. GEO AREAS METRICS
  27. 27. Analytic MDM
  28. 28. Faster time to solution High scale: up to 48TB Fast Track Low TCO with better price performance; industry standard hardware Data Warehouse Better performance out of the box and predictable performance offers customers Reduced risk through balanced hardware & Best practices Integration with Madison Hub & Spoke Architecture Twelve reference architectures from HP, Dell, Bull, EMC SQL Server Fast Track Data and IBM Warehouse has 2 components System Integrators with industry solution templates – Avanade, HP, Hitachi, Cognizant and EMC
  29. 29. • Fast Track Data Warehouse offers − − − − − • Parallel Data Warehouse offers − − − − • − − −
  30. 30. © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

×