Fast track foundations getting serious about sequential io

4,598 views
4,534 views

Published on

SQLBits VI Presentation on SQL Server Fast Track Data Warehousing Concepts

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
4,598
On SlideShare
0
From Embeds
0
Number of Embeds
2,551
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 1
  • SMP - Symmetric MultiProcessing
    Scale up design that is balanced provides
  • T1117 Ensures even growth of all files in a file group
    E Extent Allocation now 64 contiguous extents per file
  • 1 Gigabit = 128 MegaBytes
  • 1 Gigabit = 128 MegaBytes
  • Disk blocks per database file should be kept contiguous on the physical platter within the NTFS file system
  • 31
  • 1 Gigabit = 128 MegaBytes
  • Fast track foundations getting serious about sequential io

    1. 1. 1© Copyright 2009 EMC Corporation. All rights reserved. Fast Track Foundations: Serious about Sequential I/O James Rowland-Jones & Allan Mitchell
    2. 2. 2© Copyright 2009 EMC Corporation. All rights reserved. WTF IS FAST TRACK? Introduction
    3. 3. 3© Copyright 2009 EMC Corporation. All rights reserved. Blend of Hardware & Software
    4. 4. 4© Copyright 2009 EMC Corporation. All rights reserved. Predictable Performance Balanced Configurations – CPU – Memory – Network – Storage
    5. 5. 5© Copyright 2009 EMC Corporation. All rights reserved. Targeted to the Data Warehouse Workload Sequential Scanning of Data Leads to fewer disks
    6. 6. 6© Copyright 2009 EMC Corporation. All rights reserved. Prescriptive Best Practices Recommendations You don’t get to pick and choose!
    7. 7. 7© Copyright 2009 EMC Corporation. All rights reserved. WHY SQL 2008 ONLY?
    8. 8. 8© Copyright 2009 EMC Corporation. All rights reserved. SQL Server 2008 Enterprise... ...was the Data Warehousing release... Compression Star Joins Few Outer Rows Parallelism Partitioned Table Parallelism Resource Governor Improved Read Ahead
    9. 9. 9© Copyright 2009 EMC Corporation. All rights reserved. SQL 2008 Settings Startup Parameters – T1117 – E Resource Governor – Reduce Max memory per session Page Compression ON!
    10. 10. 10© Copyright 2009 EMC Corporation. All rights reserved. Windows 2008 Settings Multipath I/O Feature is required Lock Pages In Memory Enabled
    11. 11. 11© Copyright 2009 EMC Corporation. All rights reserved. SEQUENTIAL VS RANDOM I/O DEMO TIME:
    12. 12. 12© Copyright 2009 EMC Corporation. All rights reserved. Counters Performance Counter  Avg. Disk Bytes / Read  Avg. Disk Sec / Read  Disk Read Bytes / Sec  Disk Reads / Sec Meaning  Size of I/Os  Latency “Seek Time”  Throughput – MB/s  I/Os per second  sys.dm_db_index_physical_file_stats  sys.dm_io_virtual_file_stats  sys.dm_io_pending_io_requests DMVs
    13. 13. 13© Copyright 2009 EMC Corporation. All rights reserved. Time to Take the Red Pill...
    14. 14. 14© Copyright 2009 EMC Corporation. All rights reserved.
    15. 15. 15© Copyright 2009 EMC Corporation. All rights reserved. DB DESIGN CONSIDERATIONS Fast Track Thinking
    16. 16. 16© Copyright 2009 EMC Corporation. All rights reserved. Indexing Think Index Light  NC Indexes aimed at low volume row lookups  Indexes are Expensive to maintain
    17. 17. 17© Copyright 2009 EMC Corporation. All rights reserved. Aggregation Think Aggregation Light  Introduces Updates (fragmentation)  Removing aggs reduces maintenance
    18. 18. 18© Copyright 2009 EMC Corporation. All rights reserved. Partitioning Partition Qualification Data Management P1 P4P3P2P 2
    19. 19. 19© Copyright 2009 EMC Corporation. All rights reserved. RG1 RG2 RG3 RG4 RG5 RG6 RG7 RG8 RG9 RG10 Conceptual View of Fast Track Hardware 1 3 2 4 5 7 6 8 H H 4GB 4GB 4GB 4GB 4GB 4GB 4GB RAID 1 LUN 01 LUN 02 LUN 03 LUN 04 LUN 17 (Logs) LUN 05 LUN 07 LUN 06 LUN 08 SPARE SPASPB FC1 FC2 FC3 FC4 FibreSwitch LUN 09 LUN 11 LUN 10 LUN 12 LUN 18 (Logs) LUN 13 LUN 15 LUN 14 LUN 16 SPARE SPCSPD FC5 FC6 FC7 FC8 8 Core 32 GB RAM Server 4GB
    20. 20. 20© Copyright 2009 EMC Corporation. All rights reserved. RG1 RG2 RG3 RG4 RG5 RG6 RG7 RG8 RG9 RG10 Performance View of Fast Track Hardware 1 3 2 4 5 7 6 8 H H LUN 01 LUN 02 LUN 03 LUN 04 LUN 17 (Logs) LUN 05 LUN 07 LUN 06 LUN 08 SPASPB FC1 FC2 FC3 FC4 FibreSwitch LUN 09 LUN 11 LUN 10 LUN 12 LUN 18 (Logs) LUN 13 LUN 15 LUN 14 LUN 16 SPCSPD FC5 FC6 FC7 FC8 200 mb/s per CPU core is central to Fast Track Architecture 32GB RAM800mb/s 800mb/s 400mb/s per port 1 LUN = 125 MB/S SP Port 650MB/s FCPort 400MB/S
    21. 21. 21© Copyright 2009 EMC Corporation. All rights reserved. Performance Summary Component Throughput 8 CPU Cores 1600 MB/S 4 HBA Ports (2 adapters) 1600 MB/S 8 FC Ports (4 per array) 3200 MB/S 4 Storage Processors (2 per array) 2600 MB/S 16 LUNS (Data Only) 2000 MB/S
    22. 22. 22© Copyright 2009 EMC Corporation. All rights reserved. Capacity Summary 16x 300GB Data Disks Raw (formatted) 4352 GB 4x 300GB Log Disks Raw (formatted) 1088 GB LESS RAID 1 8x RAID 1 Raid Groups for Data 2176 GB 2x RAID 1 Raid Groups for Logs 544 GB LESS TEMPDB SPACE Tempdb Data files 576 GB User Data Uncompressed Available 1600 GB Data Compression Ratio 2.5:1 4000 GB
    23. 23. 23© Copyright 2009 EMC Corporation. All rights reserved. Database Layout Fi1 Fi2 Fi3 Fi4 Fi5 Fi6 Fi7 Fi9 Fi10 Fi11 Fi12 Fi13 Fi14 Fi15 Fi1 Fi2 Fi3 Fi4 Fi5 Fi6 Fi7 Fi1 Fi2 Fi3 Fi4 Fi5 Fi6 Fi7 Fi9 Fi10 Fi11 Fi12 Fi13 Fi14 Fi15 Fi9 Fi10 Fi11 Fi12 Fi13 Fi14 Fi15 tempLog Live Log Stage Log Live FG1 Stage FG1 Tempdb Primary Fi8 Fi8 Fi8 Fi16 Fi16 Fi16 Fi0 Fi0 PrimaryPrimary
    24. 24. 24© Copyright 2009 EMC Corporation. All rights reserved. MANAGING FRAGMENTATION The arch enemy of Sequential I/O
    25. 25. 25© Copyright 2009 EMC Corporation. All rights reserved. Fragmentation File System Extent Index
    26. 26. 26© Copyright 2009 EMC Corporation. All rights reserved. File System Fragmentation Pre-Allocate your Database files Avoid O/S de-fraggers They know not what they do...
    27. 27. 27© Copyright 2009 EMC Corporation. All rights reserved. Extent Fragmentation Occurs Courtesy of Concurrent DML Updates Deletes Follow the guidelines! Otherwise you’ll be re-loading the tables....
    28. 28. 28© Copyright 2009 EMC Corporation. All rights reserved. Index Fragmentation  Don’t use ALTER INDEX REORGANIZE  Do one of the following – ALTER INDEX REBUILD ... SORT_IN_TEMPDB = TRUE (MAXDOP 1) – INSERT ... SELECT into a new table
    29. 29. 29© Copyright 2009 EMC Corporation. All rights reserved. File Groups Isolate Volatile Data  Create a separate Filegroup  One file per LUN to spread the I/O Isolate Small / Dimension Tables Stage Data in a Separate DB or File Group
    30. 30. 30© Copyright 2009 EMC Corporation. All rights reserved. DATA LOADING FOR SEQUENTIAL I/O Over to Mitchell...
    31. 31. where Allan and Darren live

    ×