Table Partitioning: Secret Weapon for Big Data Problems

2,095 views

Published on

Published in: Education, Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,095
On SlideShare
0
From Embeds
0
Number of Embeds
845
Actions
Shares
0
Downloads
49
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Table Partitioning: Secret Weapon for Big Data Problems

  1. 1. Table Partitioning: Secret Weapon for Big Data Problems October 15-18, 2013 Charlotte, NC John Sterrett, Sr. Database Admin Advisor DELL
  2. 2. About John 2
  3. 3. How to contact me? http://johnsterrett.com/go/partition Blog: http://JohnSterrett.com/ Twitter: @JohnSterrett LinkedIn: http://linkedin.com/in/johnsterrett 3
  4. 4. Agenda Big Data starting to slow you down? Data growth putting your maintenance tasks in danger of not meeting your SLAs? Wish you could archive old data with minimal impact to your tables during the archive process or that you could eliminate most of the data in your tables when you query them? If so, it’s time you consider implementing table partitioning to help with general performance and reduce your window for completing maintenance tasks. •  Learn if Partitioning can help you manage big data. •  Understand how partitioning works. •  Learn how to maintain table partitioning with an automated sliding window. 4
  5. 5. Assumptions… •  You are not familiar with Table Partitioning and want to see how it works. •  You have Enterprise Edition •  Highly Transactional Tables with over 50 GB of data •  General Maintenance Tasks are in danger of not meeting your SLA •  Purge and/or Archive process is slowing you down. 5
  6. 6. The Big Question… HOW CAN TABLE PARTITIONG MAKE MY LIFE EASIER? 6
  7. 7. How partitioning helps me? •  Reduce Maintenance Tasks 7
  8. 8. How partitioning helps me? •  Reduce Maintenance Tasks •  Improves Purging and/or Archiving 8
  9. 9. How partitioning helps me? •  Reduce Maintenance Tasks •  Improves Purging and/or Archiving •  Improves Performance 9
  10. 10. Big Question… HOW DOES TABLE PARTITIONING WORK? 10
  11. 11. High Level… 11
  12. 12. High Level… Partition  Table Partition  Scheme Partition  Function 12
  13. 13. Selecting The Partition Column Only get one column, use it wisely! Column should be a highly used filter. •  Review index usage statistics •  Review Missing index statistics •  Review Queries and Talk to developers ;-) Column must be part of clustered index 13
  14. 14. Partition Function Defines the data type used to distribute data into partitions. Assigns boundary values to split data between partitions. Assigns the RANGE for boundary values Partitions = Boundary values + 1 NULL values go to left most partition 14
  15. 15. Partition Function Partition functions are not static. They can change over time with SPLIT and MERGE statements. Range LEFT is used by default. 15
  16. 16. Partition Function - Range {min….100}, {101…200}, {201...300}, {301…max} {min…99}, {100…199}, {200…299}, {300…max} 16
  17. 17. Partition Scheme Assigns a partition function to a partition scheme Assigns filegroups to partitions 17
  18. 18. Partitioning DEMO October 15-18, 2013 | Charlotte, NC
  19. 19. Big Question… HOW DOES TABLE PARTITIONING IMPROVE MY MAINTENANCE TASKS? 19
  20. 20. Improving Maintenance Tasks Backup and restore filegroups based on business priorities. Index Maintenance by partition New Features in SQL 2014 •  Rebuild index online by partition •  Incremental Statistics by partition (SQL 2014 CTP 2) 20
  21. 21. Incremental Statistics Just added in SQL Server 2014 CTP2 From Books Online: “When ON, the statistics are recreated as per partition statistics.” Cannot be used for the following: •  Statistics created with indexes that are not partitionaligned with the base table •  Filtered Indexes 21
  22. 22. HOW DOES TABLE PARTITIONING IMPROVE PERFORMANCE? 22
  23. 23. Partition Elimination 23
  24. 24. Skip-Scan: Seek keys 24
  25. 25. Database Compression by Partition 25
  26. 26. Partitioning DEMO October 15-18, 2013 | Charlotte, NC
  27. 27. HOW DOES PARTITIONING IMPROVE ARCHIVING AND PURGING? 27
  28. 28. Real World Example: 28
  29. 29. Quickest way to move billions of rows Meta data swap is quickest way to move billions of rows assuming you can get a schema lock. 29
  30. 30. Sliding Window Goals SPLIT and MERGE with empty partitions Use SWITCH to do meta-data swaps Minimize all physical data movement 30
  31. 31. Range Right - MERGE 31
  32. 32. Range Left - MERGE 32
  33. 33. Range Right - SPLIT 33
  34. 34. Range Left - SPLIT 34
  35. 35. Visual Sliding Window Example 35
  36. 36. Visual Sliding Window Example Partiton1 (EMPTY) Partiton2 (DATA) Partiton3 (DATA) Partiton4 (DATA) Partiton5 (DATA) FG1 FG2 FG3 FG4 FG5
  37. 37. Visual Sliding Window Example Partiton1 (EMPTY) Partiton2 (DATA) Staging   Table     (EMPTY) FG2 Partiton4 (DATA) Partiton5 (DATA) FG3 FG1 Partiton3 (DATA) FG4 FG5
  38. 38. Visual Sliding Window Example Partiton1 (EMPTY) FG1 Partiton2 (EMPTY) Staging   Table    (DATA) FG2 Partiton4 (DATA) Partiton5 (DATA) FG3 SWAP Partiton3 (DATA) FG4 FG5
  39. 39. Visual Sliding Window Example Partiton1 (EMPTY) MERGE Partiton2 (EMPTY) Staging   Table    (DATA) FG2 Partiton4 (DATA) Partiton5 (DATA) FG3 FG1 Partiton3 (DATA) FG4 FG5
  40. 40. Visual Sliding Window Example Partiton1 (EMPTY) Partiton2 (DATA) Partiton3 (DATA) Partiton4 (DATA) FG1 FG3 FG4 FG5 Staging   Table    (DATA) FG2
  41. 41. Visual Sliding Window Example Partiton1 (EMPTY) Partiton2 (DATA) Partiton3 (DATA) Partiton4 (DATA) FG1 FG3 FG4 FG5 NEXT  USED Staging   Table     (EMPTY) FG2
  42. 42. Visual Sliding Window Example Partiton1 (EMPTY) Partiton2 (DATA) Partiton3 (DATA) Partiton4 (DATA) Partiton5 (DATA) FG1 FG3 FG4 FG5 FG2
  43. 43. Sliding Window Steps... Create partition swap table (if it doesn’t exist) 1.  Insert partition swap meta data 2.  Create staging table 3.  Meta-data swap (partition 2 with staging table) 4.  Merge Partitions #1 and #2 5.  Mark next used partition 6.  Split to create new partition 7.  Update processed partition swap meta data. 43
  44. 44. Partitioning DEMO October 15-18, 2013 | Charlotte, NC
  45. 45. Questions… http://johnsterrett.com/go/partition Blog: http://JohnSterrett.com/ Twitter: @JohnSterrett LinkedIn: http://linkedin.com/in/johnsterrett 45
  46. 46. Thank you for attending this session and the 2013 PASS Summit in Charlotte, NC 46 October 15-18, 2013 | Charlotte, NC

×