This document discusses various data compression methods that can be used to reduce the size of databases and improve performance. It covers physical compression techniques like data compression, archiving, and using smaller data types. It also covers logical compression methods like partitioning data horizontally and vertically, creating covering indexes, and filtered indexes. The goal of these compression methods is to address challenges of storing and accessing large amounts of data by minimizing the data size and amount of data that needs to be processed for queries and other operations.

1. Smaller is Betterישי גרין| ראש צוות DBA| הבורסה לניירות ערך

2. Data Compression – Why bother?Physical Compression MethodsLogical Compression Methods

3. 3

4. 4

5. 5

6. 6

7. Data Compression – Whybother?The bigger the data – the bigger the challenges

8. Data Compression – Whybother?Every action takes longerData Access Queries

9. Loading data volumes

10. Archiving

11. Maintenance (backups, fragmentation, statistics, data replications…)

12. Storage constraintsPhysical Compression MethodsArchivingData compression (SQL 2008)Smaller data typesHorizontal PartitioningVertical Partitioning

13. Archiving The most efficient way to minimize data in OLTPGood: Archive data into a Archive Repository

14. Better: Archive data to flat files or other media (BCP out)

15. Best: Purge data. Retrieve data from backup ifneeded.Data compression (SQL 2008)Compression TypeData CompressionTable level

16. Index levelBackup CompressionCompression MethodsRow CompressionPage Compression

17. Data compression (SQL 2008)Row CompressionStoring fixed-length data types in variable-length storage format. A compressed row uses 4 bits per compressed column to store the length of the data in the column.

18. Data compression (SQL 2008)Row Compression13

19. Data compression (SQL 2008)PAGE Compressiona superset of row compression. optimizes storage of multiple rows in a page, by minimizing the data redundancy. Page compression uses prefix compression and dictionary compression. Prefix compression looks for common patterns in the beginning of the column values on a given column across all rows on each page.

20. Dictionary compression looks for exact value matches across all columns and rows on each page. 1414

21. Data compression (SQL 2008)PAGE Compression Prefix Compression Dictionary Compression15

22. Data compression (SQL 2008)Performance16

23. Data compression (SQL 2008)Performance17

24. Data compression (SQL 2008)Performance - Summary CPU IOBetter performance when IO is a bottleneck

25. Reduces storage needs significantly

26. Use advisory procedure to calculate compressionUsing Smallest Data Type Available19

27. Using Smallest Data Type AvailableExamples:Char (large fixed value)  Varchar (large value)

28. UniCode data type  NonUnicode

29. Datetime  SmallDateTime

30. Datetime  Date

31. Datetime  Time20

32. PartitioningDefinition: Breakup a table or index into smaller parts, that when stitched recreate the original object.Types of Partitioning:HorizontalVertical21

33. Horizontal PartitioningEach table maintains a similar schema, but contains fewer rows.

34. There is no data overlap between the partitions.

35. A logical union of partitions recreates the original logical table.22

36. Horizontal Partitioning

37. Horizontal PartitioningSlow storageMedium speed storageFast storage24

38. Horizontal Partitioning 3 Types of implementing Horizontal Partitions Partitioned Views

39. Partitioned Indexes

40. Partitioned TablesHorizontal Partitioning - until SQL 2005Partitioned viewYou can have differences between the tables the partitioned view references

41. The optimizer must consider each partition separately and duplicate the query's filter against the view for all partitions in the plan

42. Limits on update or delete operations 26

43. Native Horizontal PartitioningSQL 2005 and onOne Object!!!One table

44. One index

45. One PartitionFunction

46. One Partition SchemeVertical PartitioningDivide a logical table into multiple physical tables that contain fewer columns, but the same number of rows.

47. To Recreate the original table, join the partitioned tables on the primary key columns.Vertical Partitioning

48. Partitioning - Summaryuse Vertical Partitioning when some columns are more queried than others(columns is the Select list)

49. Use Horizontal Partitioning when part of the data is queried more than others(columns is the Where clause)DEMOLogical Compression Methods3131

50. Logical CompressionMethodsCovering IndexesFiltered IndexesSparse Columns Refactoring TSQL to retrieve less data

51. When it comes to dataSmaller is Better !!33