Data Compression for Large Multidimensional Data Warehouses<br />Supervisor:<br />Presented by:<br />Dr. K.M. Azharul Hasa...
Presentation Layout<br /><ul><li> Objectives
 Existing Compression Schemes
 Traditional Extendible Array
 Proposed Compression Scheme
 EXCS</li></ul>   (Extendible Array Based Compression Scheme)<br /><ul><li>Comparative Analysis
Conclusion</li></ul>2<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
<ul><li>Data compression technology reduces:
effective price of logical data storage capacity
improves query performance
 Multidimensional array is widely used in large number of scientific research.
 An efficient compression of multidimensional array can handle large multidimensional data sets of data warehouses</li></u...
Existing Compression Schemes   (1/ 3)<br /><ul><li> Bitmap compression
 Run Length Encoding
 Header compression
 Compressed Column Storage
 Compressed Row Storage</li></ul>4<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
Existing Compression Schemes   (2/ 3)<br />5<br />(a) A sparse array.       (b) The CRS scheme<br />Abdullah Al Mahmud, St...
Upcoming SlideShare
Loading in …5
×

Data compression for Large Multidimensional Data Warehouses

1,617 views

Published on

This presentation is prepared for the presentation of thesis titled as "Data compression for Large Multidimensional Data Warehouses" which was done for the partial fulfillment of the undergrad course in Dept of CSE, KUET, Bangladesh.

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,617
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
34
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Data compression for Large Multidimensional Data Warehouses

  1. 1. Data Compression for Large Multidimensional Data Warehouses<br />Supervisor:<br />Presented by:<br />Dr. K.M. Azharul Hasan<br />Associate Professor,<br />Head of the Department,<br />Department of CSE, KUET<br />Abdullah Al Mahmud,<br />Roll : 0507006<br />Md. Mushfiqur Rahman, <br />Roll : 0507029 <br />1<br />This slide is prepared by Abdullah Al Mahmud for the presentation of Thesis which was done as the partial fulfillment of degree of in undergrad course in Khulna University of Engineering & Technology(KUET), Bangladesh<br />
  2. 2. Presentation Layout<br /><ul><li> Objectives
  3. 3. Existing Compression Schemes
  4. 4. Traditional Extendible Array
  5. 5. Proposed Compression Scheme
  6. 6. EXCS</li></ul> (Extendible Array Based Compression Scheme)<br /><ul><li>Comparative Analysis
  7. 7. Conclusion</li></ul>2<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  8. 8. <ul><li>Data compression technology reduces:
  9. 9. effective price of logical data storage capacity
  10. 10. improves query performance
  11. 11. Multidimensional array is widely used in large number of scientific research.
  12. 12. An efficient compression of multidimensional array can handle large multidimensional data sets of data warehouses</li></ul>3<br />Objectives<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  13. 13. Existing Compression Schemes (1/ 3)<br /><ul><li> Bitmap compression
  14. 14. Run Length Encoding
  15. 15. Header compression
  16. 16. Compressed Column Storage
  17. 17. Compressed Row Storage</li></ul>4<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  18. 18. Existing Compression Schemes (2/ 3)<br />5<br />(a) A sparse array. (b) The CRS scheme<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  19. 19. Existing Compression Schemes (3/ 3)<br /><ul><li>Classical methods cannot support updates without completely readjusting runs .
  20. 20. Compressing sparse array
  21. 21. Do not support extendibility</li></ul>6<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  22. 22. Traditional Extendible Array<br /><ul><li>TEA supports dynamic extension of dimension size.</li></ul>7<br />Position <1,3><br />H1[1]<H2[3]<br />Address of Cell=Address1[3]+1=10<br />0<br />History Counter=<br />0<br />1<br />2<br />3<br />4<br />5<br />Figure 1: TEA Construction And Access<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  23. 23. Proposed Compression Scheme<br /><ul><li>Multidimensional arrays are important for sparse array operations
  24. 24. Extendibility of multidimensional arrays
  25. 25. A compression technique that can work on multidimensional extendible array
  26. 26. Our proposed compression scheme is EXCS (Extendible array based Compression Scheme)</li></ul>8<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  27. 27. Extendible array based Compression Scheme (EXCS) 1/3<br /><ul><li>We implemented the multidimensional extendible array in secondary memory
  28. 28. We have considered dimension =3 in our experimental approach
  29. 29. The sub-arrays are distinguished to store them individually in the secondary memory</li></ul>9<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  30. 30. Extendible array based Compression Scheme (EXCS) 2/3<br /><ul><li>The sub-arrays are of n-1(=2) dimension
  31. 31. A large no. of sub-arrays are generated to be compressed
  32. 32. Sub-arrays are dynamically taken as input
  33. 33. Only the max no of sub-arrays is to be given</li></ul>10<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  34. 34. 11<br />Extendible array based Compression Scheme (EXCS) 3/3<br /><ul><li>Each sub-array is compressed individually
  35. 35. The compression technique used is similar to CRS
  36. 36. The compressed elements are written in the secondary memory as RO, CO, VL of subarray_1, subarray_2, … … subarray_N</li></ul>Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  37. 37. Performance Measurement <br /><ul><li>Performance is measured by measuring two key factors of the compression schemes:
  38. 38. Data Density
  39. 39. Length of Dimension/ Number of Data
  40. 40. compression ratio=</li></ul>(compressed data/ original data)<br /><ul><li> space savings = 1 – compression ratio
  41. 41. we have considered space savings in percent</li></ul>12<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  42. 42. Comparative Analysis (1/4)<br />13<br />No. of data <br /> Figure: Comparison with fixed density = 20%<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  43. 43. 14<br /> Comparative Analysis (2/4)<br />No. of data <br /> Figure: Comparison with fixed density = 25%<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  44. 44. Comparative Analysis (3/4)<br />15<br />Density of data <br />Figure: Comparison with fixed no. of data=64<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  45. 45. Comparative Analysis (4/4)<br />16<br />Density of data <br />Figure: Comparison with fixed no. of data=4096<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  46. 46. Performance Measurement <br /><ul><li> Extendibility of arrays
  47. 47. Using multidimensional arrays
  48. 48. Extendibility toward any dimension
  49. 49. EXCS allows dynamic extension of arrays.
  50. 50. In analysis, we can extend data up to n dimensions
  51. 51. Performance is good for large no. of data</li></ul>17<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />
  52. 52. Conclusion<br /><ul><li> Our proposed compression scheme is experimentally done up to 3 dimension data
  53. 53. It can be extended experimentally for compressing n dimension data in future.
  54. 54. EXCS is effective for large multidimensional data warehouses</li></ul>18<br />Abdullah Al Mahmud, Student ID: 0507006, Dept. of CSE, KUET, Bangladesh<br />

×