This document discusses the I/O costs of performing a DISTINCT operation on the Reserves relation using a two-phase hash-based approach. It analyzes the I/O costs for partitioning the data into buckets in the first phase and eliminating duplicates in the second phase under different buffer sizes. The total I/O cost is shown to be linear in the number of input and output pages.

1. Projection
)
(
,...,
, 2
1 R
m
attr
attr
attr

1

2. 2

3.  Tuple Length
◦ Reserves
 40 bytes
◦ Sailors
 50 bytes
 # of Tuples
◦ Reserves
 100,000
◦ Sailors
 40,000
 Page Size
◦ 4k
 Size on Disk
◦ Reserves ???
◦ Sailors ???
3

4.  SELECT DISTINCT R.sid, R.bid
FROM Reserves R
)
(
,...,
, 2
1 R
m
attr
attr
attr

Reserves
bid
sid,

4

5.  M:= # of pages of R
 T:= M*F
 Scan and Project (in Temp)
◦ O(M)+O(T)
 Sort (Temp)
◦ O (TlogT)
 2-Way Sort
 Scan (Temp) and Eliminate Duplicates
◦ O(T)
h
TupleLengt
attr
of
Size
F
m
i
i /
)
(
_
1




5

6.  sid+bid = 10 bytes
 Buffer Pages = 20
 Scan and Project (in Temp)
◦ 1000+250 = 1250
 Sort (Temp)
◦ Two passes
 Pass 0 (⌈250/20⌉ = 13 runs)
 Pass 1 (13 Way Merges)
◦ 2(2*250) = 1000
 Scan (Temp) and Eliminate Duplicates
◦ 250
Total I/Os:
1250
1000
250
2500
6

7.  Buffer Pages = B
 Pass 0
◦ Project Out Unwanted Attributes
◦ Read B Pages Write run of B*F Pages
 Runs of 2B pages with aggressive implementation
 Subsequent Passes
◦ Eliminate Duplicates while Merging
h
TupleLengt
attr
of
Size
F
m
i
i /
)
(
_
1




7

8. ◦ sid+bid = 10 bytes
◦ Buffer Pages = 20
 Pass 0
◦ 1000+250 = 1250
◦ 50 Reads of 20 Pages each
◦ 50 Writes of 5 Pages each
 Pass 1
◦ 250*2 = 500
◦ 19-Way Merges with Duplicates Elimination
◦ Two runs of 19*5 pages, One run of 60 pages
 Pass 2
◦ 250*1 = 250
◦ 3-Way Merge with Duplicates Elimination
Total I/Os:
1250
500
250
2000
8

9.  Using Aggressive Implementation
 Pass 0
◦ 1000+250 = 1250
◦ 6 runs of 40 pages each
◦ 1 run of 10 pages
 Pass 1
◦ 250*1 = 250
◦ 7-Way Merge with Duplicates Elimination
Total I/Os:
1250
250
1500
9

10.  B (buffer pages) is typically large
◦ f:= fudge factor
 2-Phase Process
◦ Phase 1
 Partitioning
 (After Projecting OUT Unwanted Attributes)
◦ Phase 2
 Duplicates Elimination
f
T
B .

10

11.  One input buffer page
 B −1 output buffer pages
 For each tuple
◦ Project out the unwanted attributes
◦ Apply a hash function h to the combination of all
remaining attributes
 h is chosen so that tuples are distributed
uniformly to one of B −1 partitions
11

12. 12
At the end of the partitioning phase,
we have B −1 partitions

13.  X and Y are identical records only if
◦ hash(X) == hash(Y)
 For each Partition (made in phase 1)
◦ Insert each Tuple into an in-memory Hash table
 Using a different Hash function
 Why different ???
◦ Compare Colliding Tuples for Duplicates
◦ Discard Duplicates when Found
13

14.  Phase 1: Partitioning
◦ Read M Pages
◦ Write T Pages
 Phase 2: Duplicates Elimination
◦ Read T Pages
 Total I/Os
◦ M+2T
14

15.  Index-Only Scan
15