SlideShare a Scribd company logo
1 of 16
Hash Join
Background
It is very common that people says hash join is good for joining huge table while nested loop is good for small
table. I will not give any comment regarding that statement, but to me, the fundamental and the main
different between hash join and nested loop is that in nested loop we can look-up the inner table using any
value from the outer table (and get benefit of index access if any) while in hash join we cannot do that. The
only possibility to look-up to the inner table (probe table in term of hash join) is by using constant value.
In below example, actually we can divide the query into 2 separate queries.
SELECT /*+ leading(a) use_hash(b) */ a.*, b.*
FROM tbuild a, tprobe b
WHERE a.id = b.id;

Query 1:
SELECT a.* FROM tbuild;

Query 2:
SELECT b.* FROM tprobe;

So, even though we have index on ID column in those 2 tables, Oracle won’t be able to use that index access
(instead of full table scan).
In general, there are only 2 major steps in performing hash join:
1. Build hash table from 1 table based on pre-defined hash function
2. Probe the other table to get the result
Based on how Oracle do those 2 steps , there are 3 types of hash join:
1. Optimal
2. Onepass
3. Multipass
The objective of this article is to see the different between those 3 types along with few other scenarios to see
how Oracle handle it.
During the build phase, Oracle will create in-memory hash buckets (we may call it as hash table). The number
of hash bucket should be more than enough to avoid hash collision. Technically the hash buckets are split into
several partitions. In every partition there are several slots and those slots will be having several blocks. If I can
give analogy, it is very close to partition table. Apart from that, there is bitmap structure that maintain the slot
usage.
In-Memory Hash Table

Table Segment

Hash partition

Table partition

Every partition consists several slots

Every segment consists several extents

Every slot consists several blocks

Every extent consists several blocks

Start the Exercise
In this first 3 exercises I will use below attached script (create_tables.txt) to build the required tables.
Each table has 10,000 unique rows. To get the details information on hash join operation, we need to turn on
event 10104. Beside that we need to change ”workarea_size_policy” to MANUAL and we need to
configure the ”hash_area_size” to create the required scenarios (ibegin.sql).

create_tables.txt

ibegin.sql

The query for this exercise is:
SELECT /*+ leading(a) */ a.*, b.*
FROMtbuild a, tprobe b
WHERE a.id=b.id;
From above execution plan, the join will produces 10,000 rows and requires around 2 MB memory for hash
join. The size of hash join can be calculated as below. The query selects all columns in those 2 tables.Total size
of the columns are 201 + 4 = 205. We have 10,000 rows in the table, so in total we require 205 * 10,000 / 1,024
= 2,000 kB.

Hash Join - Optimal
workarea_size_policy = MANUAL
hash_area_size = 10485760 (10 MB)

Optimal hash join is the best type, where Oracle doesn’t requires any temporary space to store the hash
bucket. Everything is done in the memory since there is enough memory to do that.
Let’s analyze the statistics which we can see in above section of trace file.
In the first 3 lines after “Join Type” we can see the size of memory:
o Hash Area is 10,304,443
o

Slot Table is 9,478,144
This value is calculated later as 13 * 712 * 1,024, where:
 13 is number of slot
 712kB is size of each slot
 1,024 of course size of 1 kB

o

Overhead is 826,299
This is calculated as Hash Area – Slot Table
Number of slot/ cluster is 13
Number of partition is 8
Number of block in every slot is 89
Block size is 8 kB
Slot size is 89 * 8 kB = 712 kB
Bitmap size for each partition is 64 kB
Bitmap for all partitions is 8 * 64 kB
Size of row is 220
The size for overhead is approximately 15 bytes in this table (220 – 205).
Next in below section, the Build phase is started and since we have the optimal hash join, we cannot see any
operation in the temporary tablespace.

Later is the Probe phase. Let’s check few statistics from the trace file:
All partitions (8) are fitted and available in memory
Only 8 out of 13 slots are used
In the “Partition Distribution” we can see:
I take 1 line as example
o

Number of rows across all partitions

o

How many number of cluster/ slot in each partition

o
o

How many number of slot is available in the memory for each partition
The status,an indication whether the partition is still in memory or not
 If all partitions have kept=1  optimal hash join
 If all partitions have kept=0 multipass hash join
 If at least 1 partition has kept=1 onepass hash join

The number of bucket is 16,384
This value is 2^14 which is the closest one to 10,000  with this value, we are sure that hash collision
will not happened

Partitions
distribution

Below part shows the histogram of number of rows inside the hash bucket.
The last part is the overall statistics information which are quite self-explanation. The most interested part is
the first line after the title. From 16,384 available buckets, Oracle only use 7,538 buckets (the other 8,846
buckets are empty). That means there are some buckets that hold more than 1 row value (it can be saw in the
above histogram as well). So the hash function is efficient enough to manage/ address more than 1 row into
the bucket without any collision.
Number of rows

Lastly, below is the output of autotrace from SQL*Plus session along with few session statistics. We can see the
statistic for “workarea executions – optimal” is increasing by 3, where actually only 1 is relevant for above
query (the other scenarios also show the increment of 2 in this statistic as well, so we should consider only 1).
There is no temporary tablespace activity in this test case.

Hash Join - Onepass
workarea_size_policy = MANUAL
hash_area_size = 1572864 (1.5 MB)
In onepass type, Oracle needs to dump the data in the temporary tablespace due to insufficient hash area
memory, but when probing the second table, for each available partitions, Oracle only iterates ONCE. This is
why it is called onepass.

In the above capture, the important point is number of block in each slot, which is only 13 (compare to 89 in
the optimal type). It makes the slot size is 104 kB since there are 13 slots in this exercise. The reason why
Oracle reduce the number of block is to manage at least 1 slot in every partition is available in the memory.
The next section (Build phase) is quite interested, Oracle start spilling the data to temporary tablespace.

Once Build phase completed, Oracle start the Probe phase as below. Let’s highlight few points:
In first operation, only 6 slots are in memory
2 partitions are in memory
2,481 rows are processed
Above 3 items are also expressed in below capture
Number of bucket is 4,096
Again, this value is 2^12 (so we can conclude that the value is the closest power of 2)
All 13 slots are used

In the trace file output we see a lot of writing and reading, and we see new section in Probe phase like below
(HASH JOIN GET FLUSHED PARTITIONS). This is the process of reading back the build table and then continue
with probing the second table to get the result. Oracle will do this operation for rest of partitions, and since
the memory is not sufficient to do the operation in one shoot, Oracle will iterates the operation.
We can see clearly in below partition, 1,224 rows are being processed from build and probe table (which is
Partition: 0 if we trace back to the initial step of Probe phase), and at the end of iteration, the number of rows
left to be iterated over is 0.
These is the list of all iterations in this test case. Not sure why Oracle didn’t do the operation in ordered
fashion (from Partition 0 to 5 or from Partition 5 down to 0).

This is the overall statistics for onepass type (not all rows are showed).
Below is the output of autotrace from SQL*Plus session along with few session statistics. We can see the
statistic for “workarea executions – onepass” is increasing by 1 (we can ignore the increment of “workarea
executions – optimal” as I mentioned before). There is balanced activity between read and write against
temporary tablespace, which means Oracle only write once to temporary tablespaceand read once from
temporary tablespace.

Hash Join - Multipass
workarea_size_policy = MANUAL
hash_area_size = 131072 (128 kB)

In multipass type, Oracle also needs to dump the data in the temporary tablespace due to insufficient hash
area memory, but when probing the second table, for each available partitions, Oracle will iterates SEVERAL
times. This is why it is called multipass. This is the least efficient type of hash join.
In above output, every slot has single block only, so the size of slot is 8 kB. The total memory for slot table is 14
* 8 * 1024 = 114,688. The Build probe is getting longer since the size of slot is less (for efficiency please go to
the attached trace file if you want to know how long the “writing” activity of Build phase .

This is few capture of Probe phase for multipass type.

Again power of 2
Below is the details iteration for Partition 0. There are 4 iterations for processing 1,224 rows in this partition.
Theoretically if we want to change the type to onepass operation, we need to multiply “hash_area_size” by 4,
so in this case we need to configure at least 131,072 * 4 = 524,288 (512 kB).
Below is the output of autotrace from SQL*Plus session along with few session statistics. We can see the
statistic for “workarea executions – multipass” is increasing by 1 (we can ignore the increment of “workarea
executions – optimal” as I mentioned before). There is imbalance activity between read and write against
temporary tablespace. Oracle does the read part more compare to write part.

Which One Should Be The Build Table
Now let’s create another test case which will shows us the impact of build table’s size to the performance and
memory size. The configuration will be like below:
Create 2 tables, one table with 10,000 rows (TBIG) and the other with 2,500 rows (TSMALL). Both the
tables has 100 distinct values
Set “workarea_size_policy” to MANUAL
Set “hash_area_size” to 3 MB
Create 2 scenarios, first scenario will uses TBIG as build table and the second will uses TSMALL as build
table
The complete table creation script is attached

create_tables2.txt

The trace files output are attached
These are the execution plan for both queries. The consistent gets is bigger when we use TSMALL as build
table.(unfortunately I didn’t turn on event 10200 so I don’t know from where those consistent gets are
coming).The reason behind this symptom can be explained if we turn on event 10200 (for dumping consistent
gets). Please find below attached excel for the details of consistent gets, along with its trace files.

DBA series - Hash
Join.xlsx

Again, I attach below statistics from TBIG and TSMALL table. The next capture is summary of consistent gets
for both scenarios (TSMALL as “build” table and TBIG as “build” table)

TSMALL as “build” table

TBIG as “build” table

During Build phase, what Oracle reads all rows from “build” table. So in case of TBIG, Oracle requires 1,000
consistent gets (number of block in TBIG) and in case of TSMALL, Oracle requires 292 consistent gets (yet I
cannot explain the 78 different in this case)
The next Probe phase is more interesting,instead of loading all available blocks in“probe” table, it looks like
Oracle reads the second table row by row, 1 consistent get for single row. The result is 10,000 consistent gets
when we use TSMALL as “build” table (there are 10,000 rows in TBIG). In case of TBIG as “build” table, Oracle
requires 2,542 consistent gets (there are 2,500 rows in TSMALL  again I cannot explain the 42 different, but
during the test I filled-up the buffer by doing full table scan against TBIG and TSMALL, not sure if this was the
RC).
Apart from that, everything is similar.
Now let’s analyze the trace file to see the different from hash memory configuration and components.

From above comparison, we see Oracle works more efficient when the build table is small. We can see the
number of block in the slot is bigger and the number of bucket is smaller. It makes the overall memory
consumption is smaller when we have smaller build table.
Before we go to the conclusion that smaller build table is better than the bigger one, let’s retry the test case
with “workarea_size_policy” = AUTO which is default and recommended by Oracle.

Again we see that the memory consumption is better for smaller build table. But in this time, Oracle decided to
configure more blocks for each slot when bigger build table is used (this is in the reverse way if we compare to
previous test case when we set MANUAL for“workarea_size_policy”)

Conclusion
1. Multipass hash join is the most in-efficient type, and we can change it to, at least, onepass by multiply
the “hash_area_size“ by the number of iterations in one of hash partition.
2. Smaller table is always good as starting point for Build table until unless you see significant downgrade
in the performance. Small table will leads to smaller in-memory hash table.
2. Smaller table is not always good as “build” table, it depends ;-)
o It is good as “build”table as it requires smaller in-memory hash table to start the join
o In the other hand, it generates more consistent gets (again it depends of the size of Probe
table, the number of rows)
Saying this table is good as “build”table, or that table is not good (without confirmed by the number) is
not wise.

-heri-

More Related Content

What's hot

CS 542 -- Query Optimization
CS 542 -- Query OptimizationCS 542 -- Query Optimization
CS 542 -- Query OptimizationJ Singh
 
Query Optimization
Query OptimizationQuery Optimization
Query Optimizationrohitsalunke
 
Linked List, Types of Linked LIst, Various Operations, Applications of Linked...
Linked List, Types of Linked LIst, Various Operations, Applications of Linked...Linked List, Types of Linked LIst, Various Operations, Applications of Linked...
Linked List, Types of Linked LIst, Various Operations, Applications of Linked...Balwant Gorad
 
SQL: Query optimization in practice
SQL: Query optimization in practiceSQL: Query optimization in practice
SQL: Query optimization in practiceJano Suchal
 
Query Optimization - Brandon Latronica
Query Optimization - Brandon LatronicaQuery Optimization - Brandon Latronica
Query Optimization - Brandon Latronica"FENG "GEORGE"" YU
 
Hashing and Hashtable, application of hashing, advantages of hashing, disadva...
Hashing and Hashtable, application of hashing, advantages of hashing, disadva...Hashing and Hashtable, application of hashing, advantages of hashing, disadva...
Hashing and Hashtable, application of hashing, advantages of hashing, disadva...NaveenPeter8
 
14. Query Optimization in DBMS
14. Query Optimization in DBMS14. Query Optimization in DBMS
14. Query Optimization in DBMSkoolkampus
 
Overview of query evaluation
Overview of query evaluationOverview of query evaluation
Overview of query evaluationavniS
 
11. Hashing - Data Structures using C++ by Varsha Patil
11. Hashing - Data Structures using C++ by Varsha Patil11. Hashing - Data Structures using C++ by Varsha Patil
11. Hashing - Data Structures using C++ by Varsha Patilwidespreadpromotion
 
Cost estimation for Query Optimization
Cost estimation for Query OptimizationCost estimation for Query Optimization
Cost estimation for Query OptimizationRavinder Kamboj
 
Deletion from single way linked list and search
Deletion from single way linked list and searchDeletion from single way linked list and search
Deletion from single way linked list and searchEstiak Khan
 
Assignment on different types of addressing modes
Assignment on different types of addressing modesAssignment on different types of addressing modes
Assignment on different types of addressing modesNusratJahan263
 
Array and Pointers
Array and PointersArray and Pointers
Array and PointersProf Ansari
 

What's hot (20)

Data Structure
Data StructureData Structure
Data Structure
 
Hashing_UNIT2.pptx
Hashing_UNIT2.pptxHashing_UNIT2.pptx
Hashing_UNIT2.pptx
 
CS 542 -- Query Optimization
CS 542 -- Query OptimizationCS 542 -- Query Optimization
CS 542 -- Query Optimization
 
linear probing
linear probinglinear probing
linear probing
 
Chapter 12 ds
Chapter 12 dsChapter 12 ds
Chapter 12 ds
 
Query Optimization
Query OptimizationQuery Optimization
Query Optimization
 
single linked list
single linked listsingle linked list
single linked list
 
Linked List, Types of Linked LIst, Various Operations, Applications of Linked...
Linked List, Types of Linked LIst, Various Operations, Applications of Linked...Linked List, Types of Linked LIst, Various Operations, Applications of Linked...
Linked List, Types of Linked LIst, Various Operations, Applications of Linked...
 
Indexing Data Structure
Indexing Data StructureIndexing Data Structure
Indexing Data Structure
 
SQL: Query optimization in practice
SQL: Query optimization in practiceSQL: Query optimization in practice
SQL: Query optimization in practice
 
Query Optimization - Brandon Latronica
Query Optimization - Brandon LatronicaQuery Optimization - Brandon Latronica
Query Optimization - Brandon Latronica
 
Hashing and Hashtable, application of hashing, advantages of hashing, disadva...
Hashing and Hashtable, application of hashing, advantages of hashing, disadva...Hashing and Hashtable, application of hashing, advantages of hashing, disadva...
Hashing and Hashtable, application of hashing, advantages of hashing, disadva...
 
14. Query Optimization in DBMS
14. Query Optimization in DBMS14. Query Optimization in DBMS
14. Query Optimization in DBMS
 
Overview of query evaluation
Overview of query evaluationOverview of query evaluation
Overview of query evaluation
 
11. Hashing - Data Structures using C++ by Varsha Patil
11. Hashing - Data Structures using C++ by Varsha Patil11. Hashing - Data Structures using C++ by Varsha Patil
11. Hashing - Data Structures using C++ by Varsha Patil
 
Cost estimation for Query Optimization
Cost estimation for Query OptimizationCost estimation for Query Optimization
Cost estimation for Query Optimization
 
Hashing
HashingHashing
Hashing
 
Deletion from single way linked list and search
Deletion from single way linked list and searchDeletion from single way linked list and search
Deletion from single way linked list and search
 
Assignment on different types of addressing modes
Assignment on different types of addressing modesAssignment on different types of addressing modes
Assignment on different types of addressing modes
 
Array and Pointers
Array and PointersArray and Pointers
Array and Pointers
 

Viewers also liked

Something about oracle joins
Something about oracle joinsSomething about oracle joins
Something about oracle joinsmysqlops
 
Corba and-java
Corba and-javaCorba and-java
Corba and-javaafreen58
 
Rmi, corba and java beans
Rmi, corba and java beansRmi, corba and java beans
Rmi, corba and java beansRaghu nath
 
Database , 8 Query Optimization
Database , 8 Query OptimizationDatabase , 8 Query Optimization
Database , 8 Query OptimizationAli Usman
 
7. Relational Database Design in DBMS
7. Relational Database Design in DBMS7. Relational Database Design in DBMS
7. Relational Database Design in DBMSkoolkampus
 
13. Query Processing in DBMS
13. Query Processing in DBMS13. Query Processing in DBMS
13. Query Processing in DBMSkoolkampus
 

Viewers also liked (12)

Gpu Join Presentation
Gpu Join PresentationGpu Join Presentation
Gpu Join Presentation
 
Something about oracle joins
Something about oracle joinsSomething about oracle joins
Something about oracle joins
 
Corba and-java
Corba and-javaCorba and-java
Corba and-java
 
Rmi, corba and java beans
Rmi, corba and java beansRmi, corba and java beans
Rmi, corba and java beans
 
Database , 8 Query Optimization
Database , 8 Query OptimizationDatabase , 8 Query Optimization
Database , 8 Query Optimization
 
Oracle: Joins
Oracle: JoinsOracle: Joins
Oracle: Joins
 
Algorithm.ppt
Algorithm.pptAlgorithm.ppt
Algorithm.ppt
 
7. Relational Database Design in DBMS
7. Relational Database Design in DBMS7. Relational Database Design in DBMS
7. Relational Database Design in DBMS
 
13. Query Processing in DBMS
13. Query Processing in DBMS13. Query Processing in DBMS
13. Query Processing in DBMS
 
Webinar Smile et WSO2
Webinar Smile et WSO2Webinar Smile et WSO2
Webinar Smile et WSO2
 
Ejb
Ejb Ejb
Ejb
 
Slideshare ppt
Slideshare pptSlideshare ppt
Slideshare ppt
 

Similar to Hash join

Myth busters - performance tuning 103 2008
Myth busters - performance tuning 103 2008Myth busters - performance tuning 103 2008
Myth busters - performance tuning 103 2008paulguerin
 
Application sql issues_and_tuning
Application sql issues_and_tuningApplication sql issues_and_tuning
Application sql issues_and_tuningAnil Pandey
 
Oracle Join Methods and 12c Adaptive Plans
Oracle Join Methods and 12c Adaptive PlansOracle Join Methods and 12c Adaptive Plans
Oracle Join Methods and 12c Adaptive PlansFranck Pachot
 
CBO choice between Index and Full Scan: the good, the bad and the ugly param...
CBO choice between Index and Full Scan:  the good, the bad and the ugly param...CBO choice between Index and Full Scan:  the good, the bad and the ugly param...
CBO choice between Index and Full Scan: the good, the bad and the ugly param...Franck Pachot
 
Db2 performance tuning for dummies
Db2 performance tuning for dummiesDb2 performance tuning for dummies
Db2 performance tuning for dummiesAngel Dueñas Neyra
 
Applyinga blockcentricapproach
Applyinga blockcentricapproachApplyinga blockcentricapproach
Applyinga blockcentricapproachoracle documents
 
Database Performance Tuning
Database Performance Tuning Database Performance Tuning
Database Performance Tuning Arno Huetter
 
Sql server scalability fundamentals
Sql server scalability fundamentalsSql server scalability fundamentals
Sql server scalability fundamentalsChris Adkin
 
あなたのScalaを爆速にする7つの方法
あなたのScalaを爆速にする7つの方法あなたのScalaを爆速にする7つの方法
あなたのScalaを爆速にする7つの方法x1 ichi
 
Hive query optimization infinity
Hive query optimization infinityHive query optimization infinity
Hive query optimization infinityShashwat Shriparv
 
Reliable Windows Heap Exploits
Reliable Windows Heap ExploitsReliable Windows Heap Exploits
Reliable Windows Heap Exploitsamiable_indian
 
Myth busters - performance tuning 102 2008
Myth busters - performance tuning 102 2008Myth busters - performance tuning 102 2008
Myth busters - performance tuning 102 2008paulguerin
 

Similar to Hash join (20)

Myth busters - performance tuning 103 2008
Myth busters - performance tuning 103 2008Myth busters - performance tuning 103 2008
Myth busters - performance tuning 103 2008
 
Application sql issues_and_tuning
Application sql issues_and_tuningApplication sql issues_and_tuning
Application sql issues_and_tuning
 
Database Sizing
Database SizingDatabase Sizing
Database Sizing
 
Nested loop join technique - part2
Nested loop join technique - part2Nested loop join technique - part2
Nested loop join technique - part2
 
Oracle Join Methods and 12c Adaptive Plans
Oracle Join Methods and 12c Adaptive PlansOracle Join Methods and 12c Adaptive Plans
Oracle Join Methods and 12c Adaptive Plans
 
CBO choice between Index and Full Scan: the good, the bad and the ugly param...
CBO choice between Index and Full Scan:  the good, the bad and the ugly param...CBO choice between Index and Full Scan:  the good, the bad and the ugly param...
CBO choice between Index and Full Scan: the good, the bad and the ugly param...
 
Db2 performance tuning for dummies
Db2 performance tuning for dummiesDb2 performance tuning for dummies
Db2 performance tuning for dummies
 
Applyinga blockcentricapproach
Applyinga blockcentricapproachApplyinga blockcentricapproach
Applyinga blockcentricapproach
 
Interview Preparation
Interview PreparationInterview Preparation
Interview Preparation
 
Migration
MigrationMigration
Migration
 
Database Performance Tuning
Database Performance Tuning Database Performance Tuning
Database Performance Tuning
 
15 Jo P Mar 08
15 Jo P Mar 0815 Jo P Mar 08
15 Jo P Mar 08
 
Oracle NOLOGGING
Oracle NOLOGGINGOracle NOLOGGING
Oracle NOLOGGING
 
Extentnumbers
ExtentnumbersExtentnumbers
Extentnumbers
 
Sql server scalability fundamentals
Sql server scalability fundamentalsSql server scalability fundamentals
Sql server scalability fundamentals
 
あなたのScalaを爆速にする7つの方法
あなたのScalaを爆速にする7つの方法あなたのScalaを爆速にする7つの方法
あなたのScalaを爆速にする7つの方法
 
Hive query optimization infinity
Hive query optimization infinityHive query optimization infinity
Hive query optimization infinity
 
Reliable Windows Heap Exploits
Reliable Windows Heap ExploitsReliable Windows Heap Exploits
Reliable Windows Heap Exploits
 
8 tune tusc
8 tune tusc8 tune tusc
8 tune tusc
 
Myth busters - performance tuning 102 2008
Myth busters - performance tuning 102 2008Myth busters - performance tuning 102 2008
Myth busters - performance tuning 102 2008
 

More from Heribertus Bramundito

More from Heribertus Bramundito (9)

Subquery factoring for FTS
Subquery factoring for FTSSubquery factoring for FTS
Subquery factoring for FTS
 
Few useful features
Few useful featuresFew useful features
Few useful features
 
MV sql profile and index
MV sql profile and indexMV sql profile and index
MV sql profile and index
 
The internals
The internalsThe internals
The internals
 
10053 - null is not nothing
10053 - null is not nothing10053 - null is not nothing
10053 - null is not nothing
 
Not in vs not exists
Not in vs not existsNot in vs not exists
Not in vs not exists
 
Introduction to oracle optimizer
Introduction to oracle optimizerIntroduction to oracle optimizer
Introduction to oracle optimizer
 
Correlated update vs merge
Correlated update vs mergeCorrelated update vs merge
Correlated update vs merge
 
Checking clustering factor to detect row migration
Checking clustering factor to detect row migrationChecking clustering factor to detect row migration
Checking clustering factor to detect row migration
 

Recently uploaded

Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhisoniya singh
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 

Recently uploaded (20)

Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 

Hash join

  • 1. Hash Join Background It is very common that people says hash join is good for joining huge table while nested loop is good for small table. I will not give any comment regarding that statement, but to me, the fundamental and the main different between hash join and nested loop is that in nested loop we can look-up the inner table using any value from the outer table (and get benefit of index access if any) while in hash join we cannot do that. The only possibility to look-up to the inner table (probe table in term of hash join) is by using constant value. In below example, actually we can divide the query into 2 separate queries. SELECT /*+ leading(a) use_hash(b) */ a.*, b.* FROM tbuild a, tprobe b WHERE a.id = b.id; Query 1: SELECT a.* FROM tbuild; Query 2: SELECT b.* FROM tprobe; So, even though we have index on ID column in those 2 tables, Oracle won’t be able to use that index access (instead of full table scan). In general, there are only 2 major steps in performing hash join: 1. Build hash table from 1 table based on pre-defined hash function 2. Probe the other table to get the result Based on how Oracle do those 2 steps , there are 3 types of hash join: 1. Optimal 2. Onepass 3. Multipass The objective of this article is to see the different between those 3 types along with few other scenarios to see how Oracle handle it. During the build phase, Oracle will create in-memory hash buckets (we may call it as hash table). The number of hash bucket should be more than enough to avoid hash collision. Technically the hash buckets are split into several partitions. In every partition there are several slots and those slots will be having several blocks. If I can give analogy, it is very close to partition table. Apart from that, there is bitmap structure that maintain the slot usage.
  • 2. In-Memory Hash Table Table Segment Hash partition Table partition Every partition consists several slots Every segment consists several extents Every slot consists several blocks Every extent consists several blocks Start the Exercise In this first 3 exercises I will use below attached script (create_tables.txt) to build the required tables. Each table has 10,000 unique rows. To get the details information on hash join operation, we need to turn on event 10104. Beside that we need to change ”workarea_size_policy” to MANUAL and we need to configure the ”hash_area_size” to create the required scenarios (ibegin.sql). create_tables.txt ibegin.sql The query for this exercise is: SELECT /*+ leading(a) */ a.*, b.* FROMtbuild a, tprobe b WHERE a.id=b.id;
  • 3. From above execution plan, the join will produces 10,000 rows and requires around 2 MB memory for hash join. The size of hash join can be calculated as below. The query selects all columns in those 2 tables.Total size of the columns are 201 + 4 = 205. We have 10,000 rows in the table, so in total we require 205 * 10,000 / 1,024 = 2,000 kB. Hash Join - Optimal workarea_size_policy = MANUAL hash_area_size = 10485760 (10 MB) Optimal hash join is the best type, where Oracle doesn’t requires any temporary space to store the hash bucket. Everything is done in the memory since there is enough memory to do that.
  • 4. Let’s analyze the statistics which we can see in above section of trace file. In the first 3 lines after “Join Type” we can see the size of memory: o Hash Area is 10,304,443 o Slot Table is 9,478,144 This value is calculated later as 13 * 712 * 1,024, where:  13 is number of slot  712kB is size of each slot  1,024 of course size of 1 kB o Overhead is 826,299 This is calculated as Hash Area – Slot Table Number of slot/ cluster is 13 Number of partition is 8 Number of block in every slot is 89 Block size is 8 kB Slot size is 89 * 8 kB = 712 kB Bitmap size for each partition is 64 kB Bitmap for all partitions is 8 * 64 kB Size of row is 220 The size for overhead is approximately 15 bytes in this table (220 – 205). Next in below section, the Build phase is started and since we have the optimal hash join, we cannot see any operation in the temporary tablespace. Later is the Probe phase. Let’s check few statistics from the trace file: All partitions (8) are fitted and available in memory Only 8 out of 13 slots are used In the “Partition Distribution” we can see: I take 1 line as example
  • 5. o Number of rows across all partitions o How many number of cluster/ slot in each partition o o How many number of slot is available in the memory for each partition The status,an indication whether the partition is still in memory or not  If all partitions have kept=1  optimal hash join  If all partitions have kept=0 multipass hash join  If at least 1 partition has kept=1 onepass hash join The number of bucket is 16,384 This value is 2^14 which is the closest one to 10,000  with this value, we are sure that hash collision will not happened Partitions distribution Below part shows the histogram of number of rows inside the hash bucket.
  • 6. The last part is the overall statistics information which are quite self-explanation. The most interested part is the first line after the title. From 16,384 available buckets, Oracle only use 7,538 buckets (the other 8,846 buckets are empty). That means there are some buckets that hold more than 1 row value (it can be saw in the above histogram as well). So the hash function is efficient enough to manage/ address more than 1 row into the bucket without any collision. Number of rows Lastly, below is the output of autotrace from SQL*Plus session along with few session statistics. We can see the statistic for “workarea executions – optimal” is increasing by 3, where actually only 1 is relevant for above query (the other scenarios also show the increment of 2 in this statistic as well, so we should consider only 1). There is no temporary tablespace activity in this test case. Hash Join - Onepass workarea_size_policy = MANUAL hash_area_size = 1572864 (1.5 MB)
  • 7. In onepass type, Oracle needs to dump the data in the temporary tablespace due to insufficient hash area memory, but when probing the second table, for each available partitions, Oracle only iterates ONCE. This is why it is called onepass. In the above capture, the important point is number of block in each slot, which is only 13 (compare to 89 in the optimal type). It makes the slot size is 104 kB since there are 13 slots in this exercise. The reason why Oracle reduce the number of block is to manage at least 1 slot in every partition is available in the memory. The next section (Build phase) is quite interested, Oracle start spilling the data to temporary tablespace. Once Build phase completed, Oracle start the Probe phase as below. Let’s highlight few points: In first operation, only 6 slots are in memory 2 partitions are in memory 2,481 rows are processed Above 3 items are also expressed in below capture
  • 8. Number of bucket is 4,096 Again, this value is 2^12 (so we can conclude that the value is the closest power of 2) All 13 slots are used In the trace file output we see a lot of writing and reading, and we see new section in Probe phase like below (HASH JOIN GET FLUSHED PARTITIONS). This is the process of reading back the build table and then continue with probing the second table to get the result. Oracle will do this operation for rest of partitions, and since the memory is not sufficient to do the operation in one shoot, Oracle will iterates the operation. We can see clearly in below partition, 1,224 rows are being processed from build and probe table (which is Partition: 0 if we trace back to the initial step of Probe phase), and at the end of iteration, the number of rows left to be iterated over is 0.
  • 9. These is the list of all iterations in this test case. Not sure why Oracle didn’t do the operation in ordered fashion (from Partition 0 to 5 or from Partition 5 down to 0). This is the overall statistics for onepass type (not all rows are showed).
  • 10. Below is the output of autotrace from SQL*Plus session along with few session statistics. We can see the statistic for “workarea executions – onepass” is increasing by 1 (we can ignore the increment of “workarea executions – optimal” as I mentioned before). There is balanced activity between read and write against temporary tablespace, which means Oracle only write once to temporary tablespaceand read once from temporary tablespace. Hash Join - Multipass workarea_size_policy = MANUAL hash_area_size = 131072 (128 kB) In multipass type, Oracle also needs to dump the data in the temporary tablespace due to insufficient hash area memory, but when probing the second table, for each available partitions, Oracle will iterates SEVERAL times. This is why it is called multipass. This is the least efficient type of hash join.
  • 11. In above output, every slot has single block only, so the size of slot is 8 kB. The total memory for slot table is 14 * 8 * 1024 = 114,688. The Build probe is getting longer since the size of slot is less (for efficiency please go to the attached trace file if you want to know how long the “writing” activity of Build phase . This is few capture of Probe phase for multipass type. Again power of 2
  • 12. Below is the details iteration for Partition 0. There are 4 iterations for processing 1,224 rows in this partition. Theoretically if we want to change the type to onepass operation, we need to multiply “hash_area_size” by 4, so in this case we need to configure at least 131,072 * 4 = 524,288 (512 kB).
  • 13. Below is the output of autotrace from SQL*Plus session along with few session statistics. We can see the statistic for “workarea executions – multipass” is increasing by 1 (we can ignore the increment of “workarea executions – optimal” as I mentioned before). There is imbalance activity between read and write against temporary tablespace. Oracle does the read part more compare to write part. Which One Should Be The Build Table Now let’s create another test case which will shows us the impact of build table’s size to the performance and memory size. The configuration will be like below: Create 2 tables, one table with 10,000 rows (TBIG) and the other with 2,500 rows (TSMALL). Both the tables has 100 distinct values Set “workarea_size_policy” to MANUAL Set “hash_area_size” to 3 MB Create 2 scenarios, first scenario will uses TBIG as build table and the second will uses TSMALL as build table The complete table creation script is attached create_tables2.txt The trace files output are attached
  • 14. These are the execution plan for both queries. The consistent gets is bigger when we use TSMALL as build table.(unfortunately I didn’t turn on event 10200 so I don’t know from where those consistent gets are coming).The reason behind this symptom can be explained if we turn on event 10200 (for dumping consistent gets). Please find below attached excel for the details of consistent gets, along with its trace files. DBA series - Hash Join.xlsx Again, I attach below statistics from TBIG and TSMALL table. The next capture is summary of consistent gets for both scenarios (TSMALL as “build” table and TBIG as “build” table) TSMALL as “build” table TBIG as “build” table During Build phase, what Oracle reads all rows from “build” table. So in case of TBIG, Oracle requires 1,000 consistent gets (number of block in TBIG) and in case of TSMALL, Oracle requires 292 consistent gets (yet I cannot explain the 78 different in this case) The next Probe phase is more interesting,instead of loading all available blocks in“probe” table, it looks like Oracle reads the second table row by row, 1 consistent get for single row. The result is 10,000 consistent gets when we use TSMALL as “build” table (there are 10,000 rows in TBIG). In case of TBIG as “build” table, Oracle requires 2,542 consistent gets (there are 2,500 rows in TSMALL  again I cannot explain the 42 different, but during the test I filled-up the buffer by doing full table scan against TBIG and TSMALL, not sure if this was the RC). Apart from that, everything is similar.
  • 15. Now let’s analyze the trace file to see the different from hash memory configuration and components. From above comparison, we see Oracle works more efficient when the build table is small. We can see the number of block in the slot is bigger and the number of bucket is smaller. It makes the overall memory consumption is smaller when we have smaller build table.
  • 16. Before we go to the conclusion that smaller build table is better than the bigger one, let’s retry the test case with “workarea_size_policy” = AUTO which is default and recommended by Oracle. Again we see that the memory consumption is better for smaller build table. But in this time, Oracle decided to configure more blocks for each slot when bigger build table is used (this is in the reverse way if we compare to previous test case when we set MANUAL for“workarea_size_policy”) Conclusion 1. Multipass hash join is the most in-efficient type, and we can change it to, at least, onepass by multiply the “hash_area_size“ by the number of iterations in one of hash partition. 2. Smaller table is always good as starting point for Build table until unless you see significant downgrade in the performance. Small table will leads to smaller in-memory hash table. 2. Smaller table is not always good as “build” table, it depends ;-) o It is good as “build”table as it requires smaller in-memory hash table to start the join o In the other hand, it generates more consistent gets (again it depends of the size of Probe table, the number of rows) Saying this table is good as “build”table, or that table is not good (without confirmed by the number) is not wise. -heri-