Christo Kutrovsky, profile picture
Uploaded byChristo Kutrovsky
PPTX, PDF715 views

Christo Kutrovsky - Maximize Data Warehouse Performance with Parallel Queries

The presentation discusses maximizing data warehouse performance using parallel queries, presented by Christo Kutrovsky of Pythian. Key topics include parallel query concepts, Exadata data flow capabilities, and performance metrics for complex IT environments. The document emphasizes Pythian's expertise and global reach in database administration and consulting services.

Embed presentation

Downloaded 12 times
Maximize Data Warehouse Performance with Parallel Queries Presented by: Christo Kutrovsky
© 2015 Pythian – Presentation for DOAG Who Am I 2 • Oracle ACE • 15 years in Oracle field • Joined Pythian 2003 • Part of Pythian Consulting Group • Special projects • Performance tuning • Critical services • Presenter at: IOUG, RMOUG, UKOUG, OpenWorld “oracle pinup”
© 2015 Pythian – Presentation for DOAG3 © 2011 Pythian - Confidential Why Companies Trust Pythian • Recognized Leader: • Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server • Work with over 150 multinational companies such as Forbes.com, Fox Sports, Nordion and Western Union to help manage their complex IT deployments • Expertise: • One of the world’s largest concentrations of dedicated, full-time DBA expertise. Employ 7 Oracle ACEs/ACE Directors. • Hold 7 Specializations under Oracle Platinum Partner program, including Oracle Exadata, Oracle GoldenGate & Oracle RAC. • Global Reach & Scalability: • 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response
© 2015 Pythian – Presentation for DOAG Agenda 4 • Review Parallel Query concepts • Review Exadata Data Flow capabilities • The Big Sort • The Big Group By • The Big Join • Final Thoughts
5 Parallel Query Concepts
© 2015 Pythian – Presentation for DOAG Oracle Parallel Query 6 • Producer/Consumer model • A PQ slave is either a producer or consumer, never both Figure 8-2 Inter-operation Parallelism and Dynamic Partitioning Oracle® Database VLDB and Partitioning Guide
© 2015 Pythian – Presentation for DOAG Oracle Parallel Query 7 • Producer/Consumer model • A PQ slave is either a producer or consumer, never both Figure 8-2 Inter-operation Parallelism and Dynamic Partitioning Oracle® Database VLDB and Partitioning Guide
© 2015 Pythian – Presentation for DOAG Oracle Parallel Query 8 • A PQ slave is either a producer or consumer in relation to other PQ slaves PQ1 PQ2 PQ3 PQ4 PQ5 PQ6 PQ7 PQ8 S G A DB
© 2015 Pythian – Presentation for DOAG Oracle Parallel Query 9 • But logically, step-wise, when there are two sets it’s flow-through PQ1 PQ2 PQ3 PQ4 PQ5 PQ6 PQ7 PQ8 S G A DB ONE STEP (1)
© 2015 Pythian – Presentation for DOAG Oracle Parallel Query 10 • QC is always flow-throught PQ1 PQ2 PQ3 PQ4 PQ5 PQ6 PQ7 PQ8 S G A DB S G A QC SQLNET CLIENTONE STEP (2)
© 2015 Pythian – Presentation for DOAG Oracle Parallel Query 11 • select count(*) from table; PQ1 PQ2 PQ3 PQ4 S G A DB ONE STEP QC SQLNET CLIENT
© 2015 Pythian – Presentation for DOAG Oracle Parallel Query 12 • Producer/consumer model • A slave is either producer and consumer in relation to other slaves.
13 Exadata Data Flow
© 2015 Pythian – Presentation for DOAG Multiple Tiers 14 • Every database has multiple tiers • Memory • SAN/NFS – network • SAN Performance • CPU Capacity • Client side connectivity • Each tier has it’s own nuances, properties and bandwidth limitations
© 2015 Pythian – Presentation for DOAG Exadata X2-2 Data Flow (intra) 15 • Each DB node (compute) and CELL node (storage) • Single dual port 40 Gbit infiniband (QDR) • 3,200 Mb/sec (full-duplex) • Each CELL node • Single dual port 40 Gbit infiniband (QDR) • 3,200 Mb/sec (full-duplex) • 12 x 4 TB SAS drives (7200 rpm) • Total bandwidth: 1,000 - 2,000 Mb/sec • 4 x Sun Flash Arrays • Total bandwidth: 4,100 Mb/sec
© 2015 Pythian – Presentation for DOAG Exadata Data Flow (External) 16 • SQL Net Clients talk only to DB nodes • Each DB node • 4 x 10 gigabit network ( 4 x 1.08 Gbytes/sec) • 2 x 10 Gigabit network (opt) ( 2 x 1.08 Gbytes/sec) • Alternatively, can use IP over infiniband • Exalogic • RMAN backups • Etc.
© 2015 Pythian – Presentation for DOAG Exadata Data Flow (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET
© 2015 Pythian – Presentation for DOAG DISK: 2 GB/sec + FLASH: 4 GB/sec = 6 GB/sec X 7 cells = 42 GB/sec Single Cell (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET CELL 01
© 2015 Pythian – Presentation for DOAG Cell output (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET CELL to SWITCH: 3.2 GB/sec X 7 cells = 22.4 GB/sec
© 2015 Pythian – Presentation for DOAG DB Node Consumption (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET SWITCH to DB NODE: 3.2 GB/sec X 4 nodes = 12.8 GB/sec
© 2015 Pythian – Presentation for DOAG Memory speed (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET DB to SGA (memory) 22 GB/sec (very CPU dependant) X 4 DB = 88 GB/sec
© 2015 Pythian – Presentation for DOAG DB Node to Client (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET DB to SINGLE SQLNET Client 112 MB/sec (1 GBIT) 1.08 GB/sec (10 GBIT)
© 2015 Pythian – Presentation for DOAG Exadata Data Flow (External) 23 • Ethernet link aggregation requires • Single conversation runs on a single path • Ethernet standard • One database sessions runs on single TCP connection, which must run on single path • Switches and network cards can implement different hashing algorithms to determine paths • Most are on IP level. Between 2 IPs the same NICs are used, regardless of number of connections
© 2015 Pythian – Presentation for DOAG Exadata Data Flow (Half Rack) DB4 eth0 DB to THIS Client: 112 MB/sec (theoretical) (18 MB/sec per session) IP: 10.31.33.1 Session Session Session Session Session Session eth1 eth2 eth3 eth0 eth1 eth2 eth3 S w i t c h
© 2015 Pythian – Presentation for DOAG Exadata Data Flow (External) 25 • As per Exadata Documentation: • Data Load Rate: Up to 10.5 TB/hour • 764 MB/sec per DB node • 3 GB/sec – can your other infrastructure provide such performance?
26 The Big Sort
© 2015 Pythian – Presentation for DOAG Sample Data 27 • My favorite table – CKK • 39 GiB of data • 10,000,000 rows • id, mod5_id, mod5000_id, dt_dec, dt_pos etc. create table ckk /*parallel 8 */nologging tablespace ckbig as select rownum id, mod(rownum,5) mod5_id, mod(rownum,5000) mod5000_id, sysdate dt_fixed, sysdate - rownum/24/60 dt_dec, sysdate + rownum/24/60 dt_pos, sysdate + ora_hash(rownum,65,535)/24 dt_rand, sysdate+mod(rownum,10) dt_mod10, rpad('x',3500,'x') filler from (select rownum r from dual connect by level <= 10000) r1, (select rownum r from dual connect by level <= 1000) ;
© 2015 Pythian – Presentation for DOAG Parallel Sort 28 select /*+ parallel(4) */ * from ckk c1 order by id; ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 33G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (ORDER) | :TQ10001 | 10M| 33G| | Q1,01 | P->S | QC (ORDER) | | 3 | SORT ORDER BY | | 10M| 33G| 38G| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 5 | PX SEND RANGE | :TQ10000 | 10M| 33G| | Q1,00 | P->P | RANGE | | 6 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- PQ1 PQ2 PQ3 PQ4 PQ5 – up ID=2.5 M PQ6 – up ID=5.0 M PQ7 – up ID=7.5 M PQ8 – up ID=10 M S G A DB
© 2015 Pythian – Presentation for DOAG STORAGE FULL (OFFLOADED) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET Decompression, Projection and Filter Happens on EVERY CELL NODE regardless of query parallelism. MPP mode.
© 2015 Pythian – Presentation for DOAG Parallel Sort 30 select dfo_number "d", tq_id as "t", server_type, num_rows,rpad('x',round(num_rows*10/nullif(max(num_rows) over (partition by dfo_number, tq_id, server_type),0)),'x') as "pr", round(bytes/1024/1024) mb, process, instance i,round(ratio_to_report (num_rows) over (partition by dfo_number, tq_id, server_type)*100) as "%", open_time, avg_latency, waits, timeouts,round(bytes/nullif(num_rows,0)) as "b/r” from v$pq_tqstat order by dfo_number, tq_id, server_type desc, process;; d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 372 xxxxxxxxxx 2 QC 1 100 1 0 Producer 2,692,340 xxxxxxxxxx 9,161 P002 1 27 1 0 Producer 2,499,910 xxxxxxxxx 8,506 P002 2 25 1 0 Producer 2,692,340 xxxxxxxxxx 9,161 P003 2 27 1 0 Producer 2,115,410 xxxxxxxx 7,198 P003 1 21 1 0 Consumer 6,146,614 xxxxxxxxxx 20,915 P000 2 61 1 0 Consumer 781,680 x 2,660 P000 1 8 1 0 Consumer 1,968,128 xxx 6,697 P001 2 20 1 0 Consumer 1,103,578 xx 3,755 P001 1 11 1 1 Producer 13 x 0 P000 1 5 1 1 Producer 212 xxxxxxxxxx 1 P000 2 84 1 1 Producer 13 x 0 P001 2 5 1 1 Producer 13 x 0 P001 1 5 1 1 Consumer 200 xxxxxxxxxx 1 QC 1 100
© 2015 Pythian – Presentation for DOAG Parallel Sort 31 select /*+ parallel(4) */ * from ckk c1 order by id; ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 33G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (ORDER) | :TQ10001 | 10M| 33G| | Q1,01 | P->S | QC (ORDER) | | 3 | SORT ORDER BY | | 10M| 33G| 38G| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 5 | PX SEND RANGE | :TQ10000 | 10M| 33G| | Q1,00 | P->P | RANGE | | 6 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- PQ1 PQ2 PQ3 PQ4 PQ5 – up to ID=6.1 M (6.1 M) PQ6 – up to ID=6.9 M (0.8 M) PQ7 – up to ID=8.8 M (1.9 M) PQ8 – up to ID=10.0 M (1.1M S G A DB
© 2015 Pythian – Presentation for DOAG The Ranger 32 • The Ranger Bug • The RANGER uses a sample from the data to determine the boundaries for PQ sort operations • Sometimes it’s off by a lot • It will ALWAYS miss when the input data is already somewhat sorted • Limit your scalability – no matter how big your machine is
© 2015 Pythian – Presentation for DOAG Parallel Sort – random data 33 select /*+ parallel(4) */ id, mod5_id, mod5000_id, dt_fixed, dt_dec, dt_pos, dt_rand, dt_mod10, substr(filler,1,2000) from ckk c1 order by dt_rand; --90 sec d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 372 xxxxxxxxxx 1 QC 1 100 1 0 Producer 2,500,030 xxxxxxxxxx 4,925 P002 1 25 1 0 Producer 2,500,030 xxxxxxxxxx 4,925 P002 2 25 1 0 Producer 2,499,910 xxxxxxxxxx 4,925 P003 1 25 1 0 Producer 2,500,030 xxxxxxxxxx 4,925 P003 2 25 1 0 Consumer 2,424,554 xxxxxxxxx 4,776 P000 2 24 1 0 Consumer 2,576,164 xxxxxxxxx 5,075 P000 1 26 1 0 Consumer 2,272,136 xxxxxxxx 4,476 P001 2 23 1 0 Consumer 2,727,146 xxxxxxxxxx 5,373 P001 1 27 1 1 Producer 218 xxxxxxxxxx 0 P000 1 76 1 1 Producer 23 x 0 P000 2 8 1 1 Producer 23 x 0 P001 2 8 1 1 Producer 23 x 0 P001 1 8 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
© 2015 Pythian – Presentation for DOAG Parallel Sort – random data 34 select /*+ parallel(32) */ id, mod5_id, mod5000_id, dt_fixed, dt_dec, dt_pos, dt_rand, dt_mod10, substr(filler,1,2000) from ckk c1 order by dt_rand; --10 sec d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 3,872 xxxxxxxxxx 11 QC 1 100 1 0 Producer 312,520 xxxxxxxxxx 616 P016 1 3 1 0 Producer 312,520 xxxxxxxxxx 616 P016 2 3 1 0 Producer 312,520 xxxxxxxxxx 616 P017 1 3 1 0 Producer 312,520 xxxxxxxxxx 616 P017 2 3 ... 1 0 Consumer 151,262 xxx 298 P003 1 2 1 0 Consumer 454,696 xxxxxxxxxx 896 P003 2 5 1 0 Consumer 302,320 xxxxxxx 596 P004 1 3 1 0 Consumer 151,731 xxx 299 P004 2 2 1 0 Consumer 303,429 xxxxxxx 598 P005 1 3 1 0 Consumer 454,650 xxxxxxxxxx 896 P005 2 5 1 0 Consumer 303,259 xxxxxxx 597 P006 2 3 1 0 Consumer 455,172 xxxxxxxxxx 897 P006 1 5 1 0 Consumer 302,945 xxxxxxx 597 P012 1 3 1 0 Consumer 303,318 xxxxxxx 598 P012 2 3 1 0 Consumer 303,245 xxxxxxx 597 P013 1 3 1 0 Consumer 304,029 xxxxxxx 599 P013 2 3 1 0 Consumer 301,937 xxxxxxx 595 P014 2 3 1 0 Consumer 454,903 xxxxxxxxxx 896 P014 1 5 1 0 Consumer 151,887 xxx 299 P015 1 2 1 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
© 2015 Pythian – Presentation for DOAG Parallel Sort – random data 35 select /*+ parallel(32) */ id, mod5_id, mod5000_id, dt_fixed, dt_dec, dt_pos, dt_rand, dt_mod10, substr(filler,1,2000) from ckk c1 order by id; --20 sec d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 3,872 xxxxxxxxxx 11 QC 1 100 1 0 Producer 240,400 xxxxxx 474 P016 2 2 1 0 Producer 384,640 xxxxxxxxx 758 P016 1 4 1 0 Producer 216,360 xxxxx 426 P017 1 2 1 0 Producer 192,320 xxxx 379 P017 2 2 ... 1 0 Consumer 373,702 xxxx 736 P000 1 4 1 0 Consumer 372,530 xxxx 734 P000 2 4 1 0 Consumer 124,801 x 246 P001 2 1 1 0 Consumer 183,552 xx 361 P001 1 2 1 0 Consumer 734,398 xxxxxxx 1,446 P002 1 7 1 0 Consumer 22,402 44 P002 2 0 1 0 Consumer 191,810 xx 378 P003 1 2 1 0 Consumer 132,990 x 262 P003 2 1 1 0 Consumer 32,640 64 P004 1 0 1 0 Consumer 920,448 xxxxxxxxx 1,813 P004 2 9 1 0 Consumer 167,232 xx 329 P005 1 2 1 0 Consumer 110,466 x 218 P005 2 1 1 0 Consumer 1,055,669 xxxxxxxxxx 2,080 P014 2 11 1 0 Consumer 130,944 x 258 P014 1 1 1 0 Consumer 60,250 x 119 P015 2 1 1 0 Consumer 20,298 40 P015 1 0 ... 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
© 2015 Pythian – Presentation for DOAG The Ranger issue 36 • Distribution can be much worse • When using analytics or other “input” into the sort operation that is pre-sorted • Often 90% goes to one slave, 10% to remaining • “Been worked on” – I am told • Problem made even worse due to lack of large buffers
© 2015 Pythian – Presentation for DOAG Writing to disk – lack of buffering 37 • When a slave is writing to disk – it cannot receive any messages or can receive a very small number of messages • Results in slowdowns when Oracle has to write to TEMP • Massive slowdown when only a few slaves have to write to TEMP • All producers have to wait • Significant slowdown when all slaves have to write to TEMP • Cannot parallelise sufficiently to leverage available disk bandwidth
38 The Big Group By
© 2015 Pythian – Presentation for DOAG Group By 39 select /*+ parallel(4) */ mod5_id, count(*) from ckk group by mod5_id; --------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| TQ |IN-OUT| PQ Distrib | --------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 5 | 15 | | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 5 | 15 | Q1,01 | P->S | QC (RAND) | | 3 | HASH GROUP BY | | 5 | 15 | Q1,01 | PCWP | | | 4 | PX RECEIVE | | 5 | 15 | Q1,01 | PCWP | | | 5 | PX SEND HASH | :TQ10000 | 5 | 15 | Q1,00 | P->P | HASH | | 6 | HASH GROUP BY | | 5 | 15 | Q1,00 | PCWP | | | 7 | PX BLOCK ITERATOR | | 10M| 28M| Q1,00 | PCWC | | | 8 | TABLE ACCESS STORAGE FULL| CKK | 10M| 28M| Q1,00 | PCWP | | --------------------------------------------------------------------------------------------------- select /*+ parallel(4) NO_GBY_PUSHDOWN*/ mod5_id, count(*) from ckk group by mod5_id; -------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| TQ |IN-OUT| PQ Distrib | -------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 5 | 15 | | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 5 | 15 | Q1,01 | P->S | QC (RAND) | | 3 | HASH GROUP BY | | 5 | 15 | Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 28M| Q1,01 | PCWP | | | 5 | PX SEND HASH | :TQ10000 | 10M| 28M| Q1,00 | P->P | HASH | | 6 | PX BLOCK ITERATOR | | 10M| 28M| Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK | 10M| 28M| Q1,00 | PCWP | | --------------------------------------------------------------------------------------------------
© 2015 Pythian – Presentation for DOAG Oracle Parallel Query 40 • But logically, step-wise, when there are two sets it’s flow-through PQ1 PQ2 PQ3 PQ4 S G A DB STEP (1) GROUP PGA GROUP PGA GROUP PGA GROUP PGA PQ5 PQ6 PQ7 PQ8 GROUP PGA GROUP PGA GROUP PGA GROUP PGA S G A QCSTEP (2)
© 2015 Pythian – Presentation for DOAG Group by – small result set 41 select /*+ parallel(4) */ mod5_id, count(*) from ckk group by mod5_id; -- 4.8 sec d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 5 xxxxxxxxxx 0 P002 2 25 1 0 Producer 5 xxxxxxxxxx 0 P002 1 25 1 0 Producer 5 xxxxxxxxxx 0 P003 2 25 1 0 Producer 5 xxxxxxxxxx 0 P003 1 25 1 0 Consumer 4 xxxxx 0 P000 2 20 1 0 Consumer 8 xxxxxxxxxx 0 P000 1 40 1 0 Consumer 4 xxxxx 0 P001 2 20 1 0 Consumer 4 xxxxx 0 P001 1 20 1 1 Producer 2 xxxxxxxxxx 0 P000 1 40 1 1 Producer 1 xxxxx 0 P000 2 20 1 1 Producer 1 xxxxx 0 P001 1 20 1 1 Producer 1 xxxxx 0 P001 2 20 1 1 Consumer 5 xxxxxxxxxx 0 QC 1 100
© 2015 Pythian – Presentation for DOAG Group by – small result set 42 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 2,692,340 xxxxxxxxxx 10 P004 2 27 1 0 Producer 2,307,600 xxxxxxxxx 8 P004 1 23 1 0 Producer 2,500,030 xxxxxxxxx 9 P005 2 25 1 0 Producer 2,500,030 xxxxxxxxx 9 P005 1 25 1 0 Consumer 4,000,000 xxxxxxxxxx 15 P002 2 40 1 0 Consumer 2,000,000 xxxxx 8 P002 1 20 1 0 Consumer 2,000,000 xxxxx 6 P003 2 20 1 0 Consumer 2,000,000 xxxxx 8 P003 1 20 1 1 Producer 1 xxxxx 0 P002 1 20 1 1 Producer 2 xxxxxxxxxx 0 P002 2 40 1 1 Producer 1 xxxxx 0 P003 1 20 1 1 Producer 1 xxxxx 0 P003 2 20 1 1 Consumer 5 xxxxxxxxxx 0 QC 1 100 select /*+ parallel(4) NO_GBY_PUSHDOWN*/ mod5_id, count(*) from ckk group by mod5_id; -- 4.8 sec
© 2015 Pythian – Presentation for DOAG Group by – small result set 43 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 264,440 xxxxxx 1 P016 2 3 1 0 Producer 312,520 xxxxxxxx 1 P016 1 3 1 0 Producer 240,400 xxxxxx 1 P017 1 2 1 0 Producer 360,600 xxxxxxxxx 1 P017 2 4 ... 1 0 Consumer 0 0 P000 2 0 1 0 Consumer 2,000,000 xxxxxxxxxx 8 P000 1 20 1 0 Consumer 0 0 P001 2 0 ... 1 0 Consumer 0 0 P004 1 0 1 0 Consumer 0 0 P005 2 0 1 0 Consumer 2,000,000 xxxxxxxxxx 6 P005 1 20 1 0 Consumer 0 0 P006 2 0 1 0 Consumer 0 0 P006 1 0 1 0 Consumer 0 0 P007 2 0 1 0 Consumer 2,000,000 xxxxxxxxxx 8 P007 1 20 1 0 Consumer 2,000,000 xxxxxxxxxx 8 P008 1 20 1 0 Consumer 2,000,000 xxxxxxxxxx 8 P010 2 20 ... select /*+ parallel(32) NO_GBY_PUSHDOWN*/ mod5_id, count(*) from ckk group by mod5_id; -- 4.8 sec
© 2015 Pythian – Presentation for DOAG Group by – small result set 44 • When the result set from a group by is small – there are a lot of optimisations that can kick in • PUSHDOWN saves on PQ traffic • Receiving end has very little work to do • Very efficient to parallelise
© 2015 Pythian – Presentation for DOAG select /*+ parallel(4) */ id, count(*) from ckk group by id; -- 4.8 sec Group by – large result set HASH 45 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 2,500,030 xxxxxxxx 17 P002 2 25 1 0 Producer 2,115,410 xxxxxx 14 P002 1 21 1 0 Producer 2,115,410 xxxxxx 14 P003 2 21 1 0 Producer 3,269,150 xxxxxxxxxx 21 P003 1 33 1 0 Consumer 2,499,213 xxxxxxxxxx 16 P000 1 25 1 0 Consumer 2,499,807 xxxxxxxxxx 16 P000 2 25 1 0 Consumer 2,500,981 xxxxxxxxxx 16 P001 1 25 1 0 Consumer 2,499,999 xxxxxxxxxx 16 P001 2 25 1 1 Producer 1,465 xxxxx 0 P000 2 17 1 1 Producer 2,929 xxxxxxxxxx 0 P000 1 33 1 1 Producer 2,928 xxxxxxxxxx 0 P001 1 33 1 1 Producer 1,464 xxxxx 0 P001 2 17 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
© 2015 Pythian – Presentation for DOAG select /*+ parallel(4) */ id, count(*) from ckk group by id order by id; -- 6.5 sec Group by – large result set SORT 46 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 372 xxxxxxxxxx 0 QC 1 100 1 0 Producer 2,499,910 xxxxxxxxx 17 P002 1 25 1 0 Producer 2,692,340 xxxxxxxxxx 18 P002 2 27 1 0 Producer 2,500,030 xxxxxxxxx 17 P003 2 25 1 0 Producer 2,307,720 xxxxxxxxx 15 P003 1 23 1 0 Consumer 6,154,678 xxxxxxxxxx 40 P000 2 62 1 0 Consumer 793,712 x 5 P000 1 8 1 0 Consumer 1,095,514 xx 7 P001 1 11 1 0 Consumer 1,956,096 xxx 13 P001 2 20 1 1 Producer 4,391 xxxxxxxx 0 P000 1 24 1 1 Producer 5,379 xxxxxxxxxx 0 P000 2 29 1 1 Producer 4,391 xxxxxxxx 0 P001 2 24 1 1 Producer 4,391 xxxxxxxx 0 P001 1 24 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
© 2015 Pythian – Presentation for DOAG select /*+ parallel(32) */ id, count(*) from ckk group by id; -- 4.8 sec Group by – large result set HASH 47 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 360,600 xxxxxxxxx 2 P016 1 4 1 0 Producer 240,400 xxxxxx 2 P016 2 2 1 0 Producer 312,520 xxxxxxxx 2 P017 1 3 1 0 Producer 336,560 xxxxxxxxx 2 P017 2 3 ... 1 0 Consumer 312,829 xxxxxxxxxx 2 P000 2 3 1 0 Consumer 311,690 xxxxxxxxxx 2 P000 1 3 1 0 Consumer 312,371 xxxxxxxxxx 2 P001 1 3 1 0 Consumer 312,854 xxxxxxxxxx 2 P001 2 3 1 0 Consumer 312,255 xxxxxxxxxx 2 P002 2 3 1 0 Consumer 312,321 xxxxxxxxxx 2 P002 1 3 1 0 Consumer 313,241 xxxxxxxxxx 2 P003 2 3 1 0 Consumer 312,605 xxxxxxxxxx 2 P003 1 3 1 0 Consumer 312,202 xxxxxxxxxx 2 P004 1 3 1 0 Consumer 311,294 xxxxxxxxxx 2 P004 2 3 1 0 Consumer 312,338 xxxxxxxxxx 2 P005 1 3 1 0 Consumer 312,206 xxxxxxxxxx 2 P005 2 3 1 0 Consumer 312,898 xxxxxxxxxx 2 P006 1 3 1 0 Consumer 312,817 xxxxxxxxxx 2 P006 2 3 1 0 Consumer 312,291 xxxxxxxxxx 2 P007 1 3 1 0 Consumer 312,872 xxxxxxxxxx 2 P007 2 3 ... 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
© 2015 Pythian – Presentation for DOAG Group by – large result set HASH 48 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 312,520 xxxxxxxxxx 1,047 P016 1 3 1 0 Producer 312,520 xxxxxxxxxx 1,047 P016 2 3 1 0 Producer 312,520 xxxxxxxxxx 1,047 P017 1 3 ... 1 0 Consumer 311,603 xxxxxxxxxx 1,044 P000 1 3 1 0 Consumer 312,470 xxxxxxxxxx 1,047 P000 2 3 1 0 Consumer 312,484 xxxxxxxxxx 1,047 P001 2 3 1 0 Consumer 312,662 xxxxxxxxxx 1,048 P001 1 3 1 0 Consumer 312,341 xxxxxxxxxx 1,047 P002 2 3 1 0 Consumer 312,692 xxxxxxxxxx 1,048 P002 1 3 1 0 Consumer 313,114 xxxxxxxxxx 1,049 P003 2 3 1 0 Consumer 311,889 xxxxxxxxxx 1,045 P003 1 3 1 0 Consumer 312,323 xxxxxxxxxx 1,047 P004 1 3 1 0 Consumer 313,310 xxxxxxxxxx 1,050 P004 2 3 1 0 Consumer 312,715 xxxxxxxxxx 1,048 P005 2 3 1 0 Consumer 313,394 xxxxxxxxxx 1,050 P005 1 3 1 0 Consumer 312,718 xxxxxxxxxx 1,048 P006 1 3 1 0 Consumer 312,090 xxxxxxxxxx 1,046 P006 2 3 1 0 Consumer 312,720 xxxxxxxxxx 1,048 P007 1 3 1 0 Consumer 311,889 xxxxxxxxxx 1,045 P007 2 3 2 ... 1 1 Consumer 200 xxxxxxxxxx 1 QC 1 100 select /*+ parallel(32) */ id, filler, count(*) from ckk group by id,filler; -- 18 sec
© 2015 Pythian – Presentation for DOAG Group by – large result set SORT 49 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 3,872 xxxxxxxxxx 23 QC 1 100 1 0 Producer 288,480 xxxxxxx 967 P016 2 3 1 0 Producer 288,480 xxxxxxx 967 P016 1 3 1 0 Producer 336,560 xxxxxxxx 1,128 P017 2 3 1 0 Producer 240,400 xxxxxx 806 P017 1 2 ... 1 0 Consumer 722,115 xxxxxx 2,420 P002 2 7 1 0 Consumer 16,386 55 P002 1 0 1 0 Consumer 210,112 xx 704 P003 2 2 1 0 Consumer 112,510 x 377 P003 1 1 1 0 Consumer 940,928 xxxxxxxx 3,153 P004 1 9 1 0 Consumer 34,688 116 P004 2 0 1 0 Consumer 171,392 xx 574 P005 2 2 1 0 Consumer 110,466 x 370 P005 1 1 1 0 Consumer 163,658 x 548 P014 2 2 1 0 Consumer 1,113,523 xxxxxxxxxx 3,732 P014 1 11 1 0 Consumer 11,100 37 P015 1 0 1 0 Consumer 4,150 14 P015 2 0 ... 1 1 Consumer 200 xxxxxxxxxx 1 QC 1 100 select /*+ parallel(32) */ id, filler, count(*) from ckk group by id,filler order by id,filler; -- 41 sec
© 2015 Pythian – Presentation for DOAG Group by – summary 50 • Theoretical performance is reachable • SORT based operations have major issues • Especially with sorted inputs, such as from analytics • model clause can help – partition by is hash based • Partial disk staging causes major slowdowns • Balanced operation is critical
© 2015 Pythian – Presentation for DOAG Group by – Partitioning 51 • What about partitioning? create table ckk_p (id, mod5_id, mod5000_id, dt_fixed, dt_dec, dt_pos, dt_rand, dt_mod10, filler) partition by hash (id) partitions 32 tablespace ckbig nologging parallel 4 as select * from ckk;
© 2015 Pythian – Presentation for DOAG Group by – with partitioning 52 select /*+ parallel(4) */ id,count(*) from ckk_p group by id,filler; ------------------------------------------------------------------------------------------------------------------------ | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | Pstart| Pstop | TQ |IN-OUT| PQ Distrib | ------------------------------------------------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | 7071K| 23G| | | | | | | | 1 | PX COORDINATOR | | | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | 7071K| 23G| | | | Q1,00 | P->S | QC (RAND) | | 3 | PX PARTITION HASH ALL | | 7071K| 23G| | 1 | 32 | Q1,00 | PCWC | | | 4 | HASH GROUP BY | | 7071K| 23G| 38G| | | Q1,00 | PCWP | | | 5 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 32G| | 1 | 32 | Q1,00 | PCWP | | ------------------------------------------------------------------------------------------------------------------------ PQ1 PQ2 PQ3 PQ4 S G A STEP (1) GROUP PGA GROUP PGA GROUP PGA GROUP PGA QC LIST P1 P2 P3 P4 P5 P6 P7 P8
© 2015 Pythian – Presentation for DOAG select /*+ parallel(64) */ * from ckk c1 order by id; ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 7071K| 23G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 7071K| 23G| | Q1,01 | P->S | QC (RAND) | | 3 | HASH GROUP BY | | 7071K| 23G| 38G| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 32G| | Q1,01 | PCWP | | | 5 | PX SEND HASH | :TQ10000 | 10M| 32G| | Q1,00 | P->P | HASH | | 6 | PX BLOCK ITERATOR | | 10M| 32G| | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 32G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- Group by – enough partitions 53 PQ1 PQ2 PQ3 PQ4 S G A STEP (1) GROUP PGA GROUP PGA GROUP PGA GROUP PGA QC LIST P1 P2 P3 P4 P5 P6 P7 P8
© 2015 Pythian – Presentation for DOAG Analytics don’t speak parallel 54 select /*+ parallel(4) */ id, row_number() over (partition by id,filler order by null) r from ckk_p ; ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 32G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 32G| | Q1,01 | P->S | QC (RAND) | | 3 | WINDOW SORT | | 10M| 32G| 38G| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 32G| | Q1,01 | PCWP | | | 5 | PX SEND HASH | :TQ10000 | 10M| 32G| | Q1,00 | P->P | HASH | | 6 | PX BLOCK ITERATOR | | 10M| 32G| | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 32G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- select /*+ NO_parallel */ id, row_number() over (partition by id,filler order by null) r from ckk_p ; -------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | Pstart| Pstop | -------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 32G| | | | | 1 | PARTITION HASH ALL | | 10M| 32G| | 1 | 32 | | 2 | WINDOW SORT | | 10M| 32G| 38G| | | | 3 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 32G| | 1 | 32 | --------------------------------------------------------------------------------------
55 The Big Join
© 2015 Pythian – Presentation for DOAG Test Data 56 • Copy of ckk – ckk2 • ckk_m5 – 5 rows dimension create table ckk_m5 as select rownum mod5_id, rpad('x',1000,'x') filler from dual connect by level <=5; alter table ckk_m5 modify (mod5_id not null);
© 2015 Pythian – Presentation for DOAG Simple Hash Join select /*+ parallel(4) */ * from ckk c1 join ckk_m5 m5 on (c1.mod5_id = m5.mod5_id); -------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| TQ |IN-OUT| PQ Distrib | -------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 42G| | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 42G| Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN | | 10M| 42G| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 5 | 5020 | Q1,01 | PCWP | | | 5 | PX SEND BROADCAST | :TQ10000 | 5 | 5020 | Q1,00 | P->P | BROADCAST | | 6 | PX BLOCK ITERATOR | | 5 | 5020 | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK_M5 | 5 | 5020 | Q1,00 | PCWP | | | 8 | PX BLOCK ITERATOR | | 10M| 33G| Q1,01 | PCWC | | | 9 | TABLE ACCESS STORAGE FULL | CKK | 10M| 33G| Q1,01 | PCWP | | --------------------------------------------------------------------------------------------------
© 2015 Pythian – Presentation for DOAG Simple Hash Join 58 PQ1 PQ2 PQ3 PQ4 S G A STEP (1) PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A QC CKK _M5 0 GiB All Hash Areas: 0 GiB
© 2015 Pythian – Presentation for DOAG Simple Hash Join 59 PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A STEP (2) QC CKK (39 GiB)
© 2015 Pythian – Presentation for DOAG Big Hash Join select /*+ parallel(4) */ * from ckk c1join ckk2 c2 on (c1.id = c2.id) ------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp |IN-OUT| PQ Distrib | ------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10002 | 10M| 66G| | P->S | QC (RAND) | |* 3 | HASH JOIN BUFFERED | | 10M| 66G| 8502M| PCWP | | | 4 | PX RECEIVE | | 10M| 33G| | PCWP | | | 5 | PX SEND HASH | :TQ10000 | 10M| 33G| | P->P | HASH | | 6 | PX BLOCK ITERATOR | | 10M| 33G| | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | PCWP | | | 8 | PX RECEIVE | | 10M| 33G| | PCWP | | | 9 | PX SEND HASH | :TQ10001 | 10M| 33G| | P->P | HASH | | 10 | PX BLOCK ITERATOR | | 10M| 33G| | PCWC | | | 11 | TABLE ACCESS STORAGE FULL| CKK2 | 10M| 33G| | PCWP | | -------------------------------------------------------------------------------------------------
© 2015 Pythian – Presentation for DOAG Big Hash Join 61 PQ1 PQ2 PQ3 PQ4 S G A STEP (1) PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A QC CKK 39 GiB All Hash Areas: 39 GiB
© 2015 Pythian – Presentation for DOAG Big Hash Join 62 PQ1 PQ2 PQ3 PQ4 S G A STEP (2) PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A QC CKK 2 39 GiB All Hash Areas: 78 GiB
© 2015 Pythian – Presentation for DOAG Simple Hash Join 63 PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A STEP (3) QC
© 2015 Pythian – Presentation for DOAG Hash Join summary 64 • HASH JOINS with small tables are extremely fast • Uses BROADCAST to send copy of table to every slave • As much PGA work area as parallel degree * small table size • HASH JOINS with big tables can be extremely expensive • Almost entire result set written to temp • Unless – partitioning
© 2015 Pythian – Presentation for DOAG Semi-Partitioned Hash Join select /*+ parallel(32) */ * from ckk c1 join ckk_p c2 on (c1.id = c2.id) ----------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ----------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 66G| | Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN | | 10M| 66G| 1062M| Q1,01 | PCWP | | | 4 | PART JOIN FILTER CREATE | :BF0000 | 10M| 33G| | Q1,01 | PCWP | | | 5 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 6 | PX SEND PARTITION (KEY) | :TQ10000 | 10M| 33G| | Q1,00 | P->P | PART (KEY) | | 7 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 8 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | | 9 | PX PARTITION HASH JOIN-FILTER| | 10M| 33G| | Q1,01 | PCWC | | | 10 | TABLE ACCESS STORAGE FULL | CKK_P | 10M| 33G| | Q1,01 | PCWP | | -----------------------------------------------------------------------------------------------------------
© 2015 Pythian – Presentation for DOAG Semi Partitioned Hash Join 66 PQ1 PQ2 PQ3 PQ4 S G A STEP (1) PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A QC CKK 39 GiB All Hash Areas: 39 GiB
© 2015 Pythian – Presentation for DOAG Semi Partitioned Hash Join 67 PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A STEP (2) QC CKK_P (39 GiB) LIST P1 P2 P3 P4 P5 P6 P7 P8
© 2015 Pythian – Presentation for DOAG Semi Partitioned Hash Join 68 • Dramatically reduces HASH Area sizes • Very likely to fit in RAM – no disk • Reduces from 3 steps to 2 steps • Always reads the non-partitioned table into HASH areas • Each slave reads it’s own partition
© 2015 Pythian – Presentation for DOAG Partitioned Hash Join select /*+ parallel(4) */ * from ckk_p c1 join ckk_p c2 on (c1.id = c2.id) -------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | -------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | 10M| 66G| | Q1,00 | P->S | QC (RAND) | | 3 | PX PARTITION HASH ALL | | 10M| 66G| | Q1,00 | PCWC | | |* 4 | HASH JOIN | | 10M| 66G| 8502M| Q1,00 | PCWP | | | 5 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 33G| | Q1,00 | PCWP | | | 6 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 33G| | Q1,00 | PCWP | | --------------------------------------------------------------------------------------------------------
© 2015 Pythian – Presentation for DOAG Partitioned Hash Join select /*+ parallel(4) */ * from ckk_p c1 join ckk_p c2 on (c1.id = c2.id) -------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | -------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | 10M| 66G| | Q1,00 | P->S | QC (RAND) | | 3 | PX PARTITION HASH ALL | | 10M| 66G| | Q1,00 | PCWC | | |* 4 | HASH JOIN | | 10M| 66G| 8502M| Q1,00 | PCWP | | | 5 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 33G| | Q1,00 | PCWP | | | 6 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 33G| | Q1,00 | PCWP | | --------------------------------------------------------------------------------------------------------
© 2015 Pythian – Presentation for DOAG Switching places select /*+ parallel(32) */ * from ckk_p c1 join ckk c2 on (c1.id = c2.id); ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 66G| | Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN BUFFERED | | 10M| 66G| 1062M| Q1,01 | PCWP | | | 4 | PX PARTITION HASH ALL | | 10M| 33G| | Q1,01 | PCWC | | | 5 | TABLE ACCESS STORAGE FULL | CKK_P | 10M| 33G| | Q1,01 | PCWP | | | 6 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 7 | PX SEND PARTITION (KEY) | :TQ10000 | 10M| 33G| | Q1,00 | P->P | PART (KEY) | | 8 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 9 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- select /*+ parallel(32) LEADING(c2) */ * from ckk_p c1 join ckk c2 on (c1.id = c2.id); ----------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ----------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 66G| | Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN | | 10M| 66G| 1062M| Q1,01 | PCWP | | | 4 | PART JOIN FILTER CREATE | :BF0000 | 10M| 33G| | Q1,01 | PCWP | | | 5 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 6 | PX SEND PARTITION (KEY) | :TQ10000 | 10M| 33G| | Q1,00 | P->P | PART (KEY) | | 7 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 8 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | | 9 | PX PARTITION HASH JOIN-FILTER| | 10M| 33G| | Q1,01 | PCWC | | | 10 | TABLE ACCESS STORAGE FULL | CKK_P | 10M| 33G| | Q1,01 | PCWP | | -----------------------------------------------------------------------------------------------------------
© 2015 Pythian – Presentation for DOAG Outer Joins select /*+ parallel(32) */ * from ckk_p c1 left join ckk c2 on (c1.id = c2.id); ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 66G| | Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN OUTER BUFFERED | | 10M| 66G| 1062M| Q1,01 | PCWP | | | 4 | PX PARTITION HASH ALL | | 10M| 33G| | Q1,01 | PCWC | | | 5 | TABLE ACCESS STORAGE FULL | CKK_P | 10M| 33G| | Q1,01 | PCWP | | | 6 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 7 | PX SEND PARTITION (KEY) | :TQ10000 | 10M| 33G| | Q1,00 | P->P | PART (KEY) | | 8 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 9 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- select /*+ parallel(32) SWAP_JOIN_INPUTS(c2)*/ * from ckk_p c1 left join ckk c2 on (c1.id = c2.id); ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 66G| | Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN RIGHT OUTER | | 10M| 66G| 1062M| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 5 | PX SEND PARTITION (KEY) | :TQ10000 | 10M| 33G| | Q1,00 | P->P | PART (KEY) | | 6 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | | 8 | PX PARTITION HASH ALL | | 10M| 33G| | Q1,01 | PCWC | | | 9 | TABLE ACCESS STORAGE FULL | CKK_P | 10M| 33G| | Q1,01 | PCWP | | ----------------------------------------------------------------------------------------------------------
73 The Big Join – with Index
© 2015 Pythian – Presentation for DOAG Adding a few indexes 74 create index ckk$id on ckk(id) parallel 32 tablespace ckbig nologging; --6.5 sec create index ckk$mod5000_id on ckk(mod5000_id) parallel 32 tablespace ckbig nologging compress; --5.8 sec create index ckk_p$mod5000_id on ckk_p(mod5000_id) parallel 32 tablespace ckbig nologging compress local; --16 sec
© 2015 Pythian – Presentation for DOAG Direct index access select /*+ parallel(4) */ * from ckk where mod5000_id=100; ----------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| ----------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 2000 | 6941K| | 1 | TABLE ACCESS BY INDEX ROWID| CKK | 2000 | 6941K| |* 2 | INDEX RANGE SCAN | CKK$MOD5000_ID | 2000 | | ----------------------------------------------------------------------- select /*+ parallel(4) */ * from ckk_p where mod5000_id=100; -------------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| TQ |IN-OUT| PQ Distrib | -------------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 2000 | 6941K| | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | 2000 | 6941K| Q1,00 | P->S | QC (RAND) | | 3 | PX PARTITION HASH ALL | | 2000 | 6941K| Q1,00 | PCWC | | | 4 | TABLE ACCESS BY LOCAL INDEX ROWID| CKK_P | 2000 | 6941K| Q1,00 | PCWP | | |* 5 | INDEX RANGE SCAN | CKK_P$MOD5000_ID | 2000 | | Q1,00 | PCWP | | --------------------------------------------------------------------------------------------------------------
© 2015 Pythian – Presentation for DOAG Join with Index select /*+ parallel(4) */ * from ckk c1 join ckk2 c2 on (c1.id = c2.id) where c2.mod5000_id=5; ------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| TQ |IN-OUT| PQ Distrib | ------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 2017 | 13M| | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | | | Q1,00 | P->S | QC (RAND) | | 3 | NESTED LOOPS | | | | Q1,00 | PCWP | | | 4 | NESTED LOOPS | | 2017 | 13M| Q1,00 | PCWP | | | 5 | PX BLOCK ITERATOR | | | | Q1,00 | PCWC | | |* 6 | TABLE ACCESS STORAGE FULL| CKK2 | 2000 | 6941K| Q1,00 | PCWP | | |* 7 | INDEX RANGE SCAN | CKK$ID | 1 | | Q1,00 | PCWP | | | 8 | TABLE ACCESS BY INDEX ROWID| CKK | 1 | 3554 | Q1,00 | PCWP | | -------------------------------------------------------------------------------------------------
© 2015 Pythian – Presentation for DOAG Indexes summary 77 • Parallel access on direct value queries only on partitioned tables and on local indexes • Parallel access on global or non-partitioned only from joins (multiple different values)
© 2015 Pythian – Presentation for DOAG Final Thoughts 78 • Exadata allows Parallel Query to make sense • High Bandwidth everywhere • Very efficient table scans • Optimiser sometimes picks up wrong access plan • There are deficiencies in SORT operations – balancing the workloads • There are deficiencies in handling disk write operations in PQ Slaves • Can be extremely fast under the right circumstances
© 2009/2010 Pythian - Presentation for ABC Company79 The End Thank You Questions? I blog at http://www.pythian.com/news/author/kutrovsky/

More Related Content

PPT
PLNOG15: Practical deployments of Kea, a high performance scalable DHCP - Tom...
byPROIDEA
 
PDF
Sfo15 408- odp-dpdk platform performance
byLinaro
 
PDF
Virtualizing the Network to enable a Software Defined Infrastructure (SDI)
byOdinot Stanislas
 
PDF
ONS Summit 2017 SKT TINA
byJunho Suh
 
PDF
Introduction to Data Centre 3.0
byTũi Wichets
 
PDF
Open stackdaykorea2016 wedge
byJunho Suh
 
PPTX
Birds of a feather - network engineering
byJisc
 
PDF
OpenFlow @ Google
byOpen Networking Summits
 
PLNOG15: Practical deployments of Kea, a high performance scalable DHCP - Tom...
byPROIDEA
 
Sfo15 408- odp-dpdk platform performance
byLinaro
 
Virtualizing the Network to enable a Software Defined Infrastructure (SDI)
byOdinot Stanislas
 
ONS Summit 2017 SKT TINA
byJunho Suh
 
Introduction to Data Centre 3.0
byTũi Wichets
 
Open stackdaykorea2016 wedge
byJunho Suh
 
Birds of a feather - network engineering
byJisc
 
OpenFlow @ Google
byOpen Networking Summits
 

What's hot

PPTX
Hannes end-of-the-router-tnc17
byHannes Gredler
 
PDF
Service Function Chaining in Openstack Neutron
byMichelle Holley
 
PPTX
Building a Router
byHannes Gredler
 
PPTX
GPORCA: Query Optimization as a Service
byPivotalOpenSourceHub
 
PDF
【EPN Seminar Nov.10.2015】 Services Function Chaining Architecture, Standardiz...
byシスコシステムズ合同会社
 
PPTX
Barak Perlman, ConteXtream - SFC (Service Function Chaining) Using Openstack ...
byCloud Native Day Tel Aviv
 
PDF
OCP Summit 2016 - Transforming Networks to All-IT Network with OCP and Open N...
byJunho Suh
 
PPTX
Transactional operations in Apache Hive: present and future
byDataWorks Summit
 
PPTX
LISP and NSH in Open vSwitch
bymestery
 
PPSX
MQTC V2.0.1.3 - WMQ & TCP Buffers – Size DOES Matter! (pps)
byArt Schanz
 
PPTX
Opensample: A Low-latency, Sampling-based Measurement Platform for Software D...
byJunho Suh
 
PDF
Segment routing tutorial
byYi-Sung Chiu
 
PPTX
Ipv6 deployment at the university of reading - Networkshop44
byJisc
 
PPTX
DEVNET-1175 OpenDaylight Service Function Chaining
byCisco DevNet
 
PPTX
APNIC IPv6 Deployment
byAPNIC
 
ODP
CentOS NFV SIG Introduction and Update
byTom Herbert
 
PDF
Traffic Engineering Using Segment Routing
byCisco Canada
 
PDF
DPDK Summit 2015 - Sprint - Arun Rajagopal
byJim St. Leger
 
PDF
Criteo Labs Infrastructure Tech Talk Meetup Nov. 7
byShuo LI
 
PDF
IPv6 at CSCS
bySwiss IPv6 Council
 
Hannes end-of-the-router-tnc17
byHannes Gredler
 
Service Function Chaining in Openstack Neutron
byMichelle Holley
 
Building a Router
byHannes Gredler
 
GPORCA: Query Optimization as a Service
byPivotalOpenSourceHub
 
【EPN Seminar Nov.10.2015】 Services Function Chaining Architecture, Standardiz...
byシスコシステムズ合同会社
 
Barak Perlman, ConteXtream - SFC (Service Function Chaining) Using Openstack ...
byCloud Native Day Tel Aviv
 
OCP Summit 2016 - Transforming Networks to All-IT Network with OCP and Open N...
byJunho Suh
 
Transactional operations in Apache Hive: present and future
byDataWorks Summit
 
LISP and NSH in Open vSwitch
bymestery
 
MQTC V2.0.1.3 - WMQ & TCP Buffers – Size DOES Matter! (pps)
byArt Schanz
 
Opensample: A Low-latency, Sampling-based Measurement Platform for Software D...
byJunho Suh
 
Segment routing tutorial
byYi-Sung Chiu
 
Ipv6 deployment at the university of reading - Networkshop44
byJisc
 
DEVNET-1175 OpenDaylight Service Function Chaining
byCisco DevNet
 
APNIC IPv6 Deployment
byAPNIC
 
CentOS NFV SIG Introduction and Update
byTom Herbert
 
Traffic Engineering Using Segment Routing
byCisco Canada
 
DPDK Summit 2015 - Sprint - Arun Rajagopal
byJim St. Leger
 
Criteo Labs Infrastructure Tech Talk Meetup Nov. 7
byShuo LI
 
IPv6 at CSCS
bySwiss IPv6 Council
 

Viewers also liked

PDF
Ge Careers How To Apply
byBruno Pitzer
 
PDF
被災校の先生方を対象としたリフレッシュ研修
byKoyo Yamamori
 
PDF
10 Things to Make API Users Like You
byDavid Yun
 
PDF
Bestavros Memorial Book
byvmstj
 
PPS
Mike baker ovarb, fidelidad
byMike Baker Ovarb
 
PDF
お台場学園前砂浜における海苔の生育について
byKoyo Yamamori
 
PPTX
"Social Innovation for Employment" information session
byNahdet El Mahrousa
 
PPT
Edmodo 8-6-12
byOlga Kuharets
 
PPT
แต่งตัวตุ๊กตา
byJirachaya Sritha
 
PDF
White Paper Employee Retention
byewayton
 
PPTX
Dreamworks
byrussian21
 
PDF
Anteprima Pianificazione strategica per gli studi professionali della Guida a...
byeAgisco
 
PDF
Virtual + Magazine (2012 vEN)
bymichaelhandforth
 
PPT
BYOiPad Parent Briefing, July 31, 2012
byjarruzza
 
PPTX
Ajay 61 b 201214
byAjay Pareek
 
PDF
LVSA Annrual Report 2010_2011
byBill Ritchie
 
PDF
Support The Young Innovators Awards Program
byNahdet El Mahrousa
 
PDF
W.A.Y. Where are you? Strumento di a
byeAgisco
 
PPTX
Learning Solutions
byjaylambert
 
PDF
150520 教育学特殊XIV 第5講
byKoyo Yamamori
 
Ge Careers How To Apply
byBruno Pitzer
 
被災校の先生方を対象としたリフレッシュ研修
byKoyo Yamamori
 
10 Things to Make API Users Like You
byDavid Yun
 
Bestavros Memorial Book
byvmstj
 
Mike baker ovarb, fidelidad
byMike Baker Ovarb
 
お台場学園前砂浜における海苔の生育について
byKoyo Yamamori
 
"Social Innovation for Employment" information session
byNahdet El Mahrousa
 
Edmodo 8-6-12
byOlga Kuharets
 
แต่งตัวตุ๊กตา
byJirachaya Sritha
 
White Paper Employee Retention
byewayton
 
Dreamworks
byrussian21
 
Anteprima Pianificazione strategica per gli studi professionali della Guida a...
byeAgisco
 
Virtual + Magazine (2012 vEN)
bymichaelhandforth
 
BYOiPad Parent Briefing, July 31, 2012
byjarruzza
 
Ajay 61 b 201214
byAjay Pareek
 
LVSA Annrual Report 2010_2011
byBill Ritchie
 
Support The Young Innovators Awards Program
byNahdet El Mahrousa
 
W.A.Y. Where are you? Strumento di a
byeAgisco
 
Learning Solutions
byjaylambert
 
150520 教育学特殊XIV 第5講
byKoyo Yamamori
 

Similar to Christo Kutrovsky - Maximize Data Warehouse Performance with Parallel Queries

PDF
Parallel Execution With Oracle Database 12c - Masterclass
byIvica Arsov
 
PDF
Px execution in rac
byRiyaj Shamsudeen
 
PPTX
Manjeet Singh.pptx
byRAMCHANDRASHARMA7
 
PPTX
Analysing and troubleshooting Parallel Execution IT Tage 2015
byRandolf Geist
 
PPTX
All (that i know) about exadata external
byPrasad Chitta
 
PPT
Ch22 parallel d_bs_cs561
byShobhit Saxena
 
PPTX
database slide on modern techniques for optimizing database queries.pptx
byaky52024
 
PPTX
Oracle Database 12c features for DBA
byKaran Kukreja
 
PDF
Parallel and Distributed Databases NOTES.pdf
bywrushabsirsat
 
PDF
Randolf Geist – IT-Tage 2015 – Oracle Parallel Execution – Analyse und Troubl...
byInformatik Aktuell
 
PDF
Oracle Parallel Distribution and 12c Adaptive Plans
byFranck Pachot
 
PDF
Parallel Query on Exadata
byEnkitec
 
PPTX
Powering GIS Application with PostgreSQL and Postgres Plus
byAshnikbiz
 
PPT
Introduction to Parallel Execution
byDoug Burns
 
PDF
Technical Introduction to PostgreSQL and PPAS
byAshnikbiz
 
PDF
OracleDatabase12cPXNewFeatures_ITOUG_2018.pdf
by7vkx8892hv
 
PPTX
CS 542 -- Query Execution
byJ Singh
 
PPTX
Oracle Database 12.1.0.2 New Features
byDeiby Gómez
 
PDF
POLARDB for MySQL - Parallel Query
byoysteing
 
PDF
OOW13: Accelerate your Exadata deployment with the DBA skills you already have
byMarc Fielding
 
Parallel Execution With Oracle Database 12c - Masterclass
byIvica Arsov
 
Px execution in rac
byRiyaj Shamsudeen
 
Manjeet Singh.pptx
byRAMCHANDRASHARMA7
 
Analysing and troubleshooting Parallel Execution IT Tage 2015
byRandolf Geist
 
All (that i know) about exadata external
byPrasad Chitta
 
Ch22 parallel d_bs_cs561
byShobhit Saxena
 
database slide on modern techniques for optimizing database queries.pptx
byaky52024
 
Oracle Database 12c features for DBA
byKaran Kukreja
 
Parallel and Distributed Databases NOTES.pdf
bywrushabsirsat
 
Randolf Geist – IT-Tage 2015 – Oracle Parallel Execution – Analyse und Troubl...
byInformatik Aktuell
 
Oracle Parallel Distribution and 12c Adaptive Plans
byFranck Pachot
 
Parallel Query on Exadata
byEnkitec
 
Powering GIS Application with PostgreSQL and Postgres Plus
byAshnikbiz
 
Introduction to Parallel Execution
byDoug Burns
 
Technical Introduction to PostgreSQL and PPAS
byAshnikbiz
 
OracleDatabase12cPXNewFeatures_ITOUG_2018.pdf
by7vkx8892hv
 
CS 542 -- Query Execution
byJ Singh
 
Oracle Database 12.1.0.2 New Features
byDeiby Gómez
 
POLARDB for MySQL - Parallel Query
byoysteing
 
OOW13: Accelerate your Exadata deployment with the DBA skills you already have
byMarc Fielding
 

Recently uploaded

PDF
MySQL Routing Guidelines: Designing Smarter Query Routing Together
byMiguel Araújo
 
PPTX
40m_wAIred_DigitalPlatformRabobank_10022026.pptx
bySimonedeGijt
 
PPTX
Prime Teams – Real-Time Employee Monitoring & Project Management Software
byPrime Teams
 
PDF
Building Smarter AI: 16 RAG Approaches for Accuracy, Memory, and Reasoning Vi...
byVineet Sharma
 
PPTX
How CFD Simulations Support Sustainable Data Center Design Optimization
byWeb Synergies
 
PDF
Ontwikkel sneller en veiliger met GitHub Copilot
byDelta-N
 
PDF
Building production-ready AI Agents with Strands Agents and Amazon Bedrock Ag...
byVadym Kazulkin
 
PDF
Jira, Powered by Enterprise-Grade Intelligence from NeoroTalks.pdf
byNeoro Talks
 
PDF
The Modern Architect's Codex: Design Patterns in .NET 9
byVineet Sharma
 
PDF
ACE Aarhus February 2026: Atlassian news
byDanielEriksen5
 
PDF
Powering Spring AI with Model Context Protocol Integration
byJuarez Junior
 
PDF
SAP ERP to SAP S_4HANA Cloud Private Edition Delta Scope
bychuck78
 
PDF
Routing Guidelines: Unlocking Smarter Query Routing in MySQL Architectures
byMiguel Araújo
 
PPTX
Connecting to MCP Servers in OutSystems Platform
byOzanCali
 
PDF
Lost Android Phone Recovery_ Steps That Work Fast.pdf
byIsabella Reed
 
PPTX
Data Skills for Business Analysts: What You Need to Succeed
byScott W. Ambler
 
PDF
Clean Architecture with Vertical Slice Architecture in .NET 8
byVineet Sharma
 
PDF
On Demand Massage Therapist Booking App Development
byV3CUBE TECHNOLABS
 
PDF
Mastering Zero-Allocation Parsers: A Deep Dive into High-Performance C#
byVineet Sharma
 
PDF
Prompt Engineering vs Content Engineering vs RAG.pdf
byVineet Sharma
 
MySQL Routing Guidelines: Designing Smarter Query Routing Together
byMiguel Araújo
 
40m_wAIred_DigitalPlatformRabobank_10022026.pptx
bySimonedeGijt
 
Prime Teams – Real-Time Employee Monitoring & Project Management Software
byPrime Teams
 
Building Smarter AI: 16 RAG Approaches for Accuracy, Memory, and Reasoning Vi...
byVineet Sharma
 
How CFD Simulations Support Sustainable Data Center Design Optimization
byWeb Synergies
 
Ontwikkel sneller en veiliger met GitHub Copilot
byDelta-N
 
Building production-ready AI Agents with Strands Agents and Amazon Bedrock Ag...
byVadym Kazulkin
 
Jira, Powered by Enterprise-Grade Intelligence from NeoroTalks.pdf
byNeoro Talks
 
The Modern Architect's Codex: Design Patterns in .NET 9
byVineet Sharma
 
ACE Aarhus February 2026: Atlassian news
byDanielEriksen5
 
Powering Spring AI with Model Context Protocol Integration
byJuarez Junior
 
SAP ERP to SAP S_4HANA Cloud Private Edition Delta Scope
bychuck78
 
Routing Guidelines: Unlocking Smarter Query Routing in MySQL Architectures
byMiguel Araújo
 
Connecting to MCP Servers in OutSystems Platform
byOzanCali
 
Lost Android Phone Recovery_ Steps That Work Fast.pdf
byIsabella Reed
 
Data Skills for Business Analysts: What You Need to Succeed
byScott W. Ambler
 
Clean Architecture with Vertical Slice Architecture in .NET 8
byVineet Sharma
 
On Demand Massage Therapist Booking App Development
byV3CUBE TECHNOLABS
 
Mastering Zero-Allocation Parsers: A Deep Dive into High-Performance C#
byVineet Sharma
 
Prompt Engineering vs Content Engineering vs RAG.pdf
byVineet Sharma
 

Christo Kutrovsky - Maximize Data Warehouse Performance with Parallel Queries

  • 1.
    Maximize Data Warehouse Performancewith Parallel Queries Presented by: Christo Kutrovsky
  • 2.
    © 2015 Pythian– Presentation for DOAG Who Am I 2 • Oracle ACE • 15 years in Oracle field • Joined Pythian 2003 • Part of Pythian Consulting Group • Special projects • Performance tuning • Critical services • Presenter at: IOUG, RMOUG, UKOUG, OpenWorld “oracle pinup”
  • 3.
    © 2015 Pythian– Presentation for DOAG3 © 2011 Pythian - Confidential Why Companies Trust Pythian • Recognized Leader: • Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server • Work with over 150 multinational companies such as Forbes.com, Fox Sports, Nordion and Western Union to help manage their complex IT deployments • Expertise: • One of the world’s largest concentrations of dedicated, full-time DBA expertise. Employ 7 Oracle ACEs/ACE Directors. • Hold 7 Specializations under Oracle Platinum Partner program, including Oracle Exadata, Oracle GoldenGate & Oracle RAC. • Global Reach & Scalability: • 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response
  • 4.
    © 2015 Pythian– Presentation for DOAG Agenda 4 • Review Parallel Query concepts • Review Exadata Data Flow capabilities • The Big Sort • The Big Group By • The Big Join • Final Thoughts
  • 5.
  • 6.
    © 2015 Pythian– Presentation for DOAG Oracle Parallel Query 6 • Producer/Consumer model • A PQ slave is either a producer or consumer, never both Figure 8-2 Inter-operation Parallelism and Dynamic Partitioning Oracle® Database VLDB and Partitioning Guide
  • 7.
    © 2015 Pythian– Presentation for DOAG Oracle Parallel Query 7 • Producer/Consumer model • A PQ slave is either a producer or consumer, never both Figure 8-2 Inter-operation Parallelism and Dynamic Partitioning Oracle® Database VLDB and Partitioning Guide
  • 8.
    © 2015 Pythian– Presentation for DOAG Oracle Parallel Query 8 • A PQ slave is either a producer or consumer in relation to other PQ slaves PQ1 PQ2 PQ3 PQ4 PQ5 PQ6 PQ7 PQ8 S G A DB
  • 9.
    © 2015 Pythian– Presentation for DOAG Oracle Parallel Query 9 • But logically, step-wise, when there are two sets it’s flow-through PQ1 PQ2 PQ3 PQ4 PQ5 PQ6 PQ7 PQ8 S G A DB ONE STEP (1)
  • 10.
    © 2015 Pythian– Presentation for DOAG Oracle Parallel Query 10 • QC is always flow-throught PQ1 PQ2 PQ3 PQ4 PQ5 PQ6 PQ7 PQ8 S G A DB S G A QC SQLNET CLIENTONE STEP (2)
  • 11.
    © 2015 Pythian– Presentation for DOAG Oracle Parallel Query 11 • select count(*) from table; PQ1 PQ2 PQ3 PQ4 S G A DB ONE STEP QC SQLNET CLIENT
  • 12.
    © 2015 Pythian– Presentation for DOAG Oracle Parallel Query 12 • Producer/consumer model • A slave is either producer and consumer in relation to other slaves.
  • 13.
  • 14.
    © 2015 Pythian– Presentation for DOAG Multiple Tiers 14 • Every database has multiple tiers • Memory • SAN/NFS – network • SAN Performance • CPU Capacity • Client side connectivity • Each tier has it’s own nuances, properties and bandwidth limitations
  • 15.
    © 2015 Pythian– Presentation for DOAG Exadata X2-2 Data Flow (intra) 15 • Each DB node (compute) and CELL node (storage) • Single dual port 40 Gbit infiniband (QDR) • 3,200 Mb/sec (full-duplex) • Each CELL node • Single dual port 40 Gbit infiniband (QDR) • 3,200 Mb/sec (full-duplex) • 12 x 4 TB SAS drives (7200 rpm) • Total bandwidth: 1,000 - 2,000 Mb/sec • 4 x Sun Flash Arrays • Total bandwidth: 4,100 Mb/sec
  • 16.
    © 2015 Pythian– Presentation for DOAG Exadata Data Flow (External) 16 • SQL Net Clients talk only to DB nodes • Each DB node • 4 x 10 gigabit network ( 4 x 1.08 Gbytes/sec) • 2 x 10 Gigabit network (opt) ( 2 x 1.08 Gbytes/sec) • Alternatively, can use IP over infiniband • Exalogic • RMAN backups • Etc.
  • 17.
    © 2015 Pythian– Presentation for DOAG Exadata Data Flow (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET
  • 18.
    © 2015 Pythian– Presentation for DOAG DISK: 2 GB/sec + FLASH: 4 GB/sec = 6 GB/sec X 7 cells = 42 GB/sec Single Cell (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET CELL 01
  • 19.
    © 2015 Pythian– Presentation for DOAG Cell output (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET CELL to SWITCH: 3.2 GB/sec X 7 cells = 22.4 GB/sec
  • 20.
    © 2015 Pythian– Presentation for DOAG DB Node Consumption (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET SWITCH to DB NODE: 3.2 GB/sec X 4 nodes = 12.8 GB/sec
  • 21.
    © 2015 Pythian– Presentation for DOAG Memory speed (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET DB to SGA (memory) 22 GB/sec (very CPU dependant) X 4 DB = 88 GB/sec
  • 22.
    © 2015 Pythian– Presentation for DOAG DB Node to Client (Half Rack) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET DB to SINGLE SQLNET Client 112 MB/sec (1 GBIT) 1.08 GB/sec (10 GBIT)
  • 23.
    © 2015 Pythian– Presentation for DOAG Exadata Data Flow (External) 23 • Ethernet link aggregation requires • Single conversation runs on a single path • Ethernet standard • One database sessions runs on single TCP connection, which must run on single path • Switches and network cards can implement different hashing algorithms to determine paths • Most are on IP level. Between 2 IPs the same NICs are used, regardless of number of connections
  • 24.
    © 2015 Pythian– Presentation for DOAG Exadata Data Flow (Half Rack) DB4 eth0 DB to THIS Client: 112 MB/sec (theoretical) (18 MB/sec per session) IP: 10.31.33.1 Session Session Session Session Session Session eth1 eth2 eth3 eth0 eth1 eth2 eth3 S w i t c h
  • 25.
    © 2015 Pythian– Presentation for DOAG Exadata Data Flow (External) 25 • As per Exadata Documentation: • Data Load Rate: Up to 10.5 TB/hour • 764 MB/sec per DB node • 3 GB/sec – can your other infrastructure provide such performance?
  • 26.
  • 27.
    © 2015 Pythian– Presentation for DOAG Sample Data 27 • My favorite table – CKK • 39 GiB of data • 10,000,000 rows • id, mod5_id, mod5000_id, dt_dec, dt_pos etc. create table ckk /*parallel 8 */nologging tablespace ckbig as select rownum id, mod(rownum,5) mod5_id, mod(rownum,5000) mod5000_id, sysdate dt_fixed, sysdate - rownum/24/60 dt_dec, sysdate + rownum/24/60 dt_pos, sysdate + ora_hash(rownum,65,535)/24 dt_rand, sysdate+mod(rownum,10) dt_mod10, rpad('x',3500,'x') filler from (select rownum r from dual connect by level <= 10000) r1, (select rownum r from dual connect by level <= 1000) ;
  • 28.
    © 2015 Pythian– Presentation for DOAG Parallel Sort 28 select /*+ parallel(4) */ * from ckk c1 order by id; ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 33G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (ORDER) | :TQ10001 | 10M| 33G| | Q1,01 | P->S | QC (ORDER) | | 3 | SORT ORDER BY | | 10M| 33G| 38G| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 5 | PX SEND RANGE | :TQ10000 | 10M| 33G| | Q1,00 | P->P | RANGE | | 6 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- PQ1 PQ2 PQ3 PQ4 PQ5 – up ID=2.5 M PQ6 – up ID=5.0 M PQ7 – up ID=7.5 M PQ8 – up ID=10 M S G A DB
  • 29.
    © 2015 Pythian– Presentation for DOAG STORAGE FULL (OFFLOADED) DB1 INFINIBAND SWITCHES DB2 DB3 DB4 CELL 01 CELL 02 CELL 03 CELL 04 CELL 05 CELL 06 CELL 07 SGA SGA SGA SGA CLIENT RESULT SET Decompression, Projection and Filter Happens on EVERY CELL NODE regardless of query parallelism. MPP mode.
  • 30.
    © 2015 Pythian– Presentation for DOAG Parallel Sort 30 select dfo_number "d", tq_id as "t", server_type, num_rows,rpad('x',round(num_rows*10/nullif(max(num_rows) over (partition by dfo_number, tq_id, server_type),0)),'x') as "pr", round(bytes/1024/1024) mb, process, instance i,round(ratio_to_report (num_rows) over (partition by dfo_number, tq_id, server_type)*100) as "%", open_time, avg_latency, waits, timeouts,round(bytes/nullif(num_rows,0)) as "b/r” from v$pq_tqstat order by dfo_number, tq_id, server_type desc, process;; d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 372 xxxxxxxxxx 2 QC 1 100 1 0 Producer 2,692,340 xxxxxxxxxx 9,161 P002 1 27 1 0 Producer 2,499,910 xxxxxxxxx 8,506 P002 2 25 1 0 Producer 2,692,340 xxxxxxxxxx 9,161 P003 2 27 1 0 Producer 2,115,410 xxxxxxxx 7,198 P003 1 21 1 0 Consumer 6,146,614 xxxxxxxxxx 20,915 P000 2 61 1 0 Consumer 781,680 x 2,660 P000 1 8 1 0 Consumer 1,968,128 xxx 6,697 P001 2 20 1 0 Consumer 1,103,578 xx 3,755 P001 1 11 1 1 Producer 13 x 0 P000 1 5 1 1 Producer 212 xxxxxxxxxx 1 P000 2 84 1 1 Producer 13 x 0 P001 2 5 1 1 Producer 13 x 0 P001 1 5 1 1 Consumer 200 xxxxxxxxxx 1 QC 1 100
  • 31.
    © 2015 Pythian– Presentation for DOAG Parallel Sort 31 select /*+ parallel(4) */ * from ckk c1 order by id; ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 33G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (ORDER) | :TQ10001 | 10M| 33G| | Q1,01 | P->S | QC (ORDER) | | 3 | SORT ORDER BY | | 10M| 33G| 38G| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 5 | PX SEND RANGE | :TQ10000 | 10M| 33G| | Q1,00 | P->P | RANGE | | 6 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- PQ1 PQ2 PQ3 PQ4 PQ5 – up to ID=6.1 M (6.1 M) PQ6 – up to ID=6.9 M (0.8 M) PQ7 – up to ID=8.8 M (1.9 M) PQ8 – up to ID=10.0 M (1.1M S G A DB
  • 32.
    © 2015 Pythian– Presentation for DOAG The Ranger 32 • The Ranger Bug • The RANGER uses a sample from the data to determine the boundaries for PQ sort operations • Sometimes it’s off by a lot • It will ALWAYS miss when the input data is already somewhat sorted • Limit your scalability – no matter how big your machine is
  • 33.
    © 2015 Pythian– Presentation for DOAG Parallel Sort – random data 33 select /*+ parallel(4) */ id, mod5_id, mod5000_id, dt_fixed, dt_dec, dt_pos, dt_rand, dt_mod10, substr(filler,1,2000) from ckk c1 order by dt_rand; --90 sec d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 372 xxxxxxxxxx 1 QC 1 100 1 0 Producer 2,500,030 xxxxxxxxxx 4,925 P002 1 25 1 0 Producer 2,500,030 xxxxxxxxxx 4,925 P002 2 25 1 0 Producer 2,499,910 xxxxxxxxxx 4,925 P003 1 25 1 0 Producer 2,500,030 xxxxxxxxxx 4,925 P003 2 25 1 0 Consumer 2,424,554 xxxxxxxxx 4,776 P000 2 24 1 0 Consumer 2,576,164 xxxxxxxxx 5,075 P000 1 26 1 0 Consumer 2,272,136 xxxxxxxx 4,476 P001 2 23 1 0 Consumer 2,727,146 xxxxxxxxxx 5,373 P001 1 27 1 1 Producer 218 xxxxxxxxxx 0 P000 1 76 1 1 Producer 23 x 0 P000 2 8 1 1 Producer 23 x 0 P001 2 8 1 1 Producer 23 x 0 P001 1 8 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
  • 34.
    © 2015 Pythian– Presentation for DOAG Parallel Sort – random data 34 select /*+ parallel(32) */ id, mod5_id, mod5000_id, dt_fixed, dt_dec, dt_pos, dt_rand, dt_mod10, substr(filler,1,2000) from ckk c1 order by dt_rand; --10 sec d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 3,872 xxxxxxxxxx 11 QC 1 100 1 0 Producer 312,520 xxxxxxxxxx 616 P016 1 3 1 0 Producer 312,520 xxxxxxxxxx 616 P016 2 3 1 0 Producer 312,520 xxxxxxxxxx 616 P017 1 3 1 0 Producer 312,520 xxxxxxxxxx 616 P017 2 3 ... 1 0 Consumer 151,262 xxx 298 P003 1 2 1 0 Consumer 454,696 xxxxxxxxxx 896 P003 2 5 1 0 Consumer 302,320 xxxxxxx 596 P004 1 3 1 0 Consumer 151,731 xxx 299 P004 2 2 1 0 Consumer 303,429 xxxxxxx 598 P005 1 3 1 0 Consumer 454,650 xxxxxxxxxx 896 P005 2 5 1 0 Consumer 303,259 xxxxxxx 597 P006 2 3 1 0 Consumer 455,172 xxxxxxxxxx 897 P006 1 5 1 0 Consumer 302,945 xxxxxxx 597 P012 1 3 1 0 Consumer 303,318 xxxxxxx 598 P012 2 3 1 0 Consumer 303,245 xxxxxxx 597 P013 1 3 1 0 Consumer 304,029 xxxxxxx 599 P013 2 3 1 0 Consumer 301,937 xxxxxxx 595 P014 2 3 1 0 Consumer 454,903 xxxxxxxxxx 896 P014 1 5 1 0 Consumer 151,887 xxx 299 P015 1 2 1 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
  • 35.
    © 2015 Pythian– Presentation for DOAG Parallel Sort – random data 35 select /*+ parallel(32) */ id, mod5_id, mod5000_id, dt_fixed, dt_dec, dt_pos, dt_rand, dt_mod10, substr(filler,1,2000) from ckk c1 order by id; --20 sec d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 3,872 xxxxxxxxxx 11 QC 1 100 1 0 Producer 240,400 xxxxxx 474 P016 2 2 1 0 Producer 384,640 xxxxxxxxx 758 P016 1 4 1 0 Producer 216,360 xxxxx 426 P017 1 2 1 0 Producer 192,320 xxxx 379 P017 2 2 ... 1 0 Consumer 373,702 xxxx 736 P000 1 4 1 0 Consumer 372,530 xxxx 734 P000 2 4 1 0 Consumer 124,801 x 246 P001 2 1 1 0 Consumer 183,552 xx 361 P001 1 2 1 0 Consumer 734,398 xxxxxxx 1,446 P002 1 7 1 0 Consumer 22,402 44 P002 2 0 1 0 Consumer 191,810 xx 378 P003 1 2 1 0 Consumer 132,990 x 262 P003 2 1 1 0 Consumer 32,640 64 P004 1 0 1 0 Consumer 920,448 xxxxxxxxx 1,813 P004 2 9 1 0 Consumer 167,232 xx 329 P005 1 2 1 0 Consumer 110,466 x 218 P005 2 1 1 0 Consumer 1,055,669 xxxxxxxxxx 2,080 P014 2 11 1 0 Consumer 130,944 x 258 P014 1 1 1 0 Consumer 60,250 x 119 P015 2 1 1 0 Consumer 20,298 40 P015 1 0 ... 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
  • 36.
    © 2015 Pythian– Presentation for DOAG The Ranger issue 36 • Distribution can be much worse • When using analytics or other “input” into the sort operation that is pre-sorted • Often 90% goes to one slave, 10% to remaining • “Been worked on” – I am told • Problem made even worse due to lack of large buffers
  • 37.
    © 2015 Pythian– Presentation for DOAG Writing to disk – lack of buffering 37 • When a slave is writing to disk – it cannot receive any messages or can receive a very small number of messages • Results in slowdowns when Oracle has to write to TEMP • Massive slowdown when only a few slaves have to write to TEMP • All producers have to wait • Significant slowdown when all slaves have to write to TEMP • Cannot parallelise sufficiently to leverage available disk bandwidth
  • 38.
  • 39.
    © 2015 Pythian– Presentation for DOAG Group By 39 select /*+ parallel(4) */ mod5_id, count(*) from ckk group by mod5_id; --------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| TQ |IN-OUT| PQ Distrib | --------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 5 | 15 | | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 5 | 15 | Q1,01 | P->S | QC (RAND) | | 3 | HASH GROUP BY | | 5 | 15 | Q1,01 | PCWP | | | 4 | PX RECEIVE | | 5 | 15 | Q1,01 | PCWP | | | 5 | PX SEND HASH | :TQ10000 | 5 | 15 | Q1,00 | P->P | HASH | | 6 | HASH GROUP BY | | 5 | 15 | Q1,00 | PCWP | | | 7 | PX BLOCK ITERATOR | | 10M| 28M| Q1,00 | PCWC | | | 8 | TABLE ACCESS STORAGE FULL| CKK | 10M| 28M| Q1,00 | PCWP | | --------------------------------------------------------------------------------------------------- select /*+ parallel(4) NO_GBY_PUSHDOWN*/ mod5_id, count(*) from ckk group by mod5_id; -------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| TQ |IN-OUT| PQ Distrib | -------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 5 | 15 | | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 5 | 15 | Q1,01 | P->S | QC (RAND) | | 3 | HASH GROUP BY | | 5 | 15 | Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 28M| Q1,01 | PCWP | | | 5 | PX SEND HASH | :TQ10000 | 10M| 28M| Q1,00 | P->P | HASH | | 6 | PX BLOCK ITERATOR | | 10M| 28M| Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK | 10M| 28M| Q1,00 | PCWP | | --------------------------------------------------------------------------------------------------
  • 40.
    © 2015 Pythian– Presentation for DOAG Oracle Parallel Query 40 • But logically, step-wise, when there are two sets it’s flow-through PQ1 PQ2 PQ3 PQ4 S G A DB STEP (1) GROUP PGA GROUP PGA GROUP PGA GROUP PGA PQ5 PQ6 PQ7 PQ8 GROUP PGA GROUP PGA GROUP PGA GROUP PGA S G A QCSTEP (2)
  • 41.
    © 2015 Pythian– Presentation for DOAG Group by – small result set 41 select /*+ parallel(4) */ mod5_id, count(*) from ckk group by mod5_id; -- 4.8 sec d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 5 xxxxxxxxxx 0 P002 2 25 1 0 Producer 5 xxxxxxxxxx 0 P002 1 25 1 0 Producer 5 xxxxxxxxxx 0 P003 2 25 1 0 Producer 5 xxxxxxxxxx 0 P003 1 25 1 0 Consumer 4 xxxxx 0 P000 2 20 1 0 Consumer 8 xxxxxxxxxx 0 P000 1 40 1 0 Consumer 4 xxxxx 0 P001 2 20 1 0 Consumer 4 xxxxx 0 P001 1 20 1 1 Producer 2 xxxxxxxxxx 0 P000 1 40 1 1 Producer 1 xxxxx 0 P000 2 20 1 1 Producer 1 xxxxx 0 P001 1 20 1 1 Producer 1 xxxxx 0 P001 2 20 1 1 Consumer 5 xxxxxxxxxx 0 QC 1 100
  • 42.
    © 2015 Pythian– Presentation for DOAG Group by – small result set 42 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 2,692,340 xxxxxxxxxx 10 P004 2 27 1 0 Producer 2,307,600 xxxxxxxxx 8 P004 1 23 1 0 Producer 2,500,030 xxxxxxxxx 9 P005 2 25 1 0 Producer 2,500,030 xxxxxxxxx 9 P005 1 25 1 0 Consumer 4,000,000 xxxxxxxxxx 15 P002 2 40 1 0 Consumer 2,000,000 xxxxx 8 P002 1 20 1 0 Consumer 2,000,000 xxxxx 6 P003 2 20 1 0 Consumer 2,000,000 xxxxx 8 P003 1 20 1 1 Producer 1 xxxxx 0 P002 1 20 1 1 Producer 2 xxxxxxxxxx 0 P002 2 40 1 1 Producer 1 xxxxx 0 P003 1 20 1 1 Producer 1 xxxxx 0 P003 2 20 1 1 Consumer 5 xxxxxxxxxx 0 QC 1 100 select /*+ parallel(4) NO_GBY_PUSHDOWN*/ mod5_id, count(*) from ckk group by mod5_id; -- 4.8 sec
  • 43.
    © 2015 Pythian– Presentation for DOAG Group by – small result set 43 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 264,440 xxxxxx 1 P016 2 3 1 0 Producer 312,520 xxxxxxxx 1 P016 1 3 1 0 Producer 240,400 xxxxxx 1 P017 1 2 1 0 Producer 360,600 xxxxxxxxx 1 P017 2 4 ... 1 0 Consumer 0 0 P000 2 0 1 0 Consumer 2,000,000 xxxxxxxxxx 8 P000 1 20 1 0 Consumer 0 0 P001 2 0 ... 1 0 Consumer 0 0 P004 1 0 1 0 Consumer 0 0 P005 2 0 1 0 Consumer 2,000,000 xxxxxxxxxx 6 P005 1 20 1 0 Consumer 0 0 P006 2 0 1 0 Consumer 0 0 P006 1 0 1 0 Consumer 0 0 P007 2 0 1 0 Consumer 2,000,000 xxxxxxxxxx 8 P007 1 20 1 0 Consumer 2,000,000 xxxxxxxxxx 8 P008 1 20 1 0 Consumer 2,000,000 xxxxxxxxxx 8 P010 2 20 ... select /*+ parallel(32) NO_GBY_PUSHDOWN*/ mod5_id, count(*) from ckk group by mod5_id; -- 4.8 sec
  • 44.
    © 2015 Pythian– Presentation for DOAG Group by – small result set 44 • When the result set from a group by is small – there are a lot of optimisations that can kick in • PUSHDOWN saves on PQ traffic • Receiving end has very little work to do • Very efficient to parallelise
  • 45.
    © 2015 Pythian– Presentation for DOAG select /*+ parallel(4) */ id, count(*) from ckk group by id; -- 4.8 sec Group by – large result set HASH 45 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 2,500,030 xxxxxxxx 17 P002 2 25 1 0 Producer 2,115,410 xxxxxx 14 P002 1 21 1 0 Producer 2,115,410 xxxxxx 14 P003 2 21 1 0 Producer 3,269,150 xxxxxxxxxx 21 P003 1 33 1 0 Consumer 2,499,213 xxxxxxxxxx 16 P000 1 25 1 0 Consumer 2,499,807 xxxxxxxxxx 16 P000 2 25 1 0 Consumer 2,500,981 xxxxxxxxxx 16 P001 1 25 1 0 Consumer 2,499,999 xxxxxxxxxx 16 P001 2 25 1 1 Producer 1,465 xxxxx 0 P000 2 17 1 1 Producer 2,929 xxxxxxxxxx 0 P000 1 33 1 1 Producer 2,928 xxxxxxxxxx 0 P001 1 33 1 1 Producer 1,464 xxxxx 0 P001 2 17 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
  • 46.
    © 2015 Pythian– Presentation for DOAG select /*+ parallel(4) */ id, count(*) from ckk group by id order by id; -- 6.5 sec Group by – large result set SORT 46 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 372 xxxxxxxxxx 0 QC 1 100 1 0 Producer 2,499,910 xxxxxxxxx 17 P002 1 25 1 0 Producer 2,692,340 xxxxxxxxxx 18 P002 2 27 1 0 Producer 2,500,030 xxxxxxxxx 17 P003 2 25 1 0 Producer 2,307,720 xxxxxxxxx 15 P003 1 23 1 0 Consumer 6,154,678 xxxxxxxxxx 40 P000 2 62 1 0 Consumer 793,712 x 5 P000 1 8 1 0 Consumer 1,095,514 xx 7 P001 1 11 1 0 Consumer 1,956,096 xxx 13 P001 2 20 1 1 Producer 4,391 xxxxxxxx 0 P000 1 24 1 1 Producer 5,379 xxxxxxxxxx 0 P000 2 29 1 1 Producer 4,391 xxxxxxxx 0 P001 2 24 1 1 Producer 4,391 xxxxxxxx 0 P001 1 24 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
  • 47.
    © 2015 Pythian– Presentation for DOAG select /*+ parallel(32) */ id, count(*) from ckk group by id; -- 4.8 sec Group by – large result set HASH 47 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 360,600 xxxxxxxxx 2 P016 1 4 1 0 Producer 240,400 xxxxxx 2 P016 2 2 1 0 Producer 312,520 xxxxxxxx 2 P017 1 3 1 0 Producer 336,560 xxxxxxxxx 2 P017 2 3 ... 1 0 Consumer 312,829 xxxxxxxxxx 2 P000 2 3 1 0 Consumer 311,690 xxxxxxxxxx 2 P000 1 3 1 0 Consumer 312,371 xxxxxxxxxx 2 P001 1 3 1 0 Consumer 312,854 xxxxxxxxxx 2 P001 2 3 1 0 Consumer 312,255 xxxxxxxxxx 2 P002 2 3 1 0 Consumer 312,321 xxxxxxxxxx 2 P002 1 3 1 0 Consumer 313,241 xxxxxxxxxx 2 P003 2 3 1 0 Consumer 312,605 xxxxxxxxxx 2 P003 1 3 1 0 Consumer 312,202 xxxxxxxxxx 2 P004 1 3 1 0 Consumer 311,294 xxxxxxxxxx 2 P004 2 3 1 0 Consumer 312,338 xxxxxxxxxx 2 P005 1 3 1 0 Consumer 312,206 xxxxxxxxxx 2 P005 2 3 1 0 Consumer 312,898 xxxxxxxxxx 2 P006 1 3 1 0 Consumer 312,817 xxxxxxxxxx 2 P006 2 3 1 0 Consumer 312,291 xxxxxxxxxx 2 P007 1 3 1 0 Consumer 312,872 xxxxxxxxxx 2 P007 2 3 ... 1 1 Consumer 200 xxxxxxxxxx 0 QC 1 100
  • 48.
    © 2015 Pythian– Presentation for DOAG Group by – large result set HASH 48 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Producer 312,520 xxxxxxxxxx 1,047 P016 1 3 1 0 Producer 312,520 xxxxxxxxxx 1,047 P016 2 3 1 0 Producer 312,520 xxxxxxxxxx 1,047 P017 1 3 ... 1 0 Consumer 311,603 xxxxxxxxxx 1,044 P000 1 3 1 0 Consumer 312,470 xxxxxxxxxx 1,047 P000 2 3 1 0 Consumer 312,484 xxxxxxxxxx 1,047 P001 2 3 1 0 Consumer 312,662 xxxxxxxxxx 1,048 P001 1 3 1 0 Consumer 312,341 xxxxxxxxxx 1,047 P002 2 3 1 0 Consumer 312,692 xxxxxxxxxx 1,048 P002 1 3 1 0 Consumer 313,114 xxxxxxxxxx 1,049 P003 2 3 1 0 Consumer 311,889 xxxxxxxxxx 1,045 P003 1 3 1 0 Consumer 312,323 xxxxxxxxxx 1,047 P004 1 3 1 0 Consumer 313,310 xxxxxxxxxx 1,050 P004 2 3 1 0 Consumer 312,715 xxxxxxxxxx 1,048 P005 2 3 1 0 Consumer 313,394 xxxxxxxxxx 1,050 P005 1 3 1 0 Consumer 312,718 xxxxxxxxxx 1,048 P006 1 3 1 0 Consumer 312,090 xxxxxxxxxx 1,046 P006 2 3 1 0 Consumer 312,720 xxxxxxxxxx 1,048 P007 1 3 1 0 Consumer 311,889 xxxxxxxxxx 1,045 P007 2 3 2 ... 1 1 Consumer 200 xxxxxxxxxx 1 QC 1 100 select /*+ parallel(32) */ id, filler, count(*) from ckk group by id,filler; -- 18 sec
  • 49.
    © 2015 Pythian– Presentation for DOAG Group by – large result set SORT 49 d t SERVER_TYPE NUM_ROWS pr MB PROCESS I % 1 0 Ranger 3,872 xxxxxxxxxx 23 QC 1 100 1 0 Producer 288,480 xxxxxxx 967 P016 2 3 1 0 Producer 288,480 xxxxxxx 967 P016 1 3 1 0 Producer 336,560 xxxxxxxx 1,128 P017 2 3 1 0 Producer 240,400 xxxxxx 806 P017 1 2 ... 1 0 Consumer 722,115 xxxxxx 2,420 P002 2 7 1 0 Consumer 16,386 55 P002 1 0 1 0 Consumer 210,112 xx 704 P003 2 2 1 0 Consumer 112,510 x 377 P003 1 1 1 0 Consumer 940,928 xxxxxxxx 3,153 P004 1 9 1 0 Consumer 34,688 116 P004 2 0 1 0 Consumer 171,392 xx 574 P005 2 2 1 0 Consumer 110,466 x 370 P005 1 1 1 0 Consumer 163,658 x 548 P014 2 2 1 0 Consumer 1,113,523 xxxxxxxxxx 3,732 P014 1 11 1 0 Consumer 11,100 37 P015 1 0 1 0 Consumer 4,150 14 P015 2 0 ... 1 1 Consumer 200 xxxxxxxxxx 1 QC 1 100 select /*+ parallel(32) */ id, filler, count(*) from ckk group by id,filler order by id,filler; -- 41 sec
  • 50.
    © 2015 Pythian– Presentation for DOAG Group by – summary 50 • Theoretical performance is reachable • SORT based operations have major issues • Especially with sorted inputs, such as from analytics • model clause can help – partition by is hash based • Partial disk staging causes major slowdowns • Balanced operation is critical
  • 51.
    © 2015 Pythian– Presentation for DOAG Group by – Partitioning 51 • What about partitioning? create table ckk_p (id, mod5_id, mod5000_id, dt_fixed, dt_dec, dt_pos, dt_rand, dt_mod10, filler) partition by hash (id) partitions 32 tablespace ckbig nologging parallel 4 as select * from ckk;
  • 52.
    © 2015 Pythian– Presentation for DOAG Group by – with partitioning 52 select /*+ parallel(4) */ id,count(*) from ckk_p group by id,filler; ------------------------------------------------------------------------------------------------------------------------ | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | Pstart| Pstop | TQ |IN-OUT| PQ Distrib | ------------------------------------------------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | 7071K| 23G| | | | | | | | 1 | PX COORDINATOR | | | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | 7071K| 23G| | | | Q1,00 | P->S | QC (RAND) | | 3 | PX PARTITION HASH ALL | | 7071K| 23G| | 1 | 32 | Q1,00 | PCWC | | | 4 | HASH GROUP BY | | 7071K| 23G| 38G| | | Q1,00 | PCWP | | | 5 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 32G| | 1 | 32 | Q1,00 | PCWP | | ------------------------------------------------------------------------------------------------------------------------ PQ1 PQ2 PQ3 PQ4 S G A STEP (1) GROUP PGA GROUP PGA GROUP PGA GROUP PGA QC LIST P1 P2 P3 P4 P5 P6 P7 P8
  • 53.
    © 2015 Pythian– Presentation for DOAG select /*+ parallel(64) */ * from ckk c1 order by id; ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 7071K| 23G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 7071K| 23G| | Q1,01 | P->S | QC (RAND) | | 3 | HASH GROUP BY | | 7071K| 23G| 38G| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 32G| | Q1,01 | PCWP | | | 5 | PX SEND HASH | :TQ10000 | 10M| 32G| | Q1,00 | P->P | HASH | | 6 | PX BLOCK ITERATOR | | 10M| 32G| | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 32G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- Group by – enough partitions 53 PQ1 PQ2 PQ3 PQ4 S G A STEP (1) GROUP PGA GROUP PGA GROUP PGA GROUP PGA QC LIST P1 P2 P3 P4 P5 P6 P7 P8
  • 54.
    © 2015 Pythian– Presentation for DOAG Analytics don’t speak parallel 54 select /*+ parallel(4) */ id, row_number() over (partition by id,filler order by null) r from ckk_p ; ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 32G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 32G| | Q1,01 | P->S | QC (RAND) | | 3 | WINDOW SORT | | 10M| 32G| 38G| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 32G| | Q1,01 | PCWP | | | 5 | PX SEND HASH | :TQ10000 | 10M| 32G| | Q1,00 | P->P | HASH | | 6 | PX BLOCK ITERATOR | | 10M| 32G| | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 32G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- select /*+ NO_parallel */ id, row_number() over (partition by id,filler order by null) r from ckk_p ; -------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | Pstart| Pstop | -------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 32G| | | | | 1 | PARTITION HASH ALL | | 10M| 32G| | 1 | 32 | | 2 | WINDOW SORT | | 10M| 32G| 38G| | | | 3 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 32G| | 1 | 32 | --------------------------------------------------------------------------------------
  • 55.
  • 56.
    © 2015 Pythian– Presentation for DOAG Test Data 56 • Copy of ckk – ckk2 • ckk_m5 – 5 rows dimension create table ckk_m5 as select rownum mod5_id, rpad('x',1000,'x') filler from dual connect by level <=5; alter table ckk_m5 modify (mod5_id not null);
  • 57.
    © 2015 Pythian– Presentation for DOAG Simple Hash Join select /*+ parallel(4) */ * from ckk c1 join ckk_m5 m5 on (c1.mod5_id = m5.mod5_id); -------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| TQ |IN-OUT| PQ Distrib | -------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 42G| | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 42G| Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN | | 10M| 42G| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 5 | 5020 | Q1,01 | PCWP | | | 5 | PX SEND BROADCAST | :TQ10000 | 5 | 5020 | Q1,00 | P->P | BROADCAST | | 6 | PX BLOCK ITERATOR | | 5 | 5020 | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK_M5 | 5 | 5020 | Q1,00 | PCWP | | | 8 | PX BLOCK ITERATOR | | 10M| 33G| Q1,01 | PCWC | | | 9 | TABLE ACCESS STORAGE FULL | CKK | 10M| 33G| Q1,01 | PCWP | | --------------------------------------------------------------------------------------------------
  • 58.
    © 2015 Pythian– Presentation for DOAG Simple Hash Join 58 PQ1 PQ2 PQ3 PQ4 S G A STEP (1) PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A QC CKK _M5 0 GiB All Hash Areas: 0 GiB
  • 59.
    © 2015 Pythian– Presentation for DOAG Simple Hash Join 59 PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A STEP (2) QC CKK (39 GiB)
  • 60.
    © 2015 Pythian– Presentation for DOAG Big Hash Join select /*+ parallel(4) */ * from ckk c1join ckk2 c2 on (c1.id = c2.id) ------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp |IN-OUT| PQ Distrib | ------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10002 | 10M| 66G| | P->S | QC (RAND) | |* 3 | HASH JOIN BUFFERED | | 10M| 66G| 8502M| PCWP | | | 4 | PX RECEIVE | | 10M| 33G| | PCWP | | | 5 | PX SEND HASH | :TQ10000 | 10M| 33G| | P->P | HASH | | 6 | PX BLOCK ITERATOR | | 10M| 33G| | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | PCWP | | | 8 | PX RECEIVE | | 10M| 33G| | PCWP | | | 9 | PX SEND HASH | :TQ10001 | 10M| 33G| | P->P | HASH | | 10 | PX BLOCK ITERATOR | | 10M| 33G| | PCWC | | | 11 | TABLE ACCESS STORAGE FULL| CKK2 | 10M| 33G| | PCWP | | -------------------------------------------------------------------------------------------------
  • 61.
    © 2015 Pythian– Presentation for DOAG Big Hash Join 61 PQ1 PQ2 PQ3 PQ4 S G A STEP (1) PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A QC CKK 39 GiB All Hash Areas: 39 GiB
  • 62.
    © 2015 Pythian– Presentation for DOAG Big Hash Join 62 PQ1 PQ2 PQ3 PQ4 S G A STEP (2) PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A QC CKK 2 39 GiB All Hash Areas: 78 GiB
  • 63.
    © 2015 Pythian– Presentation for DOAG Simple Hash Join 63 PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A STEP (3) QC
  • 64.
    © 2015 Pythian– Presentation for DOAG Hash Join summary 64 • HASH JOINS with small tables are extremely fast • Uses BROADCAST to send copy of table to every slave • As much PGA work area as parallel degree * small table size • HASH JOINS with big tables can be extremely expensive • Almost entire result set written to temp • Unless – partitioning
  • 65.
    © 2015 Pythian– Presentation for DOAG Semi-Partitioned Hash Join select /*+ parallel(32) */ * from ckk c1 join ckk_p c2 on (c1.id = c2.id) ----------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ----------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 66G| | Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN | | 10M| 66G| 1062M| Q1,01 | PCWP | | | 4 | PART JOIN FILTER CREATE | :BF0000 | 10M| 33G| | Q1,01 | PCWP | | | 5 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 6 | PX SEND PARTITION (KEY) | :TQ10000 | 10M| 33G| | Q1,00 | P->P | PART (KEY) | | 7 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 8 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | | 9 | PX PARTITION HASH JOIN-FILTER| | 10M| 33G| | Q1,01 | PCWC | | | 10 | TABLE ACCESS STORAGE FULL | CKK_P | 10M| 33G| | Q1,01 | PCWP | | -----------------------------------------------------------------------------------------------------------
  • 66.
    © 2015 Pythian– Presentation for DOAG Semi Partitioned Hash Join 66 PQ1 PQ2 PQ3 PQ4 S G A STEP (1) PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A QC CKK 39 GiB All Hash Areas: 39 GiB
  • 67.
    © 2015 Pythian– Presentation for DOAG Semi Partitioned Hash Join 67 PQ5 PQ6 PQ7 PQ8 HASH PGA HASH PGA HASH PGA HASH PGA S G A STEP (2) QC CKK_P (39 GiB) LIST P1 P2 P3 P4 P5 P6 P7 P8
  • 68.
    © 2015 Pythian– Presentation for DOAG Semi Partitioned Hash Join 68 • Dramatically reduces HASH Area sizes • Very likely to fit in RAM – no disk • Reduces from 3 steps to 2 steps • Always reads the non-partitioned table into HASH areas • Each slave reads it’s own partition
  • 69.
    © 2015 Pythian– Presentation for DOAG Partitioned Hash Join select /*+ parallel(4) */ * from ckk_p c1 join ckk_p c2 on (c1.id = c2.id) -------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | -------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | 10M| 66G| | Q1,00 | P->S | QC (RAND) | | 3 | PX PARTITION HASH ALL | | 10M| 66G| | Q1,00 | PCWC | | |* 4 | HASH JOIN | | 10M| 66G| 8502M| Q1,00 | PCWP | | | 5 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 33G| | Q1,00 | PCWP | | | 6 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 33G| | Q1,00 | PCWP | | --------------------------------------------------------------------------------------------------------
  • 70.
    © 2015 Pythian– Presentation for DOAG Partitioned Hash Join select /*+ parallel(4) */ * from ckk_p c1 join ckk_p c2 on (c1.id = c2.id) -------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | -------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | 10M| 66G| | Q1,00 | P->S | QC (RAND) | | 3 | PX PARTITION HASH ALL | | 10M| 66G| | Q1,00 | PCWC | | |* 4 | HASH JOIN | | 10M| 66G| 8502M| Q1,00 | PCWP | | | 5 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 33G| | Q1,00 | PCWP | | | 6 | TABLE ACCESS STORAGE FULL| CKK_P | 10M| 33G| | Q1,00 | PCWP | | --------------------------------------------------------------------------------------------------------
  • 71.
    © 2015 Pythian– Presentation for DOAG Switching places select /*+ parallel(32) */ * from ckk_p c1 join ckk c2 on (c1.id = c2.id); ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 66G| | Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN BUFFERED | | 10M| 66G| 1062M| Q1,01 | PCWP | | | 4 | PX PARTITION HASH ALL | | 10M| 33G| | Q1,01 | PCWC | | | 5 | TABLE ACCESS STORAGE FULL | CKK_P | 10M| 33G| | Q1,01 | PCWP | | | 6 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 7 | PX SEND PARTITION (KEY) | :TQ10000 | 10M| 33G| | Q1,00 | P->P | PART (KEY) | | 8 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 9 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- select /*+ parallel(32) LEADING(c2) */ * from ckk_p c1 join ckk c2 on (c1.id = c2.id); ----------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ----------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 66G| | Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN | | 10M| 66G| 1062M| Q1,01 | PCWP | | | 4 | PART JOIN FILTER CREATE | :BF0000 | 10M| 33G| | Q1,01 | PCWP | | | 5 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 6 | PX SEND PARTITION (KEY) | :TQ10000 | 10M| 33G| | Q1,00 | P->P | PART (KEY) | | 7 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 8 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | | 9 | PX PARTITION HASH JOIN-FILTER| | 10M| 33G| | Q1,01 | PCWC | | | 10 | TABLE ACCESS STORAGE FULL | CKK_P | 10M| 33G| | Q1,01 | PCWP | | -----------------------------------------------------------------------------------------------------------
  • 72.
    © 2015 Pythian– Presentation for DOAG Outer Joins select /*+ parallel(32) */ * from ckk_p c1 left join ckk c2 on (c1.id = c2.id); ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 66G| | Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN OUTER BUFFERED | | 10M| 66G| 1062M| Q1,01 | PCWP | | | 4 | PX PARTITION HASH ALL | | 10M| 33G| | Q1,01 | PCWC | | | 5 | TABLE ACCESS STORAGE FULL | CKK_P | 10M| 33G| | Q1,01 | PCWP | | | 6 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 7 | PX SEND PARTITION (KEY) | :TQ10000 | 10M| 33G| | Q1,00 | P->P | PART (KEY) | | 8 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 9 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | ---------------------------------------------------------------------------------------------------------- select /*+ parallel(32) SWAP_JOIN_INPUTS(c2)*/ * from ckk_p c1 left join ckk c2 on (c1.id = c2.id); ---------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes|E-Temp | TQ |IN-OUT| PQ Distrib | ---------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10M| 66G| | | | | | 1 | PX COORDINATOR | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10001 | 10M| 66G| | Q1,01 | P->S | QC (RAND) | |* 3 | HASH JOIN RIGHT OUTER | | 10M| 66G| 1062M| Q1,01 | PCWP | | | 4 | PX RECEIVE | | 10M| 33G| | Q1,01 | PCWP | | | 5 | PX SEND PARTITION (KEY) | :TQ10000 | 10M| 33G| | Q1,00 | P->P | PART (KEY) | | 6 | PX BLOCK ITERATOR | | 10M| 33G| | Q1,00 | PCWC | | | 7 | TABLE ACCESS STORAGE FULL| CKK | 10M| 33G| | Q1,00 | PCWP | | | 8 | PX PARTITION HASH ALL | | 10M| 33G| | Q1,01 | PCWC | | | 9 | TABLE ACCESS STORAGE FULL | CKK_P | 10M| 33G| | Q1,01 | PCWP | | ----------------------------------------------------------------------------------------------------------
  • 73.
    73 The Big Join– with Index
  • 74.
    © 2015 Pythian– Presentation for DOAG Adding a few indexes 74 create index ckk$id on ckk(id) parallel 32 tablespace ckbig nologging; --6.5 sec create index ckk$mod5000_id on ckk(mod5000_id) parallel 32 tablespace ckbig nologging compress; --5.8 sec create index ckk_p$mod5000_id on ckk_p(mod5000_id) parallel 32 tablespace ckbig nologging compress local; --16 sec
  • 75.
    © 2015 Pythian– Presentation for DOAG Direct index access select /*+ parallel(4) */ * from ckk where mod5000_id=100; ----------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| ----------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 2000 | 6941K| | 1 | TABLE ACCESS BY INDEX ROWID| CKK | 2000 | 6941K| |* 2 | INDEX RANGE SCAN | CKK$MOD5000_ID | 2000 | | ----------------------------------------------------------------------- select /*+ parallel(4) */ * from ckk_p where mod5000_id=100; -------------------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| TQ |IN-OUT| PQ Distrib | -------------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 2000 | 6941K| | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | 2000 | 6941K| Q1,00 | P->S | QC (RAND) | | 3 | PX PARTITION HASH ALL | | 2000 | 6941K| Q1,00 | PCWC | | | 4 | TABLE ACCESS BY LOCAL INDEX ROWID| CKK_P | 2000 | 6941K| Q1,00 | PCWP | | |* 5 | INDEX RANGE SCAN | CKK_P$MOD5000_ID | 2000 | | Q1,00 | PCWP | | --------------------------------------------------------------------------------------------------------------
  • 76.
    © 2015 Pythian– Presentation for DOAG Join with Index select /*+ parallel(4) */ * from ckk c1 join ckk2 c2 on (c1.id = c2.id) where c2.mod5000_id=5; ------------------------------------------------------------------------------------------------- | Id | Operation | Name | E-Rows |E-Bytes| TQ |IN-OUT| PQ Distrib | ------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 2017 | 13M| | | | | 1 | PX COORDINATOR | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | | | Q1,00 | P->S | QC (RAND) | | 3 | NESTED LOOPS | | | | Q1,00 | PCWP | | | 4 | NESTED LOOPS | | 2017 | 13M| Q1,00 | PCWP | | | 5 | PX BLOCK ITERATOR | | | | Q1,00 | PCWC | | |* 6 | TABLE ACCESS STORAGE FULL| CKK2 | 2000 | 6941K| Q1,00 | PCWP | | |* 7 | INDEX RANGE SCAN | CKK$ID | 1 | | Q1,00 | PCWP | | | 8 | TABLE ACCESS BY INDEX ROWID| CKK | 1 | 3554 | Q1,00 | PCWP | | -------------------------------------------------------------------------------------------------
  • 77.
    © 2015 Pythian– Presentation for DOAG Indexes summary 77 • Parallel access on direct value queries only on partitioned tables and on local indexes • Parallel access on global or non-partitioned only from joins (multiple different values)
  • 78.
    © 2015 Pythian– Presentation for DOAG Final Thoughts 78 • Exadata allows Parallel Query to make sense • High Bandwidth everywhere • Very efficient table scans • Optimiser sometimes picks up wrong access plan • There are deficiencies in SORT operations – balancing the workloads • There are deficiencies in handling disk write operations in PQ Slaves • Can be extremely fast under the right circumstances
  • 79.
    © 2009/2010 Pythian- Presentation for ABC Company79 The End Thank You Questions? I blog at http://www.pythian.com/news/author/kutrovsky/

Editor's Notes

  • #4 - Successful growing business for more than 14 years - Served many customers with complex requirements/infrastructure just like yours. - Operate globally for 24 x 7 “always awake” services