Tuning for Oracle RAC
Wait Events
Confio Software

12/4/2013

1
Focus on the buffer cache and its
operations
• Monitor for all the same waits as with single instances
• In addition, monitor the buffer cache and its operation (the
amount of requests going back and forth over the RAC
interconnect)
• The buffer cache is global across all instances in the cluster
– When a process in a RAC database needs to modify or read
data, Oracle first checks to see if it already exists in the local
buffer cache
– If the data isn’t in the local buffer cache, the global buffer cache
is reviewed to see if another instance already has it in their
buffer cache
– If the data is in another instance, the remote instance will send
the data to the local instance via the high-speed interconnect,
thus avoiding a disk read

12/4/2013

2
Focus on the buffer cache and its
operations
• The most common wait events related to the
global buffer cache are:
– gc cr request
– gc buffer busy

12/4/2013

3
gc cr request waits
• Specifies the time it takes to retrieve the data from the remote
cache
– Also known as global cache cr request in Oracle 9i and prior

• Possible causes:
– RAC traffic using slow connection
• Sometimes Oracle may not pick the correct interconnect to transfer data
between instances and instead route traffic over a slower public network
• Verify which network is being used for RAC traffic using oradebug command:

SQL> oradebug setmypid
SQL> oradebug ipc
• Dumps a trace file to location specified by the user_dump_dest Oracle
parameter containing information about the network and protocols being
used for the RAC interconnect

– Inefficient queries
• Poorly tuned queries will increase the amount of data blocks requested by an
Oracle session
• The more blocks requested typically means the more often a block will need
to be read from a remote instance via the interconnect
12/4/2013

4
gc buffer busy acquire and gc
buffer busy release waits
•

Specify the time the remote instance locally spends accessing the requested data
block
– In Oracle 11g you will see gc buffer busy acquire wait event when the global cache open
request originated from the local instance and gc buffer busy release when the open request
originated from a remote instance
– In Oracle 10g these two wait events were represented in a single gc buffer busy wait
– In Oracle 9i these two wait events were represented in a single global cache buffer busy wait

•

Very similar to the buffer busy wait events in a single-instance database and often
the result of:
– Hot Blocks
•
•
•
•

Multiple sessions may be requesting a block that is either not in buffer cache or is in an incompatible
mode
Deleting some of the hot rows and re-inserting them back into the table may alleviate the problem
Most of the time the rows will be placed into a different block and reduce contention on the block
May also need to adjust the pctfree and/or pctused parameters for the table to ensure the rows are
placed into a different block

– Inefficient Queries
•
•

12/4/2013

The more blocks requested from the buffer cache the more likelihood of a session having to wait for
other sessions
Tuning queries to access fewer blocks will often result in less contention for the same block

5
Monitor with Ignite
for Oracle RAC
• Measures the performance for the entire RAC as well as
individual RAC nodes at a query level to help you find:
– Imbalance between RAC nodes
– Excessive overhead for cache access between nodes
– Improper RAC configuration leading to degraded performance

12/4/2013

6
Find Root Cause of Oracle RAC
Waits Quickly
• Improve database performance by 65% with
Confio Ignite for Oracle RAC
– Installs in minutes
– Puts no load on monitored servers
– Identify bottlenecks and root causes of Oracle RAC
waits like these in minutes

Download Ignite trial for free
www.confio.com/freetrial
12/4/2013

7

Tuning for Oracle RAC Wait Events

  • 1.
    Tuning for OracleRAC Wait Events Confio Software 12/4/2013 1
  • 2.
    Focus on thebuffer cache and its operations • Monitor for all the same waits as with single instances • In addition, monitor the buffer cache and its operation (the amount of requests going back and forth over the RAC interconnect) • The buffer cache is global across all instances in the cluster – When a process in a RAC database needs to modify or read data, Oracle first checks to see if it already exists in the local buffer cache – If the data isn’t in the local buffer cache, the global buffer cache is reviewed to see if another instance already has it in their buffer cache – If the data is in another instance, the remote instance will send the data to the local instance via the high-speed interconnect, thus avoiding a disk read 12/4/2013 2
  • 3.
    Focus on thebuffer cache and its operations • The most common wait events related to the global buffer cache are: – gc cr request – gc buffer busy 12/4/2013 3
  • 4.
    gc cr requestwaits • Specifies the time it takes to retrieve the data from the remote cache – Also known as global cache cr request in Oracle 9i and prior • Possible causes: – RAC traffic using slow connection • Sometimes Oracle may not pick the correct interconnect to transfer data between instances and instead route traffic over a slower public network • Verify which network is being used for RAC traffic using oradebug command: SQL> oradebug setmypid SQL> oradebug ipc • Dumps a trace file to location specified by the user_dump_dest Oracle parameter containing information about the network and protocols being used for the RAC interconnect – Inefficient queries • Poorly tuned queries will increase the amount of data blocks requested by an Oracle session • The more blocks requested typically means the more often a block will need to be read from a remote instance via the interconnect 12/4/2013 4
  • 5.
    gc buffer busyacquire and gc buffer busy release waits • Specify the time the remote instance locally spends accessing the requested data block – In Oracle 11g you will see gc buffer busy acquire wait event when the global cache open request originated from the local instance and gc buffer busy release when the open request originated from a remote instance – In Oracle 10g these two wait events were represented in a single gc buffer busy wait – In Oracle 9i these two wait events were represented in a single global cache buffer busy wait • Very similar to the buffer busy wait events in a single-instance database and often the result of: – Hot Blocks • • • • Multiple sessions may be requesting a block that is either not in buffer cache or is in an incompatible mode Deleting some of the hot rows and re-inserting them back into the table may alleviate the problem Most of the time the rows will be placed into a different block and reduce contention on the block May also need to adjust the pctfree and/or pctused parameters for the table to ensure the rows are placed into a different block – Inefficient Queries • • 12/4/2013 The more blocks requested from the buffer cache the more likelihood of a session having to wait for other sessions Tuning queries to access fewer blocks will often result in less contention for the same block 5
  • 6.
    Monitor with Ignite forOracle RAC • Measures the performance for the entire RAC as well as individual RAC nodes at a query level to help you find: – Imbalance between RAC nodes – Excessive overhead for cache access between nodes – Improper RAC configuration leading to degraded performance 12/4/2013 6
  • 7.
    Find Root Causeof Oracle RAC Waits Quickly • Improve database performance by 65% with Confio Ignite for Oracle RAC – Installs in minutes – Puts no load on monitored servers – Identify bottlenecks and root causes of Oracle RAC waits like these in minutes Download Ignite trial for free www.confio.com/freetrial 12/4/2013 7