The document is a presentation about Exadata and Oracle 11gR2. It provides background on the speaker's experience with parallel query over the years. It then covers key features of Exadata storage solutions and new parallel query capabilities in 11gR2 like auto degree of parallelism and statement queuing. The presentation notes some practical considerations for implementing Exadata and parallel query, such as partitioning workloads and monitoring statement queuing.

1. Doug Burns

2.  Introduction
 A Little History
 Exadata
 Oracle 11gR2
 Practicalities
 Summary
13/08/2012 Slide 2

3.  My real name is Douglas
 But only my family call me that
 My official intro is on the E4 website
 There is even a picture of a Cuddly Toy
 Please note that they are Cuddly not Plushy
 I'm from Scotland which is part of Great Britain
 Not Ireland
 Definitely not English
 Speaking of where I'm from …
13/08/2012 Slide 3

4.  Is a liar
 Is offering you a useful
simplification
 Doesn't have much faith in your
knowledge of British geography
13/08/2012 Slide 4

5.  From https://twitter.com/TheTumshie/status/227488415061508097/photo/1
13/08/2012 Slide 5

6. 13/08/2012 Slide 6

7. 13/08/2012 Slide 7

8.  Because Tanel asked
 I've worked with Exadata V2 and X2 in high-throughput
Production environments for the past two years
 2 x Half-Rack V2
 Full-Rack V2
 Lots of Quarter-Rack X2-2s on the way
 I have a long-standing interest in Parallel Execution
 Not that the two went together particularly well at this client
 It saved you all the bitter 'early V2 experiences' diatribe
13/08/2012 Slide 8

9.  This presentation will include opinions
 I will not regurgitate large chunks of the documentation,
Oracle White Papers or Expert Oracle Exadata
 Client use of Parallelism on Exadata has been more
limited than I expected
13/08/2012 Slide 9

10.  Introduction
 A Little History
 Exadata
 Oracle 11gR2
 Practicalities
 Summary
13/08/2012 Slide 10

11.  First time I heard of Oracle Parallel Query was in 1993
 Boss returned quite giddy from IOUG Conference
 Was a while longer before it was released in 7.1.6 and
didn't hear much about it until 1996
 Dev DBA decided it would be a really cool thing to try out
 V7.3 time-frame
 Was switched off not long afterwards
 I've worked on many Data Warehouses so have been
interested in Parallelism, Stats Collection etc for a long
time
13/08/2012 Slide 11

12.  Tuning Parallel Execution
 http://oracledoug.com/px.pdf
 Parallel Adaptive Multi-User debate with Tom Kyte
SELECT name, value FROM v$sysstat WHERE name LIKE 'Parallel%';
NAME VALUE
-------------------------------------------------- ------
Parallel operations not downgraded 546353
Parallel operations downgraded to serial 432
Parallel operations downgraded 75 to 99 pct 790
Parallel operations downgraded 50 to 75 pct 1454
Parallel operations downgraded 25 to 50 pct 7654
Parallel operations downgraded 1 to 25 pct 11873
▪ Downgraded to serial could be a particular problem!
▪ PX is great at delivering blistering but inconsistent performance
13/08/2012 Slide 12

13.  How Many Slaves?
 http://oracledoug.com/px_slaves.pdf
 Parallel Execution and the Magic of Two
 Inspired by Cary Millsap's work on the Magic of Two for batch-
type processes
 Sharply diminishing returns with higher DOPs
 ISP4400 quad-socket server with 4 x SCSI Ultra 160
 E10k with 5 disk stripe-set on EMC array
 Had never seen many benefits of DOPs over 2-4, confirmed by
feedback on the paper
 PX is great at uncovering storage bottlenecks
13/08/2012 Slide 13

14.  Introduction
 A Little History
 Exadata
 Oracle 11gR2
 Practicalities
 Summary
13/08/2012 Slide 14

15. (Insert Mandatory slide here
with a pretty picture of
a full-rack X2-2 and
lots of very impressive
throughput numbers)
13/08/2012 Slide 15

16.  The most common real world Parallel Query limitation in
my experience of numerous sites was storage
bandwidth
 Exadata can certainly reduce that!
 Not necessarily better than other modern storage solutions
 Dedicated to Oracle and controlled by a dedicated team
 Smart Scan reduces bandwidth pressure by reducing traffic
 Balanced Configuration
 Despite Oracle and the market deciding it's a mixed-
workload/consolidation platform, it excels as a DW one
13/08/2012 Slide 16

17. 13/08/2012 Slide 17

18.  Introduction
 A Little History
 Exadata
 Oracle 11gR2
 Practicalities
 Summary
13/08/2012 Slide 18

19.  Exadata implies Oracle 11gR2
 Let's all try to forget 11.2.0.1 as soon as possible
 Assume 11.2.0.2
 Lots of new Parallel features
 Auto DOP
 Statement Queuing
 In-memory Parallel Query
13/08/2012 Slide 19

20.  Let Oracle decide what is the most-appropriate Degree
of Parallelism (DOP) for queries
 Set parallel_degree_policy to
 LIMITED – Calculates DOP where objects have PARALLEL set
to DEFAULT
 AUTO – Calculates DOP for all objects, potentially
 Default parallel_min_time_threshold is 10 seconds
 Anything estimated to run for less time will not be considered
 Auto-DOP will not work until you have run CALIBRATE_IO
13/08/2012 Slide 20

21.  Procedure in DBMS_RESOURCE_MANAGER
 Support Note 727062.1
 Oracle Best Practice advice in Support Note 1297112.1
is to set MAX_PMBPS to 200 on Exadata
 Manual Update of resource_io_calibrate$
 Restart each Instance
13/08/2012 Slide 21

22.  Parallel_degree_limit
 Absolute DOP limit for an individual query
 Default is CPU
 parallel_threads_per_cpu * cpu_count * active_instance_count
 Can set it to a fixed value
13/08/2012 Slide 22

23.  The problems with Parallel Adaptive Multi-User (PAMU)
 Aggressive
 DOP determined at run-time
 Alternative approach
 If slaves are available, then
▪ Use as many resources as possible
▪ Complete and let others in
 If slaves are unavailable then queue until they are
 Degraded DOP not decided at run-time
13/08/2012 Slide 23

24.  Set parallel_degree_policy to
 AUTO
 But that includes Auto-DOP and In-Memory PQ
 _parallel_statement_queuing = TRUE
 Alternatively – enable/disable at query-level using hints
 /*+ NO_STMT_QUEUING */
 /*+ STMT_QUEUING */
 Waits on 'resmgr:pq queued' event (Scheduler)
13/08/2012 Slide 24

25.  The flip-side of the fact we can now do Direct Path
Reads for queries executed Serially
 Can now use reads into the Buffer Cache for Parallel
Queries
 Uses aggregated Buffer Cache across RAC cluster
 But also works single-instance
13/08/2012 Slide 25

26. 13/08/2012 Slide 26

27.  Introduction
 A Little History
 Exadata
 Oracle 11gR2
 Practicalities
 Summary
13/08/2012 Slide 27

28.  Exadata nodes are not that powerful
 Exadata storage and Flash Cache are not necessarily
state-of-the art
 Lots of quarter-racks out there
 Companies are cheap ;-)
 Kind of reasonable to partition workloads.
 Technically, separating heavy write and read workloads probably
makes sense, but …
13/08/2012 Slide 28

29. Node 2
Node 1
Read/Reporting Workload
Write workload
App A
Node 4
Node 3
Read/Reporting Workload
Read/Reporting Workload
App B
App A
plus Standby
13/08/2012 Slide 29

30.  Remember those users who hate inconsistent response
times?
 Parallel Statement Queuing
 Some visibility in OEM Performance Pages
 Plenty of information in V$ views but not very useful to
users!
 Support Note 1359043.1 contains queries to help
monitor Statement Queuing
13/08/2012 Slide 30

31. 13/08/2012 Slide 31

32.  Well, we're not doing Direct Path Reads any more
 So we're not using Smart Scan any more
 Tricky to get working properly
 Needs to be pay-back
 If you get a lot of pay-back, is Exadata really the way to
go? Aren't these two different kinds of system?
13/08/2012 Slide 32

33.  Introduction
 A Little History
 Exadata
 Oracle 11gR2
 Practicalities
 Summary
13/08/2012 Slide 33

34.  Exadata is very likely to be better than previous solution
 The bigger the rack, the better
 Cross-instance Parallelism for truly impressive results
 One method of driving towards Direct Path Reads to
take advantage of Smart Scan
13/08/2012 Slide 34

35.  Auto-DOP
 Statement Queuing
 Balance cache benefits of In-Memory PQ against initial
read overhead
13/08/2012 Slide 35

36.  "Oracle Database Parallel Execution Fundamentals"
 Oracle Corp. White Paper – 11gR2 focus
 Oracle Support Notes
 1297112.1 – Best Practices for DW on X2-2
 727062.1 – CALIBRATE_IO
 1264548.1 – New 11gR2 Parallel Query Parameters
 1269321.1 – Auto-DOP
 1340180.1 – Recommended patches for PX Statement Queuing
 1359043.1 – Configure Statement Queuing for mixed workloads
 Chapter 6 of Expert Oracle Exadata
13/08/2012 Slide 36

37.  To various people at Oracle who keep working
on improving this stuff
 To Enkitec for taking such good care of the
speakers
 To you, for your time
 P.S. If you really want me to explain what's going on with the toys ...
13/08/2012 Slide 37

38. Mail: dougburns@yahoo.com
Twitter: @orcldoug
Web: http://oracledoug.com