Oracle Exadata is the equivalent of a F1 car in terms of performance but are you sure your application is driving it at its full potential? A simple "lift&shift" approach to Exadata migration might lose significant opportunities for improvements. This session highlights a few example where making little changes dramatically changed the application performance

1. Is your SQL Exadata-aware?
Mauro Pagano & Carlos Sierra

2. What is Exadata?
• Complete solution to run Oracle RDBMS
• Lots of top shelf hardware
– That you can buy anywhere...
• Specific software capabilities
– That you can’t buy anywhere else J
1/29/17 2

3. Very partial list of Exa-specific features
• Some are more advertised
– Smart Scan (since 11.1)
– Hybrid Columnar Compression (11.2.1.2)
– Smart Flash Cache (11.2.1.2)
– Smart Flash Log (11.2.2.4)
• Some are less
– Columnar Flash Caching (12.1.2.1.0)
– Smart Block Transfer (12.1.2.2.0)
1/29/17 3

4. Exadata is a F1 car
• ”Smart” features solve many old issues
– Very large I/O bandwidth
• Lots of CPU and memory available
• Exadata is basically a supercar
• But do you drive it at its full potential?
– Do your SQLs fully leverage Exadata?
1/29/17 4

5. 1/29/17 5
How I drove Exadata the first time
NOTICE: Next is a pic of my
first time with Exadata.
Please be nice, I’m very
sensitive about it

6. How many people drive Exadata
• Many migrations are drop-in
– Nothing special to make SQLs more Exa-friendly
• SQLs often run much faster
– Hardware is very powerful
– Exa-specific SW features enabled by default
– Many features require almost no configuration
1/29/17 6

7. Driving lesson #1 - Memo
• Smart Scan is your friend J
– Opens the door to extremely powerful scan
– Requires Full Scan and Direct Path Reads
• That you spent 20y trying to avoid!!!
– Doesn’t work for objects mostly in memory
• IMPQ, ABTC, Full DB Caching
– Doesn’t work for every object
• Cluster tables, IOT, orascn/rowdependencies, etc (*)
1/29/17 7

8. Driving lesson #2 - Memo
• CBO has no specific intelligence for Exadata
– Same formulas, algorithms and params as before
• That you spent 20y tweaking to avoid full scans
– Won’t favor Exadata features explicitly (*)
• Might need a little push to get more Exa-friendly
– Exadata system statistics help a bit
1/29/17 8

9. Example of simple drop-in
Analytical SQL on large dataset (Exa SW ON/OFF)
select cust_gender, calendar_week_number,
sum(amount_sold), sum(quantity_sold)
from countries, -- from SH schema
sales_big, -- ~2.5B rows
customers, -- from SH schema
times -- from SH schema
where sales_big.cust_id = customers.cust_id
and sales_big.time_id = times.time_id
and customers.country_id = countries.country_id
and countries.country_name = 'Italy'
and calendar_quarter_desc = '2001-02'
group by cust_gender, calendar_week_number;
1/29/17 9

10. Exadata, Smart Scan OFF
Exadata X3
11.2.0.3
Exadata X2
12.1.0.2
1/29/17 10
Oracle 12c generated a
better performing plan
than 11g, regardless of
Exadata

11. Exadata, Smart Scan OFF, 11g
1/29/17 11

12. Exadata, Smart Scan OFF, 12c
1/29/17 12

13. Exadata HW, Smart Scan ON
Exadata X3
11.2.0.3
Exadata X2
12.1.0.2
1/29/17 13

14. Exadata, Smart Scan ON, 11g
1/29/17 14

15. Exadata, Smart Scan ON, 12c
1/29/17 15

16. Cool, I got to do nothing!
• Partially true
– Many features require almost no attention
– They just work and give you decent gains
• Partially false
– Better use of some features if you know them
– Using them right makes significant difference
1/29/17 16

17. 1/29/17 17

18. Exadata, Smart Scan ON, SQL Exa-friendly
Exadata X3
11.2.0.3
Exadata X2
12.1.0.2
1/29/17 18

19. 1/29/17 19

20. Exadata, Smart Scan ON, 12c, SQL Exa-friendly
1/29/17 20

21. Help the joins
• Smart Scan reads / filters data efficiently
– But it doesn’t help with joins
• Is there a way to write a join as a filter?
– Kind of, using Bloom filters to reduce data before join
– Less data returned to compute nodes to join
• Trick was hint to encourage diff join order -> 2 BF
– One BF per dimension, similar to VT idea
1/29/17 21

22. Does it always work?
• Of course not J
• Efficiency depends on
– Selectivity of filters from the dimensions
– Amount of false positives
– Number of Bloom Filters applied on large table
• 1 BF on many cols vs many BFs on 1 col (*)
1/29/17 22

23. Another drop-in
Large table filtered by lookup table(s)
select sum(amount_sold),
sum(quantity_sold)
from sales_big
where sales_big.cust_id in
(select cust_id
from customers,
countries
where customers.country_id = countries.country_id
and countries.country_name = 'Italy'
and cust_credit_limit = 15000)
1/29/17 23

24. X2, 12c, drop-in vs Exa-friendly
Drop-in
Exa-friendly
1/29/17 24

25. Exadata, Smart Scan ON, SQL Exa-friendly
1/29/17 25

26. Remove the subquery
• Subquery produces low NDV
• Can be pre-computed and used as filter
• Hint /*+ PRECOMPUTE_SUBQUERY */
• Doesn’t always work
– If the subquery alone takes long to execute
– If high NDV then Smart Scan disabled (uses DPR)
1/29/17 26

27. Drop-in #3
Simple SQL using complex predicate
SELECT COUNT(*)
FROM martin.bigtab
WHERE v1 = DECODE(
CASE WHEN (
CASE WHEN 'ED' IN ('ED','AIS') THEN 1
WHEN ('NA'='P' OR 'ED'='SAQ') AND v1 <> 'Y' THEN 1
ELSE 0
END = 1)
THEN 'X'
ELSE 'N'
END ,'X','0001161826', 'Y')
1/29/17 27
'0001161826'

28. 12c, drop-in vs Exa-friendly
Drop-in
Exa-friendly
1/29/17 28
cell pIO bytes eligible for pred offload 83886137344
cell pIO bytes saved by storage index 0
cell pIO inter bytes ret by smart scan 14740392
cell pIO bytes eligible for pred offload 83886137344
cell pIO bytes saved by storage index 31303966720
cell pIO inter bytes ret by smart scan 9241000

29. Drop-in #4
Simple SQL using less complex predicate
SELECT COUNT(*)
FROM sales_big
WHERE TO_CHAR(time_id,'YYYY-MM-DD HH24') = '1997-10-09 10'
1/29/17 29

30. 12c, drop-in vs Exa-friendly
Drop-in
Exa-friendly
1/29/17 30
cell pIO bytes eligible for pred offload 103534297088
cell pIO bytes saved by storage index 0
cell pIO inter bytes ret by smart scan 5985839720
cell pIO bytes eligible for pred offload 103534297088
cell pIO bytes saved by storage index 81859526656
cell pIO inter bytes ret by smart scan 950385176

31. Complex filters
• CBO attempts to simplify them if possible
– No logic to simplify ALL of them
• Storage Index works with simple operators
– Complex filters won’t use SI
• Simplify logic when possible
• Use super-set filter predicates
– Can be offloaded and use storage index
– Reduce num of rows to apply complex pred on
1/29/17 31

32. V$SQLFN_METADATA
• List of functions that can [not] be offloaded
– Increase version after version
• Should be familiar when writing SQL
• If SQL includes a not offloadable then
– Rewrite if possible using offloadable function
– If not, add super-set with offloadable function
• Specific functions can be turned off
1/29/17 32

33. So far we learned
• Clean your CBO environment
– Let the CBO do its job J
• Filter sooner to process less later
– Help the CBO if needed
• Applies to
– Joins -> Bloom Filters
– Subqueries -> Unnest+BF or precompute subq
– Complex pred -> simplify or super-set pred
• Old concept, indexes serve same purpose
1/29/17 33

34. I’m ready to race now!
1/29/17 34
My first time with no
training wheels.

35. What if I get an engine problem?
• Previous examples increase offload efficiency
– It’s like going faster with a functioning car
• What is the car doesn’t start?
– Several conditions disable / impact smart scan
– Learning them to
• Avoid unexpected surprises
• Identify way to alleviate impact
1/29/17 35

36. Mixed load (OLTP/DW) sample
1/29/17 36

37. DW load sample
1/29/17 37

38. Smart Scan party poopers
• Requirements not met
– No FullScan+DPR, mostly cached, etc
• Not available
– Cluster, IOT, rowscn, etc
• Rows need resolution in compute node
– Chained rows with rowpiece in different AU
– Consistent reads not helped by minscn cache
– Saturated cells shipping data to balance CPU
1/29/17 38

39. Drop-in surprise #1
SQL doesn’t use Smart Scan on every partition
1/29/17 39

40. Drop-in surprise #1
SQL doesn’t use Smart Scan on every partition
1/29/17 40

41. Drop-in surprise #1
• Segment(s) too small for Smart Scan
– Legacy heavy partitioning to extreme of tiny parts
– Can also happen as consequence of compression
• Confirmed by nsmtio trace
SqlId = 9rxuuptqp3ru2, phv = 2502846130, Part# = 1
NSMTIO: kcbism: islarge 1 next 0 nblks 915656 ...
…
NSMTIO: qertbFetch:[MTT < OBJECT_SIZE < VLOT]:…
NSMTIO: kcbdpc:DirectRead
SqlId = 1vrwjrfzxmknd, phv = 2502846130, Part# = 2
NSMTIO: kcbism: islarge 0 next 0 nblks 836 …
…
NSMTIO: qertbFetch:NoDirectRead:[- STT < OBJECT_SIZE < MTT]
1/29/17 41

42. Drop-in surprise #2
1/29/17 42
Smart Scan spending 65% on single block reads

43. Drop-in surprise #2
• Usually because of
– Large UPD causing chained rows
– Tables with > 255 cols in different AU
• Chained rows are processed
– In the cell if next piece available (less likely)
• chained rows processed by cell
– In the compute node if not (more likely)
• chained rows skipped by cell
• Some fixes (e.g. 9373758) try to minimize impact
1/29/17 43

44. Drop-in surprise #3
CR needed, returning data to compute node
-------------------------------------------------------------------------------------------------------
SID @INST, USERNAME , TYPE, STATISTIC , DELTA
-------------------------------------------------------------------------------------------------------
1497 @1, EO00 , STAT, cell physical IO interconnect bytes , 650748816
1497 @1, EO00 , STAT, cell physical IO bytes eligible for predicate offload , 612196352
1497 @1, EO00 , STAT, cell physical IO interconnect bytes returned by smart scan, 613436648
1497 @1, EO00 , STAT, cell blocks processed by cache layer , 76033
1497 @1, EO00 , STAT, cell commit cache queries , 76033
1/29/17 44

45. Drop-in surprise #3
• Uncommited changes
– Smart Scan returns data needs CR to compute
• Not Exadata specific
– Just push work back to compute
• For commited changes MINSCN cache helps
– cell blocks helped by minscn optimization
1/29/17 45

46. Conclusion
• Drop-in migration often sub-optimally use Exadata
• Legacy SQLs may need adjustments:
– To fully leverage Exadata features
– To improve areas where Exadata doesn’t automatically help
• Most of the adjustments are simple:
– Clean CBO environment
– Streamline filter predicates
• Sometimes CBO needs a little push
• Get familiar with Smart Scan disablers
– To predict behaviors and avoid surprises
1/29/17 46

47. 47

48. References
• Oracle® Exadata Storage Server Software
User's Guide 12c Release 1 (12.1)
• Oracle® Exadata Database Machine System
Overview 12c Release 1 (12.1)
• Expert Oracle Exadata
• Google.com J
48

49. Contact Information
• http://mauro-pagano.com
– Email
• mauro.pagano@gmail.com
– Download(s)
• SQLd360 vYYMM (date)
• TUNAs360 vYYMM (date)
• Pathfinder vYYMM (date)
49