Benchmarking Oracle I/O Performance with Orion by Alex Gorbachev
Upcoming SlideShare
Loading in...5
×
 

Benchmarking Oracle I/O Performance with Orion by Alex Gorbachev

on

  • 11,682 views

 

Statistics

Views

Total Views
11,682
Views on SlideShare
8,601
Embed Views
3,081

Actions

Likes
7
Downloads
513
Comments
0

11 Embeds 3,081

http://iusoltsev.wordpress.com 2370
http://www.scoop.it 616
http://www.hiteshgondalia.dev 70
https://twitter.com 8
https://iusoltsev.wordpress.com 6
http://webcache.googleusercontent.com 5
http://ning.com 2
http://prlog.ru 1
http://translate.googleusercontent.com 1
http://www.kred.com 1
http://nf2xg33morzwk5q.o5xxezdqojsxg4zomnxw2.cmle.ru 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Benchmarking Oracle I/O Performance with Orion by Alex Gorbachev Benchmarking Oracle I/O Performance with Orion by Alex Gorbachev Document Transcript

  • Benchmarking Oracle I/OPerformance with ORIONAlex GorbachevOttawa, ON4-Feb-2013
  • Alex Gorbachev • CTO, The Pythian Group • Blogger • OakTable Network member • Oracle ACE Director • BattleAgainstAnyGuess.com • IOUG, Director of Communities2 © 2011-2012 Pythian
  • Why Pythian Recognized Leader: • Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server • Work with over 150 multinational companies such as Forbes.com, Fox Sports, Nordion and Western Union to help manage their complex IT deployments Expertise: • One of the world’s largest concentrations of dedicated, full-time DBA expertise. Employ 7 Oracle ACEs/ACE Directors • Hold 7 Specializations under Oracle Platinum Partner program, including Oracle Exadata, Oracle GoldenGate & Oracle RAC Global Reach & Scalability: • 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response © 2011-2012 PythianWe are a managed services, consulting and solution provider of elite database and system administration skills in Oracle, MySQL andMicrosoft SQL Server environments.
  • 4 © 2011-2012 PythianApply at hr@pythian.com
  • ORION - ORacle I/O Numbers Generate I/O workload similar to database patterns & measure I/O performance5 © 2011-2012 Pythian
  • Orion is designed to stress test the I/O subsystem6 © 2011-2012 Pythian
  • Orion isnot perfect for simulation but good enough7 © 2011-2012 Pythian
  • Use Orion before moving/ deploying databases to the new platform8 © 2011-2012 Pythian
  • Orion is used in two scenarios9 © 2011-2012 Pythian
  • You know what you need and want to ensure you have it or You have no idea what you need and want to ensure you get the best you can 10 © 2011-2012 PythianThe first one is based on capacity planning.The second you can call an infrastructure tuning
  • Infrastructure tuning - what’s the goal? • When you don’t know how much you need you try at least to ensure you take all you can • Assess what’s your possible bottlenecks • 1 Gbit Ethernet => 100+ MBPS or 10,000+ IOPS (8K) • 15K RPM disk • will easily serve 100-150 IOPS with average resp. time <10ms • can get to 200-250 IOPS but response time increase to 20 ms • SSD - see vendors specs • reads: random vs sequential... small vs large... no matter • writes: pattern matters11 © 2011-2012 Pythian
  • Orion • Uses code-base similar to Oracle database kernel • Standalone binary or part of Oracle home since 11.2.0.1 • Standalone Orion downloadable version is only 11.1 • Tests only I/O subsystem • Minimal CPU consumption • Async I/O is used to submit concurrent I/O requests • Each run includes multiple data points / tests • Scaling concurrency of small and large I/Os12 © 2011-2012 Pythian
  • Controlling Orion • Workload patterns • Small random I/O size and scale • Large I/O size, scale and pattern (random vs sequential) • Write percentage • Cache warming • Duration of each test (data point) • Data layout (concatenation vs striping)13 © 2011-2012 Pythian
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... .............................. .............................. .............................. 14 © 2011-2012 PythianEach Orion run performs several tests and collects metrics for each test. The set of metrics for one test is a datapoint. Based on the run configuration, Orion collects several data points scaling concurrency of small random IOsand concurrency of large IOs.Each data point is defined by the number of concurrent small I/O requests and the number of concurrent large IOstreams.Orion iterates through concurrency of large I/Os from minimal to maximum (which can be the only one dependingon the run configuration) and then for each large IO concurrency level, it iterates through concurrency levels ofsmall IOs from minimum to maximum (which can be the only one as well depending on the run configuration). Wewill see how it these ranges are selected later.If you look at the matrix then you can imagine this process as running the tests row by row from top to bottomand for each row, the sequence of tests is from left to right. Just like in English writing.As Orion performs the tests, it writes the results in the trace file and at the end of the test it produces severalmatrix files with collected metrics.
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... -run advanced -matrix detailed # of tests = (Xlarge + 1) * (Xsmall + 1) 15 © 2011-2012 PythianThere are several types of runs. Let’s first look into “advanced” mode and the rest of the runs are simpler versionswhich present some of the parameters for you. You can think of them as wizard modes.To define which data points are collected by Orion, the matrix type is defined. Detailed matrix is the most timeconsuming to run - Orion will test every combination of large and small I/O workload - it will iterate from 0concurrency level to maximum concurrency level for both large and small IOs.
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... -run advanced -matrix row -num_large 2 # of tests = Xsmall + 1 16 © 2011-2012 PythianMatrix row fixes number of concurrent large I/O streams to a configurable number (can be zero) and iteratesthrough concurrency of small IOs.
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... -run advanced -matrix col -num_small 3 # of tests = Xlarge + 1 17 © 2011-2012 PythianMatrix col fixes number of concurrent small IOs to a configurable number (can be zero) and iterates throughconcurrency of large IO streams.
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... -run advanced -matrix basic # of tests = Xlarge + Xsmall + 1 18 © 2011-2012 PythianMatrix basic performs tests of non-mixed small and large workloads.First, Orion iterates through different concurrency levels of small IOs without any large IO streams.Then, Orion iterates through concurrency of large IO streams without any small IOs.
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 0, x, x, x, x, x, x, x, x, x x 1, x, x, x, x, x, x, x, x, x x 2, x, x, x, x, x, x, x, x, x x 3, x, x, x, x, x, x, x, x, x x 4, x, x, x, x, x, x, x, x, x x 5, x, x, x, x, x, x, x, x, x x 6, x, x, x, x, x, x, x, x, x x 7, x, x, x, x, x, x, x, x, x x 8, x, x, x, x, x, x, x, x, x x 9, x, x, x, x, x, x, x, x, x x 10, x, x, x, x, x, x, x, x, x x 11, x, x, x, x, x, x, x, x, x, x -run advanced -matrix max # of tests = Xlarge + Xsmall + 1 19 © 2011-2012 PythianMatrix max is similar to basic but instead of performing no large IO activity while iterating through small IOs,Orion performs maximum number of large IO streams. The same with iterating through large IO streamsconcurrency -- Orion will run at maximum concurrent small I/Os.
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... -run advanced -matrix point -num_large 2 -num_small 3 # of tests = 1 20 © 2011-2012 PythianMatrix point is the fastest run as it runs exactly one test defined.
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... -run simple 21 © 2011-2012 PythianNon-advanced runs automatically define matrix type as well as most of other parameters.
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... -run normal 22 © 2011-2012 PythianNon-advanced runs automatically define matrix type as well as most of other parameters.
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... -run oltp 23 © 2011-2012 PythianNon-advanced runs automatically define matrix type as well as most of other parameters.
  • Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... -run dss 24 © 2011-2012 PythianNon-advanced runs automatically define matrix type as well as most of other parameters.
  • Orion I/O Performance Metrics • Small IOs • iops - average number of IOs per second • {test name}_{date}_{time}_iops.csv • lat - average IO response time • {test name}_{date}_{time}_lat.csv • Large IOs • mbps - throughput MB per second • {test name}_{date}_{time}_mbps.csv25 © 2011-2012 Pythian
  • Sample for -matrix detailed iops Large/Small, 1, 2, 3, 4, 5 0, 58, 114, 117, 127, 84 1, 11, 29, 49, 63, 81 2, 12, 23, 30, 24, 31 lat (us) Large/Small, 1, 2, 3, 4, 5 0, 17184.84, 17487.14, 25594.11, 31505.73, 59205.26 1, 88272.75, 66781.92, 60642.59, 62514.76, 61699.40 2, 80854.55, 83085.06, 99019.72, 155528.65, 156500.44 mbps Large/Small, 0, 1, 2, 3, 4, 5 1, 18.35, 12.14, 15.99, 16.99, 16.48, 16.37 2, 29.74, 27.07, 25.19, 21.18, 13.04, 13.33 26 © 2011-2012 PythianOrion 11.1.0.7 and earlier reports response time in ms.11.2.0.1+ reports latency in us (microseconds)Note how matrix is slightly different:- iops and lat matrix exclude column with zero small IOs- mbps matrix excludes row with zero large IOs
  • Sample for -matrix basic iops Large/Small, 1, 2, 3, 4, 5 0, 80, 153, 165, 163, 197 1 2 lat (us) Large/Small, 1, 2, 3, 4, 5 0, 12370.09, 13060.23, 18112.16, 24448.27, 25250.33 1 2 mbps Large/Small, 0, 1, 2, 3, 4, 5 1, 31.84 2, 29.8727 © 2011-2012 Pythian
  • Trace file content ran (small): VLun = 0 Size = 10737418240 ran (small): Index = 0 Avg Lat = 22996.61 us Count = 431 ran (small): Index = 1 Avg Lat = 23825.39 us Count = 417 ran (small): nio=848 nior=652 niow=196 req w%=25 act w%=23 ran (small): my 2 oth 1 iops 65 lat 26081 us, bw = 0.51 MBps dur 9.96 s size 8 K, min lat 932 us, max lat 227524 us READ ran (small): my 2 oth 1 iops 19 lat 14499 us, bw = 0.15 MBps dur 9.96 s size 8 K, min lat 1422 us, max lat 120529 us WRITE ran (small): my 2 oth 1 iops 85 lat 23404 us, bw = 0.66 MBps dur 9.96 s size 8 K, min lat 932 us, max lat 227524 us TOTAL seq (large): VLun = 0 Size = 10737418240 seq (large): Index = 0 Avg Lat = 22038.99 us Count = 450 seq (large): Stream = 0 VLun = 0 Start = 2675965952 End = 3152019456 seq (large): Stream = 0 Avg Lat = 22038.99 us CIO = 1 NIO Count = 450 seq (large): nio=450 nior=450 niow=0 req w%=25 act w%=0 seq (large): my 1 oth 2 iops 45 lat 22039 us, bw = 45.22 MBps dur 9.95 s size 1024 K, min lat 9976 us, max lat 223534 us READ seq (large): my 1 oth 2 iops 0 lat 0 us, bw = 0.00 MBps dur 9.95 s size 1024 K, min lat 18446744073709551614 us, max lat 0 us WRITE seq (large): my 1 oth 2 iops 45 lat 22039 us, bw = 45.22 MBps dur 9.95 s size 1024 K, min lat 9976 us, max lat 223534 us TOTAL 28 © 2011-2012 PythianSeparate read and write statistics.Actual write percentage is important for sequential large I/Obecause it assigns streams to write or read.IOPS, LAT and MBPS are actually calculated for all types of IO butmatrix doesn’t report them all.Can parse trace file to extract all statistics available.Note: write stats for large sequential IO is bogus since there wasno writes done.
  • Concurrent I/O requests = number of outstanding I/Os Separate process for large and small I/Os 29 © 2011-2012 PythianFor each task, Orion forks 2 separate processes performing largeand small IOs. If only large or only small IOs are performed thenonly one process is forked.
  • Setting Scale of Concurrent I/Os • Range of concurrency is {0..max} • unless specified with -num_small or -num_large or fixed by run type • max for small IOs • num_disks * 5 for advanced, simple and normal runs • num_disks * 20 for OLTP run • max for large IOs • num_disks * 2 for advanced, simple and normal runs • num_disks * 15 for DSS run30 © 2011-2012 Pythian
  • OLTP and DSS runs are impractical* • Range 20 steps with interval num_disks of concurrency is {0..max} {num_disks..num_disks*20) • unless specified with -num_small or -num_large or fixed by run type • max for small IOs • num_disks * 5 for advanced, simple and normal runs • num_disks * 20 for oltp run To much concurrency • max for large IOs • num_disks * 2 for advanced, simple and normal runs • num_disks * 15 for dss run 15 steps with interval num_disks {num_disks..num_disks*15) * 11.2.0.3 behavior31 © 2011-2012 Pythian
  • Orion command-line syntax required arguments: -testname & -run orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -num_streamIO {num} Defines input file with the -size_large {Kb} list of disks {testname}.lun -type rand|seq in the current directory -num_small {num} # cat mytest.lun -size_small {Kb} -simulate concat|raid0 /dev/sdc -stripe {Mb} /dev/sdd -write {%} /dev/sde -cache_size {MB} -verbose 32 © 2011-2012 PythianThis is the full command-line syntax.The two parameters that are always required are -testname and -run.-testname identifies the only input file that Orion needs with thelist of disks - each disk is a path on the new line. The file namemust be testname with added .lun extension and the file must bein the current directory. Orion will also prefix the output resultswith testname.-run defines types of Orion run and the rest of parameters dependon it.
  • Orion command-line syntax -run normal orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration 60 -num_disks {disks} -num_large {num} -type rand -num_streamIO {num} -size_large 1024 -num_small {num} -size_small 8 -simulate concat -stripe 1 -write 0 -cache_size {MB} -verbose this is preset this can’t be this can be set 33 © 2011-2012 PythianFor -run normal, Orion sets most of the parameters to predefinedvalue and you can only specify -num_disks, -cache_size and -verbose.
  • Orion command-line syntax -run simple orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration 60 -num_disks {disks} -num_large {num} -type rand -num_streamIO {num} -size_large 1024 -num_small {num} -size_small 8 -simulate concat -stripe 1 -write 0 -cache_size {MB} -verbose this is preset this can’t be this can be set 34 © 2011-2012 Pythian-run simple has identical settings but the the -matrix is basic.
  • Orion command-line syntax -run oltp orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose this is preset this can’t be this can be set 35 © 2011-2012 Pythian-run oltp (make sure it’s lower case) lets you specify most of theother parameters but you really only need to care aboutparameters affecting small IOs. Defaults are used if you don’tdefine a specific value.
  • Orion command-line syntax -run dss orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose this is preset this can’t be this can be set 36 © 2011-2012 Pythian-run dss (make sure it’s lower case) lets you specify most of theother parameters (except switching to sequential large IO streams)but parameter controlling small IOs don’t matter. Defaults areused if you don’t define a specific value.
  • Orion command-line syntax -run advanced -matrix detailed | basic | max orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose this is preset this can’t be this can be set 37 © 2011-2012 PythianRun -advanced is the most flexible mode and depending on thematrix type selected, most of the parameters can be specified.When selecting -matrix detailed, basic or max, Orion selectsconcurrency ranges for large and small IOs based on -num_disksso -num_large and -num_small cannot be set explicitly.
  • Orion command-line syntax -run advanced -matrix col orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose this is preset this can’t be this can be set 38 © 2011-2012 PythianWhen selecting -matrix col (for column), you must specify -num_small to define the column of data points to collect while -num_large is not relevant.
  • Orion command-line syntax -run advanced -matrix row orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose this is preset this can’t be this can be set 39 © 2011-2012 Pythian-matrix row is reverse to col - you must specify -num_large todefine the row of data points to collect while -num_small is notrelevant.
  • Orion command-line syntax -run advanced -matrix point orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose this is preset this can’t be this can be set 40 © 2011-2012 PythianTo specify -matrix point, you need to explicitly set both -num_small and -num_large to identify the data point to collect.
  • Orion command-line syntax -simulate raid0 orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq Great way to -num_streamIO {num} simulate ASM -size_large {Kb} -num_small {num} striping -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 41 © 2011-2012 PythianParameter -simulate controls how Orion treats multiple disks andit has two options:1. “concat” - all disks are concatenated sequentially into onesingle virtual disk against which Orion submits IO requests.2. “raid0” - Orion organizes a sing virtual disk by striping acrossall disks defined in the testname.lun file using stripe size that canbe set by -stripe parameter (default 1Mb). This is the best way tosimulate ASM striping.
  • Orion command-line syntax -type seq orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num default 4} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 42 © 2011-2012 PythianParameter -type controls large IO pattern:1. “rand” - Orion performs large IOs across randomly selecting theoffset for each IO request from the whole virtual disk.2. “seq” - Orion establishes multiple sequential IO streamsstarting from predefined offsets of the virtual disk (that producedby concatenating or striping). The starting offsets are selected atthe beginning of each test by splitting the virtual disks in equalchunks of number of concurrent stream.
  • Orion Sequential I/O e t on wai ule d e h ed st an on ait Sc ue le w q e du and re ch est IO S u q re e one IO Sc hedul d wait st an IO reque -num_streamIO 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 -num_streamIO 4 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 Sch Sch IO re edule fo IO re edule fo Sch ques u ques u ts an r IO re edule fo ts an r d wa ques u d wa it ts an r it d wa it 43 © 2011-2012 PythianEach stream can also have multiple IO threads simulated (and bydefault there are 4 threads). Thus when you are testing sequentiallarge IO, your real number of concurrent IO requests mightactually by much higher than you think because of default valuefor -num_streamIO set to 4.
  • What I/O in Oracle behaves like -num_streamIO 4? *n • Some examples: ee ds • serial direct parallel read ve r ifi • ARCH reads of redo logs ca tio • some operations with temporary segments n * • How do you verify/know? • Enable 10046 trace and OS trace (strace/truss/tusc)44 © 2011-2012 Pythian
  • Orion Flexibility (Inflexibility?) • Single Orion run is enough to assess scalability at defined settings • Need several separate Orion runs to vary • write % • large IO pattern • IO size • striping • Need multiple concurrent runs to • simulate more complex IO patterns • simulate RAC 45 © 2011-2012 PythianOrion has lots of flexibility in the settings. However, for a singlerun there is very limited control on data points collected. Variationof any settings other then concurrency requires separate Orionruns.When simulating more complex scenarios, you would also need tocombine multiple run and make sure they are running in sync. Tosimplify synchronization, you would use -matrix point.Otherwise, sync different data points is a nightmare especially thatOrion can’t be used to collect the same data point multiple timesover and over in the same run while another run (or runs) iteratesthrough other data points.
  • Scenarios: OLTP traffic • -run advanced -matrix row -large_num 0 • Shadow processes’ “db file sequential reads” • DBWR’s “db file parallel write” • Optionally several runs with different settings like -write % • Analyze IOPS & response time 46 © 2011-2012 PythianInstead of using “-run oltp” use advanced run settings. This runwill simulate random reads that foreground processes are doingas well as background random writes performed by DBWR.One almost universally good variation to drill into is writepercentage - this will let you assess how well I/O subsystem canhandle random writes as opposed to random reads. These testsusually show that no matter what storage vendors claim abouttheir super smart storage arrays and caching algorithms,sustained random writes ruin any parity based mirroring.
  • Scenarios: OLTP traffic visualization Oracle Database Appliance example ODA: Small IOPS scalability / HDDs 5,000 25 IOPS Response Time 4,000 20 IO Response Time, ms 3,000 15Throughput, IOPS 2,000 10 1,000 5 0 0 1 2 3 4 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 47 © 2011-2012 PythianThis is an example of the first Orion run of Oracle DatabaseAppliance to assess OLTP traffic scalability for read only workload.
  • Scenarios: OLTP traffic variation analysis Varying write percentage in ODA Small IOPS by writes percentage Oracle Database Appliance / OLPT / whole HDDs 7,000 80 6,000 70 60 5,000 IO Response Time, ms Throughput, IOPS 50 4,000 40 3,000 30 2,000 20 1,000 10 0 0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 Concurrent IO requests IOPS wrt 0% IOPS wrt 10% IOPS wrt 20% IOPS wrt 40% IOPS wrt 60% Latency wrt 0% Latency wrt 10% Latency wrt 20% Latency wrt 40% Latency wrt 60% 48 © 2011-2012 PythianNow let’s introduce variable write percentage and assess theimpact. Because ODA doesn’t use any RAID technology, we seealmost no degradation.However, since ASM will be doing host based tripple mirroring (forthis purpose comparable to RAID1), this IOPS metrics are fromdisks perspective and not from the database perspective. We needto adjust IOPS and write percentage to see the numbers fromdatabase perspective after ASM mirroring.
  • Scenarios: OLTP traffic variation analysis Write percentage adjusted for ASM mirroring Small IOPS by writes percentage Oracle Database Appliance / OLPT / whole HDDs 7,000 80 6,000 70 60 5,000 IO Response Time, msThroughput, IOPS 50 4,000 40 3,000 30 2,000 20 1,000 10 0 0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 Concurrent IO requests IOPS wrt 0% IOPS wrt 4% IOPS wrt 8% IOPS wrt 18% IOPS wrt 33% Latency wrt 0% Latency wrt 4% Latency wrt 8% Latency wrt 18% Latency wrt 33% 49 © 2011-2012 PythianThis is the adjusted IOPS and percentage values.
  • Impact of writes on RAID5 is huge 40% writes => 4 times lower IOPS 50 © 2011-2012 PythianHere is an explicit example of RAID5 shortcomings.
  • Same disks reconfigured as RAID1+0 40% writes => less than 50% hit 51 © 2011-2012 PythianMuch less writes impact with RAID10 which actually becomesnoticeable closer to saturation point anyway.
  • Scenarios: Data Warehouse queries • -runadvanced -matrix col -small_num 0 • Keep read only (-write 0) • Concurrent users environment • -type rand • Single dedicated user performance • -type seq • -num_streamIO 1 • Most reads in the DB are synchronous • Analyze MBPS 52 © 2011-2012 PythianTo simulate data warehousing workload from concurrent users useread only workload with random large reads. Even thoughindividual queries might be scanning tables in more sequentialmanner, the high concurrency level makes them look like random.Environments with low concurrency levels will probably look morelike multiple sequential scan streams.For data warehouse performance you are normally interested inthe scan throughput measured as MB per second.
  • Scenarios: Data Warehouse IO visualization Large IOs throughput 300 225 Throughput, MBPS 150 75 0 1 2 4 6 8 10 12 14 18 20 22 24 26 28 30 32 Concurrent threads 53 © 2011-2012 PythianSimple way to visualize.You could also add throughput per reading stream to seeperformance that each user doing serial scans will get, forexample.
  • Scenarios: RMAN backup • -runadvanced -matrix col -small_num 0 -type seq -num_streamIO 1 • Backup source only => -write 0 • Backup destination only => -write 100 • Database and backup destination combined => -write 50 • Watch for actual write percentage • 1 thread => 0% actual writes • 2 threads => 50% actual writes • 3 threads => 33% actual writes • 4 threads => 50% actual writes and etc... • Analyze MBPS 54 © 2011-2012 PythianNo backup compression overhead accounted for.Orion will be actually more aggressive sending IO requestsbecause it will keep either writing non-stop or reading non-stopwhile an RMAN process needs to read and write, read and write,read ...
  • Scenarios: LGWR writes • -run advanced -matrix point -small_num 0 -type seq -num_streamIO 1 -write 100 -num_large 1 -size_large 5 • -size_large should be set to average LGWR write size which is often about 5-20k for OLTP systems • -num_large n • multiple instances • multiple LGWR threads in RAC • redo logs multiplexing • Analyze IOPS and response time • Gather from Orion run’s trace file55 © 2011-2012 Pythian
  • Scenarios: LGWR writes visualization ODA SSD sequential 32K IO streams (tripple mirroring) 8000 1.00 6400 0.80 Average Response time, ms Writes per second 4800 0.60 3200 0.40 1600 0.20 0 0 2 4 6 8 10 12 14 16 Concurrent Threads IOPS Response Time, ms 56 © 2011-2012 PythianBecause you can’t throttle down each thread, each thread will goas fast as it can so you you always pushing some kind of a limitand you will be throttled by the maximum what an I/O subsystemcan deliver or by CPU but Orion consumes very little CPU so youignore it.
  • Combining different workloads • Startmultiple parallel Orion runs • OLTP -matrix point -num_large 0 -num_small X • LGWR -matrix point -num_large 1 -num_small 0 -write 100 • ARCH -matrix point -num_small 0 -write {0 | 50} • RMAN - matrix point -num_small 0 -write {0 | 50} • Add batch data load with large parallel writes • Add batch reporting (DW-like) with large reads Cannot schedule a run Cannot throttle IO other with repetitive data points than controlling number - must schedule multiple of outstanding IOs consecutive runs 57 © 2011-2012 PythianCombining multiple runs is only reliable if using -matrix point.
  • EC2 large 5 EBS disks: first run to test scalability Initial OTLP test with 5 disks and 20% writes 1,500 20 IOPS Response time, ms 1,125 15 Average response time, msIOPS 750 10 375 5 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Number of concurrent IOs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 IOPS 156 326 178 411 532 729 928 1,103 1,023 1,070 964 1,202 1,285 1,232 1,204 1,245 1,352 1,338 1,360 1,149 1,379 1,327 1,334 1,362 1,363Response time, ms 6.4 6.1 10.2 9.7 9.3 8.2 7.5 7.2 8.8 9.3 11.4 10 10.1 11.3 12.4 12.8 12.6 13.4 14 17.3 15.2 16.5 17.2 17.6 18.2 58 © 2011-2012 PythianMy initial run gives me general idea how my subsystem wouldscale under different OLTP load with 20% writes. If I’m curios, Imight go further and perform few runs with different writepercentage and visualize the difference.
  • Let’s mix in additional I/O workloads DUR=60 # OLTP test of scalability - original first run # /root/orion11203/orion -testname baseoltp -run advanced -duration $DUR -matrix row -num_large 0 -write 20 # OLTP point /root/orion11203/orion -testname oltp -run advanced -duration $DUR -matrix point -num_large 0 -num_small 10 -write 20 & # Adding LGWR /root/orion11203/orion -testname lgwr -run advanced -duration $DUR -matrix point -num_large 1 -num_small 0 -type seq -num_streamIO 1 -size_large 5 -write 100 & # Adding ARCH /root/orion11203/orion -testname arch -run advanced -duration $DUR -matrix point -num_large 2 -num_small 0 -type seq -num_streamIO 1 -size_large 1024 -write 50 & # Backup in 1 channel # /root/orion11203/orion -testname backup -run advanced -duration $DUR -matrix point -num_large 1 -num_small 0 -type seq -num_streamIO 1 -size_large 1024 -write 0 & # Backup in 4 channels # /root/orion11203/orion -testname backup -run advanced -duration $DUR -matrix point -num_large 4 -num_small 0 -type seq -num_streamIO 1 -size_large 1024 -write 0 & wait 59 © 2011-2012 PythianThe first commented out command I used to assess initialscalability and build the run visualized on the previous slide.I then take it and convert to “OLTP point” and run it.Next step I add “Adding LGWR” to run in parallel.After that I add ARCH and collect another data point and etc.Note that they are all starting at the same time and run in parallelin the background and the script waits for all background jobs tocomplete at the end using “wait” command.
  • EC2... visualizing combined workload impact 150 75 LGWR writes per second 120 60 LGWR write, ms 90 45 60 30 30 15 0 0 OLTP IOPS Response time, ms LGWR writes LGWR write, ms 1500 20 Response time, ms 1200 16 900 12 IOPS 600 8 300 4 0 0 y R N1 N4 OLT P onl +LGW +ARC H RMA RMA OLTP +LGWR +LG WR+ +LG WR+ OLTP OLTP OLTP OLTP IOPS Response time, ms LGWR writes LGWR write, ms OLTP only 1306 7.7 OLTP +LGWR 1239 8.1 139 7.1 OLTP+LGWR+ARCH 576 17.4 17 56.0 OLTP+LGWR+RMAN1 778 12.8 38 26.1 OLTP+LGWR+RMAN4 571 17.5 49 20.3 60 © 2011-2012 PythianI can then record how my OLTP traffic is affected in differentscenarios including LGWR performance.
  • The best Orion 11.2 new feature Histograms! Bucket LGWR no LGWR with no ARCH ARCH ARCH 0 - 128 0 0 128 - 256 0 0 256 - 512 0 0 512 - 1024 1085 1 1024 - 2048 3376 8 2048 - 4096 395 1 with ARCH 4096 - 8192 845 0 8192 - 16384 1406 2 16384 - 32768 1115 161 32768 - 65536 161 699 65536 - 131072 4 169 0 - 128 128 - 256 256 - 512 512 - 1024 1024 - 2048 2048 - 4096 4096 - 8192 8192 - 16384 16384 - 32768 32768 - 65536 65536 - 131072 131072 - 262144 262144 - 524288 524288 - 1048576 1048576 - 2097152 131072 - 262144 0 17 262144 - 524288 1 10 524288 - 1048576 0 2 1048576 - 2097152 0 161 © 2011-2012 Pythian
  • Got RAC? Schedule parallel runs on each node HP blades HP Virtual Connect Flex10 Big NetApp box 100 disks62 © 2011-2012 Pythian
  • Example of Failed Expectations NetApp NAS, 1 Gbit Ethernet, 42 disks 5000 30.0 4000 Read only 22.5 Latency, ms 3000 IOPS 15.0 2000 7.5 1000 0 0 1 2 3 4 5 10 20 30 40 50 60 70 80 90 100 IOPS Latency 5000 50 4000 40 Read write Latency, ms 3000 30 IOPS 2000 20 1000 10 0 0 1 2 3 4 5 10 20 30 40 50 60 70 80 90 10063 © 2011-2012 Pythian
  • Tune-Up Results Switched from Intel to Broadcom NICs and disabled snapshots IOPS Latency 10000 12 10 8000 8 Latency, ms 6000 IOPS 6 4000 4 2000 2 0 0 1 2 3 4 5 10 20 30 40 50 60 70 80 90 100 15000 8 12500 6 10000 Latency, ms IOPS 7500 4 5000 2 2500 0 0 1 2 3 4 5 10 20 30 40 50 60 70 80 90 10064 © 2011-2012 Pythian
  • Possible “What-If” scenarios • Impact of a failed disk in a RAID group • Different block size • Different ASM allocation unit size (-stripe) • Assess foreign workload impact (shared SAN with other servers) • Test impact of configuration / infrastructure changes • Impact of backup or a batch job • Impact of decreased MTTR target (higher -write %) • Platform stability test (repeating the same data point for many days) • Impact of CPU starvation65 © 2011-2012 Pythian
  • Concurrent IOs on axis X is not always the best... ODA: Small IOPS scalability and data placement / HDDs 6,000 25 IOPS whole disk IOPS outside 40% IOPS inside 60% Latency whole disk Latency outside 40% Latency inside 60% 4,800 20 IO Response Time, msThroughput, IOPS 3,600 15 2,400 10 1,200 5 0 0 1 2 3 4 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 66 © 2011-2012 Pythian
  • Smarter presentation 50% IOPS at the same response time ODA: Improving IO throughput by data placement 6000 4800 3600 IOPS 2400 whole disk 1200 outside 40% inside 60% 0 0 5 10 15 20 25 IO response time67 © 2011-2012 Pythian
  • Storage types • Anything as long as ASYNC IO is supported • Local storage (LUNs or filesystem) • NAS via NFS • iSCSI / FC devices (any block or raw device) • Cluster filesystem should work just fine68 © 2011-2012 Pythian
  • Beware of thin provisioning and other NAS magic • Smart storage technologies play bad jokes • If in doubt - “initialize” disks with non-zeroes69 © 2011-2012 Pythian
  • Orion 11.2 Included in • Database • Grid home • Client (tested Administrative option) Dependencies 11.2.0.1 11.2.0.2 11.2.0.3 libcell11.so x x x libclntsh.so.11.1 x x libskgxp11.so x x libnnz11.so x x70 © 2011-2012 Pythian
  • Orion with SLOB (Silly Little Oracle Benchmark) • Orion gives more control • Orion is easier to setup • Orion uses very little CPU - it doesn’t do anything with data • Easier to saturate IO subsystem without CPU starvation • Less realistic results if you want to account database CPU use for LIO and processing the data • Less realistic for multiprocess orchestration • SLOB - is more realistic but more difficult to control71 © 2011-2012 Pythian
  • Visualization is the Key72 © 2011-2012 Pythian
  • Thank you and Q&A To contact us… sales@pythian.com or hr@pythian.com 1-866-PYTHIAN gorbachev@pythian.com To follow us… http://www.pythian.com/news/ http://www.facebook.com/pages/The-Pythian-Group/ http://twitter.com/pythian http://www.linkedin.com/company/pythian73 © 2011-2012 Pythian