Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OTN tour 2015 benchmarking oracle io performance with Orion by Alex Gorbachev

327 views

Published on

Every time Alex demonstrates charts he produces during IO benchmarks with ORION tool (Oracle I/O Numbers), he hears "Wow! How do you build these?" In this session Alex will teach how to benchmark your storage subsystem and capacity and how to stress test it to the limits. You will learn how easy it is to setup ORION benchmark and collect I/O performance characteristics of your platform and assess scalability of small random I/Os, impact of writes on I/Operformance, impact of different RAID levels, how backups can affect your OTLP traffic, performance of outer areas of disks vs inner areas, and compare SSD with HDD performance. ORION tests are very repeatable so it's a great measuring tool in your Measure, Analyze, Change, Measure cycle.

Published in: Education
  • Be the first to comment

  • Be the first to like this

OTN tour 2015 benchmarking oracle io performance with Orion by Alex Gorbachev

  1. 1. Benchmarking Oracle I/O Performance with ORION Alex Gorbachev November 2015
  2. 2. Alex Gorbachev • Chief Technology Officer at Pythian • Blogger • Cloudera Champion of Big Data • OakTable Network member • Oracle ACE Director • Founder of BattleAgainstAnyGuess.com • Founder of Sydney Oracle Meetup • EVP, IOUG • EVP, Ottawa Oracle User Group
  3. 3. © 2014 Pythian 3
  4. 4. ABOUT PYTHIAN Pythian is a global data outsourcing and consulting company that specializes in optimizing and managing mission-critical data systems. Pythian blends the world’s leading data experts with advanced, secure service delivery processes to create the industry’s best standard of care for its clients. Since its inception, Pythian has managed some of the world’s largest, most business-critical data infrastructures. 4 11,500Pythian currently manages over 11,500 systems. 385Pythian currently employs more than 385 people in 30 countries worldwide. 1997Pythian was founded in 1997 New Q2 2015
  5. 5. © 2011-2012 Pythian5
  6. 6. © 2011-2012 Pythian6 ORION - ORacle I/O Numbers Generate I/O workload similar to database patterns & measure I/O performance
  7. 7. © 2011-2012 Pythian7 Orion is designed to stress test the I/O subsystem
  8. 8. © 2011-2012 Pythian8 Orion is not perfect for simulation but good enough
  9. 9. © 2011-2012 Pythian9 Use Orion before moving/ deploying databases to the new platform
  10. 10. © 2011-2012 Pythian10 Orion is used in two scenarios
  11. 11. © 2011-2012 Pythian11 You have no idea what you need and want to ensure you get You know what you need and want to ensure you have it or
  12. 12. © 2011-2012 Pythian Infrastructure tuning - what’s the goal? • When you don’t know how much you need you try at least to ensure you take all you can • Assess what’s your possible bottlenecks • 1 Gbit Ethernet => 100+ MBPS or 10,000+ IOPS (8K) • 15K RPM disk • will easily serve 100-150 IOPS with average resp. time <10ms • can get to 200-250 IOPS but response time increase to 20 ms • SSD - see vendors specs • reads: random vs sequential... small vs large... no matter • writes: pattern matters 12
  13. 13. © 2011-2012 Pythian Orion • Uses code-base similar to Oracle database kernel • Standalone binary or part of Oracle home since 11.2.0.1 • Standalone Orion downloadable version is only 11.1 • Tests only I/O subsystem • Minimal CPU consumption • Async I/O is used to submit concurrent I/O requests • Each run includes multiple data points / tests • Scaling concurrency of small and large I/Os 13
  14. 14. © 2011-2012 Pythian Controlling Orion • Workload patterns • Small random I/O size and scale • Large I/O size, scale and pattern (random vs sequential) • Write percentage • Cache warming • Duration of each test (data point) • Data layout (concatenation vs striping) 14
  15. 15. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... .............................. .............................. .............................. 15
  16. 16. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... 16 -run advanced -matrix detailed # of tests = (Xlarge + 1) * (Xsmall + 1)
  17. 17. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... 17 -run advanced -matrix row -num_large 2 # of tests = Xsmall + 1
  18. 18. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... 18 -run advanced -matrix col -num_small 3 # of tests = Xlarge + 1
  19. 19. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... 19 -run advanced -matrix basic # of tests = Xlarge + Xsmall + 1
  20. 20. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 0, x, x, x, x, x, x, x, x, x x 1, x, x, x, x, x, x, x, x, x x 2, x, x, x, x, x, x, x, x, x x 3, x, x, x, x, x, x, x, x, x x 4, x, x, x, x, x, x, x, x, x x 5, x, x, x, x, x, x, x, x, x x 6, x, x, x, x, x, x, x, x, x x 7, x, x, x, x, x, x, x, x, x x 8, x, x, x, x, x, x, x, x, x x 9, x, x, x, x, x, x, x, x, x x 10, x, x, x, x, x, x, x, x, x x 11, x, x, x, x, x, x, x, x, x, x 20 -run advanced -matrix max # of tests = Xlarge + Xsmall + 1
  21. 21. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... 21 -run advanced -matrix point -num_large 2 -num_small 3 # of tests = 1
  22. 22. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... 22 -run simple
  23. 23. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... 23 -run normal
  24. 24. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... 24 -run oltp
  25. 25. © 2011-2012 Pythian Data Points Matrix Large/Small, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9... 0, x, x, x, x, x, x, x, x, x, x... 1, x, x, x, x, x, x, x, x, x, x... 2, x, x, x, x, x, x, x, x, x, x... 3, x, x, x, x, x, x, x, x, x, x... 4, x, x, x, x, x, x, x, x, x, x... 5, x, x, x, x, x, x, x, x, x, x... 6, x, x, x, x, x, x, x, x, x, x... 7, x, x, x, x, x, x, x, x, x, x... 8, x, x, x, x, x, x, x, x, x, x... 9, x, x, x, x, x, x, x, x, x, x... 10, x, x, x, x, x, x, x, x, x, x... 11, x, x, x, x, x, x, x, x, x, x... 25 -run dss
  26. 26. © 2011-2012 Pythian Orion I/O Performance Metrics • Small IOs • iops - average number of IOs per second • {test name}_{date}_{time}_iops.csv • lat - average IO response time • {test name}_{date}_{time}_lat.csv • Large IOs • mbps - throughput MB per second • {test name}_{date}_{time}_mbps.csv 26
  27. 27. © 2011-2012 Pythian Sample for -matrix detailed 27 Large/Small, 1, 2, 3, 4, 5 0, 58, 114, 117, 127, 84 1, 11, 29, 49, 63, 81 2, 12, 23, 30, 24, 31 iops Large/Small, 1, 2, 3, 4, 5 0, 17184.84, 17487.14, 25594.11, 31505.73, 59205.26 1, 88272.75, 66781.92, 60642.59, 62514.76, 61699.40 2, 80854.55, 83085.06, 99019.72, 155528.65, 156500.44 lat (us) Large/Small, 0, 1, 2, 3, 4, 5 1, 18.35, 12.14, 15.99, 16.99, 16.48, 16.37 2, 29.74, 27.07, 25.19, 21.18, 13.04, 13.33 mbps
  28. 28. © 2011-2012 Pythian Sample for -matrix basic 28 Large/Small, 1, 2, 3, 4, 5 0, 80, 153, 165, 163, 197 1 2 iops Large/Small, 1, 2, 3, 4, 5 0, 12370.09, 13060.23, 18112.16, 24448.27, 25250.33 1 2 lat (us) Large/Small, 0, 1, 2, 3, 4, 5 1, 31.84 2, 29.87 mbps
  29. 29. © 2011-2012 Pythian Trace file content ran (small): VLun = 0 Size = 10737418240 ran (small): Index = 0 Avg Lat = 22996.61 us Count = 431 ran (small): Index = 1 Avg Lat = 23825.39 us Count = 417 ran (small): nio=848 nior=652 niow=196 req w%=25 act w%=23 ran (small): my 2 oth 1 iops 65 lat 26081 us, bw = 0.51 MBps dur 9.96 s size 8 K, min lat 932 us, max lat 227524 us READ ran (small): my 2 oth 1 iops 19 lat 14499 us, bw = 0.15 MBps dur 9.96 s size 8 K, min lat 1422 us, max lat 120529 us WRITE ran (small): my 2 oth 1 iops 85 lat 23404 us, bw = 0.66 MBps dur 9.96 s size 8 K, min lat 932 us, max lat 227524 us TOTAL seq (large): VLun = 0 Size = 10737418240 seq (large): Index = 0 Avg Lat = 22038.99 us Count = 450 seq (large): Stream = 0 VLun = 0 Start = 2675965952 End = 3152019456 seq (large): Stream = 0 Avg Lat = 22038.99 us CIO = 1 NIO Count = 450 seq (large): nio=450 nior=450 niow=0 req w%=25 act w%=0 seq (large): my 1 oth 2 iops 45 lat 22039 us, bw = 45.22 MBps dur 9.95 s size 1024 K, min lat 9976 us, max lat 223534 us READ seq (large): my 1 oth 2 iops 0 lat 0 us, bw = 0.00 MBps dur 9.95 s size 1024 K, min lat 18446744073709551614 us, max lat 0 us WRITE seq (large): my 1 oth 2 iops 45 lat 22039 us, bw = 45.22 MBps dur 9.95 s size 1024 K, min lat 9976 us, max lat 223534 us TOTAL 29
  30. 30. © 2011-2012 Pythian Concurrent I/O requests = number of outstanding I/ 30 Separate process for large and small
  31. 31. © 2011-2012 Pythian Setting Scale of Concurrent I/Os • Range of concurrency is {0..max} • unless specified with -num_small or -num_large or fixed by run type • max for small IOs • num_disks * 5 for advanced, simple and normal runs • num_disks * 20 for OLTP run • max for large IOs • num_disks * 2 for advanced, simple and normal runs • num_disks * 15 for DSS run 31
  32. 32. © 2011-2012 Pythian OLTP and DSS runs are impractical* • Range of concurrency is {0..max} • unless specified with -num_small or -num_large or fixed by run type • max for small IOs • num_disks * 5 for advanced, simple and normal runs • num_disks * 20 for oltp run • max for large IOs • num_disks * 2 for advanced, simple and normal runs • num_disks * 15 for dss run 32 20 steps with interval num_disks {num_disks..num_disks*20) To much concurrency 15 steps with interval num_disks {num_disks..num_disks*15) * 11.2.0.3 behavior
  33. 33. © 2011-2012 Pythian Orion command-line syntax required arguments: -testname & -run orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -num_streamIO {num} -size_large {Kb} -type rand|seq -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 33 Defines input file with the list of disks {testname}.lun in the current directory # cat mytest.lun /dev/sdc /dev/sdd /dev/sde
  34. 34. © 2011-2012 Pythian Orion command-line syntax -run normal orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration 60 -num_disks {disks} -num_large {num} -type rand -num_streamIO {num} -size_large 1024 -num_small {num} -size_small 8 -simulate concat -stripe 1 -write 0 -cache_size {MB} -verbose 34 this is preset this can’t be this can be set
  35. 35. © 2011-2012 Pythian Orion command-line syntax -run simple orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration 60 -num_disks {disks} -num_large {num} -type rand -num_streamIO {num} -size_large 1024 -num_small {num} -size_small 8 -simulate concat -stripe 1 -write 0 -cache_size {MB} -verbose 35 this is preset this can’t be this can be set
  36. 36. © 2011-2012 Pythian Orion command-line syntax -run oltp orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large 0 -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 36 this is preset this can’t be this can be set
  37. 37. © 2011-2012 Pythian Orion command-line syntax -run dss orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small 0 -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 37 this is preset this can’t be this can be set
  38. 38. © 2011-2012 Pythian Orion command-line syntax -run advanced -matrix detailed | basic | max orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 38 this can’t be this can be set
  39. 39. © 2011-2012 Pythian Orion command-line syntax -run advanced -matrix col orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 39 this must be set this can’t be this can be set
  40. 40. © 2011-2012 Pythian Orion command-line syntax -run advanced -matrix row orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 40 this must be set this can’t be this can be set
  41. 41. © 2011-2012 Pythian Orion command-line syntax -run advanced -matrix point orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 41 this must be set this can’t be this can be set
  42. 42. © 2011-2012 Pythian Orion command-line syntax -simulate raid0 orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 42 Great way to simulate ASM striping
  43. 43. © 2011-2012 Pythian Orion command-line syntax -type seq orion -testname {testname} -run advanced | normal | simple | oltp | dss -matrix detailed | col | row | basic | max | point -duration {seconds} -num_disks {disks} -num_large {num} -type rand|seq -num_streamIO {num default 4} -size_large {Kb} -num_small {num} -size_small {Kb} -simulate concat|raid0 -stripe {Mb} -write {%} -cache_size {MB} -verbose 43
  44. 44. © 2011-2012 Pythian Orion Sequential I/O 44 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 -num_streamIO 1 -num_streamIO 4 Schedule one IO request and wait Schedule one IO request and wait Schedule one IO request and wait Schedule four IO requests and wait Schedule four IO requests and wait Schedule four IO requests and wait
  45. 45. © 2011-2012 Pythian What I/O in Oracle behaves like - num_streamIO 4? • Some examples: • serial direct parallel read • ARCH reads of redo logs • some operations with temporary segments • How do you verify/know? • Enable 10046 trace and OS trace (strace/truss/tusc) 45 * needs verification *
  46. 46. © 2011-2012 Pythian Orion Flexibility (Inflexibility?) 46 • Single Orion run is enough to assess scalability at defined settings • Need several separate Orion runs to vary • write % • large IO pattern • IO size • striping • Need multiple concurrent runs to • simulate more complex IO patterns • simulate RAC
  47. 47. © 2011-2012 Pythian Scenarios: OLTP traffic • -run advanced -matrix row -large_num 0 • Shadow processes’ “db file sequential reads” • DBWR’s “db file parallel write” • Optionally several runs with different settings like -write % • Analyze IOPS & response time 47
  48. 48. © 2011-2012 Pythian Scenarios: OLTP traffic visualization Oracle Database Appliance example 48 ODA: Small IOPS scalability / HDDs IOResponseTime,ms 0 5 10 15 20 25 Throughput,IOPS 0 1 000 2 000 3 000 4 000 5 000 1 2 3 4 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 IOPS Response Time
  49. 49. © 2011-2012 Pythian Scenarios: OLTP traffic variation analysis Varying write percentage in ODA 49 Small IOPS by writes percentage Oracle Database Appliance / OLPT / whole HDDs IOResponseTime,ms 0 10 20 30 40 50 60 70 80 Throughput,IOPS 0 1 000 2 000 3 000 4 000 5 000 6 000 7 000 Concurrent IO requests 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 IOPS wrt 0% IOPS wrt 10% IOPS wrt 20% IOPS wrt 40% IOPS wrt 60% Latency wrt 0% Latency wrt 10% Latency wrt 20% Latency wrt 40% Latency wrt 60%
  50. 50. © 2011-2012 Pythian Scenarios: OLTP traffic variation analysis Write percentage adjusted for ASM mirroring 50 Small IOPS by writes percentage Oracle Database Appliance / OLPT / whole HDDs IOResponseTime,ms 0 10 20 30 40 50 60 70 80 Throughput,IOPS 0 1 000 2 000 3 000 4 000 5 000 6 000 7 000 Concurrent IO requests 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 IOPS wrt 0% IOPS wrt 4% IOPS wrt 8% IOPS wrt 18% IOPS wrt 33% Latency wrt 0% Latency wrt 4% Latency wrt 8% Latency wrt 18% Latency wrt 33%
  51. 51. © 2011-2012 Pythian Impact of writes on RAID5 is huge 40% writes => 4 times lower IOPS 51
  52. 52. © 2011-2012 Pythian Same disks reconfigured as RAID1+0 40% writes => less than 50% hit 52
  53. 53. © 2011-2012 Pythian Scenarios: Data Warehouse queries • -run advanced -matrix col -small_num 0 • Keep read only (-write 0) • Concurrent users environment • -type rand • Single dedicated user performance • -type seq • -num_streamIO 1 • Most reads in the DB are synchronous • Analyze MBPS 53
  54. 54. © 2011-2012 Pythian Scenarios: Data Warehouse IO visualization 54 Large IOs throughput Throughput,MBPS 0 75 150 225 300 Concurrent threads 1 2 4 6 8 10 12 14 18 20 22 24 26 28 30 32
  55. 55. © 2011-2012 Pythian Scenarios: RMAN backup • -run advanced -matrix col -small_num 0 -type seq -num_streamIO 1 • Backup source only => -write 0 • Backup destination only => -write 100 • Database and backup destination combined => -write 50 • Watch for actual write percentage • 1 thread => 0% actual writes • 2 threads => 50% actual writes • 3 threads => 33% actual writes • 4 threads => 50% actual writes and etc... • Analyze MBPS 55
  56. 56. © 2011-2012 Pythian Scenarios: LGWR writes • -run advanced -matrix point -small_num 0 -type seq -num_streamIO 1 -write 100 -num_large 1 -size_large 5 • -size_large should be set to average LGWR write size which is often about 5-20k for OLTP systems • -num_large n • multiple instances • multiple LGWR threads in RAC • redo logs multiplexing • Analyze IOPS and response time • Gather from Orion run’s trace file 56
  57. 57. © 2011-2012 Pythian Scenarios: LGWR writes visualization 57 ODA SSD sequential 32K IO streams (tripple mirroring) AverageResponsetime,ms 0,00 0,20 0,40 0,60 0,80 1,00 Writespersecond 0 1600 3200 4800 6400 8000 Concurrent Threads 2 4 6 8 10 12 14 16 IOPS Response Time, ms
  58. 58. © 2011-2012 Pythian Combining different workloads • Start multiple parallel Orion runs • OLTP -matrix point -num_large 0 -num_small X • LGWR -matrix point -num_large 1 -num_small 0 -write 100 • ARCH -matrix point -num_small 0 -write {0 | 50} • RMAN - matrix point -num_small 0 -write {0 | 50} • Add batch data load with large parallel writes • Add batch reporting (DW-like) with large reads 58 Cannot throttle IO other than controlling number of outstanding IOs Cannot schedule a run with repetitive data points - must schedule multiple consecutive runs
  59. 59. © 2011-2012 Pythian EC2 large 5 EBS disks: first run to test scalability 59 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 IOPS 156 326 178 411 532 729 928 1 103 1 023 1 070 964 1 202 1 285 1 232 1 204 1 245 1 352 1 338 1 360 1 149 1 379 1 327 1 334 1 362 1 363 Response time, ms 6,4 6,1 10,2 9,7 9,3 8,2 7,5 7,2 8,8 9,3 11,4 10 10,1 11,3 12,4 12,8 12,6 13,4 14 17,3 15,2 16,5 17,2 17,6 18,2 Initial OTLP test with 5 disks and 20% writes Averageresponsetime,ms 0 5 10 15 20 IOPS 0 350 700 1 050 1 400 Number of concurrent IOs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 IOPS Response time, ms
  60. 60. © 2011-2012 Pythian Let’s mix in additional I/O workloads DUR=60 # OLTP test of scalability - original first run # /root/orion11203/orion -testname baseoltp -run advanced -duration $DUR -matrix row -num_large 0 -write 20 # OLTP point /root/orion11203/orion -testname oltp -run advanced -duration $DUR -matrix point -num_large 0 -num_small 10 -write 20 & # Adding LGWR /root/orion11203/orion -testname lgwr -run advanced -duration $DUR -matrix point -num_large 1 -num_small 0 -type seq -num_streamIO 1 -size_large 5 -write 100 & # Adding ARCH /root/orion11203/orion -testname arch -run advanced -duration $DUR -matrix point -num_large 2 -num_small 0 -type seq -num_streamIO 1 -size_large 1024 -write 50 & # Backup in 1 channel # /root/orion11203/orion -testname backup -run advanced -duration $DUR -matrix point -num_large 1 -num_small 0 -type seq -num_streamIO 1 -size_large 1024 -write 0 & # Backup in 4 channels # /root/orion11203/orion -testname backup -run advanced -duration $DUR -matrix point -num_large 4 -num_small 0 -type seq -num_streamIO 1 -size_large 1024 -write 0 & wait 60
  61. 61. © 2011-2012 Pythian61 OLTP IOPS Response time, ms LGWR writes LGWR write, ms OLTP only 1306 7,7 OLTP +LGWR 1239 8,1 139 7,1 OLTP+LGWR+ARCH 576 17,4 17 56 OLTP+LGWR+RMAN1 778 12,8 38 26,1 OLTP+LGWR+RMAN4 571 17,5 49 20,3 Responsetime,ms 0 4 8 12 16 20 IOPS 0 300 600 900 1200 1500 OLTP only OLTP +LGWR OLTP+LGWR+ARCH OLTP+LGWR+RMAN1 OLTP+LGWR+RMAN4 OLTP IOPS Response time, ms LGWRwrite,ms 0 15 30 45 60 75 LGWRwritespersecond 0 30 60 90 120 150 LGWR writes LGWR write, ms EC2... visualizing combined workload impact
  62. 62. © 2011-2012 Pythian The best Orion 11.2 new feature 62 Bucket LGWR no ARCH LGWR with ARCH 0 - 128 0 0 128 - 256 0 0 256 - 512 0 0 512 - 1024 1085 1 1024 - 2048 3376 8 2048 - 4096 395 1 4096 - 8192 845 0 8192 - 16384 1406 2 16384 - 32768 1115 161 32768 - 65536 161 699 65536 - 131072 4 169 131072 - 262144 0 17 262144 - 524288 1 10 524288 - 1048576 0 2 1048576 - 2097152 0 1 no ARCH 0-128 128-256 256-512 512-1024 1024-2048 2048-4096 4096-8192 8192-16384 16384-32768 32768-65536 65536-131072 131072-262144 262144-524288 524288-1048576 1048576-2097152 with ARCH Histograms!
  63. 63. © 2011-2012 Pythian Got RAC? Schedule parallel runs on each node 63 HP blades HP Virtual Connect Flex10 Big NetApp box 100 disks
  64. 64. © 2011-2012 Pythian Example of Failed Expectations NetApp NAS, 1 Gbit Ethernet, 42 disks 64 Latency,ms 0,0 7,5 15,0 22,5 30,0 IOPS 0 1000 2000 3000 4000 5000 1 2 3 4 5 10 20 30 40 50 60 70 80 90 100 IOPS Latency Read only Latency,ms 0 10 20 30 40 50 IOPS 0 1000 2000 3000 4000 5000 1 2 3 4 5 10 20 30 40 50 60 70 80 90 100 Read write
  65. 65. © 2011-2012 Pythian Tune-Up Results Switched from Intel to Broadcom NICs and disabled 65 Latency,ms 0 2 4 6 8 10 12 IOPS 0 2000 4000 6000 8000 10000 1 2 3 4 5 10 20 30 40 50 60 70 80 90 100 IOPS Latency Latency,ms 0 2 4 6 8 IOPS 0 2500 5000 7500 10000 12500 15000 1 2 3 4 5 10 20 30 40 50 60 70 80 90 100
  66. 66. © 2011-2012 Pythian Possible “What-If” scenarios 66 • Impact of a failed disk in a RAID group • Different block size • Different ASM allocation unit size (-stripe) • Assess foreign workload impact (shared SAN with other servers) • Test impact of configuration / infrastructure changes • Impact of backup or a batch job • Impact of decreased MTTR target (higher -write %) • Platform stability test (repeating the same data point for many days) • Impact of CPU starvation
  67. 67. © 2011-2012 Pythian Concurrent IOs on axis X is not always the best... 67 ODA: Small IOPS scalability and data placement / HDDs IOResponseTime,ms 0 5 10 15 20 25 Throughput,IOPS 0 1 200 2 400 3 600 4 800 6 000 1 2 3 4 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 IOPS whole disk IOPS outside 40% IOPS inside 60% Latency whole disk Latency outside 40% Latency inside 60%
  68. 68. © 2011-2012 Pythian Smarter presentation 50% IOPS at the same response time 68 ODA: Improving IO throughput by data placement IOPS 0 1200 2400 3600 4800 6000 IO response time 0 5 10 15 20 25 whole disk outside 40% inside 60%
  69. 69. © 2011-2012 Pythian Storage types • Anything as long as ASYNC IO is supported • Local storage (LUNs or filesystem) • NAS via NFS • iSCSI / FC devices (any block or raw device) • Cluster filesystem should work just fine 69
  70. 70. © 2011-2012 Pythian Beware of thin provisioning and other NAS magic • Smart storage technologies play bad jokes • If in doubt - “initialize” disks with non-zeroes 70
  71. 71. © 2011-2012 Pythian Orion 11.2 71 11.2.0.1 11.2.0.2 11.2.0.3 libcell11.so x x x libclntsh.so.11.1 x x libskgxp11.so x x libnnz11.so x x Included in • Database • Grid home • Client (tested Administrative option) Dependencies
  72. 72. © 2011-2012 Pythian Orion with SLOB (Silly Little Oracle Benchmark) • Orion gives more control • Orion is easier to setup • Orion uses very little CPU - it doesn’t do anything with data • Easier to saturate IO subsystem without CPU starvation • Less realistic results if you want to account database CPU use for LIO and processing the data • Less realistic for multiprocess orchestration • SLOB - is more realistic but more difficult to control 72
  73. 73. © 2011-2012 Pythian73 Visualization is the Key
  74. 74. © 2011-2012 Pythian Thank you and Q&A 74 http://www.pythian.com/news/ http://www.facebook.com/pages/The-Pythian-Group/ http://twitter.com/pythian http://www.linkedin.com/company/pythian 1-866-PYTHIAN sales@pythian.com or hr@pythian.com To contact us… To follow us… gorbachev@pythian.com

×