Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kickass benchmarking with JMH Riviera Dev 2017

303 views

Published on

In this session, we will introduce you to JMH, an OpenJDK harness for building, running and analysing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM. It should help you find spots to optimise performance and, which may be even more important, it will show you parts that you don't really need to optimise. It not only will make your benchmarks more accurate, but also much easier to write.

Published in: Software
  • Be the first to comment

Kickass benchmarking with JMH Riviera Dev 2017

  1. 1. Kickass benchmarking with JMH Leonardo F. Gomes Nenad Bogojevic 0
  2. 2. Done, works, fast! 1
  3. 3. Premature optimization Donald Knuth, 1974 is the root of all evil 2
  4. 4. Donald E. Knuth Professor Emeritus at Stanford University ACM Grace Murray Hopper Award Turing Award Author of The Art Of Computer Programming Creator of TeX 3
  5. 5. 4
  6. 6. Donald E. Knuth We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunity in that critical 3%. 5
  7. 7. Donald E. Knuth A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified. 6
  8. 8. Donald E. Knuth It is often a mistake to make a priori judgements about what parts of a program are really critical. 7
  9. 9. Donald E. Knuth The universal experience of programmers who have been using measurement tools has been that the intuitive guesses fail. 8
  10. 10. Good programmers measure before optimizing 9
  11. 11. Benchmarking is hard 11
  12. 12. Why? 12
  13. 13. Warmup phase 13
  14. 14. Java source code Bytecodes HotSpot Java VM compile execute _Ahead-of- time _Using javac _Instructions for an abstract machine 14
  15. 15. Bytecodes HotSpot Java VM Interpreter Heap Stack Garbage collector execute access access manage vC1 C2 Machine code Debug info Compiled method Object maps compile produce Compilation system 15
  16. 16. Compiler optimizations 16
  17. 17. ©2016AmadeusITGroupanditsaffiliatesandsubsidiaries Don’t roll out your own benchmarking harness 18
  18. 18. JMH is your friend 19
  19. 19. Java Microbenchmark Harness 20
  20. 20. JVM Microbenchmark Harness 21
  21. 21. JVM Millibenchmark Harness 22
  22. 22. JVM Macrobenchmark Harness 23
  23. 23. JVM Nanobenchmark Harness 24
  24. 24. JMH is for benchmarking what JUnit is for unit testing 25
  25. 25. Macro 1 … 1000s Milli 1 … 1000ms Micro 1 … 1000us Nano 1 … 1000ns 27 Granularity
  26. 26. Benchmark modes Throughput ops/time_unit AverageTime time/operation SampleTime percentiles SingleShotTime cold performance 28
  27. 27. 29 @Warmup(iterations=5, time=1, timeUnit=SECONDS) @Measurement(iterations=5, time=1, timeUnit=SECONDS)
  28. 28. Multithreading 30
  29. 29. Multithreading made easy @Threads(20) @State(Scope.Thread) 31
  30. 30. Multithreading made easy results are aggregated for you 32
  31. 31. Anatomy of 33 Hashtable
  32. 32. 34 table lock thread 0 get ( key0 ) thread 1 get ( key1 ) put ( key0, value0 ) put ( key1, value1 ) lock (thread 0)
  33. 33. Anatomy of 35 ConcurrentHashMap
  34. 34. 36 Segment Segment Segment Segment lock lock lock lock thread 0 put ( key0, value0) segmentFor ( hash0 ) thread 1 put ( key1, value1) segmentFor ( hash1 ) thread 2 put ( key2, value2) segmentFor ( hash2 ) lock (thread 0) lock (thread 1)
  35. 35. 37 Segment Segment Segment Segment lock lock lock lock thread 0 get ( key0 ) segmentFor ( hash0 ) thread 2 get ( key2 ) segmentFor ( hash2 ) read volatile read volatile
  36. 36. Built-in profilers can show If compilation is happening while measuring If class loading is happening while measuring How much object allocation is happening Which methods are consuming CPU time 39
  37. 37. External profilers can be used Linux perf_events Windows xperf Java Mission Control (pluggable) Yourkit, etc. 40
  38. 38. JMH’s adopters 42
  39. 39. Our experience at amadeus
  40. 40. Verify that new code matches expectations Check that no regression is introduced Validate optimization ideas Cover performance fixes with related test
  41. 41. Continuous Integration
  42. 42. Care about a warmup phase Reduce noise Define regression Make sure backlog is handled
  43. 43. Key takeaways 50 Benchmark is tricky Measure before optimizing JMH helps a lot
  44. 44. • Caliper: https://www.flickr.com/photos/andrewthecook/14026422669/sizes/l • Geometric forms: https://www.flickr.com/photos/internetarchivebookimages/14753972274/sizes/l • Metric tape: https://www.flickr.com/photos/ilianov/3345314090/sizes/l/ • Mountain: https://www.flickr.com/photos/pthread/8151096195/sizes/l • Friends: https://www.flickr.com/photos/livenature/13895494231/sizes/l • Root: https://www.flickr.com/photos/paperpariah/19937816358/sizes/l/ • Knuth: https://www.flickr.com/photos/ioerror/56360019/sizes/l • Warmup: https://www.flickr.com/photos/komunews/2085730526/sizes/o/ • Multithreading: https://www.flickr.com/photos/slimjim/4329655445/sizes/l • Stop: https://www.flickr.com/photos/thematthewknot/3924980314/sizes/l • Boats: https://www.flickr.com/photos/cuppini/8465318134/sizes/l • Next steps: https://www.flickr.com/photos/gebagia/22346547334/sizes/l • Marines: https://www.flickr.com/photos/dvids/14007373489/sizes/l • Artic ice: https://commons.wikimedia.org/wiki/File:ICESCAPE.jpg • Demo time: https://www.flickr.com/photos/abstractbynature/6111219203 • Blue sky: https://www.flickr.com/photos/foctavian/16371691937/ 51
  45. 45. Questions? 52
  46. 46. 53 Follow us @lgomes @nenadbo github.com/kickass-jmh

×