Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JVM: need for speed - tooling overview for jdk.io 2016

473 views

Published on

Monitoring your production JVMs is extremely important. Though, not always you have a luxury of having direct access to production servers, which suffer from performance peaks, memory leaks or concurrency issues. Not always you are able to catch JVM issues without an ability to debug the code in-place. There are quite many tools that can help analyzing performance issues of your JVMs based on real-time monitoring data as well as on logs available only post-mortem. This presentation will give an overview of these tools.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

JVM: need for speed - tooling overview for jdk.io 2016

  1. 1. 01
  2. 2. About me 02
  3. 3. Andrey Adamovich Java/Groovy developer (Sometimes) application performance analyst Co­author of Groovy 2 Cookbook Co­organizer of @latcraft and @devternity • • • • 03
  4. 4. Contact details E­mail: andrey@aestasit.com Linkedin: http://www.linkedin.com/in/andreyadamovich Lanyrd: http://lanyrd.com/profile/andrey­adamovich GitHub: https://github.com/aadamovich SO: http://stackoverflow.com/users/162792/andrey­adamovich Twitter: @codingandrey, @aestasit • • • • • • 04
  5. 5. What's in this presentation? 05
  6. 6. The tools 06
  7. 7. The game 07
  8. 8. Rules Picture is shown You guess what the next topic is First one to guess gets a sticker! • • • 08
  9. 9. Let's try! 09
  10. 10. 10
  11. 11. Java's (Hash)Map 11
  12. 12. 12
  13. 13. BigData 13
  14. 14. Let the games begin! 14
  15. 15. 15
  16. 16. GC Pauses 16
  17. 17. What are the good parameters?17
  18. 18. So many options.... 18
  19. 19. So many options.... > java ‐XX:+PrintFlagsFinal ‐version | wc ‐l java version "1.8.0_45" Java(TM) SE Runtime Environment (build 1.8.0_45‐b15) Java HotSpot(TM) 64‐Bit Server VM (build 25.45‐b02, mixed mode) 714 01. 02. 03. 04. 05. 19
  20. 20. So many options.... > java ‐XX:+PrintFlagsFinal ‐version | wc ‐l java version "1.7.0_80" Java(TM) SE Runtime Environment (build 1.7.0_80‐b15) Java HotSpot(TM) 64‐Bit Server VM (build 24.80‐b11, mixed mode) 668 01. 02. 03. 04. 05. 20
  21. 21. So many options.... > java ‐XX:+PrintFlagsFinal ‐version | wc ‐l java version "1.6.0_45" Java(TM) SE Runtime Environment (build 1.6.0_45‐b06) Java HotSpot(TM) 64‐Bit Server VM (build 20.45‐b01, mixed mode) 670 01. 02. 03. 04. 05. 21
  22. 22. So many options.... > java ‐XX:+PrintFlagsFinal ‐version | grep GC | wc ‐l 65 01. 02. 22
  23. 23. Use cases Web application Streaming application Background job • • • 23
  24. 24. Memory tuning? ‐Xms<heap size>[g|m|k]  ‐Xmx<heap size>[g|m|k] ‐XX:PermSize=<perm gen size>[g|m|k]  ‐XX:MaxPermSize=<perm gen size>[g|m|k] ‐Xmn<young size>[g|m|k] ‐XX:SurvivorRatio=<ratio> 01. 02. 03. 04. 05. 06. 24
  25. 25. Leverage multi­core ‐XX:+UseConcMarkSweepGC  ‐XX:+UseCMSInitiatingOccupancyOnly  ‐XX:+ScavengeBeforeFullGC  ‐XX:+CMSScavengeBeforeRemark ‐XX:+CMSConcurrentMTEnabled ‐XX:CMSInitiatingOccupancyFraction=70 01. 02. 03. 04. 05. 06. 25
  26. 26. GC logging ‐XX:+PrintGCDetails  ‐XX:+PrintGCDateStamps  ‐Xloggc:<file‐path>  ‐XX:+PrintGCApplicationConcurrentTime  ‐XX:+PrintSafepointStatistics  ‐XX:+PrintHeapAtGC 01. 02. 03. 04. 05. 06. 26
  27. 27. Log analysis 27
  28. 28. Open source: GCViewer 28
  29. 29. Comercial: Censum 29
  30. 30. Next riddle 30
  31. 31. 31
  32. 32. Memory dumps 32
  33. 33. How to take? On out of memory: ‐XX:+HeapDumpOnOutOfMemoryError  ‐XX:HeapDumpPath=<path to dump>`date`.hprof Or manually: jmap ‐dump:file=<path to dump> <java process id> 01. 02. 01. 33
  34. 34. jhat 34
  35. 35. Eclipse MAT 35
  36. 36. OQL 36
  37. 37. Useful characteristics Object Retained Size Object Shallow Size Stale Objects (not connected to GC roots) • • • 37
  38. 38. Flight Recording 38
  39. 39. Comercial YourKit jProfiler • • 39
  40. 40. Next riddle 40
  41. 41. 41
  42. 42. Thread dumps 42
  43. 43. Situations Dead locks 100% CPU load Hanging I/O • • • 43
  44. 44. jstack jstack <java process pid>01. 44
  45. 45. JMX Use MBean with name  java.lang:type=Threading Call  getAllThreadIds Loop through all retrieved IDs and call  getThreadInfo • • • 45
  46. 46. The cheapest profiler for ((a=1; a <= 60; a++)) do    jstack 7789 > thread‐dump.$a   sleep 1   echo $a done 01. 02. 03. 04. 05. 06. 46
  47. 47. Thread dump analysis 47
  48. 48. Thread dump structure 48
  49. 49. Thread states I A thread can be in one of the following states: NEW  A thread that has not yet started is in this state. RUNNABLE  A thread executing in the Java virtual machine is in this state. BLOCKED  A thread that is blocked waiting for a monitor lock is in this state. • • • 49
  50. 50. Thread states II WAITING  A thread that is waiting indefinitely for another thread to perform a particular action is in this state. TIMED_WAITING  A thread that is waiting for another thread to perform an action for up to a specified waiting time is in this state. TERMINATED  A thread that has exited is in this state. • • • 50
  51. 51. TDA 51
  52. 52. Samurai TDA 52
  53. 53. IBM TMDA 53
  54. 54. Use custom scripts 54
  55. 55. Next riddle 55
  56. 56. 56
  57. 57. Debugging 57
  58. 58. We need it when we have... Non­reproducable issues Hard­to­understand code • • 58
  59. 59. Enable debugging ‐agentlib:jdwp=transport=dt_socket,server=y,address=133701. 59
  60. 60. Use your IDE (Eclipse) 60
  61. 61. Use your IDE (Intellij IDEA) 61
  62. 62. jdb 62
  63. 63. jdb 63
  64. 64. YouDebug 64
  65. 65. YouDebug YouDebug is a non­interactive debugger scripted by Groovy to assist remote troubleshooting and data collection to analyze failures. “ 65
  66. 66. YouDebug http://youdebug.kohsuke.org/ https://github.com/kohsuke/youdebug • • 66
  67. 67. YouDebug (buggy class) public class SubStringTest {   public static void main(String[] args) {     String s = someLengthComputationOfString();     System.out.println(s.substring(5));   }   private static String someLengthComputationOfString() {     ...;   } } 01. 02. 03. 04. 05. 06. 07. 08. 09. 67
  68. 68. YouDebug (SubStringMonitor.ydb) breakpoint("com.acme.SubStringTest",3) {   println "s="+s; } 01. 02. 03. 68
  69. 69. YouDebug (run debugger) > java ‐jar youdebug.jar ‐socket 1337 SubStringMonitor.ydb s=test 01. 02. 69
  70. 70. Next riddle 70
  71. 71. 71
  72. 72. Decompiling 72
  73. 73. Java Decompiler 73
  74. 74. Java Decompiler 74
  75. 75. Issues Lack of support for latest JVM features Lack of support for other JVM langauges Obfuscation is always an issue • • • 75
  76. 76. Next riddle 76
  77. 77. 77
  78. 78. Profiling 78
  79. 79. Types of profiling Memory allocation profiling CPU profiling Lock profiling • • • 79
  80. 80. VisualVM 80
  81. 81. Honest Profiler 81
  82. 82. Comercial YourKit jProfiler • • 82
  83. 83. Next riddle 83
  84. 84. 84
  85. 85. Benchmarks 85
  86. 86. JMH MH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targetting the JVM. “ 86
  87. 87. JMH: example package org.openjdk.jmh.samples; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.runner.Runner; import org.openjdk.jmh.runner.RunnerException; import org.openjdk.jmh.runner.options.Options; import org.openjdk.jmh.runner.options.OptionsBuilder; 01. 02.03. 04. 05. 06. 07. 87
  88. 88. JHM: example public class JMHSample_01_HelloWorld {   @Benchmark   public void wellHelloThere() {     // this method was intentionally left blank.   } 01. 02. 03. 04. 05. 88
  89. 89. JMH: example   public static void main(String[] args) throws RunnerException {     Options opt = new OptionsBuilder()             .include(JMHSample_01_HelloWorld.class.getSimpleName()             .forks(1)             .build();     new Runner(opt).run();   } } 01. 02. 03. 04. 05. 06. 07. 08. 89
  90. 90. JMH: example > mvn clean install > java ‐jar target/benchmarks.jar JMHSample_01_HelloWorld 01. 02. 90
  91. 91. JMH: annotations @Benchmark @BenchmarkMode @CompilerControl  @Fork @Group @GroupThreads @Level @Measurement @Mode @OperationsPerInvocation @OutputTimeUnit @Param @Scope @Setup @State @TearDown @Threads @Timeout @Warmup 01. 02. 03. 04. 05. 06. 91
  92. 92. JMH: output 92
  93. 93. Load testing tools jMeter ab gatling • • • 93
  94. 94. Next riddle 94
  95. 95. 95
  96. 96. Leak detection 96
  97. 97. Open source Leak Prevention Library Leak Canary • • 97
  98. 98. Comercial Plumbr AppDynamics NewRelic • • • 98
  99. 99. Next riddle 99
  100. 100. 100
  101. 101. Monitoring 101
  102. 102. JMX is your friend! 102
  103. 103. VisualVM (again) 103
  104. 104. Mission Control 104
  105. 105. Metrics collection 105
  106. 106. JMXTrans This is effectively the missing connector between speaking to a JVM via JMX on one end and whatever logging / monitoring / graphing package that you can dream up on the other end. “ 106
  107. 107. jHiccup jHiccup is a non­intrusive instrumentation tool that logs and records platform "hiccups" ­ including the JVM stalls that often happen when Java applications are executed and/or any OS or hardware platform noise that may cause the running application to not be continuously runnable. “ 107
  108. 108. jHiccup > java ‐javaagent:jHiccup.jar MyProgram01. 108
  109. 109. jHiccup 109
  110. 110. Drop Wizard Metrics Metrics provides a powerful toolkit of ways to measure the behavior of critical components in your production environment. “ 110
  111. 111. Netflix Hysterix 111
  112. 112. Stagemonitor 112
  113. 113. 113
  114. 114. Single JVM? 114
  115. 115. Really? 115
  116. 116. 116
  117. 117. Dead JVM 117
  118. 118. 118
  119. 119. Many JVMs 119
  120. 120. Open­source tools Graphite Graphana Logstash Kibana Graylog2 Fluentd • • • • • • 120
  121. 121. Comercial tools NewRelic Plumbr AppDynamics Dynatrace Takipi Hyperic • • • • • • 121
  122. 122. Final riddle 122
  123. 123. 123
  124. 124. Links 124
  125. 125. Links: GC http://spin.atomicobject.com/2014/09/03/visualizing­garbage­ collection­algorithms/ http://techblog.netflix.com/2013/05/garbage­collection­ visualization.html http://blog.sokolenko.me/2014/11/javavm­options­production.html https://blog.codecentric.de/en/2013/10/useful­jvm­flags­part­7­cms­ collector/ • • • • 125
  126. 126. Links: GC http://blog.ragozin.info/2011/09/hotspot­jvm­garbage­collection­ options.html http://blog.tier1app.com/2014/11/19/understand­garbage­collection­ log/ http://www.tagtraum.com/gcviewer­vmflags.html • • • 126
  127. 127. Links: GC http://fasterj.com/tools/gcloganalysers.shtml https://github.com/chewiebug/GCViewer http://www.jclarity.com/censum/ • • • 127
  128. 128. Links: Thread dumps http://www.javacodegeeks.com/2012/03/jvm­how­to­analyze­thread­ dump.html http://www.slideshare.net/buzdin/thread­dump­analysis http://docs.oracle.com/javase/7/docs/api/java/lang/Thread.State.html • • • 128
  129. 129. Links: Tools https://github.com/akullpp/awesome­java https://github.com/aragozin/jvm­tools http://openjdk.java.net/projects/code­tools/jmh/ • • • 129
  130. 130. Links: Tools http://youdebug.kohsuke.org/ https://java.net/projects/tda http://samuraism.jp/samurai/en/index.html • • • 130
  131. 131. Links: Tools https://github.com/jmxtrans/jmxtrans http://www.stagemonitor.org/ http://jamonapi.sourceforge.net/ https://github.com/Netflix/Hystrix • • • • 131
  132. 132. Links: Tools http://www.eclipse.org/tptp/home/documents/tutorials/profilingtool/profilingexample https://github.com/mjiderhamn/classloader­leak­prevention • • 132
  133. 133. Reading material 133
  134. 134. Java Performance 134
  135. 135. Java Performance: The Defenetive Guide 135
  136. 136. Java Performance And Scalability 136
  137. 137. This is it! 137
  138. 138. Questions? 138
  139. 139. Thank you! 139
  140. 140. Devternity 1­2 of December, 2016, Riga Software Excellence Conference 4 tracks, 30+ speakers, 6 workshops http://devternity.com • • • • 140

×