Monitoring and Tuning GlassFish


Published on

'In the Wild'
presentation by Steve Millidge

Published in: Technology
1 Comment
  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Monitoring and Tuning GlassFish

  1. 1. GlassFish Monitoring and Troubleshooting In the Wild Steve Millidge C2B2 Consulting Limited
  2. 2. CAVEAT If this works it’ll be a miracle!
  3. 3. Agenda <ul><li>Setting the Scene </li></ul><ul><ul><li>A Real World? Scenario </li></ul></ul><ul><li>Command Line Tools </li></ul><ul><li>Graphical Tools </li></ul>
  4. 4. Setting the Scene Troubleshooting in the Wild! (An Apology)
  5. 5. C2B2 Consulting <ul><li>Fast </li></ul><ul><ul><li>Performance and Scalability </li></ul></ul><ul><li>Reliable </li></ul><ul><ul><li>High Availability and Recoverability </li></ul></ul><ul><li>Manageable </li></ul><ul><ul><li>Monitoring and Supporting </li></ul></ul><ul><li>Secure </li></ul><ul><ul><li>Security </li></ul></ul>
  6. 6. First Day At Work
  7. 7. It’s All Gone to *#*@!
  8. 11. Meet the Project Manager
  9. 18. Where do You Start?
  10. 19. JVM Versions <ul><li>JDK 1.4 </li></ul><ul><ul><li>Black Box </li></ul></ul><ul><li>JDK 1.5 </li></ul><ul><ul><li>You can watch from outside </li></ul></ul><ul><li>JDK 1.6 </li></ul><ul><ul><li>You can sneak in by the back door </li></ul></ul>
  11. 20. JVM Command Line Tools <ul><li>jps </li></ul><ul><ul><li>Identified Running Machines on the Host </li></ul></ul><ul><li>jstat </li></ul><ul><ul><li>Provides Statistics on Running VMs </li></ul></ul><ul><li>jmap </li></ul><ul><ul><li>Provides Memory Information </li></ul></ul><ul><li>jstack </li></ul><ul><ul><li>Provides Stack Trace Information </li></ul></ul>
  12. 21. jps <ul><li>Lists the Running VMs on a Host </li></ul><ul><li>-l shows you the class name </li></ul><ul><li>-m The Command Line Parameters </li></ul><ul><li>-v full Java Options </li></ul>
  13. 22. GlassFish Command Line Tools <ul><li>asadmin </li></ul><ul><ul><li>Command line access to GlassFish </li></ul></ul><ul><li>Orientation </li></ul><ul><ul><li>list-domains </li></ul></ul><ul><ul><li>list-clusters </li></ul></ul><ul><ul><li>get-health </li></ul></ul><ul><ul><li>generate-diagnostic-report </li></ul></ul><ul><ul><li>generate-jvm-report </li></ul></ul>
  14. 23. GlassFish Monitoring <ul><li>Check Monitoring </li></ul><ul><ul><li>get server.monitoring-service.module-monitoring-levels.* </li></ul></ul><ul><li>Switch Monitoring On </li></ul><ul><ul><li>set server.monitoring-service.module-monitoring-levels.jvm=LOW </li></ul></ul><ul><li>Monitor </li></ul><ul><ul><li>Monitor –type jvm </li></ul></ul>
  15. 24. jstack <ul><li>Dumps a stack trace </li></ul><ul><li>Adds monitors and Synchronizers </li></ul><ul><li>C stack as well on Linux </li></ul>
  16. 25. jstat <ul><li>Provides Memory Stats </li></ul><ul><ul><li>Perm Gen, New Gen, Old Gen </li></ul></ul><ul><li>Provide Class Loading Stats </li></ul><ul><li>Provide Compilation Stats </li></ul>
  17. 26. jinfo <ul><li>Provides Information on the Running VM </li></ul><ul><ul><li>Command Line </li></ul></ul><ul><ul><li>System Properties </li></ul></ul><ul><ul><li>VM flags </li></ul></ul><ul><li>Changes the flags in the VM </li></ul><ul><li>Limited use on Windows </li></ul>
  18. 27. jmap <ul><li>Gives you visibility into the memory </li></ul><ul><li>See the Class Histogram </li></ul><ul><li>Dump the Memory to a File </li></ul>
  19. 28. jhat <ul><li>Heap Analyzer </li></ul><ul><li>Works from a Dump File </li></ul><ul><li>Browse the Heap via a Browser </li></ul>
  20. 29. Graphical Tools
  21. 30. GlassFish Admin Console
  22. 31. Admin Console
  23. 32. Visual VM <ul><li>New in 1.6u7 </li></ul><ul><li>Replaces JConsole </li></ul><ul><li>Number of plugins </li></ul><ul><li>Enables Heap Dumps </li></ul><ul><li>Thread Dumps </li></ul><ul><li>Simple Profiling </li></ul><ul><ul><li>Pretty Dodgy (Often Crashes VM) </li></ul></ul>
  24. 33. Third Party Tool
  25. 34. Memory Analyzer Tool <ul><li>Open Source </li></ul><ul><li>Developed by SAP Support </li></ul><ul><li>Analyses Heap Dumps </li></ul><ul><ul><li>Finds Culprits Easily </li></ul></ul><ul><ul><li>Indexes the dump for fast navigation </li></ul></ul>
  26. 35. Analysis <ul><li>We are Leaking Memory in a static </li></ul><ul><ul><li>Class name and variable </li></ul></ul><ul><li>We have a rogue thread spinning the cpu </li></ul><ul><ul><li>Full stack trace </li></ul></ul>
  27. 37. Exotica When things get really tough!
  28. 38. Exotica <ul><li>Oracle JRockit </li></ul><ul><li>Attach API </li></ul><ul><li>Instrumentation API </li></ul><ul><li>BTrace </li></ul>
  29. 39. Oracle JRockit <ul><li>Supports 1.4, 1.5, 1.6 </li></ul><ul><li>Ships with Many Advanced Tools </li></ul><ul><li>Mission Control </li></ul><ul><li>Leak Analyzer </li></ul><ul><ul><li>Looks for Increasing Dominators in Real Time </li></ul></ul><ul><li>Latency Analyzer </li></ul><ul><ul><li>Records cause of Thread Latency in Real Time </li></ul></ul><ul><li>Instrumentation in the VM itself </li></ul>
  30. 40. Attach API <ul><li>Enables tools to Dynamically Attach to a JVM </li></ul><ul><li>// attach to target VM VirtualMachine vm = VirtualMachine.attach(&quot;2177&quot;); // construct path to management String agent = &quot;management-agent.jar&quot;; // load agent into target VM vm.loadAgent(agent) </li></ul><ul><li>vm.loadAgent(agent); // detach vm.detach(); </li></ul>
  31. 41. Agents <ul><li>Agent is a JAR file </li></ul><ul><li>Define Premain Class in the Manifest </li></ul><ul><ul><li>Has method premain(String args) </li></ul></ul><ul><li>Define Agent Class in the Manifest </li></ul><ul><ul><li>Has Method agentmain(String args) </li></ul></ul>
  32. 42. Instrumentation <ul><li>JDK 6 support in place Class Replacement </li></ul><ul><li>Agents implement ClassFileTransformer </li></ul><ul><li>Register themselves as a Transformer </li></ul><ul><ul><li>Instrumentation.addTransformer </li></ul></ul><ul><li>Call restransform </li></ul><ul><ul><li>Instrumentation.retransform </li></ul></ul>
  33. 43. BTrace <ul><li>Uses Attach API and Instrumentation </li></ul><ul><li>Attaches an Agent into the VM </li></ul><ul><li>Injects “scripts” into a running JVM </li></ul><ul><li>Script transform class files and inject code </li></ul><ul><li> </li></ul>
  34. 44. Example Script // import all BTrace annotations import com.sun.btrace.annotations.*; // import statics from BTraceUtils class import static com.sun.btrace.BTraceUtils.*; // @BTrace annotation tells that this is a BTrace program @BTrace public class HelloWorld { @OnMethod ( clazz=&quot;java.lang.Thread&quot;, method=&quot;start&quot; ) public static void func() { println(&quot;about to start a thread!&quot;); } }
  35. 45. Thank you Questions?