JavaOne2013: Build Your Own Runtime Monitoring for the IBM JDK with the Health Center API

  • 743 views
Uploaded on

In the recently released Health Center version 2.2 of the IBM JDK, a new API was made available that makes it possible to create your own monitoring and profiling tools that uses the Health Center …

In the recently released Health Center version 2.2 of the IBM JDK, a new API was made available that makes it possible to create your own monitoring and profiling tools that uses the Health Center data and recommendations. This session provides an overview of the API, shows you how to use it to create simple alerts based on the occurrence of defined conditions, and explores how it is being used by IBM to integrate the Health Center data into its own products.

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
743
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
45
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. © 2013 IBM Corporation Chris Bailey – IBM Java Service Architect 26th September 2013 Build your own Monitoring With the Java Remote Monitoring API Document number
  • 2. © 2013 IBM Corporation Important Disclaimers THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES. ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS 2
  • 3. © 2013 IBM Corporation3 Introduction to the speaker  13 years experience developing and deploying Java SDKs  Recent work focus: – Java applications in the cloud – Java usability and quality – Debugging tools and capabilities – Requirements gathering – Highly resilient and scalable deployments  My contact information: – baileyc@uk.ibm.com – http://www.linkedin.com/in/chrisbaileyibm – http://www.slideshare.net/cnbailey/
  • 4. © 2013 IBM Corporation4 Goals of this talk  Introduce the IBM tools for monitoring, debugging and diagnosing problems in the deployment life cycle  Understand the capabilities of Health Center  Learn how to build your own tools using the Health Center data feed  See how IBM is building on top of the Java tools platform
  • 5. © 2013 IBM Corporation5 IBM Tools for Java
  • 6. © 2013 IBM Corporation6 IBM Monitoring and Diagnostic Tools for Java  A free suite of tools for understanding different aspects of Java applications – Garbage Collection and Memory Visualizer (GCMV) – Memory Analyzer (and IBM Extensions for Memory Analyzer) – Interactive Diagnostic Data Explorer (IDDE) – Health Center  Provide visualization, analysis and recommendations  Available in the IBM Support Assistant – Provides the “toolbox” in which analysis tools reside – Assist with opening PMRs and working with IBM Support – Search feature to query IBM and non-IBM knowledge banks
  • 7. © 2013 IBM Corporation7 Garbage Collection and Memory Visualizer  Graphical Display of Data – Allows graphing of all available data: pause times, heap size etc – Allows zoom, cropping and change of axes value and units – Allows comparison of multiple files  Analysis and Recommendations – Provides tuning recommendations based on data and flags errors. – Analysis can be limited using cropping. – Values and units used in analysis can be changed by changing axes values and units
  • 8. © 2013 IBM Corporation8 Memory Analyzer  Overview: – Overview of the dump including Java heap size and total number of objects. – Provides links to continued analysis  Path to GC Roots: – Provides the reference chain that prevents an object being garbage collected.  Dominator Tree grouped by Class Loader: – Lists the biggest objects using a “keep alive tree”. Grouping by Class – Loader limits the analysis to a single application in a JEE environment  Provides extension API – Used to provide product based extensions
  • 9. © 2013 IBM Corporation9 Interactive Diagnostic Data Explorer  Multi-platform system dump analyzer – AIX, Linux, i/OS, Windows, z/OS  Provides dump portability – Analyse dump on any platform  Has knowledge of data structures in dump: – Java, JVM, and some OS structures  Provides collaborative analysis – Store progress and notes with the dump  Provides extension API – Used to provide automated analysis
  • 10. © 2013 IBM Corporation10 Health Center  Live monitoring tool with very low overhead – Pre-installed in IBM JDKs since Java 5 – Enable at startup: -Xhealthcenter – Enable at runtime: java -jar healthcenter.jar ID=<pid>  Understand how your application is behaving – Monitor Class loading, File I/O, Environment settings, Garbage Collection, Allocations, Method Profiling, Locking, Native memory use  Diagnose potential problems, with recommendations  Suitable for all Java applications
  • 11. © 2013 IBM Corporation11 Health Center  Environment reporting – Detects invalid Java options – Detects options which may hurt performance or serviceability – Useful for remote diagnosis of configuration-related problems  Class loading visualization – Shows all loaded classes – Shows load time – Identifies shared classes  File I/O – Monitor application file open/close events as they occur – Lists currently open files
  • 12. © 2013 IBM Corporation12 Health Center  Garbage Collection visualization – Visualizes heap usage and gc pause times over time – Identifies memory leaks – Suggests command-line and tuning parameters – Java Object allocation profiling by allocating code or type  Native Memory – Detect native memory leaks in application – Determine if external forces are using more memory – Memory counters showing which parts of the JVM are using the most native memory  Threads view – List of current threads and states – Number of threads over time – See contended monitors
  • 13. © 2013 IBM Corporation13 Health Center  Method Profiling – Always-on profiling shows application activity – Identifies the hottest methods in an application – Full call stacks to identify where methods are being called from and what methods they call – No byte code instrumentation, no recompiling  Java Lock Profiling – Always-on lock monitoring – Quickly allows the usage of all locks to be profiled – Helps to identify points of contention in the application that are preventing scaling  Live control of application – Trigger dumps – Enable verbosegc collection
  • 14. © 2013 IBM Corporation14 Health Center Demo
  • 15. © 2013 IBM Corporation15 Health Center: Overhead  Measured using WebSphere App Server and the DayTrader benchmark with 50 clients – Running WAS 8.5.5, IBM Java 7 SR5, AIX 7.1, POWER7 – Throughput determined by number of completed transactions on 4 saturated CPUs 94 95 96 97 98 99 100 101 102 103 100 Application Throughput (percentage of baseline) Baseline Dormant (level=off) Network Client Headless client (level=headless) Headless client with ZIP (level=headless)
  • 16. © 2013 IBM Corporation16 Health Center: Overhead  Measured using WebSphere App Server and the DayTrader benchmark with 50 clients – Running WAS 8.5.5, IBM Java 7 SR5, AIX 7.1, POWER7 – Throughput determined by number of completed transactions on 4 saturated CPUs 94 95 96 97 98 99 100 101 102 103 100 101.76 Application Throughput (percentage of baseline) Baseline Dormant (level=off) Network Client Headless client (level=headless) Headless client with ZIP (level=headless)
  • 17. © 2013 IBM Corporation17 Health Center: Overhead  Measured using WebSphere App Server and the DayTrader benchmark with 50 clients – Running WAS 8.5.5, IBM Java 7 SR5, AIX 7.1, POWER7 – Throughput determined by number of completed transactions on 4 saturated CPUs 94 95 96 97 98 99 100 101 102 103 100 101.76 100.4 Application Throughput (percentage of baseline) Baseline Dormant (level=off) Network Client Headless client (level=headless) Headless client with ZIP (level=headless)
  • 18. © 2013 IBM Corporation18 Health Center: Overhead  Measured using WebSphere App Server and the DayTrader benchmark with 50 clients – Running WAS 8.5.5, IBM Java 7 SR5, AIX 7.1, POWER7 – Throughput determined by number of completed transactions on 4 saturated CPUs 94 95 96 97 98 99 100 101 102 103 100 101.76 100.4 99.01 Application Throughput (percentage of baseline) Baseline Dormant (level=off) Network Client Headless client (level=headless) Headless client with ZIP (level=headless)
  • 19. © 2013 IBM Corporation19 Health Center: Overhead  Measured using WebSphere App Server and the DayTrader benchmark with 50 clients – Running WAS 8.5.5, IBM Java 7 SR5, AIX 7.1, POWER7 – Throughput determined by number of completed transactions on 4 saturated CPUs 94 95 96 97 98 99 100 101 102 103 100 101.76 100.4 99.01 96.99 Application Throughput (percentage of baseline) Baseline Dormant (level=off) Network Client Headless client (level=headless) Headless client with ZIP (level=headless)
  • 20. © 2013 IBM Corporation20 Health Center  Low overhead is achieved by surfacing existing data used by the JVM internally – Performance overhead is therefore cost of storage or IO  Garbage Collection Data: – GC collects metrics to determine how to self tune • Heap occupancy: expansion or contraction of the heap • GC durations: expansion/contraction, concurrent kick-off  Method Profiling: – JIT compiler collects metrics to determine: • When to compile: high use methods compiled first • How to compile: optimize for most common flow through code • When to recompile: assumptions are now incorrect
  • 21. © 2013 IBM Corporation21 Java Monitoring API
  • 22. © 2013 IBM Corporation22 Health Center and the Java Monitoring API  Health Center obtains its data directly from the Java runtime and produces analysis and recommendations shown in the Health Center client  The Java runtime's data is now provided to any consumer via the Java Monitoring API – Allows any tool to receive, analyze and interpret the data – Allows those tools to additionally obtain the analysis and recommendations  IBM Java Runtime provides a monitoring platform!
  • 23. © 2013 IBM Corporation23 Using the API  The API can be used in three modes: – Remote: connecting over a socket connection (with security/authentication etc) – Local: running inside the monitored application (using a local loop socket) – Offline: loading a pre-saved binary “.hcd” file  Performance cost to monitored application is the same as for Health Center – Running in “local” mode adds has additional impact due to the the cost of parsing data
  • 24. © 2013 IBM Corporation24 Using the API  Three step process for obtaining data from the API:  Obtain a Health Center object – Using either a File or a Connection  Obtain a data type object from the Health Center object – getCpuData() / getGCData() / getThreadsData() / getIOData() / etc  Request data points from the data type object: – getCpuData().getProcessUse() / getCpuData.getSystemUse()
  • 25. © 2013 IBM Corporation25 Obtain a Health Center Object: File import com.ibm.java.diagnostics.healthcenter.api.ConnectionProperties; import com.ibm.java.diagnostics.healthcenter.api.HealthCenter; import com.ibm.java.diagnostics.healthcenter.api.HealthCenterException; import com.ibm.java.diagnostics.healthcenter.api.factory.HealthCenterFactory; public class GCGraphPrototype { static HealthCenter hcAPI = null; public static void main(String[] args) { new GCGraphPrototype(); } GCGraphPrototype() { try { hcAPI = HealthCenterFactory.connect(new File("C:healthcenter.hcd")); } catch (HealthCenterException | FileNotFoundException e) { e.printStackTrace(); } testGCAPI(); } ... Use factory to create Health Center Object Connect to a File provider
  • 26. © 2013 IBM Corporation26 Obtain a Health Center Object: Connection import com.ibm.java.diagnostics.healthcenter.api.ConnectionProperties; import com.ibm.java.diagnostics.healthcenter.api.HealthCenter; import com.ibm.java.diagnostics.healthcenter.api.HealthCenterException; import com.ibm.java.diagnostics.healthcenter.api.factory.HealthCenterFactory; public class GCGraphPrototype { static HealthCenter hcAPI = null; public static void main(String[] args) { new GCGraphPrototype(); } GCGraphPrototype() { ConnectionProperties props = new ConnectionProperties(); try { hcAPI = HealthCenterFactory.connect(props, true); } catch (HealthCenterException e) { e.printStackTrace(); } testGCAPI(); } ... Describe your connection target: localhost:1972 Connect to your target
  • 27. © 2013 IBM Corporation27 Obtain a Health Center Object: Connection import com.ibm.java.diagnostics.healthcenter.api.ConnectionProperties; import com.ibm.java.diagnostics.healthcenter.api.HealthCenter; import com.ibm.java.diagnostics.healthcenter.api.HealthCenterException; import com.ibm.java.diagnostics.healthcenter.api.factory.HealthCenterFactory; public class GCGraphPrototype { static HealthCenter hcAPI = null; public static void main(String[] args) { new GCGraphPrototype(); } GCGraphPrototype() { ConnectionProperties props = new ConnectionProperties("192.168.1.105", 35535)); try { hcAPI = HealthCenterFactory.connect(props, true); } catch (HealthCenterException e) { e.printStackTrace(); } testGCAPI(); } ... Describe a target of 192.168.1.105:35535
  • 28. © 2013 IBM Corporation28 Obtain a Health Center Object: Connection Security import com.ibm.java.diagnostics.healthcenter.api.ConnectionProperties; import com.ibm.java.diagnostics.healthcenter.api.HealthCenter; import com.ibm.java.diagnostics.healthcenter.api.HealthCenterException; import com.ibm.java.diagnostics.healthcenter.api.factory.HealthCenterFactory; public class GCGraphPrototype { static HealthCenter hcAPI = null; public static void main(String[] args) { new GCGraphPrototype(); } GCGraphPrototype() { ConnectionProperties props = new ConnectionProperties("192.168.1.105", 35535)); props.setJmxUsername("Username"); props.setJmxPassword("Password"); try { hcAPI = HealthCenterFactory.connect(props, true); } catch (HealthCenterException e) { e.printStackTrace(); } testGCAPI(); } ... Add JMX authentication
  • 29. © 2013 IBM Corporation29 Obtain a Health Center Object: Connection Security import com.ibm.java.diagnostics.healthcenter.api.ConnectionProperties; import com.ibm.java.diagnostics.healthcenter.api.HealthCenter; import com.ibm.java.diagnostics.healthcenter.api.HealthCenterException; import com.ibm.java.diagnostics.healthcenter.api.factory.HealthCenterFactory; public class GCGraphPrototype { static HealthCenter hcAPI = null; public static void main(String[] args) { new GCGraphPrototype(); } GCGraphPrototype() { ConnectionProperties props = new ConnectionProperties("192.168.1.105", 35535)); props.setSSLTruststoreLocation("Location"); props.setSSLTruststorePassword("Password"); try { hcAPI = HealthCenterFactory.connect(props, true); } catch (HealthCenterException e) { e.printStackTrace(); } testGCAPI(); } ... Add SSL encryption
  • 30. © 2013 IBM Corporation30 testGCAPI(): private static void testGCAPI() { GCGraph demo = new GCGraph(); demo.updateDataSet(hcAPI.getGCData().getGCPauseTimeData()); demo.pack(); demo.setVisible(true); while (true) { demo.updateDataSet(hcAPI.getGCData().getGCPauseTimeData()); demo.repaint(); try { Thread.sleep(2 * 1000); } catch (InterruptedException e) { e.printStackTrace(); } } } Get the GC Data provider Get information on Pause Times
  • 31. © 2013 IBM Corporation31 testGCAPI(): private static void testGCAPI() { GCGraph demo = new GCGraph(); demo.updateDataSet(hcAPI.getGCData().getUsedHeapData()); demo.pack(); demo.setVisible(true); while (true) { demo.updateDataSet(hcAPI.getGCData().getUsedHeapData()); demo.repaint(); try { Thread.sleep(2 * 1000); } catch (InterruptedException e) { e.printStackTrace(); } } } Get Java Heap usage information
  • 32. © 2013 IBM Corporation32 testGCAPI(): private static void testGCAPI() { GCGraph demo = new GCGraph(); demo.updateDataSet(hcAPI.getGCData().getHeapSizeData()); demo.pack(); demo.setVisible(true); while (true) { demo.updateDataSet(hcAPI.getGCData().getHeapSizeData()); demo.repaint(); try { Thread.sleep(2 * 1000); } catch (InterruptedException e) { e.printStackTrace(); } } } Get Java Heap size information
  • 33. © 2013 IBM Corporation33 testCPUAPI(): private static void testCPUAPI() { GCGraph demo = new GCGraph(); demo.updateDataSet(hcAPI.getCpuData().getProcessUse()); demo.pack(); demo.setVisible(true); while (true) { demo.updateDataSet(hcAPI.getCpuData().getProcessUse()); demo.repaint(); try { Thread.sleep(2 * 1000); } catch (InterruptedException e) { e.printStackTrace(); } } } Get the CPU Data provider Get information on JVM CPU usage
  • 34. © 2013 IBM Corporation34 testCPUAPI(): private static void testCPUAPI() { GCGraph demo = new GCGraph(); demo.updateDataSet(hcAPI.getCpuData().getSystemUse()); demo.pack(); demo.setVisible(true); while (true) { demo.updateDataSet(hcAPI.getCpuData().getSystemUse()); demo.repaint(); try { Thread.sleep(2 * 1000); } catch (InterruptedException e) { e.printStackTrace(); } } } Get information on overall CPU usage
  • 35. © 2013 IBM Corporation35 GCGraph Class: public class GCPauseGraph extends JFrame { private XYSeriesCollection gcDataSet; private static final long serialVersionUID = 1L; public GCGraph() { super(); gcDataSet = new XYSeriesCollection(); JFreeChart lineChartObject = ChartFactory.createXYLineChart( “gc pauses”, "time (seconds)", "pause time (ms)", getGCDataSet(), PlotOrientation.VERTICAL, true, true, false); ChartPanel chartPanel = new ChartPanel(lineChartObject); chartPanel.setPreferredSize(new java.awt.Dimension(500, 270)); setContentPane(chartPanel); } Extends Swing JFrame Store data as X,Y values Create a JFreeChart line plot Add the JFreeChart to the JFrame
  • 36. © 2013 IBM Corporation36 GCGraph Class: public void updateDataSet(PauseData[] pD) { final XYSeries series1 = new XYSeries("Pause Times"); if (pD != null) { for (int i = 0; i < pD.length; i++) { series1.add(pD[i].getTime() / 1000, pD[i].getPauseTime()); } getGCDataSet().removeAllSeries(); getGCDataSet().addSeries(series1); } } Create a JFree XYSeries Populate with data Remove old data Add new data
  • 37. © 2013 IBM Corporation37 Monitoring API Demo
  • 38. © 2013 IBM Corporation38 Usage by IBM
  • 39. © 2013 IBM Corporation39 Rational Application Developer 9.0  In addition to supporting trace-based profiling, Rational Application Developer v9.0 Beta now supports sample-based profiling for diagnosing execution time problems.  Using the Monitoring API, introduced in Health Center v2.1 – Very low overhead compare with trace-based profiling. – Available on all supported profiling launch types, including profiling on WAS servers.
  • 40. © 2013 IBM Corporation40 Rational Application Developer 9.0
  • 41. © 2013 IBM Corporation41 Rational Application Developer 9.0
  • 42. © 2013 IBM Corporation42 Rational Application Developer 9.0
  • 43. © 2013 IBM Corporation43 Rational Application Developer 9.0
  • 44. © 2013 IBM Corporation44 Summary  IBM is providing a very low overhead mechanism for accessing monitoring and profiling data  Allows building of your own tools  Same mechanism is being used by IBMs tools
  • 45. © 2013 IBM Corporation45 http://ibm.co/JavaOne2013
  • 46. © 2013 IBM Corporation46 References  IBM Monitoring and Diagnostic Tools for Java™ – http://www.ibm.com/developerworks/java/jdk/tools/  IBM Extensions for Memory Analyzer – http://www.alphaworks.ibm.com/tech/iema  Debugging WebSphere Application Server from Dumps (MA) – http://www.ibm.com/developerworks/websphere/techjournal/1103_supauth/1103_supaut h.html  IBM Support Assistant (ISA) – http://www.ibm.com/software/support/isa  Email: javatool@uk.ibm.com
  • 47. © 2013 IBM Corporation47 Copyright and Trademarks © IBM Corporation 2013. All Rights Reserved. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., and registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web – see the IBM “Copyright and trademark information” page at URL: www.ibm.com/legal/copytrade.shtml