While JMX is one of the oldest components of Java (JSR-3), few know about its actual power. This talk gives a short introduction on the basics of monitoring in general, what JMX is and how both, tech and business can benefit from a proper implementation. After this talk, you will know, how to export JMX metrics in your own projects, which common frameworks and libraries also provide JMX metrics and which tools for JMX monitoring are available. This talk contains content for Devs, Ops and Managers, as all of them can benefit from doing monitoring right.

1. Monitoring 101
Leverage on the Power of JMX
... and beyond
Martin Gutenbrunner
Dynatrace InnovationLab
@MartinGoodwell
Java Forum Stuttgart, July 7 #jfs2016

2. @MartinGoodwell
About me
 Started with Commodore 8-bit (VC-20 and C-64)
 Built Null-Modem connections for playing Doom and WarCraft I
 Went on to IPX/SPX networks between MS-DOS 6.22 and
WfW 3.11
 Did DevOps before it was a thing (mainly Java and Web)
for ~ 10 years
 Now at Dynatrace Innovation Lab
 Find me on Twitter: @MartinGoodwell
 Austria
Passionate about life,
technology, and the people
behind both of them.

3. @MartinGoodwell
Monitoring 101
Leverage on the Power of JMX
... and beyond
1
Agenda
2
3

4. @MartinGoodwell
Warm up
 Please, feel free to ask and interrupt anytime
 This is about you, after all
 How many developers, operators and business people are here?
 I love Java and Spring Framework
 Anyone here who hates one of them?
 Any previous experience with JMX or monitoring, anyone?
 Who knows what APM is?

5. @MartinGoodwell
Monitoring
... your application, not your environment

6. @MartinGoodwell
Why monitoring, when we can debug?
 Debugging is for Developers only
 Operations need monitoring
 Monitoring can be done via web apps
 Debugging requires a dev-env to be setup
 Monitoring is about two things
 Tracking problems
 Optimizing performance
 More often than not, those two things will be based on production data
 Plus: when you can track errors, you can also track business

7. @MartinGoodwell
Active vs passive approach
Two ways of thinking, two ways of integrating

8. @MartinGoodwell
Active
 JMX is an active way of monitoring
 You need to know, which metrics you want to monitor
 And actively publish/export them
 Pro
 You can monitor all the metrics you want, even those specific to your application
 Con
 Pollutes your code
 But there‘s ways around that, like AOP

9. @MartinGoodwell
Passive
 As opposed to passive monitoring
 Where tools automatically pick up the most common metrics, like
 Number of requests
 Round-trip times
 HTTP status codes
 by eg. AOP or Java agent interface
 I.e. without actively integrating monitoring code into your business logic
 Pro:
 No code pollution
 Con
 Only collects technology-related metrics (nr of requests, ...), no business metrics (like nr of orders)

10. @MartinGoodwell
JMX
A short introduction

11. @MartinGoodwell
JMX Trivia
 Java Management Extensions
 Counters, gauges and strings
 JMX is ooold:
 Integral part of Java since Java 5
 MBeans can consist of
 Readable/writable attributes (right, not only for reading values)
 Invokable operations (I am not really in favor of those)
 A description (rules of proper documentation apply)

12. @MartinGoodwell
Standard, Dynamic, Model, Open, MX
 MBean
 Static
 Dynamic
 ModelMBean (configurable)
 OpenMBean (specific data-types)
 MXBean (came with Java 6)
 XMBean is not an MXBean with a typo
 XMBean is a JBoss-specific MBean
 Bottomline: use MXBean

13. @MartinGoodwell
Why JMX?
 For Ops:
 Because most Java services / apps provide JMX metrics. You get it „for free“
 For Devs:
 It‘s not too hard to implement, even really easy with eg. Spring Framework
 For DevOps (ie both):
 Lots of visualization tools available, both free and commercial
 It‘s a unified way of monitoring (no matter whether it‘s a queue, a database or a
cache)
 For Business:
 It allows your devs to provide and your ops to report the metrics you need.

14. @MartinGoodwell
Which metrics should I monitor?
 Common process metrics
 CPU
 Memory
 Service specific metrics
 Webservers: Nr of requests and response times
 Databases: Connection pools utilization, etc.
 Caches: hits and misses
 Queues: size, fill-rate
 Business Intelligence
 Visitors
 Orders

15. @MartinGoodwell
Who provides JMX metrics out-of-the-box?
 JVM
 Threading, Memory usage, Garbage Collection, CPU usage (system, process, …)
 Web Servers
 Tomcat
 Jetty
 Netflix OSS
 Eureka Discovery Server
 Hystrix Circuit Breaker
 Connection Pools
 RDBMS (Tomcat DBCP, c3p0, BoneCP, HikariCP, …)
 MongoDB
 Messaging Queues (ie JMS)
 HornetQ
 Active MQ

16. @MartinGoodwell
... Wait, there‘s still more!
 Hibernate
 Spring
 Adds JMX to most libraries it wraps
 Quartz (the scheduler)
 EhCache (the cache)

17. @MartinGoodwell
Some examples

18. @MartinGoodwell
Java JMX

19. @MartinGoodwell
Tomcat JMX

20. @MartinGoodwell
MongoDB JMX

21. @MartinGoodwell
HornetQ

22. @MartinGoodwell
Business metrics
 Feature usage
 Number of placed orders
 Where do my customers come from?
 Track a customer‘s path:
 Catalog
 Shopping Cart
 Checkout
 Payment
 Dropouts

23. @MartinGoodwell
Correlating metrics
 On a technical basis
 Nr of requests vs response time
 Increasing response time alongside increasing number of requests probably pinpoints a
scalability problem
 The real fun starts when you correlate BI with technical metrics
 Eg. Feature usage vs error rate or response times
 Increasing errors in service X
 And decreasing usage of feature A at the same time
 There seems to be a relation
 Order rate vs response time?

24. @MartinGoodwell
Correlating Business and Tech
 Any error count makes a great candidate for correlation with most metrics

25. @MartinGoodwell
Build 17 testNewsAlert OK
testSearch OK
Build # Use Case Stat # API Calls # SQL Payload CPU
1 5 2kb 70ms
1 3 5kb 120ms
Tests Metrics
Build 26 testNewsAlert OK
testSearch OK
Build 25 testNewsAlert OK
testSearch OK
1 4 1kb 60ms
34 171 104kb 550ms
Ops
#ServInst Usage RT
1 0.5% 7.2s
1 63% 5.2s
1 4 1kb 60ms
2 3 10kb 150ms
1 0.2% 5.2s
5 75% 2.5s
Build 35 testNewsAlert -
testSearch OK
- - - -
2 3 10kb 150ms
- - -
8 80% 2.0s
Metrics from and for Dev(to)Ops
Re-architecture -> Performance Fixes

26. @MartinGoodwell
How to export your own metrics
For Ops: How to tell your Devs how to export their own metrics

27. @MartinGoodwell
By Donsez - self-made, CC BY-SA 3.0, https://en.wikipedia.org/w/index.php?curid=6721989

28. @MartinGoodwell
Vanilla JMX
https://docs.oracle.com/javase/tutorial/jmx/mbeans/standard.html

29. @MartinGoodwellhttps://docs.oracle.com/javase/tutorial/jmx/mbeans/standard.html

30. @MartinGoodwell
https://docs.oracle.com/javase/tutorial/jmx/mbeans/standard.html

31. @MartinGoodwell
JMX with Spring Boot
 Spring simplifies JMX by
 creating an MBean server automatically (as does eg Tomcat)
 less boilerplate code for registering your own MBeans

32. @MartinGoodwell

33. @MartinGoodwell
JMX Tools

34. @MartinGoodwell
JConsole, JVisualVM
 Pro
 Comes with JVM
 Con
 Once you quit, all data is lost

35. @MartinGoodwell
Nagios Core
 Pro
 Allows you to combine all different types of
metrics
 Host
 Application
 Con
 Very tedious to setup
 Dedicated plugins for each technology

36. @MartinGoodwell
Dynatrace
 Allows you to combine all different types of
metrics
 Host
 Application
 Zero-conf, auto-detection

37. @MartinGoodwell
Bridges (eg JMX to HTTP)
https://jolokia.org/reference/html/protocol.html

38. @MartinGoodwell
Downsides of JMX
 For Java code only
 Finding the right spots for monitoring might require some iterations
 Potential source of „Hell Breaks Loose“
 Triggering methods out of context
 Changing configuration values in-memory only

39. @MartinGoodwell
Alternatives?

40. @MartinGoodwell
statsd real quick
http://www.slideshare.net/DatadogSlides/dev-opsdays-tokyo2013effectivestatsdmonitoring
@MartinGoodwell

41. @MartinGoodwell
Java code calling statsd client
http://rick-hightower.blogspot.co.uk/2015/05/working-with-statsd-and-java.html

42. @MartinGoodwell
Why JMX over other alternatives?
 You might be using it already for JVM-metrics (or Tomcat, etc)
 GC, CPU-usage, requests, etc
 While still „polluting“ your code, interfaces allow for good structures
 Allows to interact with the application (although I wouldn‘t recommend this)
 Notifications on a code-level
(https://docs.oracle.com/javase/tutorial/jmx/notifs/index.html)
 Comes with JVM – saves you from bloating small solutions with dependencies

43. @MartinGoodwell
Shout out
Right after lunch break at 14.30
In Room Silcher-Saal

44. @MartinGoodwell
“Go then, there are other worlds than these.”
— Jake Chambers, The Dark Tower
martin.gutenbrunner@dynatrace.com
@MartinGoodwell
Dynatrace Innovation Lab
http://blog.ruxit.com
http://www.dynatrace.com

45. @MartinGoodwell
References
 JMX-Technology homepage
 http://www.oracle.com/technetwork/java/javase/tech/javamanagement-140525.html
 Monitoring and Managing the Java Platform using JMX technology
 https://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html
 Call-tracing and performance management in microservice environments
 http://www.slideshare.net/MartinGoodwell/performance-monitoring-and-call-tracing-in-
microservice-environments
 Jolokia https://jolokia.org/
 Nagios https://www.nagios.org/
 statsd https://github.com/etsy/statsd/wiki
 Sleepless Dev http://rick-hightower.blogspot.co.uk