Download to read offline
Uploaded byLeonardo Gomes
Java Flight Recorder & Flamegraphs
Presentation done at Riviera Dev 18 by Nenad Bogojevic & Leonardo F. Gomes
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Java Flight Recorder & Flamegraphs
- 1. Fire & Fury: JavaFlight Recorder & Flamegraphs ©AmadeusITGroupanditsaffiliatesandsubsidiaries
- 2. _Java Flight Recorder _JavaMission Control & jcmd _Flamegraphs _Flamegrapher 2 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Agenda
- 3. A little bitof history ©AmadeusITGroupanditsaffiliatesandsubsidiaries
- 4.
- 5.
- 6. _An “airplane flightrecorder” for JVM _Recording telemetry information while the JVM is running _Has very low overhead (< 1% for the long running application) _Bundled with JVM _Extensible _CPU profiling outside of safepoints 6 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Java Flight Recorder
- 7. _CPU & threadstack sampling _Garbage collection information _Exceptions created _File and socket operations _Locks that were held for longer than 10ms _Allocations on Thread Local Allocation Buffer (TLAB) and outside TLABs. 7 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Java Flight Recorder (A subset of) collected events
- 8. _Use cases • Profilingof running application • Debugging or investigating issues • Black box analysis & anomaly detection _Use • Can be started at launch • Or dynamically for the running project 8 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Java Flight Recorder Use Cases
- 9.
- 10. _Tool for managing,monitoring, profiling JVM processes • $JAVA_HOME/bin/jmc • JMX Console • Java Flight Recorder _Online from running program or offline from JFR dump 10 ©AmadeusITGroupanditsaffiliatesandsubsidiaries What is Mission Control
- 11.
- 12. _Tool to senddiagnostic commands to JVM _Replaces: jheap, jmap, jps, jstack, jstat _Usage: • jcmd <process id|main class> <command> • jcmd • jcmd my.MainClass help • jcmd my.MainClass VM.command_line • jcmd 3871 Thread.print 18 ©AmadeusITGroupanditsaffiliatesandsubsidiaries jcmd jcmd
- 13.
- 14. 20 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Oracle JDK andOpen JDK, 8 vs 10 jcmd help The following commands are available: JFR.configure JFR.stop JFR.start JFR.dump JFR.check VM.log VM.native_memory VM.check_commercial_features VM.unlock_commercial_features ManagementAgent.status ManagementAgent.stop ManagementAgent.start_local ManagementAgent.start Compiler.directives_clear Compiler.directives_remove Compiler.directives_add Compiler.directives_print VM.print_touched_methods Compiler.codecache Compiler.codelist Compiler.queue VM.classloader_stats Thread.print JVMTI.data_dump JVMTI.agent_load VM.stringtable VM.symboltable VM.class_hierarchy VM.systemdictionary GC.class_stats GC.class_histogram GC.heap_dump GC.finalizer_info GC.heap_info GC.run_finalization GC.run VM.info VM.uptime VM.dynlibs VM.set_flag VM.flags VM.system_properties VM.command_line VM.version help The following commands are available: JFR.stop JFR.start JFR.dump JFR.check VM.native_memory VM.check_commercial_features VM.unlock_commercial_features ManagementAgent.stop ManagementAgent.start_local ManagementAgent.start GC.rotate_log Thread.print GC.class_stats GC.class_histogram GC.heap_dump GC.run_finalization GC.run VM.uptime VM.flags VM.system_properties VM.command_line VM.version help JDK 8 JDK 10 The following commands are available: GC.rotate_log Thread.print GC.class_stats GC.class_histogram GC.heap_dump GC.run_finalization GC.run VM.uptime VM.flags VM.system_properties VM.command_line VM.version help Open JDK
- 15.
- 16. _Data visualization developedby Brendan Gregg _Trying to make sense of the output provided by Linux perf _Allows you to: • Identify most frequent code-paths easily • Once you understand how it works :) 22 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Flamegraphs?
- 19.
- 21.
- 22.
- 23. SAMPLING Main Parse HTTP headers ParseHTTP parameters
- 24.
- 25.
- 26. SAMPLING Main Call service Call ORM Calldatabase driver
- 27.
- 28.
- 29.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36. Credits: Brendan Gregg a( ) b ( ) c ( ) d ( ) e ( ) f ( ) g ( ) h ( ) i ( )
- 37. Credits: Brendan Gregg a( ) b ( ) c ( ) d ( ) e ( ) f ( ) g ( ) h ( ) i ( ) Top down shows the stack
- 38. a ( ) b( ) c ( ) d ( ) e ( ) f ( ) g ( ) h ( ) i ( ) What’s running on CPU Credits: Brendan Gregg
- 39. a ( ) b( ) c ( ) d ( ) e ( ) f ( ) g ( ) h ( ) i ( ) How much (width) Credits: Brendan Gregg
- 40. Credits: Brendan Gregg a( ) b ( ) c ( ) d ( ) e ( ) f ( ) g ( ) h ( ) i ( ) Widths are proportional to presence in samples
- 41. _Flamegraphs are great! _Weneed a way to get them out of JFR recordings 47 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Java Flight Recorder & Flamegraphs
- 42.
- 43.
- 44. 50 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Java Flight Recorder& Flamegraphs As off today, JFR is a commercial feature of Oracle JDK Non-production usage is free
- 45. 51 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Java Flight Recorder& Flamegraphs Start a new record Using the profiling settings
- 46. 52 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Java Flight Recorder& Flamegraphs Dump the recording Specify location
- 47. 53 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Java Flight Recorder& Flamegraphs Dumping a recording doesn’t stop it We need to explicitly stop it.
- 48.
- 49. 55 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Java Flight Recorder& Flamegraphs By default, creates a CPU flamegraph. Needs to have Brendan Gregg’s repository cloned locally for SVG generation
- 50.
- 51.
- 52. _ To generatea flamegraph: • Run JCMD • Start a recording • Dump the record • Stop the recording • Run tool to generate SVG • Open in the your favorite browser 58 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Why Flamegrapher?
- 53. 59 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Wouldn’t it benicer to simply open the browser?
- 54.
- 55.
- 57. a ( ) b( ) c ( ) d ( ) e ( ) f ( ) g ( ) h ( ) i ( ) For how long threads have been held waiting to acquire this lock? Locks flamegraphs
- 58. a ( ) b( ) c ( ) d ( ) e ( ) f ( ) g ( ) h ( ) i ( ) Width is time in milliseconds Locks flamegraphs
- 59. a ( ) b( ) c ( ) d ( ) e ( ) f ( ) g ( ) h ( ) i ( ) Stack trace leads to an exception (or error) being created Exceptions flamegraphs
- 60. g ( ) a( ) b ( ) c ( ) d ( ) e ( ) f ( ) h ( ) i ( ) Width is the number of instances of that exception (or error), that have been created Exceptions flamegraphs
- 61.
- 62.
- 63.
- 64. _https://github.com/google/pprof (golang) _https://github.com/evanhempel/python-flamegraph (python) _https://nodejs.org/en/blog/uncategorized/profiling-node-js/(nodejs) _https://github.com/jstepien/flames/ (clojure) 70 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Other languages
- 65.
- 66.
- 67. Thank you! ©AmadeusITGroupanditsaffiliatesandsubsidiaries You canfollow us on: AmadeusITgroup amadeus.com amadeus.com/blog
- 68.
Editor's Notes
- #28 Sampling is like taking a snapshot of what’s on CPU at a given moment; Imagine we have a program running and it’s like an HTTP-based server.
- #29 - It receives a request and starts by parsing its headers
- #30 - Next time we take a snapshot it’s parsing HTTP parameters
- #31 - On the next snapshot it’s calling a certain service
- #32 - New snapshot and the service is calling the ORM layer
- #33 - We sample again the the ORM layer is calling the code of a database driver
- #34 New snapshot and it’s the service code that is executing
- #35 Again, service code
- #36 We sample again and it’s building an HTTP response body
- #37 New sample and it’s calling a method to add http headers to the response
- #38 We’re finally back to main, probably waiting for new requests to be processed
- #39 Now imagine grouping all the parts that are common to the different samples
- #41 And if we look at the them grouped we can have an idea of the where we’re spending time in our program.
- #42 Finally, that flamegraph probably starts making more sense
- #51 http://openjdk.java.net/jeps/328
- #74 Backcover option 2