This document discusses Java memory leaks. It provides an overview of Java memory management and the behaviors observed with memory leaks. It describes how to generate heap dumps and analyze them using tools like Eclipse Memory Analyzer to identify leaked objects and their referencing paths. Specifically, it outlines how to use MAT to find leak suspects, inspect objects' references, and view thread stacks to locate the root cause of memory leaks.

1. Java Memory Leak

2. Agenda
1. Supported JDK Versions in Mule
2. Introduction to JVM Memory Management
3. What is memory leak and what behaviours do you observe?
4. Tools for memory analysis
5. What is Heap Dump?
6. Basic theory
7. Example
8. Summary

3. Supported Java Versions
 Mule Runtime Environments:
 3.8 (Extended Support till Nov/2021) -> JDK 1.7 and recommended 1.8.0_91/92
 3.9 (Extended Support till Mar/2024) -> JDK 1.8.0_144
 4.1 (Extended Support till Nov/2022) -> JDK 1.8.0_151/52
 4.2 (Extended Support till May/2022) -> OpenJDK 8 recommended over Java
11/other JDK 8 versions
Links:
https://www.mulesoft.com/legal/versioning-back-support-policy
https://docs.mulesoft.com/release-notes/mule-runtime/mule-4.2.0-release-notes

4. Java 8 JVM Memory Management

5. Java 8 Memory Management
 There are three main parts in JVM:
1. Young/New Generation:
 Newly created/started objects are stored in this region.
 When the young generation fills up, a minor garbage collection happens and either objects get cleaned
or moved to the Survivor region.
 World Stop Events
2. Old Generation:
 Long surviving objects get stored in this region.
 A threshold is set in the Young Generation and when objects are met, they get moved to the Old
Generation.
 When a garbage collection happens in the Old Generation, it is called a major garbage collection.
 World Stop Events: Longer than a minor garbage collection
3. Metaspace:
 Store meta information about loaded classes.
 No instances of class Class.

6. Two Main Memory Leaks
 Memory Leak in HEAP (What we are going to look at)
[Exception in thread "main" java.lang.OutOfMemoryError: Java heap space]
 Used objects are no longer required and supposed to be garbage collected.
 Somewhere they are being referenced during the life time of the JVM process.
 This results in consuming more memory.
 Memory Leak in METASPACE
[java.lang.OutOfMemoryError: Metaspace]
 This could be due to the inadequate sizing of the Metaspace.
 There is a classloader/classes leak.
 This is quite rare in Mule.

7. Behaviours in Anypoint Platform
 Used objects are not de-referenced and some active objects are referencing the used
objects. Therefore the conditions for the JVM garbage collector to delete used objects
are no longer met.
 In Anypoint platform, Memory Usage goes up and CPU Usage goes up too.
 Slow down the application and responses from the application take longer.

8. How do we get Heap Dump?
 Download and install the on-prem runtime environment from MuleSoft.
 Set JVM Config under mule-enterprise-standalone-4.1.5-
hf1/conf/wrapper.conf
wrapper.java.additional.19=-XX:+PrintGCDetails
wrapper.java.additional.20=-XX:+PrintGCDateStamps
wrapper.java.additional.21=-Xloggc:%MULE_HOME%/logs/gc.log
wrapper.java.additional.22=-XX:+UseGCLogFileRotation
wrapper.java.additional.23=-XX:NumberOfGCLogFiles=5
wrapper.java.additional.24=-XX:GCLogFileSize=10M
 Set JAVA_HOME and other environment variables (Linux):
MULE_HOME=/home/jee/Tools/Environments/mules/mule-enterprise-standalone-4.1.5-hf1.nolicense/
JAVA_HOME=/home/jee/Tools/JDK/jdk1.8.0_131
PATH="$JAVA_HOME/bin:$MAVEN_HOME/bin:$HOME/bin:$HOME/.local/bin:$PATH"
 Deploy app under $MULE_HOME/apps
 To Start Mule locally: cd $MULE_HOME/bin
./mule start

9. Taking Heap Dumps
 Several ways to take Heap Dumps from Java processes and HPROF is a
preferred format.
1. jmap -dump:format=b,file=cheap.hprof <PID>
2. jcmd <PID> GC.heap_dump <full path/filename.hprof>
3. VisualVM: GUI to generate a heap dump

10. Tools For Heap Dump Analysis
There are many tools available online and here are free ones listed.
 Eclipse Java Memory Analyser Tool
https://www.eclipse.org/mat/
I prefer using MAT.
 VisualVM
https://visualvm.github.io/
 Netbeans Profiler
https://netbeans.org/kb/docs/java/profiler-intro.html

11. Understanding of Heap Dump
 The snapshot contains information about the Java objects and classes in the
heap at the moment the snapshot is triggered.
 Typical information in a heap dump
 All Objects: Class, fields, primitive values, and references.
 All Classes: Class loader, name, super class, and static fields.
 Garbage collection roots: Objects defined to be reachable by the JVM.
 Thread Stacks and Local variables
 The data structure of Java heap is a directed graph of object references.
 Think of it as a graph structured data containing references that may have
primitives such as byte, integers, longs, etc.

12. Understanding of Heap Dump
 Incoming References: Think of it as Parents
 Outgoing References: Think of it as Children
 These terms can’t really be used as the definite terms as they can be referenced
to each other.

13. Understanding of Heap Dump
 Shallow heap is the size of an Object and its primitives.
 Retained heap (effective size of an object) is the shallow heap + the retained
heaps of lifetime dependent outgoing references.
 Retained Set is the set of objects that would be removed by garbage
collection when an object, or multiple objects, is garbage collected.

14. Understanding of Heap Dump
 Dominator Tree: Transform object graph to identify the biggest parts of
retained memory and the keep-alive dependencies among objects.

15. Finding Leaks
 Eclipse Memory Analyzer Tool
1. Open Heap Dump File:
2. Select “Leak Select Suspects”.
This will give suspected memory leaks
and it does a great job to find one.
3. List Objects with incoming or outgoing
references.
4. If the name is prefixed with “class”, it is
the static instance of that class or just
referencing to that class. So every
instance has a reference to its class.
Generally skip that.
5. If you see unreachable objects, that
means they are eligible for garbage
collection and by default MAT shows live
HEAP.

16. Finding Leaks
 Inspector: This shows a complete information of an live object.

17. Finding Leaks
 Thread Stacks and many other information available.

18. Practical Example

19. Summary
 Understand Heap Dump: A directed graph referencing from/to objects.
 Using local environment, obtain Heap Dump after reproducing the leak
problem.
 Using MAT, just click memory leak suspect or use dominator tree. This will tell
what the problem is most of times.