Fundamental knowledge on GCs, JVM Architecture, GC Algorithms, how to tune applications using GC parameters

1. Performance Tuning
Garbage Collection
Haribabu Nandyal
Performance Engineering Expert

2. Agenda - Garbage Collection
1. JVM Architecture
 Components of GC
2. Fundamentals of Garbage Collection
3. GC Algorithms
• Generational GC
 Young generation
 Old Generation
 Minor and Major GC
• Serial and Parallel GC
• STW and Concurrent GC
• CMS GC

3. • More time in GC means more application thread pauses.
• Higher the number of objects, higher is the memory foot print
and thereby more work for GC to reclaim memory.
• Large heap - more time for GC to trigger.
• Small heap - less time but frequent GCs.
Why GC Monitoring important?
• GC compute intensive - CPU overhead. More the time taken
by GC, slower will be your application.
• Throughput : Total time spent in not doing GC.
• Pause Time: The time for which the application threads
stopped while collecting.
• Promptness: time between objects death and its collection

4. HotSpot JVM: Architecture

5. 1) Classloader:
Classloader is a subsystem of JVM that is used to load class files.
2) Class(Method) Area:
It stores per-class structures such as the runtime constant pool, field and
method data, the code for methods.
3) Heap:
It is the runtime data area in which objects are allocated.
4) Stack:
Each thread has its own PC register and a Java stack. A new frame is created
each time a method is invoked and is destroyed when its method invocation
completes. The stack stores primitive local variables and object references along
with the call stack.
5) Program Counter Register:
It contains the address of the Java virtual machine instruction currently being
executed.
HotSpot JVM: Architecture

6. 6) Native Method Stack:
It contains all the native methods used in the application.
7) Execution Engine:
 A virtual processor
 Interpreter:
Reads bytecode stream and executes the instructions.
 Just-In-Time(JIT) compiler:
JIT compiles parts of the byte code that have similar functionality at the
same time, and hence reduces the amount of time needed to compile. Here the
term “compiler” refers to a translator from the instruction set of a JVM to the
instruction set of a specific CPU.
HotSpot JVM: Architecture

7. JVM Heap Structure

8. Before Marking
After Marking
Garbage Collector first performs a task called
Marking.
Each object which the GC meets is marked as
being used and will not be deleted in the
sweeping stage.
Fundamentals of Garbage
Collection

9. Fundamentals of Garbage
Collection

10. The Sweeping stage is where the deletion of the objects takes
place.
The traditional way is to let the allocator methods use complex data
structures to search the memory for the required space.
Fundamentals of Garbage Collection

11. Deletion with
compacting
Compact the memory by moving objects close to each other.
Object allocation is faster.
Fundamentals of Garbage Collection

12. Objects Lifetime

13. Generational Garbage
Collection

14. Generational Garbage Collection
HotSpot uses “Generational Collectors”
HotSpot Java heap is allocated into generational spaces.
Memory space is divided into three sections:
• Young Generation (for young objects)
 Eden
 A “from” survivor space
 A “to” survivor space
• Tenured (old) generation (for old objects)
• Permanent generation (meta data, classes and so on)

15. Features of Young Generational Space
• GCs occur relatively frequent.
• GCs are fast and efficient because young generation space is
usually small and likely to contain a lot of short lived objects.
• Objects that survive some number of young generation
collections are promoted to old generation heap space.
Features of Old Generational Space
• Typically larger than young generation heap space
• Its occupancy grows slowly
• GCs are infrequent but takes significantly longer time to
complete than young generational heap space.
 GCs in old generation space should be minimized.
Generational Garbage Collection

16. Generational Garbage Collection

17. New objects are allocated to the Eden space.
When Eden space is full, a minor GC is triggered
(Stop the world event)
Young GC process
Generational Garbage Collection

18. Unreferenced objects are garbage collected.
Referenced objects are copied to survivor space and have
their age incremented.
Young GC process
Generational Garbage Collection

19. After objects are moved to the survivor space, Eden space is
cleared.
The from survivor space is also cleared.
Young GC process
Generational Garbage Collection

20. Next Minor GC
Referenced objects from last GC become "from" Survivor
space.
Referenced objects are copied to the "to" survivor space.
Surviving objects ages are incremented.
Young GC process
Generational Garbage Collection

21. Young GC process
Generational Garbage Collection

22. Promoted to Old Space
When age threshold is reached , objects are eventually promoted to
tenured space.
Young GC process
Generational Garbage Collection

23. Process repeats at each minor GC
When objects reach an age threshold, they are copied to
old generation
Generational Garbage Collection

24. • Young Generation
 Eden
 Survivor Space
 Objects age here
• Minor garbage collections are always "Stop the
World" events
• Minor garbage collections can be
 Single-threaded
Serial GC
 Multithreaded (Parallel)
• Parallel GC
• Concurrent GC
• G1 GC
Young GC process - Summary

25. Improve performance of GC
For young generation (Minor GC)
threads
timegc
threads
Default GC Parallel GC
Young
Generation
Serial GC vs Parallel GC

26. Reduce pause time to collect Old Generation
For old generation (Full GC)
Enabled by -XX:+UseConcMarkSweepGC
threads
timegc
threads
STW GC Concurrent GC
Old
Generation
STW GC vs Concurrent GC

27. Serial Mark Sweep vs Concurrent
Mark Sweep (CMS)

28. • Eliminating dead object in “Eden” space.
• Moving live object from “Eden” to empty survival space (“To” space).
• Object that are too big, are copied directly to old space.
• Eliminating dead object in survival “From” space
• Mature objects are moved to old space
• Moving live object from used survival space (“From”) to empty survival
space (“To” space).
• Object that are too big, are copied directly to old space.
Serial Collector

29. Initial Mark
• Identifies root objects.
• Stop the world phase
Concurrent
Mark
• Marks live object that are reachable from the root object graph.
• Concurrent
Remark
• Revisits changed objects for liveliness check (Objects change
during the concurrent phases)
• Stop the world phase
Concurrent
Sweep
• All garbage objects are swept.
• Concurrent
Concurrent Mark-Sweep

30. • 32 bit Java processes heap size
 Varies according to the OS and platform
 determined by the process memory layout
 32bit architecture has an addressable range of: 2^32 is
4GB
• 64 bit processes do not have this limit
 Limit exists, but is so large it can be effectively ignored
 Addressability usually between 2^44 and 2^64 : 16+
TeraBytes
Maximum Possible Heap Size
A 32 bit Java process has a 4 GB memory which is shared
by the Java Heap, Native Heap and the Operating System.

31. GC will adapt heap size to keep occupancy between 40% and
70%
• Heap occupancy over 70% causes frequent GC cycles
 Which, In general means reduced performance
• Heap occupancy below 40% means infrequent GC cycles, but
cycles longer than they need to be
 Which means longer pause times than necessary
 Which generally means reduced performance (high latency)
• The maximum heap size setting should therefore be 43% larger
than the maximum occupancy of the application
 Maximum occupancy + 43% means occupancy at 70% of total
heap
Eg. For 70MB occupancy, 100MB heap size required (70MB + 43%
of 70MB)
“The Right” Java heap size

32. Fixed heap vs. Variable heap
Minimum heap size (-Xms) = Maximum heap size (-Xmx)?
• Variable Heap Sizes
 GC will adapt heap size to keep occupancy between 40% and
70% which expands and shrinks the Java heap
 Allows for scenarios where usage varies over time, where
variations would take usage outside of the 40-70% window
• Fixed Heap Sizes
 Does not expand or shrink the Java heap

33. Requirement Problem Solution
Fast and responsive Requires low pause GC CMS Collector
Needs to support a large
number of concurrent
users
• Generates a lot of
short lived objects
• Hardware support
Multi-threaded Hardware
Large Eden memory space
(1 GB)
Generates a large
number of proxy
classes
Requires Permanent
generation tuning
Large permanent generation
space (384 MB)
Holds large data
collections in memory
Requires a large old
memory space
Large Old memory space (4
GB)
Heap size is 5 GB 32 Bit JVM is not enough 64 Bit JVM and Hardware
GC Related Problems & Solutions

34. Process Flow to Tune JVM