“Just leave the garbage outside and we will take care of it for you”. This is the panacea promised by garbage collection mechanisms built into most software stacks available today. So, we don’t need to think about it anymore, right? Wrong! When misused, garbage collectors can fail miserably. When this happens they slow down your application and lead to unacceptable pauses. In this talk we will go over different garbage collectors approaches and understand under which conditions they function well.

1. Show Me the Garbage
Haim Yadid @ Next Insurance

6. @lifeyx

7. NEXT
7
Today

8. NEXT
8
Talk Focus
Mainstream applications HPC/ HFT/ Huge heaps
Web apps latency Low latency
Monitor and Tune no allocation

9. NEXT
9
Agenda
● Memory Management Terminology
● GC Building Blocks
● Current OpenJDK GCs
● Monitoring GC and Memory
● Analysis of common problems

10. NEXT
Let’s talk
memory
10

11. NEXT
Heap
Thread
Stacks
Static
11
Memory (Data)

12. NEXT
Allocator - Memory Allocation mechanism
Collector - memory reclamation mechanism
12
Participants

13. Mutator

14. NEXT
Allocator - Memory Allocation mechanism
Mutators - the application threads
Collectors - the GC threads
14
Participants

15. NEXT
● new / malloc () to create object
● delete() / free() release an object
15
malloc / free

16. NEXT
● Dangling pointers / Double free
● Memory leaks
● Crashes
● Security Vulnerabilities
● Fragmentation
● Performance Degradation
16
I don’t miss it

17. NEXT
17
Automatic Reference Counting (ARC)

18. NEXT
1 1
Heap
2
1
1
1 0
Non Heap
18
Automatic Reference Counting (ARC)

19. NEXT
1 1
Heap
2
1
1
1 1
19
Automatic Reference Counting - Cycles

20. NEXT
Concurrency - Every change to reference requires a memory barrier
Very bad for highly concurrent systems Mostly relevant for single threaded
20
Reference Counting Concurrency

21. NEXT
The (Tracing) Garbage Collector
21

22. NEXT
22

23. NEXT
The 3 KPIs
(key performance indicators)
23

24. NEXT
CPU/Time Overhead
The amount of time the GC consumes / requires
○ Wall clock time
○ CPU time
24

25. NEXT
Memory Overhead
The amount of memory the GC requires
○ Internal Data structures
○ Headroom
25

26. NEXT
Latency (Pauses)
What is the probability that the GC will affect your
transaction time and for how long
26

27. NEXT
Terminology
(Heap)
27

28. NEXT
Heap
Static &
Stack
28
To Heap or not to Heap

29. NEXT
Heap
29
Root Set

30. NEXT
Heap
Stack
30
Live Set (Reachable objects)

31. NEXT
Heap
Stack
31
Garbage (Unreachable Objects)

32. NEXT
Heap
Stack
32
Garbage (Unreachable Objects)

33. NEXT
Heap
Stack
33
Garbage (Unreachable Objects)

34. NEXT
Heap
Stack
34
Free List

35. NEXT
Building Blocks
&
Algorithms
35

36. NEXT
Heap
36
Root Scanning & Safe Points
Work: O(#threads & stack frames)

37. NEXT
Heap
Work: O(#Objects in LiveSet)
Stack
37
Marking (a.k.a. tracing)

38. NEXT
Sweep through the heap and collect all unmarked objects
Maintain a list of free memory portions
Work: O(#Objects and
fragments in Heap)
Heap
Unreachable
Reachable
Free
38
Mark & Sweep : Sweep

39. NEXT
39
Sweep: Fragmentation

40. NEXT
Moving objects to make bigger free contiguous spaces
Heap Heap
Work:
O( Size of Liveset in Heap)
40
Mark Compact

41. NEXT
41
Moving object around
● fix incoming references
● Stop application ?
● Read and write memory barriers
○ Brook pointer (shenandoah)
○ colored pointers (zgc, C4)

42. NEXT
Move all live objects to another space
From Space To Space
Work: O(Size of live set)
Memory: x2
42
Copy collector

43. NEXT
43
Thread Local Bump allocation

44. NEXT
Weak Generational Hypothesis - Most object die young
The surviving ones tend to live for a long time
44
Generational collectors

45. NEXT
Old
Non
Heap
From To
45
New/ Old Generation
there are also survivor spaces (I know)

46. NEXT
Higher throughput
Shorter new generation collection
46
Generational collectors

47. NEXT
Heap
Non
Heap
47
Regional

48. NEXT
Single Threaded
Stop the World Concurrent
Parallel
VS
VS
Monolithic Incremental
VS
48
Collectors Attributes

49. NEXT
49
STW Collectors - Serial (single threaded) - Monolithic
(Non incremental)
stop the world
Application
Threads
GC
Thread

50. NEXT
50
STW Collectors - Incremental

51. NEXT
51
STW Collectors - Parallel
Application
Threads
GC
Threads

52. NEXT
52
Concurrent Collectors - (single threaded)

53. NEXT
53
Concurrent Collectors -Parallel

54. NEXT
Let’s Talk about Mostly
54

55. NEXT
When do we want to collect Garbage?
55

56. NEXT
56

57. NEXT
57
The Open JDK GCs - Serial
Property Serial
-XX:+UseSerialGC
Compacting
Regional
Generational
Concurrent
Mostly Concurrent
When To Use Small Heaps
Single cpu
containers
(*) can be found as part of Corretto JDK

58. NEXT
58
The Open JDK GCs - Parallel
Property Serial
-XX:+UseSerialGC
Parallel
-XX:+UseParallelGC
Compacting
Regional
Generational
Concurrent
Mostly Concurrent
When To Use Small Heaps
Single cpu
containers
High throughput
Latency of a
single response is
not important
(*) can be found as part of Corretto JDK

59. NEXT
59
The Open JDK GCs - G1GC
Property Serial
-XX:+UseSerialGC
Parallel
-XX:+UseParallelGC
G1
-XX:+UseG1GC
Compacting
Regional
Generational
Concurrent <200ms
Mostly Concurrent Old gen only
When To Use Small Heaps
Single cpu
containers
High throughput
Latency of a
single response is
not important
Mainstream
applications
(*) can be found as part of Corretto JDK

60. NEXT
60
The Open JDK GCs - ZGC
Property Serial
-XX:+UseSerialGC
Parallel
-XX:+UseParallelGC
G1
-XX:+UseG1GC
ZGC
-XX:+UseG1GC
Compacting
Regional
Generational WIP
Concurrent <200ms <1ms
Mostly Concurrent Old gen only
When To Use Small Heaps
Single cpu
containers
High throughput
Latency of a
single response is
not important
Mainstream
applications
Large Heaps
Highly
responsive
(*) can be found as part of Corretto JDK

61. NEXT
61
The Open JDK GCs - Shenandoah
Property Serial
-XX:+UseSerialGC
Parallel
-XX:+UseParallelGC
G1
-XX:+UseG1GC
ZGC
-XX:+UseG1GC
Shenandoah(*)
-XX:+UseShenandoahGC
Compacting
Regional
Generational WIP WIP
Concurrent <200ms <1ms <1ms
Mostly Concurrent Old gen only
When To Use Small Heaps
Single cpu
containers
High throughput
Latency of a
single response is
not important
Mainstream
applications
Large Heaps
Highly
responsive
Large Heaps
Highly responsive
(*) can be found as part of Corretto JDK

62. NEXT
62
Azul’s C4
Property C4
Compacting
Regional
Generational
Concurrent
Mostly
Concurrent
When To Use Large
Heaps
low latency

63. NEXT
63
Let’s go practical
Dall-e: practical engineer picasso style

64. NEXT
Monitoring
64

65. NEXT
● CPU/Time overhead
● Memory Overhead
● Latency and Pauses
65
So what should we monitor ?

66. NEXT
-Xlog:gc=info:
file=<file_name>:
time,uptime,level,tags:
filecount=5,filesize=20m
66
GC Logs
Output file
decorators
Output Options
Tag

67. NEXT
67
GC Logs
[2022-09-01T17:30:40.433+0000][0.004s][info][gc] Using G1
[2022-09-01T17:30:41.975+0000][1.546s][info][gc] GC(0) Pause Young (Concurrent Start) (Metadata GC
Threshold) 126M->30M(4096M) 12.598ms
[2022-09-01T17:30:41.975+0000][1.546s][info][gc] GC(1) Concurrent Mark Cycle
[2022-09-01T17:30:41.988+0000][1.558s][info][gc] GC(1) Pause Remark 32M->22M(4096M) 2.119ms
[2022-09-01T17:30:41.988+0000][1.558s][info][gc] GC(1) Pause Cleanup 22M->22M(4096M) 0.003ms
[2022-09-01T17:30:42.052+0000][1.622s][info][gc] GC(1) Concurrent Mark Cycle 76.584ms
[2022-09-01T17:30:42.824+0000][2.395s][info][gc] GC(2) Pause Young (Concurrent Start) (Metadata GC

68. NEXT
68
GC Logs - Description
[2022-09-01T17:32:31.262+0000][110.832s][info][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause)
486M->130M(4096M) 93.914ms

69. NEXT
69
GC Logs - Time Stamp (when)
[2022-09-01T17:32:31.262+0000][110.832s]
[info][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause)
486M->130M(4096M) 93.914ms

70. NEXT
70
GC Logs - Memory released (how much)
[2022-09-01T17:32:31.262+0000][110.832s]
[info][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause)
486M->130M(4096M) 93.914ms

71. NEXT
71
GC Logs - Memory released (how long )
[2022-09-01T17:32:31.262+0000][110.832s]
[info][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause)
486M->130M(4096M) 93.914ms

72. NEXT
72
GC Logs
[2022-09-01T17:32:31.262+0000][110.832s][info][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause) 486M->130M(4096M) 93.914ms
[2022-09-01T17:33:48.974+0000][188.545s][info][gc] GC(16) Pause Young (Normal) (G1 Evacuation Pause) 1240M->169M(4096M) 26.671ms
.
.
.
Start and End Time
Freed memory
Overhead
Frequency
Allocation Rate
Pause Time

73. NEXT
73
GC Logs - GC Viewer

74. NEXT
74
GC Logs - GC Viewer

75. NEXT
75
GC Logs - GC Viewer

76. NEXT
76
GCEasy.io

77. NEXT
77
GCEasy.io

78. NEXT
● MBeans
● Mission Control
78
JMX (Java.lang.GarbageCollector)

79. NEXT
79
Visual GC
* does not support ZGC and Shenandoah

80. NEXT
● Wake Up every x ms
● measure amount time passed
● https://github.com/giltene/jHiccup
80
Latency - Hiccup Probe (inspired by jHiccup)
T T+x + p
{
Sleep X ms

81. NEXT
● GC logs and - System.gc()
● Histogram of live set : jmap -histo:live <PID>
● Grab a Heap dump (or at least a heap histogram)
81
Estimate Live Set

82. NEXT
● Long lasting memory ( maybe slowly changing)
● Service with large state will benefit from generational GC
● If State is bigger than old gen …. very bad
82
Live set - Long lasting - state

83. NEXT
● The (maximal/average) amount of memory needed to serve
a request
83
Live set - Working memory

84. NEXT
Common
Problems
84

85. NEXT
85
https://github.com/lifey/gcKryptonites
Pause Detector
every 10ms detects (10& 100ms)
State Holder
Holds 2GB of smalls object (100m)
Brutal Allocator
Allocates 500 MB/s
2X

86. NEXT
86
Some Results

87. NEXT
87
Some Results
G1
Shenandoah
ZGC

88. NEXT
88
Some Results(2 min execution 4GB heap)
G1 Shenandoah zGC Parallel
# 10ms pause 27 3 10 107
# 100 ms pause 0 1 4 9
Average runtime
of iteration
302ms 794ms 352ms 434ms
LiveSet 2GB 2GB 2.5GB 2GB

89. NEXT
● Is it over > 5%
● Improve by :
○ Increase heap size
○ Use Generational Garbage collector
89
Excess GC (Temporal) Overhead

90. NEXT
● What can you tolerate ?
● Move to a concurrent
● Upgrade to the newest JVM
● Use Zing JVM
90
Latency

91. NEXT
● Use Generational Garbage collector
● Trade throughput
91
Excess GC (Memory) Overhead

92. NEXT
● Compacting GC should not have fragmentation
● What about humongous objects ?
● G1 handles it poorly
● ZGC Does better
92
Fragmentation

93. NEXT
● long term state should fit in old gen
● short term memory should not spill to old gen
93
Generational GC anomalies
New Generation Old Generation
Application State

94. NEXT
94
Your Service Questionnaire:
Question Value
Heap size the process is using (-Xmx)
GC overhead over an hour of peak time
Max Pause Time over a 24 hour period
Number of collections above GC pause time targets
Live set size (MB)
Allocation Rate

95. NEXT
95
Additional Reading