Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Gclogs
1. Do Your GC Logs Speak
To You
Using Measurements to Tune Java Memory
Management
Copyright 2011 Kodewerk Ltd. All rights reserved
2. Kodewerk
About Me
tm
Java Performance Services
• Independent consultant (www.kodewerk.com)
• performance tuning, benchmarking, training
course
• Co-author www.javaperformancetuning.com
• Speak frequently about performance
• Member of Java Champion program
• Other stuff... (google is you care to)
Copyright 2011 Kodewerk Ltd. All rights reserved
3. Kodewerk
About Me
tm
Java Performance Services
• Independent consultant (www.kodewerk.com)
• performance tuning, benchmarking, training
course
• Co-author www.javaperformancetuning.com
• Speak frequently about performance
• Member of Java Champion program
• Other stuff... (google is you care to)
Copyright 2011 Kodewerk Ltd. All rights reserved
4. Kodewerk
Disclaimer
tm
Java Performance Services
The resemblance of any
opinion, recommendation or
comment made during this
presentation to performance
tuning advice is merely
coincidental.
Copyright 2011 Kodewerk Ltd. All rights reserved
5. Why should I monitor GC?
Copyright 2011 Kodewerk Ltd. All rights reserved
6. hy
G
C d
at
be o
Kodewerk fo tha we
g
in
is
tm
Java Performance Services
re t
oc ha
no nn
g
th cu ve
in
e
tting a heap
ru
I am ge en rs th
th
ju is
an is
or no apparent d
OOM f of st lo
ha 0% PU
d
Hi!
can
W hat .
reasonc d
zen. C
g
the
o uld ay
in
stra
size
be t h he
why
GC’s
e
en
w
f re
d u r in e rea
h th
feT
gc
som
son f
gc.l
it h
g s ta
pp
10
nge
r tu p or co
que
ntinu
heap
CM
t f ro
exe
m w
ous f
r a
. Wi t
og.
eon
ull
, I
Wh
ntly
S G
cut
y
afte
oble
fr do
It s
e h
mca
t im e
ance
wan
ee es
C e
es?
se ze ou
and
elp
co up
eem
r a
and
a pr
t to
es to
nd
xec
fo
or m
pp
e of
s? r 1 lic
to
why
0s at
s v
Hello
ute
io
rows
kno
of
I’ve
perf
inut
n
see
laps
er y
s so
Copyright 2011 Kodewerk Ltd. All rights reserved fu l
w
8. GC logs contain key
performance indicators
Copyright 2011 Kodewerk Ltd. All rights reserved
9. Kodewerk
tm
Garbage Collection Logs
Java Performance Services
-verbose:gc
-Xloggc:gc.log
7.0 has new flags for rollover
Some other useful flags
-XX:+PrintGCDetails
-XX:+PrintTenuringDistribution
-XX:+PrintHeapAtGC
-XX:+PrintApplicationStoppedTime
Copyright 2011 Kodewerk Ltd. All rights reserved
10. Kodewerk
tm
Collector Choices
Java Performance Services
uint i = 0;
if (UseSerialGC) i++;
if (UseConcMarkSweepGC || UseParNewGC) i++;
if (UseParallelGC || UseParallelOldGC) i++;
if (UseG1GC) i++;
if (i > 1) {
jio_fprintf(defaultStream::error_stream(),
"Conflicting collector combinations in option list; "
"please refer to the release notes for the combinations "
"allowedn");
status = false;
}
Copyright 2011 Kodewerk Ltd. All rights reserved
11. Kodewerk
tm
Java Memory Pools
Java Performance Services
Eden S0 S1 Tenured Perm
Young Old Gen
Gen
Normal Flow
- objects created in Eden
- copied into the active survivor space
- copied between survivor spaces
- copied to Old Gen when ages reaches
tenuring threshold
Copyright 2011 Kodewerk Ltd. All rights reserved
12. Kodewerk
tm
Java Memory Pools
Java Performance Services
Eden S0 S1 Tenured Perm
Young Old Gen
Gen
Abnormal Situations
- memory leaks (loitering objects)
- long pause times (heap too big, poor
parallelization)
- too frequent full GC (zombies, poor throughput)
- survivor space flooding (premature promotion)
- fragmented tenured space (compaction)
- high rates of object creation
- etc.....
Copyright 2011 Kodewerk Ltd. All rights reserved
13. Kodewerk tm
Java Performance Services Puzzler #1
600
Heap Usage after GC (MB)
500
400
300
200
100
1000 2000 3000 4000
Time (seconds)
Copyright 2011 Kodewerk Ltd. All rights reserved
14. Kodewerk tm
Java Performance Services Puzzler #1
600
Heap Usage after GC (MB)
500
400
300
what happened here?
200
100
1000 2000 3000 4000
Time (seconds)
Copyright 2011 Kodewerk Ltd. All rights reserved
15. Kodewerk
tm
GC Log Fragment
Java Performance Services
21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K-
>89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,
real=0.32 secs]
163.028: [GC 163.028: [DefNew
Desired survivor size 115671040 bytes, new threshold 15 (max 15)
- age 1: 736 bytes, 736 total
- age 2: 112 bytes, 848 total
- age 4: 40 bytes, 888 total
- age 6: 232 bytes, 1120 total
- age 7: 32 bytes, 1152 total
- age 11: 96 bytes, 1248 total
: 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times:
user=0.00 sys=0.00, real=0.00 secs]
Heap
def new generation total 451968K, used 193472K [0x10020000, 0x39620000, 0x39620000)
eden space 226048K, 85% used [0x10020000, 0x1bd0fe48, 0x1dce0000)
from space 225920K, 0% used [0x1dce0000, 0x1dce0560, 0x2b980000)
to space 225920K, 0% used [0x2b980000, 0x2b980000, 0x39620000)
tenured generation total 677888K, used 443015K [0x39620000, 0x62c20000, 0x62c20000)
the space 677888K, 65% used [0x39620000, 0x546c1d78, 0x546c1e00, 0x62c20000)
compacting perm gen total 36608K, used 36480K [0x62c20000, 0x64fe0000, 0x66c20000)
the space 36608K, 99% used [0x62c20000, 0x64fc0290, 0x64fc0400, 0x64fe0000)
No shared spaces configured.
Copyright 2011 Kodewerk Ltd. All rights reserved
16. Kodewerk
tm
Java Performance Services Full GC
21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K-
>89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,
real=0.32 secs]
Full GC requested @, started @
Copyright 2011 Kodewerk Ltd. All rights reserved
17. Kodewerk
tm
Java Performance Services Full GC
21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K-
>89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,
real=0.32 secs]
tenured space statistics
perm space statistics
- other stats are for the entire heap
Copyright 2011 Kodewerk Ltd. All rights reserved
18. Kodewerk
tm
Java Performance Services Full GC
21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K-
>89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,
real=0.32 secs]
- occupancy before->after collection (memory pool configured size)
- reporting on tenured, perm memory pools plus total for all Java
heap spaces
Copyright 2011 Kodewerk Ltd. All rights reserved
19. Kodewerk
tm
Java Performance Services Full GC
21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K-
>89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,
real=0.32 secs]
- pause time statistics for tenured and all Java heap
Copyright 2011 Kodewerk Ltd. All rights reserved
20. Kodewerk
tm
GC Log Fragment
Java Performance Services
163.028: [GC 163.028: [DefNew
Desired survivor size 115671040 bytes, new threshold 15 (max 15)
- age 1: 736 bytes, 736 total
- age 2: 112 bytes, 848 total
- age 4: 40 bytes, 888 total
- age 6: 232 bytes, 1120 total
- age 7: 32 bytes, 1152 total
- age 11: 96 bytes, 1248 total
: 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times:
user=0.00 sys=0.00, real=0.00 secs]
young gen statistics
statistics for all Java heap
Copyright 2011 Kodewerk Ltd. All rights reserved
21. Kodewerk
tm
GC Log Fragment
Java Performance Services
163.028: [GC 163.028: [DefNew
Desired survivor size 115671040 bytes, new threshold 15 (max 15)
- age 1: 736 bytes, 736 total
- age 2: 112 bytes, 848 total
- age 4: 40 bytes, 888 total
- age 6: 232 bytes, 1120 total
- age 7: 32 bytes, 1152 total
- age 11: 96 bytes, 1248 total
: 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times:
user=0.00 sys=0.00, real=0.00 secs]
surivor space sizing and tenuring thresholds
breakdown of bytes @ age
Copyright 2011 Kodewerk Ltd. All rights reserved
22. Kodewerk
tm
GC Log Fragment
Java Performance Services
Heap
def new generation total 451968K, used 193472K [0x10020000, 0x39620000, 0x39620000)
eden space 226048K, 85% used [0x10020000, 0x1bd0fe48, 0x1dce0000)
from space 225920K, 0% used [0x1dce0000, 0x1dce0560, 0x2b980000)
to space 225920K, 0% used [0x2b980000, 0x2b980000, 0x39620000)
tenured generation total 677888K, used 443015K [0x39620000, 0x62c20000, 0x62c20000)
the space 677888K, 65% used [0x39620000, 0x546c1d78, 0x546c1e00, 0x62c20000)
compacting perm gen total 36608K, used 36480K [0x62c20000, 0x64fe0000, 0x66c20000)
the space 36608K, 99% used [0x62c20000, 0x64fc0290, 0x64fc0400, 0x64fe0000)
No shared spaces configured.
- breakdown of occupancy and configured size of each memory pool
- pointer values describing each space (for debugging)
Copyright 2011 Kodewerk Ltd. All rights reserved