Gclogs jdd

  • 604 views
Uploaded on

My GC log presentation in Krakow.

My GC log presentation in Krakow.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
604
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
23
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • turn them on in production\n
  • \n
  • \n
  • \n
  • + many others\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Transcript

  • 1. Do Your GC Logs Speak To YouUsing Measurements to Tune Java Memory Management Copyright 2011 Kodewerk Ltd. All rights reserved
  • 2. Kodewerk tmJava Performance Services About Me • Independent consultant (www.kodewerk.com) • performance tuning, benchmarking, training course • Co-author www.javaperformancetuning.com • Speak frequently about performance • Member of Java Champion program • Other stuff... (google is you care to) Copyright 2011 Kodewerk Ltd. All rights reserved
  • 3. Kodewerk tmJava Performance Services About Me • Independent consultant (www.kodewerk.com) • performance tuning, benchmarking, training course • Co-author www.javaperformancetuning.com • Speak frequently about performance • Member of Java Champion program • Other stuff... (google is you care to) Copyright 2011 Kodewerk Ltd. All rights reserved
  • 4. Kodewerk tmJava Performance Services Disclaimer The resemblance of any opinion, recommendation or comment made during this presentation to performance tuning advice is merely coincidental. Copyright 2011 Kodewerk Ltd. All rights reserved
  • 5. Why should I monitor GC? Copyright 2011 Kodewerk Ltd. All rights reserved
  • 6. hy G C d at Kodewerk be o fo tha we g tm in is re t Java Performance Services oc ha no nn g th cu ve in e tting a heap ru I am ge en rs th th ju is an is or no apparent d OOM f of st lo ha 0% PU d Hi! can W hat . reasonc d zen. C g the o uld ay in stra size be t h he why GC’s e en w f re d u r in e rea h th feT gc som son f gc.l it h g s ta pp 10 nge r tu p or co que ntinu heap CM t f ro exe m w ous f r a . Wi t og. eon ull , I Wh ntly S G cut y afte oble fr do It s e h mca t im e ance wan ee es C e es? se ze ou and elp co up eem r a and a pr t to es to nd xec fo or m pp e of s? r 1 lic to why 0s at s v Hello ute io rows kno of I’ve perf inut n seelaps er y s so Copyright 2011 Kodewerk Ltd. All rights reserved fu l w
  • 7. Kodewerk tmJava Performance Services ? ? ? ?? ?? ? ? ? ? ?? ? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ? ?? ? Copyright 2011 Kodewerk Ltd. All rights reserved
  • 8. GC logs contain keyperformance indicatorsCopyright 2011 Kodewerk Ltd. All rights reserved
  • 9. Kodewerk tm Java Memory PoolsJava Performance Services Eden S0 S1 Tenured Perm Young Old Gen Gen Normal Flow - objects created in Eden - copied into the active survivor space - copied between survivor spaces - copied to Old Gen when ages reaches tenuring threshold Copyright 2011 Kodewerk Ltd. All rights reserved
  • 10. Kodewerk tm Java Performance Services Collector Choicesuint i = 0;if (UseSerialGC) i++;if (UseConcMarkSweepGC || UseParNewGC) i++;if (UseParallelGC || UseParallelOldGC) i++;if (UseG1GC) i++;if (i > 1) { jio_fprintf(defaultStream::error_stream(), "Conflicting collector combinations in option list; " "please refer to the release notes for the combinations " "allowedn"); status = false;} Copyright 2011 Kodewerk Ltd. All rights reserved
  • 11. Kodewerk tm Java Memory PoolsJava Performance Services Eden S0 S1 Tenured Perm Young Old Gen Gen Abnormal Situations - memory leaks (loitering objects) - long pause times (heap too big, poor parallelization) - too frequent full GC (zombies, poor throughput) - survivor space flooding (premature promotion) - fragmented tenured space (compaction) - high rates of object creation - etc..... Copyright 2011 Kodewerk Ltd. All rights reserved
  • 12. Kodewerk tm Garbage Collection LogsJava Performance Services -verbose:gc -Xloggc:gc.log 7.0 has new flags for rollover Some many useful flags for monitoring GC -XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintHeapAtGC -XX:+PrintApplicationStoppedTime Copyright 2011 Kodewerk Ltd. All rights reserved
  • 13. Kodewerk tm GC Log Fragment Java Performance Services21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K->89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,real=0.32 secs]163.028: [GC 163.028: [DefNewDesired survivor size 115671040 bytes, new threshold 15 (max 15)- age 1: 736 bytes, 736 total- age 2: 112 bytes, 848 total- age 4: 40 bytes, 888 total- age 6: 232 bytes, 1120 total- age 7: 32 bytes, 1152 total- age 11: 96 bytes, 1248 total: 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times:user=0.00 sys=0.00, real=0.00 secs]Heap def new generation total 451968K, used 193472K [0x10020000, 0x39620000, 0x39620000) eden space 226048K, 85% used [0x10020000, 0x1bd0fe48, 0x1dce0000) from space 225920K, 0% used [0x1dce0000, 0x1dce0560, 0x2b980000) to space 225920K, 0% used [0x2b980000, 0x2b980000, 0x39620000) tenured generation total 677888K, used 443015K [0x39620000, 0x62c20000, 0x62c20000) the space 677888K, 65% used [0x39620000, 0x546c1d78, 0x546c1e00, 0x62c20000) compacting perm gen total 36608K, used 36480K [0x62c20000, 0x64fe0000, 0x66c20000) the space 36608K, 99% used [0x62c20000, 0x64fc0290, 0x64fc0400, 0x64fe0000)No shared spaces configured. Copyright 2011 Kodewerk Ltd. All rights reserved
  • 14. Kodewerk tm Java Performance Services Full GC21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K->89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,real=0.32 secs] tenured space statistics perm space statistics- other stats are for the entire heap Copyright 2011 Kodewerk Ltd. All rights reserved
  • 15. Kodewerk tm Java Performance Services Full GC21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K->89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,real=0.32 secs]- occupancy before->after collection (memory pool configured size)- reporting on tenured, perm memory pools plus total for all Java heap spaces Copyright 2011 Kodewerk Ltd. All rights reserved
  • 16. Kodewerk tm Java Performance Services Full GC21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K->89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,real=0.32 secs]- pause time statistics for tenured and all Java heap Copyright 2011 Kodewerk Ltd. All rights reserved
  • 17. Kodewerk tm GC Log Fragment Java Performance Services163.028: [GC 163.028: [DefNewDesired survivor size 115671040 bytes, new threshold 15 (max 15)- age 1: 736 bytes, 736 total- age 2: 112 bytes, 848 total- age 4: 40 bytes, 888 total- age 6: 232 bytes, 1120 total- age 7: 32 bytes, 1152 total- age 11: 96 bytes, 1248 total: 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times:user=0.00 sys=0.00, real=0.00 secs] young gen statistics statistics for all Java heap Copyright 2011 Kodewerk Ltd. All rights reserved
  • 18. Kodewerk tm GC Log Fragment Java Performance Services163.028: [GC 163.028: [DefNewDesired survivor size 115671040 bytes, new threshold 15 (max 15)- age 1: 736 bytes, 736 total- age 2: 112 bytes, 848 total- age 4: 40 bytes, 888 total- age 6: 232 bytes, 1120 total- age 7: 32 bytes, 1152 total- age 11: 96 bytes, 1248 total: 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times:user=0.00 sys=0.00, real=0.00 secs] surivor space sizing and tenuring thresholds breakdown of bytes @ age Copyright 2011 Kodewerk Ltd. All rights reserved
  • 19. Kodewerk tm GC Log Fragment Java Performance ServicesHeap def new generation total 451968K, used 193472K [0x10020000, 0x39620000, 0x39620000) eden space 226048K, 85% used [0x10020000, 0x1bd0fe48, 0x1dce0000) from space 225920K, 0% used [0x1dce0000, 0x1dce0560, 0x2b980000) to space 225920K, 0% used [0x2b980000, 0x2b980000, 0x39620000) tenured generation total 677888K, used 443015K [0x39620000, 0x62c20000, 0x62c20000) the space 677888K, 65% used [0x39620000, 0x546c1d78, 0x546c1e00, 0x62c20000) compacting perm gen total 36608K, used 36480K [0x62c20000, 0x64fe0000, 0x66c20000) the space 36608K, 99% used [0x62c20000, 0x64fc0290, 0x64fc0400, 0x64fe0000)No shared spaces configured.- breakdown of occupancy and configured size of each memory pool- pointer values describing each space (for debugging) Copyright 2011 Kodewerk Ltd. All rights reserved
  • 20. Kodewerk tm Java Performance Services Puzzler #1 600Heap Usage after GC (MB) 500 400 300 200 100 1000 2000 3000 4000 Time (seconds) Copyright 2011 Kodewerk Ltd. All rights reserved
  • 21. Kodewerk tm Java Performance Services Puzzler #1 600Heap Usage after GC (MB) 500 400 300 what happened here? 200 100 1000 2000 3000 4000 Time (seconds) Copyright 2011 Kodewerk Ltd. All rights reserved
  • 22. Kodewerk tm Java Performance Services Puzzler #2147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ... ????Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved
  • 23. Kodewerk tm Java Performance Services Puzzler #2147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ...Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved
  • 24. Kodewerk tm Java Performance Services Puzzler #2147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ...Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved
  • 25. Kodewerk tm Java Performance Services Puzzler #2147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ...Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs]147.823 + 0.8074580 = 148.630 ~= 148.632 Copyright 2011 Kodewerk Ltd. All rights reserved
  • 26. Kodewerk tm Java Performance Services Puzzler #2147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ...Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved
  • 27. Kodewerk tm Java Performance Services Puzzler #2147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ... ????Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved
  • 28. Kodewerk tmJava Performance Services Case Study one web app four different memory configurations Copyright 2011 Kodewerk Ltd. All rights reserved
  • 29. Kodewerk tmJava Performance Services Case Study One Web app Four different memory configurations heap size new ratio survivor ratio Copyright 2011 Kodewerk Ltd. All rights reserved
  • 30. Kodewerk tmJava Performance Services 1G, NR=8, SR=8 Ave: 149 Medien: 84 90%: 345 Max: 1728 Error: 0% Copyright 2011 Kodewerk Ltd. All rights reserved
  • 31. Kodewerk tmJava Performance Services 2G, NR=8, SR=8 Ave: 129 Medien: 80 90%: 287 Max: 4920 Error: 0% Copyright 2011 Kodewerk Ltd. All rights reserved
  • 32. Kodewerk tmJava Performance Services 1G, NR=1, SR=1 Ave: 82 Medien: 60 90%: 171 Max: 650 Error: 0% Copyright 2011 Kodewerk Ltd. All rights reserved
  • 33. Kodewerk tmJava Performance Services 1G, NR=8, SR=8 Copyright 2011 Kodewerk Ltd. All rights reserved
  • 34. Kodewerk tmJava Performance Services 1G, NR=1, SR=1 Copyright 2011 Kodewerk Ltd. All rights reserved
  • 35. Kodewerk tmJava Performance Services 2G, NR=1, SR=1 Ave: 326 Medien: 156 90%: 831 Max: 4976 Error: 0% Copyright 2011 Kodewerk Ltd. All rights reserved
  • 36. QuestionsCopyright 2011 Kodewerk Ltd. All rights reserved