Your SlideShare is downloading. ×
  • Like
Gclogs
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Gclogs

  • 476 views
Published

GC log talk presented at JavaONE 2011

GC log talk presented at JavaONE 2011

Published in Technology , News & Politics
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
476
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
49
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • turn them on in production\n
  • + many others\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Transcript

  • 1. Do Your GC Logs Speak To YouUsing Measurements to Tune Java Memory Management Copyright 2011 Kodewerk Ltd. All rights reserved
  • 2. Kodewerk About Me tmJava Performance Services • Independent consultant (www.kodewerk.com) • performance tuning, benchmarking, training course • Co-author www.javaperformancetuning.com • Speak frequently about performance • Member of Java Champion program • Other stuff... (google is you care to) Copyright 2011 Kodewerk Ltd. All rights reserved
  • 3. Kodewerk About Me tmJava Performance Services • Independent consultant (www.kodewerk.com) • performance tuning, benchmarking, training course • Co-author www.javaperformancetuning.com • Speak frequently about performance • Member of Java Champion program • Other stuff... (google is you care to) Copyright 2011 Kodewerk Ltd. All rights reserved
  • 4. Kodewerk Disclaimer tmJava Performance Services The resemblance of any opinion, recommendation or comment made during this presentation to performance tuning advice is merely coincidental. Copyright 2011 Kodewerk Ltd. All rights reserved
  • 5. Why should I monitor GC? Copyright 2011 Kodewerk Ltd. All rights reserved
  • 6. hy G C d at be o Kodewerk fo tha we g in is tm Java Performance Services re t oc ha no nn g th cu ve in e tting a heap ru I am ge en rs th th ju is an is or no apparent d OOM f of st lo ha 0% PU d Hi! can W hat . reasonc d zen. C g the o uld ay in stra size be t h he why GC’s e en w f re d u r in e rea h th feT gc som son f gc.l it h g s ta pp 10 nge r tu p or co que ntinu heap CM t f ro exe m w ous f r a . Wi t og. eon ull , I Wh ntly S G cut y afte oble fr do It s e h mca t im e ance wan ee es C e es? se ze ou and elp co up eem r a and a pr t to es to nd xec fo or m pp e of s? r 1 lic to why 0s at s v Hello ute io rows kno of I’ve perf inut n seelaps er y s so Copyright 2011 Kodewerk Ltd. All rights reserved fu l w
  • 7. Kodewerk ? ? ? tmJava Performance Services ?? ?? ? ? ? ? ?? ? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ? ?? ? Copyright 2011 Kodewerk Ltd. All rights reserved
  • 8. GC logs contain keyperformance indicatorsCopyright 2011 Kodewerk Ltd. All rights reserved
  • 9. Kodewerk tm Garbage Collection LogsJava Performance Services -verbose:gc -Xloggc:gc.log 7.0 has new flags for rollover Some other useful flags -XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintHeapAtGC -XX:+PrintApplicationStoppedTime Copyright 2011 Kodewerk Ltd. All rights reserved
  • 10. Kodewerk tm Collector Choices Java Performance Servicesuint i = 0;if (UseSerialGC) i++;if (UseConcMarkSweepGC || UseParNewGC) i++;if (UseParallelGC || UseParallelOldGC) i++;if (UseG1GC) i++;if (i > 1) { jio_fprintf(defaultStream::error_stream(), "Conflicting collector combinations in option list; " "please refer to the release notes for the combinations " "allowedn"); status = false;} Copyright 2011 Kodewerk Ltd. All rights reserved
  • 11. Kodewerk tm Java Memory PoolsJava Performance Services Eden S0 S1 Tenured Perm Young Old Gen GenNormal Flow- objects created in Eden- copied into the active survivor space- copied between survivor spaces- copied to Old Gen when ages reaches tenuring threshold Copyright 2011 Kodewerk Ltd. All rights reserved
  • 12. Kodewerk tm Java Memory PoolsJava Performance Services Eden S0 S1 Tenured Perm Young Old Gen GenAbnormal Situations- memory leaks (loitering objects)- long pause times (heap too big, poorparallelization)- too frequent full GC (zombies, poor throughput)- survivor space flooding (premature promotion)- fragmented tenured space (compaction)- high rates of object creation- etc..... Copyright 2011 Kodewerk Ltd. All rights reserved
  • 13. Kodewerk tm Java Performance Services Puzzler #1 600Heap Usage after GC (MB) 500 400 300 200 100 1000 2000 3000 4000 Time (seconds) Copyright 2011 Kodewerk Ltd. All rights reserved
  • 14. Kodewerk tm Java Performance Services Puzzler #1 600Heap Usage after GC (MB) 500 400 300 what happened here? 200 100 1000 2000 3000 4000 Time (seconds) Copyright 2011 Kodewerk Ltd. All rights reserved
  • 15. Kodewerk tm GC Log Fragment Java Performance Services21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K->89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,real=0.32 secs]163.028: [GC 163.028: [DefNewDesired survivor size 115671040 bytes, new threshold 15 (max 15)- age 1: 736 bytes, 736 total- age 2: 112 bytes, 848 total- age 4: 40 bytes, 888 total- age 6: 232 bytes, 1120 total- age 7: 32 bytes, 1152 total- age 11: 96 bytes, 1248 total: 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times:user=0.00 sys=0.00, real=0.00 secs]Heap def new generation total 451968K, used 193472K [0x10020000, 0x39620000, 0x39620000) eden space 226048K, 85% used [0x10020000, 0x1bd0fe48, 0x1dce0000) from space 225920K, 0% used [0x1dce0000, 0x1dce0560, 0x2b980000) to space 225920K, 0% used [0x2b980000, 0x2b980000, 0x39620000) tenured generation total 677888K, used 443015K [0x39620000, 0x62c20000, 0x62c20000) the space 677888K, 65% used [0x39620000, 0x546c1d78, 0x546c1e00, 0x62c20000) compacting perm gen total 36608K, used 36480K [0x62c20000, 0x64fe0000, 0x66c20000) the space 36608K, 99% used [0x62c20000, 0x64fc0290, 0x64fc0400, 0x64fe0000)No shared spaces configured. Copyright 2011 Kodewerk Ltd. All rights reserved
  • 16. Kodewerk tm Java Performance Services Full GC21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K->89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,real=0.32 secs]Full GC requested @, started @ Copyright 2011 Kodewerk Ltd. All rights reserved
  • 17. Kodewerk tm Java Performance Services Full GC21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K->89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,real=0.32 secs] tenured space statistics perm space statistics- other stats are for the entire heap Copyright 2011 Kodewerk Ltd. All rights reserved
  • 18. Kodewerk tm Java Performance Services Full GC21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K->89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,real=0.32 secs]- occupancy before->after collection (memory pool configured size)- reporting on tenured, perm memory pools plus total for all Java heap spaces Copyright 2011 Kodewerk Ltd. All rights reserved
  • 19. Kodewerk tm Java Performance Services Full GC21.026: [Full GC 21.026: [Tenured: 87725K->89582K(146212K), 0.3151328 secs] 117886K->89582K(243748K), [Perm : 30974K->30958K(30976K)], 0.3152271 secs] [Times: user=0.31 sys=0.02,real=0.32 secs]- pause time statistics for tenured and all Java heap Copyright 2011 Kodewerk Ltd. All rights reserved
  • 20. Kodewerk tm GC Log Fragment Java Performance Services163.028: [GC 163.028: [DefNewDesired survivor size 115671040 bytes, new threshold 15 (max 15)- age 1: 736 bytes, 736 total- age 2: 112 bytes, 848 total- age 4: 40 bytes, 888 total- age 6: 232 bytes, 1120 total- age 7: 32 bytes, 1152 total- age 11: 96 bytes, 1248 total: 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times:user=0.00 sys=0.00, real=0.00 secs] young gen statistics statistics for all Java heap Copyright 2011 Kodewerk Ltd. All rights reserved
  • 21. Kodewerk tm GC Log Fragment Java Performance Services163.028: [GC 163.028: [DefNewDesired survivor size 115671040 bytes, new threshold 15 (max 15)- age 1: 736 bytes, 736 total- age 2: 112 bytes, 848 total- age 4: 40 bytes, 888 total- age 6: 232 bytes, 1120 total- age 7: 32 bytes, 1152 total- age 11: 96 bytes, 1248 total: 226048K->1K(451968K), 0.0018054 secs] 669064K->443016K(1129856K), 0.0018580 secs] [Times:user=0.00 sys=0.00, real=0.00 secs] surivor space sizing and tenuring thresholds breakdown of bytes @ age Copyright 2011 Kodewerk Ltd. All rights reserved
  • 22. Kodewerk tm GC Log Fragment Java Performance ServicesHeap def new generation total 451968K, used 193472K [0x10020000, 0x39620000, 0x39620000) eden space 226048K, 85% used [0x10020000, 0x1bd0fe48, 0x1dce0000) from space 225920K, 0% used [0x1dce0000, 0x1dce0560, 0x2b980000) to space 225920K, 0% used [0x2b980000, 0x2b980000, 0x39620000) tenured generation total 677888K, used 443015K [0x39620000, 0x62c20000, 0x62c20000) the space 677888K, 65% used [0x39620000, 0x546c1d78, 0x546c1e00, 0x62c20000) compacting perm gen total 36608K, used 36480K [0x62c20000, 0x64fe0000, 0x66c20000) the space 36608K, 99% used [0x62c20000, 0x64fc0290, 0x64fc0400, 0x64fe0000)No shared spaces configured.- breakdown of occupancy and configured size of each memory pool- pointer values describing each space (for debugging) Copyright 2011 Kodewerk Ltd. All rights reserved
  • 23. Kodewerk tm Java Performance Services Puzzler #20.194: [GC [PSYoungGen: 10489K->720K(305856K)] 10489K->720K(1004928K), 0.0061190 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]    0.200: [Full GC (System) [PSYoungGen: 720K->0K(305856K)] [ParOldGen: 0K->594K(699072K)] 720K->594K(1004928K) [PSPermGen: 6645K->6641K(204800K)], 0.0516540 secs] [Times: user=0.10 sys=0.00, real=0.06 secs]    6.184: [GC [PSYoungGen: 262208K->14797K(305856K)] 262802K->15392K(1004928K), 0.0354510 secs] [Times: user=0.18 sys=0.04, real=0.03 secs]    9.549: [GC [PSYoungGen: 277005K->43625K(305856K)] 277600K->60736K(1004928K), 0.0781960 secs] [Times: user=0.56 sys=0.07, real=0.08 secs]  11.768: [GC [PSYoungGen: 305833K->43645K(305856K)] 322944K->67436K(1004928K), 0.0584750 secs] [Times: user=0.40 sys=0.05, real=0.06 secs]  15.037: [GC [PSYoungGen: 305853K->43619K(305856K)] 329644K->72932K(1004928K), 0.0688340 secs] [Times: user=0.42 sys=0.01, real=0.07 secs]  19.372: [GC [PSYoungGen: 273171K->43621K(305856K)] 302483K->76957K(1004928K), 0.0573890 secs] [Times: user=0.41 sys=0.01, real=0.06 secs]  19.430: [Full GC (System) [PSYoungGen: 43621K->0K(305856K)] [ParOldGen: 33336K->54668K(699072K)] 76957K->54668K(1004928K) [PSPermGen:36356K->36296K(204800K)], 0.4569500 secs] [Times: user=1.77 sys=0.02, real=0.46 secs]  19.924: [GC [PSYoungGen: 4280K->128K(305856K)] 58949K->54796K(1004928K), 0.0041070 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  19.928: [Full GC (System) [PSYoungGen: 128K->0K(305856K)] [ParOldGen: 54668K->54532K(699072K)] 54796K->54532K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.3531480 secs] [Times: user=1.19 sys=0.10, real=0.35 secs]  20.284: [GC [PSYoungGen: 4280K->64K(305856K)] 58813K->54596K(1004928K), 0.0040580 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  20.288: [Full GC (System) [PSYoungGen: 64K->0K(305856K)] [ParOldGen: 54532K->54532K(699072K)] 54596K->54532K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2360580 secs] [Times: user=1.01 sys=0.01, real=0.24 secs]  20.525: [GC [PSYoungGen: 4280K->96K(305856K)] 58813K->54628K(1004928K), 0.0030960 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]  20.528: [Full GC (System) [PSYoungGen: 96K->0K(305856K)] [ParOldGen: 54532K->54533K(699072K)] 54628K->54533K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2311320 secs] [Times: user=0.88 sys=0.00, real=0.23 secs]  20.760: [GC [PSYoungGen: 4280K->96K(305856K)] 58814K->54629K(1004928K), 0.0034940 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]  20.764: [Full GC (System) [PSYoungGen: 96K->0K(305856K)] [ParOldGen: 54533K->54533K(699072K)] 54629K->54533K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2381600 secs] [Times: user=0.85 sys=0.01, real=0.24 secs]  21.201: [GC [PSYoungGen: 5160K->354K(305856K)] 59694K->54888K(1004928K), 0.0019950 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]  21.204: [Full GC (System) [PSYoungGen: 354K->0K(305856K)] [ParOldGen: 54533K->54792K(699072K)] 54888K->54792K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2358570 secs] [Times: user=0.98 sys=0.01, real=0.24 secs]  21.442: [GC [PSYoungGen: 4280K->64K(305856K)] 59073K->54856K(1004928K), 0.0022190 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  21.444: [Full GC (System) [PSYoungGen: 64K->0K(305856K)] [ParOldGen: 54792K->54792K(699072K)] 54856K->54792K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2475970 secs] [Times: user=0.95 sys=0.00, real=0.24 secs]  21.773: [GC [PSYoungGen: 11200K->741K(305856K)] 65993K->55534K(1004928K), 0.0030230 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  21.776: [Full GC (System) [PSYoungGen: 741K->0K(305856K)] [ParOldGen: 54792K->54376K(699072K)] 55534K->54376K(1004928K) [PSPermGen:36538K->36537K(204800K)], 0.2550630 secs] [Times: user=1.05 sys=0.00, real=0.25 secs]   Copyright 2011 Kodewerk Ltd. All rights reserved
  • 24. Kodewerk tm Java Performance Services Puzzler #20.194: [GC [PSYoungGen: 10489K->720K(305856K)] 10489K->720K(1004928K), 0.0061190 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]    0.200: [Full GC (System) [PSYoungGen: 720K->0K(305856K)] [ParOldGen: 0K->594K(699072K)] 720K->594K(1004928K) [PSPermGen: 6645K->6641K(204800K)], 0.0516540 secs] [Times: user=0.10 sys=0.00, real=0.06 secs]    6.184: [GC [PSYoungGen: 262208K->14797K(305856K)] 262802K->15392K(1004928K), 0.0354510 secs] [Times: user=0.18 sys=0.04, real=0.03 secs]    9.549: [GC [PSYoungGen: 277005K->43625K(305856K)] 277600K->60736K(1004928K), 0.0781960 secs] [Times: user=0.56 sys=0.07, real=0.08 secs]  11.768: [GC [PSYoungGen: 305833K->43645K(305856K)] 322944K->67436K(1004928K), 0.0584750 secs] [Times: user=0.40 sys=0.05, real=0.06 secs]  15.037: [GC [PSYoungGen: 305853K->43619K(305856K)] 329644K->72932K(1004928K), 0.0688340 secs] [Times: user=0.42 sys=0.01, real=0.07 secs]  19.372: [GC [PSYoungGen: 273171K->43621K(305856K)] 302483K->76957K(1004928K), 0.0573890 secs] [Times: user=0.41 sys=0.01, real=0.06 secs]  19.430: [Full GC (System) [PSYoungGen: 43621K->0K(305856K)] [ParOldGen: 33336K->54668K(699072K)] 76957K->54668K(1004928K) [PSPermGen:36356K->36296K(204800K)], 0.4569500 secs] [Times: user=1.77 sys=0.02, real=0.46 secs]  19.924: [GC [PSYoungGen: 4280K->128K(305856K)] 58949K->54796K(1004928K), 0.0041070 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  19.928: [Full GC (System) [PSYoungGen: 128K->0K(305856K)] [ParOldGen: 54668K->54532K(699072K)] 54796K->54532K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.3531480 secs] [Times: user=1.19 sys=0.10, real=0.35 secs]  20.284: [GC [PSYoungGen: 4280K->64K(305856K)] 58813K->54596K(1004928K), 0.0040580 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  20.288: [Full GC (System) [PSYoungGen: 64K->0K(305856K)] [ParOldGen: 54532K->54532K(699072K)] 54596K->54532K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2360580 secs] [Times: user=1.01 sys=0.01, real=0.24 secs]  20.525: [GC [PSYoungGen: 4280K->96K(305856K)] 58813K->54628K(1004928K), 0.0030960 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]  20.528: [Full GC (System) [PSYoungGen: 96K->0K(305856K)] [ParOldGen: 54532K->54533K(699072K)] 54628K->54533K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2311320 secs] [Times: user=0.88 sys=0.00, real=0.23 secs]  20.760: [GC [PSYoungGen: 4280K->96K(305856K)] 58814K->54629K(1004928K), 0.0034940 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]  20.764: [Full GC (System) [PSYoungGen: 96K->0K(305856K)] [ParOldGen: 54533K->54533K(699072K)] 54629K->54533K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2381600 secs] [Times: user=0.85 sys=0.01, real=0.24 secs]  21.201: [GC [PSYoungGen: 5160K->354K(305856K)] 59694K->54888K(1004928K), 0.0019950 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]  21.204: [Full GC (System) [PSYoungGen: 354K->0K(305856K)] [ParOldGen: 54533K->54792K(699072K)] 54888K->54792K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2358570 secs] [Times: user=0.98 sys=0.01, real=0.24 secs]  21.442: [GC [PSYoungGen: 4280K->64K(305856K)] 59073K->54856K(1004928K), 0.0022190 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  21.444: [Full GC (System) [PSYoungGen: 64K->0K(305856K)] [ParOldGen: 54792K->54792K(699072K)] 54856K->54792K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2475970 secs] [Times: user=0.95 sys=0.00, real=0.24 secs]  21.773: [GC [PSYoungGen: 11200K->741K(305856K)] 65993K->55534K(1004928K), 0.0030230 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  21.776: [Full GC (System) [PSYoungGen: 741K->0K(305856K)] [ParOldGen: 54792K->54376K(699072K)] 55534K->54376K(1004928K) [PSPermGen:36538K->36537K(204800K)], 0.2550630 secs] [Times: user=1.05 sys=0.00, real=0.25 secs]   Copyright 2011 Kodewerk Ltd. All rights reserved
  • 25. Kodewerk tm Java Performance Services Puzzler #20.194: [GC [PSYoungGen: 10489K->720K(305856K)] 10489K->720K(1004928K), 0.0061190 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]    0.200: [Full GC (System) [PSYoungGen: 720K->0K(305856K)] [ParOldGen: 0K->594K(699072K)] 720K->594K(1004928K) [PSPermGen: 6645K->6641K(204800K)], 0.0516540 secs] [Times: user=0.10 sys=0.00, real=0.06 secs]    6.184: [GC [PSYoungGen: 262208K->14797K(305856K)] 262802K->15392K(1004928K), 0.0354510 secs] [Times: user=0.18 sys=0.04, real=0.03 secs]    9.549: [GC [PSYoungGen: 277005K->43625K(305856K)] 277600K->60736K(1004928K), 0.0781960 secs] [Times: user=0.56 sys=0.07, real=0.08 secs]  11.768: [GC [PSYoungGen: 305833K->43645K(305856K)] 322944K->67436K(1004928K), 0.0584750 secs] [Times: user=0.40 sys=0.05, real=0.06 secs]  15.037: [GC [PSYoungGen: 305853K->43619K(305856K)] 329644K->72932K(1004928K), 0.0688340 secs] [Times: user=0.42 sys=0.01, real=0.07 secs]  19.372: [GC [PSYoungGen: 273171K->43621K(305856K)] 302483K->76957K(1004928K), 0.0573890 secs] [Times: user=0.41 sys=0.01, real=0.06 secs]  19.430: [Full GC (System) [PSYoungGen: 43621K->0K(305856K)] [ParOldGen: 33336K->54668K(699072K)] 76957K->54668K(1004928K) [PSPermGen:36356K->36296K(204800K)], 0.4569500 secs] [Times: user=1.77 sys=0.02, real=0.46 secs]  19.924: [GC [PSYoungGen: 4280K->128K(305856K)] 58949K->54796K(1004928K), 0.0041070 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  19.928: [Full GC (System) [PSYoungGen: 128K->0K(305856K)] [ParOldGen: 54668K->54532K(699072K)] 54796K->54532K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.3531480 secs] [Times: user=1.19 sys=0.10, real=0.35 secs]  20.284: [GC [PSYoungGen: 4280K->64K(305856K)] 58813K->54596K(1004928K), 0.0040580 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  20.288: [Full GC (System) [PSYoungGen: 64K->0K(305856K)] [ParOldGen: 54532K->54532K(699072K)] 54596K->54532K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2360580 secs] [Times: user=1.01 sys=0.01, real=0.24 secs]  20.525: [GC [PSYoungGen: 4280K->96K(305856K)] 58813K->54628K(1004928K), 0.0030960 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]  20.528: [Full GC (System) [PSYoungGen: 96K->0K(305856K)] [ParOldGen: 54532K->54533K(699072K)] 54628K->54533K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2311320 secs] [Times: user=0.88 sys=0.00, real=0.23 secs]  20.760: [GC [PSYoungGen: 4280K->96K(305856K)] 58814K->54629K(1004928K), 0.0034940 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]  20.764: [Full GC (System) [PSYoungGen: 96K->0K(305856K)] [ParOldGen: 54533K->54533K(699072K)] 54629K->54533K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2381600 secs] [Times: user=0.85 sys=0.01, real=0.24 secs]  21.201: [GC [PSYoungGen: 5160K->354K(305856K)] 59694K->54888K(1004928K), 0.0019950 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]  21.204: [Full GC (System) [PSYoungGen: 354K->0K(305856K)] [ParOldGen: 54533K->54792K(699072K)] 54888K->54792K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2358570 secs] [Times: user=0.98 sys=0.01, real=0.24 secs]  21.442: [GC [PSYoungGen: 4280K->64K(305856K)] 59073K->54856K(1004928K), 0.0022190 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  21.444: [Full GC (System) [PSYoungGen: 64K->0K(305856K)] [ParOldGen: 54792K->54792K(699072K)] 54856K->54792K(1004928K) [PSPermGen:36300K->36300K(204800K)], 0.2475970 secs] [Times: user=0.95 sys=0.00, real=0.24 secs]  21.773: [GC [PSYoungGen: 11200K->741K(305856K)] 65993K->55534K(1004928K), 0.0030230 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]  21.776: [Full GC (System) [PSYoungGen: 741K->0K(305856K)] [ParOldGen: 54792K->54376K(699072K)] 55534K->54376K(1004928K) [PSPermGen:36538K->36537K(204800K)], 0.2550630 secs] [Times: user=1.05 sys=0.00, real=0.25 secs]   Copyright 2011 Kodewerk Ltd. All rights reserved
  • 26. Kodewerk tm Java Performance Services Puzzler #3848463.696: [GC [1 CMS-initial-mark: 989633K(1075200K)] 1003551K(1397760K), 6.3838946 secs][Times: user=0.06 sys=0.01, real=6.39 secs]848470.085: [CMS-concurrent-mark-start]848605.512: [CMS-concurrent-mark: 135.377/135.426 secs] [Times: user=7.83 sys=9.92, real=135.44 secs]848605.512: [CMS-concurrent-preclean-start]848605.726: [CMS-concurrent-preclean: 0.207/0.214 secs] [Times: user=0.06 sys=0.00, real=0.22 secs]848605.726: [CMS-concurrent-abortable-preclean-start] CMS: abort preclean due to time 848610.846: [CMS-concurrent-abortable-preclean: 1.014/5.120 secs][Times: user=1.05 sys=0.08, real=5.13 secs]848610.871: [GC[YG occupancy: 160577 K (322560 K)]848610.871: [Rescan (parallel) , 0.1280728secs]848610.999: [weak refs processing, 1.3120556 secs] [1 CMS-remark: 989633K(1075200K)]1150210K(1397760K), 1.4413516 secs] Application runtime details can help threads stall for 60 seconds during pause Copyright 2011 Kodewerk Ltd. All rights reserved
  • 27. Kodewerk tm Java Performance Services Puzzler #4147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ... ????Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved
  • 28. Kodewerk tm Java Performance Services Puzzler #4147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ...Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved
  • 29. Kodewerk tm Java Performance Services Puzzler #4147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ...Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved
  • 30. Kodewerk tm Java Performance Services Puzzler #4147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ...Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs]147.823 + 0.8074580 = 148.630 ~= 148.632 Copyright 2011 Kodewerk Ltd. All rights reserved
  • 31. Kodewerk tm Java Performance Services Puzzler #4147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ...Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved
  • 32. Kodewerk tm Java Performance Services Puzzler #4147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs]148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs]java.lang.OutOfMemoryError: Java heap spaceDumping heap to java_pid18706.hprof ... ????Heap dump file created [83874513 bytes in 3.330 secs]160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00,real=0.28 secs] Copyright 2011 Kodewerk Ltd. All rights reserved
  • 33. Kodewerk tmJava Performance Services Case Study Copyright 2011 Kodewerk Ltd. All rights reserved
  • 34. QuestionsCopyright 2011 Kodewerk Ltd. All rights reserved