Successfully reported this slideshow.
Your SlideShare is downloading. ×

Tuning the g1gc

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
Trouble with memory
Trouble with memory
Loading in …3
×

Check these out next

1 of 58 Ad
Advertisement

More Related Content

Slideshows for you (20)

Similar to Tuning the g1gc (20)

Advertisement

Recently uploaded (20)

Tuning the g1gc

  1. 1. Copyright 2018 Kirk Pepperdine TUNING G1GC A GUIDE TO
  2. 2. Copyright 2018 Kirk Pepperdine HOW TO TUNE THE G1GC ▸ Set -XX:+UseG1GC ▸ Set max heap size using -mx ▸default value is 1/4 of physical RAM ▸ Set pause time goal using -XX:MaxGCPauseMillis=200 (default value) jClarity
  3. 3. Copyright 2018 Kirk Pepperdine Questions? jClarity
  4. 4. Copyright 2018 Kirk Pepperdine jClarity I lift, you grab. … Was that concept just a little too complex?
  5. 5. Copyright 2018 Kirk Pepperdine jClarity If only it was that easy!
  6. 6. Copyright 2018 Kirk Pepperdine Questions? jClarity
  7. 7. Copyright 2018 Kirk Pepperdine jClarity ▸ Kirk Pepperdine ▸ Author of jPDM, a performance diagnostic model ▸ Author of the original Java Performance Tuning workshop ▸ Co-founded jClarity ▸ Building the smart generation of performance diagnostic tooling ▸ Bring predictability into the diagnostic process ▸ Co-founded JCrete ▸ The hottest unconference on the planet ▸ Java Champion(s) OUR MARKETING SLIDE
  8. 8. Copyright 2018 Kirk Pepperdine ▸Models help us to ▸ understand how things work ▸ develop a cost model ▸Memory management consists of ▸ allocators ▸ mutators ▸ Garbage Collector IMPORTANCE OF MODELS jClarity
  9. 9. Copyright 2018 Kirk Pepperdine ▸Allocator/Mutator takes on all responsibility for managing memory ▸ unmanaged runtimes ▸Workload is shared between Allocator/Mutator and Garbage Collectors ▸ balance between allocation/mutation throughput and GC pause time ▸more work to GC equates to longer GC pause times ▸more work to allocator/mutators equates to reduces application throughput ▸ Lets develop an understanding so we can build a cost model ALLOCATORS VS COLLECTORS jClarity
  10. 10. Copyright 2018 Kirk Pepperdine THINGS WE NEED ▸ Java Heap ▸Regions ▸ Mark-Sweep (copy) ▸Young generational collection ▸Tenured Mark ▸Mixed collection ▸ Supporting Data Structures ▸ Collection set (CSet) ▸ Remembered Set (RSet) ▸ RSet refinement queue ▸ Tuning G1GC jClarity
  11. 11. Copyright 2018 Kirk Pepperdine THE G1GC HEAP jClarity ▸Reserved as a single contiguous region at JVM startup ▸Divide into 2048 to 4095 regions ▸ heap size is specified with -mx ▸ regions size is one of 1, 2, 4, 8, 16, or 32m ▸-XX:G1HeapRegionSize=<n> ▸example for -mx10G Region size = 10240M/2048 = 5m Number of regions = 10G/4m = 2560 regions
  12. 12. Copyright 2018 Kirk Pepperdine G1GC ALLOCATION jClarity All regions are put into a free region list
  13. 13. Copyright 2018 Kirk Pepperdine G1GC ALLOCATION jClarity Free Region List
  14. 14. Copyright 2018 Kirk Pepperdine ERGONOMICS jClarity Free Region List Regions allotted for Eden
  15. 15. Copyright 2018 Kirk Pepperdine G1GC ALLOCATION jClarity Free Region List MyDominObject mdo = new MyDomainObject Take region from free list
  16. 16. Copyright 2018 Kirk Pepperdine G1GC ALLOCATION jClarity Free Region List MyDominObject mdo = new MyDomainObject Label it as Eden
  17. 17. Copyright 2018 Kirk Pepperdine G1GC ALLOCATION jClarity Free Region List MyDominObject mdo = new MyDomainObject allocate
  18. 18. Copyright 2018 Kirk Pepperdine G1GC ALLOCATION jClarity Free Region List Continue until allotted Regions are full
  19. 19. Copyright 2018 Kirk Pepperdine G1GC ALLOCATION jClarity Free Region List Allocation failure triggers Young Gen collection
  20. 20. Copyright 2018 Kirk Pepperdine FORMING A CSET jClarity Free Region List Allocation failure triggers Young Gen collection CSet
  21. 21. Copyright 2018 Kirk Pepperdine YOUNG COLLECTION ▸ Mark/Sweep (copy) ▸Mix of parallel and serial phases jClarity 1. Place all young gen regions into a CSet 2. Calculate a root set for the CSet 3. Mark all live data in the CSet 4. Evacuate all live data to the ‘to’ space
 The ‘to’ space regions allocated from the free region list 5. Place eden regions back on free list 6. Ergonomics recalculates number of regions to allocate to Eden
  22. 22. Copyright 2018 Kirk Pepperdine G1GC SIZING jClarity Free Region List Head of List New allocation limit -XX:G1MaxNewSizePercentage=60, -XX:G1NewSizePercentage=5 Survivor regions
  23. 23. Copyright 2018 Kirk Pepperdine THE G1GC HEAP Eden Regions Unallocated Regions Young jClarity Survivor Region aka: to space
  24. 24. Copyright 2018 Kirk Pepperdine TENURING Young jClarity Survivor Region aka: to space Tenured Region Eden Regions Unallocated Regions
  25. 25. Copyright 2018 Kirk Pepperdine YOUNG COLLECTION ▸ Mark/Sweep (copy) ▸Mix of parallel and serial phases ▸Place all young gen regions into a CSet ▸Calculation a root set for the CSet ▸Mark all live ▸Evacuate all live to survivor regions allocated from the free region list ▸Place eden regions back on free list ▸Ergonomics recalculates number of regions to allocate to Eden jClarity Time complexity!!!!
  26. 26. Copyright 2018 Kirk Pepperdine GC ROOTS jClarity
  27. 27. Copyright 2018 Kirk Pepperdine GC ROOTS jClarity
  28. 28. Copyright 2018 Kirk Pepperdine GC ROOTS jClarity Metaspace
  29. 29. Copyright 2018 Kirk Pepperdine GC ROOTS jClarity Metaspace Compressed Classspace
  30. 30. Copyright 2018 Kirk Pepperdine GC ROOTS jClarity JNI Metaspace Compressed Classspace
  31. 31. Copyright 2018 Kirk Pepperdine GC ROOTS jClarity Stack FramesJNI Metaspace Compressed Classspace
  32. 32. Copyright 2018 Kirk Pepperdine GC ROOTS jClarity Code Cache Stack FramesJNI Metaspace Compressed Classspace
  33. 33. Copyright 2018 Kirk Pepperdine GC ROOTS jClarity Code Cache Stack FramesJNI Metaspace Compressed Classspace C0 L1 L2 L3 Cn L1 L2… … … … CPU Registers
  34. 34. Copyright 2018 Kirk Pepperdine SCAN FOR ROOTS ▸ Multi-threaded but…. ▸Each individual space can be managed by a single thread ▸Large collections can only be managed by a single thread ▸ Scan for root time will be the longest time for scanning any single space ▸Often scan for roots is linear to the size of tenured ▸ G1 avoids this by introducing a remembered set (RSet) jClarity
  35. 35. Copyright 2018 Kirk Pepperdine RSET ▸Each Young gen region has an RSet ▸ Tracks pointers from Tenured regions into target region ▸ Requires a write barrier ▸ Has an associated space complexity ▸Complex implemetation solves this ▸RSet maintenance costs are non-linear to size of the region ▸ Mutations place an unwanted burden on the mutator threads jClarity
  36. 36. Copyright 2018 Kirk Pepperdine RSET REFINEMENT QUEUE No threads are working Number of RSets that can be processed run less than 10% of the pause time goal All refinement threads are working G1UpdatingPauseTimePercent > 10% Application threads are involved in RSet refinement. Slow mutation rates which acts as back pressure jClarity
  37. 37. Copyright 2018 Kirk Pepperdine [15.316s][info ][gc,start ] GC(0) Pause Young (G1 Evacuation Pause) [15.316s][info ][gc,start ] GC(0) Pause Young (G1 Evacuation Pause) [15.316s][info ][gc,task ] GC(0) Using 8 workers of 8 for evacuation [15.322s][info ][gc,phases ] GC(0) Pre Evacuate Collection Set: 0.0ms [15.322s][info ][gc,phases ] GC(0) Evacuate Collection Set: 4.9ms [15.322s][info ][gc,phases ] GC(0) Post Evacuate Collection Set: 1.0ms [15.322s][info ][gc,phases ] GC(0) Other: 0.2ms [15.322s][info ][gc,heap ] GC(0) Eden regions: 24->0(150) [15.322s][info ][gc,heap ] GC(0) Survivor regions: 0->3(3) [15.322s][info ][gc,heap ] GC(0) Old regions: 0->2 [15.322s][info ][gc,heap ] GC(0) Humongous regions: 0->0 [15.322s][info ][gc,metaspace ] GC(0) Metaspace: 16151K->16151K(1064960K) [15.322s][info ][gc ] GC(0) Pause Young (G1 Evacuation Pause) 24M->4M(256M) 6.183ms [15.322s][info ][gc,cpu ] GC(0) User=0.04s Sys=0.00s Real=0.00s YOUNG COLLECTION jClarity
  38. 38. Copyright 2018 Kirk Pepperdine [15.322s][info ][gc,phases ] GC(0) Pre Evacuate Collection Set: 0ms [15.322s][debug][gc,phases ] GC(0) Choose Collection Set: 0.0ms [15.322s][debug][gc,phases ] GC(0) Humongous Register: 0.0ms YOUNG COLLECTION jClarity
  39. 39. Copyright 2018 Kirk Pepperdine [15.322s][info ][gc,phases ] GC(0) Evacuate Collection Set: 4.9ms [15.322s][debug][gc,phases ] GC(0) Ext Root Scanning (ms): Min: 0.3, Avg: 0.8, Max: 2.5, Diff: 2.2, Sum: 6.3, Workers: 8 [15.322s][debug][gc,phases ] GC(0) Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0, Workers: 8 [15.322s][debug][gc,phases ] GC(0) Processed Buffers: Min: 0, Avg: 0.0, Max: 0, Diff: 0, Sum: 0, Workers: 8 [15.322s][debug][gc,phases ] GC(0) Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0, Workers: 8 [15.322s][debug][gc,phases ] GC(0) Code Root Scanning (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 1.7, Workers: 8 [15.322s][debug][gc,phases ] GC(0) Object Copy (ms): Min: 1.8, Avg: 3.4, Max: 4.3, Diff: 2.5, Sum: 27.0, Workers: 8 [15.322s][debug][gc,phases ] GC(0) Termination (ms): Min: 0.0, Avg: 0.4, Max: 0.6, Diff: 0.6, Sum: 3.4, Workers: 8 [15.322s][debug][gc,phases ] GC(0) Termination Attempts: Min: 1, Avg: 17.4, Max: 34, Diff: 33, Sum: 139, Workers: 8 [15.322s][debug][gc,phases ] GC(0) GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.1, Workers: 8 [15.322s][debug][gc,phases ] GC(0) GC Worker Total (ms): Min: 4.8, Avg: 4.8, Max: 4.8, Diff: 0.0, Sum: 38.6, Workers: 8 YOUNG COLLECTION jClarity
  40. 40. Copyright 2018 Kirk Pepperdine [15.322s][info ][gc,phases ] GC(0) Post Evacuate Collection Set: 1.0ms [15.322s][debug][gc,phases ] GC(0) Code Roots Fixup: 0.1ms [15.322s][debug][gc,phases ] GC(0) Preserve CM Refs: 0.0ms [15.322s][debug][gc,phases ] GC(0) Reference Processing: 0.7ms [15.322s][debug][gc,phases ] GC(0) Clear Card Table: 0.0ms [15.322s][debug][gc,phases ] GC(0) Reference Enqueuing: 0.0ms [15.322s][debug][gc,phases ] GC(0) Merge Per-Thread State: 0.0ms [15.322s][debug][gc,phases ] GC(0) Code Roots Purge: 0.0ms [15.322s][debug][gc,phases ] GC(0) Redirty Cards: 0.0ms [15.322s][debug][gc,phases ] GC(0) Free Collection Set: 0.2ms [15.322s][debug][gc,phases ] GC(0) Humongous Reclaim: 0.0ms [15.322s][debug][gc,phases ] GC(0) Expand Heap After Collection: 0.0ms YOUNG COLLECTION jClarity
  41. 41. Copyright 2018 Kirk Pepperdine [73.082s][info ][gc ] GC(263) Concurrent Cycle [73.082s][info ][gc,marking ] GC(263) Concurrent Clear Claimed Marks [73.082s][info ][gc,marking ] GC(263) Concurrent Clear Claimed Marks 0.018ms [73.082s][info ][gc,marking ] GC(263) Concurrent Scan Root Regions [73.084s][info ][gc,marking ] GC(263) Concurrent Scan Root Regions 2.325ms [73.084s][info ][gc,marking ] GC(263) Concurrent Mark (73.084s) [73.084s][info ][gc,marking ] GC(263) Concurrent Mark From Roots [73.084s][info ][gc,task ] GC(263) Using 2 workers of 2 for marking [73.138s][info ][gc,marking ] GC(263) Concurrent Mark From Roots 53.902ms [73.138s][info ][gc,marking ] GC(263) Concurrent Mark (73.084s, 73.138s) 53.954ms [73.139s][info ][gc,start ] GC(263) Pause Remark [73.160s][info ][gc,stringtable] GC(263) Cleaned string and symbol table, strings: 7924 processed, 21 removed, symbols: 55530 processed, 17 removed [73.160s][info ][gc ] GC(263) Pause Remark 211M->211M(256M) 21.685ms [73.160s][info ][gc,cpu ] GC(263) User=0.03s Sys=0.00s Real=0.02s [73.160s][info ][gc,marking ] GC(263) Concurrent Create Live Data [73.168s][info ][gc,marking ] GC(263) Concurrent Create Live Data 8.089ms [73.169s][info ][gc,start ] GC(263) Pause Cleanup [73.169s][info ][gc ] GC(263) Pause Cleanup 223M->213M(256M) 0.271ms [73.169s][info ][gc,cpu ] GC(263) User=0.00s Sys=0.00s Real=0.00s [73.169s][info ][gc,marking ] GC(263) Concurrent Complete Cleanup [73.169s][info ][gc,marking ] GC(263) Concurrent Complete Cleanup 0.013ms [73.169s][info ][gc,marking ] GC(263) Concurrent Cleanup for Next Mark [73.171s][info ][gc,marking ] GC(263) Concurrent Cleanup for Next Mark 1.646ms [73.171s][info ][gc ] GC(263) Concurrent Cycle 89.437ms TENURED COLLECTION jClarity
  42. 42. Copyright 2018 Kirk Pepperdine COLLECTING TENURED ▸ Triggered at IHOP=45% ▸ Tenured regions are marked ▸region liveliness is calculated by the mark 23.615: [GC pause (Metadata GC Threshold) (young) (initial-mark) 23.649: [GC concurrent-root-region-scan-start] 23.698: [GC concurrent-root-region-scan-end, 0.0484826 secs] 23.698: [GC concurrent-mark-start] 23.889: [GC concurrent-mark-end, 0.1907474 secs] 23.889: [GC remark 23.889: [Finalize Marking, 0.0002867 secs] 23.889: [GC ref-proc, 0.0001423 secs] 23.889: [Unloading, 0.0094320 secs], 0.0123489 secs] 23.902: [GC cleanup 504M->500M(1024M), 0.0015885 secs] 23.903: [GC concurrent-cleanup-start] 23.903: [GC concurrent-cleanup-end, 0.0000172 secs] jClarity
  43. 43. Copyright 2018 Kirk Pepperdine BUILDING A CSET REVISITED ▸ CSet contains all eden and survivor regions ▸a mixed collection will add a subset of tenured regions to the CSet ▸ CSet size is controlled by ▸the pause time goal (200ms default) ▸minimum collection percentages (10%) ▸other factors jClarity
  44. 44. Copyright 2018 Kirk Pepperdine MARK AND CALCULATE LIVELINESS Tenured regions jClarity
  45. 45. Copyright 2018 Kirk Pepperdine SORT BY LIVELINESS jClarity
  46. 46. Copyright 2018 Kirk Pepperdine SORT BY LIVELINESS jClarity free eligible for collection not eligible -XX:G1MixedGCLiveThresholdPercent=85
  47. 47. Copyright 2018 Kirk Pepperdine ELIGIBLE REGIONS jClarity
  48. 48. Copyright 2018 Kirk Pepperdine MIXED COLLECTION COUNT jClarity -XX:G1OldCSetRegionThresholdPercent=10 -XX:G1HeapWastePercent=10 mixed 1 mixed 2 mixed 3 mixed 8…
  49. 49. Copyright 2018 Kirk Pepperdine HUMONGOUS ALLOCATIONS Tenured Young jClarity Tenured Region Humungous Region (start) Humungous Region (cont) Survivor Region aka: to space Eden Region Unallocated Region
  50. 50. Copyright 2018 Kirk Pepperdine HUMONGOUS ALLOCATIONS Humungous Region (start) Humungous Region (cont) jClarity ▸Allocation that is 1/2 the size of a region ▸Fragmented heaps may not have enough space to satisfy an allocation ▸ dip into reserved space ▸ trigger a young gen collection with and initial-mark ▸commit more regions to the heap ▸ worst case can trigger a Full GC
  51. 51. Copyright 2018 Kirk Pepperdine RESERVED SPACE Tenured Region Humungous Region (start) Humungous Region (cont) Survivor Region aka: to space Eden Region Unallocated Region Tenured Young jClarity Reserved
  52. 52. Copyright 2018 Kirk Pepperdine SOMETHINGS GONE OFF jClarity
  53. 53. Copyright 2018 Kirk Pepperdine HOW TO GET A GC LOG jClarity -Xloggc:gc.log -XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+PrintReferenceGC -Xlog:gc*,gc+ref,gc+phases=debug,gc+age=debug,safepoint:file=gc.log
  54. 54. Copyright 2018 Kirk Pepperdine jClarity DON’T LIKE READING GC LOGS? I’ve got your back
  55. 55. Copyright 2018 Kirk Pepperdine ▸ JDK 9 ▸ G1GC is default collector ▸ improve ability to determine important memory setting ▸ migrate GC logging to Unified logging ▸ JDK 10 ▸ Parallel full GC ▸ JDK 11 ▸ new perf counters for STW during concurrent collection ▸ adaptive parallel reference processing ▸ Adaptive scaling for threads RELEASE NOTES jClarity
  56. 56. Copyright 2018 Kirk Pepperdine WHY IS IT TAKING SO LONG??? jClarity Lets look at a GC Log using Censum
  57. 57. Copyright 2018 Kirk Pepperdine CONTROLLING OBJECT COPY COSTS volume of reachable objects jClarity Occupancy Age
  58. 58. Copyright 2018 Kirk Pepperdine TEXT TITLE TEXT ▸ Body Level One ▸ Body Level Two ▸ Body Level Three ▸ Body Level Four ▸ Body Level Five Java Performance Tuning Workshop Send us a Java 11 GC log or tweet about @jclarity and #censum and receive a free Censum License

×