Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Use Ruby GC in full..

2,840 views

Published on

Alexander Naumchuk - «GC time garbage collection» (Ideeli)

Use Ruby GC in full..

  1. 1. GCNO effort CLEAN memory
  2. 2. GCNO effort CLEAN memory SOME
  3. 3. Who am I? Alex NaumchukSoftware Architect at Consulter at Brightgrove alex.naumchuk@gmail.com
  4. 4. What IS GC?
  5. 5. What IS GC?Developers
  6. 6. 80k req/min 10k db queries/sec 125 prod servers avg response time: 25ms
  7. 7. Automatic collector reclaiming garbage C, C++
  8. 8. Automatic collector reclaiming garbage C, C++ Ruby, Java, .NET
  9. 9. Automatic collector reclaiming garbage C, C++ ALLY MA NU D o it Ruby, Java, .NET
  10. 10. Automatic collector reclaiming garbage C, C++ ALLY MA NU D o it Ruby, Java, .NET about it ’t C ARE Don
  11. 11. Should you care about memory?
  12. 12. Should you care about memory? YES!
  13. 13. Should you care about memory? YES! at least know how it works
  14. 14. REE/MRI Memory organizationHeaps slot slot slot slot slot slot slot slot slot slot slot slot slot slot slot
  15. 15. Slot RVALUEtypedef struct RVALUE { union { struct { unsigned long flags; /* always 0 for freed obj */ struct RVALUE *next; } free; struct RBasic basic; struct RObject object; struct RClass klass; struct RFloat flonum; struct RString string; struct RArray array; struct RRegexp regexp; struct RHash hash; struct RData data; struct RStruct rstruct; struct RBignum bignum; struct RFile file; struct RNode node; struct RMatch match; struct RVarmap varmap; struct SCOPE scope; } as;} RVALUE;
  16. 16. REE GC Options
  17. 17. Default SettingsRUBY_HEAP_SLOTS 10000
  18. 18. Default SettingsRUBY_HEAP_SLOTS 10000RUBY_HEAP_SLOTS_INCREMENT 10000
  19. 19. Default SettingsRUBY_HEAP_SLOTS 10000RUBY_HEAP_SLOTS_INCREMENT 10000RUBY_HEAP_SLOTS_GROWTH_FACTOR 1.8
  20. 20. Default SettingsRUBY_HEAP_SLOTS 10000RUBY_HEAP_SLOTS_INCREMENT 10000RUBY_HEAP_SLOTS_GROWTH_FACTOR 1.8RUBY_GC_MALLOC_LIMIT 8000000
  21. 21. Default SettingsRUBY_HEAP_SLOTS 10000RUBY_HEAP_SLOTS_INCREMENT 10000RUBY_HEAP_SLOTS_GROWTH_FACTOR 1.8RUBY_GC_MALLOC_LIMIT 8000000RUBY_HEAP_FREE_MIN 4096
  22. 22. Ideeli Settings MM FERUBY_HEAP_SLOTS 6000000 2400000RUBY_HEAP_SLOTS_INCREMENT 100000 100000RUBY_HEAP_SLOTS_GROWTH_FACTOR 1 1RUBY_GC_MALLOC_LIMIT 100000000 40000000RUBY_HEAP_FREE_MIN 3500000 100000
  23. 23. When GC runs malloc limit exceeded min free slots < free slotsnot enough slots to allocate value
  24. 24. Mark Phase
  25. 25. GC Flow Mark Sweep Finalize
  26. 26. Statistics >>ObjectSpace.statisticsNumber of objects : 1631452 (1291302 AST nodes, 79.15%)Heap slot size : 40GC cycles so far : 32Number of heaps : 9Total size of objects : 63728.59 KBTotal size of heaps : 119579.81 KB (55851.22 KB = 46.71% unused)Leading free slots : 1154340 (45091.41 KB = 37.71%)Trailing free slots : 4 (0.16 KB = 0.00%)Number of contiguous groups of 16 slots: 76771 (40.13%)Number of terminal objects: 8973 (0.29%)
  27. 27. Other StatsGC.enable_statsGC.disable_statsGC.collectionsGC.timeGC.growthGC.clear_statsGC.dumpGC.allocated_sizeGC.num_allocationsObjectSpace.live_objectsObjectSpace.allocated_objects
  28. 28. Let’s Play Let’s play
  29. 29. CodeGC.enable_statsrequire ‘config/environment’GC.timeGC.num_allocationsGC.collections
  30. 30. Default optionsGC.enable_statsrequire ‘config/environment’GC.time ~2 SecondsGC.num_allocations 5_635_070GC.collections 31
  31. 31. ObjectSpace.statisticsNumber of objects : 1643029 (1265158 AST nodes, 77.00%)Heap slot size : 40GC cycles so far : 31Number of heaps : 9Total size of objects : 64180.82 KBTotal size of heaps : 119579.42 KB (55398.60 KB = 46.33% unused)Leading free slots : 1093423 (42711.84 KB = 35.72%)Trailing free slots : 0 (0.00 KB = 0.00%)Number of contiguous groups of 16 slots: 73754 (38.55%)Number of terminal objects: 7274 (0.24%)
  32. 32. Default options• Too many GC cycles• Many heaps• Bad heap utilization
  33. 33. New optionsRUBY_HEAP_MIN_SLOTS = 1_800_000RUBY_GC_MALLOC_LIMIT = 50_000_000 GC.time ~1 Second GC.num_allocations 5_634_856 GC.collections 6
  34. 34. New options RUBY_HEAP_MIN_SLOTS = 1_800_000 RUBY_GC_MALLOC_LIMIT = 50_000_000Number of objects : 1682325 (1272351 AST nodes, 75.63%)Heap slot size : 40GC cycles so far :6Number of heaps : 1Total size of objects : 65715.82 KBTotal size of heaps : 70312.51 KB (4596.69 KB = 6.54% unused)Leading free slots : 39246 (1533.05 KB = 2.18%)Trailing free slots : 0 (0.00 KB = 0.00%)Number of contiguous groups of 16 slots: 4305 (3.83%)Number of terminal objects: 7435 (0.41%)
  35. 35. Can we do it even better?
  36. 36. Can we do it even better? YES!
  37. 37. Enhanced optionsRUBY_HEAP_MIN_SLOTS = 2_100_000RUBY_GC_MALLOC_LIMIT = 100_000_000 GC.time ~0.8 Second GC.num_allocations 5_634_892 GC.collections 3
  38. 38. Enhanced options RUBY_HEAP_MIN_SLOTS = 2_100_000 RUBY_GC_MALLOC_LIMIT = 100_000_000Number of objects : 1816515 (1297270 AST nodes, 71.42%)Heap slot size : 40GC cycles so far :3Number of heaps : 1Total size of objects : 70957.62 KBTotal size of heaps : 82031.26 KB (11073.64 KB = 13.50% unused)Leading free slots : 0 (0.00 KB = 0.00%)Trailing free slots : 0 (0.00 KB = 0.00%)Number of contiguous groups of 16 slots: 5807 (4.42%)Number of terminal objects: 7969 (0.38%)
  39. 39. What about memory? Before 325.9M After 347.2M 21.3M
  40. 40. Garbage Recon
  41. 41. Garbage Recon AVG GC AVG Object AVG Req Action Rounds Allocations between GC /cart_items/index 0.61 330651 2 /cart_items/ID/ 1.21 651679 1 /cart_items 0.93 498881 1/store/confirm_reservation_address 0.15 76084 6 /store/confirm_reservation 1.37 724095 1
  42. 42. In between request GC User Request Response GC
  43. 43. It’s a trap
  44. 44. The Codeclass Test attr_accessor :i_val def initialize @i_val = [] 300_000.times do |i| @i_val << "ruby-#{i}" * 10 end end def say_hello "hello" endend
  45. 45. The Testtest = Test.newGC.startt.i_val = nilt.say_helloGC.startt = []GC.start
  46. 46. Resultstest = Test.new # 1504032t.i_val = nil # 3922t = [] # 3916
  47. 47. Conclusion• be careful with the big sets of data• don’t store big amount of data in object with long life time• even if current request wont spend much time in GC the next request would do it• the more objects - the longer is mark phase• symbols are not collected but increase mark phase time
  48. 48. Questions?

×