Rails performance: Ruby GC tweaking


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Rails performance: Ruby GC tweaking

  1. 1. Rails performance: Ruby GC tweaking #ParisRB Vlad ZLOTEANU August 2, 2001 @vladzloteanu Software engineer @ DimeloCopyright Dimelo SA www.dimelo.com
  2. 2. What we develop Frontend products -Collaborative platforms, ‘forum-like’, white-labeled, for big accounts (a kind of GetSatisfaction / UserVoice, but.. Better ;) ) Backend product -SocialAPIs client -(a kind of TweetDeck, but for multiple channels, and designed for teams/multiple users per account)Copyright Dimelo SA www.dimelo.com
  3. 3. Stats on Dynamic requests - on HTML/XHTML dynamic requests (we don’t count CSS, JS, images, etc) 20 req/s Web Peaks of 40 req/s 10 req/s APIs Peaks of 25 req/sCopyright Dimelo SA www.dimelo.com
  4. 4. Problem Medium response time ~ 400 ms WTF??Copyright Dimelo SA www.dimelo.com
  5. 5. Ruby perf toolbox Benchmark Server logs Apache Benchmark (ab, ab2) Profile Rack::Bug RailsBench RubyProf RBTrace Memprof Scrap BleakHouse .. Valgrind ltrace Std surveillance Cacti Nagios Surveilance & profiling NewRelic RPM UnionStationCopyright Dimelo SA www.dimelo.com
  6. 6. Finding the problem(s) - DB queries (unoptimized queries, n+1) - External services (EG external RSS feeds) - Lots of partials, low or inexistent cache - Slow session storage - Memory issues - Unoptimized GC - Memory leaksCopyright Dimelo SA www.dimelo.com
  7. 7. Memory / GC issues - How do I know it’s a memory/GC issue? - Server logs - Profilers (memprof) - Monitorization services (newrelic)Copyright Dimelo SA www.dimelo.com
  8. 8. Ruby Garbage Collector - Conservative - Stop-the-world - Mark-and-sweep processCopyright Dimelo SA www.dimelo.com
  9. 9. Ruby default GC settings RUBY_HEAP_MIN_SLOTS=10000 -Initial number of heap slots. It also represents the minimum number of slots, at all times -a new rails app boots up with almost 500k objects on the heap(mostly nodes) RUBY_HEAP_SLOTS_INCREMENT=10000 -The number of new slots to allocate when all initial slots are used RUBY_HEAP_SLOTS_GROWTH_FACTOR=1.8 -Next time Ruby needs new heap slots it will use a multiplicator -New stab is almost double sizedCopyright Dimelo SA www.dimelo.com
  10. 10. Ruby default GC settings (2) RUBY_GC_MALLOC_LIMIT=8000000 -force GC after 8 MB malloc/realloc -High traffic Rails servers can easily allocate more than 8 MB per request RUBY_HEAP_FREE_MIN=4096 -The number of free slots that should be present after GC finishes running. If there are fewer slots than those defined it will allocate new onesCopyright Dimelo SA www.dimelo.com
  11. 11. Find your numbers - Start with enough memory to hold Rails (Ruby’s default is practically nothing) - Increase it linearly if you need more (Ruby’s default is exponential increase) - Only garbage-collect every XX million malloc calls (Ruby’s default is way too small) - Benchmark, benchmark, benchmark..Copyright Dimelo SA www.dimelo.com
  12. 12. GitHub numbers • RUBY_HEAP_MIN_SLOTS=800000 (10000) • RUBY_HEAP_FREE_MIN=100000 (4096) • RUBY_HEAP_SLOTS_INCREMENT=300000(100 00) • RUBY_HEAP_SLOTS_GROWTH_FACTOR=1(1.8) • RUBY_GC_MALLOC_LIMIT=79000000(8000000 )Copyright Dimelo SA www.dimelo.com
  13. 13. Demo (using ApacheBench) - Frontend app, 15 requests, concurrency 1 Before optimisations: Memory usage: 130 MB Execution time(ms): 228 344 322.0 316 4797 After optimisations: Memory usage: 160 MB Execution time(ms): 120 181 190.0 150 2256Copyright Dimelo SA www.dimelo.com
  14. 14. Conclusions - use REE on prod (or Ruby 1.9) - measure, measure, measure (don’t forget to GC.enable_stats) - optimise what matters - And yes, 5 lines of code can make a difference ;)Copyright Dimelo SA www.dimelo.com
  15. 15. Thank you! • Questions?Copyright Dimelo SA www.dimelo.com