Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Parallel Ruby: Managing the Memory Monster

141 views

Published on

From RubyConf 2019. How to run parallel ruby with significantly reduced memory usage

Published in: Software
  • Be the first to comment

  • Be the first to like this

Parallel Ruby: Managing the Memory Monster

  1. 1. Parallel Ruby: Managing the Memory Monster Kevin Miller kevin.miller@flexport.com
  2. 2. We run a lot of background jobs
  3. 3. The Great Migration Old ● 240 Ruby Processes ● 20 Servers ● 12 Processes per Server ● 1 Thread per Process
  4. 4. The Great Migration Old ● 240 Ruby Processes ● 20 Servers ● 12 Processes per Server ● 1 Thread per Process New ● 400 Ruby Threads ● 10 Servers ● 4 Processes per Server ● 10 Threads per Processes
  5. 5. Things started getting… slow
  6. 6. Side Lesson If you deploy every 10 minutes, you can get away with a lot
  7. 7. And now back to business
  8. 8. Ruby Heap: 40 byte slots
  9. 9. Ruby Heap: 40 byte slots “String”
  10. 10. Ruby Heap: 40 byte slots “String” Class.new
  11. 11. Ruby Heap: 40 byte slots “String” Class.new {a: 2}
  12. 12. Ruby Heap: 40 byte slots “String” Class.new {a: 2} [1]
  13. 13. Ruby Heap: 40 byte slots “String” 12132425 Class.new {a: 2} [1]
  14. 14. Ruby Heap: 40 byte slots “String” 12132425 Class.new 121325.224 {a: 2} [1]
  15. 15. Ruby Heap: 40 byte slots “String” 12132425 Class.new 121325.224 {a: 2} /regular?/ [1]
  16. 16. Ruby Heap: 40 byte slots “String” 12132425 Class.new 121325.224 {a: 2} /regular?/ [1] ...
  17. 17. “String” 12132425 Class.new 121325.224 {a: 2} /regular?/ [1] ...
  18. 18. “String” 12132425 [“aaaaaaaaa... Class.new 121325.224 {a: 2} /regular?/ [1] ...
  19. 19. “String” 12132425 [“aaaaaaaaa... Class.new 121325.224 {a: 2} /regular?/ [1] ...
  20. 20. C (OS) Heap Pages: 4KB of space per page Big ArrayBig String
  21. 21. “String” 12132425 ArraySlot Class.new 121325.224 {a: 2} /regular?/ [1] ... Big Array Big String [String, String...]
  22. 22. “String” 12132425 ArraySlot “aaaa…” Class.new 121325.224 “aaaa…” “aaaa…” {a: 2} /regular?/ “aaaa…” “aaaa…” [1] ... “aaaa…” “aaaa…” Big Array Big String [String, String...]
  23. 23. “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “String” 12132425 ArraySlot “aaaa…” Class.new 121325.224 “aaaa…” “aaaa…” {a: 2} /regular?/ “aaaa…” “aaaa…” [1] ... “aaaa…” “aaaa…” Big Array Big String [String, String...]
  24. 24. “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “String” 12132425 ArraySlot StringSlot Class.new 121325.224 StringSlot StringSlot {a: 2} /regular?/ StringSlot StringSlot [1] ... StringSlot StringSlot Big Array Big String [String, String...]
  25. 25. Big Array Big String [String, String...] “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “String” 12132425 ArraySlot StringSlot Class.new 121325.224 StringSlot StringSlot {a: 2} /regular?/ StringSlot StringSlot [1] ... StringSlot StringSlot Big Array Big String [String, String...]
  26. 26. Total Slots = 1 for Array + 1024 for the 1024 strings = 1025 Total Ruby Heap Size = 1025 Slots * 40 bytes per slot = ~40KB = ~ .04 MB Total OS Heap Size = 4 bytes per array elem * 1024 array elems + 1024 bytes per string * 1024 strings = = 1028KB = ~1MB
  27. 27. We expect memory to be around here
  28. 28. Big Array Big String [String, String...] “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “String” 12132425 ArraySlot StringSlot Class.new 121325.224 StringSlot StringSlot {a: 2} /regular?/ StringSlot StringSlot [1] ... StringSlot StringSlot Big Array Big String [String, String...]
  29. 29. Big Array Big String [String, String...] “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “aaaa…” “String” 12132425 ArraySlot StringSlot Class.new 121325.224 StringSlot StringSlot {a: 2} /regular?/ StringSlot StringSlot [1] ... StringSlot StringSlot Big Array Big String Wasted Space
  30. 30. Ruby Threads Are Not Memory Friendly
  31. 31. Thread 1 Fragmented OS Heap 1
  32. 32. Thread 1 Fragmented OS Heap 1 Thread 2
  33. 33. Thread 1 Fragmented OS Heap 1 Thread 2
  34. 34. C Default: One OS Heap per Thread Thread 1 Thread 2 Fragmented OS Heap 1 Fragmented OS Heap 2
  35. 35. C Default: One OS Heap per Thread Thread 1 Thread 2 Thread 20 ... Fragmented OS Heap 1 Fragmented OS Heap 2 Fragmented OS Heap 20
  36. 36. Option 1: Reduce Number of OS Heaps Number of OS Heaps (a.k.a Arenas) can be configured with MALLOC_ARENA_MAX
  37. 37. We expect memory to be around here
  38. 38. Option 2: Use a Different Memory Allocator Can switch to jemalloc* over standard glibc malloc with LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1 * After installing it via homebrew/apt-get/yum/etc
  39. 39. We expect memory to be around here
  40. 40. So… What should you do? Default Reduced Arena Count jemalloc Pros Out of the Box Works with all ruby Significant memory reduction Best Memory Usage Cons Heavy Fragmentation Increases contention in ruby implementations with true threading (JRuby) Requires reinstalling ruby and installing jemalloc Use Cases Single-threaded Ruby Programs on OS’s you don’t control Can set environment variables but cannot install packages Anytime you’re able to
  41. 41. So… What should you do? 1) If you control the environment, jemalloc is king 2) If using MRI/CRuby, always set MALLOC_ARENA_MAX to 2
  42. 42. So… What should you do? 1) If you control the environment, jemalloc is king 2) If using MRI/CRuby, always set MALLOC_ARENA_MAX to 2 “I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is terrible with memory, a huge memory hog with their Rails apps. My users switch to jemalloc and a miracle occurs: their memory usage drops massively” Taken from https://bugs.ruby-lang.org/issues/14718: proposal to make jemalloc part of standard Ruby
  43. 43. So… What should you do? 1) If you control the environment, jemalloc is king 2) If using MRI/CRuby, always set MALLOC_ARENA_MAX to 2 “I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is terrible with memory, a huge memory hog with their Rails apps. My users switch to jemalloc and a miracle occurs: their memory usage drops massively” “On Ubuntu Linux, with Rails Ruby Bench, jemalloc gives an overall speedup (not just memory, end-to-end speedup) of around 11%” Taken from https://bugs.ruby-lang.org/issues/14718: proposal to make jemalloc part of standard Ruby
  44. 44. So… What should you do? 1) If you control the environment, jemalloc is king 2) If using MRI/CRuby, always set MALLOC_ARENA_MAX to 2 Taken from https://bugs.ruby-lang.org/issues/14718: proposal to make jemalloc part of standard Ruby “I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is terrible with memory, a huge memory hog with their Rails apps. My users switch to jemalloc and a miracle occurs: their memory usage drops massively” “On Ubuntu Linux, with Rails Ruby Bench, jemalloc gives an overall speedup (not just memory, end-to-end speedup) of around 11%” “I'm another Ruby user that used to have memory bloat problems and switched to jemalloc as well”
  45. 45. Want to Learn More? Go to Aaron Patterson’s Talk: “Compacting Heaps in Ruby 2.7”

×