Download to read offline
Uploaded byAmoniac OÜ
Distributed Cluster in Ruby
This document discusses distributed clustering in Ruby. It begins with background on the presenter and then describes typical solutions using multiple processing boxes to scale horizontally. However, this leads to underutilization of resources when load is not evenly distributed. The document proposes a new approach called di-cluster that scales different parts of an application independently based on actual load. It discusses challenges with existing concurrency approaches in Ruby and introduces an early prototype of di-cluster that requires further development and testing.
Distributed Cluster in Ruby
- 1.
- 2. ABOUT ME • MikhailBortnyk • Ruby developer at Amoniac OÜ • Co-founder of Kottans • Github: @vessi • Twitter: @mikhailbortnyk • 2 cats
- 3.
- 4. BACKGROUND STORY 0 25 50 75 100 125 0 7.515 22.5 30 Loadpercentage Amount of requests Application load
- 5.
- 6. PROBLEMS OF TYPICAL SOLUTION •you are now available to process 2 times (less in reality) of your typical load • for 102% load 98% is not utilised • and you scale all parts even ones that don’t need to be scaled • spending money, alcoholism, drugs, depression, life is over
- 7. LOOK OUT OFIN THE BOX Special processing box Your data Something useful in reality is: Your data request processo r databas e request business logic respons e processo r Somethin g useful
- 8. SO WHAT? • inreality, your 100% of processing power is: • 5% of request processor • 20% of database requests • 60% of business logic • 10% of serialisation • 5% of response processor • Load differs from stage to stage.
- 9. LOOK LIKE ISEE THE IDEA! • What if we will scale only most loaded part of application? • Typical case: request processor + database + business logic + serialiser + response * 2 = 200% • This case: request processor + database + business logic * 2 + serialiser + response = 160% • For 102% of load only 58% is not utilised
- 10. OK, OK, IJUST STOLE IDEA FROM ERLANG
- 11. SIDE STORY OF CONCURRENCYIN RUBY • Threads • DRuby • Celluloid • concurrent-ruby • “Guild” model (planned for Ruby 3, author: Koichi Sasada)
- 12. THREADS CAVEATS • Somepeople, when confronted with a problem, think, "I know, I'll use threads," and then two they hav erpoblesms. • All multithreading low-level stuff • Aware thread-safety! • Ruby VM is shared/copied to child process
- 13. LONG STORY SHORT •DRuby is too low-level • Have you seen anyone who uses this in reality? • concurrent-ruby does not have network solutions • Guild is not out yet
- 14.
- 15. OK, WHAT’S WRONGWITH THIS? • you don’t know which node runs what • Celluloid. NUFF SAID • No, seriously. Thread per actor is too expensive • But they gave the idea
- 16.
- 17. I AM PROUDTO PRESENT
- 18. TOTALLY NOT FINISHED CONCEPT •di-cluster gem • github://vessi/di-cluster.git • lot of things to complete • clean code • write tests • write documentation
- 19. DICLUSTER • NIH syndrome •role-based registry • based on concurrent-ruby • buggy and unfinished • only JSON-compatible data
- 20. DEMO I will needcourageous person from listeners
- 21.
- 22.
- 23. SO LONG ANDTHANKS FOR ALL THE LINKS