Your SlideShare is downloading. ×
How to build a state-of-the-art rails cluster
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

How to build a state-of-the-art rails cluster

14,271
views

Published on

presented at 'ostrava on rails' on 2007-06-22. there exists also a video recording: …

presented at 'ostrava on rails' on 2007-06-22. there exists also a video recording:

http://www.rails.cz/articles/2007/06/27/oor-videos

Published in: Economy & Finance, Technology

6 Comments
17 Likes
Statistics
Notes
No Downloads
Views
Total Views
14,271
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
415
Comments
6
Likes
17
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. How to build a state-of-the-art rails cluster Tim Lossen Infopark AG, Berlin
  • 2. You mean ... like this?
  • 3. System X • Virginia Tech • 1100 x Apple XServe G5 • 2 CPUs each (2,3 GHz) • Infiniband network • 12 Teraflops
  • 4. How to build a state-of-the-art rails cluster for 100.000 Koruny Tim Lossen Infopark AG, Berlin
  • 5. 100.000 Koruny = 4700 $ = 3500 €
  • 6. • Evolving a cluster setup • Scaling • Make or buy? • Discussion
  • 7. Aims • performance • efficiency • availability • scalability
  • 8. Performance • pages / second • throughput • 10 Mbit connection = max. about 1 million unique page impressions per day
  • 9. Efficiency • bang per buck • bang per watt
  • 10. Availability • uptime • can be expressed in ‚nines‘ • 99% (2 nines) = max. 100 minutes downtime per week • 99,9999% (6 nines) = max. 0,6 seconds downtime per week
  • 11. Scalability • grow (or shrink) app as needed • ideal: linear with number of machines
  • 12. Setup 1
  • 13. Example software • nginx • mongrel cluster • mysql
  • 14. Ideas • put everything on one machine • use top quality hardware • go for high processor speed
  • 15. Example hardware • Sun Fire X2200 M2 • 2 x Opteron 2220 (dual core / 2,8 GHz) • 4 GB ram • 250 GB disk • 3290 €
  • 16. Result • performance ✔ • efficiency • availability • scalability
  • 17. Setup 2
  • 18. Ideas • put app and database on separate machines • use cheap commodity hardware • like Google! • go for high number of CPUs
  • 19. Example hardware • Dell PowerEdge SC 1435 • 2 x Opteron 2212 (dual core / 2 GHz) • 4 GB ram, 160 GB disk • 2 machines = 8 cores • 3340 €
  • 20. Result • performance ✔ • efficiency ✔ • availability • scalability
  • 21. Setup 3
  • 22. Ideas • cluster of identical nodes • loadbalancing • failover • eliminate single points of failure through redundancy
  • 23. Setup 3a
  • 24. Virtualization • partition one physical machine into multiple virtual machines • assign dedicated ressources (cpu, ram) to virtual machines • example: xen (linux), zones (solaris)
  • 25. Why virtualization? • flexibility • simplified deployment • move images between machines • optimization • hardware utilization • tuning
  • 26. Why virtualization? • isolation • use different operating systems • contain intrusion • multiple environments (staging, production) • but: small performance overhead (ca. 5%)
  • 27. Setup 3a
  • 28. Loadbalancer • openBSD • CARP: shared virtual ip, automatic failover • pf: firewall + loadbalancer • openBSD on xen: currently only on ramdisk • better use dedicated hardware instead
  • 29. Redundant firewall
  • 30. Database • mysql • master-master replication • caveat: avoid primary key clashes! • other possibilities: • master-slave replication • mysql cluster
  • 31. Result • performance ✔ • efficiency ✔ • availability ✔ • scalability ?
  • 32. • Evolving a cluster setup • Scaling • Make or buy? • Discussion
  • 33. Scaling • not a rails-specific problem • find and eliminate bottlenecks!
  • 34. Scaling approaches • scaling up • increase power of machines • scaling out • increase number of machines
  • 35. Scaling up: memory • add more memory • mongrel can be very memory-hungry • useful for caching (memcached) • hint: check crucial.com for low prices
  • 36. Scaling up: CPU • upgrade to quad-core Opterons • available later this year • same socket • same power / heat envelope • use as drop-in replacement
  • 37. Scaling out
  • 38. Scaling out • app servers scale (almost) linearly • use same hardware (if possible) • one type of processor, memory, disk • simplifies maintenance / repair
  • 39. Bottleneck: application • lots of concurrent users • rails is single threaded (mutex lock) • solution: add more mongrels • rule of thumb: 10 mongrels per cpu • twitter runs on ca. 300 mongrels
  • 40. Bottleneck: database • avoid to hit database • cache as much as possible • scale up database masters • more disks = faster access • add read-only db slaves
  • 41. Scaling 2.0
  • 42. Scaling storage • disk failure very common • use RAID1 for database masters • network storage • example: ATA-over-Ethernet (AoE) • accessible as block device • combine with cluster file system
  • 43. Coraid EtherDrive, 11 TB, 200MB/s
  • 44. • Evolving a cluster setup • Scaling • Make or buy? • Discussion
  • 45. Make or buy? • think about total cost of ownership (TCO) • not only hardware costs • rackspace, power, bandwidth • administration overhead • „Utility? Or generator in the back yard?“ (David Young)
  • 46. Our cluster • hardware: 3340 € • colocation with burst.net (for 1 year) • 60 $ + 12 x 140 $ = 900 $ = 670 € • 4000 € + administration
  • 47. Rails hosting • example: Engine Yard • 1 slice = 1 CPU core, 400 MB ram • 4 slices (for 1 year) • 796 $ + 12 x 1116 $ = 14.200 $ • 10.500 €
  • 48. • Evolving a cluster setup • Scaling • Make or buy? • Discussion
  • 49. Summary • 2-machine cluster is cool (and quite affordable) setup • virtualization is a Good Thing • scaling the database can be tricky • make or buy depends on concrete situation
  • 50. Questions?
  • 51. http://tim.lossen.de ❦ photo credits http://www.flickr.com/photos/shivayanamahohm/192322513 http://www.flickr.com/photos/nicholas_t/286352669 http://www.flickr.com/photos/redjar/480709870