Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com

Ruby Proxies for Scale, Performance and Monitoring Ilya Grigorik CTO / AideRSS Event-Driven Architectures @igrigorik #meshu #eda http://bit.ly/eda

postrank.com/topic/ruby The slides… Twitter My blog

Proxy Love

Myth: Slow Frameworks
“ Rails, Django, Seaside, Scala…” cant scale.

The Proxy Solution

The “More” Proxy Solution

Transparent, Cut-Through Proxy
Transparent

Transparent Proxy = Scalability Power Tool
90% use case

Proxies for Monitoring, Performance and Scale welcome to the wonderful world of intercepting, caching, … , proxy servers

Problem: Staging Environment Production Simulated User

“ Representative Load / Staging” Duplication Simulating traffic?

Staging fail. Profile of queries has changed Fail Load on production has changed Fail Parallel environment Fail Slower release cycle Fail

Replay log data, rinse, repeat github.com/igrigorik/autoperf

Benchmarking Proxy
flash of the obvious
Real (production) traffic Duplex Ruby Proxy, FTW!

Transparent Duplex Proxy, FTW

Same response, different turnaround time Different response body!

github.com/igrigorik/em-proxy
Duplex Ruby Proxy, FTW!

EM-Proxy Workflow
1. Accept connection - Connect & Forward to Production - Connect & Forward to Staging 2. Stream response to client from Production - Buffer response from Staging 3. Run “Processor / Analyzer” once both return

Ruby EventMachine crash course EM::Connection #receive_data - new data ready for processing #unbind - connection terminated #connection_completed - connected

Duplex incoming data

Forward Production Response Analyze Results

Open connection to application server

Buffer until connected

Pass response to Duplex Notify when done

Analyze Time, Response…

Analyze Time, Response… Response Time

Cute, right? but so what…

%w[ <Transparent> Intercepting Caching … ] solution for every problem

Validating Proxy easy, real-time diagnostics Woops!

Who said you’re not allowed to touch the data? Intercepting Proxy for those hard-to-reach spots

SMTP / Mail: inject SPAM detection De(encrypt): stream data MySQL: inject sharding logic Intercepting Proxy for those hard-to-reach spots

@PostRank: Beanstalkd + Ruby Proxy because RAM is still expensive

Beanstalkd Math
~ 93 Bytes of overhead per job ~300 Bytes of data / job x 80,000,000 jobs in memory ~ 30 GB of RAM = 2 X-Large EC2 instances Oi, expensive!

Extending Beanstalkd
Observations: 1. Each job is rescheduled several times 2. > 95% are scheduled for > 3 hours into the future 3. Beanstalkd does not have overflow page-to-disk Memory is wasted… We’ll add it ourselves!

@PostRank: “Chronos Scheduler” 1 “Medium” EC2 Instance Intercepting Proxy

Extend the protocol Chronos Scheduler Beanstalkd persistence Process incoming data Buffer “SCHEDULE” requests, otherwise forward

Archive to MySQL Chronos Scheduler Beanstalkd persistence Buffer “SCHEDULE”

@PostRank: “Chronos Scheduler” Extend the protocol ARCHIVE RESERVE, … 10 line patch to client: queue.archive(job)

@PostRank: “Chronos Scheduler” ~79,000,000 jobs, 4GB RAM ARCHIVE RESERVE, … Upcoming jobs: ~ 1M 400% cheaper + extensible! 1 Proxy, 1 Beanstalkd, 1 MySQL = 200 req/s

%w[ Transparent <Intercepting> Auth Cache … ] easy to make & a powerful hammer

Cheers. Questions? The slides… Twitter My blog

http://www.igvita.com	5991
http://www.slideshare.net	38
http://www.ultrasaurus.com	27
http://coderwall.com	8
http://www.railsfire.com	4
http://localhost	3
http://www.linkedin.com	3
http://www.zhuaxia.com	2
http://sexydev.soup.io	2
http://www.hanrss.com	2
http://www.protopage.com	1
https://webcache.googleusercontent.com	1
http://xss.yandex.net	1
http://127.0.0.1:8795	1
http://www.artisopensource.net	1
http://feeds.igvita.com	1
http://feeds.feedburner.com	1

Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com

by Ilya Grigorik on Apr 17, 2009

Accessibility

Categories

Upload Details

Usage Rights

17 Embeds 6,087

Statistics

Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com Presentation Transcript