Ruby Proxies for Scale, Performance and Monitoring Ilya Grigorik CTO / AideRSS Event-Driven Architectures @igrigorik #meshu #eda http://bit.ly/eda

postrank.com/topic/ruby The slides… Twitter My blog

Proxy Love

Proxy Love

Myth: Slow Frameworks “ Rails, Django, Seaside, Scala…” cant scale.

Myth: Slow Frameworks

The Proxy Solution

The Proxy Solution

The “More” Proxy Solution

The “More” Proxy Solution

Transparent, Cut-Through Proxy Transparent

Transparent, Cut-Through Proxy

Transparent Proxy = Scalability Power Tool 90% use case

Transparent Proxy = Scalability Power Tool

Proxies for Monitoring, Performance and Scale welcome to the wonderful world of intercepting, caching, … , proxy servers

Problem: Staging Environment Production Simulated User

“ Representative Load / Staging” Duplication Simulating traffic?

Staging fail. Profile of queries has changed Fail Load on production has changed Fail Parallel environment Fail Slower release cycle Fail

Replay log data, rinse, repeat github.com/igrigorik/autoperf

Benchmarking Proxy flash of the obvious Real (production) traffic Duplex Ruby Proxy, FTW!

Benchmarking Proxy

flash of the obvious

Transparent Duplex Proxy, FTW

Same response, different turnaround time Different response body!

github.com/igrigorik/em-proxy Duplex Ruby Proxy, FTW!

github.com/igrigorik/em-proxy

EM-Proxy Workflow 1. Accept connection - Connect & Forward to Production - Connect & Forward to Staging 2. Stream response to client from Production - Buffer response from Staging 3. Run “Processor / Analyzer” once both return

EM-Proxy Workflow

Ruby EventMachine crash course EM::Connection #receive_data - new data ready for processing #unbind - connection terminated #connection_completed - connected

Duplex incoming data

Forward Production Response Analyze Results

Open connection to application server

Buffer until connected

Pass response to Duplex Notify when done

Analyze Time, Response…

Analyze Time, Response… Response Time

Cute, right? but so what…

%w[ <Transparent> Intercepting Caching … ] solution for every problem

Validating Proxy easy, real-time diagnostics Woops!

Who said you’re not allowed to touch the data? Intercepting Proxy for those hard-to-reach spots

SMTP / Mail: inject SPAM detection De(encrypt): stream data MySQL: inject sharding logic Intercepting Proxy for those hard-to-reach spots

@PostRank: Beanstalkd + Ruby Proxy because RAM is still expensive

Beanstalkd Math ~ 93 Bytes of overhead per job ~300 Bytes of data / job x 80,000,000 jobs in memory ~ 30 GB of RAM = 2 X-Large EC2 instances Oi, expensive!

Beanstalkd Math

Extending Beanstalkd Observations: 1. Each job is rescheduled several times 2. > 95% are scheduled for > 3 hours into the future 3. Beanstalkd does not have overflow page-to-disk Memory is wasted… We’ll add it ourselves!

Extending Beanstalkd

@PostRank: “Chronos Scheduler” 1 “Medium” EC2 Instance Intercepting Proxy

Extend the protocol Chronos Scheduler Beanstalkd persistence Process incoming data Buffer “SCHEDULE” requests, otherwise forward

Archive to MySQL Chronos Scheduler Beanstalkd persistence Buffer “SCHEDULE”

@PostRank: “Chronos Scheduler” Extend the protocol ARCHIVE RESERVE, … 10 line patch to client: queue.archive(job)

@PostRank: “Chronos Scheduler” ~79,000,000 jobs, 4GB RAM ARCHIVE RESERVE, … Upcoming jobs: ~ 1M 400% cheaper + extensible! 1 Proxy, 1 Beanstalkd, 1 MySQL = 200 req/s

%w[ Transparent <Intercepting> Auth Cache … ] easy to make & a powerful hammer

Cheers. Questions? The slides… Twitter My blog