Scaling application
servers for efficiency
  Tomas Doran <bobtfish@bobtfish.net>
Catalyst web framework core team member
    ...
mod_$lang considered
        harmful

• Web servers send bytes
• Application servers generate pages
• These two goals are ...
EPIC FAIL
• 80 Mb mod_perl processes
• Serving static media
• Reading stuff off disk/network in a while
  loop
• Sending i...
Pushing bytes - how to fail.

• Serve static content from the same servers
  running your application (mod_perl,
  mod_pyt...
Pushing bytes - Success


• X-Sendfile (lighty and mod_sendfile)
• X-Accel-Redirect (nginx)
App server maps media

• Check ACLs / Resolve one-time URI
• Locate media
• Serve X-Accel-Redirect
• Aquire PIE, CAKE and ...
My setup

•   App runs FCGI

•   Run nCPUs x 1.2 procs (measure this!)

•   Looks up asset mapping (memcache)
            ...
•   X-Accel-Redirect

•   Bytes sent by nginx

• Serving mp3s - filesize 1-7Mb (ish).
• > 30k sessions
• > 200 reqs/s
• Fil...
Several 100 Tb of media online

      Technology stack:

           nginx
            perl
           FCGI
          Catal...
Even if extra context switching has zero overhead
       you serve people sooner if you queue.

    A     B       A   B   ...
RUN MOST EFFICIENT NUMBER OF
          WORKERS

 QUEUE REQUESTS AFTER THAT

PROFILE PROFILE PROFILE
• App is notwork bound after tuning
• Best efficiency ~ n CPUs x 1.2 (for me!)
• CONTEXT SWITCHING HURTS
Thanks

• Questions?
• t0m <bobtfish@bobtfish.net>
• http://catalystframework.org
• http://search.cpan.org/author/BOBTFISH
•...
Upcoming SlideShare
Loading in …5
×

Scaling application servers for efficiency

1,895 views

Published on

Slides for the talk/ discussion I gave at ScaleCamp uk 2009, and then repeated the day afterwards at London perl workshop.

This presentation covers the key points about serving media files efficiently with 100s or 1000s of concurrent streams, still using a high level web framework in combination with X-Accel-Redirect.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,895
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
25
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Scaling application servers for efficiency

    1. 1. Scaling application servers for efficiency Tomas Doran <bobtfish@bobtfish.net> Catalyst web framework core team member Serving lots of media for a living ScaleCamp London 09
    2. 2. mod_$lang considered harmful • Web servers send bytes • Application servers generate pages • These two goals are orthogonal
    3. 3. EPIC FAIL • 80 Mb mod_perl processes • Serving static media • Reading stuff off disk/network in a while loop • Sending it to people on Virgin, using bittorrent, ‘3g’, via a damp piece of string. • With MaxClients x proc size being waaaay higher than physical memory
    4. 4. Pushing bytes - how to fail. • Serve static content from the same servers running your application (mod_perl, mod_python epic fail, mod_php moderate fail) • Static content AT ALL. I don’t care if you need to check ACLs
    5. 5. Pushing bytes - Success • X-Sendfile (lighty and mod_sendfile) • X-Accel-Redirect (nginx)
    6. 6. App server maps media • Check ACLs / Resolve one-time URI • Locate media • Serve X-Accel-Redirect • Aquire PIE, CAKE and PONY, profit.
    7. 7. My setup • App runs FCGI • Run nCPUs x 1.2 procs (measure this!) • Looks up asset mapping (memcache) (Tm) • UPDATE download_attempts + 1 in MyFirstSQL • X-Accel-Redirect
    8. 8. • X-Accel-Redirect • Bytes sent by nginx • Serving mp3s - filesize 1-7Mb (ish). • > 30k sessions • > 200 reqs/s • Filling 1Gb of pipe • 1 box.
    9. 9. Several 100 Tb of media online Technology stack: nginx perl FCGI Catalyst MyFirstSQL memcache X-Accel-Redirect nginx-mogilefs-module MogileFS lighty
    10. 10. Even if extra context switching has zero overhead you serve people sooner if you queue. A B A B A B A B A B A finishes significantly before B in the lower diagram B finishes at the same time in both
    11. 11. RUN MOST EFFICIENT NUMBER OF WORKERS QUEUE REQUESTS AFTER THAT PROFILE PROFILE PROFILE
    12. 12. • App is notwork bound after tuning • Best efficiency ~ n CPUs x 1.2 (for me!) • CONTEXT SWITCHING HURTS
    13. 13. Thanks • Questions? • t0m <bobtfish@bobtfish.net> • http://catalystframework.org • http://search.cpan.org/author/BOBTFISH • http://github.com/bobtfish

    ×