Scaling application servers for efficiency

•Download as KEY, PDF•

3 likes•986 views

Slides for the talk/ discussion I gave at ScaleCamp uk 2009, and then repeated the day afterwards at London perl workshop. This presentation covers the key points about serving media files efficiently with 100s or 1000s of concurrent streams, still using a high level web framework in combination with X-Accel-Redirect.

Technology

Scaling application
servers for efﬁciency
Tomas Doran <bobtﬁsh@bobtﬁsh.net>
Catalyst web framework core team member
Serving lots of media for a living

ScaleCamp London 09

mod_$lang considered
harmful

• Web servers send bytes
• Application servers generate pages
• These two goals are orthogonal

EPIC FAIL
• 80 Mb mod_perl processes
• Serving static media
• Reading stuff off disk/network in a while
loop
• Sending it to people on Virgin, using
bittorrent, ‘3g’, via a damp piece of string.
• With MaxClients x proc size being waaaay
higher than physical memory

Pushing bytes - how to fail.

• Serve static content from the same servers
running your application (mod_perl,
mod_python epic fail, mod_php moderate
fail)
• Static content AT ALL. I don’t care if you
need to check ACLs

Pushing bytes - Success

• X-Sendﬁle (lighty and mod_sendﬁle)
• X-Accel-Redirect (nginx)

App server maps media

• Check ACLs / Resolve one-time URI
• Locate media
• Serve X-Accel-Redirect
• Aquire PIE, CAKE and PONY, proﬁt.

My setup

• App runs FCGI

• Run nCPUs x 1.2 procs (measure this!)

• Looks up asset mapping (memcache)
(Tm)

• UPDATE download_attempts + 1 in MyFirstSQL

• X-Accel-Redirect

• X-Accel-Redirect

• Bytes sent by nginx

• Serving mp3s - ﬁlesize 1-7Mb (ish).
• > 30k sessions
• > 200 reqs/s
• Filling 1Gb of pipe
• 1 box.

Several 100 Tb of media online

Technology stack:

nginx
perl
FCGI
Catalyst
MyFirstSQL
memcache
X-Accel-Redirect
nginx-mogilefs-module
MogileFS
lighty

Even if extra context switching has zero overhead
you serve people sooner if you queue.

A B A B A B A B

A B

A ﬁnishes signiﬁcantly before B in the lower diagram

B ﬁnishes at the same time in both

• App is notwork bound after tuning
• Best efﬁciency ~ n CPUs x 1.2 (for me!)
• CONTEXT SWITCHING HURTS

Thanks

• Questions?
• t0m <bobtﬁsh@bobtﬁsh.net>
• http://catalystframework.org
• http://search.cpan.org/author/BOBTFISH
• http://github.com/bobtﬁsh

RedHat built a distributed object storage solution named Ceph which first debuted ten years ago. Now we are seeing rapid developments in the industry and we want to take advantage of them. In this talk, we will briefly introduce Ceph, revisit the problems we are seeing when profiling its I/O performance with flash device, and explain why we want to embrace the future by switching to Seastar. We’ll share our experiences with the audience of how and when we are porting our software to this framework.

Gobblin on-aws

Vasanth Rajamani

Crash reports pycodeconflauraxthomson

Kafka Summit SF 2017 - Shopify Flash Sales with Apache Kafka

confluent

High Concurrency Architecture and Laravel Performance Tuning

Albert Chen

"Have you ever crossed your fingers before performing an upgrade or switching storage engines, because you weren't quite sure what would happen? Have you ever been bitten by a slight change in behavior that turned out to be unexpectedly significant for your workload? At Parse we have developed a workflow that lets us repeatedly capture and replay real production workloads offline. This has allowed us to confidently perform upgrades across a large fleet with a minimum amount of canarying, and has helped us load test a variety of storage engines with real workloads so we can compare and understand the performance tradeoffs. In this talk we will cover best practices for upgrades and migrations, and we will walk through how to use our open-sourced tooling to demonstrate how you can do the same. We will also share some fun war stories about various disasters found and averted *before* putting them into production thanks to offline benchmarking."

Infrastructure as Code with Terraform and Ansible

DevOps Meetup Bern

CRX2Oak - all the secrets of repository migration

Tomasz Rękawek

NRD: Nagios Result Distributor

Jose Luis Martínez

Developing Scylla Applications: Practical Tips

ScyllaDB

Pulsarctl & Pulsar Manager

StreamNative

Snabb Switch: Riding the HPC wave to simpler, better network appliances (FOSD...

Igalia

By Katerina Barone-Adesi. Driven by the needs of scientific computing, rapid rises in memory bandwidth have made it possible to implement high-performance network functions in a radically simpler way. Snabb Switch rides this wave, bypassing the kernel to process network packets in terse Lua, leaving the programmer free to focus on the essence of their problem. This talk presents our experiences delivering a carrier-grade implementation of "lightweight 4 over 6", an IPv4-as-a-service architecture that tunnels access to the IPv4 internet through specialized Snabb appliances. We report on our recent experience implementing a carrier-grade virtualized network function, with observations on what it is like to build real-world, high-performance Snabb applications. (and kernel bypass). Each instance runs at essentially line speed on two ten-gigabit Ethernet cards. Lightweight 4-over-6 (lw4o6) defines an IPv4-as-a-service architecture that allows ISPs to internally operate an IPv6-only network, tunneling IPv4 connections between lw4o6-aware endpoints installed at the customer's site (e.g. in OpenWRT) and an internet-facing "lwAFTR". Lw4o6 was specified in 2015 as RFC 7596 and has the architectural advantage that the carrier-side lwAFTR only needs per-customer state, not per-flow state. An lw4o6 system can also be configured to share IPv4 addresses between multiple customers as part of an IPv4 exhaustion strategy. It allows IPv4 networks to interoperate smoothly, while a carrier between them runs a pure-IPv6 network. Igalia has built an open source "lwAFTR" implementation that is ready to deploy in production. We describe the joys of hacking with Snabb, giving a quick intro to Snabb, modern x86, and lw4o6 along the way. (c) 2016 FOSDEM VZW CC BY 2.0 BE https://archive.fosdem.org/2016/

PROCESS WARP

祐司伊藤

Camel Desing Patterns Learned Through Blood, Sweat, and Tears

Bilgin Ibryam

Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013

Puppet

"Running at Scale: Practical Performance Tuning with Puppet" by Sam Kottler Engineer, Red Hat. Presentation Overview: This session will talk about some production issues I've seen running Puppet in large environments. From how to manage a single master with hundreds of hosts to real-life patterns for building high availability clusters that scale to 10's of thousands of agents. Another important topic that will be covered is how to deploy networked filesystems that perform well under high load and streaming files to many hosts simultaneously. Speaker Bio: Sam Kottler is a software engineer in the Virtualization R&D group at Red Hat. He's helped build infrastructure for leading startups, including Digg.com, Acquia, and Venmo and is a contributor to Puppet, the Fedora Project, Drupal, and the Rubygems.org. Sam speaks around the world on the topics of internet security, systems automation, and software architecture.

Riga Dev Day 2016 - Microservices with Apache Camel & fabric8 on Kubernetes

Claus Ibsen

The so-called experts are saying microservices and containers will change the way we build, maintain, operate, and integrate applications. This talk is intended for Java developers who wants to hear and see how you can develop Java microservices that runs in containers. This talk uses Apache Camel as the Java library to build microservice architectured applications. At first we introduce you to Apache Camel and show how you can easily get started with Camel on your computer, and build a microservice application that runs on CDI and Spring-Boot. The second part of this talk is about running Camel (or any Java project) on Docker and Kubernetes. We start covering the basic concepts you as a Java developer must understand about Kubernetes. Then we show how to migrate Java projects to build as Docker images and deployable on Kubernetes, with help from fabric8 Maven tooling. You will also hear about how to make your microservices scalable and distributed by leveraging the facilities that Kubernetes provides for truly distributed services with load balancing and location independence. You will also see how to manage your container using the Kubernetes CLI and the fabric8 web console. At the end we have a bit of fun with scaling up and down your Camel application to see how resilient the application is, when we kill containers. This talk is a 50/50 mix between slides and demo.

Scale your Alfresco Solutions

Alfresco Software

John adams talk cloudy

John Adams

What's hot

3.2 Streaming and Messaging

振东刘

Nashorn: JavaScript that doesn't suck - Tomer Gabel, Wix

Codemotion Tel Aviv

HBaseCon2017 gohbase: Pure Go HBase Client

HBaseCon

Cloud Foundry on OpenStack - An Experience Report | anynines anynines GmbH

Cloud Infrastructures Slide Set 8 - More Cloud Technologies - Mesos, Spark | ...

anynines GmbH

Apache Bookkeeper and Apache Zookeeper for Apache Pulsar

Enrico Olivelli

Apache Zeppelin on Kubernetes with Spark and Kafka - meetup @twitter

Apache Zeppelin

Mad scalability: Scaling when you are not Google

Abel Muíño

Performance Benchmarking: Tips, Tricks, and Lessons Learned

Tim Callaghan

Benchmarking, Load Testing, and Preventing Terrible Disasters

MongoDB

Infrastructure as Code with Terraform and Ansible

DevOps Meetup Bern

CRX2Oak - all the secrets of repository migration

Tomasz Rękawek

NRD: Nagios Result Distributor

Jose Luis Martínez

Developing Scylla Applications: Practical Tips

ScyllaDB

Pulsarctl & Pulsar Manager

StreamNative

Snabb Switch: Riding the HPC wave to simpler, better network appliances (FOSD...

Igalia

PROCESS WARP

祐司伊藤

Camel Desing Patterns Learned Through Blood, Sweat, and Tears

Bilgin Ibryam

Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013

Puppet

Riga Dev Day 2016 - Microservices with Apache Camel & fabric8 on Kubernetes

Claus Ibsen

What's hot (20)

3.2 Streaming and Messaging

Nashorn: JavaScript that doesn't suck - Tomer Gabel, Wix

HBaseCon2017 gohbase: Pure Go HBase Client

Cloud Foundry on OpenStack - An Experience Report | anynines

Cloud Infrastructures Slide Set 8 - More Cloud Technologies - Mesos, Spark | ...

Apache Bookkeeper and Apache Zookeeper for Apache Pulsar

Apache Zeppelin on Kubernetes with Spark and Kafka - meetup @twitter

Mad scalability: Scaling when you are not Google

Performance Benchmarking: Tips, Tricks, and Lessons Learned

Benchmarking, Load Testing, and Preventing Terrible Disasters

Infrastructure as Code with Terraform and Ansible

CRX2Oak - all the secrets of repository migration

NRD: Nagios Result Distributor

Developing Scylla Applications: Practical Tips

Pulsarctl & Pulsar Manager

Snabb Switch: Riding the HPC wave to simpler, better network appliances (FOSD...

PROCESS WARP

Camel Desing Patterns Learned Through Blood, Sweat, and Tears

Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013

Riga Dev Day 2016 - Microservices with Apache Camel & fabric8 on Kubernetes

Similar to Scaling application servers for efficiency

Scale your Alfresco Solutions

Alfresco Software

John adams talk cloudy

John Adams

Tech4Africa 2014

FAschenbrenner

Kernel Recipes 2015: Solving the Linux storage scalability bottlenecks

Anne Nicolas

lash devices introduced a sudden shift in the performance profile of direct attached storage. With IOPS rates orders of magnitude higher than rotating storage, it became clear that Linux needed a re-design of its storage stack to properly support and get the most out of these new devices. This talk will detail the architecture of blk-mq, the redesign of the core of the Linux storage stack, and the later set of changes made to adapt the SCSI stack to this new queuing model. Early results of running Facebook infrastructure production workloads on top of the new stack will also be shared. Jense Axboe, Facebook

Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...

ScyllaDB

ScyllaDB is a distributed database designed to scale horizontally and vertically — in theory. What about in practice? ScyllaDB’s Benny Halevy, Director, Software Engineering, will take you through the process and results of benchmarking our NoSQL database at the petabyte level, showing how you can use advanced features like workload prioritization to control priorities of transactional (read-write) and analytic (read-only) queries on the same cluster with smooth and predictable performance. To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.

Openstack meetup lyon_2017-09-28

Xavier Lucas

Machine Learning With H2O vs SparkML

Arnab Biswas

Meetup#2: Building responsive Symbology & Suggest WebServiceMinsk MongoDB User Group

High performace network of Cloud Native Taiwan User Group

HungWei Chiu

Flexible compute

Peter Clapham

Sanger, upcoming Openstack for Bio-informaticians

Peter Clapham

Data Science

Ahmet Bulut

Fixing twitter

Roger Xia

Fixing_Twitterliujianrong

Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...

smallerror

Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...xlight

Scaling a MeteorJS SaaS app on AWS

Brett McLain

Realtime traffic analyser

Alex Moskvin

5 Pitfalls to Avoid with MongoDB

Tim Callaghan

(WEB401) Optimizing Your Web Server on AWS | AWS re:Invent 2014

Amazon Web Services

Tuning your EC2 web server will help you to improve application server throughput and cost-efficiency as well as reduce request latency. In this session we will walk through tactics to identify bottlenecks using tools such as CloudWatch in order to drive the appropriate allocation of EC2 and EBS resources. In addition, we will also be reviewing some performance optimizations and best practices for popular web servers such as Nginx and Apache in order to take advantage of the latest EC2 capabilities.

Similar to Scaling application servers for efficiency (20)

Scale your Alfresco Solutions

John adams talk cloudy

Tech4Africa 2014

Kernel Recipes 2015: Solving the Linux storage scalability bottlenecks

Scylla Summit 2022: Operating at Monstrous Scales: Benchmarking Petabyte Work...

Openstack meetup lyon_2017-09-28

Machine Learning With H2O vs SparkML

Meetup#2: Building responsive Symbology & Suggest WebService

High performace network of Cloud Native Taiwan User Group

Flexible compute

Sanger, upcoming Openstack for Bio-informaticians

Data Science

Fixing twitter

Fixing_Twitter

Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...

Scaling a MeteorJS SaaS app on AWS

Realtime traffic analyser

5 Pitfalls to Avoid with MongoDB

(WEB401) Optimizing Your Web Server on AWS | AWS re:Invent 2014

More from Tomas Doran

Empowering developers to deploy their own data stores

Tomas Doran

Dockersh and a brief intro to the docker internals

Tomas Doran

Dockersh is a new tool to give a login shell into per-user Docker containers. (https://github.com/Yelp/dockersh) This talk will be an illustrated tour of what dockersh does, and why it might be useful to you. During this journey we’ll dive into the Go programming language, + libcontainer (the technologies Docker is built on) in addition to the facilities Docker uses in the kernel (Namespaces, Cgroups and Capabilities), how these work, and how normal mortals can (ab)use them for fun and profit

Sensu and Sensibility - Puppetconf 2014

Tomas Doran

As the Yelp infrastructure and engineering team grew, so did the pain of managing Nagios. Problems like splitting alerting across multiple teams, providing high availability and managing nagios systems in multiple environments had become pressing. As we grew towards a service oriented architecture and pushed some services out into the cloud, we rapidly needed more automated monitoring configuration. An evolutionary solution wasn’t going to solve all of our problems, we needed to revolutionize our monitoring. Sensu is built from the ground up to solve many of our issues and be easy to extend. This talk covers our puppet ‘monitoring_check’ API (that sets up monitoring for our services within puppet), how and why we deploy Sensu and our custom handlers and escalations, along with how we provide automatic ‘self service’ monitoring for dynamic services and how we deal with the challenges posed by the more ephemeral nature of cloud architectures.

Steamlining your puppet development workflow

Tomas Doran

Building a smarter application stack - service discovery and wiring for Docker

Tomas Doran

There are many advantages to a container based, microservices architecture - however, as always, there is no silver bullet. Any serious deployment will involve multiple host machines, and will have a pressing need to migrate containers between hosts at some point. In such a dynamic world hard coding IP addresses, or even host names is not a viable solution. This talk will take a journey through how Yelp has solved the discovery problems using Airbnb’s SmartStack to dynamically discover service dependencies, and how this is helping unify our architecture, from traditional metal to EC2 ‘immutable’ SOA images, to Docker containers.

Chasing AMI - Building Amazon machine images with Puppet, Packer and Jenkins

Tomas Doran

Using puppet when configuring EC2 machines seems a natural fit. However bringing up new machines from a community image with puppet is not trivial and can be slow, and so not useful for auto-scaling. The cloud also offers a solution to ongoing server maintenance, allowing you to launch fresh instances whenever you upgrade your applications (Immutable or Phoenix servers). However to predictably succeed, you need to freeze the puppet code alongside the application version for deployment. The solution to these issues is generating custom machine images (AMIs) with your software inlined. This talk will cover Yelp's use of a Packer, Jenkins and Puppet for generating AMIs. This will include how we deal with issues like bootstrapping, getting canonical information about a machine's environment and cluster state at launch time, as well as supporting immutable/phoenix servers in combination with more traditional long lived servers inside our hybrid cloud infrastructure.

Deploying puppet code at light speed

Tomas Doran

Thinking through puppet code layout

Tomas Doran

Docker puppetcamp london 2013

Tomas Doran

"The worst code I ever wrote"

Tomas Doran

Test driven infrastructure development (2 - puppetconf 2013 edition)Tomas Doran

Test driven infrastructure development

Tomas Doran

London devops - orc

Tomas Doran

London devops loggingTomas Doran

Message:Passing - lpw 2012Tomas Doran

Webapp security testingTomas Doran

Dates aghhhh!!?!?!?!

Tomas Doran

Messaging, interoperability and log aggregation - a new framework

Tomas Doran

In this talk, I will talk about why log files are horrible, logging log lines, and more structured performance metrics from large scale production applications as well as building reliable, scaleable and flexible large scale software systems in multiple languages. Why (almost) all log formats are horrible will be explained, and why JSON is a good solution for logging will be discussed, along with a number of message queuing, middleware and network transport technologies, including STOMP, AMQP and ZeroMQ. The Message::Passing framework will be introduced, along with the logstash.net project which the perl code is interoperable with. These are pluggable frameworks in ruby/java/jruby and perl with pre-written sets of inputs, filters and outputs for many many different systems, message formats and transports. They were initially designed to be aggregators and filters of data for logging. However they are flexible enough to be used as part of your messaging middleware, or even as a replacement for centralised message queuing systems. You can have your cake and eat it too - an architecture which is flexible, extensible, scaleable and distributed. Build discrete, loosely coupled components which just pass messages to each other easily. Integrate and interoperate with your existing code and code bases easily, consume from or publish to any existing message queue, logging or performance metrics system you have installed. Simple examples using common input and output classes will be demonstrated using the framework, as will easily adding your own custom filters. A number of common messaging middleware patterns will be shown to be trivial to implement. Some higher level use-cases will also be explored, demonstrating log indexing in ElasticSearch and how to build a responsive platform API using webhooks. Interoperability is also an important goal for messaging middleware. The logstash.net project will be highlighted and we'll discuss crossing the single language barrier, allowing us to have full integration between java, ruby and perl components, and to easily write bindings into libraries we want to reuse in any of those languages.

Zero mq logsTomas Doran

More from Tomas Doran (20)

Empowering developers to deploy their own data stores

Dockersh and a brief intro to the docker internals

Sensu and Sensibility - Puppetconf 2014

Steamlining your puppet development workflow

Building a smarter application stack - service discovery and wiring for Docker

Chasing AMI - Building Amazon machine images with Puppet, Packer and Jenkins

Deploying puppet code at light speed

Thinking through puppet code layout

Docker puppetcamp london 2013

"The worst code I ever wrote"

Test driven infrastructure development (2 - puppetconf 2013 edition)

Test driven infrastructure development

London devops - orc

London devops logging

Message:Passing - lpw 2012

Webapp security testing

Dates aghhhh!!?!?!?!

Messaging, interoperability and log aggregation - a new framework

Zero mq logs

Recently uploaded

DevOps and Testing slides at DASA Connect

Kari Kakkonen

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

Product School

JMeter webinar - integration with InfluxDB and Grafana

RTTS

Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application. In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics. Length: 30 minutes Session Overview ------------------------------------------- During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana: - What out-of-the-box solutions are available for real-time monitoring JMeter tests? - What are the benefits of integrating InfluxDB and Grafana into the load testing stack? - Which features are provided by Grafana? - Demonstration of InfluxDB and Grafana using a practice web application To view the webinar recording, go to: https://www.rttsweb.com/jmeter-integration-webinar

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...

Thierry Lestable

The Art of the Pitch: WordPress Relationships and Sales

Laura Byrne

Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes? All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.

Key Trends Shaping the Future of Infrastructure.pdf

Cheryl Hung

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Sri Ambati

Assuring Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

Product School

Neuro-symbolic is not enough, we need neuro-*semantic*

Frank van Harmelen

Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”. All of this illustrated with link prediction over knowledge graphs, but the argument is general.

When stars align: studies in data quality, knowledge graphs, and machine lear...

Elena Simperl

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

Product School

Designing Great Products: The Power of Design and Leadership by Chief Designe...

Product School

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

BookNet Canada

The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more. Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/ Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

How world-class product teams are winning in the AI era by CEO and Founder, P...

Product School

The Future of Platform Engineering

Jemma Hussein Allen

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

FIDO Alliance

Recently uploaded (20)

DevOps and Testing slides at DASA Connect

FIDO Alliance Osaka Seminar: Overview.pdf

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

JMeter webinar - integration with InfluxDB and Grafana

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...

The Art of the Pitch: WordPress Relationships and Sales

Key Trends Shaping the Future of Infrastructure.pdf

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Assuring Contact Center Experiences for Your Customers With ThousandEyes

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

Neuro-symbolic is not enough, we need neuro-*semantic*

When stars align: studies in data quality, knowledge graphs, and machine lear...

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

Designing Great Products: The Power of Design and Leadership by Chief Designe...

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

How world-class product teams are winning in the AI era by CEO and Founder, P...

The Future of Platform Engineering

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

Scaling application servers for efficiency

1. Scaling application servers for efficiency Tomas Doran <bobtfish@bobtfish.net> Catalyst web framework core team member Serving lots of media for a living ScaleCamp London 09

2. mod_$lang considered harmful • Web servers send bytes • Application servers generate pages • These two goals are orthogonal

3. EPIC FAIL • 80 Mb mod_perl processes • Serving static media • Reading stuff off disk/network in a while loop • Sending it to people on Virgin, using bittorrent, ‘3g’, via a damp piece of string. • With MaxClients x proc size being waaaay higher than physical memory

4. Pushing bytes - how to fail. • Serve static content from the same servers running your application (mod_perl, mod_python epic fail, mod_php moderate fail) • Static content AT ALL. I don’t care if you need to check ACLs

5. Pushing bytes - Success • X-Sendﬁle (lighty and mod_sendﬁle) • X-Accel-Redirect (nginx)

6. App server maps media • Check ACLs / Resolve one-time URI • Locate media • Serve X-Accel-Redirect • Aquire PIE, CAKE and PONY, proﬁt.

7. My setup • App runs FCGI • Run nCPUs x 1.2 procs (measure this!) • Looks up asset mapping (memcache) (Tm) • UPDATE download_attempts + 1 in MyFirstSQL • X-Accel-Redirect

8. • X-Accel-Redirect • Bytes sent by nginx • Serving mp3s - ﬁlesize 1-7Mb (ish). • > 30k sessions • > 200 reqs/s • Filling 1Gb of pipe • 1 box.

9. Several 100 Tb of media online Technology stack: nginx perl FCGI Catalyst MyFirstSQL memcache X-Accel-Redirect nginx-mogilefs-module MogileFS lighty

10. Even if extra context switching has zero overhead you serve people sooner if you queue. A B A B A B A B A B A finishes significantly before B in the lower diagram B finishes at the same time in both

11. RUN MOST EFFICIENT NUMBER OF WORKERS QUEUE REQUESTS AFTER THAT PROFILE PROFILE PROFILE

12. • App is notwork bound after tuning • Best efﬁciency ~ n CPUs x 1.2 (for me!) • CONTEXT SWITCHING HURTS

13. Thanks • Questions? • t0m <bobtfish@bobtfish.net> • http://catalystframework.org • http://search.cpan.org/author/BOBTFISH • http://github.com/bobtfish

Scaling application servers for efficiency

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Scaling application servers for efficiency

Similar to Scaling application servers for efficiency (20)

More from Tomas Doran

More from Tomas Doran (20)

Recently uploaded

Recently uploaded (20)

Scaling application servers for efficiency