Building a scalable application- architecture considerations


Published on

Nate Johnson is CEO/Founder of Reliam. He worked on some of the first internet application architectures and continues to stay a step ahead of the industry by providing his clients with state of the art services. Nate founded Reliam (formerly Ristech) in 2001, and continually pushes the boundaries with company reliability and employee expertise. Reliam provides Internet Application Management, the next generation of managed web hosting. Reliam delivers solutions for the largest internet events in the world, the top interactive media agencies and the world's most recognized brands. For more information please check out If you are interested in having Nate speak at one of your events please e-mail

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • My Name is Nate Johnson and I’m the Founder and CEO of Reliam. I’d like to thank ShaRonand the HTML5 Developer group for inviting me to talk tonight. Who here is a developer? Who works exclusively with Internet based applications via desktop browsers or mobile platforms? Based on the attendee list, I wasn’t positive on how technical the audience was going to be, so I tried to make this relatable for everyone and if you have any technical questions or want to dive in further - we can get into that in the QandA.
  • Before we dive into it. I thought I’d give you a quick overview on what Reliam does. Reliam stands for Reliable Internet Application Management. The first think people ask when we meet them, is “What do you mean by Internet Application Management” and The short answer is that we do managed hosting and a whole lot more.We started “way back” (in Internet time) in 2001 as R.I.S. Technology and our core services at the time were pure managed hosting. We saw that websites were evolving from simple information delivery sites to complex programs that were integral to our lives. And just as there were desktop applications at the time and there have evolved to be applications for your phone - We view complex web sites to be Internet applications. So in 2008 we rebranded the company - RIS stood for Reliable Internet Services - We kept the Reliable and integrated it with our I A M (Internet Application Management) service and viola! Reliam was born. Since then we’ve been building our IAM service level – which actually goes deeper into the customer’s application code than any other service provider dares! ;-) We have embraced the DevOps culture and truly become integrated into our customer’s teams. Our service not only manages the traditional layers that a managed hosting company would - Datacenters, network infrastucture, servers, Operating systems, and application servers, monitoring, backup, support, etc. We also manage the performance of the entire application - performing services like Scalability and performance analysis, db query and code optimization, automation, correlative custom monitoring, and much more. Basically the only thing we won’t do, is develop new functionality for your application. This service level has been really successful with our customers in the advertising and entertainment industries. We’ve had quite a bit of real life experience with scaling web applications.
  • Some of the more well known applications we have scaled and managed are: Nissan’s Super bowl ads for the eXterra and we managed the global online auction for the release of the Nissan GTR. If anyone is a car person here, they probably know the GTR - There were only 1000 initial cars and collectors world wide were bidding to get one. In entertainment some of the high scalabilty events we’ve managed are – The Miss Universepagents, The Grammys website, The Golden Globes, and many applications for the Oscars, like their press photography distribution and award database. These are all examples of applications that become extremely busy for very short periods of time. Media companies we work with – MyDamnChannel– which is on online comedy TV channel that hosts David Wains and the Hilarious You suck at photoshop, SpinMedia– which hosts some of their high traffic sites with us – Most notably EgotasticOur most recent client They just relaunched their applicationwith us. I’m sure that if anyone has ever bought concert tickets, you know how scalable that application has to be. Working with these types of applications has given us some insight into what is required to build and maintain scalability. And what I wanted to talk about tonight is at a high level - how to architect and build something that’s going to need to scale.
  • ** So what is a Scalable application? What is Scalability?** In our view - Scalability is the ability to grow effectively and continue to achieve your goals regardless of the number of users using your services. And this isn’t a trivial goal. Its actually quite hard. And sometimes it’s a little bit of an after thought in the development process. If I had a dollar for every time I’ve heard someone say - “What? This can’t be breaking - it works great on my laptop!” I’d be rich. So designing with scalability in mind can save you a lot of frustration in the future. And as we all know, it’s a competitive, impatient, world out there. Many times you don’t get a second chance to make a first impression. ** So how does one build applications that scale? ** To build applications that scale, it requires considering scalability at every layer of the application.The enemy of every application is concurrency. – The amount of connections/sessions/users at the same time.And the more efficient you can be with your resources at every layer, the more concurrency you can handle in a single instance. A good example of real world scalability is Costco - They have economized their stores to be as efficient as possible. No advertising displays, no sales people, everything is self serve- grab and go. Costco has a lot of similarities to scalable applications actually - Everything is sold in a large size so you don’t have to come back often (think optimizing MTU on your network for large packets). Their Isles are wide (think high bandwidth) and they have a ton of cashiers (think load balanced webservers… ) There is one HUGE difference between Costco and most internet applications that need to scale though. The difference is that Costco knows they are going to be busy – they are always busy – they’ve had time to evolve these techniques. With many Internet applications, and the viral nature of content, You could suddenly find yourself on the front page of Reddit and be faced with a tidal wave of traffic. So in the next couple of slides, we’ll take a look at each layer of an application architecture and give some tips on how to build scalably.
  • There are a lot of layers that all work together to make an application scalable. I like to break them down into 4 high level categories.The first is Application Design Layer - This includes the code layer. The HOW and WHEN of your application. How is that query structured, when does the table update? How will user created content and processing be handled?The second is the Application Architecture Layer - This is the lego layer, What types of databases servers will be uses, how many? Will there be a display caching tier? Will there be a DB caching tier? How many web servers, load balancers, application servers will be required? The third is the Configuration Layer - How to tune the operating system and various service process for maximum performance. Webserver services, database services, caching services, queuing services, etc. The fourth is the Physical Layer - This includes optimization of physical hardware, raid controllers, network cards, Ram, Processors, etc. It also includes the network layer, switches, routers, firewalls. Security devices. In the next few slides, I’ll cover a couple of considerations that can be made at each of these layers to enhance application scalability.
  • Things to consider on the Design Layer – Integrating scalable techniques into the design of the application is critical for getting the best performance with heavy loads. The thing to remember is to prevent resource over utilization and blocking at all costs. Here are a couple of things to think about at the Design layer Page and Object Caching - Whenever possible, think about utilizing caching methods for static objects and even entire pages. Its 100,000 times faster to retrieve data from ram than from disk. To put that into perspective in our timescale, if I was hungry and wanted to grab a cookie and the cookie jar on my counter was RAM, If it took me 1 minute to gtab myself a tasty snack from the cookie jar, grabbing a cookie from “disk” would take me 70 days. Even at the nanosecond/millisecond timescale, It’s a huge difference that really adds up over millions of operations. Plan for growth – Each time your application has to personalize, that consumes CPU and memory. Try to only set cookies and create session IDs when absolutely necessary and when you do store that information in a shared location so that you can easily scale to multiple instance later on. One thing to consider is serving CSS, JavaScript, and static assets from a cookie less DNS name. That enables paralleling requests for quick retrieval. It goes without saying, but its something we still see… don’t use static IP addresses in code, use DNS names so that there is a central location for changes. Additionally, think about using relative paths in references to objects so that your code is independent from web server configuration. Read write splitting – When dealing with a data in a database, structure your code so that you can segment the reads and writes to different data sources if need be. This allows for easy scalability in the database tier in the future. Consider new methods – Web sockets. HTML5 spec includes support for web sockets, which is a way to maintain an open two way socket between the client and server. Although this may not be fully supported on al platforms, its something to keep in mind, as the performance improvements can be massive, up to 600 times more efficient.
  • The Architecture layer defines how requests flow through your application. Here are a couple of scalability considerations to be made at the architecture layer: Caching and Content distribution– Consider using a proxy cache like Varnish or Squid. Again RAM is way faster than disk and if you can offload the delivery of content to a caching tier, your application will run faster. Use Memcache to cache query results or any objects really. Content distribution with a CDN is a similar concept - Except the goal is to cache content as close to the end user as possible. The also minimizes traffic to your infrastructure. Load balancing – Even if your application may not need load balancing during normal operation, its good to be prepared so consider running a single instance behind a load balancer. You are then prepared to add additional instances and scale when needed. Also depending on the type of load balancer, tune the balancing algorithm to match your application requirements. Database scaling- When it comes to webservers, scaling is fairly straightforward – you clone and add more. But with Databases, it becomes a bit more difficult. Maintaining the integrity of the data and ensuring performance sometimes have opposing requirements. One simple method we have found effective is to create a replication scenario with a single write master and read slaves. In most cases, application read from the database to create dynamic pages more than write, so if your application understands read/write splitting this becomes a powerful way to scale your database tier. If your application doesn’t, you can still use this technique by introducing a proxy tier using something like MySQL Proxy which can be configured to automatically split reads from writes to different database targets. Monitoring - A critical piece of the scalability equation is monitoring. We recommend comprehensive instrumentation at all tiers of the application and even within the application itself. Trending data points like CPU, RAM, Disk, Network, response time, etc. Can assist in so many ways when designing for scalability. One tip for monitoring would be to create a specialized page that exercises the critical components of the application, connects to the database, access shared storage, verifies queues, etc. And then point your monitoring to that page to verify response times and availability.Load Testing - Load testing falls under the architecture layer by default. It’s vital part of the application development process. Without it, you have no way of knowing how your application will perform when the going gets rough. There are many tools that make basic load testing less of a headache then it was in the past, utilities like Apache bench and Seige make executing a simple load test fun. Our goal in load testing is to establish a performance baseline for an application and then plan to maintain between 50% to 75% overhead in the application to accommodate for initial bursts as a buffer while additional resources come online.
  • The Configuration Layer is probably the most complex and also the most crucial. This layer includes the optimization and tuning of not just the underlying operating system, but also of your webserver, database, application server, and any other services required to make the application run. Operating systems and service software are designed to be extremely general. An operating system has to be equally suited for being a file server or running complex computations and webservers and database servers are designed with flexibility in mind as well. To customize them for your particular application takes skill, experience, and immense amounts of google-fu. Here are a couple of considerations to be made at the configuration layer: Operating systems – Hardening for security and tuning for performance should be your primary goal here. The operating system should be tuned for its specific hardware platform with things like swap space, open file handles, i/o buffers, etc. If the OS wasn’t installed using the the bare minimum of configurations, disable any un-needed processes and daemons, and optimize the startup sequences for the critical ones. Webservers – There are a lot of webservers in the world today. We find that a well tuned apache instance provides the performance, security, and compatibility balance that is still best. Optimizing settings like compression for JavaScript, CSS, and HTML, enabling keep alives, tuning timeouts that are appropriate for the application, and setting the maximum connection limit and max resource utilization to match the application and underlying hardware platform will keep your webserver handling requests in the fastest way possible. Databases – Again a lot of platforms to choose from. New concepts like NOSQL are worth exploring if it fits your model. For most traditional web application running a MySQL or PostgreSQL platform, consider things like, choosing the right the table type for your requirements, tuning the buffer sizes to match the number of possible concurrent connections from your wed tier, and tuning the query cache size to that it fits the results of your most common queries. Configuration management – All of the changes made to each of the files that make up a complex internet application can be overwhelming. Use a configuration management tool like Puppet or Chef to document, manage, and automate those changes. This helps a lot in reducing human workload and error.
  • Physical Layer – The physical layer includes the networks, servers, virtualization platforms, and clouds that form the bedrock of your application. The physical layer is so often ignored today. Listening to many developers, it seems they could care less about where their applications run. I can understand their lack of enthusiasm. The physical layer isn’t very sexy. Public cloud services like Amazon have almost reduced the physical layer to a commodity. However, there are still many considerations to be made on the physical layer. At the network layer, tuning your TCP stack especially in between the tiers of your application can greatly improve network performance. Increasing the MTU size on storage networks, and optimizing window sizes across distances can be ways to improve your latency and throughput. Scaling your application also means protecting it from the bad guys. Things like DDoS, exploit attempts, and general brute force attacks often come right at the time when you are most busy, and failing to prepare for that eventuality can mean disaster. Wire speed unified threat management devices are the way to do here. Don’t rely on your front end servers to use CPU cycles to block unwanted traffic, perform it at the network layer. Physical servers have gotten a bad rap. When it comes to needing raw CPU power, nothing beats a well configured physical server. Optimization strategies on disks, networks, and CPU power can give you better performance at a better price point than many virtualized offerings. Finally cloud services - Cloud is really a marketing term, we all know that. But virtualization technology combines with automated orchestration processes, and configuration management does provide real benefits to some tiers of your application and the flexibility offered with snapshots, cloning, and virtualized networks can make a big difference when trying to scale dynamically. The big considerations to make in cloud services are CPU measurement (they all do it differently) and I/O performance. Unfortunately when using a 3rd party cloud service, there aren’t many things you can do to change the way their physical layer works. But that’s the tradeoff with not having to think about the physical layer.
  • So to sum up- As we all know, there is a lot of competition for eyeballs on the internet today and attention spans are very short. As the saying goes – You never get a second chance to make a first impression. If your application is the subject of the next viral craze or you experience the high quality problem of success, having a scalable application is going to allow you to make the most of that opportunity. Design with scalability in mind at all levels of your architecture, and if this is something that you don’t want to have to think about… come talk to us. We’ll help you out.
  • Thanks for listening… any questions?
  • Building a scalable application- architecture considerations

    1. 1. How to Build a ScalableApplicationNate Johnson, Reliam CEO & Founder
    2. 2. What is Scalability?
    3. 3. Lots of Layers
    4. 4. The Design Layer• Page and Object Caching• Plan for Growth• Read/Write Splitting• Consider New Methods
    5. 5. The Architecture Layer• Caching and Content Distribution• Load Balancing• Database Scaling• Monitoring• Load Testing
    6. 6. Configuration Layer• Operating Systems• Web Servers• Databases• Configuration Management
    7. 7. Physical Layer• Network Layer• Physical Servers• Cloud Services
    8. 8. Thank you!