From One to a Cluster

Brian Moon discusses the evolution of the architecture of dealnews.com from a single server setup in the late 1990s to a clustered architecture in 2008. The initial setup encountered bottlenecks with software load balancing and using NFS. They overcame these by implementing hardware load balancing, dropping NFS, and using Memcached for caching. As traffic increased from sites like Digg and Yahoo!, they added more servers, offloaded static content to a CDN, and implemented a custom caching proxy and "pushed cache" to prevent stampeding. Their current architecture loads balances incoming traffic with F5 BIG-IP and uses replication and load balancing for the database.