Experience In Building Scalable Web Sites Through Infrastructure's View

Experience in Building Scalable Web Sites through Infrastructure's Aspect Barcamp Bangkok 3 May 23-24, 2009 Sripathum University

Gen#1 Web + DB Cost effective for startup

Gen#2 Web DB After boom!!

Gen#3 Web Web Web DB More boom!!, add more web servers but DB is the problem

Gen#4 Web Web Web WRITE Query DB Replication READ ONLY DB Replication could save your life for a while

Gen#... Reverse Proxy Web Reverse Proxy Web Memcached WRITE Query Memcached DB Replication READ ONLY DB Memcached: reduce READ query at DB Reverse Proxy: serves static contents

Finally: Horizontal Scaling Reverse Proxy LB Storage Reverse Proxy Memcached Global Database - Multimaster Web (dynamic) masterA masterB LB Replication Web (dynamic) Slave 1 Slave 2 UserDB1 UserDB2 UserDB3 UserDB..n

Behind the Scene Hardware (LB, Server) Software architecture OS Web Server Caching DB Infra: dns, network, storage, firewall, security process, log, backup, ntp, logging, source control, monitoring, ...

Good to Know 80% of the end-user response time is spent on the front-end Ref: developer.yahoo.com

YAHOO's “Best Practices for Speeding Up Your Web Site” Minimize HTTP requests Enable GZIP Add “Cache-Control” HTTP header Put stylesheet at the top Put scripts at the bottom ............... more at developer.yahoo.com

TIPS Enable “keep-alive” at all termination hops ISP Reverse Web LB (may do caching) Proxy Server

TIPS Split STATIC contents server Set up new top level domain to reduce cookie in HTTP header

TIPS No more cookie in HTTP request

Lot of Things to find out Hardware (LB, Server) Software architecture OS Web Server Caching DB Infra: dns, network, storage, firewall, security process, log, backup, ntp, logging, source control, monitoring, ...

Hardware: Server Any class can be used (blade, rack, local brand) DB Server: Raid 1+0, SAS, SCSI, FC (15K) Prefer 64 bits Battery-backed write cache could improve disk I/O performance We have HP BL20p, BL25p, BL460c, DL380 G3 G5, DL360 G3 G5 … Dell PE, Dell R300 Out of band management is required

Hardware: Load Balancer Evaluate first (L4 - L7) Foundry ServerIron GTE, G4 (now Brocade) We tried, Citrix Netscalr Radware Modern box comes with HTTP accelerate Crescendo

OS. 64 Bits Choose any os. that you can fully control it Monitor your servers CPU, load average, mem, disk, network

Web Server Apache 2.2 Still use prefork Comment unused modules Hardening php.ini Lighttpd, Nginx are ok Be careful about FastCGI IIS for some applications

Web Server (cont.) Keep monitor

Database MySQL 5.x Use memory table for unimportant data MyISAM, table locking InnoDB, difficult to backup PostgreSQL is coming Oracle just acquired SUN (April 2009) Enterprise features

Database (cont.) Examine all queries (if you can) Index hitting for SELECT ? Query monitoring

Database (cont.) Avoid full text searching Use Sphinx Use Solr Use EXPLAIN

Caching Memcached (32GB/server)

Caching Reverse Proxy Squid (designed as Forward proxy) Apache Mod_Proxy (never tried) Varnish (good response time, only UTF8)

Storage NAS is painful for scalable engineer SAN is not for sharing across servers Solutions Put the scripts at local disk (rsync) Put only shared folder at NAS Benchmark by iozone Caching Caching & Caching

Clustering Storage [1] Lustre: for HPC Very good performance No redundant Gluster Mountable in userspace Slow Painful for operation

Clustering Storage [2] MogileFS: good performance Build on commodity hardware Acces via API (php, python, perl) Hadoop Includes Map/Reduce, HDFS, HBase May suitable for logging application Good for batch processing

Security Hardening at all components Perform security audit if you can DNS blacklist monitoring Try network scan for ports changed Default: DENY policy

KB Train the new programmer Never knew about indexing Never used Memcache Never knew reverse proxy Internal KB (recommend MediaWiki, TWIKI)

Other Use version control Find “one click deployment system” All servers must sync their clock Centralize log server + Analyzer Backup & restore Perform URL monitoring from outside & inside

Try ITIL V3 to improve the quality of IT services

Twitter: @AuntiSpam

Experience In Building Scalable Web Sites Through Infrastructure's View

by Phuwadon D on May 23, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 10

Statistics

Experience In Building Scalable Web Sites Through Infrastructure’s View — Presentation Transcript

http://wp.monotechnology.com:8088	9
http://www.slideshare.net	1