• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Experience In Building Scalable Web Sites Through Infrastructure's View
 

Experience In Building Scalable Web Sites Through Infrastructure's View

on

  • 3,167 views

Presented at #Barcampbkk3 23-24 May 2009

Presented at #Barcampbkk3 23-24 May 2009

Statistics

Views

Total Views
3,167
Views on SlideShare
3,157
Embed Views
10

Actions

Likes
4
Downloads
3
Comments
2

2 Embeds 10

http://wp.monotechnology.com:8088 9
http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

CC Attribution-NonCommercial LicenseCC Attribution-NonCommercial License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • ยินดีครับ ที่เป็นประโยชน์ ไว้โอกาสหน้าคงได้ share กันอีก
    Are you sure you want to
    Your message goes here
    Processing…
  • ผมไ้ด้ฟังพี่พูดในงานนี้ด้วย...

    สุดยอดมากๆ เสียดายเวลาน้อยไปนิด ขอบคุณค้าบ....
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Experience In Building Scalable Web Sites Through Infrastructure's View Experience In Building Scalable Web Sites Through Infrastructure's View Presentation Transcript

    • Experience in Building Scalable Web Sites through Infrastructure's Aspect Barcamp Bangkok 3 May 23-24, 2009 Sripathum University
    • Gen#1 Web + DB Cost effective for startup
    • Gen#2 Web DB After boom!!
    • Gen#3 Web Web Web DB More boom!!, add more web servers but DB is the problem
    • Gen#4 Web Web Web WRITE Query DB Replication READ ONLY DB Replication could save your life for a while
    • Gen#... Reverse Proxy Web Reverse Proxy Web Memcached WRITE Query Memcached DB Replication READ ONLY DB Memcached: reduce READ query at DB Reverse Proxy: serves static contents
    • Finally: Horizontal Scaling Reverse Proxy LB Storage Reverse Proxy Memcached Global Database - Multimaster Web (dynamic) masterA masterB LB Replication Web (dynamic) Slave 1 Slave 2 UserDB1 UserDB2 UserDB3 UserDB..n
    • Behind the Scene Hardware (LB, Server) Software architecture OS Web Server Caching DB Infra: dns, network, storage, firewall, security process, log, backup, ntp, logging, source control, monitoring, ...
    • Good to Know 80% of the end-user response time is spent on the front-end Ref: developer.yahoo.com
    • YAHOO's “Best Practices for Speeding Up Your Web Site” Minimize HTTP requests Enable GZIP Add “Cache-Control” HTTP header Put stylesheet at the top Put scripts at the bottom ............... more at developer.yahoo.com
    • TIPS Enable “keep-alive” at all termination hops ISP Reverse Web LB (may do caching) Proxy Server
    • TIPS Split STATIC contents server Set up new top level domain to reduce cookie in HTTP header
    • TIPS No more cookie in HTTP request
    • Lot of Things to find out Hardware (LB, Server) Software architecture OS Web Server Caching DB Infra: dns, network, storage, firewall, security process, log, backup, ntp, logging, source control, monitoring, ...
    • Hardware: Server Any class can be used (blade, rack, local brand) DB Server: Raid 1+0, SAS, SCSI, FC (15K) Prefer 64 bits Battery-backed write cache could improve disk I/O performance We have HP BL20p, BL25p, BL460c, DL380 G3 G5, DL360 G3 G5 … Dell PE, Dell R300 Out of band management is required
    • Hardware: Load Balancer Evaluate first (L4 - L7) Foundry ServerIron GTE, G4 (now Brocade) We tried, Citrix Netscalr Radware Modern box comes with HTTP accelerate Crescendo
    • OS. 64 Bits Choose any os. that you can fully control it Monitor your servers CPU, load average, mem, disk, network
    • Web Server Apache 2.2 Still use prefork Comment unused modules Hardening php.ini Lighttpd, Nginx are ok Be careful about FastCGI IIS for some applications
    • Web Server (cont.) Keep monitor
    • Database MySQL 5.x Use memory table for unimportant data MyISAM, table locking InnoDB, difficult to backup PostgreSQL is coming Oracle just acquired SUN (April 2009) Enterprise features
    • Database (cont.) Examine all queries (if you can) Index hitting for SELECT ? Query monitoring
    • Database (cont.) Avoid full text searching Use Sphinx Use Solr Use EXPLAIN
    • Caching Memcached (32GB/server)
    • Caching Reverse Proxy Squid (designed as Forward proxy) Apache Mod_Proxy (never tried) Varnish (good response time, only UTF8)
    • Storage NAS is painful for scalable engineer SAN is not for sharing across servers Solutions Put the scripts at local disk (rsync) Put only shared folder at NAS Benchmark by iozone Caching Caching & Caching
    • Clustering Storage [1] Lustre: for HPC Very good performance No redundant Gluster Mountable in userspace Slow Painful for operation
    • Clustering Storage [2] MogileFS: good performance Build on commodity hardware Acces via API (php, python, perl) Hadoop Includes Map/Reduce, HDFS, HBase May suitable for logging application Good for batch processing
    • Security Hardening at all components Perform security audit if you can DNS blacklist monitoring Try network scan for ports changed Default: DENY policy
    • KB Train the new programmer Never knew about indexing Never used Memcache Never knew reverse proxy Internal KB (recommend MediaWiki, TWIKI)
    • Other Use version control Find “one click deployment system” All servers must sync their clock Centralize log server + Analyzer Backup & restore Perform URL monitoring from outside & inside
    • Try ITIL V3 to improve the quality of IT services
    • Twitter: @AuntiSpam