Your SlideShare is downloading. ×
Experience in Building Scalable Web Sites
    through Infrastructure's Aspect




             Barcamp Bangkok 3
         ...
Gen#1




     Web + DB




Cost effective for startup
Gen#2




  Web



   DB



After boom!!
Gen#3



      Web          Web           Web




                    DB



More boom!!, add more web servers but DB is th...
Gen#4



  Web               Web         Web

                                   WRITE Query

                     DB
    ...
Gen#...


   Reverse Proxy
                                         Web
       Reverse Proxy
                             ...
Finally: Horizontal Scaling


     Reverse Proxy
LB                    Storage
     Reverse Proxy


                      ...
Behind the Scene


Hardware (LB, Server)
Software architecture
OS
Web Server
Caching
DB
Infra: dns, network, storage, fire...
Good to Know



80% of the end-user response
 time is spent on the front-end




                         Ref: developer.y...
YAHOO's “Best Practices for Speeding Up Your
                 Web Site”


Minimize HTTP requests
Enable GZIP
Add “Cache-Co...
TIPS


Enable “keep-alive” at all termination hops




              ISP                  Reverse     Web
                ...
TIPS


Split STATIC contents server
  Set up new top level domain to reduce cookie in
   HTTP header
TIPS


No more cookie in HTTP request
Lot of Things to find out


Hardware (LB, Server)
Software architecture
OS
Web Server
Caching
DB
Infra: dns, network, stor...
Hardware: Server


Any class can be used (blade, rack, local brand)
DB Server: Raid 1+0, SAS, SCSI, FC (15K)
Prefer 64 bit...
Hardware: Load Balancer


Evaluate first (L4 - L7)
Foundry ServerIron GTE, G4 (now Brocade)
We tried,
  Citrix Netscalr
  ...
OS.


64 Bits
Choose any os. that you can fully control it
Monitor your servers
  CPU, load average, mem, disk, network
Web Server


Apache 2.2
   Still use prefork
   Comment unused modules
   Hardening php.ini
Lighttpd, Nginx are ok
   Be c...
Web Server (cont.)


Keep monitor
Database


MySQL 5.x
  Use memory table for unimportant data
  MyISAM, table locking
  InnoDB, difficult to backup
Postgre...
Database (cont.)


Examine all queries (if you can)
Index hitting for SELECT ?
Query monitoring
Database (cont.)


Avoid full text searching
  Use Sphinx
  Use Solr
Use EXPLAIN
Caching


Memcached (32GB/server)
Caching


Reverse Proxy
  Squid (designed as Forward proxy)
  Apache Mod_Proxy (never tried)
  Varnish (good response time...
Storage


NAS is painful for scalable engineer
SAN is not for sharing across servers
Solutions
  Put the scripts at local ...
Clustering Storage [1]


Lustre:
  for HPC
  Very good performance
  No redundant
Gluster
  Mountable in userspace
  Slow
...
Clustering Storage [2]


MogileFS:
  good performance
  Build on commodity hardware
  Acces via API (php, python, perl)
Ha...
Security


Hardening at all components
Perform security audit if you can
DNS blacklist monitoring
Try network scan for por...
KB


Train the new programmer
  Never knew about indexing
  Never used Memcache
  Never knew reverse proxy
Internal KB (re...
Other


Use version control
Find “one click deployment system”
All servers must sync their clock
Centralize log server + A...
Try ITIL V3 to improve the quality of IT services
Twitter: @AuntiSpam
Upcoming SlideShare
Loading in...5
×

Experience In Building Scalable Web Sites Through Infrastructure's View

2,153

Published on

Presented at #Barcampbkk3 23-24 May 2009

Published in: Technology
2 Comments
4 Likes
Statistics
Notes
  • ยินดีครับ ที่เป็นประโยชน์ ไว้โอกาสหน้าคงได้ share กันอีก
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • ผมไ้ด้ฟังพี่พูดในงานนี้ด้วย...

    สุดยอดมากๆ เสียดายเวลาน้อยไปนิด ขอบคุณค้าบ....
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
2,153
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
2
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "Experience In Building Scalable Web Sites Through Infrastructure's View"

  1. 1. Experience in Building Scalable Web Sites through Infrastructure's Aspect Barcamp Bangkok 3 May 23-24, 2009 Sripathum University
  2. 2. Gen#1 Web + DB Cost effective for startup
  3. 3. Gen#2 Web DB After boom!!
  4. 4. Gen#3 Web Web Web DB More boom!!, add more web servers but DB is the problem
  5. 5. Gen#4 Web Web Web WRITE Query DB Replication READ ONLY DB Replication could save your life for a while
  6. 6. Gen#... Reverse Proxy Web Reverse Proxy Web Memcached WRITE Query Memcached DB Replication READ ONLY DB Memcached: reduce READ query at DB Reverse Proxy: serves static contents
  7. 7. Finally: Horizontal Scaling Reverse Proxy LB Storage Reverse Proxy Memcached Global Database - Multimaster Web (dynamic) masterA masterB LB Replication Web (dynamic) Slave 1 Slave 2 UserDB1 UserDB2 UserDB3 UserDB..n
  8. 8. Behind the Scene Hardware (LB, Server) Software architecture OS Web Server Caching DB Infra: dns, network, storage, firewall, security process, log, backup, ntp, logging, source control, monitoring, ...
  9. 9. Good to Know 80% of the end-user response time is spent on the front-end Ref: developer.yahoo.com
  10. 10. YAHOO's “Best Practices for Speeding Up Your Web Site” Minimize HTTP requests Enable GZIP Add “Cache-Control” HTTP header Put stylesheet at the top Put scripts at the bottom ............... more at developer.yahoo.com
  11. 11. TIPS Enable “keep-alive” at all termination hops ISP Reverse Web LB (may do caching) Proxy Server
  12. 12. TIPS Split STATIC contents server Set up new top level domain to reduce cookie in HTTP header
  13. 13. TIPS No more cookie in HTTP request
  14. 14. Lot of Things to find out Hardware (LB, Server) Software architecture OS Web Server Caching DB Infra: dns, network, storage, firewall, security process, log, backup, ntp, logging, source control, monitoring, ...
  15. 15. Hardware: Server Any class can be used (blade, rack, local brand) DB Server: Raid 1+0, SAS, SCSI, FC (15K) Prefer 64 bits Battery-backed write cache could improve disk I/O performance We have HP BL20p, BL25p, BL460c, DL380 G3 G5, DL360 G3 G5 … Dell PE, Dell R300 Out of band management is required
  16. 16. Hardware: Load Balancer Evaluate first (L4 - L7) Foundry ServerIron GTE, G4 (now Brocade) We tried, Citrix Netscalr Radware Modern box comes with HTTP accelerate Crescendo
  17. 17. OS. 64 Bits Choose any os. that you can fully control it Monitor your servers CPU, load average, mem, disk, network
  18. 18. Web Server Apache 2.2 Still use prefork Comment unused modules Hardening php.ini Lighttpd, Nginx are ok Be careful about FastCGI IIS for some applications
  19. 19. Web Server (cont.) Keep monitor
  20. 20. Database MySQL 5.x Use memory table for unimportant data MyISAM, table locking InnoDB, difficult to backup PostgreSQL is coming Oracle just acquired SUN (April 2009) Enterprise features
  21. 21. Database (cont.) Examine all queries (if you can) Index hitting for SELECT ? Query monitoring
  22. 22. Database (cont.) Avoid full text searching Use Sphinx Use Solr Use EXPLAIN
  23. 23. Caching Memcached (32GB/server)
  24. 24. Caching Reverse Proxy Squid (designed as Forward proxy) Apache Mod_Proxy (never tried) Varnish (good response time, only UTF8)
  25. 25. Storage NAS is painful for scalable engineer SAN is not for sharing across servers Solutions Put the scripts at local disk (rsync) Put only shared folder at NAS Benchmark by iozone Caching Caching & Caching
  26. 26. Clustering Storage [1] Lustre: for HPC Very good performance No redundant Gluster Mountable in userspace Slow Painful for operation
  27. 27. Clustering Storage [2] MogileFS: good performance Build on commodity hardware Acces via API (php, python, perl) Hadoop Includes Map/Reduce, HDFS, HBase May suitable for logging application Good for batch processing
  28. 28. Security Hardening at all components Perform security audit if you can DNS blacklist monitoring Try network scan for ports changed Default: DENY policy
  29. 29. KB Train the new programmer Never knew about indexing Never used Memcache Never knew reverse proxy Internal KB (recommend MediaWiki, TWIKI)
  30. 30. Other Use version control Find “one click deployment system” All servers must sync their clock Centralize log server + Analyzer Backup & restore Perform URL monitoring from outside & inside
  31. 31. Try ITIL V3 to improve the quality of IT services
  32. 32. Twitter: @AuntiSpam

×