How we build caibangzi.com

How we builtCaibangzi.com Robin Lu (caibangzi.com)

IntroductionPersonal Investment Management. Tool+ CommunityFounders: We are engineers!Technology: Ruby On Rails MySQL，Linux，Lighttpd ...

TopicsProblems we are facingArchitecture evolution of caibangziMaintenanceQ&A

Peaks!High PeakHeavy ComputationVisiting Peak, Computation Peak and Cacheexpiration Peak are in the same time. :...(

Evolution - StartLaunched at March, 2007Hardware: 1 x 1U server 1xAMD Opteron 250, 2G Memory, 250G SATA, RAID 1Bandwidth Shared less than 3Mbps

Evolution - Start Apache 2 + mod_proxy_balancer mongrel x 8 All inmemcached caibangzi one box rails 1.1.6 MySQL 5

First ChallengeMay, 2007Extremely time consuming actionsblocked the site.Performance tuningOptimize Rails cache

Second ChallengeAugust, 2007We were running out of bandwidthSolution: Money upgrade to 5Mbps

Server upgradingHigh growth rate after August, 2007Hardware upgrading in Sept. 2007 still 1 x 1U 2 x AMD 2214 2.2GHz 4G RAM

Until Sept. 2007 Apache 2 + mod_proxy_balancer mongrel x 20 Still inmemcached caibangzi one box rails 1.1.6 MySQL 5

Something happened in Oct. 2007Stock Index reached 6000Daily pv reached 1 millionBigger challenge came

Several Changes We MadeUpgrade bandwidth to 10MbpsSeparate APP server and DB serverLoad BalancerDistributed Computation

2 Boxes Web Server event mongrel x 30 APmemcached caibangzi P rails MySQL 5 Tianyuanji a

Load balancer Run with 2 mongrels behind balancer

Load balancerSlow actions block the otherswith balancer: by request Apach e by traffic HAproxy round-robin Nginx Lighttpd 1.4 Note: Nginx fixed it with ‘fair’ balancer later

Caibangzi Choice of BalancerLighttpd 1.5 beta proxy-core.balancer = "sqf" http://redmine.lighttpd. net/issues/show/1362For Rails application, choice of balanceris sometimes more important thanchoice of web server

More on Web Serverdisable keep-aliveserve static files with web server insteadof mongreltune configurations according to yourweb server documentations

Distributed solutionBackgrounDRB based on Distributed Ruby (DRB) use ruby thread high overhead during context switching very slow when concurrent connections higher than 10

Distributed solutionFastBackground our own distributed computation solution based on Eventmachine simple, but fast and works for us

Rails upgradingEarly of 2008, we upgraded to Rails 2.0 expired several plugins refactory to be more RESTful cookie session multi view => wap.caibangzi.com

Open APIAPI Based on OAuth integrate caibangzi into your own sitesOther APPs support several F8 clones xiaonei.com, hainei.com, UCHome

MaintenanceMunin easy to use and extendMonitCrontab

With all that....we handled: pv/day 1M rails req/day 1.8M peak rails req/sec 160 total req/day 5M peak req/sec 430 It’s the fact. Not the capacity. Our CPU usage never goes higher than 30%

How we build caibangzi.com

by Robin Lu on Mar 27, 2011

Accessibility

Tags

Upload Details

Usage Rights

Statistics

How we build caibangzi.com — Presentation Transcript