豆瓣技术架构的发展历程 @ QCon Beijing 2009

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    66 Favorites

    豆瓣技术架构的发展历程 @ QCon Beijing 2009 - Presentation Transcript

    1. 2009.4 hongqn@douban.com
    2. • 2005 3 • • •
    3. • 2.9M 1/4 • • 23M / 500~600/sec • 23 PC (1U*15/2U*8) • 12 • 38G memcached
    4. • 1U (frodo) • AMD Athlon 64 1.8GHz • 1G 160G SATA*2 • Gentoo Linux • MySQL 5 • Quixote (a Python web framework) • Lighttpd + SCGI (shire) • Memcached (!)
    5. Gentoo Linux • • emerge mysql • ebuild patch • • • GLSA(Gentoo Linux Security Advisories)
    6. MySQL • The world’s most popular open source database • / ==> MyISAM • ==> InnoDB • Replicate for backup
    7. Python • • Battery Included • • • CPUG: http://python.cn/
    8. Quixote • REST URL • Django, TurboGears, Pylons ZOPE • http://www.douban.com/subject/1000001 # luz/subject/__init__.py def _q_lookup(request, name): subject = get_subject(name) return lambda req: subject_ui(req, subject) # luz/subject/subject_ui.ptl def subject_ui [html] (request, subject): site_header(request) “<h1>%s</h1>” % subject.title site_footer(request)
    9. Lighttpd • • SCGI • SCGI: FastCGI Quixote • 80 lighttpd SCGI localhost Quixote
    10. Memcache • MySQL • libmemcache python Pyrex python 3x+ def get_subject(subject_id): subject = mc.get(‘s:’+subject_id) if subject is None: store.farm.execute(“select xxx, xxx from subject where id=%s”, subject_id) subject = Subject(*store.farm.fetchone()) mc.set(‘s:’+subject_id, subject) return subject
    11. Internet Lighttpd SCGI App FS MySQL Memcache Static Files
    12. • 1.2M / • IO •
    13. • 1U • pippin meriadoc ( merry) • , 4G 250G SATA*3 • • IP DNS IP -_-b • frodo (subversion, trac, etc...)
    14. Internet Lighttpd (#$) DNS HTTP Proxy SCGI App Lighttpd (!\") FS Static Files Memcache MySQL
    15. • • innodb_buffer_pool_size • • IP
    16. • 1.5M / • • IP
    17. • IP(BGP) • (arwen) • 74G 1w SATA * 3 • •
    18. Internet Data Lighttpd Mining read SCGI write App Replicate MySQL MySQL Master Slave Memcache Static Files
    19. • 2M / • IO • , etc... •
    20. • 4G 250G SATA*3 • 10000 • mod_rewrite URL • lighttpd mod_memcache • IO • web • •
    21. Internet store.farm App MySQL SCGI Master Lighttpd Memcache Replicate store.farmr HTTP Proxy WebDAV Web Service Replicate Lighttpd (w/ mod_memcache) MySQL write Slave Spiders Lighttpd WebDAV Memcache Static Files !\"#$% read Data Mining MySQL Slave
    22. • store farmr • replicate delay • • • cache • • • ...... but it works
    23. replicate delay def get_subject(sid): sbj = mc.get(‘s:’+sid) if sbj is None: sbj = flush_subject(sid, store.farmr) return sbj def flush_subject(sid, cursor=None): cursor = cursor or store.farm cursor.execute(“select ... from subject”) subject = Subject(*cursor.fetchone()) mc.set(‘s:’+sid, subject) return subject def update_subject(subject, props): store.farm.execute(“update subject ...”) store.farm.connection.commit() flush_subject(subject.id, store.farm)
    24. • 2.5M / • • / • SATA •
    25. • Scale Up 1U • 16G 147G SCSI *2 + 500G SATA • SCSI RAID-0 • MySQL Slave • memcached • MyISAM InnoDB • • Sphinx
    26. Internet Sphinx MySQL Master store.farm Web Service Replicate Lighttpd SCGI App HTTP Proxy Memcache Lighttpd (w/ mod_memcache) WebDAV store.farmr Memcache Lighttpd Web Service WebDAV MySQL Static Files Slave Spiders Memcache Memcache
    27. • 5.2M / • • Web IO • •
    28. • :) • • • • 3 1U 4 32G 1T SATA * 3 • otho.douban.com lotho.douban.com • lighttpd 1.5 with aio support • LVS • Scale Up: 4G -> 8G
    29. Internet www.douban.com otho.douban.com LVS LB (Master) Lighttpd Lighttpd 1.5 (w/ mod_cache) HTTP Proxy Lighttpd Keepalived LVS LB (backup) Lighttpd Lighttpd 1.5 WebDAV (w/ mod_cache) Static Files
    30. write !\"#$% replicate read Data Mining MySQL MySQL Slave Master read Replicate write !\"#$% Data Mining MySQL Slave
    31. • 6.4M / (5M PV) • • • CPU memcache /
    32. • • lighttpd mod_scgi round-robin • lighttpd 1.5 • mod_proxy • proxy.balance = fair (load based, passive balancing) • • spread
    33. Internet Lighttpd SCGI HTTP Proxy App Memcache Lighttpd HTTP Proxy spread HTTP Proxy Lighttpd Lighttpd Log Aggregator SCGI spread Lighttpd Memcache App WebDAV Static Files
    34. • 11M / 3 • • Sphinx • • load
    35. • • • • Sphinx -> Xapian • MogileFS
    36. !\" Master replicate replicate !\" %&'() Slave #$ Master read replicate Data #$ read Slave Mining write *+,- !\" Master Slave replicate *+,- Slave write replicate Data %&'() Mining #$ Slave
    37. • • store.farm[r] -> store.get_cursor(table=‘xxx’, ro=True/False) def flush_subject(sid, ro=False): cursor = store.get_cursor(table=‘subject’, ro=ro) cursor.execute(“select ... from subject”) subject = Subject(*cursor.fetchone()) mc.set(‘s:’+sid, subject) return subject •
    38. Internet upload.douban.com otho.douban.com nginx signed POST form Uploader (w/ proxy_store) HTTP redirect FileStorage MogileFS Tracker Gateway App MogileFS MogileFS Node Node MogileFS Master
    39. • libmemcache -> libmemcached consistent hash memcache • libmemcached consistent hash bug • CPU • libmemcached failover bug • nginx lighttpd load balance • spread • nginx
    40. Consistent Hash charlee http:// tech.idv2.com/2008/07/24/ memcached-004/
    41. Internet App SCGI Lighttpd HTTP Proxy Lighttpd HTTP Proxy App HTTP Proxy SCGI Lighttpd HTTP Proxy Nginx Lighttpd HTTP Proxy App Lighttpd SCGI Lighttpd WebDAV Static Files
    42. • 13M / • MogileFS • Tracker DB •
    43. • 8 • 32G CPU • (300G SCSI×2 + 1T SATA) × 3 • (1T SATA × 3) × 5 •6 2 • DoubanFS
    44. DoubanFS • hash • hash hash • Merkle Tree • consistent hash • WebDAV • MogileFS 3 50
    45. Merkle Tree
    46. Internet upload.douban.com otho.douban.com nginx signed POST form Uploader (w/ proxy_store) HTTP redirect FileStorage Gateway App DoubanFS DoubanFS Node Node
    47. • 16M / • • DoubanFS IO •
    48. • DoubanDB • • MySQL • MySQL Master • failover • replicate delay
    49. DoubanDB • Key-Value • Amazon Dynamo • set(key, value), get(key), delete(key) • memcache • Merkle Tree • Consistent Hash • TokyoCabinet • DoubanDB • DoubanFS 2.0 DoubanDB
    50. charlee http:// tech.idv2.com/2008/07/24/ memcached-004/
    51. !\" Master1 replicate replicate !\" %&'() Slave #$ Master1 read replicate Data #$ read Slave Mining write *+,- !\" Master Master2 replicate *+,- Slave write replicate Data %&'() Mining #$ Master2
    52. • DoubanFS • • • ngnix • www.douban.com LVS • RabbitMQ spread
    53. • • profile • memcache cache • join • • •
    SlideShare Zeitgeist 2009

    + hongqnhongqn Nominate

    custom

    6822 views, 66 favs, 13 embeds more stats

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 6822
      • 6619 on SlideShare
      • 203 from embeds
    • Comments 0
    • Favorites 66
    • Downloads 461
    Most viewed embeds
    • 84 views on http://www.kgblog.net
    • 47 views on http://www.kakee.net
    • 15 views on http://www.groops.cn
    • 13 views on http://keeng2008.appspot.com
    • 11 views on http://www.jondy.net

    more

    All embeds
    • 84 views on http://www.kgblog.net
    • 47 views on http://www.kakee.net
    • 15 views on http://www.groops.cn
    • 13 views on http://keeng2008.appspot.com
    • 11 views on http://www.jondy.net
    • 11 views on http://www.am82.cn
    • 9 views on http://blog.ncuhome.cn
    • 4 views on http://www.w4t.net.cn
    • 4 views on http://static.slidesharecdn.com
    • 2 views on http://jondy.net
    • 1 views on http://www.mefeedia.com
    • 1 views on http://203.208.37.132
    • 1 views on http://cache.baidu.com

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories