SlideShare a Scribd company logo
1 of 54
Download to read offline
2009.4

hongqn@douban.com
•   2005 3

•

•

•
• 2.9M            1/4
•
• 23M        /        500~600/sec
• 23      PC      (1U*15/2U*8)
  • 12
  • 38G memcached
•        1U         (frodo)
    •        AMD Athlon 64 1.8GHz
    •   1G       160G SATA*2
•   Gentoo Linux
•   MySQL 5
•   Quixote (a Python web framework)
•   Lighttpd + SCGI (shire)
•   Memcached (!)
Gentoo Linux
•
    • emerge mysql
    • ebuild          patch
•
•
    • GLSA(Gentoo Linux Security Advisories)
MySQL

•   The world’s most popular open source database

•            /             ==> MyISAM

•                ==> InnoDB

•   Replicate for backup
Python

•
•   Battery Included

•
•
    •   CPUG: http://python.cn/
Quixote
•                           REST         URL
•              Django, TurboGears, Pylons
              ZOPE
• http://www.douban.com/subject/1000001
    # luz/subject/__init__.py
    def _q_lookup(request, name):
        subject = get_subject(name)
        return lambda req: subject_ui(req, subject)

    # luz/subject/subject_ui.ptl
    def subject_ui [html] (request, subject):
        site_header(request)
        “<h1>%s</h1>” % subject.title
        site_footer(request)
Lighttpd
•
•       SCGI
    • SCGI:               FastCGI
      Quixote
•                  80        lighttpd
                   SCGI    localhost
    Quixote
Memcache
•                                           MySQL
•     libmemcache   python                          Pyrex
      python    3x+
    def get_subject(subject_id):

       subject = mc.get(‘s:’+subject_id)

       if subject is None:

          store.farm.execute(“select xxx, xxx from subject where id=%s”,

                              subject_id)

          subject = Subject(*store.farm.fetchone())

          mc.set(‘s:’+subject_id, subject)

       return subject
Internet




                  Lighttpd

                SCGI



          App

                       FS




MySQL   Memcache             Static Files
• 1.2M   /
• IO
•
•                1U
    •   pippin       meriadoc (      merry)
    •         , 4G          250G SATA*3
•
•                      IP            DNS
    IP -_-b
•                            frodo
    (subversion, trac, etc...)
Internet




        Lighttpd (#$)                               DNS
                                        HTTP
                                        Proxy
                    SCGI


                           App                  Lighttpd (!quot;)

               FS




Static Files          Memcache                    MySQL
•
    • innodb_buffer_pool_size
•
•              IP
• 1.5M   /
•
• IP
•                            IP(BGP)
•                         (arwen)
    • 74G 1w   SATA * 3
    •
•
Internet




                                                               Data
               Lighttpd                                       Mining
                                                               read
                   SCGI                               write



                          App                 Replicate




                                     MySQL                    MySQL
                                     Master                   Slave


                          Memcache
Static Files
• 2M              /
•                     IO
  •
       , etc...
•
•                               4G   250G SATA*3
•                                10000
    •   mod_rewrite       URL
•              lighttpd              mod_memcache

    •           IO
•                web
    •
•
Internet

                                                                store.farm
                                              App

                                                                                       MySQL
                                SCGI
                                                                                       Master
          Lighttpd                                         Memcache                        Replicate
                                                                     store.farmr
        HTTP Proxy
                                   WebDAV   Web Service
                                                                                                 Replicate
      Lighttpd
(w/ mod_memcache)

                                                                                     MySQL               write

                                                                                     Slave
                                            Spiders
                     Lighttpd
                     WebDAV
                                                          Memcache
Static Files


                                                                                    !quot;#$%
                                                                                       read
                                                                                                        Data
                                                                                                       Mining
                                                                                   MySQL
                                                                                   Slave
•   store          farmr
•                replicate delay
    •
    •

    •                          cache
        •
•

    •   ......             but it works
replicate delay
def get_subject(sid):
   sbj = mc.get(‘s:’+sid)
   if sbj is None:
      sbj = flush_subject(sid, store.farmr)
      return sbj

def flush_subject(sid, cursor=None):
   cursor = cursor or store.farm
   cursor.execute(“select ... from subject”)
   subject = Subject(*cursor.fetchone())
   mc.set(‘s:’+sid, subject)
   return subject

def update_subject(subject, props):
   store.farm.execute(“update subject ...”)
   store.farm.connection.commit()
   flush_subject(subject.id, store.farm)
• 2.5M       /
•
  •      /
• SATA
•
•   Scale Up               1U
    •   16G        147G SCSI *2 + 500G SATA
    •   SCSI     RAID-0
•       MySQL Slave
•         memcached
•              MyISAM            InnoDB
    •
•                       Sphinx
Internet
                                                     Sphinx


                                                                                          MySQL
                                                                                          Master
                                                                    store.farm
                                                Web Service
                                                                                                    Replicate
          Lighttpd
                                SCGI

                                              App
        HTTP Proxy

                                                                                         Memcache
      Lighttpd
(w/ mod_memcache)                  WebDAV
                                                                           store.farmr
                                                               Memcache

                     Lighttpd
                                            Web Service
                     WebDAV

                                                                                          MySQL
Static Files
                                                                                          Slave

                                            Spiders


                                                              Memcache
                                                                                         Memcache
• 5.2M   /
•
• Web        IO


•
•
•                             :)
    •
    •
    •
•        3 1U             4        32G        1T SATA * 3
•                      otho.douban.com        lotho.douban.com
    •   lighttpd 1.5 with aio support
    •        LVS
•   Scale Up:                            4G -> 8G
Internet


                          www.douban.com          otho.douban.com


                                                    LVS LB
                                                   (Master)
          Lighttpd
                                                                        Lighttpd 1.5
                                                                      (w/ mod_cache)
          HTTP Proxy




          Lighttpd                                                  Keepalived

                                                   LVS LB
                                                  (backup)
                       Lighttpd
                                                                        Lighttpd 1.5
                       WebDAV
                                                                      (w/ mod_cache)

Static Files
write




                                      !quot;#$%
                replicate
                                         read        Data
                                                    Mining
                                     MySQL
MySQL
                                     Slave
Master

                                             read
    Replicate
                            write




                                      !quot;#$%
                                           Data
                                          Mining
MySQL
Slave
• 6.4M       /   (5M PV)
•
  •
 • CPU   memcache          /
•
    •   lighttpd     mod_scgi     round-robin
    •   lighttpd 1.5
    •   mod_proxy
        •   proxy.balance = fair (load based, passive
            balancing)
•
•           spread
Internet


                                                     Lighttpd

                                                       SCGI
                              HTTP Proxy

                                                       App      Memcache
          Lighttpd

          HTTP Proxy


                                            spread
                              HTTP Proxy
          Lighttpd


                                                     Lighttpd
                          Log
                       Aggregator                      SCGI
                                           spread

                       Lighttpd
                                                                Memcache
                                                       App
                       WebDAV
Static Files
• 11M      /      3
•
• Sphinx
•
•
           load
•
    •
    •
• Sphinx -> Xapian
• MogileFS
!quot;
             Master

                       replicate

replicate
                                                !quot;
                                   %&'()
                                                Slave
               #$
              Master                                     read
                       replicate
                                                                  Data
                                                #$        read
                                                Slave            Mining
                                                 write



                                            *+,-
              !quot;
                                            Master
              Slave
                       replicate


             *+,-
              Slave                     write



 replicate
                                         Data
                                    %&'()
                                        Mining
              #$
              Slave
•
•   store.farm[r] -> store.get_cursor(table=‘xxx’,
    ro=True/False)
    def flush_subject(sid, ro=False):
        cursor = store.get_cursor(table=‘subject’, ro=ro)
        cursor.execute(“select ... from subject”)
        subject = Subject(*cursor.fetchone())
        mc.set(‘s:’+sid, subject)
        return subject

•
Internet

                         upload.douban.com otho.douban.com




                                                                   nginx
      signed POST form                 Uploader
                                                             (w/ proxy_store)

               HTTP redirect
                                                             FileStorage
                                  MogileFS Tracker
                                                              Gateway
App




                                                  MogileFS                 MogileFS
                                                   Node                     Node
                               MogileFS
                                Master
•   libmemcache -> libmemcached              consistent
    hash    memcache
    •       libmemcached consistent hash           bug
•                              CPU
    •       libmemcached failover           bug
•       nginx     lighttpd   load balance
•                            spread
    •           nginx
Consistent Hash




                                charlee http://
                  tech.idv2.com/2008/07/24/
                  memcached-004/
Internet


                                                                  App
                                                           SCGI
                                                Lighttpd




                                   HTTP Proxy
  Lighttpd
           HTTP Proxy
                                                                  App
  HTTP Proxy                                               SCGI
                                                Lighttpd
                                   HTTP Proxy
                        Nginx
 Lighttpd

                                   HTTP Proxy




                                                                  App
                        Lighttpd                           SCGI
                                                Lighttpd
                        WebDAV
Static Files
• 13M   /
•                 MogileFS
  •         Tracker DB


•
•       8
    • 32G            CPU
    • (300G SCSI×2 + 1T SATA) × 3
    • (1T SATA × 3) × 5
    •6          2
•       DoubanFS
DoubanFS
•                            hash


•      hash                              hash


•                          Merkle Tree
•      consistent hash
•   WebDAV
•             MogileFS 3     50
Merkle Tree
Internet

                         upload.douban.com otho.douban.com




                                                                   nginx
      signed POST form                 Uploader
                                                             (w/ proxy_store)

               HTTP redirect
                                                             FileStorage
                                                              Gateway
App




                                DoubanFS                                   DoubanFS
                                  Node                                       Node
• 16M        /
•
• DoubanFS       IO
•
•       DoubanDB
    •
    •                 MySQL


• MySQL Master
  • failover
  • replicate delay
DoubanDB
•         Key-Value
•     Amazon Dynamo
•              set(key, value), get(key), delete(key)
•   memcache
•         Merkle Tree
•   Consistent Hash
•     TokyoCabinet
•                                DoubanDB
•   DoubanFS 2.0        DoubanDB
charlee http://
tech.idv2.com/2008/07/24/
memcached-004/
!quot;
             Master1

                        replicate

replicate
                                                 !quot;
                                    %&'()
                                                 Slave
               #$
              Master1                                     read
                        replicate
                                                                   Data
                                                 #$        read
                                                 Slave            Mining
                                                  write



                                             *+,-
              !quot;
                                             Master
             Master2
                        replicate


             *+,-
              Slave                      write



 replicate
                                          Data
                                     %&'()
                                         Mining
              #$
             Master2
•       DoubanFS
    •
    •
• ngnix
• www.douban.com       LVS
• RabbitMQ         spread
•
•              profile
•   memcache            cache


•                join
•
•
•
豆瓣技术架构的发展历程 @ QCon Beijing 2009

More Related Content

What's hot

微博cache设计谈
微博cache设计谈微博cache设计谈
微博cache设计谈
Tim Y
 

What's hot (20)

Global Azure Virtual 2020 What's new on Azure IaaS for SQL VMs
Global Azure Virtual 2020 What's new on Azure IaaS for SQL VMsGlobal Azure Virtual 2020 What's new on Azure IaaS for SQL VMs
Global Azure Virtual 2020 What's new on Azure IaaS for SQL VMs
 
NoSQL: Cassadra vs. HBase
NoSQL: Cassadra vs. HBaseNoSQL: Cassadra vs. HBase
NoSQL: Cassadra vs. HBase
 
High Performance Weibo QCon Beijing 2011
High Performance Weibo QCon Beijing 2011High Performance Weibo QCon Beijing 2011
High Performance Weibo QCon Beijing 2011
 
Redis vs Memcached
Redis vs MemcachedRedis vs Memcached
Redis vs Memcached
 
Hadoop World 2011: Building Realtime Big Data Services at Facebook with Hadoo...
Hadoop World 2011: Building Realtime Big Data Services at Facebook with Hadoo...Hadoop World 2011: Building Realtime Big Data Services at Facebook with Hadoo...
Hadoop World 2011: Building Realtime Big Data Services at Facebook with Hadoo...
 
Scylla Summit 2018: In-Memory Scylla - When Fast Storage is Not Fast Enough
Scylla Summit 2018: In-Memory Scylla - When Fast Storage is Not Fast EnoughScylla Summit 2018: In-Memory Scylla - When Fast Storage is Not Fast Enough
Scylla Summit 2018: In-Memory Scylla - When Fast Storage is Not Fast Enough
 
Power Saturday 2019 B6 - SQL Server installation cookbook
Power Saturday 2019 B6 - SQL Server installation cookbookPower Saturday 2019 B6 - SQL Server installation cookbook
Power Saturday 2019 B6 - SQL Server installation cookbook
 
Hadoop World 2011: Apache HBase Road Map - Jonathan Gray - Facebook
Hadoop World 2011: Apache HBase Road Map - Jonathan Gray - FacebookHadoop World 2011: Apache HBase Road Map - Jonathan Gray - Facebook
Hadoop World 2011: Apache HBase Road Map - Jonathan Gray - Facebook
 
Apache HBase Performance Tuning
Apache HBase Performance TuningApache HBase Performance Tuning
Apache HBase Performance Tuning
 
Implementing High Availability Caching with Memcached
Implementing High Availability Caching with MemcachedImplementing High Availability Caching with Memcached
Implementing High Availability Caching with Memcached
 
Scaling PHP apps
Scaling PHP appsScaling PHP apps
Scaling PHP apps
 
[Pgday.Seoul 2018] PostgreSQL 성능을 위해 개발된 라이브러리 OS 소개 apposha
[Pgday.Seoul 2018]  PostgreSQL 성능을 위해 개발된 라이브러리 OS 소개 apposha[Pgday.Seoul 2018]  PostgreSQL 성능을 위해 개발된 라이브러리 OS 소개 apposha
[Pgday.Seoul 2018] PostgreSQL 성능을 위해 개발된 라이브러리 OS 소개 apposha
 
HBase 0.20.0 Performance Evaluation
HBase 0.20.0 Performance EvaluationHBase 0.20.0 Performance Evaluation
HBase 0.20.0 Performance Evaluation
 
微博cache设计谈
微博cache设计谈微博cache设计谈
微博cache设计谈
 
HBase: Extreme Makeover
HBase: Extreme MakeoverHBase: Extreme Makeover
HBase: Extreme Makeover
 
HBaseCon 2012 | HBase and HDFS: Past, Present, Future - Todd Lipcon, Cloudera
HBaseCon 2012 | HBase and HDFS: Past, Present, Future - Todd Lipcon, ClouderaHBaseCon 2012 | HBase and HDFS: Past, Present, Future - Todd Lipcon, Cloudera
HBaseCon 2012 | HBase and HDFS: Past, Present, Future - Todd Lipcon, Cloudera
 
Storage infrastructure using HBase behind LINE messages
Storage infrastructure using HBase behind LINE messagesStorage infrastructure using HBase behind LINE messages
Storage infrastructure using HBase behind LINE messages
 
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
HBaseCon 2012 | Content Addressable Storages for Fun and Profit - Berk Demir,...
 
20080611accel
20080611accel20080611accel
20080611accel
 
HBaseCon 2015: Elastic HBase on Mesos
HBaseCon 2015: Elastic HBase on MesosHBaseCon 2015: Elastic HBase on Mesos
HBaseCon 2015: Elastic HBase on Mesos
 

Viewers also liked (12)

Python于Web 2.0网站的应用 - QCon Beijing 2010
Python于Web 2.0网站的应用 - QCon Beijing 2010Python于Web 2.0网站的应用 - QCon Beijing 2010
Python于Web 2.0网站的应用 - QCon Beijing 2010
 
New Design of OneRing
New Design of OneRingNew Design of OneRing
New Design of OneRing
 
服务框架: Thrift & PasteScript
服务框架: Thrift & PasteScript服务框架: Thrift & PasteScript
服务框架: Thrift & PasteScript
 
OneRing @ OSCamp 2010
OneRing @ OSCamp 2010OneRing @ OSCamp 2010
OneRing @ OSCamp 2010
 
Python高级编程(二)
Python高级编程(二)Python高级编程(二)
Python高级编程(二)
 
DAE
DAEDAE
DAE
 
Python在豆瓣的应用
Python在豆瓣的应用Python在豆瓣的应用
Python在豆瓣的应用
 
合久必分,分久必合
合久必分,分久必合合久必分,分久必合
合久必分,分久必合
 
Data Storage Practice
Data Storage PracticeData Storage Practice
Data Storage Practice
 
Scaling MySQL in Amazon Web Services
Scaling MySQL in Amazon Web ServicesScaling MySQL in Amazon Web Services
Scaling MySQL in Amazon Web Services
 
Build scalable microblog qcon beijing 2010
Build scalable microblog qcon beijing 2010Build scalable microblog qcon beijing 2010
Build scalable microblog qcon beijing 2010
 
淘宝网前端应用与发展
淘宝网前端应用与发展淘宝网前端应用与发展
淘宝网前端应用与发展
 

Similar to 豆瓣技术架构的发展历程 @ QCon Beijing 2009

豆瓣技术架构的发展历程
豆瓣技术架构的发展历程豆瓣技术架构的发展历程
豆瓣技术架构的发展历程
George Ang
 
豆瓣 Qcon2009 Beijing
豆瓣 Qcon2009 Beijing豆瓣 Qcon2009 Beijing
豆瓣 Qcon2009 Beijing
yiditushe
 
WordPress Performance & Scalability
WordPress Performance & ScalabilityWordPress Performance & Scalability
WordPress Performance & Scalability
Joseph Scott
 
Running A Realtime Stats Service On My Sql
Running A Realtime Stats Service On My SqlRunning A Realtime Stats Service On My Sql
Running A Realtime Stats Service On My Sql
PerconaPerformance
 
Make Your Life Easier With Maatkit
Make Your Life Easier With MaatkitMake Your Life Easier With Maatkit
Make Your Life Easier With Maatkit
MySQLConference
 
State of the art - server side JavaScript - web-5 2012
State of the art - server side JavaScript - web-5 2012State of the art - server side JavaScript - web-5 2012
State of the art - server side JavaScript - web-5 2012
Alexandre Morgaut
 
Galera Multi Master Synchronous My S Q L Replication Clusters
Galera  Multi Master  Synchronous  My S Q L  Replication  ClustersGalera  Multi Master  Synchronous  My S Q L  Replication  Clusters
Galera Multi Master Synchronous My S Q L Replication Clusters
PerconaPerformance
 

Similar to 豆瓣技术架构的发展历程 @ QCon Beijing 2009 (20)

Hong Qiangning in QConBeijing
Hong Qiangning in QConBeijingHong Qiangning in QConBeijing
Hong Qiangning in QConBeijing
 
豆瓣技术架构的发展历程
豆瓣技术架构的发展历程豆瓣技术架构的发展历程
豆瓣技术架构的发展历程
 
豆瓣 Qcon2009 Beijing
豆瓣 Qcon2009 Beijing豆瓣 Qcon2009 Beijing
豆瓣 Qcon2009 Beijing
 
EC2ナイトセミナプレゼン資料 Rel2
EC2ナイトセミナプレゼン資料 Rel2EC2ナイトセミナプレゼン資料 Rel2
EC2ナイトセミナプレゼン資料 Rel2
 
Memcached Study
Memcached StudyMemcached Study
Memcached Study
 
Running your Java EE 6 Applications in the Cloud
Running your Java EE 6 Applications in the CloudRunning your Java EE 6 Applications in the Cloud
Running your Java EE 6 Applications in the Cloud
 
2008 MySQL Conference Recap
2008 MySQL Conference Recap2008 MySQL Conference Recap
2008 MySQL Conference Recap
 
OpenSolaris Web Stack MySQL BOF
OpenSolaris Web Stack MySQL BOFOpenSolaris Web Stack MySQL BOF
OpenSolaris Web Stack MySQL BOF
 
Scaling Mapufacture on Amazon Web Services
Scaling Mapufacture on Amazon Web ServicesScaling Mapufacture on Amazon Web Services
Scaling Mapufacture on Amazon Web Services
 
WordPress Performance & Scalability
WordPress Performance & ScalabilityWordPress Performance & Scalability
WordPress Performance & Scalability
 
Running A Realtime Stats Service On My Sql
Running A Realtime Stats Service On My SqlRunning A Realtime Stats Service On My Sql
Running A Realtime Stats Service On My Sql
 
Fisl - Deployment
Fisl - DeploymentFisl - Deployment
Fisl - Deployment
 
Lightweight Grids With Terracotta
Lightweight Grids With TerracottaLightweight Grids With Terracotta
Lightweight Grids With Terracotta
 
Make Your Life Easier With Maatkit
Make Your Life Easier With MaatkitMake Your Life Easier With Maatkit
Make Your Life Easier With Maatkit
 
State of the art - server side JavaScript - web-5 2012
State of the art - server side JavaScript - web-5 2012State of the art - server side JavaScript - web-5 2012
State of the art - server side JavaScript - web-5 2012
 
Running your Java EE 6 applications in the Cloud (FISL 12)
Running your Java EE 6 applications in the Cloud (FISL 12)Running your Java EE 6 applications in the Cloud (FISL 12)
Running your Java EE 6 applications in the Cloud (FISL 12)
 
JFokus 2011 - Running your Java EE 6 apps in the Cloud
JFokus 2011 - Running your Java EE 6 apps in the CloudJFokus 2011 - Running your Java EE 6 apps in the Cloud
JFokus 2011 - Running your Java EE 6 apps in the Cloud
 
Galera Multi Master Synchronous My S Q L Replication Clusters
Galera  Multi Master  Synchronous  My S Q L  Replication  ClustersGalera  Multi Master  Synchronous  My S Q L  Replication  Clusters
Galera Multi Master Synchronous My S Q L Replication Clusters
 
What's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea Luzzardi
What's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea LuzzardiWhat's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea Luzzardi
What's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea Luzzardi
 
What's New in Docker 1.12 by Mike Goelzer and Andrea Luzzardi
What's New in Docker 1.12 by Mike Goelzer and Andrea LuzzardiWhat's New in Docker 1.12 by Mike Goelzer and Andrea Luzzardi
What's New in Docker 1.12 by Mike Goelzer and Andrea Luzzardi
 

Recently uploaded

Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
panagenda
 

Recently uploaded (20)

Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptxBT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 

豆瓣技术架构的发展历程 @ QCon Beijing 2009

  • 2. 2005 3 • • •
  • 3. • 2.9M 1/4 • • 23M / 500~600/sec • 23 PC (1U*15/2U*8) • 12 • 38G memcached
  • 4. 1U (frodo) • AMD Athlon 64 1.8GHz • 1G 160G SATA*2 • Gentoo Linux • MySQL 5 • Quixote (a Python web framework) • Lighttpd + SCGI (shire) • Memcached (!)
  • 5. Gentoo Linux • • emerge mysql • ebuild patch • • • GLSA(Gentoo Linux Security Advisories)
  • 6. MySQL • The world’s most popular open source database • / ==> MyISAM • ==> InnoDB • Replicate for backup
  • 7. Python • • Battery Included • • • CPUG: http://python.cn/
  • 8. Quixote • REST URL • Django, TurboGears, Pylons ZOPE • http://www.douban.com/subject/1000001 # luz/subject/__init__.py def _q_lookup(request, name): subject = get_subject(name) return lambda req: subject_ui(req, subject) # luz/subject/subject_ui.ptl def subject_ui [html] (request, subject): site_header(request) “<h1>%s</h1>” % subject.title site_footer(request)
  • 9. Lighttpd • • SCGI • SCGI: FastCGI Quixote • 80 lighttpd SCGI localhost Quixote
  • 10. Memcache • MySQL • libmemcache python Pyrex python 3x+ def get_subject(subject_id): subject = mc.get(‘s:’+subject_id) if subject is None: store.farm.execute(“select xxx, xxx from subject where id=%s”, subject_id) subject = Subject(*store.farm.fetchone()) mc.set(‘s:’+subject_id, subject) return subject
  • 11. Internet Lighttpd SCGI App FS MySQL Memcache Static Files
  • 12. • 1.2M / • IO •
  • 13. 1U • pippin meriadoc ( merry) • , 4G 250G SATA*3 • • IP DNS IP -_-b • frodo (subversion, trac, etc...)
  • 14. Internet Lighttpd (#$) DNS HTTP Proxy SCGI App Lighttpd (!quot;) FS Static Files Memcache MySQL
  • 15. • innodb_buffer_pool_size • • IP
  • 16. • 1.5M / • • IP
  • 17. IP(BGP) • (arwen) • 74G 1w SATA * 3 • •
  • 18. Internet Data Lighttpd Mining read SCGI write App Replicate MySQL MySQL Master Slave Memcache Static Files
  • 19. • 2M / • IO • , etc... •
  • 20. 4G 250G SATA*3 • 10000 • mod_rewrite URL • lighttpd mod_memcache • IO • web • •
  • 21. Internet store.farm App MySQL SCGI Master Lighttpd Memcache Replicate store.farmr HTTP Proxy WebDAV Web Service Replicate Lighttpd (w/ mod_memcache) MySQL write Slave Spiders Lighttpd WebDAV Memcache Static Files !quot;#$% read Data Mining MySQL Slave
  • 22. store farmr • replicate delay • • • cache • • • ...... but it works
  • 23. replicate delay def get_subject(sid): sbj = mc.get(‘s:’+sid) if sbj is None: sbj = flush_subject(sid, store.farmr) return sbj def flush_subject(sid, cursor=None): cursor = cursor or store.farm cursor.execute(“select ... from subject”) subject = Subject(*cursor.fetchone()) mc.set(‘s:’+sid, subject) return subject def update_subject(subject, props): store.farm.execute(“update subject ...”) store.farm.connection.commit() flush_subject(subject.id, store.farm)
  • 24. • 2.5M / • • / • SATA •
  • 25. Scale Up 1U • 16G 147G SCSI *2 + 500G SATA • SCSI RAID-0 • MySQL Slave • memcached • MyISAM InnoDB • • Sphinx
  • 26. Internet Sphinx MySQL Master store.farm Web Service Replicate Lighttpd SCGI App HTTP Proxy Memcache Lighttpd (w/ mod_memcache) WebDAV store.farmr Memcache Lighttpd Web Service WebDAV MySQL Static Files Slave Spiders Memcache Memcache
  • 27. • 5.2M / • • Web IO • •
  • 28. :) • • • • 3 1U 4 32G 1T SATA * 3 • otho.douban.com lotho.douban.com • lighttpd 1.5 with aio support • LVS • Scale Up: 4G -> 8G
  • 29. Internet www.douban.com otho.douban.com LVS LB (Master) Lighttpd Lighttpd 1.5 (w/ mod_cache) HTTP Proxy Lighttpd Keepalived LVS LB (backup) Lighttpd Lighttpd 1.5 WebDAV (w/ mod_cache) Static Files
  • 30. write !quot;#$% replicate read Data Mining MySQL MySQL Slave Master read Replicate write !quot;#$% Data Mining MySQL Slave
  • 31. • 6.4M / (5M PV) • • • CPU memcache /
  • 32. • lighttpd mod_scgi round-robin • lighttpd 1.5 • mod_proxy • proxy.balance = fair (load based, passive balancing) • • spread
  • 33. Internet Lighttpd SCGI HTTP Proxy App Memcache Lighttpd HTTP Proxy spread HTTP Proxy Lighttpd Lighttpd Log Aggregator SCGI spread Lighttpd Memcache App WebDAV Static Files
  • 34. • 11M / 3 • • Sphinx • • load
  • 35. • • • Sphinx -> Xapian • MogileFS
  • 36. !quot; Master replicate replicate !quot; %&'() Slave #$ Master read replicate Data #$ read Slave Mining write *+,- !quot; Master Slave replicate *+,- Slave write replicate Data %&'() Mining #$ Slave
  • 37. • • store.farm[r] -> store.get_cursor(table=‘xxx’, ro=True/False) def flush_subject(sid, ro=False): cursor = store.get_cursor(table=‘subject’, ro=ro) cursor.execute(“select ... from subject”) subject = Subject(*cursor.fetchone()) mc.set(‘s:’+sid, subject) return subject •
  • 38. Internet upload.douban.com otho.douban.com nginx signed POST form Uploader (w/ proxy_store) HTTP redirect FileStorage MogileFS Tracker Gateway App MogileFS MogileFS Node Node MogileFS Master
  • 39. libmemcache -> libmemcached consistent hash memcache • libmemcached consistent hash bug • CPU • libmemcached failover bug • nginx lighttpd load balance • spread • nginx
  • 40. Consistent Hash charlee http:// tech.idv2.com/2008/07/24/ memcached-004/
  • 41. Internet App SCGI Lighttpd HTTP Proxy Lighttpd HTTP Proxy App HTTP Proxy SCGI Lighttpd HTTP Proxy Nginx Lighttpd HTTP Proxy App Lighttpd SCGI Lighttpd WebDAV Static Files
  • 42. • 13M / • MogileFS • Tracker DB •
  • 43. 8 • 32G CPU • (300G SCSI×2 + 1T SATA) × 3 • (1T SATA × 3) × 5 •6 2 • DoubanFS
  • 44. DoubanFS • hash • hash hash • Merkle Tree • consistent hash • WebDAV • MogileFS 3 50
  • 46. Internet upload.douban.com otho.douban.com nginx signed POST form Uploader (w/ proxy_store) HTTP redirect FileStorage Gateway App DoubanFS DoubanFS Node Node
  • 47. • 16M / • • DoubanFS IO •
  • 48. DoubanDB • • MySQL • MySQL Master • failover • replicate delay
  • 49. DoubanDB • Key-Value • Amazon Dynamo • set(key, value), get(key), delete(key) • memcache • Merkle Tree • Consistent Hash • TokyoCabinet • DoubanDB • DoubanFS 2.0 DoubanDB
  • 51. !quot; Master1 replicate replicate !quot; %&'() Slave #$ Master1 read replicate Data #$ read Slave Mining write *+,- !quot; Master Master2 replicate *+,- Slave write replicate Data %&'() Mining #$ Master2
  • 52. DoubanFS • • • ngnix • www.douban.com LVS • RabbitMQ spread
  • 53. • • profile • memcache cache • join • • •