LVS Project: 14
              Years Development
              and Experience

                            Wensong Zhang
                    The 11th Northeast Asia OSS Promotion Forum
                                     2012.11.14




The 11th Northeast Asia
                                        Page 1
OSS Promotion Forum
About Me
        Wensong Zhang,
         Ph.D.
        Working at Alibaba,
         leading software
         infrastructure team
        LVS founder &
         developer
        Was the chief scientist
         and co-founder of
         TelTel, an associate
         professor at NUDT,
         the co-founder of
         ChinaCluster, and Red
         Hat Kernel Developer



The 11th Northeast Asia
                                   Page 2
OSS Promotion Forum
LVS Framework

     The Linux Virtual Server
     Project
      The   goal of the Linux Virtual Server
         project
            to provide a basic framework for building
             highly scalable and highly available
             network services using a large cluster of
             commodity servers.
      http://www.LinuxVirtualServer.org


The 11th Northeast Asia
                               Page 3
OSS Promotion Forum
LVS Framework

     Framework




The 11th Northeast Asia
                          Page 4
OSS Promotion Forum
LVS Framework

     IP Virtual Server
      Implemented   in the Linux kernel
      Three IP load balancing techniques
            Virtual Server via NAT
            Virtual Server via IP Tunneling
            Virtual Server via Direct Routing
      Ten          scheduling algorithms
          IPVS is an advanced Layer-4 switching
The 11th Northeast Asia
                                 Page 5
OSS Promotion Forum
Building Scalable Network Service using LVS

     General Architecture
                                       3   tiers
                                          Load balancer
                                          Server cluster
                                          Shared storage
                                        Clusters
                                          Web
                                          Cache
                                          Email, etc
The 11th Northeast Asia
                              Page 6
OSS Promotion Forum
Characteristics of LVS
     Characteristics
      LVS  extends Linux kernel to support
       three IP load balancing techniques
      Ten scheduling algorithms
      High scalability (up to 100 nodes)

      High availability

      Supporting most TCP and UDP
       services, no modifications to either
       clients or servers
The 11th Northeast Asia
                              Page 7
OSS Promotion Forum
Characteristics of LVS
     Compared to Other
     Commercial Products
      More  IP load balancing techniques
      Multiple scheduling algorithms

      State synchronization

      A robust and stable code base, a large
       user and developer base.
      Reliability proven in big real world
       applications
      Free to everyone
The 11th Northeast Asia
                              Page 8
OSS Promotion Forum
Comments from LVS Users
        “ We tried virtually all of the commercial load
         balancers, LVS beats them all for reliability, cost,
         manageability, you-name-it”
       Jerry Glomph Black, Director, Internet & Technical Operations, Real
        Networks, Seattle Washington, USA
         http://marc.theaimsgroup.com/?1=linux-virtual-server&m=95385809030794&w=2

        “I can say without a doubt that lvs toasts F5/BigIP
         solutions, at least in our real world
         implementations. I wouldn’t trade a good lvs box for
         a Cisco Local Director either”
         Drew Streib, Information Architect, VA Linux Systems, USA
         http://marc.theaimsgroup.com/?1=linux-virtual-server&m=95385694529750&w=2
The 11th Northeast Asia
                                               Page 9
OSS Promotion Forum
Some Sites using LVS
      QQ.com   has deployed thousands of LVS
       in the production
      Alibaba has used about 500 LVS systems
      Baidu.com has used about 500 too
      sourceforge.net & linux.com
      real.com
      IBM and EDS use LVS in their solutions
      One of largest PC manufacturers
      There is about tens of thousands of LVS
       deployment on this planet
The 11th Northeast Asia
                          Page 10
OSS Promotion Forum
LVS Development
        I got my first Linux CD at the end of 1995, mainly
         installed Linux for fun, later used Linux servers in
         some SI solutions.
        May 1998: the first version of LVS was released
         as a patch for Linux kernel 2.0, which only
         supports VS/NAT and Round-Robin scheduling,
         very simple!
        Later, added more scheduling algorithms, and
         included some bug fixes, and so on.
        Nov 1998: implemented the VS/IPTunneling load
         balancing.
        May 1999: implemented the VS/DirectRouting
         load balancing, then ported it to the Linux kernel
         2.2

The 11th Northeast Asia
OSS Promotion Forum
                                 Page 11
                                                 11
LVS Development (cont’d)
      More and more people joined the project.
      May 2000: ported to the Linux kernel 2.4
      Sep 2002: ported to the Linux kernel 2.5
      Oct 2003: included into the Linux kernel 2.5
       devel trunk, and into the Linux kernel 2.4
       stable branch too.
      Nov 2003: Linux kernel 2.4.23 includes LVS
      Dec 2003: Linux kernel 2.6.0 includes LVS
      Nov 2008: Google engineers helped to add
       IPv6 support
      Feb 2010: SCTP protocol support was added


The 11th Northeast Asia
OSS Promotion Forum
                            Page 12
                                          12
LVS Global Collaboration




        Source: WIRED Magazine, Vol. 6, 2004

The 11th Northeast Asia
OSS Promotion Forum
                                    Page 13
                                                13
Some Insights
        It’s better to do things well first, then reward
         follows.
        Open source is a good way to get more users,
         users will test and give feedbacks.
        Open source is also a good way to collect
         requirements.
        Through open source development, we can get
         help from a lot of developers and talents, we can
         improve our skills through collaboration.
         Software quality is improved as well.
        Open source can make our code to have much
         longer lifecycle.



The 11th Northeast Asia
OSS Promotion Forum
                                Page 14
                                               14
Some Insights (Cont’d)
      The   ways that users run our software
       usually beyond our imagination.
      Do not think that it’s too hard to do open
       source, just write software and open
       source it.
      It’s important to write documents for
       open source projects, especially the
       QuickStart.
      It’s also very important to give
       acknowledgements to every contribution,
       then community can grow.

The 11th Northeast Asia
OSS Promotion Forum
                          Page 15
                                       15
Collateral
      The          Linux Virtual Server Project




                          (http://www.LinuxVirtualServer.org)
      wensong@linux-vs.org
The 11th Northeast Asia
                                           Page 16
OSS Promotion Forum
Q & A


                          Thanks!




The 11th Northeast Asia
                             Page 17
OSS Promotion Forum

LVS development and experience

  • 1.
    LVS Project: 14 Years Development and Experience Wensong Zhang The 11th Northeast Asia OSS Promotion Forum 2012.11.14 The 11th Northeast Asia Page 1 OSS Promotion Forum
  • 2.
    About Me  Wensong Zhang, Ph.D.  Working at Alibaba, leading software infrastructure team  LVS founder & developer  Was the chief scientist and co-founder of TelTel, an associate professor at NUDT, the co-founder of ChinaCluster, and Red Hat Kernel Developer The 11th Northeast Asia Page 2 OSS Promotion Forum
  • 3.
    LVS Framework The Linux Virtual Server Project  The goal of the Linux Virtual Server project  to provide a basic framework for building highly scalable and highly available network services using a large cluster of commodity servers.  http://www.LinuxVirtualServer.org The 11th Northeast Asia Page 3 OSS Promotion Forum
  • 4.
    LVS Framework Framework The 11th Northeast Asia Page 4 OSS Promotion Forum
  • 5.
    LVS Framework IP Virtual Server  Implemented in the Linux kernel  Three IP load balancing techniques  Virtual Server via NAT  Virtual Server via IP Tunneling  Virtual Server via Direct Routing  Ten scheduling algorithms IPVS is an advanced Layer-4 switching The 11th Northeast Asia Page 5 OSS Promotion Forum
  • 6.
    Building Scalable NetworkService using LVS General Architecture 3 tiers  Load balancer  Server cluster  Shared storage  Clusters  Web  Cache  Email, etc The 11th Northeast Asia Page 6 OSS Promotion Forum
  • 7.
    Characteristics of LVS Characteristics  LVS extends Linux kernel to support three IP load balancing techniques  Ten scheduling algorithms  High scalability (up to 100 nodes)  High availability  Supporting most TCP and UDP services, no modifications to either clients or servers The 11th Northeast Asia Page 7 OSS Promotion Forum
  • 8.
    Characteristics of LVS Compared to Other Commercial Products  More IP load balancing techniques  Multiple scheduling algorithms  State synchronization  A robust and stable code base, a large user and developer base.  Reliability proven in big real world applications  Free to everyone The 11th Northeast Asia Page 8 OSS Promotion Forum
  • 9.
    Comments from LVSUsers  “ We tried virtually all of the commercial load balancers, LVS beats them all for reliability, cost, manageability, you-name-it” Jerry Glomph Black, Director, Internet & Technical Operations, Real Networks, Seattle Washington, USA http://marc.theaimsgroup.com/?1=linux-virtual-server&m=95385809030794&w=2  “I can say without a doubt that lvs toasts F5/BigIP solutions, at least in our real world implementations. I wouldn’t trade a good lvs box for a Cisco Local Director either” Drew Streib, Information Architect, VA Linux Systems, USA http://marc.theaimsgroup.com/?1=linux-virtual-server&m=95385694529750&w=2 The 11th Northeast Asia Page 9 OSS Promotion Forum
  • 10.
    Some Sites usingLVS  QQ.com has deployed thousands of LVS in the production  Alibaba has used about 500 LVS systems  Baidu.com has used about 500 too  sourceforge.net & linux.com  real.com  IBM and EDS use LVS in their solutions  One of largest PC manufacturers  There is about tens of thousands of LVS deployment on this planet The 11th Northeast Asia Page 10 OSS Promotion Forum
  • 11.
    LVS Development  I got my first Linux CD at the end of 1995, mainly installed Linux for fun, later used Linux servers in some SI solutions.  May 1998: the first version of LVS was released as a patch for Linux kernel 2.0, which only supports VS/NAT and Round-Robin scheduling, very simple!  Later, added more scheduling algorithms, and included some bug fixes, and so on.  Nov 1998: implemented the VS/IPTunneling load balancing.  May 1999: implemented the VS/DirectRouting load balancing, then ported it to the Linux kernel 2.2 The 11th Northeast Asia OSS Promotion Forum Page 11 11
  • 12.
    LVS Development (cont’d)  More and more people joined the project.  May 2000: ported to the Linux kernel 2.4  Sep 2002: ported to the Linux kernel 2.5  Oct 2003: included into the Linux kernel 2.5 devel trunk, and into the Linux kernel 2.4 stable branch too.  Nov 2003: Linux kernel 2.4.23 includes LVS  Dec 2003: Linux kernel 2.6.0 includes LVS  Nov 2008: Google engineers helped to add IPv6 support  Feb 2010: SCTP protocol support was added The 11th Northeast Asia OSS Promotion Forum Page 12 12
  • 13.
    LVS Global Collaboration  Source: WIRED Magazine, Vol. 6, 2004 The 11th Northeast Asia OSS Promotion Forum Page 13 13
  • 14.
    Some Insights  It’s better to do things well first, then reward follows.  Open source is a good way to get more users, users will test and give feedbacks.  Open source is also a good way to collect requirements.  Through open source development, we can get help from a lot of developers and talents, we can improve our skills through collaboration. Software quality is improved as well.  Open source can make our code to have much longer lifecycle. The 11th Northeast Asia OSS Promotion Forum Page 14 14
  • 15.
    Some Insights (Cont’d)  The ways that users run our software usually beyond our imagination.  Do not think that it’s too hard to do open source, just write software and open source it.  It’s important to write documents for open source projects, especially the QuickStart.  It’s also very important to give acknowledgements to every contribution, then community can grow. The 11th Northeast Asia OSS Promotion Forum Page 15 15
  • 16.
    Collateral  The Linux Virtual Server Project (http://www.LinuxVirtualServer.org)  wensong@linux-vs.org The 11th Northeast Asia Page 16 OSS Promotion Forum
  • 17.
    Q & A Thanks! The 11th Northeast Asia Page 17 OSS Promotion Forum