Running the World’s Internet Servers




 MongoDB
 Chinese Use Cases

 By Steve Mushero
 CEO of ChinaNetCloud



      Server Management Ÿ Cloud computing Ÿ Audit & Optimization   Copyright © 2013 ChinaNetCloud
About Steve

              Founder, CEO & CTO of ChinaNetCloud
                 •  Before that, CTO of Tudou (土豆网)

              8 years in China
                 •  Lived in Silicon Valley, Seattle, and New York

              25 years experience as a CTO
                 •  Experience in development and operations
                 •  Decades of DBA, performance, security experience
                 •  Remembers flat file, network, IMS databases




                 Running the World’s Internet Servers   www.ChinaNetCloud.com   2
About ChinaNetCloud

§  Founded in Shanghai in 2008
   •  By Silicon Valley Technology Guys

§  Vision: Run All the World's Internet Servers
   •  It’s out job to run the Internet

§  We manage servers and systems
   •  For Chinese Internet and Game Companies

§  Thousands of Servers - Hundreds of Customers
   •  We've seen systems of every type and size

•  We are hiring – Linux Engineers / System Admins
                            Running the World’s Internet Servers   www.ChinaNetCloud.com   3
Chinese Internet Situation


                                            Mostly PHP & MySQL


                                                Some MongoDB
      Databases
                                               Less PostgreSQL


                                                  A bit of Oracle

                                                Hadoop & things



                  Running the World’s Internet Servers      www.ChinaNetCloud.com
Chinese Internet Situation II

                         Still growing fast

                         Startups move fast

    Startups &           Value flexibility (schema-free companies)
   Applications
                         Resume-building helps adoption

                         Lots of mobile apps in 2012/2013

                         Lots of new startups

                         E-Commerce FINALLY growing

                         But lots of dating, SNS, games, etc.


                  Running the World’s Internet Servers   www.ChinaNetCloud.com
MongoDB General Use Cases

      1   Flexible-Structure Document Database


      2   Non-relational Database, networks, analysis


      3   High-Performance Database

      4   Replication with multi-Master Write

      5   Sharding for large data sets

      6   Basic datastore to replace RDBMS

                  Running the World’s Internet Servers   www.ChinaNetCloud.com
Major Challenges

      Stability

      Locking

      Tuning

      Competition from Redis, MySQL

      In China, lack of skilled engineers

      In China, Hadoop suddenly sexy


                     Running the World’s Internet Servers   www.ChinaNetCloud.com   7
Cases

        Mobile Gaming


        SNS Travel


        IPTV Company


        Baby Care Site


        Job & Resume Site



                     Running the World’s Internet Servers   www.ChinaNetCloud.com   8
Big Game Company

   Why         §     Big data for logs, Users to a lot of things
               §     Unstructured logs


                §    Good
Experience      §    User log analysis and data
                §    Core data simple key-value, Redis/MySQL


               §     Lazy write to DB, first in Memcache                    q  Runs several mobile games
Interesting    §     Works well for write-heavy systems
                        §  Also for Game status data, all write
                                                                             q  Each in several countries

Problems      §      Early on stability, locking                            q  Most on several platforms
              §      Now runs well



              §      High-speed, schema-free
 Pro/Con      §      Slow to restart, reload data




                                      Running the World’s Internet Servers           www.ChinaNetCloud.com   9
SNS Travel Site / App

              §    Schema free
   Why
              §    High Performance



              §    Good
Experience    §    Core data system, relatively unusual


              §    Core data system
Interesting   §    Core user relationships & travel plans
              §    Backup every hour via Mongo Dump for safety
                                                                         q  Lots of data

                                                                         q  Lots of pictures
              §    Version 1.6 crashed and lost all data
 Problems     §    Now runs well with 2.0                               q  Complex travel


 Pro/Con      §    Schema-free
              §    Need experienced people when there are problems




                                  Running the World’s Internet Servers   www.ChinaNetCloud.com   10
IPTV Company

              §    Complements MySQL
   Why        §    For unstructured & business-related data



              §    Good with 2.0 now
Experience    §    Lock time ratios high for a long time, 75%+
              §    User log analysis and related data


              §    Find high performance IO powerful in 2.0
Interesting   §    Load only 100 requests/second, multi-instance       q  Lots of users
              §    Mostly 32GB RAM, medium-sized
                                                                        q  Lots of data

 Problems     §    1.x Stability issues                                q  High uptime
              §    Lack of experts to help them



              §    Complex analysis easy
 Pro/Con      §    Lack of skills



                                 Running the World’s Internet Servers   www.ChinaNetCloud.com   11
Baby Care Site
              §    Performance to 50M PV/day
   Why        §    Lots of data (~100GB already)
              §    Sharding if needed


              §    Good so far
Experience    §    Not really scaled yet
              §    Data growing fast (comments / BBS)



              §    Lots of text / comments
Interesting   §    No MySQL or other backing DB                        q  Fast

                                                                        q  Large data

 Problems     §    Locks, server seems normal                          q  Flexible
                      §  Have to restart – ongoing issue



              §    Schema Free data
 Pro/Con      §    Closely matches business model


                                 Running the World’s Internet Servers   www.ChinaNetCloud.com   12
Job & Hiring Site
               §    Very fast development / limited time
    Why        §    Flexibility as needs / app changing
               §    Popular with staff, improve resumes


               §    Mostly 64 GB RAM
Experience     §    Medium load – 500-1500 requests/sec
               §    Random Global Locks suddenly in 2.0



               §    Using sharding
 Interesting   §    Using replication to balance load                   q  Complex data
               §    Also tracking logs, analyis

                                                                         q  High concurrency
               §    Locks, server seems normal
 Problems
                       §  Have to restart – ongoing issue               q  Constant change
               §    Lack experience on tuning
               §    RAM overflow on Master

               §    Schema Free data
               §    Good for storing OO objects in DB
  Pro/Con      §    Closely matches business model
               §    Stability Issue / Manageability

                                  Running the World’s Internet Servers   www.ChinaNetCloud.com   13
Summary

§  MongoDB is new in China
§  MongoDB is sexy in China
§  Lots of interest

§  Well-suited for many startups
   §  Especially for flexibility and speed

§  Some challenges
   §  Stability
   §  2.x much better

§  Lack of talent
   §  More events !

§  Bright future in China

                              Running the World’s Internet Servers   www.ChinaNetCloud.com   14
Questions




    ?
            Running the World’s Internet Servers   www.ChinaNetCloud.com

How MongoDB is Being Used in China - Case Studies

  • 1.
    Running the World’sInternet Servers MongoDB Chinese Use Cases By Steve Mushero CEO of ChinaNetCloud Server Management Ÿ Cloud computing Ÿ Audit & Optimization Copyright © 2013 ChinaNetCloud
  • 2.
    About Steve Founder, CEO & CTO of ChinaNetCloud •  Before that, CTO of Tudou (土豆网) 8 years in China •  Lived in Silicon Valley, Seattle, and New York 25 years experience as a CTO •  Experience in development and operations •  Decades of DBA, performance, security experience •  Remembers flat file, network, IMS databases Running the World’s Internet Servers www.ChinaNetCloud.com 2
  • 3.
    About ChinaNetCloud §  Foundedin Shanghai in 2008 •  By Silicon Valley Technology Guys §  Vision: Run All the World's Internet Servers •  It’s out job to run the Internet §  We manage servers and systems •  For Chinese Internet and Game Companies §  Thousands of Servers - Hundreds of Customers •  We've seen systems of every type and size •  We are hiring – Linux Engineers / System Admins Running the World’s Internet Servers www.ChinaNetCloud.com 3
  • 4.
    Chinese Internet Situation Mostly PHP & MySQL Some MongoDB Databases Less PostgreSQL A bit of Oracle Hadoop & things Running the World’s Internet Servers www.ChinaNetCloud.com
  • 5.
    Chinese Internet SituationII Still growing fast Startups move fast Startups & Value flexibility (schema-free companies) Applications Resume-building helps adoption Lots of mobile apps in 2012/2013 Lots of new startups E-Commerce FINALLY growing But lots of dating, SNS, games, etc. Running the World’s Internet Servers www.ChinaNetCloud.com
  • 6.
    MongoDB General UseCases 1 Flexible-Structure Document Database 2 Non-relational Database, networks, analysis 3 High-Performance Database 4 Replication with multi-Master Write 5 Sharding for large data sets 6 Basic datastore to replace RDBMS Running the World’s Internet Servers www.ChinaNetCloud.com
  • 7.
    Major Challenges Stability Locking Tuning Competition from Redis, MySQL In China, lack of skilled engineers In China, Hadoop suddenly sexy Running the World’s Internet Servers www.ChinaNetCloud.com 7
  • 8.
    Cases Mobile Gaming SNS Travel IPTV Company Baby Care Site Job & Resume Site Running the World’s Internet Servers www.ChinaNetCloud.com 8
  • 9.
    Big Game Company Why §  Big data for logs, Users to a lot of things §  Unstructured logs §  Good Experience §  User log analysis and data §  Core data simple key-value, Redis/MySQL §  Lazy write to DB, first in Memcache q  Runs several mobile games Interesting §  Works well for write-heavy systems §  Also for Game status data, all write q  Each in several countries Problems §  Early on stability, locking q  Most on several platforms §  Now runs well §  High-speed, schema-free Pro/Con §  Slow to restart, reload data Running the World’s Internet Servers www.ChinaNetCloud.com 9
  • 10.
    SNS Travel Site/ App §  Schema free Why §  High Performance §  Good Experience §  Core data system, relatively unusual §  Core data system Interesting §  Core user relationships & travel plans §  Backup every hour via Mongo Dump for safety q  Lots of data q  Lots of pictures §  Version 1.6 crashed and lost all data Problems §  Now runs well with 2.0 q  Complex travel Pro/Con §  Schema-free §  Need experienced people when there are problems Running the World’s Internet Servers www.ChinaNetCloud.com 10
  • 11.
    IPTV Company §  Complements MySQL Why §  For unstructured & business-related data §  Good with 2.0 now Experience §  Lock time ratios high for a long time, 75%+ §  User log analysis and related data §  Find high performance IO powerful in 2.0 Interesting §  Load only 100 requests/second, multi-instance q  Lots of users §  Mostly 32GB RAM, medium-sized q  Lots of data Problems §  1.x Stability issues q  High uptime §  Lack of experts to help them §  Complex analysis easy Pro/Con §  Lack of skills Running the World’s Internet Servers www.ChinaNetCloud.com 11
  • 12.
    Baby Care Site §  Performance to 50M PV/day Why §  Lots of data (~100GB already) §  Sharding if needed §  Good so far Experience §  Not really scaled yet §  Data growing fast (comments / BBS) §  Lots of text / comments Interesting §  No MySQL or other backing DB q  Fast q  Large data Problems §  Locks, server seems normal q  Flexible §  Have to restart – ongoing issue §  Schema Free data Pro/Con §  Closely matches business model Running the World’s Internet Servers www.ChinaNetCloud.com 12
  • 13.
    Job & HiringSite §  Very fast development / limited time Why §  Flexibility as needs / app changing §  Popular with staff, improve resumes §  Mostly 64 GB RAM Experience §  Medium load – 500-1500 requests/sec §  Random Global Locks suddenly in 2.0 §  Using sharding Interesting §  Using replication to balance load q  Complex data §  Also tracking logs, analyis q  High concurrency §  Locks, server seems normal Problems §  Have to restart – ongoing issue q  Constant change §  Lack experience on tuning §  RAM overflow on Master §  Schema Free data §  Good for storing OO objects in DB Pro/Con §  Closely matches business model §  Stability Issue / Manageability Running the World’s Internet Servers www.ChinaNetCloud.com 13
  • 14.
    Summary §  MongoDB isnew in China §  MongoDB is sexy in China §  Lots of interest §  Well-suited for many startups §  Especially for flexibility and speed §  Some challenges §  Stability §  2.x much better §  Lack of talent §  More events ! §  Bright future in China Running the World’s Internet Servers www.ChinaNetCloud.com 14
  • 15.
    Questions ? Running the World’s Internet Servers www.ChinaNetCloud.com