SDEC2011 Using Couchbase for social game scaling and speed
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

SDEC2011 Using Couchbase for social game scaling and speed

on

  • 4,940 views

A social game, by it's very nature, can spread very quickly to a large user population. Because the game is typically interactive, the speed of retrieving information needed for the user's ...

A social game, by it's very nature, can spread very quickly to a large user population. Because the game is typically interactive, the speed of retrieving information needed for the user's interactions with the system is critical. When building their new game Animal Party, the developers at Tribal Crossing needed to get away from the complexity of sharding an SQL database. They also were looking for a solution to the administration cost associated with the operation of traditional data stores. When evaluating multiple different NoSQL solutions, they realized that Couchbase's Membase server meets most of their critical requirements in developing their game software. Simple to use, Couchbase's model allows Tribal Crossing to easily model their game interactions with the key/value data store. Fast read and write performance is required with interactive, social games, and they found that support in Membase as well. Elastic scalability is easily achieved by simply adding more nodes to the Couchbase cluster without any modifications required to the application. Relying on Couchbase's technology Tribal Crossing has been able to quickly build and scale Animal Party with a small team and no dedicated system administrators.

http://sdec.kr/

Statistics

Views

Total Views
4,940
Views on SlideShare
4,939
Embed Views
1

Actions

Likes
8
Downloads
242
Comments
0

1 Embed 1

http://twitter.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

SDEC2011 Using Couchbase for social game scaling and speed Presentation Transcript

  • 1. 1  
  • 2. USING  COUCHBASE  FOR  SOCIAL   GAME  SCALING  AND  SPEED     Chiyoung Seo, Couchbase Inc. Matt Ingenthron, Couchbase Inc. 2  
  • 3. Agenda   •  Introduc-on   •  What  is  Couchbase  Server?   –  Simple,  Fast,  Elas-c   –  Technology  Overview  (Architecture,  data  flow,  rebalancing)   •  Tribal  Crossing  Inc:  Animal  Party   –  Challenges  before  Couchbase   •  Original  Architecture   –  Why  Couchbase?   •  Simplicity   •  Performance   •  Flexibility   –  Deploying  Couchbase   •  New  Architecture   •  EC2   •  Data  Model   •  Accessing  data  in  Couchbase   •  Product  Roadmap   •  Q&A   3  
  • 4. Couchbase  Inc.  •  Membase  and  CouchOne  have  merged  to  form  Couchbase   Inc.  (headquartered  in  Silicon  Valley)  •  Team   –  Brings  together  the  creators  and  core  contributors  of  Memcached,   Membase  and  CouchDB  technologies   –  Doubles  technical  team  size,  accelerates  roadmaps  by  over  a  year  •  Products   –  Couchbase  Server  (Formerly  Membase)   –  Couchbase  Single  Server   –  Mobile  Couchbase  (iPhone  and  Android)  •  Technology   –  Most  mature,  reliable  and  widely  deployed  NoSQL  technologies   –  Fully  featured,  open  source  document  datastore   –  First  complete,  end-­‐to-­‐end  NoSQL  database  product   4  
  • 5. Modern Interactive Web Application Architecture Application Scales Out Just add more commodity web servers www.facebook.com/animalparty   Load  Balancer   Web  Servers   Rela-onal     Database Scales Up Get a bigger, more complex server Database  -­‐  Expensive  and  disrup-ve  sharding  -­‐  Doesn’t  perform  at  Web  Scale   5  
  • 6. Couchbase  Server  is  a  distributed  database   Couchbase  Web  Console   Applica-on  user   Web    applica-on    server   Couchbase  Servers   6  
  • 7. Couchbase data layer scales like application logic tierData layer now scales with linear cost and constant performance. Application Scales Out www.facebook.com/animalparty   Just add more commodity web servers Load  Balancer   Web  Servers   Couchbase    Servers   Database Scales Out Just add more commodity data servers Horizontally  scalable,  schema-­‐less,   auto-­‐sharding,  high-­‐performance  at   Web  Scale   Scaling out flattens the cost and performance curves. 7  
  • 8. Couchbase  Server  is  Simple,  Fast,  Elas-c   •  Five  minutes  or  less  to  a  working  cluster   –  Downloads  for  Windows,  Linux  and  OSX   –  Start  with  a  single  node   –  One  bucon  press  joins  nodes  to  a  cluster   •  Easy  to  develop  against   –  Just  SET  and  GET  –  no  schema  required   –  Drop  it  in.  10,000+  exis-ng  applica-ons   already  “speak  Couchbase”  (via  memcached)   –  Prac-cally  every  language  and  applica-on   framework  is  supported,  out  of  the  box   •  Easy  to  manage   –  One-­‐click  failover  and  cluster  rebalancing   –  Graphical  and  programma-c  interfaces   –  Configurable  aler-ng   8  
  • 9. Couchbase  Server  is  Simple,  Fast,  Elas-c   •  Predictable   –  “Never  keep  an  applica-on  wai-ng”   –  Quasi-­‐determinis-c  latency  and  throughput   •  Low  latency   –  Built-­‐in  Memcached  technology   –  Auto-­‐migra-on  of  hot  data  to  lowest  latency   storage  technology  (RAM,  SSD,  Disk)   –  Selectable  write  behavior  –  asynchronous,   synchronous  (on  replica-on,  persistence)   •  High  throughput   –  Mul--­‐threaded   –  Low  lock  conten-on   –  Asynchronous  wherever  possible   –  Automa-c  write  de-­‐duplica-on   9  
  • 10. Couchbase  Server  is  Simple,  Fast,  Elas-c   •  Zero-­‐down-me  elas-city   Couchbase  Web  Console   –  Spread  I/O  and  data  across  commodity   servers  (or  VMs)     –  Consistent  performance  with  linear  cost   –  Dynamic  rebalancing  of  a  live  cluster   •  All  nodes  are  created  equal   –  No  special  case  nodes   –  Clone  to  grow   •  Extensible   –  Change  feeds   –  Real-­‐-me  map-­‐reduce   –  RESTful  interface  for  management   10  
  • 11. Proven  at  Small,  and  Extra  Large  Scale   •  Leading cloud service (PAAS) •  Social game leader – FarmVille, provider Mafia Wars, Empires and Allies, •  Over 150,000 hosted applications Café World, FishVille •  Couchbase Server serving over •  Over 230 million monthly users 6,200 Heroku customers •  Couchbase Server is the primary database behind key Zynga properties 11  
  • 12. Customers  and  Partners   Customers  (par-al  lis-ng)   Partners   12  
  • 13. Couchbase  Server  Architecture   11211   11210   memcapable  1.0   memcapable  2.0   moxi   REST  management  API/Web  UI   vBucket  state  and  replica-on  manager   Global  singleton  supervisor   Rebalance  orchestrator   Configura-on  manager   memcached   Node  health  monitor   Process  monitor   protocol  listener/sender   Heartbeat   Data  Manager   Cluster  Manager   engine  interface   Couchbase  Storage  Engine   hcp   on  each  node   one  per  cluster   Erlang/OTP   HTTP   erlang  port  mapper   distributed  erlang   8091   4369   21100  –  21199   13  
  • 14. Couchbase  Server  Architecture   11211   11210   memcapable  1.0   memcapable  2.0   moxi   vBucket  state  and  replica-on  manager   REST  management  API/Web  UI   Global  singleton  supervisor   Rebalance  orchestrator   Configura-on  manager   memcached   Node  health  monitor   Process  monitor   protocol  listener/sender   Heartbeat   engine  interface   Couchbase  Storage  Engine   hcp   on  each  node   one  per  cluster   Erlang/OTP   HTTP   erlang  port  mapper   distributed  erlang   8091   4369   21100  –  21199   14  
  • 15. Couchbase  “write”  Data  Flow  –  applica-on  view   User  ac-on  results  in  the  need  to   change  the  VALUE  of  KEY   1   Applica-on  updates  key’s  VALUE,   2   performs  SET  opera-on     4   Couchbase  client  hashes  KEY,  iden-fies   3   KEY’s  master  server   SET  request  sent  over   network  to  master  server   5   Couchbase  replicates  KEY-­‐VALUE  pair,  caches   it  in  memory  and  stores  it  to  disk   15  
  • 16. Couchbase  Data  Flow  –  under  the  hood   SET  request  arrives  at   SET  acknowledgement   KEY’s  master  server   1   3   returned  to  applica-on   Listener-­‐Sender 2   2   Listener-­‐Sender Listener-­‐Sender   RAM* RAM*   RAM* Couchbase  storage  engine   membase  storage  engine membase  storage  engine 2   SSD SSD SSD SSD SSD SSD SSD   SSD   SSD   Disk Disk Disk 4   Disk Disk Disk Disk   Disk   Disk   Replica  Server  1  for  KEY   Master  server  for  KEY   Replica  Server  2  for  KEY   16  
  • 17. Elas-city  -­‐  Rebalancing   Node  1   Node  2   Node  3  Before   vBucket  1   vBucket  7   vBucket  2   vBucket  8  •  Adding  Node  3   vBucket  3   vBucket  9  •  Node  3  is  in  pending  state   vBucket  4   vBucket  10   Pending  state  •  Clients  talk  to  Node  1,2  only   vBucket  5   vBucket  11   vBucket  6   vBucket  12   vBucket  1   vBucket  7  During   vBucket  2   vBucket  8   vBucket  3   vBucket  9  •  Rebalancing  orchestrator  recalculates   Rebalancing   vBucket  4   vBucket  10   the  vBucket  map  (including  replicas)   vBucket  5   vBucket  11  •  Migrate  vBuckets  to  the  new  server   vBucket  6   vBucket  12  •  Finalize  migra-on   vBucket    migrator   vBucket    migrator   Client  AOer   vBucket  1   vBucket  7   vBucket  5  •  Node  3  is  balanced   vBucket  2   vBucket  8   vBucket  6  •  Clients  are  reconfigured  to  talk  to   vBucket  3   vBucket  9   vBucket  11   Node  3   vBucket  4   vBucket  10   vBucket  12   17  
  • 18. Data  buckets  are  secure  Couchbase  “slices”   Applica-on  user   Web  applica-on  server   Bucket  1   Bucket  2   Aggregate  Cluster  Memory  and  Disk  Capacity   Couchbase  data  servers   In  the  data  center   On  the  administrator  console   18  
  • 19. Couchbase  and  Hadoop  Integra-on  •  Support  large-­‐scale  analy-cs  on  applica-on  data  by  streaming    data   from  Couchbase  to  Hadoop   –  Real-­‐-me  integra-on  using  Flume   –  Batch  integra-on  using  Sqoop  •  Examples   –  Various  game  sta-s-cs  (e.g.,  monthly  /  daily  /  hourly  rankings)   –  Analyze  game  pacerns  from  users  to  enhance  various  game  metrics     Flume   memcached   TAP   protocol  listener/sender   Sqoop   engine  interface   Couchbase  Storage  Engine   19  
  • 20. Agenda   •  Introduc-on   •  What  is  Couchbase  Server?   –  Simple,  Fast,  Elas-c   –  Technology  Overview  (Architecture,  data  flow,  rebalancing)   •  Tribal  Crossing  Inc:  Animal  Party   –  Challenges  before  Couchbase   •  Original  Architecture   –  Why  Couchbase?   •  Simplicity   •  Performance   •  Flexibility   –  Deploying  Couchbase   •  New  Architecture   •  EC2   •  Data  Model   •  Accessing  data  in  Couchbase   •  Product  Roadmap   •  Q&A   20  
  • 21. Tribal  Crossing:  Challenges  Common  steps  on  scaling  up  database:  ●  Tune  queries  (indexing,  explain  query)  ●  Denormaliza-on  ●  Cache  data  (APC  /  Memcache)  ●  Tune  MySQL  configura-on  ●  Replica-on  (read  slaves)    Where  do  we  go  from  here  to  prepare  for  the  scale  of  a  successful  social  game?   21  
  • 22. Tribal  Crossing:  Challenges  ●  Write-­‐heavy  requests   –  Caching  does  not  help   –  MySQL  /  InnoDB  limita-on  (Percona)  ●  Need  to  scale  dras-cally  over  night   –  My  Polls  –  100  to  1m  users  over  a  weekend  ●  Small  team,  no  dedicated  sysadmin   –  Focus  on  what  we  do  best  –  making  games  ●  Keeping  cost  down   22  
  • 23. Tribal  Crossing:  “Old”  Architecture  and  Op-ons  ●  MySQL  with  master-­‐to-­‐master  replica-on  and     sharding   –  Complex  to  setup,  high  administra-on  cost   –  Requires  applica-on  level  changes  ●  Cassandra   –  High  write,  but  low  read  throughput   –  Live  cluster  reconfigura-on  and  rebalance  is  quite  complicated   –  Eventual  consistency  gives  too  much  burden  to  applica-on   developers  ●  MongoDB   –  High  read/write,  but  unpredictable  latency   –  Live  cluster  rebalance  for  exis-ng  nodes  only   –  Eventual  consistency  with  slave  nodes   23  
  • 24. Tribal  Crossing:  Why  Couchbase  Server?  ●  SPEED,  SPEED,  SPEED  ●  Immediate  consistency  ●  Interface  is  dead  simple  to  use   –  We  are  already  using  Memcache  ●  Low  sysadmin  overhead  ●  Schema-­‐less  data  store  ●  Used  and  Proven  by  big  guys  like  Zynga  ●  …  and  lastly,  because  Tribal  CAN   –  Bigger  firms  with  legacy  code  base  =  hard  to  adapt   –  Small  team  =  ability  to  get  on  the  cuung  edge   24  
  • 25. Tribal  Crossing:  New  Challenges  With  Couchbase  ●  But,  there  are  some  different  challenges  in     using  Couchbase  (currently  1.7)  to  handle  the  game   data:   –  No  easy  way  to  query  data   –  No  transac-on  /  rollback   ➔  Couchbase  Server  2.0  resolves  them  by  using   CouchDB  as  the  underlying  database  engine  ●  Can  this  work  for  an  online  game?   –  Break  out  of  the  old  ORM  /  rela-onal  paradigm!   –  We  are  not  handling  bank  transac-ons   25  
  • 26. Tribal  Crossing:  Deploying  Couchbase  in  EC2   Web Server Apache ●  Basic  produc-on   Client-side Moxi environment  setup   ●  Dev/Stage  environment  –   feel  free  to  install   Cluster Mgmt. Requests Couchbase  on  your  web   DNS Entry server   Couchbase …   Couchbase Couchbase Cluster 26  
  • 27. Tribal  Crossing:  Deploying  Couchbase  in  EC2   Web Server ●  Amazon  Linux  AMI,     Apache 64-­‐bit,  EBS  backed  instance   Client-side Moxi ●  Setup  swap  space   Cluster Mgmt. Requests ●  Install  Couchbase’s     Membase  Server  1.7   DNS Entry ●  Access  web  console http://<hostname>:8091 ●  Start  the  new  cluster  with  a   …   Couchbase single  node   Couchbase ●  Add  the  other  nodes  to  the   Couchbase Cluster cluster  and  rebalance   27  
  • 28. Tribal  Crossing:  Deploying  Couchbase  in  EC2   Web Server Apache Moxi  figures  out  which  node  in  the   cluster  holds  data  for  a  given  key.   Client-side Moxi ●  On  each  web  server,  install  Moxi   Cluster Mgmt. Requests proxy   ●  Start  Moxi  by  poin-ng  it  to  the   DNS Entry DNS  entry  you  created   ●  Web  apps  connect  to  Moxi  that  is   running  locally memcache->addServer(‘localhost’, 11211); Couchbase …   Couchbase Couchbase Cluster 28  
  • 29. Tribal  Crossing:  Represen-ng  Game  Data  in  Couchbase  Use  case  -­‐  simple  farming  game:  ●  A  player  can  have  a  variety  of  plants  on  their  farm.  ●  A  player  can  add  or  remove  plants  from  their  farm.  ●  A  Player  can  see  what  plants  are  on  another  players   farm.   29  
  • 30. Tribal  Crossing:  Represen-ng  Game  Data  in  Couchbase  RepresenUng  Objects  ●  Simply  treat  an  object  as  an  associa-ve  array  ●  Determine  the  key  for  an  object  using  the  class  name   (or  type)  of  the  object  and  an  unique  ID  RepresenUng  Object  Lists  ●  Denormaliza-on  ●  Save  a  comma  separated  list  or  an  array  of  object  IDs   30  
  • 31. Tribal  Crossing:  Represen-ng  Game  Data  in  Couchbase   Player  Object   Key: Player1 Plant  Object   Array Key: Plant201 ( [Id] => 1 Array [Name] => Shawn ( ) [Id] => 201 [Player_Id] => 1 [Name] => StarflowerPlayerPlant  List   )Key: Player1_PlantListArray( [0] => 201 [1] => 202 [2] => 204)   31  
  • 32. Tribal  Crossing:  Schema-­‐less  Game  Data  ●  No  need  to  “ALTER  TABLE”  ●  Add  new  “fields”  all  objects  at  any  -me   –  Specify  default  value  for  missing  fields   –  Increased  development  speed  ●  Using  JSON  for  data  objects  though,  owing  to  the   ability  to  query  on  arbitrary  fields  in  Couchbase  2.0   32  
  • 33. Tribal  Crossing:  Accessing  Game  Data  in  Couchbase  Get  all  plants  belong  to  a  given  player  Request: GET /player/1/farm$plant_ids = couchbase->get(Player1_PlantList);$response = array();foreach ($plant_ids as $plant_id){ $plant = couchbase->get(Plant . $plant_id); $response[] = $plant;}echo json_encode($response); 33  
  • 34. Tribal  Crossing:  Modifying  Game  Data  in  Couchbase   Give  a  player  a  new  plant   // Create the new plant $new_plant = array ( id => 100, name => Mushroom ); $couchbase->set(Plant100, $new_plant); // Update the player plant list $plant_ids = $couchbase->get(Player1_PlantList); $plant_ids[] = $new_plant[id]; $couchbase->set(Player1_PlantList, $plant_ids); 34  
  • 35. Tribal  Crossing:  Concurrency  Concurrency  issue  can  occur  when  mul-ple  requests  are  working  with  the  same  piece  of  data.  Solu-on:  ●  CAS  (check-­‐and-­‐set)   –  Client  can  know  if  someone  else   has  modified  the  data  while  you   are  trying  to  update   –  Implement  op-mis-c   concurrency  control  ●  Locking  (try/wait  cycle)   –  GETL  (get  with  lock  +  -meout)   opera-ons   –  Pessimis-c  concurrency  control   35  
  • 36. Tribal  Crossing:  Data  Rela-onship  ●  Record  object  rela-onships  both  ways   –  Example:    Plots  and  Plants   ●  Plot  object  stores  id  of  the  plant  that  it  hosts   ●  Plant  object  stores  id  of  the  plot  that  it  grows  on   –  Resolu-on  in  case  of  mismatch  ●  Dont  sweat  the  extra  calls  to  load  data  in  a  one-­‐to-­‐ many  rela-onship   –  Use  mul3Get   –  We  can  s-ll  cache  aggregated  results  in  a  Memcache   bucket  if  needed   36  
  • 37. Tribal  Crossing:  Migra-ng  to  Couchbase  Servers  First  migrated  large  or  slow  performing  tables  and    frequently  updated  fields  from  MySQL  to  Couchbase   Web  Server     Apache  +  PHP         Client-­‐side  Moxi       MySQL   memcached   TAP   protocol  listener/sender   engine  interface   Repor-ng   TAP  Client   Applica-ons   Couchbase  Storage  Engine   37  
  • 38. Tribal  Crossing:  Deployment     38  
  • 39. Tribal  Crossing:  Deployment   39  
  • 40. Tribal  Crossing:  Conclusion  •  Significantly  reduced  the  cost  incurred  by  scaling  up   database  servers  and  managing  them.  •  Achieved  significant  improvements  in  various   performance  metrics  (e.g.,  read,  write,  latency,  etc.)  •  Allowed  them  to  focus  more  on  game  development  and   op-mizing  key  metrics  •  Plan  to  use  real-­‐-me  MapReduce,  querying,  and   indexing  abili-es  provided  by  the  upcoming  Elas-c   Couchbase  2.0   40  
  • 41. Agenda   •  Introduc-on   •  What  is  Couchbase  Server?   –  Simple,  Fast,  Elas-c   –  Technology  Overview  (Architecture,  data  flow,  rebalancing)   •  Tribal  Crossing  Inc:  Animal  Party   –  Challenges  before  Couchbase   •  Original  Architecture   –  Why  Couchbase?   •  Simplicity   •  Performance   •  Flexibility   –  Deploying  Couchbase   •  New  Architecture   •  EC2   •  Data  Model   •  Accessing  data  in  Couchbase   •  Product  Roadmap   •  Q&A   41  
  • 42. Product  Roadmap:  Couchbase    Server  2.0  •  Mobile  to  cloud  data  synchroniza-on  •  Cross  data  center  replica-on   US  West  Coast  Data  Center   US  East  Coast  Data  Center   Couchbase   Couchbase   Server   CouchSync   Server   CouchSync   CouchSync   Couchbase  Single  Server   Couchbase  Single  Server   CouchSync   CouchSync   …   …   … … … … 42  
  • 43. Product  Roadmap:  Couchbase  Server  2.0  •  Replace  Sqlite-­‐based  storage  engine  with  CouchDB  •  Support  indexing  and  querying  on  values  •  Integrate  real-­‐-me  MapReduce  into  Couchbase  server  •  SDK  for  Couchbase  server   Membase  Server  1.7   CouchDB  1.1   Couchbase  Server  2.0   The  world’s  leading  caching   The  most  reliable  and  full-­‐ The  fastest,  most  complete  and   and  clustering  technology   featured  document  database   most  reliable  database  on  the   planet   43  
  • 44. Couchbase  Product  Download  •  Community  Edi-on   –  Open  source  build   –  Free  forum  support  •  Enterprise  Edi-on   –  Free  for  non-­‐produc-on  use   –  Cer-fied,  QA  tested  version  of  open  source   –  Case  tracking  and  guaranteed  SLA  for  produc-on   environments  •  Partner  in  Korea   –  N2M  Inc.  (hcp://www.n2m.co.kr)   44  
  • 45. Q&A   Matt Ingenthron, Couchbase Inc. (matt@couchbase.com, @ingenthr) Chiyoung Seo, Couchbase Inc.(chiyoung@couchbase.com, @chiyoungseo) 45