Couchbase	  Use	  Case	  Example:                         Social	  Gaming                                    Frank	  Weige...
Tribal	  Crossing:	  Animal	  Party  • Tribal	  Crossing	  (FableLabs)	  Facebook	  game	          – Hosted	  on	  EC2  • ...
Tribal	  Crossing:	  Challenges       Common	  steps	  on	  scaling	  up	  RDBMS:  •    Tune	  queries	  (indexing,	  expl...
Tribal	  Crossing:	  Challenges  • Write-­‐heavy	  requests              • Caching	  does	  not	  help              • MySQ...
Tribal	  Crossing:	  “Old”	  Architecture	  and	  OpFons  • MySQL	  with	  master-­‐to-­‐master	  replicaFon	  and	      s...
Tribal	  Crossing:	  Why	  Couchbase	  Server?  • SPEED,	  SPEED,	  SPEED  • Immediate	  consistency  • Interface	  is	  d...
Tribal	  Crossing:	  New	  Challenges	  With	  Couchbase  • But,	  there	  are	  some	  differences	  in	      using	  Couc...
Tribal	  Crossing:	  Deploying	  Couchbase	  in	  EC2                                  •Basic	  producEon	  environment	  ...
Tribal	  Crossing:	  Deploying	  Couchbase	  in	  EC2                             1. Amazon	  Linux	  AMI,	               ...
Tribal	  Crossing:	  Deploying	  Couchbase	  in	  EC2                              •Moxi	  figures	  out	  which	  node	  i...
Tribal	  Crossing:	  RepresenFng	  Game	  Data	  in	      Use	  case	  -­‐	  simple	  version	  of	  farming	  part:  • A	...
Tribal	  Crossing:	  RepresenFng	  Game	  Data	  in	      RepresenDng	  Objects  • Simply	  treat	  an	  object	  as	  a	 ...
Tribal	  Crossing:	  RepresenFng	  Game	  Data	  in	      Player	  Object    Key: Player1                         Plant	  ...
Tribal	  Crossing:	  Schema-­‐less	  Game	  Data  • No	  need	  to	  “ALTER	  TABLE”  • Add	  new	  “fields”	  all	  object...
Tribal	  Crossing:	  Accessing	  Game	  Data	  in	  Couchbase    Get	  all	  plants	  belong	  to	  a	  given	  player    ...
Tribal	  Crossing:	  Modifying	  Game	  Data	  in	  Couchbase    Give	  a	  player	  a	  new	  plant    // Create the new ...
Tribal	  Crossing:	  Concurrency  • Concurrency	  issue	  can	  occur	  when	  mulFple	      requests	  are	  working	  wi...
Tribal	  Crossing:	  Data	  RelaFonship  • Record	  object	  relaFonships	  both	  ways              • Example:	  	  Plots...
MIGRATING	  TO	  COUCHBASE:               Moving	  from	  MySQL	  &	  memcached                                           ...
Tribal	  Crossing:	  MigraFng	  to	  Couchbase	  Servers  • First	  migrated	  large	  or	  slow	  performing	  tables	  a...
Tribal	  Crossing:	  Deployment	                                            21Monday, March 26, 2012                     21
Tribal	  Crossing:	  Deployment                                      22Monday, March 26, 2012                 22
Tribal	  Crossing:	  Conclusion  • Significantly	  reduced	  the	  cost	  incurred	      by	  scaling	  up	  database	  ser...
Thank	  You!                           Q&A                         @FrankWeigel                                          2...
Upcoming SlideShare
Loading in...5



Published on

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Couchbase  Use  Case  Example: Social  Gaming Frank  Weigel Director,  Product  Management 1Monday, March 26, 2012 1
  2. 2. Tribal  Crossing:  Animal  Party • Tribal  Crossing  (FableLabs)  Facebook  game   – Hosted  on  EC2 • “Part  Pokemon,  part  FronFerville  and  part  Mario  Galaxy!     In  Animal  Party  youll  discover  hundreds  of  amazing   animals  across  the  galaxy  and  raise  them  on  your   magical  garden.” 2Monday, March 26, 2012 2
  3. 3. Tribal  Crossing:  Challenges Common  steps  on  scaling  up  RDBMS: • Tune  queries  (indexing,  explain  query) • DenormalizaFon • Cache  data  (Memcache) • Tune  MySQL  configuraFon • ReplicaFon  (read  slaves) • Where  do  we  go  from  here  to  prepare  for  the  scale  of  a   successful  social  game? 3Monday, March 26, 2012 3
  4. 4. Tribal  Crossing:  Challenges • Write-­‐heavy  requests • Caching  does  not  help • MySQL  /  InnoDB  limitaFon  (Percona) • Need  to  scale  drasFcally  over  night • My  Polls  –  100  to  1m  users  over  a  weekend • Small  team,  no  dedicated  sysadmin • Focus  on  what  we  do  best  –  making  games • Keeping  cost  down 4Monday, March 26, 2012 4
  5. 5. Tribal  Crossing:  “Old”  Architecture  and  OpFons • MySQL  with  master-­‐to-­‐master  replicaFon  and   sharding • Complex  to  setup,  high  administraFon  cost • Requires  applicaFon  level  changes • Cassandra • High  write,  but  low  read  throughput • Live  cluster  reconfiguraFon  and  rebalance  is  quite  complicated • Eventual  consistency  gives  too  much  burden  to  applicaFon  developers • MongoDB • High  read/write,  but  unpredictable  latency • Live  cluster  rebalance  for  exisFng  nodes  only • Disk  data  corrupFon  concerns  on  node  failures 5Monday, March 26, 2012 5
  6. 6. Tribal  Crossing:  Why  Couchbase  Server? • SPEED,  SPEED,  SPEED • Immediate  consistency • Interface  is  dead  simple  to  use • We  are  already  using  Memcache • Low  sysadmin  overhead • Schema-­‐less  data  store • Used  and  Proven  by  big  guys  like  Zynga • …  and  lastly,  because  Tribal  CAN • Bigger  firms  with  legacy  code  base  =  hard  to  adapt • Small  team  =  ability  to  get  on  the  cujng  edge 6Monday, March 26, 2012 6
  7. 7. Tribal  Crossing:  New  Challenges  With  Couchbase • But,  there  are  some  differences  in   using  Couchbase  (currently  1.7)  to  handle  the  game   data: • No  easy  way  to  query  data – Couchbase  Server  2.0  resolves  this  by  using  new  persistence   layer • Can  this  work  for  an  online  game? • Break  out  of  the  old  ORM  /  relaEonal  paradigm! • Tribal:  “we  are  not  handling  criEcal  transacEons” 7Monday, March 26, 2012 7
  8. 8. Tribal  Crossing:  Deploying  Couchbase  in  EC2 •Basic  producEon  environment   setup •Dev/Stage  environment  –  feel  free   to  install  Couchbase  on  your  web   server 8Monday, March 26, 2012 8
  9. 9. Tribal  Crossing:  Deploying  Couchbase  in  EC2 1. Amazon  Linux  AMI,   64-­‐bit,  EBS  backed  instance 2. Setup  swap  space 3. Install  Couchbase’s   Membase  Server  1.7 4. Access  web  console hp://<hostname>:8091 5. Start  the  new  cluster  with  a  single   node 6. Add  the  other  nodes  to  the  cluster   and  rebalance 9Monday, March 26, 2012 9
  10. 10. Tribal  Crossing:  Deploying  Couchbase  in  EC2 •Moxi  figures  out  which  node  in   the  cluster  holds  data  for  a  given   key.    Used  with  older,  non-­‐cluster   aware  clients  like  PHP,  Ruby,  Perl. [smart  clients  for  PHP  &  Ruby  released  since] •On  each  web  server,  install  Moxi   proxy •Start  Moxi  by  poinEng  it  to  the   DNS  entry  already  created •Web  apps  connect  to  Moxi  that  is   running  locally memcache-­‐>addServer(‘localhost’,   10Monday, March 26, 2012 10
  11. 11. Tribal  Crossing:  RepresenFng  Game  Data  in   Use  case  -­‐  simple  version  of  farming  part: • A  player  can  have  a  variety  of  plants  on  their  farm. • A  player  can  add  or  remove  plants  from  their  farm. • A  Player  can  see  what  plants  are  on  another  players   farm. 11Monday, March 26, 2012 11
  12. 12. Tribal  Crossing:  RepresenFng  Game  Data  in   RepresenDng  Objects • Simply  treat  an  object  as  a  Map/dicFonary/JSON  object • Determine  the  key  for  an  object  using  the  class  name  (or   type)  of  the  object  and  an  unique  ID RepresenDng  Object  Lists • DenormalizaFon • Save  a  comma  separated  list  or  an  array  of  object  IDs 12Monday, March 26, 2012 12
  13. 13. Tribal  Crossing:  RepresenFng  Game  Data  in   Player  Object Key: Player1 Plant  Object JSON Key: Plant201 { “_id” : “Player1”, JSON “nid” : 1, { “name” : “Shawn” “_id” : “Plant201”, } “nid” : 201, “player_id” : 1 PlayerPlant  List } “name” : “Starflower” Key: Player1_PlantList JSON { “_id” : “Player1_Plantlist”, “plants” : [201, 202, 204] } 13Monday, March 26, 2012 13
  14. 14. Tribal  Crossing:  Schema-­‐less  Game  Data • No  need  to  “ALTER  TABLE” • Add  new  “fields”  all  objects  at  any  Fme • Specify  default  value  for  missing  fields • Increased  development  speed • Using  JSON  for  data  objects  though,   – Offers  the  ability  to  query  and  analyze  arbitrary  fields  in   Couchbase  2.0 14Monday, March 26, 2012 14
  15. 15. Tribal  Crossing:  Accessing  Game  Data  in  Couchbase Get  all  plants  belong  to  a  given  player Request: GET /player/1/farm // Create a new PlantList from players PlantList playersPlants = new PlantList(cbclient.get ("Player1_PlantList"); for (Plant plant : playersPlants) { aPlayer.addPlant(plant); } 15Monday, March 26, 2012 15
  16. 16. Tribal  Crossing:  Modifying  Game  Data  in  Couchbase Give  a  player  a  new  plant // Create the new plant Plant givenPlant = new Plant(100, "Mushroom"); cbclient.set("Plant100", givenPlant); // Update the player plant list Player thePlayer = player.fetch(cbclient.get ("Player1"); // Add the plant to the player thePlayer.receivePlant(givenPlant); // Store the players new plant list cbclient.set("Player1_PlantList", 16Monday, March 26, 2012 16
  17. 17. Tribal  Crossing:  Concurrency • Concurrency  issue  can  occur  when  mulFple   requests  are  working  with  the  same  piece  of  data. SoluDons: • CAS  (check-­‐and-­‐set) • Client  can  know  if  someone  else  has  modified  the  data  while  you  are   trying  to  update • Provides  opFmisFc  concurrency  control • GETL  (get  with  lock) • Locking  (try/wait  cycle) • Provides  pessimisFc  concurrency  control 17Monday, March 26, 2012 17
  18. 18. Tribal  Crossing:  Data  RelaFonship • Record  object  relaFonships  both  ways • Example:    Plots  and  Plants – Plot  object  stores  id  of  the  plant  that  it  hosts – Plant  object  stores  id  of  the  plot  that  it  grows  on • ResoluFon  in  case  of  mismatch • Dont  sweat  the  extra  calls  to  load  data  in  a  one-­‐to-­‐many   relaFonship • Use  mulFGet • Tribal:  "We  can  sFll  cache  aggregated  results  in  a  Memcache  bucket  if   needed" 18Monday, March 26, 2012 18
  19. 19. MIGRATING  TO  COUCHBASE: Moving  from  MySQL  &  memcached 19Monday, March 26, 2012 19
  20. 20. Tribal  Crossing:  MigraFng  to  Couchbase  Servers • First  migrated  large  or  slow  performing  tables  and   frequently  updated  fields  from  MySQL  to  Couchbase 20Monday, March 26, 2012 20
  21. 21. Tribal  Crossing:  Deployment   21Monday, March 26, 2012 21
  22. 22. Tribal  Crossing:  Deployment 22Monday, March 26, 2012 22
  23. 23. Tribal  Crossing:  Conclusion • Significantly  reduced  the  cost  incurred   by  scaling  up  database  servers  and  managing  them. • Achieved  significant  improvements  in  various   performance  metrics  (e.g.,  read,  write,  latency,  etc.) • Allowed  them  to  focus  more  on  game  development  and   opFmizing  key  metrics • Plan  to  use  real-­‐Fme  Map-­‐Reduce,  querying,  and   indexing  abiliFes  provided  by  Couchbase  Server  2.0 23Monday, March 26, 2012 23
  24. 24. Thank  You! Q&A @FrankWeigel 24Monday, March 26, 2012 24
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.