• Share
  • Email
  • Embed
  • Like
  • Private Content
CouchConf-London-nosql-social-gaming-usecase-and-migration
 

CouchConf-London-nosql-social-gaming-usecase-and-migration

on

  • 715 views

 

Statistics

Views

Total Views
715
Views on SlideShare
715
Embed Views
0

Actions

Likes
0
Downloads
12
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    CouchConf-London-nosql-social-gaming-usecase-and-migration CouchConf-London-nosql-social-gaming-usecase-and-migration Presentation Transcript

    • Couchbase  Use  Case  Example: Social  Gaming Frank  Weigel Director,  Product  Management frank@couchbase.com 1Monday, March 26, 2012 1
    • Tribal  Crossing:  Animal  Party • Tribal  Crossing  (FableLabs)  Facebook  game   – Hosted  on  EC2 • “Part  Pokemon,  part  FronFerville  and  part  Mario  Galaxy!     In  Animal  Party  youll  discover  hundreds  of  amazing   animals  across  the  galaxy  and  raise  them  on  your   magical  garden.” 2Monday, March 26, 2012 2
    • Tribal  Crossing:  Challenges Common  steps  on  scaling  up  RDBMS: • Tune  queries  (indexing,  explain  query) • DenormalizaFon • Cache  data  (Memcache) • Tune  MySQL  configuraFon • ReplicaFon  (read  slaves) • Where  do  we  go  from  here  to  prepare  for  the  scale  of  a   successful  social  game? 3Monday, March 26, 2012 3
    • Tribal  Crossing:  Challenges • Write-­‐heavy  requests • Caching  does  not  help • MySQL  /  InnoDB  limitaFon  (Percona) • Need  to  scale  drasFcally  over  night • My  Polls  –  100  to  1m  users  over  a  weekend • Small  team,  no  dedicated  sysadmin • Focus  on  what  we  do  best  –  making  games • Keeping  cost  down 4Monday, March 26, 2012 4
    • Tribal  Crossing:  “Old”  Architecture  and  OpFons • MySQL  with  master-­‐to-­‐master  replicaFon  and   sharding • Complex  to  setup,  high  administraFon  cost • Requires  applicaFon  level  changes • Cassandra • High  write,  but  low  read  throughput • Live  cluster  reconfiguraFon  and  rebalance  is  quite  complicated • Eventual  consistency  gives  too  much  burden  to  applicaFon  developers • MongoDB • High  read/write,  but  unpredictable  latency • Live  cluster  rebalance  for  exisFng  nodes  only • Disk  data  corrupFon  concerns  on  node  failures 5Monday, March 26, 2012 5
    • Tribal  Crossing:  Why  Couchbase  Server? • SPEED,  SPEED,  SPEED • Immediate  consistency • Interface  is  dead  simple  to  use • We  are  already  using  Memcache • Low  sysadmin  overhead • Schema-­‐less  data  store • Used  and  Proven  by  big  guys  like  Zynga • …  and  lastly,  because  Tribal  CAN • Bigger  firms  with  legacy  code  base  =  hard  to  adapt • Small  team  =  ability  to  get  on  the  cujng  edge 6Monday, March 26, 2012 6
    • Tribal  Crossing:  New  Challenges  With  Couchbase • But,  there  are  some  differences  in   using  Couchbase  (currently  1.7)  to  handle  the  game   data: • No  easy  way  to  query  data – Couchbase  Server  2.0  resolves  this  by  using  new  persistence   layer • Can  this  work  for  an  online  game? • Break  out  of  the  old  ORM  /  relaEonal  paradigm! • Tribal:  “we  are  not  handling  criEcal  transacEons” 7Monday, March 26, 2012 7
    • Tribal  Crossing:  Deploying  Couchbase  in  EC2 •Basic  producEon  environment   setup •Dev/Stage  environment  –  feel  free   to  install  Couchbase  on  your  web   server 8Monday, March 26, 2012 8
    • Tribal  Crossing:  Deploying  Couchbase  in  EC2 1. Amazon  Linux  AMI,   64-­‐bit,  EBS  backed  instance 2. Setup  swap  space 3. Install  Couchbase’s   Membase  Server  1.7 4. Access  web  console hp://<hostname>:8091 5. Start  the  new  cluster  with  a  single   node 6. Add  the  other  nodes  to  the  cluster   and  rebalance 9Monday, March 26, 2012 9
    • Tribal  Crossing:  Deploying  Couchbase  in  EC2 •Moxi  figures  out  which  node  in   the  cluster  holds  data  for  a  given   key.    Used  with  older,  non-­‐cluster   aware  clients  like  PHP,  Ruby,  Perl. [smart  clients  for  PHP  &  Ruby  released  since] •On  each  web  server,  install  Moxi   proxy •Start  Moxi  by  poinEng  it  to  the   DNS  entry  already  created •Web  apps  connect  to  Moxi  that  is   running  locally memcache-­‐>addServer(‘localhost’,   10Monday, March 26, 2012 10
    • Tribal  Crossing:  RepresenFng  Game  Data  in   Use  case  -­‐  simple  version  of  farming  part: • A  player  can  have  a  variety  of  plants  on  their  farm. • A  player  can  add  or  remove  plants  from  their  farm. • A  Player  can  see  what  plants  are  on  another  players   farm. 11Monday, March 26, 2012 11
    • Tribal  Crossing:  RepresenFng  Game  Data  in   RepresenDng  Objects • Simply  treat  an  object  as  a  Map/dicFonary/JSON  object • Determine  the  key  for  an  object  using  the  class  name  (or   type)  of  the  object  and  an  unique  ID RepresenDng  Object  Lists • DenormalizaFon • Save  a  comma  separated  list  or  an  array  of  object  IDs 12Monday, March 26, 2012 12
    • Tribal  Crossing:  RepresenFng  Game  Data  in   Player  Object Key: Player1 Plant  Object JSON Key: Plant201 { “_id” : “Player1”, JSON “nid” : 1, { “name” : “Shawn” “_id” : “Plant201”, } “nid” : 201, “player_id” : 1 PlayerPlant  List } “name” : “Starflower” Key: Player1_PlantList JSON { “_id” : “Player1_Plantlist”, “plants” : [201, 202, 204] } 13Monday, March 26, 2012 13
    • Tribal  Crossing:  Schema-­‐less  Game  Data • No  need  to  “ALTER  TABLE” • Add  new  “fields”  all  objects  at  any  Fme • Specify  default  value  for  missing  fields • Increased  development  speed • Using  JSON  for  data  objects  though,   – Offers  the  ability  to  query  and  analyze  arbitrary  fields  in   Couchbase  2.0 14Monday, March 26, 2012 14
    • Tribal  Crossing:  Accessing  Game  Data  in  Couchbase Get  all  plants  belong  to  a  given  player Request: GET /player/1/farm // Create a new PlantList from players PlantList playersPlants = new PlantList(cbclient.get ("Player1_PlantList"); for (Plant plant : playersPlants) { aPlayer.addPlant(plant); } 15Monday, March 26, 2012 15
    • Tribal  Crossing:  Modifying  Game  Data  in  Couchbase Give  a  player  a  new  plant // Create the new plant Plant givenPlant = new Plant(100, "Mushroom"); cbclient.set("Plant100", givenPlant); // Update the player plant list Player thePlayer = player.fetch(cbclient.get ("Player1"); // Add the plant to the player thePlayer.receivePlant(givenPlant); // Store the players new plant list cbclient.set("Player1_PlantList", 16Monday, March 26, 2012 16
    • Tribal  Crossing:  Concurrency • Concurrency  issue  can  occur  when  mulFple   requests  are  working  with  the  same  piece  of  data. SoluDons: • CAS  (check-­‐and-­‐set) • Client  can  know  if  someone  else  has  modified  the  data  while  you  are   trying  to  update • Provides  opFmisFc  concurrency  control • GETL  (get  with  lock) • Locking  (try/wait  cycle) • Provides  pessimisFc  concurrency  control 17Monday, March 26, 2012 17
    • Tribal  Crossing:  Data  RelaFonship • Record  object  relaFonships  both  ways • Example:    Plots  and  Plants – Plot  object  stores  id  of  the  plant  that  it  hosts – Plant  object  stores  id  of  the  plot  that  it  grows  on • ResoluFon  in  case  of  mismatch • Dont  sweat  the  extra  calls  to  load  data  in  a  one-­‐to-­‐many   relaFonship • Use  mulFGet • Tribal:  "We  can  sFll  cache  aggregated  results  in  a  Memcache  bucket  if   needed" 18Monday, March 26, 2012 18
    • MIGRATING  TO  COUCHBASE: Moving  from  MySQL  &  memcached 19Monday, March 26, 2012 19
    • Tribal  Crossing:  MigraFng  to  Couchbase  Servers • First  migrated  large  or  slow  performing  tables  and   frequently  updated  fields  from  MySQL  to  Couchbase 20Monday, March 26, 2012 20
    • Tribal  Crossing:  Deployment   21Monday, March 26, 2012 21
    • Tribal  Crossing:  Deployment 22Monday, March 26, 2012 22
    • Tribal  Crossing:  Conclusion • Significantly  reduced  the  cost  incurred   by  scaling  up  database  servers  and  managing  them. • Achieved  significant  improvements  in  various   performance  metrics  (e.g.,  read,  write,  latency,  etc.) • Allowed  them  to  focus  more  on  game  development  and   opFmizing  key  metrics • Plan  to  use  real-­‐Fme  Map-­‐Reduce,  querying,  and   indexing  abiliFes  provided  by  Couchbase  Server  2.0 23Monday, March 26, 2012 23
    • Thank  You! Q&A @FrankWeigel 24Monday, March 26, 2012 24