Java one 2011 couchbase- no sql-for-social-game-scaling-and-performance

3,330 views

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,330
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
24
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Java one 2011 couchbase- no sql-for-social-game-scaling-and-performance

  1. 1. COUCHBASE:  NOSQL  FOR  SOCIAL   GAME  SCALING  AND  PERFORMANCE Matt Ingenthron, Couchbase Inc. 1Thursday, October 6, 2011
  2. 2. Modern Interactive Web Application Architecture Application Scales Out Just add more commodity web www.facebook.com/animalparty servers Load  Balancer Web  Servers Rela9onal   Database -­‐  Expensive  and  disrup1ve  sharding -­‐  Doesn’t  perform  at  Web  Scale 2Thursday, October 6, 2011
  3. 3. Modern Interactive Web Application Architecture Application Scales Out Just add more commodity web www.facebook.com/animalparty servers Load  Balancer Web  Servers Rela9onal   Database Scales Up Get a bigger, more complex Database server -­‐  Expensive  and  disrup1ve  sharding -­‐  Doesn’t  perform  at  Web  Scale 2Thursday, October 6, 2011
  4. 4. Couchbase  Server  is  a  distributed  database Couchbase  Web  Console Applica9on  user Web    applica9on    server Couchbase  Servers 3Thursday, October 6, 2011
  5. 5. Couchbase data layer scales like application logic tier Data layer now scales with linear cost and constant performance. Application Scales Out www.facebook.com/animalparty Just add more commodity web servers Load  Balancer Web  Servers Couchbase    Servers Horizontally  scalable,  schema-­‐less,  auto-­‐ sharding,  high-­‐performance  at  Web   Scale Scaling out flattens the cost and performance curves. 4Thursday, October 6, 2011
  6. 6. Couchbase data layer scales like application logic tier Data layer now scales with linear cost and constant performance. Application Scales Out www.facebook.com/animalparty Just add more commodity web servers Load  Balancer Web  Servers Couchbase    Servers Database Scales Out Just add more commodity data servers Horizontally  scalable,  schema-­‐less,  auto-­‐ sharding,  high-­‐performance  at  Web   Scale Scaling out flattens the cost and performance curves. 4Thursday, October 6, 2011
  7. 7. Couchbase  Server  Architecture 11211 11210 memcapable  1.0 memcapable  2.0 moxi REST  management  API/Web  UI vBucket  state  and  replica9on  manager Global  singleton  supervisor Rebalance  orchestrator memcached Configura9on  manager Node  health  monitor Process  monitor protocol  listener/sender Heartbeat Data  Manager Cluster  Manager engine  interface Couchbase  Storage  Engine hJp on  each  node one  per  cluster Erlang/OTP HTTP erlang  port  mapper distributed  erlang 8091 4369 21100  –  21199 5Thursday, October 6, 2011
  8. 8. Couchbase  Server  Architecture 11211 11210 memcapable  1.0 memcapable  2.0 moxi vBucket  state  and  replica9on  manager REST  management  API/Web  UI Global  singleton  supervisor Rebalance  orchestrator memcached Configura9on  manager Node  health  monitor Process  monitor protocol  listener/sender Heartbeat engine  interface Couchbase  Storage  Engine hJp on  each  node one  per  cluster Erlang/OTP HTTP erlang  port  mapper distributed  erlang 8091 4369 21100  –  21199 6Thursday, October 6, 2011
  9. 9. Couchbase  “write”  Data  Flow  –  applica9on  view User  ac9on  results  in  the  need  to   change  the  VALUE  of  KEY 1 Applica9on  updates  key’s  VALUE,   2 performs  SET  opera9on   4 Couchbase  client  hashes  KEY,  iden9fies   3 KEY’s  master  server SET  request  sent  over   network  to  master  server 5 Couchbase  replicates  KEY-­‐VALUE  pair,  caches  it   in  memory  and  stores  it  to  disk 7Thursday, October 6, 2011
  10. 10. Couchbase  Data  Flow  –  under  the  hood SET  request  arrives  at  KEY’s   SET  acknowledgement   master  server 1 3 returned  to  applica9on 2 2 Listener-­‐Sender RAM* 2 Couchbase  storage  engine Disk Disk Disk 4 Disk Disk Disk Replica  Server  1  for  KEY Master  server  for  KEY Replica  Server  2  for  KEY 8Thursday, October 6, 2011
  11. 11. Elas9city  -­‐  Rebalancing Node  1 Node  2 Node  3 Before vBucket  1 vBucket  7 vBucket  2 • Adding  Node  3 vBucket  8 vBucket  3 vBucket  9 • Node  3  is  in  pending  state vBucket  4 vBucket  10 Pending  state • Clients  talk  to  Node  1,2  only vBucket  5 vBucket  11 vBucket  6 vBucket  12 9Thursday, October 6, 2011
  12. 12. Elas9city  -­‐  Rebalancing Node  1 Node  2 Node  3 Before vBucket  1 vBucket  7 vBucket  2 • Adding  Node  3 vBucket  8 vBucket  3 vBucket  9 • Node  3  is  in  pending  state vBucket  4 vBucket  10 Pending  state • Clients  talk  to  Node  1,2  only vBucket  5 vBucket  11 vBucket  6 vBucket  12 vBucket  1 vBucket  7 During vBucket  2 vBucket  8 vBucket  3 vBucket  9 • Rebalancing  orchestrator  recalculates   Rebalancing vBucket  4 vBucket  10 the  vBucket  map  (including  replicas) vBucket  5 vBucket  11 • Migrate  vBuckets  to  the  new  server vBucket  6 vBucket  12 • Finalize  migra9on vBucket    migrator vBucket    migrator 9Thursday, October 6, 2011
  13. 13. Elas9city  -­‐  Rebalancing Node  1 Node  2 Node  3 Before vBucket  1 vBucket  7 vBucket  2 • Adding  Node  3 vBucket  8 vBucket  3 vBucket  9 • Node  3  is  in  pending  state vBucket  4 vBucket  10 Pending  state • Clients  talk  to  Node  1,2  only vBucket  5 vBucket  11 vBucket  6 vBucket  12 vBucket  1 vBucket  7 During vBucket  2 vBucket  8 vBucket  3 vBucket  9 • Rebalancing  orchestrator  recalculates   Rebalancing vBucket  4 vBucket  10 the  vBucket  map  (including  replicas) vBucket  5 vBucket  11 • Migrate  vBuckets  to  the  new  server vBucket  6 vBucket  12 • Finalize  migra9on vBucket    migrator vBucket    migrator Client A1er vBucket  5 • Node  3  is  balanced vBucket  1 vBucket  7 vBucket  2 vBucket  6 • Clients  are  reconfigured  to  talk  to   vBucket  8 vBucket  3 vBucket  11 Node  3 vBucket  9 vBucket  4 vBucket  10 vBucket  12 9Thursday, October 6, 2011
  14. 14. Data  buckets  are  secure  Couchbase  “slices” Applica9on  user Web  applica9on  server Bucket  1 Bucket  2 Aggregate  Cluster  Memory  and  Disk  Capacity Couchbase  data  servers In  the  data  center On  the  administrator  console 10Thursday, October 6, 2011
  15. 15. Proven  at  Small,  and  Extra  Large  Scale • Leading cloud service (PAAS) • Social game leader – FarmVille, provider Mafia Wars, Empires and Allies, • Over 150,000 hosted Café World, FishVille applications • Over 230 million monthly users • Couchbase Server serving over • Couchbase Server is the 6,200 Heroku customers primary database behind key Zynga properties 11Thursday, October 6, 2011
  16. 16. Couchbase  and  Hadoop  Integra9on • Support  large-­‐scale  analy9cs  on  applica9on  data  by  streaming    data   from  Couchbase  to  Hadoop – Real-­‐1me  integra1on  using  Flume – Batch  integra1on  using  Sqoop • Examples – Various  game  sta1s1cs  (e.g.,  monthly  /  daily  /  hourly  rankings) – Analyze  game  paJerns  from  users  to  enhance  various  game  metrics   Flume memcached TAP protocol  listener/sender Sqoop engine  interface Couchbase  Storage  Engine 12Thursday, October 6, 2011
  17. 17. CASE  STUDY:  TRIBAL  CROSSING   13Thursday, October 6, 2011
  18. 18. Tribal  Crossing:  Challenges Common  steps  on  scaling  up  database: ● Tune  queries  (indexing,  explain  query) ● Denormaliza9on ● Cache  data  (Memcache) ● Tune  MySQL  configura9on ● Replica9on  (read  slaves) Where  do  we  go  from  here  to  prepare  for  the  scale  of  a   successful  social  game? 14Thursday, October 6, 2011
  19. 19. Tribal  Crossing:  Challenges ● Write-­‐heavy  requests – Caching  does  not  help – MySQL  /  InnoDB  limita9on  (Percona) ● Need  to  scale  dras9cally  over  night – My  Polls  –  100  to  1m  users  over  a  weekend ● Small  team,  no  dedicated  sysadmin – Focus  on  what  we  do  best  –  making  games ● Keeping  cost  down 15Thursday, October 6, 2011
  20. 20. Tribal  Crossing:  “Old”  Architecture  and  Op9ons ● MySQL  with  master-­‐to-­‐master  replica9on  and   sharding – Complex  to  setup,  high  administra9on  cost – Requires  applica9on  level  changes ● Cassandra – High  write,  but  low  read  throughput – Live  cluster  reconfigura9on  and  rebalance  is  quite  complicated – Eventual  consistency  gives  too  much  burden  to  applica9on   developers ● MongoDB – High  read/write,  but  unpredictable  latency – Live  cluster  rebalance  for  exis9ng  nodes  only – Eventual  consistency  with  slave  nodes 16Thursday, October 6, 2011
  21. 21. Tribal  Crossing:  Why  Couchbase  Server? ● SPEED,  SPEED,  SPEED ● Immediate  consistency ● Interface  is  dead  simple  to  use – We  are  already  using  Memcache ● Low  sysadmin  overhead ● Schema-­‐less  data  store ● Used  and  Proven  by  big  guys  like  Zynga ● …  and  lastly,  because  Tribal  CAN – Bigger  firms  with  legacy  code  base  =  hard  to  adapt – Small  team  =  ability  to  get  on  the  cugng  edge 17Thursday, October 6, 2011
  22. 22. Tribal  Crossing:  New  Challenges  With  Couchbase ● But,  there  are  some  differences  in   using  Couchbase  (currently  1.7)  to  handle  the  game   data: – No  easy  way  to  query  data – No  transac9on  /  rollback ➔ Couchbase  Server  2.0  resolves  these  by  using   CouchDB  as  the  underlying  database  engine ● Can  this  work  for  an  online  game? – Break  out  of  the  old  ORM  /  rela9onal  paradigm! – Tribal:  “we  are  not  handling  cri9cal  transac9ons” 18Thursday, October 6, 2011
  23. 23. Tribal  Crossing:  Deploying  Couchbase  in  EC2 Web Server Apache ● Basic  produc9on   Client-side Moxi environment  setup ● Dev/Stage  environment  –   Cluster Mgmt. Requests feel  free  to  install   DNS Entry Couchbase  on  your  web   server Couchbase … Couchbase Couchbase Cluster 19Thursday, October 6, 2011
  24. 24. Tribal  Crossing:  Deploying  Couchbase  in  EC2 Web Server ● Amazon  Linux  AMI,   Apache 64-­‐bit,  EBS  backed  instance Client-side Moxi ● Setup  swap  space Cluster Mgmt. Requests ● Install  Couchbase’s   Membase  Server  1.7 DNS Entry ● Access  web  console http://<hostname>:8091 ● Start  the  new  cluster  with  a   Couchbase … Couchbase single  node ● Add  the  other  nodes  to  the   Couchbase Cluster cluster  and  rebalance 20Thursday, October 6, 2011
  25. 25. Tribal  Crossing:  Deploying  Couchbase  in  EC2 Web Server Apache Moxi  figures  out  which  node  in  the   cluster  holds  data  for  a  given  key.     Client-side Moxi Used  with  older,  non-­‐cluster  aware   clients  like  PHP,  Ruby,  Perl. Cluster Mgmt. Requests ● On  each  web  server,  install  Moxi   proxy DNS Entry ● Start  Moxi  by  poin9ng  it  to  the  DNS   entry  already  created ● Web  apps  connect  to  Moxi  that  is   Couchbase … Couchbase running  locally memcache->addServer(‘localhost’, 11211); Couchbase Cluster 21Thursday, October 6, 2011
  26. 26. Tribal  Crossing:  RepresenCng  Game  Data  in  Couchbase Use  case  -­‐  simple  farming  game: ● A  player  can  have  a  variety  of  plants  on  their  farm. ● A  player  can  add  or  remove  plants  from  their  farm. ● A  Player  can  see  what  plants  are  on  another  players   farm. 22Thursday, October 6, 2011
  27. 27. Tribal  Crossing:  RepresenCng  Game  Data  in  Couchbase RepresenMng  Objects ● Simply  treat  an  object  as  a  Map/dic9onary/JSON  object ● Determine  the  key  for  an  object  using  the  class  name   (or  type)  of  the  object  and  an  unique  ID RepresenMng  Object  Lists ● Denormaliza9on ● Save  a  comma  separated  list  or  an  array  of  object  IDs 23Thursday, October 6, 2011
  28. 28. Tribal  Crossing:  RepresenCng  Game  Data  in  Couchbase Player  Object Key: Player1 Plant  Object JSON Key: Plant201 { “_id” : “Player1”, JSON “nid” : 1, { “name” : “Shawn” “_id” : “Plant201”, } “nid” : 201, “player_id” : 1 PlayerPlant  List } “name” : “Starflower” Key: Player1_PlantList JSON { “_id” : “Player1_Plantlist”, “plants” : [201, 202, 204] } 24Thursday, October 6, 2011
  29. 29. Tribal  Crossing:  Schema-­‐less  Game  Data ● No  need  to  “ALTER  TABLE” ● Add  new  “fields”  all  objects  at  any  9me – Specify  default  value  for  missing  fields – Increased  development  speed ● Using  JSON  for  data  objects  though,   ● Offers  the  ability  to  query  and  analyze  arbitrary  fields  in   Couchbase  2.0 25Thursday, October 6, 2011
  30. 30. Tribal  Crossing:  Accessing  Game  Data  in  Couchbase Get  all  plants  belong  to  a  given  player Request: GET /player/1/farm // Create a new PlantList from players PlantList playersPlants = new PlantList(cbclient.get("Player1_PlantList"); for (Plant plant : playersPlants) { aPlayer.addPlant(plant); } 26Thursday, October 6, 2011
  31. 31. Tribal  Crossing:  Modifying  Game  Data  in  Couchbase Give  a  player  a  new  plant // Create the new plant Plant givenPlant = new Plant(100, "Mushroom"); cbclient.set("Plant100", givenPlant); // Update the player plant list Player thePlayer = player.fetch(cbclient.get("Player1"); // Add the plant to the player thePlayer.receivePlant(givenPlant); // Store the players new plant list cbclient.set("Player1_PlantList", thePlayer.getPlantsArray()); 27Thursday, October 6, 2011
  32. 32. Tribal  Crossing:  Concurrency Concurrency  issue  can  occur  when   mul9ple  requests  are  working  with   the  same  piece  of  data. Solu9ons: ● CAS  (check-­‐and-­‐set) – Client  can  know  if  someone  else   has  modified  the  data  while  you   are  trying  to  update – Provides  op9mis9c  concurrency   control ● GETL  (get  with  lock) – Locking (try/wait cycle) – Provides  pessimis9c  concurrency   control 28Thursday, October 6, 2011
  33. 33. Tribal  Crossing:  Data  Rela9onship ● Record  object  rela9onships  both  ways – Example:    Plots  and  Plants ● Plot  object  stores  id  of  the  plant  that  it  hosts ● Plant  object  stores  id  of  the  plot  that  it  grows  on – Resolu9on  in  case  of  mismatch ● Dont  sweat  the  extra  calls  to  load  data  in  a  one-­‐to-­‐ many  rela9onship – Use  mul*Get – Tribal:  "We  can  s9ll  cache  aggregated  results  in  a   Memcache  bucket  if  needed" 29Thursday, October 6, 2011
  34. 34. Tribal  Crossing:  Migra9ng  to  Couchbase  Servers First  migrated  large  or  slow  performing  tables  and   frequently  updated  fields  from  MySQL  to  Couchbase Apache  +  PHP Web  Server Client-­‐side  Moxi MySQL memcached protocol  listener/sender engine  interface Couchbase  Storage  Engine 30Thursday, October 6, 2011
  35. 35. Tribal  Crossing:  Migra9ng  to  Couchbase  Servers First  migrated  large  or  slow  performing  tables  and   frequently  updated  fields  from  MySQL  to  Couchbase Apache  +  PHP Web  Server Client-­‐side  Moxi MySQL memcached TAP protocol  listener/sender engine  interface Repor1ng   TAP  Client Applica1ons Couchbase  Storage  Engine 30Thursday, October 6, 2011
  36. 36. Tribal  Crossing:  Deployment   31Thursday, October 6, 2011
  37. 37. Tribal  Crossing:  Deployment 32Thursday, October 6, 2011
  38. 38. Tribal  Crossing:  Conclusion • Significantly  reduced  the  cost  incurred  by  scaling  up   database  servers  and  managing  them. • Achieved  significant  improvements  in  various   performance  metrics  (e.g.,  read,  write,  latency,  etc.) • Allowed  them  to  focus  more  on  game  development  and   op9mizing  key  metrics • Plan  to  use  real-­‐9me  Map-­‐Reduce,  querying,  and   indexing  abili9es  provided  by  Couchbase  Server  2.0 33Thursday, October 6, 2011
  39. 39. DEMO 34Thursday, October 6, 2011
  40. 40. Client  Setup Based  on  popular  spymemcached  Java  client,  which  is   maintained/authored  by  Couchbase,  Inc. Client  Setup: CouchbaseClient cbclient; ArrayList<URI> servers = new ArrayList<URI>(); servers.add(new URI("http://10.1.5.122:8091/pools")); servers.add(new URI("http://10.1.5.123:8091/pools")); cbclient = new CouchbaseClient(servers, "default", ""); 35Thursday, October 6, 2011
  41. 41. Accessing  a  View Query query = new Query(); query.setReduce(false); query.setIncludeDocs(true); View view = cbclient.getView("dev_players", "player_by_level"); HttpFuture<ViewResponse> future = client.asyncQuery(view, query); ViewResponse response = future.get(); if (!future.getStatus().isSuccess()) { // do something to handle the errror } Iterator<ViewRow> itr = response.iterator(); while (itr.hasNext()) { ViewRow row = itr.next(); if (ITEMS.containsKey(row.getId())) { assert ITEMS.get(row.getId()).equals(row.getDocument()); } } 36Thursday, October 6, 2011
  42. 42. PLAYER  DOCUMENT {   "_id":  "Keith4540",   "_rev":  "1-­‐ab354009ce09f198c555b693e057adce",   "jsonType":  "player",   "uuid":  "35767d02-­‐a958-­‐4b83-­‐8179-­‐616816692de1",   "name":  "Keith4540",   "hitpoints":  75,   "experience":  663,   "level":  4,   "loggedIn":  false } 37Thursday, October 6, 2011
  43. 43. PLAYER  DOCUMENT 37Thursday, October 6, 2011
  44. 44. ITEM  DOCUMENT {   "_id":  "Katana_e5890c94-­‐11c6-­‐48-­‐65746ce6c560",   "_rev":  "1-­‐ d6bbd5e814c32c66e22db2918a2efcd9",   "jsonType":  "item",   "name":  "Katana_e5890c94-­‐11c6-­‐65746ce6c560",   "uuid":  "e5890c94-­‐11c6-­‐4856-­‐ a7a6-­‐65746ce6c560",   "ownerId":  "Dale9887" } 38Thursday, October 6, 2011
  45. 45. ITEM  DOCUMENT 38Thursday, October 6, 2011
  46. 46. MONSTER  DOCUMENT {   "_id":  "Bauchan9932",   "_rev":  "1-­‐5c90be58be58134a0fc5e7db77dab5f2",   "jsonType":  "monster",   "name":  "Bauchan9932",   "uuid":  "d10dfc1b-­‐0412-­‐4140-­‐b4ec-­‐affdbf2aa5ec",   "hitpoints":  370,   "experienceWhenKilled":  52,   "itemProbability":  0.5050581341872865 } 39Thursday, October 6, 2011
  47. 47. MONSTER  DOCUMENT 39Thursday, October 6, 2011
  48. 48. 40Thursday, October 6, 2011
  49. 49. GA ME O N! thanks  for  the  photo!  hsp://flic.kr/p/88fCcH   40  Thursday, October 6, 2011
  50. 50. Couchbase  Product  Download • Community  Edi9on – Open  source  build – Free  forum  support • Enterprise  Edi9on – Free  for  non-­‐produc9on  use – Cer9fied,  QA  tested  version  of  open  source – Case  tracking  and  guaranteed  SLA  for  produc9on   environments 41Thursday, October 6, 2011
  51. 51. THANKS  -­‐  Q&A Matt Ingenthron, Couchbase Inc. (matt@couchbase.com, @ingenthr) CouchConf near you? New York: October 24 Bangalore: December 7 Berlin: November 7 Israel: December 19 Chicago: November 16 Tokyo: January 27 42Thursday, October 6, 2011

×