CouchConf-London-nosql-social-gaming-usecase-and-migration

Couchbase Use Case Example: Social Gaming Frank Weigel Director, Product Management frank@couchbase.com 1Monday, March 26, 2012 1

Tribal Crossing: Animal Party • Tribal Crossing (FableLabs) Facebook game – Hosted on EC2 • “Part Pokemon, part FronFerville and part Mario Galaxy! In Animal Party youll discover hundreds of amazing animals across the galaxy and raise them on your magical garden.” 2Monday, March 26, 2012 2

Tribal Crossing: Challenges Common steps on scaling up RDBMS: • Tune queries (indexing, explain query) • DenormalizaFon • Cache data (Memcache) • Tune MySQL conﬁguraFon • ReplicaFon (read slaves) • Where do we go from here to prepare for the scale of a successful social game? 3Monday, March 26, 2012 3

Tribal Crossing: Challenges • Write-‐heavy requests • Caching does not help • MySQL / InnoDB limitaFon (Percona) • Need to scale drasFcally over night • My Polls – 100 to 1m users over a weekend • Small team, no dedicated sysadmin • Focus on what we do best – making games • Keeping cost down 4Monday, March 26, 2012 4

Tribal Crossing: “Old” Architecture and OpFons • MySQL with master-‐to-‐master replicaFon and sharding • Complex to setup, high administraFon cost • Requires applicaFon level changes • Cassandra • High write, but low read throughput • Live cluster reconﬁguraFon and rebalance is quite complicated • Eventual consistency gives too much burden to applicaFon developers • MongoDB • High read/write, but unpredictable latency • Live cluster rebalance for exisFng nodes only • Disk data corrupFon concerns on node failures 5Monday, March 26, 2012 5

Tribal Crossing: Why Couchbase Server? • SPEED, SPEED, SPEED • Immediate consistency • Interface is dead simple to use • We are already using Memcache • Low sysadmin overhead • Schema-‐less data store • Used and Proven by big guys like Zynga • … and lastly, because Tribal CAN • Bigger ﬁrms with legacy code base = hard to adapt • Small team = ability to get on the cujng edge 6Monday, March 26, 2012 6

Tribal Crossing: New Challenges With Couchbase • But, there are some diﬀerences in using Couchbase (currently 1.7) to handle the game data: • No easy way to query data – Couchbase Server 2.0 resolves this by using new persistence layer • Can this work for an online game? • Break out of the old ORM / relaEonal paradigm! • Tribal: “we are not handling criEcal transacEons” 7Monday, March 26, 2012 7

Tribal Crossing: Deploying Couchbase in EC2 •Basic producEon environment setup •Dev/Stage environment – feel free to install Couchbase on your web server 8Monday, March 26, 2012 8

Tribal Crossing: Deploying Couchbase in EC2 1. Amazon Linux AMI, 64-‐bit, EBS backed instance 2. Setup swap space 3. Install Couchbase’s Membase Server 1.7 4. Access web console hp://<hostname>:8091 5. Start the new cluster with a single node 6. Add the other nodes to the cluster and rebalance 9Monday, March 26, 2012 9

Tribal Crossing: Deploying Couchbase in EC2 •Moxi ﬁgures out which node in the cluster holds data for a given key. Used with older, non-‐cluster aware clients like PHP, Ruby, Perl. [smart clients for PHP & Ruby released since] •On each web server, install Moxi proxy •Start Moxi by poinEng it to the DNS entry already created •Web apps connect to Moxi that is running locally memcache-‐>addServer(‘localhost’, 10Monday, March 26, 2012 10

Tribal Crossing: RepresenFng Game Data in Use case -‐ simple version of farming part: • A player can have a variety of plants on their farm. • A player can add or remove plants from their farm. • A Player can see what plants are on another players farm. 11Monday, March 26, 2012 11

Tribal Crossing: RepresenFng Game Data in RepresenDng Objects • Simply treat an object as a Map/dicFonary/JSON object • Determine the key for an object using the class name (or type) of the object and an unique ID RepresenDng Object Lists • DenormalizaFon • Save a comma separated list or an array of object IDs 12Monday, March 26, 2012 12

Tribal Crossing: RepresenFng Game Data in Player Object Key: Player1 Plant Object JSON Key: Plant201 { “_id” : “Player1”, JSON “nid” : 1, { “name” : “Shawn” “_id” : “Plant201”, } “nid” : 201, “player_id” : 1 PlayerPlant List } “name” : “Starflower” Key: Player1_PlantList JSON { “_id” : “Player1_Plantlist”, “plants” : [201, 202, 204] } 13Monday, March 26, 2012 13

Tribal Crossing: Schema-‐less Game Data • No need to “ALTER TABLE” • Add new “fields” all objects at any Fme • Specify default value for missing fields • Increased development speed • Using JSON for data objects though, – Offers the ability to query and analyze arbitrary fields in Couchbase 2.0 14Monday, March 26, 2012 14

Tribal Crossing: Accessing Game Data in Couchbase Get all plants belong to a given player Request: GET /player/1/farm // Create a new PlantList from players PlantList playersPlants = new PlantList(cbclient.get ("Player1_PlantList"); for (Plant plant : playersPlants) { aPlayer.addPlant(plant); } 15Monday, March 26, 2012 15

Tribal Crossing: Modifying Game Data in Couchbase Give a player a new plant // Create the new plant Plant givenPlant = new Plant(100, "Mushroom"); cbclient.set("Plant100", givenPlant); // Update the player plant list Player thePlayer = player.fetch(cbclient.get ("Player1"); // Add the plant to the player thePlayer.receivePlant(givenPlant); // Store the players new plant list cbclient.set("Player1_PlantList", 16Monday, March 26, 2012 16

Tribal Crossing: Concurrency • Concurrency issue can occur when mulFple requests are working with the same piece of data. SoluDons: • CAS (check-‐and-‐set) • Client can know if someone else has modiﬁed the data while you are trying to update • Provides opFmisFc concurrency control • GETL (get with lock) • Locking (try/wait cycle) • Provides pessimisFc concurrency control 17Monday, March 26, 2012 17

Tribal Crossing: Data RelaFonship • Record object relaFonships both ways • Example: Plots and Plants – Plot object stores id of the plant that it hosts – Plant object stores id of the plot that it grows on • ResoluFon in case of mismatch • Dont sweat the extra calls to load data in a one-‐to-‐many relaFonship • Use mulFGet • Tribal: "We can sFll cache aggregated results in a Memcache bucket if needed" 18Monday, March 26, 2012 18

MIGRATING TO COUCHBASE: Moving from MySQL & memcached 19Monday, March 26, 2012 19

Tribal Crossing: MigraFng to Couchbase Servers • First migrated large or slow performing tables and frequently updated ﬁelds from MySQL to Couchbase 20Monday, March 26, 2012 20

Tribal Crossing: Deployment 21Monday, March 26, 2012 21

Tribal Crossing: Deployment 22Monday, March 26, 2012 22

Tribal Crossing: Conclusion • Signiﬁcantly reduced the cost incurred by scaling up database servers and managing them. • Achieved signiﬁcant improvements in various performance metrics (e.g., read, write, latency, etc.) • Allowed them to focus more on game development and opFmizing key metrics • Plan to use real-‐Fme Map-‐Reduce, querying, and indexing abiliFes provided by Couchbase Server 2.0 23Monday, March 26, 2012 23

Thank You! Q&A @FrankWeigel 24Monday, March 26, 2012 24

CouchConf-London-nosql-social-gaming-usecase-and-migration

by Couchbase

Accessibility

Categories

Upload Details

Usage Rights

Statistics

CouchConf-London-nosql-social-gaming-usecase-and-migration Presentation Transcript