Zend con 2011 couchbase mobile - the database that talks sync

Ma# Ingenthron @ingenthrThursday, October 20, 2011

* * Membase Server is renamed Couchbase Server as of release 2.0 2Thursday, October 20, 2011

Couchbase Mobile CouchSync Unique to Couchbase, CouchSync permits automated synchroniza8on of databases between intermi;ently-‐connected mobile devices and a datacenter-‐ or cloud-‐resident Couchbase cluster 3Thursday, October 20, 2011

2012 - $1.1 trillion MOBILE & CLOUD GSM/Wireless Intelligence study, The Global Cellular Industry Balance Sheet h"p://gigaom.com/mobile/global-‐mobile-‐revenues-‐will-‐be-‐1-‐1-‐trillion-‐in-‐2012-‐here-‐is-‐why/Thursday, October 20, 2011

The ‘Embedded Internet’ will consist of over 15 billion devices by 2015 30x growth expected MOBILE & CLOUD Intel Embedded Internet Projec8onsThursday, October 20, 2011

Thursday, October 20, 2011

iOSThursday, October 20, 2011

iOS AndroidThursday, October 20, 2011

NOT ONLY SMART PHONESThursday, October 20, 2011

NEEDS TO WORK OFFLINEThursday, October 20, 2011

Fallacies of Distributed Computing • The network is reliable. • Latency is zero. • Bandwidth is infinite. • The network is secure. • Topology doesnt change. • There is one administrator. • Transport cost is zero. • The network is homogeneous.Thursday, October 20, 2011

The Cost of Latency http://perspectives.mvdirona.com/2009/10/31/TheCostOfLatency.aspx • “Half a second delay caused a 20% drop in traffic.” - Greg Linden http://glinden.blogspot.com/2006/11/marissa-mayer-at-web-20.html • “Google Search found that a 400 millisecond delay resulted in a -0.59% change in searches/ user. Whats more, even after the delay was removed, these users still had -0.21% fewer searches, indicating that a slower user experience affects long term behavior.” - Steve Souders http://radar.oreilly.com/2009/07/velocity-making-your-site-fast.htmlThursday, October 20, 2011

ACTIVEThursday, October 20, 2011

GROUNDCOMPUTING http://www.ﬂickr.com/photos/mcpig/872293700/Thursday, October 20, 2011

http://www.ﬂickr.com/photos/mcpig/872293700/Thursday, October 20, 2011

CouchDB is the unifying core technology Memcached Membase CouchDB CouchDB CouchDB CouchSync CouchSync enables automated, bi-‐direc8onal synchroniza8on of Couchbase servers 13Thursday, October 20, 2011

DEMO: CLUSTERING 14Thursday, October 20, 2011

CASE STUDY: TRIBAL CROSSING 15Thursday, October 20, 2011

Tribal Crossing: Challenges Common steps on scaling up database: ● Tune queries (indexing, explain query) ● DenormalizaAon ● Cache data (APC / Memcache) ● Tune MySQL conﬁguraAon ● ReplicaAon (read slaves) Where do we go from here to prepare for the scale of a successful social game? 16Thursday, October 20, 2011

Tribal Crossing: Challenges ● Write-‐heavy requests – Caching does not help – MySQL / InnoDB limitaAon (Percona) ● Need to scale drasAcally over night – My Polls – 100 to 1m users over a weekend ● Small team, no dedicated sysadmin – Focus on what we do best – making games ● Keeping cost down 17Thursday, October 20, 2011

Tribal Crossing: Why Couchbase Server? ● SPEED, SPEED, SPEED ● Immediate consistency ● Interface is dead simple to use – We are already using Memcache ● Low sysadmin overhead ● Schema-‐less data store ● Used and Proven by big guys like Zynga ● … and lastly, because Tribal CAN – Bigger ﬁrms with legacy code base = hard to adapt – Small team = ability to get on the cu[ng edge 18Thursday, October 20, 2011

Tribal Crossing: Deploying Couchbase in EC2 Web Server Apache ● Basic producAon Client-side Moxi environment setup ● Dev/Stage environment – Cluster Mgmt. Requests feel free to install DNS Entry Couchbase on your web server Couchbase … Couchbase Couchbase Cluster 19Thursday, October 20, 2011

Tribal Crossing: Deploying Couchbase in EC2 Web Server ● Amazon Linux AMI, Apache 64-‐bit, EBS backed instance Client-side Moxi ● Setup swap space Cluster Mgmt. Requests ● Install Couchbase’s Membase Server 1.7 DNS Entry ● Access web console http://<hostname>:8091 ● Start the new cluster with a Couchbase … Couchbase single node ● Add the other nodes to the Couchbase Cluster cluster and rebalance 20Thursday, October 20, 2011

Tribal Crossing: Deploying Couchbase in EC2 Web Server Apache Moxi ﬁgures out which node in the cluster holds data for a given key. Client-side Moxi ● On each web server, install Moxi Cluster Mgmt. Requests proxy ● Start Moxi by poin8ng it to the DNS DNS Entry entry you created ● Web apps connect to Moxi that is running locally memcache->addServer(‘localhost’, 11211); Couchbase … Couchbase Couchbase Cluster 21Thursday, October 20, 2011

Tribal Crossing: Represen1ng Game Data in Couchbase Use case -‐ simple farming game: ● A player can have a variety of plants on their farm. ● A player can add or remove plants from their farm. ● A Player can see what plants are on another players farm. 22Thursday, October 20, 2011

Tribal Crossing: Represen1ng Game Data in Couchbase Represen@ng Objects ● Simply treat an object as an associaAve array ● Determine the key for an object using the class name (or type) of the object and an unique ID Represen@ng Object Lists ● DenormalizaAon ● Save a comma separated list or an array of object IDs 23Thursday, October 20, 2011

Tribal Crossing: Represen1ng Game Data in Couchbase Player Object Key: Player1 Plant Object Array Key: Plant201 ( [Id] => 1 Array [Name] => Shawn ( ) [Id] => 201 [Player_Id] => 1 [Name] => Starflower PlayerPlant List ) Key: Player1_PlantList Array ( [0] => 201 [1] => 202 [2] => 204 ) 24Thursday, October 20, 2011

Tribal Crossing: Schema-‐less Game Data ● No need to “ALTER TABLE” ● Add new “fields” all objects at any Ame – Specify default value for missing fields – Increased development speed ● Using JSON for data objects though, owing to the ability to query on arbitrary fields in Couchbase 2.0 25Thursday, October 20, 2011

Tribal Crossing: Accessing Game Data in Couchbase Get all plants belong to a given player Request: GET /player/1/farm $plant_ids = couchbase->get(Player1_PlantList); $response = array(); foreach ($plant_ids as $plant_id) { $plant = couchbase->get(Plant . $plant_id); $response[] = $plant; } echo json_encode($response); 26Thursday, October 20, 2011

Tribal Crossing: Modifying Game Data in Couchbase Give a player a new plant // Create the new plant $new_plant = array ( id => 100, name => Mushroom ); $couchbase->set(Plant100, $new_plant); // Update the player plant list $plant_ids = $couchbase->get(Player1_PlantList); $plant_ids[] = $new_plant[id]; $couchbase->set(Player1_PlantList, $plant_ids); 27Thursday, October 20, 2011

Tribal Crossing: Concurrency Concurrency issue can occur when mul1ple requests are working with the same piece of data. Solu1on: ● CAS (check-‐and-‐set) – Client can know if someone else has modiﬁed the data while you are trying to update – Implement op1mis1c concurrency control ● Locking (try/wait cycle) – GETL (get with lock + 1meout) opera1ons – Pessimis1c concurrency control 28Thursday, October 20, 2011

Tribal Crossing: Data RelaAonship ● Record object relaAonships both ways – Example: Plots and Plants ● Plot object stores id of the plant that it hosts ● Plant object stores id of the plot that it grows on – ResoluAon in case of mismatch ● Dont sweat the extra calls to load data in a one-‐to-‐ many relaAonship – Use mul$Get – We can sAll cache aggregated results in a Memcache bucket if needed 29Thursday, October 20, 2011

Tribal Crossing: MigraAng to Couchbase Servers First migrated large or slow performing tables and frequently updated ﬁelds from MySQL to Couchbase Apache + PHP Web Server Client-‐side Moxi MySQL memcached protocol listener/sender engine interface Couchbase Storage Engine 30Thursday, October 20, 2011

Tribal Crossing: MigraAng to Couchbase Servers First migrated large or slow performing tables and frequently updated ﬁelds from MySQL to Couchbase Apache + PHP Web Server Client-‐side Moxi MySQL memcached TAP protocol listener/sender engine interface ReporRng TAP Client ApplicaRons Couchbase Storage Engine 30Thursday, October 20, 2011

Tribal Crossing: Deployment 31Thursday, October 20, 2011

Tribal Crossing: Deployment 32Thursday, October 20, 2011

DEMO: GAME SIMULATION 33Thursday, October 20, 2011

PLAYER DOCUMENT { "_id": "Keith4540", "_rev": "1-‐ab354009ce09f198c555b693e057adce", "jsonType": "player", "uuid": "35767d02-‐a958-‐4b83-‐8179-‐616816692de1", "name": "Keith4540", "hitpoints": 75, "experience": 663, "level": 4, "loggedIn": false } 34Thursday, October 20, 2011

PLAYER DOCUMENT 34Thursday, October 20, 2011

ITEM DOCUMENT { "_id": "Katana_e5890c94-‐11c6-‐48-‐65746ce6c560", "_rev": "1-‐ d6bbd5e814c32c66e22db2918a2efcd9", "jsonType": "item", "name": "Katana_e5890c94-‐11c6-‐65746ce6c560", "uuid": "e5890c94-‐11c6-‐4856-‐ a7a6-‐65746ce6c560", "ownerId": "Dale9887" } 35Thursday, October 20, 2011

ITEM DOCUMENT 35Thursday, October 20, 2011

MONSTER DOCUMENT { "_id": "Bauchan9932", "_rev": "1-‐5c90be58be58134a0fc5e7db77dab5f2", "jsonType": "monster", "name": "Bauchan9932", "uuid": "d10dfc1b-‐0412-‐4140-‐b4ec-‐aﬀdbf2aa5ec", "hitpoints": 370, "experienceWhenKilled": 52, "itemProbability": 0.5050581341872865 } 36Thursday, October 20, 2011

MONSTER DOCUMENT 36Thursday, October 20, 2011

37Thursday, October 20, 2011

GA ME O N! thanks for the photo! h;p://ﬂic.kr/p/88fCcH 37 Thursday, October 20, 2011

Couchbase Product Download • Community EdiAon – Open source build – Free forum support • Enterprise EdiAon – Free for non-‐producAon use – CerAﬁed, QA tested version of open source – Case tracking and guaranteed SLA for producAon environments 38Thursday, October 20, 2011

THANKS -‐ Q&A Matt Ingenthron, Couchbase Inc. (matt@couchbase.com, @ingenthr) CouchConf near you? couchbase.com/couchconf New York: October 24 Bangalore: December 7 Berlin: November 7 Israel: December 19 Chicago: November 16 Tokyo: January 27 39Thursday, October 20, 2011

Resources http://www.couchbase.org http://guide.couchdb.org @Couchbase #CouchConfThursday, October 20, 2011

Zend con 2011 couchbase mobile - the database that talks sync

by Couchbase

Accessibility

Categories

Upload Details

Usage Rights

Statistics