Mongo or Die: How MongoDB Powers Doodle or Die

Mongo or Die!How MongoDB powers Doodle or Die Aaron Silverman (@Zugwalt)

Doodle or Die@DoodleOrDie

What is Doodle or Die?

TelephonePhrase Id like some beer!Phrase Id like some deerPhrase I see no deerPhrase I’ve no idea!

Doodle or DieDoodlePhrase Shining AppleDoodlePhrase Eat your fruit or DIE!

What Powers Doodle or Die?

Started very, very small 4 Cores 128MB RAM Node Server MongoDB Server

Got serious about our servers Small - 2GB 8 Cores MongoDB Server 256MB RAM Node Server

Called in some Reinforcements Large - 5GB 12 Cores MongoDB Server 1GB RAM Node Server

In the last 30 Days:• 2,500,000 page views• 100,000 uniques• 35,000 active player accounts• 2,000,000 new doodles and descriptions

“Small Data”Mongo DB • Player Info • Chain Info (excluding4 GB data size doodles)<1 GB index size • Group Info~10 queries/sec • Game State • LogsAmazon170 GB data size • Doodles • Static Content8 GB in/month • Compressed200 GB out/month Database Backups

MongoDB - $65 / monthDaily backups to Amazon S3: $16/mo

Amazon S3 - $70 / month

Node - $62/ month

Total Cost To Host: $197/month MongoDB $65 Amazon S3 $70 Node $62 Total $197

PAAS Provides Easy Upgrade Path

General Principles

Custom _idPartially random string generated usingShortId node module ObjectId ObjectId("4fd02d5d78315a502d15cdde") ObjectId("4fd02d5a78315a502d15cddd") ObjectId("4fd02d5878315a502d15cddc") ShortId "8rOIwh2VD" "1qyY61Lu1" "5GQnbx-1"

• Shorter, less cumbersome in code and queriesdb.players.findOne({_id: ‘58mwYlTKV’});db.chains.update({_id: ‘58mwYlTKV’}, {$set: activePlayer_id: ‘88ueYaL6V’});• Randomness could help with sharding; more importantly makes it harder to cheathttp://doodleordie.com/c/5ONtvvSGH<span class="doodle" data-jsonp="http://doodles.s3.amazonaws.com/d2/Eh8-Po2R5/1Em5kj3LY.js">

Question Oriented Subdocuments What chain is this player working on right now? What are this player’s stats? Which players are not eligible to be assigned this chain?

Goal is for most “Questions” to be able tobe answered in one query from one subdocumentdb.players.findOne({_id: ‘58mwYlTKV’}, {‘game.recentSkips’: 1});Related “Questions” will share commonancestorsdb.players.findOne({_id: ‘58mwYlTKV’}, {game: 1});

Indexes are designed to make answeringquestions easy!Which player is working on this chain?db.players.ensureIndex({‘game.activeChain_id’: 1});What chains are recently awaiting a new doodle? db.chains.ensureIndex({inUse: -1, activeState: 1 lastModified" : -1});

Doodle or Die Collections

Disclaimer: Much of the original Doodle orDie code and schema were created during aweekend long hackathon!

playersPrimary chains groups sessionsSupport log

Players

players game OftenchainHistory stats Query Frequency account login Rarely info

players game • activeState • activeChain_id chainHistory • activeStepIndex • recentSkips stats accountAnswerslogin question:“What is this player working on right now?” info db.players.findOne({_id: ‘58mwYlTKV’}, {game: 1});

players Chain1 game • datePlayed • dateViewed chainHistory stats account ChainN loginAnswers the question:“What has the player worked on?” infodb.players.findOne({_id: ‘58mwYlTKV’}, {chainHistory: 1});

players Chain1 game • datePlayed • dateViewed chainHistory stats account ChainNYUCK! loginPlan to refactor out along with a refactor of infohow chains store steps

players • totalSteps game • drawSteps • phraseSteps chainHistory • numSkips • numLikes stats account loginAnswers the question:“How active/good is this player?” infodb.players.findOne({_id: ‘58mwYlTKV’}, {stats: 1});

Chains

chains• activePlayer_id• activeState• numSteps• lastModified ineligiblePlayer_ids stepschains are assigned (not as many questions)db.chains.findOne({inUse: false, activeState : player.game.activeState, ineligiblePlayer_ids: {$ne: player._id}, lastModified: { $gte: timeRange}});

chains [player_id1 player_id2,• activePlayer_id player_id3• activeState …• numSteps player_idN]• lastModified * ineligiblePlayer_idsNote: $addToSet and $pull work great in stepsmaintaining this array

chains [• activePlayer_id Step1• activeState • player_id• numSteps • state• lastModified • date content * ineligiblePlayer_ids StepN steps ]

Description Step Content: • phraseDoodle Step Content: • url (points to S3) • time • numStrokes • numDistinctColors

We plan to stop embedding steps in chains,and link to “doodles” and “descriptions”collections doodles descriptions • player_id • player_id • date • date • url (points to S3) • text • time • numStrokes • numDistinctColors

How Does it all Work?

players queried for users who are associatedwith the authenticated twitter accountdb.players.find({‘login.twitter.uid’: ‘XXXXXXX’}, {game: 1, chainHistory: 1});chain history used to load thumbnails,previous chain is loadeddb.chains.find({_id: {$in: [player.game.activeChain_id, player.game.lastChain_id]}});

chains collection searched and atomicallyupdated for unclaimed eligible chain to begiven to player db.chains.findAndModify( {inUse: false, activeState : player.game.activeState, ineligiblePlayer_ids: {$ne: player._id}, lastModified: { $gte: timeRange}}, {$set: {inUse: true, activePlayer_id: player._id, $addToSet: {ineligiblePlayer_ids: player._id}});

Content saved to chaindb.chains.update({_id: chain._id, activePlayer_id: player._id}}, {$inc: {numSteps: 1}, $set: inUse: false, lastModified: datePlayed} $unset: {activePlayer_id: 1}, $push: {steps: { player_id: player._id, state: chain.activeState, content: content, date: datePlayed}}});

Doodle strokes will be saved to S3(but url to S3 and metadata saved to chain){"color":"#000000","size":15,"path":[307,66,308,66,308,68,308,69,308,70,308,71,308,72,308,73,306,76,305,79,305,81,302,83,302,84,302,85,302,86,301,87,300,89,300,90,300,91,300,92,300,95,300,97,300,102,300,103,300,104,300,108,300,109,300,110,301,114,303,116,304,116,305,119,305,121,307,124,309,127,310,130,310,131,313,132,314,133,316,137,317,138,320,140,321,140,323,143,326,143,328,143,333,144,337,144,341,144,343,144,347,143,352,143,354,141,357,140,358,140,359,139,362,138,363,138,365,137,368,135,369,132,370,131,371,130,374,128,375,128,376,125,378,125,379,124,380,123,381,123,382,120,385,119,386,118,388,115,391,112,394,109,394,107,394,106,397,104,397,103,397,102,397,101,397,100,397,98,397,96,397,94,397,93,397,91,397,90,397,88,397,87,397,86,397,83,395,80,395,79,395,77,394,74,393,72,393,70,393,67,392,65,391,63,389,62,388,59,386,57,383,54,383,52,381,49,379,48,378,47,376,46,375,44,374,43,372,43,370,42,369,42,368,42,364,41,362,41,359,41,355,41,351,42,349,42,347,42,346,44,343,44,342,45,339,46,337,47,336,48,333,49,332,51,330,51,328,51,327,52,326,53,323,53,322,53,321,54,320,55,317,55,316,56,314,58,309,59,306,61,306,62,305,62,304,63]}

Player’s chainHistory retrieveddb.players.find({‘login.urlSafe’: urlSafeUid}, {chainHistory: 1});chainHistory filtered and sorted onserver, Applicable chains/steps retrieveddb.chains.find({$in: chain_idArr});Retrieved chains ordered (on server) basedon previously sorted chainHistory

Stats are loaded from the counting log whichis essentially a bunch of countersincrementeddb.log.find({_id: {$in: [‘2012-06-20’, ‘2012-06-20’,‘2012-06-20’, ‘2012-06-20’]}});Extremely simple implementation usingnested subdocuments for organization

Some Additional Doodle or Die Tricks

Build assumptions into queries

Alice and Bob are in an awesome groupgroups:{ _id: ‘8rOIwh2VD’, members: [ {name: ‘Alice’, dateJoined: ‘2012-05-24’}, {name: ‘Bob’, dateJoined: ‘2012-05-25’} ] banned: [ {name: ‘Eve’, dateBanned: ‘2012-05-25’} ]}Eve tries to join despite being banned!

Bugs in our code fail to detect Eve’s trickeryand the update query is run!groups.update( {_id: ‘8rOIwh2VD’}, {$push: {members: {name: ‘Eve’, dateJoined: new Date()} }, }, function(err) { if (err) throw err; callback(); });

groups:{ _id: ‘8rOIwh2VD’, members: [ {name: ‘Alice’, dateJoined: ‘2012-05-24’}, {name: ‘Bob’, dateJoined: ‘2012-05-25’}, {name: ‘Eve’, dateJoined: ‘2012-05-26’} ] banned: [ {name: ‘Eve’, dateRequested: ‘2012-05-25’} ]}Blast! Eve got through! How can we helpprevent this?

Bake in our assumptions!groups.update( {_id: ‘8rOIwh2VD’, ‘members.name’: {$ne: ‘Eve’}, ‘banned.name’: {$ne: ‘Eve’}}, {$push: {members: {name: ‘Eve’, dateJoined: new Date()} }, }, function(err, updateCount) { if (err) throw err; if (updateCount !== 1) throw new Error(‘bugs!’); callback(updateCount === 1); });

Always Specify Fields

To assign a player a new chain, we only needtheir _id and game information players game chainHistory stats account login info

To load a player’s profile content, we justneed their history, stats, and profile info players game chainHistory stats account login info

We need to assign Bob a new chain, letsfigure out what state he needs nextdb.players.find({name: ‘Bob’});This will fetch and send back the ENTIREplayer object!

Lets specify that we only want our “game”subdocumentdb.players.find({name: ‘Bob’}, {fields: [‘game’]});Hooray! Much less to retrieve and sendover the series of tubes.

Store Everything!

If you have information available, save iteven if you don’t plan on using it!db.chains.update({_id: ‘2VmVO18hs’}, {$push: {player_id: ‘1qyY61Lu1’, state: ‘draw’, date: new Date(), content: {time: 106896, count: 27, width: 520, height: 390, step_id: ‘1i7RlFbgU’ } });

Some Pain Points

Less obvious database structuremysql> DESC players;+----------------+---------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+----------------+---------+------+-----+---------+-------+| id | int(11) | NO | PRI | NULL | || activeStateId | int(11) | YES | | NULL | || activeStepId | int(11) | YES | | NULL | |+----------------+---------+------+-----+---------+-------+

Data integrity up to applicationMysql> ALTER TABLE players ADD CONSTRAINT fk_players_chains FOREIGN KEY (activeChainId) REFERENCES chains(id);/* Oh no! This is going to mess things up! */players.update({_id: ‘c58D4’}, {$set: {‘game.activeChain_id’: ‘bad_id’}});

No mature GUI query development tools

If we could start over would we still use MongoDB?

Absolutely!• NoSQL in general is great for rapid prototyping• Fantastic performance• Intuitive language keeps it easy to run one-off queries / refactor schema• Excellent (and now officially supported) Node driver

Questions?@Zugwalt @DoodleOrDie

http://www.redditmedia.com	28
https://twitter.com	24
http://tweets.joshrobinson.com	13
http://www.twylah.com	5
https://si0.twimg.com	5
http://us-w1.rockmelt.com	4
http://localhost	3
http://www.linkedin.com	3
http://shardex.by	2

Mongo or Die: How MongoDB Powers Doodle or Die

by Zugwalt on Jun 26, 2012

Accessibility

Categories

Upload Details

Usage Rights

9 Embeds 87

Statistics

Mongo or Die: How MongoDB Powers Doodle or Die Presentation Transcript