Mongo DB in gaming industry
Upcoming SlideShare
Loading in...5
×
 

Mongo DB in gaming industry

on

  • 1,936 views

 

Statistics

Views

Total Views
1,936
Slideshare-icon Views on SlideShare
1,649
Embed Views
287

Actions

Likes
1
Downloads
18
Comments
0

4 Embeds 287

http://g33ktalk.com 244
http://www.hakkalabs.co 41
http://twimblr.appspot.com 1
https://hakka.herokuapp.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Wow that is fun

Mongo DB in gaming industry Mongo DB in gaming industry Presentation Transcript

  • MongoDB in KIXEYE AnalyticsBen Goswami
  • Game Play BP Video clips ( http://www.youtube.com/watch?v=uUHzppVn8WM)©2011 Kixeye, Private and Confidential
  • Agenda • Data scaling problems at KIXEYE analytics and what technologies we use • Example application: Cheater Detection Engine • How MongoDB helped us to solve it©2011 Kixeye, Private and Confidential
  • Game Play BP Attack Video (https://www.youtube.com/watch?v=qud_-18D2MU)©2011 Kixeye, Private and Confidential
  • Game Play and Data • Games generate lots of click data • Click = event o time of the event o what happened before or after o how the events are correlated©2011 Kixeye, Private and Confidential
  • Attack: { attacker : { user:123 player_level: 31 fleet: { 1:{ Weapons: { name:“Ripper Cannon IV” damage:306 accuracy:55 } armors:{ } … } 2: { }_id" : ObjectId("506e23001d4524122c00004b"), "ip" : "10.54.72.206", "dt" :{ "_id" :ObjectId("506e23001d4524dd2b0000bd"), "ip" : "10.54.72.206", }"&attacker_level=42&defender=0&defender_ext=salvage&defender_level=25&attack_type=PVE&attacking_ }units=%7b%224071%22%3a%5b5%2c3%2c0%5d%7d&defending_units=%7b%221040%22%3a%5b4%2c defender: {0%2c4%5d%2c%221060%22%3a%5b1%2c0%2c1%5d%7d&attacking_shipconfig=%7b%22ships%22%3a }%5b%7b%22armors%22%3a%5b333%2c333%2c333%2c333%5d%2c%22weapons%22%3a%5b134%2c1 attack_time: 10035456334%2c193%2c134%2c134%2c134%2c134%2c134%5d%2c%22specials%22%3a%5b542%2c561%2c571 attack_duration:290%2c501%5d%2c%22hullID%22%3a4071%2c%22tacticalModules%22%3a%5b%5d%2c%22actives%22%3 sector:7a%7b%22hp%22%3a9317.68457%2c%22rank%22%3a4%2c%22fltp%22%3a5%2c%22bid%22%3a54834 …79%2c%22f%22%3a0%2c%22id%22%3a110%2c%22flt%22%3a5%2c%22vxp%22%3a10.858466%7d%7d%2c%7b%22armors%22%3a%5b333%2c333%2c333%2c333.. "stage" : "new" } }
  • How much data? •Multi TBs / month (4-5x in next 6 months) •Critical : Volume reads with speed •< 1% data has actionable intelligence©2011 Kixeye, Private and Confidential
  • MongoDB in KIXEYE •Why: – fast read + writes for high volume – Document based architecture, schema less – Indexing , auto failover, auto sharding •Where: – Games , KIXEYE Platform – Analytics, Logging©2011 Kixeye, Private and Confidential
  • Data technologies at Analytics Technology Data Volume Query Type I/O speed MySQL Small Adhoc + Read fast, only < 300 GB Pre defined Aggregated data Mongo Mid Predefined Read + Write speed < 2 TB fast Hadoop X-Large Adhoc Read slow (Hive+HDFS) 2 TB+ writes: only insert no update©2011 Kixeye, Private and Confidential
  • An Example Application Cheater Detection engine for games: How mongoDB helped us©2011 Kixeye, Private and Confidential
  • A simple backyard at start in BYM
  • Currency is time or $ or both©2011 Kixeye, Private and Confidential
  • Sample record • 4kb nested jason block • base = { base_name: “sweet_mango_db” resources { 1: { name: "storage silo" level: 5, mode:“built" time:123456902 2: { … ..} ...} ...} ..}©2011 Kixeye, Private and Confidential
  • How people cheat? • Each upgrade or repair takes pre defined time • if time+payment < predefined time = cheater • victims chain create snowball effect©2011 Kixeye, Private and Confidential
  • ..and why they cheat...©2011 Kixeye, Private and Confidential
  • Data volume for cheater detection • 16Mil * 50 * 10 * 3 *.... • multiple time based snapshots • 0.01 % cheaters • Detect < 5 mins • Thats searching through lot of data fast©2011 Kixeye, Private and Confidential
  • .. and boss told • Catch and auto ban cheaters • Dont label a good guy a cheater©2011 Kixeye, Private and Confidential
  • ..team was ready for the challenge • Speed+Accuracy of application • Technology evalauation©2011 Kixeye, Private and Confidential
  • MongoDB fits the requirement • read writes are fast because memory mapped files compared to mysql+memcache or hbase • Unstructured data => need schema less • indexing on a nested data element©2011 Kixeye, Private and Confidential
  • .. more on why mongo • Auto purging thru TTL • Auto sharding during traffic spikes • Binary data support, fast uncompression • Capped collection: writes faster©2011 Kixeye, Private and Confidential
  • Development decision points • Old map-reduce (prior to 2.2) vs New Aggregation framework • Global write locks removal in 2.2 • Lazy and Non lazy inserts©2011 Kixeye, Private and Confidential
  • Result • 100+ cheaters caught per hour • Better game balancing • $$ saved©2011 Kixeye, Private and Confidential
  • Wrap up : A good partnership •Good partnership with KIXEYE and 10gen •Consulting/Training and MMS service was very helpful •More mongoDB usages in future at KIXEYE©2011 Kixeye, Private and Confidential
  • {Status: “The End”} •{ next_action: “Q&A” } • Like this jason? meet us outside – KIXEYE recruiting { Company : “KIXEYE”, “Engineers/Total _employee” : “65%”, Culture : [“Fun”, “Innovative”, “Passionate”] Analytics_engg : { technology : “mongo, hadoop , mysql”, tag : “big data, geek” } }©2011 Kixeye, Private and Confidential
  • Extra slides
  • Game Play BP Attack Video (https://www.youtube.com/watch?v=qud_-18D2MU)©2011 Kixeye, Private and Confidential
  • Usage of data• Data mining and aggregation to o Optimize User Acquisition o Optimize game analyzing game play behavior and Algo + Data Optimize r Feedback
  • Mongo - pain points • Diskspace handling • No sql syntax support • Capped collection - no deletes allowed • No good client GUI based tool©2011 Kixeye, Private and Confidential
  • Damaged building is useless
  • Data related to player • building has 5-10 levels • building can have 3 states [built, damaged, repair in progress) • an average base can have 50+ buildings • 16 million such bases • 45k people at any second [i.e base gets updated] • Each level up or repair has time delay. People pay to reduce delay©2011 Kixeye, Private and Confidential
  • Map-reduce in cheater detection function mapf() { emit(this.userid, {userid:this.userid, total_time:this.length, bldg_id:1} ; } function reducef(key, values) { var r = {userid:key, total_time:0, count:0, avg_time:0}; values.forEach(function(v) {r.total_time += v.total_time; }); return r; } function finalizef(key, value) { ... }©2011 Kixeye, Private and Confidential
  • The End •Q&A •Love numbers and big data? We have a job for you at analytics engg at KIXEYE. Talk to our recruiting team©2011 Kixeye, Private and Confidential
  • To scale data handling • Write speed for large volume data • Read speed + volume [aggregate+find] • MongoDB provides good balance compared to other sql+nosql solutions for certain use cases©2011 Kixeye, Private and Confidential
  • Games+Platforms devices mysql mongo BI Mongo Cluster ETL non Real time >= 1 hr real time 5API mins mySQL Mongo Hadoop mySQL /mongo Admin console Data access tools Dashboards Hive sql query API Scheduler, Analytics Components - Birds Eye View