Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mongo DB in gaming industry

6,394 views

Published on

Published in: Technology
  • Be the first to comment

Mongo DB in gaming industry

  1. 1. MongoDB in KIXEYE AnalyticsBen Goswami
  2. 2. Game Play BP Video clips ( http://www.youtube.com/watch?v=uUHzppVn8WM)©2011 Kixeye, Private and Confidential
  3. 3. Agenda • Data scaling problems at KIXEYE analytics and what technologies we use • Example application: Cheater Detection Engine • How MongoDB helped us to solve it©2011 Kixeye, Private and Confidential
  4. 4. Game Play BP Attack Video (https://www.youtube.com/watch?v=qud_-18D2MU)©2011 Kixeye, Private and Confidential
  5. 5. Game Play and Data • Games generate lots of click data • Click = event o time of the event o what happened before or after o how the events are correlated©2011 Kixeye, Private and Confidential
  6. 6. Attack: { attacker : { user:123 player_level: 31 fleet: { 1:{ Weapons: { name:“Ripper Cannon IV” damage:306 accuracy:55 } armors:{ } … } 2: { }_id" : ObjectId("506e23001d4524122c00004b"), "ip" : "10.54.72.206", "dt" :{ "_id" :ObjectId("506e23001d4524dd2b0000bd"), "ip" : "10.54.72.206", }"&attacker_level=42&defender=0&defender_ext=salvage&defender_level=25&attack_type=PVE&attacking_ }units=%7b%224071%22%3a%5b5%2c3%2c0%5d%7d&defending_units=%7b%221040%22%3a%5b4%2c defender: {0%2c4%5d%2c%221060%22%3a%5b1%2c0%2c1%5d%7d&attacking_shipconfig=%7b%22ships%22%3a }%5b%7b%22armors%22%3a%5b333%2c333%2c333%2c333%5d%2c%22weapons%22%3a%5b134%2c1 attack_time: 10035456334%2c193%2c134%2c134%2c134%2c134%2c134%5d%2c%22specials%22%3a%5b542%2c561%2c571 attack_duration:290%2c501%5d%2c%22hullID%22%3a4071%2c%22tacticalModules%22%3a%5b%5d%2c%22actives%22%3 sector:7a%7b%22hp%22%3a9317.68457%2c%22rank%22%3a4%2c%22fltp%22%3a5%2c%22bid%22%3a54834 …79%2c%22f%22%3a0%2c%22id%22%3a110%2c%22flt%22%3a5%2c%22vxp%22%3a10.858466%7d%7d%2c%7b%22armors%22%3a%5b333%2c333%2c333%2c333.. "stage" : "new" } }
  7. 7. How much data? •Multi TBs / month (4-5x in next 6 months) •Critical : Volume reads with speed •< 1% data has actionable intelligence©2011 Kixeye, Private and Confidential
  8. 8. MongoDB in KIXEYE •Why: – fast read + writes for high volume – Document based architecture, schema less – Indexing , auto failover, auto sharding •Where: – Games , KIXEYE Platform – Analytics, Logging©2011 Kixeye, Private and Confidential
  9. 9. Data technologies at Analytics Technology Data Volume Query Type I/O speed MySQL Small Adhoc + Read fast, only < 300 GB Pre defined Aggregated data Mongo Mid Predefined Read + Write speed < 2 TB fast Hadoop X-Large Adhoc Read slow (Hive+HDFS) 2 TB+ writes: only insert no update©2011 Kixeye, Private and Confidential
  10. 10. An Example Application Cheater Detection engine for games: How mongoDB helped us©2011 Kixeye, Private and Confidential
  11. 11. A simple backyard at start in BYM
  12. 12. Currency is time or $ or both©2011 Kixeye, Private and Confidential
  13. 13. Sample record • 4kb nested jason block • base = { base_name: “sweet_mango_db” resources { 1: { name: "storage silo" level: 5, mode:“built" time:123456902 2: { … ..} ...} ...} ..}©2011 Kixeye, Private and Confidential
  14. 14. How people cheat? • Each upgrade or repair takes pre defined time • if time+payment < predefined time = cheater • victims chain create snowball effect©2011 Kixeye, Private and Confidential
  15. 15. ..and why they cheat...©2011 Kixeye, Private and Confidential
  16. 16. Data volume for cheater detection • 16Mil * 50 * 10 * 3 *.... • multiple time based snapshots • 0.01 % cheaters • Detect < 5 mins • Thats searching through lot of data fast©2011 Kixeye, Private and Confidential
  17. 17. .. and boss told • Catch and auto ban cheaters • Dont label a good guy a cheater©2011 Kixeye, Private and Confidential
  18. 18. ..team was ready for the challenge • Speed+Accuracy of application • Technology evalauation©2011 Kixeye, Private and Confidential
  19. 19. MongoDB fits the requirement • read writes are fast because memory mapped files compared to mysql+memcache or hbase • Unstructured data => need schema less • indexing on a nested data element©2011 Kixeye, Private and Confidential
  20. 20. .. more on why mongo • Auto purging thru TTL • Auto sharding during traffic spikes • Binary data support, fast uncompression • Capped collection: writes faster©2011 Kixeye, Private and Confidential
  21. 21. Development decision points • Old map-reduce (prior to 2.2) vs New Aggregation framework • Global write locks removal in 2.2 • Lazy and Non lazy inserts©2011 Kixeye, Private and Confidential
  22. 22. Result • 100+ cheaters caught per hour • Better game balancing • $$ saved©2011 Kixeye, Private and Confidential
  23. 23. Wrap up : A good partnership •Good partnership with KIXEYE and 10gen •Consulting/Training and MMS service was very helpful •More mongoDB usages in future at KIXEYE©2011 Kixeye, Private and Confidential
  24. 24. {Status: “The End”} •{ next_action: “Q&A” } • Like this jason? meet us outside – KIXEYE recruiting { Company : “KIXEYE”, “Engineers/Total _employee” : “65%”, Culture : [“Fun”, “Innovative”, “Passionate”] Analytics_engg : { technology : “mongo, hadoop , mysql”, tag : “big data, geek” } }©2011 Kixeye, Private and Confidential
  25. 25. Extra slides
  26. 26. Game Play BP Attack Video (https://www.youtube.com/watch?v=qud_-18D2MU)©2011 Kixeye, Private and Confidential
  27. 27. Usage of data• Data mining and aggregation to o Optimize User Acquisition o Optimize game analyzing game play behavior and Algo + Data Optimize r Feedback
  28. 28. Mongo - pain points • Diskspace handling • No sql syntax support • Capped collection - no deletes allowed • No good client GUI based tool©2011 Kixeye, Private and Confidential
  29. 29. Damaged building is useless
  30. 30. Data related to player • building has 5-10 levels • building can have 3 states [built, damaged, repair in progress) • an average base can have 50+ buildings • 16 million such bases • 45k people at any second [i.e base gets updated] • Each level up or repair has time delay. People pay to reduce delay©2011 Kixeye, Private and Confidential
  31. 31. Map-reduce in cheater detection function mapf() { emit(this.userid, {userid:this.userid, total_time:this.length, bldg_id:1} ; } function reducef(key, values) { var r = {userid:key, total_time:0, count:0, avg_time:0}; values.forEach(function(v) {r.total_time += v.total_time; }); return r; } function finalizef(key, value) { ... }©2011 Kixeye, Private and Confidential
  32. 32. The End •Q&A •Love numbers and big data? We have a job for you at analytics engg at KIXEYE. Talk to our recruiting team©2011 Kixeye, Private and Confidential
  33. 33. To scale data handling • Write speed for large volume data • Read speed + volume [aggregate+find] • MongoDB provides good balance compared to other sql+nosql solutions for certain use cases©2011 Kixeye, Private and Confidential
  34. 34. Games+Platforms devices mysql mongo BI Mongo Cluster ETL non Real time >= 1 hr real time 5API mins mySQL Mongo Hadoop mySQL /mongo Admin console Data access tools Dashboards Hive sql query API Scheduler, Analytics Components - Birds Eye View

×