Mongo DB in gaming industry

5,142 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,142
On SlideShare
0
From Embeds
0
Number of Embeds
377
Actions
Shares
0
Downloads
38
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Wow that is fun
  • Mongo DB in gaming industry

    1. 1. MongoDB in KIXEYE AnalyticsBen Goswami
    2. 2. Game Play BP Video clips ( http://www.youtube.com/watch?v=uUHzppVn8WM)©2011 Kixeye, Private and Confidential
    3. 3. Agenda • Data scaling problems at KIXEYE analytics and what technologies we use • Example application: Cheater Detection Engine • How MongoDB helped us to solve it©2011 Kixeye, Private and Confidential
    4. 4. Game Play BP Attack Video (https://www.youtube.com/watch?v=qud_-18D2MU)©2011 Kixeye, Private and Confidential
    5. 5. Game Play and Data • Games generate lots of click data • Click = event o time of the event o what happened before or after o how the events are correlated©2011 Kixeye, Private and Confidential
    6. 6. Attack: { attacker : { user:123 player_level: 31 fleet: { 1:{ Weapons: { name:“Ripper Cannon IV” damage:306 accuracy:55 } armors:{ } … } 2: { }_id" : ObjectId("506e23001d4524122c00004b"), "ip" : "10.54.72.206", "dt" :{ "_id" :ObjectId("506e23001d4524dd2b0000bd"), "ip" : "10.54.72.206", }"&attacker_level=42&defender=0&defender_ext=salvage&defender_level=25&attack_type=PVE&attacking_ }units=%7b%224071%22%3a%5b5%2c3%2c0%5d%7d&defending_units=%7b%221040%22%3a%5b4%2c defender: {0%2c4%5d%2c%221060%22%3a%5b1%2c0%2c1%5d%7d&attacking_shipconfig=%7b%22ships%22%3a }%5b%7b%22armors%22%3a%5b333%2c333%2c333%2c333%5d%2c%22weapons%22%3a%5b134%2c1 attack_time: 10035456334%2c193%2c134%2c134%2c134%2c134%2c134%5d%2c%22specials%22%3a%5b542%2c561%2c571 attack_duration:290%2c501%5d%2c%22hullID%22%3a4071%2c%22tacticalModules%22%3a%5b%5d%2c%22actives%22%3 sector:7a%7b%22hp%22%3a9317.68457%2c%22rank%22%3a4%2c%22fltp%22%3a5%2c%22bid%22%3a54834 …79%2c%22f%22%3a0%2c%22id%22%3a110%2c%22flt%22%3a5%2c%22vxp%22%3a10.858466%7d%7d%2c%7b%22armors%22%3a%5b333%2c333%2c333%2c333.. "stage" : "new" } }
    7. 7. How much data? •Multi TBs / month (4-5x in next 6 months) •Critical : Volume reads with speed •< 1% data has actionable intelligence©2011 Kixeye, Private and Confidential
    8. 8. MongoDB in KIXEYE •Why: – fast read + writes for high volume – Document based architecture, schema less – Indexing , auto failover, auto sharding •Where: – Games , KIXEYE Platform – Analytics, Logging©2011 Kixeye, Private and Confidential
    9. 9. Data technologies at Analytics Technology Data Volume Query Type I/O speed MySQL Small Adhoc + Read fast, only < 300 GB Pre defined Aggregated data Mongo Mid Predefined Read + Write speed < 2 TB fast Hadoop X-Large Adhoc Read slow (Hive+HDFS) 2 TB+ writes: only insert no update©2011 Kixeye, Private and Confidential
    10. 10. An Example Application Cheater Detection engine for games: How mongoDB helped us©2011 Kixeye, Private and Confidential
    11. 11. A simple backyard at start in BYM
    12. 12. Currency is time or $ or both©2011 Kixeye, Private and Confidential
    13. 13. Sample record • 4kb nested jason block • base = { base_name: “sweet_mango_db” resources { 1: { name: "storage silo" level: 5, mode:“built" time:123456902 2: { … ..} ...} ...} ..}©2011 Kixeye, Private and Confidential
    14. 14. How people cheat? • Each upgrade or repair takes pre defined time • if time+payment < predefined time = cheater • victims chain create snowball effect©2011 Kixeye, Private and Confidential
    15. 15. ..and why they cheat...©2011 Kixeye, Private and Confidential
    16. 16. Data volume for cheater detection • 16Mil * 50 * 10 * 3 *.... • multiple time based snapshots • 0.01 % cheaters • Detect < 5 mins • Thats searching through lot of data fast©2011 Kixeye, Private and Confidential
    17. 17. .. and boss told • Catch and auto ban cheaters • Dont label a good guy a cheater©2011 Kixeye, Private and Confidential
    18. 18. ..team was ready for the challenge • Speed+Accuracy of application • Technology evalauation©2011 Kixeye, Private and Confidential
    19. 19. MongoDB fits the requirement • read writes are fast because memory mapped files compared to mysql+memcache or hbase • Unstructured data => need schema less • indexing on a nested data element©2011 Kixeye, Private and Confidential
    20. 20. .. more on why mongo • Auto purging thru TTL • Auto sharding during traffic spikes • Binary data support, fast uncompression • Capped collection: writes faster©2011 Kixeye, Private and Confidential
    21. 21. Development decision points • Old map-reduce (prior to 2.2) vs New Aggregation framework • Global write locks removal in 2.2 • Lazy and Non lazy inserts©2011 Kixeye, Private and Confidential
    22. 22. Result • 100+ cheaters caught per hour • Better game balancing • $$ saved©2011 Kixeye, Private and Confidential
    23. 23. Wrap up : A good partnership •Good partnership with KIXEYE and 10gen •Consulting/Training and MMS service was very helpful •More mongoDB usages in future at KIXEYE©2011 Kixeye, Private and Confidential
    24. 24. {Status: “The End”} •{ next_action: “Q&A” } • Like this jason? meet us outside – KIXEYE recruiting { Company : “KIXEYE”, “Engineers/Total _employee” : “65%”, Culture : [“Fun”, “Innovative”, “Passionate”] Analytics_engg : { technology : “mongo, hadoop , mysql”, tag : “big data, geek” } }©2011 Kixeye, Private and Confidential
    25. 25. Extra slides
    26. 26. Game Play BP Attack Video (https://www.youtube.com/watch?v=qud_-18D2MU)©2011 Kixeye, Private and Confidential
    27. 27. Usage of data• Data mining and aggregation to o Optimize User Acquisition o Optimize game analyzing game play behavior and Algo + Data Optimize r Feedback
    28. 28. Mongo - pain points • Diskspace handling • No sql syntax support • Capped collection - no deletes allowed • No good client GUI based tool©2011 Kixeye, Private and Confidential
    29. 29. Damaged building is useless
    30. 30. Data related to player • building has 5-10 levels • building can have 3 states [built, damaged, repair in progress) • an average base can have 50+ buildings • 16 million such bases • 45k people at any second [i.e base gets updated] • Each level up or repair has time delay. People pay to reduce delay©2011 Kixeye, Private and Confidential
    31. 31. Map-reduce in cheater detection function mapf() { emit(this.userid, {userid:this.userid, total_time:this.length, bldg_id:1} ; } function reducef(key, values) { var r = {userid:key, total_time:0, count:0, avg_time:0}; values.forEach(function(v) {r.total_time += v.total_time; }); return r; } function finalizef(key, value) { ... }©2011 Kixeye, Private and Confidential
    32. 32. The End •Q&A •Love numbers and big data? We have a job for you at analytics engg at KIXEYE. Talk to our recruiting team©2011 Kixeye, Private and Confidential
    33. 33. To scale data handling • Write speed for large volume data • Read speed + volume [aggregate+find] • MongoDB provides good balance compared to other sql+nosql solutions for certain use cases©2011 Kixeye, Private and Confidential
    34. 34. Games+Platforms devices mysql mongo BI Mongo Cluster ETL non Real time >= 1 hr real time 5API mins mySQL Mongo Hadoop mySQL /mongo Admin console Data access tools Dashboards Hive sql query API Scheduler, Analytics Components - Birds Eye View

    ×