Implementing Analytics in High-Traffic Social Games

2,287 views
2,160 views

Published on

Setting up a fail-proof analytics process that delivers information where and when they are needed comes with plenty of challenges in high traffic social gaming.
Social Point CTO, Marc Canaleta, explains in this Slideshare how Social Point has found a solution that is economic and highly scalable, with high availability.



Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,287
On SlideShare
0
From Embeds
0
Number of Embeds
1,299
Actions
Shares
0
Downloads
11
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Implementing Analytics in High-Traffic Social Games

  1. 1. Social Point Analytics in AWS Marc Canaleta (CTO) @mcanaleta AWS Summit Barcelona 2013
  2. 2. Social Games developer for Mobile & Facebook Founded in 2008, offices in Barcelona(22@) and San Francisco Top #20 mobile grossing games worldwide Top #3 Facebook developer
  3. 3.  Social Games: interaction between friends; virality  Freemium model: Play for free with in-app purchases  Midcore  Leader in Breeding & Collecting strategy games
  4. 4.  Top 20 Grossing in iOS App Store worldwide  Recently launched for Android, featured on Google Play  6M DAU on Facebook
  5. 5.  No hardware maintenance or planning: business benefits from increased speed  Flexible: Pay for use  Facilitates scalability: Auto Scaling  Facilitates high availability: multiple availability zones  Managed components: Load Balancers, Databases …
  6. 6. Analytics Driven. Vital for almost every team.  Engineers: realtime analytics, monitoring, detecting problems  Product: taking decisions, A/B testing, game balancing  Marketing: optimizing campaigns  Finance: monitoring the business
  7. 7. FLASH CLIENT IOS CLIENT ANDROID CLIENT BACKEND SERVERS BACKEND SERVERS BACKEND SERVERS Symfony 2 ANALYTICS QUEUES ANALYTICS QUEUES ANALYTICS QUEUES Redis LOGFILES STORAGE ANALYTICS DATABASE AWS S3 AWS Redshift
  8. 8.  Backend writes events in Redis lists  Why Redis?  Cost and Performance: 10K events/second/server  Problem: it’s a memory-based database; queues have to be constantly consumed  Scaled and HA: randomly distributed N servers BACKEND REDIS REDIS REDIS
  9. 9.  Python processes continuously consume queues and:  Calculate Real Time metrics  Store event logfiles to upload to S3 GENERATION Of EVENTS Redis Queue LPOP event Consumer Redis Real Time write event Event Log File  Enqueue S3 object URL to SQS INCR counter put object Amazon S3 LOAD DATA Amazon SQS enqueue S3 object URL
  10. 10. Why these technologies? GENERATION OF EVENTS  Python is well suited to developing workers and dealing with data Redis Queue  Redis: structures like counters, sets, sorted sets for Real Time metrics Consumer LPOP event put object Amazon S3  SQS reliability and availability at a higher cost than Redis Redis Real Time write event Event Log File  S3: virtually infinite space, scalable, high availability INCR counter LOAD DATA Amazon SQS enqueue S3 object URL
  11. 11. EVENT PROCESSING A process called importer: Amazon S3 Amazon SQS  Reads SQS URLs  Downloads S3 logfiles Importer  It converts them to TSV TSV  And imports multiple logfiles to Redshift at the same time RedShift
  12. 12.  Allows flexibility-> schema changes with no downtime  Highly scalable (but you cannot write while scaling)  Low implementation risk  It is an offline system. Gameplay is unaffected by downtime.  We have backups. In the worst case, we restore a backup and reload.  Minimum maintenance: only vacuums, space monitoring  Good SQL support, unlike other columnar data bases
  13. 13. Daily transformations and calculations implemented in SQL Example: UPDATE USER SET total_revenues = (SELECT SUM(amount) FROM transaction t WHERE t.user_id = user.user_id); Why not hadoop?  Much slower and complex; for now SQL operations meet all of our needs. In Redshift these SQL operations are very fast.
  14. 14. ¿Would you like to work in the video gaming sector? Talent attracts talent. You have the talent, we have the playground. www.socialpoint.es/jobs ¡Thank you! 

×