Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Game analytics - The challenges of mobile free-to-play games

776 views

Published on

A talk given at the 2015 Dundee Data Day to illustrate the usage of data and analytics in acompany for mobile games using the Freemium model.

Published in: Data & Analytics

Game analytics - The challenges of mobile free-to-play games

  1. 1. GAME ANALYTICS The challenges of mobile Free-To-Play games Christian Beckers Analyst at Outplay Entertainment Christian.beckers@outplay.com Dundee Data Day 5th of December 2015
  2. 2. Spend Hard Levels Fall Off
  3. 3. What we can learn • Levels with high drop off but low spend -> improve balancing • Losing lots of players early -> Rework tutorial to better make them stick • Levels with high spend and low drop off -> the “perfect” levels, have a closer look to determine what people like and what makes them spend …and this is only a very small picture of all the data we collect!
  4. 4. Established 2011 in Dundee Outplay Entertainment is a fast growing, venture-backed Developer and Publisher of social and mobile Games.
  5. 5. The role of a data analyst •Explore! •Explain! •Experiment! Excite!
  6. 6. Tasks of a game analyst • Measure KPI to understand performance of the games • Find and explain issues and potentials in the game from the gathered data • Optimise different factors and settings • Understand the users playing the game • Aid the decision making process • Find in-depth answers for the questions and verify the assumptions of the project team • Present all findings in a compact and understandable way
  7. 7. The challenges of a data analyst • You have to understand the game and the free-to-play model to understand the data! • You have to use databases to gather exactly the data you need! • You have to write scripts and functions to help you clean transform and summarise the data! • You have to deal with millions of data rows! • You have to know which changes are just random occurrences and which are significant developments! • You have to communicate your findings to people unfamiliar with statistics and probability!
  8. 8. No analyst can work without data! and Your analyses are only as good as your data!
  9. 9. Getting data • Every piece of data has a certain cost • Collect as little data as possible while still being able to perform all analyses • Only gather data on a significant sample of all usersMixpanel 3rd party service Postgresql database OLGA R / csv Files Report / Summary Export / Transform Query Analyse / Summarise • API export and data import into database and transformation need loads of maintenance • Analyst working tightly with DevOps team • Game might be buggy or outdated • Need to stay flexible to work around missing data or inconsistency issues Game app Send data • Games can be played offline • Some data will be reported late • We need a certain export delay to allow offline mode data to come through
  10. 10. Understanding data! • The data comes as events and properties • An event is triggered by a certain condition and sends a new data point to the servers • Every event has certain properties providing background information on the event and the user triggering it • In the database, each event corresponds to a row in a table and the properties correspond to the columns Example: I successfully finish level 12 inAlien Creeps on my iPhone and then close my session. The game now sends the two events to the server. After the export to the database we get the new rows: Game event Level_id User_id Platform Other properties AC Level_finished 12 Christian’s Id iOS e.g. Outcome = win AC Session_closed Christian’s Id iOS e.g. Duration = 120 s
  11. 11. Understanding data! Challenges are: • Multiple games, all with slightly different analytics.Templates used to prevent inconsistencies • Good knowledge of the game. Understanding how an event is fired and what it means in the context of the game • Players on different versions. Need to be aware of major difference and adapt analyses accordingly • Select data on the right users, especially if sampling was introduced
  12. 12. Cleaning and transforming data We need to tidy up all of our data to be able to use it! Reasons: • Hackers and company QA users skew the data and have to be identified and removed. • Some data is incomplete due to communication errors or bugs and has to be completed e.g. by extrapolating other events of the user • You might need data from different sources and combine the information you get through different APIs
  13. 13. ANALYSING DATA! Time to get our hands dirty!
  14. 14. Types of Analyses •Exploratory analysis •Investigative analysis •Experiment analysis
  15. 15. Exploratory Analysis • Dig into the data to find new correlations, causations or segmentations • Look at the data in many different ways, e.g. plot lots of different factors against each other and look for potential relations between variables • Use machine learning techniques to find relations the eye cannot see by itself Challenges: • Restrict to certain aspects of the game (e.g. social interactions) to keep the amount of data manageable • Correlation does not equal causation!
  16. 16. InvestigativeAnalysis • Starts with a questions, e.g. “Why do so many users quit after level x?” • Select data specifically to answer the question • Look into every aspect of the game connected to the questions (e.g. also consider in-game messaging, tutorial steps, boosts used) Challenges: • The question might be too wide or too specific • Collecting data to only answer the question might remove the big picture (e.g. the drop off is caused by the levels before level x as well) • Correlation does not equal causation!
  17. 17. Experiment Analysis Multivariate Experiment • Look at multiple factors with a distinct high and low setting • Groups implement a combination of high/low settings for the factors • Determine the impact each factor has on the KPI while keeping control of interference effects • Use experimental design to reduce the number of groups needed (otherwise 2^Number of factors groups needed) A/B test
  18. 18. Multivariate experiment
  19. 19. Multivariate Experiment Controlgroup Retention Day 7 / Day 14 Revenue Day 7 / Day 14
  20. 20. Experiment Analysis Multivariate Experiment • Look at multiple factors with a distinct high and low setting • Groups implement a combination of high/low settings for the factors • Determine the impact each factor has on the KPI while keeping control of interference effects • Use experimental design to reduce the number of groups needed (otherwise 2^Number of factors groups needed) A/B test • Look at a number of test groups (usually 1-3) and compare them against the control • The groups all change the same factor or factors to avoid unforeseen interference ruining the results • When the result is a very significant change, follow up tests are used to further optimised the tested factors • Make sure the observed differences are statistically significant!
  21. 21. HOWTO: A GAME ANALYSIS Methods and tools used for analyses
  22. 22. Segmentation We often segment the users we look at, some examples: • Country • Purchasers/Non-Purchasers • Platform/Store • Social activity • Date of first launch First launches Purchases
  23. 23. Lifestories –The backbone of any analysis All events selected of a user or cohort of users with selected properties This allows us to do all analyses! user_id event_name timestamp store session_counter current_level outcome furthest_level cumulative_game_time_seconds attempts continues_used boosts_used aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 first_launch 30/11/2015 21:36 Google Play 0 1 1 0 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 adjust_attribution 30/11/2015 21:36 Google Play 0 1 1 1 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:38 Google Play 0 1won 2 65 1 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:39 Google Play 0 2won 3 142 1 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:40 Google Play 0 3won 4 221 1 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:41 Google Play 0 4won 5 287 1 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:52 Google Play 0 5won 6 403 1 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:55 Google Play 0 6won 7 522 1 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:58 Google Play 0 7won 8 619 1 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:59 Google Play 0 8won 9 710 1 0 1 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:26 Google Play 1 10won 10 1075 1 0 1 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:27 Google Play 1 11won 12 1186 1 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:30 Google Play 1 12won 12 1312 1 0 2 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:31 Google Play 1 13won 14 1423 1 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:35 Google Play 1 14retried 14 1650 1 3 3 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:37 Google Play 1 14retried 14 1752 2 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:39 Google Play 1 14retried 14 1848 3 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:40 Google Play 1 14retried 14 1926 4 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 iap_confirmed 30/11/2015 23:42 Google Play 1 14 14 2025 0 0 aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:45 Google Play 1 14won 15 2216 5 0 2
  24. 24. R & R Studio We use the scripting language R to transform, visualise and analyse our data. R Studio is a free GUI for R with many helpful features like easily accessible help and plot windows and an environment overview.
  25. 25. R & R Studio R can be expanded easily and additional packages allow for quicker data processing and better visualisation. In R, you can make all sorts of plots very quickly and R Studio allows to quickly go through them.
  26. 26. Sankey diagram R even allows more complex visualisation like Sankey diagrams for flow explorations.
  27. 27. Social graph Using R to transform data and then exporting into Gephi allows for nice and quick network visualisations and analyses.
  28. 28. Priority is Player Satisfaction and Fun “If your game sucks, data won’t save you!” Data Analyst at Outplay
  29. 29. Thanks for listening! Any Questions?
  30. 30. • Outplay.com • Outplay.com/careers • Christian.Beckers@outplay.com Some interesting materials: • ‘Freemium Economics’, Eric B. Seufert • https://cran.r-project.org/ • https://www.rstudio.com/ • http://www.cookbook-r.com/ • https://gephi.org/

×