Game analytics - The challenges of mobile free-to-play games

GAME ANALYTICS
The challenges of mobile Free-To-Play games
Christian Beckers
Analyst at Outplay Entertainment
Christian.beckers@outplay.com
Dundee Data Day
5th of December 2015

What we can learn
• Levels with high drop off but low
spend -> improve balancing
• Losing lots of players early -> Rework
tutorial to better make them stick
• Levels with high spend and low drop
off -> the “perfect” levels, have a
closer look to determine what people
like and what makes them spend
…and this is only a very small picture
of all the data we collect!

Established 2011 in Dundee
Outplay Entertainment is a
fast growing, venture-backed
Developer and Publisher of
social and mobile Games.

The role of a data analyst
•Explore!
•Explain!
•Experiment!
Excite!

Tasks of a game analyst
• Measure KPI to understand performance of the games
• Find and explain issues and potentials in the game from the gathered data
• Optimise different factors and settings
• Understand the users playing the game
• Aid the decision making process
• Find in-depth answers for the questions and verify the assumptions of the project
team
• Present all findings in a compact and understandable way

The challenges of a data analyst
• You have to understand the game and the free-to-play model to understand the
data!
• You have to use databases to gather exactly the data you need!
• You have to write scripts and functions to help you clean transform and summarise
the data!
• You have to deal with millions of data rows!
• You have to know which changes are just random occurrences and which are
significant developments!
• You have to communicate your findings to people unfamiliar with statistics and
probability!

No analyst can work without data!
and
Your analyses are only as good as
your data!

Getting data
• Every piece of data has a certain cost
• Collect as little data as possible while still
being able to perform all analyses
• Only gather data on a significant sample of
all usersMixpanel 3rd party service
Postgresql database OLGA
R / csv Files
Report / Summary
Export /
Transform
Query
Analyse /
Summarise
• API export and data import into database
and transformation need loads of
maintenance
• Analyst working tightly with DevOps team
• Game might be buggy or outdated
• Need to stay flexible to work around
missing data or inconsistency issues
Game app
Send data
• Games can be played offline
• Some data will be reported late
• We need a certain export delay to allow
offline mode data to come through

Understanding data!
• The data comes as events and properties
• An event is triggered by a certain
condition and sends a new data point to
the servers
• Every event has certain properties
providing background information on the
event and the user triggering it
• In the database, each event corresponds
to a row in a table and the properties
correspond to the columns
Example:
I successfully finish level 12 inAlien Creeps
on my iPhone and then close my session.
The game now sends the two events to the
server.
After the export to the database we get
the new rows:
Game event Level_id User_id Platform Other properties
AC Level_finished 12 Christian’s Id iOS e.g. Outcome = win
AC Session_closed Christian’s Id iOS e.g. Duration = 120 s

Understanding data!
Challenges are:
• Multiple games, all with slightly different analytics.Templates used to prevent
inconsistencies
• Good knowledge of the game. Understanding how an event is fired and what it
means in the context of the game
• Players on different versions. Need to be aware of major difference and adapt
analyses accordingly
• Select data on the right users, especially if sampling was introduced

Cleaning and transforming data
We need to tidy up all of our data to be able to use it!
Reasons:
• Hackers and company QA users skew the data and have to be identified and
removed.
• Some data is incomplete due to communication errors or bugs and has to be
completed e.g. by extrapolating other events of the user
• You might need data from different sources and combine the information you get
through different APIs

ANALYSING DATA!
Time to get our hands dirty!

Types of Analyses
•Exploratory analysis
•Investigative analysis
•Experiment analysis

Exploratory Analysis
• Dig into the data to find new correlations, causations or segmentations
• Look at the data in many different ways, e.g. plot lots of different factors against
each other and look for potential relations between variables
• Use machine learning techniques to find relations the eye cannot see by itself
Challenges:
• Restrict to certain aspects of the game (e.g. social interactions) to keep the
amount of data manageable
• Correlation does not equal causation!

InvestigativeAnalysis
• Starts with a questions, e.g. “Why do so many users quit after level x?”
• Select data specifically to answer the question
• Look into every aspect of the game connected to the questions (e.g. also consider
in-game messaging, tutorial steps, boosts used)
Challenges:
• The question might be too wide or too specific
• Collecting data to only answer the question might remove the big picture (e.g. the
drop off is caused by the levels before level x as well)
• Correlation does not equal causation!

Experiment Analysis
Multivariate Experiment
• Look at multiple factors with a distinct
high and low setting
• Groups implement a combination of
high/low settings for the factors
• Determine the impact each factor has on
the KPI while keeping control of
interference effects
• Use experimental design to reduce the
number of groups needed (otherwise
2^Number of factors groups needed)
A/B test

Controlgroup
Retention Day 7 / Day 14
Revenue Day 7 / Day 14

Experiment Analysis
• Look at multiple factors with a distinct
high and low setting
• Groups implement a combination of
high/low settings for the factors
• Determine the impact each factor has on
the KPI while keeping control of
interference effects
• Use experimental design to reduce the
number of groups needed (otherwise
2^Number of factors groups needed)
A/B test
• Look at a number of test groups (usually
1-3) and compare them against the
control
• The groups all change the same factor or
factors to avoid unforeseen interference
ruining the results
• When the result is a very significant
change, follow up tests are used to
further optimised the tested factors
• Make sure the observed differences are
statistically significant!

HOWTO:
A GAME ANALYSIS
Methods and tools used for analyses

Segmentation
We often segment the users we look at,
some examples:
• Country
• Purchasers/Non-Purchasers
• Platform/Store
• Social activity
• Date of first launch
First launches
Purchases

Lifestories –The backbone of any analysis
All events selected of a user or cohort of users with selected properties
This allows us to do all analyses!
user_id event_name timestamp store session_counter current_level outcome furthest_level cumulative_game_time_seconds attempts continues_used boosts_used
aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 first_launch 30/11/2015 21:36 Google Play 0 1 1 0 0 0
aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 adjust_attribution 30/11/2015 21:36 Google Play 0 1 1 1 0 0
aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 21:38 Google Play 0 1won 2 65 1 0 0
aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 level_account 30/11/2015 23:35 Google Play 1 14retried 14 1650 1 3 3
aa3626d2-0a7c-48f7-8d7c-c2481c72ca20 iap_confirmed 30/11/2015 23:42 Google Play 1 14 14 2025 0 0

R & R Studio
We use the scripting language R
to transform, visualise and
analyse our data.
R Studio is a free GUI for R with
many helpful features like easily
accessible help and plot
windows and an environment
overview.

R & R Studio
R can be expanded easily and
additional packages allow for
quicker data processing and
better visualisation.
In R, you can make all sorts of
plots very quickly and R Studio
allows to quickly go through
them.

Sankey diagram
R even allows more
complex visualisation like
Sankey diagrams for flow
explorations.

Social graph
Using R to transform data and then
exporting into Gephi allows for nice
and quick network visualisations and
analyses.

Priority is Player Satisfaction and Fun
“If your game sucks,
data won’t save you!”
Data Analyst
at Outplay

Thanks for listening!
Any Questions?

• Outplay.com
• Outplay.com/careers
• Christian.Beckers@outplay.com
Some interesting materials:
• ‘Freemium Economics’, Eric B. Seufert
• https://cran.r-project.org/
• https://www.rstudio.com/
• http://www.cookbook-r.com/
• https://gephi.org/

Game analytics - The challenges of mobile free-to-play games

Recommended

Recommended

More Related Content

What's hot

What's hot (14)

Viewers also liked

Viewers also liked (18)

Similar to Game analytics - The challenges of mobile free-to-play games

Similar to Game analytics - The challenges of mobile free-to-play games (20)

Recently uploaded

Recently uploaded (20)

Game analytics - The challenges of mobile free-to-play games