What can we learn from
game analytics?
Hello
●

Osma Ahvenlampi, founder, Metrify.io
○
○

●

Formerly CTO of Sulake: Habbo Hotel
Analytics & monetization expert, advisor, consultant

Metrify does Operational Data Science
○

extracting continuous, automated value from business data
Analytics changed games forever
●

Games used to be almost completely unmonitored and analyzed once
released to the market
○

●

That, is, analysis done on them was desktop reverse engineering

Today, they’re among the most comprehensively analyzed products
○
○

●

Because they can be: fully digital, open platforms, online play
Because they have to be: free-to-play kills inefficient products

Play data shapes games through their lifetime
The four key metrics of free products
●

Acquisition: where, how & at what cost can new users be found

●

Retention: how many stay over a period of time

●

Engagement: how much time do people consume

●

Conversion: how often does all of the above lead to revenue

Without Engagement, this is referred to as the “ARC” metric
Retention beats Conversion
●

Every free product depends on repeat purchase

●

Nobody buys on the first engagement

●

High long-term retention provides more opportunity to convert

●

Optimizing near-term conversion has proved to be less effective
Why repeat purchase matters
One-time purchase Repeat purchases
Users
Free to paying conversion rate
Single purchase value
Monthly repeat customers
Six-month sales
Revenue increase

100 000    

100 000    

5%

5%

2.00 €

2.00 €

0%

10 %

10 000 €

15 000 €

-

50 %
It’s really hard to predict retention
●

Except: an engaged user is more likely to return

●

How many return one day after

●

What’s happening when people return

●

1-7-30 day retention curve

●

Typically, 30 days is enough to form a habit.
Are the next 30 days similar to the first 30?
Re-investing for growth
●

Design for repeat purchase

●

Optimize for high engagement and retention

●

Learn to recognize who will engage and retain

●

Re-invest revenues to acquire more people likely to engage
○

Paid user acquisition

○

Viral spread, eg sharing

○

Community development

○

Further product development
Do not measure averages
●

Practically all human behavior is biased towards extremes
○

Standard normal distribution applies well to physical measures, not behavior

●

This is the same power law curve as in the Long Tail

●

Average is driven by the outliers, but doesn’t represent them
○

What’s the behavior of the highest and lowest 25%, ie, Interquartile range
Retention is not the same as Churn
●

Churn = the % of users lost over a period, on average

●

Retention = the % of people of a certain cohort age who stay active

Not unreasonable to expect that Retention = 1 - Churn. Why is this wrong?
An active user is more likely to stay active than the average!
Churn vs retention, visualized
What should I measure?
●

Everything. Oh, is that not helpful?

●

Specific events during the experience

●

Frequency and periodicity of repeat events

●

As wide a set of different events as is feasible to gather

●

Clicks and other UI use is rarely meaningful, outside of UI optimization

●

What is the product meant to do?
How should I measure?
●

Event streams are semi-structured log files

●

Time, identifier, event, event-specific data, context data

●

Aim for dozens, if not hundreds of events per visit
○

●

Expect to combine multiple sources of data and build context
○

●

“Big” data: 20 MB per 1000 users per day

“Complex” data: event and source type specific processing logic

Timely feedback loops need near-realtime processes
○

Streaming data infrastructures
Okay, I’ve measured. What now?
●

Dashboards are Step 0. “What’s happening?”

●

Ability to drill down: “Who, where, why is that happening?”

●

Act on findings
●

Customer contact

●

Product changes

●

Feedback loops: “Did anything change?”

●

Testing: A/B, multivariate, pilot groups

●

Segmented and personalized experience
Recap
●

Data is essential in managing complex products

●

Understand key principles. Avoid averages.

●

You’re in the driver seat. Even real-time data is mostly a backwards mirror.

●

Use data to validate assumptions, confirm results, (dis)prove hypotheses

●

Data does not replace a product vision or design intent

●

Data Science is a specialist skill
Thank you!
Osma Ahvenlampi
osma@metrify.io
www.metrify.io
twitter.com/metrify

What can media learn from game analytics

  • 1.
    What can welearn from game analytics?
  • 2.
    Hello ● Osma Ahvenlampi, founder,Metrify.io ○ ○ ● Formerly CTO of Sulake: Habbo Hotel Analytics & monetization expert, advisor, consultant Metrify does Operational Data Science ○ extracting continuous, automated value from business data
  • 3.
    Analytics changed gamesforever ● Games used to be almost completely unmonitored and analyzed once released to the market ○ ● That, is, analysis done on them was desktop reverse engineering Today, they’re among the most comprehensively analyzed products ○ ○ ● Because they can be: fully digital, open platforms, online play Because they have to be: free-to-play kills inefficient products Play data shapes games through their lifetime
  • 4.
    The four keymetrics of free products ● Acquisition: where, how & at what cost can new users be found ● Retention: how many stay over a period of time ● Engagement: how much time do people consume ● Conversion: how often does all of the above lead to revenue Without Engagement, this is referred to as the “ARC” metric
  • 5.
    Retention beats Conversion ● Everyfree product depends on repeat purchase ● Nobody buys on the first engagement ● High long-term retention provides more opportunity to convert ● Optimizing near-term conversion has proved to be less effective
  • 6.
    Why repeat purchasematters One-time purchase Repeat purchases Users Free to paying conversion rate Single purchase value Monthly repeat customers Six-month sales Revenue increase 100 000     100 000     5% 5% 2.00 € 2.00 € 0% 10 % 10 000 € 15 000 € - 50 %
  • 7.
    It’s really hardto predict retention ● Except: an engaged user is more likely to return ● How many return one day after ● What’s happening when people return ● 1-7-30 day retention curve ● Typically, 30 days is enough to form a habit. Are the next 30 days similar to the first 30?
  • 8.
    Re-investing for growth ● Designfor repeat purchase ● Optimize for high engagement and retention ● Learn to recognize who will engage and retain ● Re-invest revenues to acquire more people likely to engage ○ Paid user acquisition ○ Viral spread, eg sharing ○ Community development ○ Further product development
  • 9.
    Do not measureaverages ● Practically all human behavior is biased towards extremes ○ Standard normal distribution applies well to physical measures, not behavior ● This is the same power law curve as in the Long Tail ● Average is driven by the outliers, but doesn’t represent them ○ What’s the behavior of the highest and lowest 25%, ie, Interquartile range
  • 10.
    Retention is notthe same as Churn ● Churn = the % of users lost over a period, on average ● Retention = the % of people of a certain cohort age who stay active Not unreasonable to expect that Retention = 1 - Churn. Why is this wrong? An active user is more likely to stay active than the average!
  • 11.
  • 12.
    What should Imeasure? ● Everything. Oh, is that not helpful? ● Specific events during the experience ● Frequency and periodicity of repeat events ● As wide a set of different events as is feasible to gather ● Clicks and other UI use is rarely meaningful, outside of UI optimization ● What is the product meant to do?
  • 13.
    How should Imeasure? ● Event streams are semi-structured log files ● Time, identifier, event, event-specific data, context data ● Aim for dozens, if not hundreds of events per visit ○ ● Expect to combine multiple sources of data and build context ○ ● “Big” data: 20 MB per 1000 users per day “Complex” data: event and source type specific processing logic Timely feedback loops need near-realtime processes ○ Streaming data infrastructures
  • 14.
    Okay, I’ve measured.What now? ● Dashboards are Step 0. “What’s happening?” ● Ability to drill down: “Who, where, why is that happening?” ● Act on findings ● Customer contact ● Product changes ● Feedback loops: “Did anything change?” ● Testing: A/B, multivariate, pilot groups ● Segmented and personalized experience
  • 15.
    Recap ● Data is essentialin managing complex products ● Understand key principles. Avoid averages. ● You’re in the driver seat. Even real-time data is mostly a backwards mirror. ● Use data to validate assumptions, confirm results, (dis)prove hypotheses ● Data does not replace a product vision or design intent ● Data Science is a specialist skill
  • 16.