Presented by Dr Dan Bergin-Holly at the 18th Games Industry Analytics Forum in London on 10th May 2018.

2. Analytics - How and why youare
making a mess of it and ruining
your analysts’ lives

3. Who the F**k areyou?
I’m Dr. Dan!
● Product Lead on Angry Birds Transformers for Exient Ltd
● Background inPerformance Technology and live theatre
● Previously Product Manager at SEGA,Producer atSixminute
● PhD in something vaguely interesting
● Married - so you know at least one person likes me

4. People
Process Tools
With Data from RealAnalysts!

5. People

6. People
You Hired the Wrong People!
You hired a data scientist when what you really wanted was a monetisation designer or game economy
expert
You hired a data scientist who ended up being a monetisation designer because your current designers
are lazy or scared of math (you hired the wrong designers)
You hired a data scientist with a big shiny degree in machine learning and big data operations so that they
could export SQLqueries into excel and make the occasional chart - without telling them (possibly
because you didn’t know)
You hired someone right outta university - and didn’t take the time to guide them through the difficult
transition towards everything being broken and horrible here in the real world.

7. People
You keep your analysts in a separate building - away from the design and
programming team!
Your designers begin to resent all the suggestions coming from ‘the numbers people’ who have no idea
what it’s like to be in the trenches, at the coalface, really working on a game.
Your analysts wonder why your designers keep iterating on the different boosters for the archery
character when none of the players actually own an archer in game
Cross-functional teams are da bomb! Put your analysts where the dev team is - after all, they’re all working
on the game right? - This includes not fobbing off all your data work to your publisher!

8. People
You are blind to theanalyst’s suggestions!
A lot of data is telling you that the FTUE funnel is broken
and that the game doesn’t run on lower spec android
phones
Your analyst has repeatedly shown you that the biggest
win would be to make the game performant on a certain
subset of devices
Your 6 year old niece who plays the game keeps saying
the bunny rabbits should beblue
You prioritise colour iterations on rabbits for the next 3
sprints

9. Tools

10. Tools
You build a shonky homebrew analytics system that never works
Your programmers will hate you and your analyst
Your analyst will hate you and your programmers
Your investors will hate you and your bad data
Your bank manager will be indifferent to your suffering

11. Tools
When is it OK to build your own Analytics Solution?
You are Daddy Warbucks.
You can dedicate significant AND
CONTINUOUS resources to the
development and maintenance of a high
quality piece of business critical software.

12. Tools
When is it OK to build your own Analytics Solution?
You have decided to quit your job and are
now making an analytics company.
You can dedicate significant AND
CONTINUOUS resources to the
development and maintenance of a high
quality piece of business critical software.

13. Tools
Right at the end of development (or even after launch) you suddenly thought ‘oh no we better get some
data’ and rush to get some analytics into the game.
It doesn’t get goodtesting.
Your analyst doesn’t get time to verify that the data will be sufficient to answer your key initial questions.
You can’t answer any key questions - and lose precious time.
Remember!
Telemetry failures can be incredibly difficult to detect and fix, and often only rear their head once the game
is ‘in the wild’ - reduce risk by implementing early!
You put your analytics inlate

14. Tools
That’s right. For one reason or another you made a mess
of it.
The data is clearly not firing correctly, and pretty much
every day that goes by not only are you adding useless
data to a database, you may even be damaging your
existing data set and making it worthless.
One day, your analyst presents you with a data table that
includes an entry being populated by the words ‘TO DO’ -
and a bill for all the additional server space
You put your analytics inwrong

15. Tools
After two months of other fixes you finally get around to fixing your data. On your next release, something
drastically changes in the game - you struggle to find out how or why, because all your baseline data from
the last two months is complete trash.
You took too long to fix your analytics

16. Process

17. Process
You ignore your analyst and product team’s pleas for some measure of sanity and prioritization in
favour of making an apple watch/appleTV/ARKit/Google Chromecast version of your game
because you know a guy, who knows a guy, who said they totally got store featuring when they did
it.
Seriously though - this annoys everyone in your team. Don’t do it.

18. Process
This is a direct result of some of the previous issues I’ve mentioned (or your team experiencing those
issues in other studios and bringing that baggage with them)
This can lead to glaring problems in your game going unfixed for far too long - for example, nobody
collecting their rewards in your game, and thus rendering your entire progression design totally redundant.
Where does this comefrom?
● Unreliable tools
● Rarely checking results of changes
● Management’s attitude
Nobody Trusts Your Data

19. Process
You still need to make your best educated
guess in absence of any data to back you up.
The fact that you have data is not an excuse for
habitual fence sitting.
You can’t fob everything off to another A/B
test.
Remember!
At the end of the day someone needs to be the
shot-caller on a project - and that means
actually calling the shots.
You stopped taking responsibility for making tough calls because data exists

20. Process
You blame analysts for thedata!
An A/B test showed promise so you rolled out a big change to the game and it's all gone wrong. When
discussing it, you keep coming back to ‘But the A/B Test said…..’
Remember!
A/B Tests aren’t always the be all and end all of a feature. Funnel progression can’t tell you everything
about your tutorial. Remember when using telemetry data about your game, it's only one piece of the
puzzle. It’s information that can (and should) be supplemented.

21. Process
You ask big questions then ignore the answers
You have an idea for afeature.
You ask the analysts to answer “some vital question”™ to prove you’re right.
After badgering your analytics team into spending hours on “some vital question”™ they deliver data that
strongly suggests you are absolutelywrong.
You promptly ignore the answers they gave you and do what you were going to do anyway.
Your analysts feel undervalued - and begin talking to recruiters….

22. Summary

23. Summary
Know what you need from a potential hire.
Be sound.
Don’t roll your own.
Test early andoften.
Listen to the data - but don’t treat it as an oracle.

24. Thanks!
Questions? Comments? Rotten Fruit?
Mail: dan@danyoutohell.com
Tweet: @danyoutohell
Address by name tomy face: Dan