Topic: What is the role of analytics in running live games? Game studios generally know that they need to instrument and use analytics, but often don't know how to use the data they are presented with to improve their games. Allison presents a "playbook" with detailed instructions on how to analyze the health of a game, what data to be focusing on to get insights and direction, and a framework by which everyone can be involved in the analysis process. This presentation pulls on Allison's experience building the game analytics function at PopCap Games, as well as bring insights from other leaders in the game industry and analytics space. [These were the supporting slides for Allison's talk at Nordic Game 2014]. Allison Bilas Bio: Hailing from Seattle, Washington, Allison earned her MBA at the University of Washington and then joined mobile games giant PopCap in 2011, where she built the game analytics team from scratch and assisted on hit titles Bejeweled Blitz and Plants vs. Zombies 2, amongst others. Now with several years of high-level user behavior analysis under her belt, Bilas joined the GameAnalytics team as VP of Product to manage the product design team and align efforts in making the best game analytics tool for game developers of all sizes.

2. Your Game Analytics
Playbook
Allison Bilas
@allisonbilas / allison@gameanalytics.com
1. Introduction
1. Currently VP of Product @ GameAnalytics
2. Just moved to Denmark, loving hygge.
2. PopCap for last 3.5 years.
1. Hired when they didn’t know what to do with data.
2. Built game analytics team from scratch.
3. Learned a LOT about how to infuse data into the decision making process behind developing and operating games.
4. Hope to share some of my experience with you.

3. Agenda
‣ What is a game analytics playbook?
‣ Why is it important?
‣ Components of a “play”
‣ Examples: pre-season to end season
This presentation is about:
!
1. What a game analytics playbook is
2. Why it’s important to have one
3. I’ll go into the component of a game analytics “play”
4. And I’ll give you some concrete examples from the pre-season to the end season
!
Let’s get started…

4. What is a game analytics
playbook?
1. A playbook comes from American football.
1. I’m not actually that into the sport.
2. But it’s a powerful tool.
2. I want you to imagine a football team on the field. The QB calls his team together.
1. Competitors position
2. Health of his team
3. Time left on the clock
2. Makes a call for a specific play > team takes action > success/fail
3. If the game isn’t over, they start again.
!
!
5. What allows the QB to make that decision is their playbook.
1. Put together at the beginning of season.
2. Anticipates every situation the team is likely to encounter.
!
A game analytics playbook is your plan for using data to inform the development and operations of your game.
!
We had one at PopCap that was explicit, but it doesn’t have to be.

5. Information resolves
uncertainty.
1. So, why is this important?
2. Assuming you’re here because you understand the power of data.
1. Playbook can be used to inform AEM and live ops.
2. In short, it helps to resolve uncertainty and speed up your decision making and development.
3. Caution that analytics doesn’t solve everything. Game still needs to be a great experience for your players.

6. Components of a Play
Context Question
1. Let’s get into the components of a “play”.
1. This is essentially the analysis process.
2. Most studios start with thinking about what to track, but there are a couple of steps that should happen before that…
2. Like the QB on the football field, first step is CONTEXT.
1. Where your game is at in its lifecycle. (We’ll go over these in the next sections.)
2. Competitive context and industry trends
3. Goals for your game in terms of KPIs
3. By assessing the context, you’ll start to come up with analysis QUESTIONS.
1. Typically focused on AEM, but also can be more technical in nature around things like bugs and frame rates.
2. Likely to have a LOT of questions.
3. Prioritize based on: measureable / actionable / most impact.

7. Components of a Play
Context Question Data / Tools Action
1. DATA AND ANALYSIS TOOLS
1. Here is where you need to start thinking about what to track in order to answer the questions you have…
2. In deciding what to track you should also consider the type of analysis you’re going to do, for example:
1. Segmentation, benchmarking, A/B testing and trend analysis.
2. Last component is ACTION. Here’s is where the playbook is most powerful.
1. If you’ve done everything right: CONTEXT > QUESTION > DATA/TOOLS
2. Deciding what action should be very easy. Typically implied in the questions you’re asking.

8. Pre-season
(Dev)
Early Season
(Beta)
Mid Season
(Launch)
Late Season
(Ongoing Ops)
Seasons
1. Next I’m going to give examples that follow the life cycle of your game from:
1. Pre-season or development
2. Early season or beta period
3. It’s not until the mid season that you’ve finally launched
4. And then the late season, or ongoing ops

9. Pre-Season (Development)
‣ Instrument tracking EARLY and TEST
‣ Be judicious in what you track
‣ Think through event hierarchies
‣ Consider multiple tracking systems
1. Pre-season is straight forward - SET UP TRACKING.
2. I recommend you start doing this very early.
1. This means you’ll track things initially that you don’t need in the end. This is okay.
2. It lets you start to refine the questions you have and ensure that you have the data needed to answer them.
3. Also gives you time to test. Every game I’ve every launched has found tracking bugs in the first day after launch.
3. Let’s you be smart about what you track. E.g. Game I was working with wanted to track every time a player turn on/off the sound.
4. Also in the early season, you should be thinking very closely about your event hierarchies.
1. These can dictate the type of analysis you can do, and it’s very nuanced.
2. It helps to start tracking and trying out analysis, and then making changes.
5. Multiple tracking systems — can add some complexity, but allows validation of data.

10. Early Season (Beta)
1. Early season is BETA period.
1. Select markets, and you have data! YAY!
2. Exciting period. Easy to loose focus.
2. Most important = look for unexpected behaviors in gameplay and monetization.
3. HIDDEN AGENDA
1. Hidden object game launched on FB a few years ago. Few hundred Australians playing it every day.
2. One player was spending a lot of money in the game. WHALE!
3. What was she spending money on > gotcha boxes of consumables.
4. How was she progressing in the game > noticed she was stuck on Level 18.
5. Needed a “finger print duster” to move forward. But that item wasn’t available until Level 20. BLESS HER!

11. Early Season (Beta)
1. Because it was the beta period, this wasn’t terrible. We were able to identify the weird behavior, find the problem and fix it.
2. In terms of analysis in the beta period:
1. Segmentation is very important. In addition to the few hundred Australians, we had thousands of Philipine installs coming and going in the
game in order to pressure test.
2. Important to filter and look at them separately.
3. I also want to mention that A/B testing isn’t necessary. Lots of info just by observing.

12. Mid Season (Launch)
1. Finally at mid-season the game launches! This is when things start moving really quickly.
2. Analysis should focus on high level metrics > dive deeper when something isn’t working.
3. One example from my experience at PopCap was when I was working with PVZ2.
1. Immediately after launch, millions of installs. Playing for a few days and then leaving. Retention was horrible. Everything else was good!
2. Digging in to engagement… players spending less than 5 minutes per session in the game. BUT, time to complete a level took 15 to 20
minutes.
3. Players had to return 3 or 4 times to the game before they had a reward moment. Felt like a slog.
4. Team identified this and set about finding ways to shorten the game lengths.

13. Mid Season (Launch)
1. Quite honestly, too late! Millions came and left and it was hard to get them to come back.
2. In launch analysis, you’ll be looking at data all the time. HOURLY, DAILY.
3. Intense period lasts probably 1 month… THEN…

14. Late Season (Ongoing Ops)
1. At some point you need to slow down and go deeper with your analysis. This is your end game, but it should never actually end!
1. Focus on refinement and longer term game design.
2. Robust content pipeline and promotional program.
2. Start looking at trends in the data.
3. Bejeweled Blitz Rare Gems
1. Super interesting game. FB for 5yrs, mobile for 3yrs.
2. Success because of content and promotions.
3. Main content is Rare Gems, extra powerful boosts.
4. Had to launch a few of them look at trends in order to understand what made them successful.
1. First one was a big hit, just because it was first.
2. 2nd and 3rd were also good…
3. Wasn’t until 6th and 7th that we started to get it.
5. Rare Gem experience is made up of:
1. In game visuals and explosions
2. Last hurrah with explosions
3. Slot machine mechanic at the end
6. Most successful had ALL of them, but with trend analysis we could understand the tradeoffs between including each of these components or
not.
7. This informed our development and let us make explicit decisions around the tradeoffs in the content design.

15. Summary
A game analytics playbook is your plan for using data
to inform the decisions you make in operating your
game.
Context Question Data / Tools Action
1. I’ve told you what a playbook is and how it relates to building and operating you game.
2. Your next step is to consider where your game is at in its lifecycle, and start developing your plays.
!
3. I would love to talk more about this with you! I love thinking through this process and how to make it both accessible and actionable.

16. Allison Bilas
@allisonbilas / allison@gameanalytics.com
THANK YOU!