This document discusses Microsoft's use of data-driven user research methods for game development. It summarizes different types of data collected, including in-game telemetry, Xbox Live usage data, and online forum discussions. Examples are provided of analyses conducted on data from games like Project Gotham Racing 4 and Halo 3 to understand player behavior and identify underutilized game elements. The goal is to better understand the player experience and Xbox ecosystem to inform game design and personalization.

1. Data-Driven
Games User Research
Thomas Zimmermann , Microsoft Research
Bruce Philips, Microsoft Game Studios
Nachiappan Nagappan , Microsoft Research
Chuck Harrison, Microsoft Game Studios
© Microsoft Corporation

2. Data! Data! Data!
In-game data
– Detailed usage data collected through telemetry (often beta)
– For example, games modes that are played, coordinates where
players died, how they alternated between weapons, etc.
Xbox Live data
– Player Experience Panel, a group of ~18,000 players (volunteered)
– Games played, achievements earned, as well as presence
information.
– Data via feed from the Xbox Community Developer Program
Online communities
– Monitor internet forums to enrich quantitative findings in our.
– Experimenting with natural language processing (NLP)
techniques
© Microsoft Corporation

3. Project Gotham Racing 4
Across all races:
• 2 of 9 game modes were used
in < 0.5% of races
• 12 of 29 event types were used
in < 1% of races
• 50 of 134 vehicles were used
in < 0.25% of races
When looking at multiplayer races:
• 2 of 4 game modes were used
in < 2% of races
• 7 of 16 event types were used
in < 0.1% of races
• 53 of 133 vehicles were used
in < 0.25% of races
Kenneth Hullett, Nachiappan Nagappan, Eric Schuh, John Hopson:
Data analytics for game development. ICSE 2011 NIER track.
© Microsoft Corporation

4. Player progression in Halo 3
Bruce Phillips. Peering into the Black Box of Player Behavior: The
Player Experience Panel at Microsoft Game Studios. GDC 2010
© Microsoft Corporation

5. Future work
© Microsoft Corporation

6. Understanding the Xbox Ecosystem
Games played
Games played
Achievements
by friends
Engagement Achievements
Friends by friends
Subscriptions
© Microsoft Corporation

7. Social play
© Microsoft Corporation

8. Personalization
© Microsoft Corporation

9. After downloading Microsoft Flight for free, players can jump
into hours of exciting gameplay on the Big Island of Hawaii. […]
Those looking to deepen their experience can purchase and
download additional content that adds new aircraft, regions
and customization options.
Microsoft Press Release, January 4th, 2012 – Microsoft Flight Takes to the Skies
© Microsoft Corporation

10. Future work
© Microsoft Corporation