Making a gameplay experience that players will like is not a trivial task. A lot of decisions need to be made along the way, “How hard should this boss be? Do players visit that corner of the map? Do they find the narrative appealing?“ are just some of the questions. Through playtesting and in-game telemetry we learn about our players – how they play and how they feel about our game as a whole, and its different aspects. The goal of this talk is to share our experience on how we use these insights to steer our game direction towards players’ preferences so we can deliver an enjoyable and thrilling gameplay experience.

1. Nikola Vasiljevic – Head of Insights & Analytics at Mad Head Games
Boosting Game Design
with Analytics

2. Why do we need analytics for game design?
Analytics’ role is to aid game designers by providing insights from player feedback that lead to more valid and reliable decision making.
In that way, it helps bridge the gap between game designers’ vision and players’ expectations.
Creative problem solving doesn’t come with a predefined set of steps we can follow. Instead, we explore the space of challenges*,
diverging and converging our focus until we reach a state we’re satisfied with. Analytics help navigate the space of challenges.
The key to success is for that state to be matching players expectations, because success is relative, and it’s anchored in delivering
on one’s expectations.
B
A ASKING QUESTIONS,
IDEA GENERATION
CONCRETIZATION AND
PROTOTYPING
DECISION MAKING
1. Opening up the
problem
(Divergence)
2. Analyzing
the problem
(Emergence)
3. Closing the
problem
(Convergence)
CHALLENGE SPACE
Initial state Desired state
*Gamestorming (2010) – Gray, Brown, Macanufo

3. Role of insights in each game-dev stage
STAGE CONTENT SCOPE KEY INSIGHTS UNDERTAKINGS
Pre-
production
Concept
It is centred on the ideation of the game Competitor evaluation
Documentation
The core plan for features and game scope Define overall UX vision and ideal player
experience from game features
Prototype
Initial implementations, like whitebox and
user flows
Usability testing on basic interaction and core
loop
Production
Production
This stage encompasses the main cycle of
the main cycle of development, where
content is added and the level of polish
increases
Evaluation of usability, behavioural, and
attitudinal aspects of the player experience at
each milestone
Alpha
Milestone reached when core gameplay
features are implemented
Usability testing on features and player
experience evaluation
Beta
Milestone reached when a complete version
of the game with full content is implemented
Usability on onboarding, full-playthrough
testing
Gold
Milestone reached when a version of the
game is at quality and is ready to be
launched for public release
Playtesting on final balancing and tuning
Post-production/Live Service
It covers all actions after the game has been
released, including patches, expansions, and
live content
Ongoing usability, balancing, and player
experience evaluation as more content is
released
GA
GUR
Games User Research – Drachen, Mirza-Babaei, Nacke

4. Game User Research (GUR) and Game Analytics (GA) as
players’ direct and indirect feedback
1. GUR: Gives insight into players’ experience - how players feel and what they think about our game. Answers the WHYs
2. GA: Gives insight into player behaviour – show us how the players are playing our game. Answers the WHATs
3. Visual-Audio Analytics (gameplay recordings with players’ comments): Observational techniques used as an
addition to previuos methods; usefil in showing us what players are doing at some exact moment; audio let’s us record
“heats of the moment“

5. GUR – Playtesting as a way to get players’ opinions and
experience
Upon playing a game, players fill out a questionnaire, professionally designed so as to avoiding auto-suggestive and biased
questions. This is how we get players’ opinion on important game matters:
• Usability: can players use the specific mechanics easily? Mechanics need to be clearly stated
• Appreciation: do players like their overall experience of the game/ specific game moments?
• Originality: does the game feel original or as something already seen
• Clarity: do the players find the mechanics easy to grasp and use, and understand what they have to do in general?
• Fun: do the players find the game fun to play?
• Challlenge difficulty: how hard is it to face the game’s challenges, such as combat, puzzles etc.?
• Balance: is any of the game’s experiences prevailing or everything is in balance?
• Satisfaction with game elements: Narrative, pacing, sound, setting & atmosphere, graphics, characters, mechanics,
animations, controls, UI, tutorials, navigation
• HLW – highlights, lowlights, wishlist
• Image questions: We can test for some statements about our game, i.e. “This is a high quality game.“
• Recommendation and Purchase Intent: Would players purchase the game if available now? Would they recommend it to
their friends?

6. Game Analytics – Telemetry signals from the game that
uncover patterns in player behaviour
Game analytics involves using quantitative measures, metrics, and tools that can be used to track events
that occur over the course of a game, with the goal of capturing such data for statistical analysis.
It is based on factual data!
Game analytics looks at users from two perspectives:
• As customers – tracking retention/churn, conversion, LTV
• As players – tracking their behaviour and experience

7. Game Analytics – The road to insights is paved with Data
Pipeline
Ensures a datastream from your in-game telemetry events to your database, and analytics tools.
You can build it yourself, or you can opt-in for a end-to-end cloud solution.
MongoDB
Tableau
Prep BigQuery Tableau
MySQL
Rstudio
Message
Broker
NoSQL
Database
ETL Tool
SQL
Warehouse
Analytics/
Vizualization
Machine
Learning
• Accepts events from game client
• Provides data structures
• Sends them to the next data node
• Stores telemetry events
• Requires no structure
• Provides flexibility and speed
• Data cleansing
• Standardization
• Deduplication
• Verification
• Sorting
• Provides structure
• Data integrity/ACID
• Complex querying
• Reporting
• Insights
• Decision making
• Forecasting/Predictive analytics (churn,
conversion)
• Classification
• Segmentation
• Sequence mining
• Decision trees
COLLECTION INJECTION PREPARATION COMPUTATION & PRESENTATION

8. Game Analytics – Events as building blocks of analytics
Events are one-off in-game actions that are tracked alongside with a set of parameters that define them more closely.
In specific occasions, you can track event streams – contextual sets of many events. A good example is combat event, which
consists of many attacks, reloads, dodges, ability usages, consumables etc.
List of some trackable events:
• Attack
• Death
• Combat
• Reload
• Dodge
• Position ping
• Tutorial_watched
• Puzzle_solved
• Purchase made
• …
event_id weapon_mod
combat_id weapon_upgrades
attack_type weapon_durability
attack_subtype death
attack_dealer death_id
attack_receiver combo_attack_name
boss_fight distance_from_enemy
combat_phase enemy_in_cover
attack_success player_in_cover
hitspot stat_effect_dealer
weakspot stat_effect_receiver
dmg_amount armor_type
ammo_left_total weapon_type
ammo_left_magazine weapon_name
session_data_block session_id, session_duration
common_data_block game_id, build_id
user_data_block user_id, control_type, difficulty, graphics_setting
timestamp, gametime, area_name, mission_name, objective_name,
player_location_xyz, health, stamina, armor_upgrades,
weapon_upgrades, status_effects, consumables_left
standard_parameter_block
data blocks
Event: ATTACK
parameters

9. General use cases of Game Analytics in Game Design
1. Identifying pain points and bottlenecks – discover where and why players get stuck
2. Feature usage/usability – discover do player use game features and mechanics (in intended
way)
3. Balancing – get familiar with how different mechanics work together and tweak the balance
between them
4. Player segmentation – discover different player’ segments based on their behaviour
5. Level design optimization – navigation heatmaps that tell us where players move, where
they fight, die, explore, find secret areas and items
6. System analytics – discover if your system design behaves in intended way, i.e. NPC AI

10. Use cases from Scars Above
1. Balancing game difficulty in general
2. Balancing a boss fight
3. Examining players’ strategy to defeating enemies and bosses
and if it’s in line with game designers intention
4. Balancing weapons in terms of general and situational usage
5. Identifying ammo depletion hotspots that break the gameplay balance
6. Balancing weakspot accuracy
7. Tweaking stamina depletion
8. Tweaking the ability system
9. Tweaking consumables and gadgets system
10. Basically, looking at how everything of the above comes together
...

11. Example: Combat analysis with GA
QUANTITY,
FREQUENCY,
OUTCOME
FUN,
FAIRNESS…
DIFFICULTY,
NPC ACCURACY,
STRATEGY…
LOCATION
CAUSE
TIMING
PLAYER STATE
PREFERENCE,
USAGE,
ACCURACY,
AMMO
DEPLETION…
RELOADING,
DODGING,
CONSUMABLES,
ENVIRONMENTAL
1. Define your approach:
• Predefined questions
• “Ammo depletion in combat“
• “Are players using the consumables in
combat?“
• Explorative approach
• Usually it’s a combination of both
2. Choose your metrics and start adding insight
layers to your area of interest

12. GUR (left) and GA (right) combo for Boss Fight Analysis
Success factors:
• Recognized the pattern
• Successful switch from defense to offense
• Weakspot accuracy
• Consumable usage
• Avoiding special attack
• Constant movement instead of dodging

13. For takeaway
1. Combine GA and GUR to a holy grail to get the full picture about:
• Players behaviour – the skeleton (usage data)
• Players experience – the meat (attitude data)
2. Understand your data – noise, limitations etc.
3. Know your stakeholders’ needs and the decisions they need to make
4. Provide actionability – answer the questions WHAT | WHY | SO WHAT | WHAT NOW
Give actionable recommendations that contribute to better decision making

14. THANK YOU FOR YOUR ATTENTION!
Q&A
nikola.vasiljevic@madheadgames.com