1) Carbine Studios wanted to identify which of its 125,000 beta testers were most likely to remain long-term players to combat customer churn. 2) Surveys and predictive modeling identified 5 unique gamer segments with different characteristics and spending behaviors. 3) Key insights included that 8% of beta testers were at risk to churn, females were twice as likely to churn, and passion for the game was more important than demographics in predicting loyalty.

1. Identifying Unique
Gamer Types Using
Predictive Analytics
Natasha Balac Ph.D.
Angel Evan
Speakers:

2. To ensure the success of their new game,
Carbine Studios would need to combat
one of the biggest challenges in gaming:
customer churn.
THE BUSINESS CHALLENGE

3. To do that they would need to identify
which of their 125,000 beta testers were
most likely to remain long-term players.
This was one of 14 questions the company wanted answers to, as part of an overall learning agenda.

4. Our Approach:
Leverage the power of the 125,000
beta testers by deploying a survey
designed with specific questions
to uncover which customers were
most likely to churn.
In general, we recommend using primary data wherever possible, as syndicated data sources are becoming less and less useful.

5. We created a learning agenda to determine which data
attributes we’d need to build our predictive models.
No. Question
1 Who are the unique customer types?
2 What are the largest segments of customers in terms of demographics?
3 Which customer segments spend the most time playing MMOs?
4 Which customers are most eagerly anticipating a new MMO?
5 Which gaming devices do WildStar gamers use, e.g., PlayStation, mobile phone etc.?
6 What are the segment’s motivations, e.g., Enjoy_Storyline/lore, Enjoy_ArtStyle, etc. How do they differ?
7 How should different gamer types be treated, i.e., marketing tactics
8 What other games do WildStar players also play?
9 What kinds of people are excited about WildStar?
10 Why are people excited about WildStar?
11 Who is most likely to be a long term player, i.e., who is most interested in PvE content
12 What are the characteristics of gamers who beta test MMOs?
13 Which attributes of a game cause people to spend money on a game?
14 Who are the characteristics of potential “churners"?
The survey, including its questions and skip logic were specifically design to understand customer churn, not to simply collect a bunch of general information on
customers.

6. This resulted in 48 million individual data cells
that would need to be whipped into shape.
In general, we recommend using primary data wherever possible, as syndicated data sources are becoming less and less useful. The importance of the data preparatio
stage cannot be overstated

7. Using a decision tree algorithm, we tried to
predict which customers were most likely to
churn by modeling various attributes.
Decision Tree Result:
83% accuracy
ROC Curve Probability
Interpretation
.90-1 = excellent (A)
.80-.90 = good (B)
.70-.80 = fair (C)
.60-.70 = poor (D)
.50-.60 = fail (F)
ROC Area =.863
Output ResultsQ1: Who are the unique customer types?
Predictive Algorithms used: Decision Tree, Decision Stump, Decision Prunenet. ROC curves are a way of visualizing performance of a binary classifier, i.e., two different
output classes. ROC curve visualizes all possible thresholds vs. a misclassifcation rate is only an error rate for a single threshold.

8. A demographics-only approach to the model
yielded low accuracy scores, so we shifted to
clustering algorithms to identify churners.
Cluster EvaluationQ1: Who are the unique customer types?
Sum of Squared Error (SSE) is
most common measure.
For each point the error is the
distance to the nearest cluster.
To get SEE, we square these
errors and then sum.
x is a data point in cluster Ci and
mi us the representative point
for cluster Ci
Given the two clusterings, we
can choose the one with the
smallest error
Rather than use the decision tree algorithm to predict churners, we instead created a segment for them, i.e., grouped them together so we could see which attributes
they commonely expressed. K-means clustering found intersting groups of gamers with different spend levels.

9. Key Insights:
1. Just because gamers play a lot doesn’t
mean they spend a lot. And just
because they make a lot doesn’t mean
they spend a lot.
2. There is no single “silver bullet”
predictive attribute for customer churn.
3. 8% of beta testers are at risk for churn.
4.Females are twice as likely to churn.
High household incomes have nothing to do with loyalty. Passion for the game (and topic) are more important.
PvE content appeals to many different clusters. But Scientists and Explorers seem to gravitate toward PvE the most.

10. SEGMENT 1
(BIG SPENDERS)
SEGMENT 2
(BREAD AND BUTTER)
SEGMENT 3A
(STUDENTS)
SEGMENT 3B
(BIG SPENDER
STUDENTS)
SEGMENT 4
(FEMALES)
Male Male Male Male Female
37% have kids 20% have kids 6% have kids 13% have kids 38% have kids
HHI $80k+ HHI $20-80k HHI $<20k HHI $20-80k HHI $20-120k
Non-Student Non-Student Student Student
Both student and non-
student
Pre-ordered a console Pre-ordered a console Didn’t pre-order console Pre-ordered a console Didn’t pre-order console
11-30 hours/wk playing
MMOs
11-20 hours/wk playing
MMOs
11-20 hours/wk playing
MMOs
40+ hours/wk playing
MMOs
11-20 hours/wk playing
MMOs
Spent $ on 20+ games Spent $ on 6-20 games Spent $ on 0-5 games Spent $ on 20+ games Spent $ on 0-5 games
PvE Content, Character
Progression
PvE Content
PvE Content, Character
Progression
PvP Storyline and Lore
Explorer, Settler Scientist, Explorer Soldier, Explorer Explorer, Scientist Explorer, Settler, Scientist
Excitement Level: 9 Excitement Level: 10 Excitement Level: 10 Excitement Level: 10 Excitement Level: 10
DEMOGRAPHICSGAMERBEHAVIOR
What did we do with all the data?
We created a customer segmentation...
High household incomes have nothing to do with loyalty. Passion for the game (and topic) are more important.
PvE content appeals to many different clusters. But Scientists and Explorers seem to gravitate toward PvE the most.

11. …and a set of marketing recommendations
that mapped to individual segment behaviors.
SEGMENT 1
(BIG SPENDERS)
SEGMENT 2
(BREAD AND BUTTER)
SEGMENT 3A
(STUDENTS)
SEGMENT 3B
(BIG SPENDER
STUDENTS)
SEGMENT 4
(FEMALES)
Mobile Display Mobile Display Mobile Display Mobile Display Mobile Display
Promoted Facebook
posts
and YouTube ads
Promoted Facebook
posts
and YouTube ads
Promoted Facebook
posts
and YouTube ads
In-game advertising (Tablet)
In-game advertising
(Tablet)
In-game advertising
(Tablet)
In-game advertising
(Tablet)
In-game advertising
(Tablet)
Interactive Trailers Interactive Trailers Interactive Trailers Interactive Trailers Interactive Trailers
In-game exclusives
(Cool Gear)
In-game exclusives
(Cool Gear)
In-game exclusives
(Cool Gear)
In-game exclusives
(Cool Gear)
In-game exclusives
(Cool Gear)
Mobile/online Coupons Mobile/online Coupons Mobile/online Coupons
High-res stylized artwork High-res stylized artwork High-res stylized artwork High-res stylized artwork High-res stylized artwork
Email (optimized for mobile)
Email (optimized for
mobile)
Email (optimized for
mobile)
Email (optimized for
mobile)
Email (optimized for
mobile)
Community-created
content, e.g., parody
videos, etc.
Community-created
content, e.g., parody
videos, etc.
Community-created
content, e.g., parody
videos, etc.
Social Media Social Media Social Media Social Media Social Media
ACQUISITIONRETENTION
Marketing recommendations were mapped to the individual gamer behavioral attributes that make up each unique segment and broken out by acquisition vs. retention.
Retention is especially important, as we were dealing with a customer churn problem.