Building an Effective LiveOps Strategy Using PlayFab
James Gwertzman
Co-founder, CEO
1. Pokemon Silver
2. Pokemon Gold
3. The Sims
4. Roller Coaster Tycoon
5. Final Fantasy 9
6. Tony Hawk Pro Skater 2
7. Diablo 2
8. Super Marios Bros. Deluxe
9. The Sims Livin’ Large
10.Who Wants to be a Millionaire?
Top Grossing Games – Then
2000
1. Harry Potter and the Sorcerer’s Stone
2. Metal Gear Solid 2
3. The Sims: Hot Date
4. The Sims
5. Pokemon: Crystal
6. Backyard Basketball
7. Grand Theft Auto 3
8. Myst 3: Exile
9. Civilization 3
10.Gran Turismo 3
2001
1. Pokemon Silver
2. Pokemon Gold
3. The Sims
4. Roller Coaster Tycoon
5. Final Fantasy 9
6. Tony Hawk Pro Skater 2
7. Diablo 2
8. Super Marios Bros. Deluxe
9. The Sims Livin’ Large
10.Who Wants to be a Millionaire?
Top Grossing Games – Then
2000
1. Harry Potter and the Sorcerer’s Stone
2. Metal Gear Solid 2
3. The Sims: Hot Date
4. The Sims
5. Pokemon: Crystal
6. Backyard Basketball
7. Grand Theft Auto 3
8. Myst 3: Exile
9. Civilization 3
10.Gran Turismo 3
10%
carryover
2001
Top Grossing Games – Now
1. Monster Strike
2. Clash of Clans
3. Game of War – Fire Age
4. Mobile Strike
5. Pokemon Go
6. Clash Royale
7. Puzzle and Dragons
8. Candy Crush Saga
9. Westward Journey
10.Clash of Kings
1. Monster Strike
2. Honor of Kings
3. Fate/Grand Order
4. Clash Royale
5. Candy Crush Saga
6. Puzzle and Dragons
7. Westward Journey
8. Lineage 2
9. Clash of Clans
10.Game of War – Fire Age
2016 2017
Top Grossing Games – Now
1. Monster Strike
2. Clash of Clans
3. Game of War – Fire Age
4. Mobile Strike
5. Pokemon Go
6. Clash Royale
7. Puzzle and Dragons
8. Candy Crush Saga
9. Westward Journey
10.Clash of Kings
1. Monster Strike
2. Honor of Kings
3. Fate/Grand Order
4. Clash Royale
5. Candy Crush Saga
6. Puzzle and Dragons
7. Westward Journey
8. Lineage 2
9. Clash of Clans
10.Game of War – Fire Age
2016 201770%
carryover
What Do Those Titles Have in Common?
LiveOp
s
“In three years our games have generated
over $80m revenue. Depending on how you
account for it, Live Ops initiatives
generated between one and two-thirds of
that revenue…. Live Ops underpins
everything we do.”
-- Simon Hade, COO, SpaceApe
“The single greatest predictor of
success and of sustainable
competitive advantage in this
business is live game operations.”
-- Owen Mahoney, CEO Nexon
A Tale of Two Games
0
500
1,000
1,500
2,000
2,500
3,000
1
5
9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
73
77
81
85
89
93
97
101
105
109
113
117
121
125
129
133
137
141
145
149
153
157
161
165
169
173
177
181
185
DAU
Thousands
What is LiveOps?
• Changes to the game after it goes live
• Designed to boost monetization, engagement, retention
• Focused on how players interact with the game and each other
• Preferably without changing game code
Live Ops
• New cosmetic item
• Limited time offer
• Weekend event
• Paid UA campaign
Not Live Ops
• New functional item
• New crafting mechanic
• New tournament mode
• New viral mechanic
LiveOps Requires Games to Be Operated as Services
Game as Packaged Good
• Publishing
• Retail
• One size fits all
• Most work done before launch
• Single-player
• Content is king
Game as Service
• Operating
• High frequency trading
• Personalized
• Most work done after launch
• Multi-player and social
• Data is king
Server
scaling
Crash
reports
Monitoring
Payments
Anti-fraud
/ Anti-
cheating
Social
Streaming
eSports
Mobile
PC
Console Multiplayer
/
Matchmaking
Chat
Trades/Gift
s
Scripting /
Rules
Matchmaking
In-Game
Commerce
Guilds /
Social
Leaderboards
/
Tournaments
Player Data
Player
Messaging
BI /
Analytics
Live Events
User
Acquisition
Offers /
Promotions
Content Mgmt A/B Testing
Community /
Support
Marketing
Automation
Advertising
Live Ops Tools & Analytics
Game Services Monetization
HostingPlayer
Acquisition
Distributi
App
Deployment
Unifying LiveOps with a single, highly-scalable backend
platform
Server
scaling
Crash
reports
Monitoring
Payments
Anti-fraud
/ Anti-
cheating
Social
Streaming
eSports
Mobile
PC
Console Multiplayer
/
Matchmaking
Chat
Trades/Gift
s
Scripting /
Rules
Matchmaking
In-Game
Commerce
Guilds /
Social
Leaderboards
/
Tournaments
Player Data
Player
Messaging
BI /
Analytics
Live Events
User
Acquisition
Offers /
Promotions
Content Mgmt A/B Testing
Community /
Support
Marketing
Automation
Advertising
Live Ops Tools & Analytics
Game Services Monetization
HostingPlayer
Acquisition
Distributi
App
Deployment
Unifying LiveOps with a single, highly-scalable backend
platform
Server
scaling
Crash
reports
Monitoring
Payments
Anti-fraud
/ Anti-
cheating
Social
Streaming
eSports
Mobile
PC
Console Multiplayer
/
Matchmaking
Chat
Trades/Gift
s
Scripting /
Rules
Matchmaking
In-Game
Commerce
Guilds /
Social
Leaderboards
/
Tournaments
Player Data
Player
Messaging
BI /
Analytics
Live Events
User
Acquisition
Offers /
Promotions
Content Mgmt A/B Testing
Community /
Support
Marketing
Automation
Advertising
Live Ops Tools & Analytics
Game Services Monetization
HostingPlayer
Acquisition
Distributi
App
Deployment
Unifying LiveOps with a single, highly-scalable backend
platform
Server
scaling
Crash
reports
Monitoring
Payments
Anti-fraud
/ Anti-
cheating
Social
Streaming
eSports
Mobile
PC
Console Multiplayer
/
Matchmaking
Chat
Trades/Gift
s
Scripting /
Rules
Matchmaking
In-Game
Commerce
Guilds /
Social
Leaderboards
/
Tournaments
Player Data
Player
Messaging
BI /
Analytics
Live Events
User
Acquisition
Offers /
Promotions
Content Mgmt A/B Testing
Community /
Support
Marketing
Automation
Advertising
Live Ops Tools & Analytics
Game Services Monetization
HostingPlayer
Acquisition
Distribut
App
Deployment
Unifying LiveOps with a single, highly-scalable backend
platform
Server
scaling
Crash
reports
Monitoring
Payments
Anti-fraud
/ Anti-
cheating
Social
Streaming
eSports
Mobile
PC
Console Multiplayer
/
Matchmaking
Chat
Trades/Gift
s
Scripting /
Rules
Matchmaking
In-Game
Commerce
Guilds /
Social
Leaderboards
/
Tournaments
Player Data
Player
Messaging
BI /
Analytics
Live Events
User
Acquisition
Offers /
Promotions
Content Mgmt A/B Testing
Community /
Support
Marketing
Automation
Advertising
Live Ops Tools & Analytics
Game Services Monetization
HostingPlayer
Acquisition
Distributi
App
Deployment
Unifying LiveOps with a single, highly-scalable backend
platform
Server
scaling
Crash
reports
Monitoring
Payments
Anti-fraud
/ Anti-
cheating
Social
Streaming
eSports
Mobile
PC
Console Multiplayer
/
Matchmaking
Chat
Trades/Gift
s
Scripting /
Rules
Matchmaking
In-Game
Commerce
Guilds /
Social
Leaderboards
/
Tournaments
Player Data
Player
Messaging
BI /
Analytics
Live Events
User
Acquisition
Offers /
Promotions
Content Mgmt A/B Testing
Community /
Support
Marketing
Automation
Advertising
Live Ops Tools & Analytics
Game Services Monetization
HostingPlayer
Acquisition
Distributi
App
Deployment
Unifying LiveOps with a single, highly-scalable backend
platform
Server
scaling
Crash
reports
Monitoring
Payments
Anti-fraud
/ Anti-
cheating
Social
Streaming
eSports
Mobile
PC
Console Multiplayer
/
Matchmaking
Chat
Trades/Gift
s
Scripting /
Rules
Matchmaking
In-Game
Commerce
Guilds /
Social
Leaderboards
/
Tournaments
Player Data
Player
Messaging
BI /
Analytics
Live Events
User
Acquisition
Offers /
Promotions
Content Mgmt A/B Testing
Community /
Support
Marketing
Automation
Advertising
Live Ops Tools & Analytics
Game Services Monetization
HostingPlayer
Acquisition
Distributi
App
Deployment
Unifying LiveOps with a single, highly-scalable backend
platform
PlayFab is a suite
of products that
complement your
existing backend
infrastructure. Mix-
and-match to meet
your needs, or adopt
the entire platform
as a powerful base
for current and
future games.
The PlayFab Platform
Fully Cross Platform and Engine Agnostic
Targeting:
• All major devices and platforms
• All popular game engines
• All popular cloud services
• All popular app stores
More Than 3,000 Developers and Growing
Building Blocks of LiveOps
• Business Intelligence (actionable analytics)
• Live events
• Player messaging
• Content updates
• Store / catalog management
• Offers and promotions
• Customer service
• User acquisition
Modern Game Success is Driven by Use of Real-Time Data
• How well does a game know its players? Across games?
• How well can the game act on that information?
Player Cross-
promotion
Tournaments
Game tuning
Targeted ads
Pricing
Events
Service resolution
Acquisition
Offers
Content
Dashboards are Just the Beginning
“Vanity Metrics” Are Not Actionable
Game Events Metrics
?
Daily Retention Cohorts Only Show the Symptoms
Hey, good
acquisition!
Huh, but weak
retention
So… Now
What?
Using Metrics More Effectively is About Learning
Source: Lean Startup Framework
Idea
Try
Data
“Ideas” Take Work – Explore the Data Idea
Try
Data
21,340
46,953
982 1,273 484 1,162 4,881 5,495 3,846 3,728 3,651
14,200
-44,257
-91,368
-12,092 -11,355
-3,308
-7,042
-19,343
-9,996
-2,423 -891 -543 -852
Acquired Level 1 Level 2 Level 3 Level 4 Level 5 Level 6+ Level 10+ Level 20+ Level 30+ Level 40+ Level 50+
Population Churned
Where are users pooling and
dropping out?
What is different between users who progress
and those who don’t?
Come Up With a Hypothesis Idea
Try
Data
21,340
46,953
982 1,273 484 1,162 4,881 5,495 3,846 3,728 3,651
14,200
-44,257
-91,368
-12,092 -11,355
-3,308
-7,042
-19,343
-9,996
-2,423 -891 -543 -852
Acquired Level 1 Level 2 Level 3 Level 4 Level 5 Level 6+ Level 10+ Level 20+ Level 30+ Level 40+ Level 50+
Population Churned
“Users who progress are earning
achievements early”
Test Your Idea With Experiments Idea
Try
Data
A/B test to try
multiple tests at
the same time
Measure the Results of Your Change Idea
Try
Data
21,340
46,953
1,212 1,354 578 1,239 5,388 6,323 4,231 3,923 4,123
15,300
-44,257
-76,368
-12,092 -10,447
-3,043
-6,479
-17,796
-9,196
-2,229 -820 -500 -784
Acquired Level 1 Level 2 Level 3 Level 4 Level 5 Level 6+ Level 10+ Level 20+ Level 30+ Level 40+ Level 50+
Population Churned
Did the change reduce the
churn?
Is the change focused in a
particular A/B treatment
(test group)?
Requirements for Actionable Business Intelligence
• Rich set of events – you can’t measure what you don’t capture
• Deeper access to your data than many analytics tools allow
– And support for semi-structured data
• Tools to easily run experiments w/o writing new code
Rules Engine
1. Collect all game data
and events in one place
Harnessing the Power of Game Data in PlayFab
2. Monitor & visualize data in real-
time
3. Exchange data with
other tools & services
5. Make real-time decisions
and take actions
Match
started
Forum
post
Crash
logge
d
+10
XP
Login
(via
FB)
Win
match
Buy
item
Chat
msg
Support
ticket
Existing
BackendPartner
ServicesPlayFab
Backend
The Game
Data
warehouse
Spent
$10
4. Store data for later
offline analysis
Capturing Events
Rules Engine
User Segmentation
Scheduled Tasks
Export to Data Warehouse
• Export your game data into a
fully featured data warehouse
• Run custom queries
• Analyze using Tableau, Looker,
or other 3rd party tools
Live Events
• Fun – boost engagement
• Monetization – boost revenues, fix economy
• Marketing – cross promote another game
• Content – promote new update
• Tactical – wielded like a scalpel
• Hybrid – mix of the above
Start With a Calendar
KPI actuals KPI forecast
Name Mechanic Fiction Duration
Ultimate Dragon Rush Special dungeon, available only
after normal dungeon
“Starlight Sanctuary” is
cleared. Legendary Plus
difficulty, stamina cost of 99
The skies are ominous and the earth trembles
beneath our feet. Behold! The legendary
dragons come forth and bring you a challenge
to remember. Do you have what it takes to
enter their realm? Or will you kneel before
their will?
3/3 (Sat), 12:00AM – 3/4 (Sun), 11:59 PM
Pal Egg Event Special Pal Egg machine, each
pull is 1000 Pal Points. Seven
special new monsters are
available, including Life
Dragon, Cao Cao, and Lu Bu.
They may be small, but their stature is great!!
Originally just Five Elements Gods, the minis
have now reached the Three Kingdoms period.
3/7 (Wed), 12:00AM – 3/13 (Tues), 11:59 PM
Descended Challenge Special dungeon with 15 floors,
each floor with powerful
enemies. No continues are
allowed. Each floor gives
special rewards.
One Descended Dungeon, with the right team,
and you think “Okay, not too bad. I can handle
this.” You think you’ve won, but instead you
keep going. They keep coming. Like meteors
from the sky, impacting on the Earth and
forcing you to run!
3/2 (Fri), 12:00AM – 3/13 (Tues), 11:59 PM
Event Mechanic vs. Fiction
Live/Recurring Event Examples
AdVenture Capitalist, Hyper Hippo
One-time Events
Recurring Events
Player Messaging
In-game message
Push notifications
Email
Content Updates
Store/Catalog Management
Offers and Promotions
Offers and Promotions
Customer Service
Customer Service 2 – Keep it Secret, Keep it Safe
Customer Service 3 – The Revenginating
User Acquisition
• Zero work to integrate
– Don’t need a separate Kochava SDK
– No code changes from existing PlayFab login
• Automatically generates attribution events
• More to come!
PlayFab Demo
Call to Action
• Create an account for free (https://developer.playfab.com)
• Consider how your LiveOps strategy can be applied
– What metrics do you use now? What continuous learnings? What
have you wanted to have, but not yet been able to?
– What data do you need to drive good decisions?
• Visit our booth, and:
– Pick up your free LiveOps guide
– Schedule a demo
• Give us feedback!
Questions?
bvanous@microsoft.com
brendan@playfab.com
@playfabnetwork
playfab.com

Introducing PlayFab -- Effective LiveOps

  • 1.
    Building an EffectiveLiveOps Strategy Using PlayFab James Gwertzman Co-founder, CEO
  • 2.
    1. Pokemon Silver 2.Pokemon Gold 3. The Sims 4. Roller Coaster Tycoon 5. Final Fantasy 9 6. Tony Hawk Pro Skater 2 7. Diablo 2 8. Super Marios Bros. Deluxe 9. The Sims Livin’ Large 10.Who Wants to be a Millionaire? Top Grossing Games – Then 2000 1. Harry Potter and the Sorcerer’s Stone 2. Metal Gear Solid 2 3. The Sims: Hot Date 4. The Sims 5. Pokemon: Crystal 6. Backyard Basketball 7. Grand Theft Auto 3 8. Myst 3: Exile 9. Civilization 3 10.Gran Turismo 3 2001
  • 3.
    1. Pokemon Silver 2.Pokemon Gold 3. The Sims 4. Roller Coaster Tycoon 5. Final Fantasy 9 6. Tony Hawk Pro Skater 2 7. Diablo 2 8. Super Marios Bros. Deluxe 9. The Sims Livin’ Large 10.Who Wants to be a Millionaire? Top Grossing Games – Then 2000 1. Harry Potter and the Sorcerer’s Stone 2. Metal Gear Solid 2 3. The Sims: Hot Date 4. The Sims 5. Pokemon: Crystal 6. Backyard Basketball 7. Grand Theft Auto 3 8. Myst 3: Exile 9. Civilization 3 10.Gran Turismo 3 10% carryover 2001
  • 4.
    Top Grossing Games– Now 1. Monster Strike 2. Clash of Clans 3. Game of War – Fire Age 4. Mobile Strike 5. Pokemon Go 6. Clash Royale 7. Puzzle and Dragons 8. Candy Crush Saga 9. Westward Journey 10.Clash of Kings 1. Monster Strike 2. Honor of Kings 3. Fate/Grand Order 4. Clash Royale 5. Candy Crush Saga 6. Puzzle and Dragons 7. Westward Journey 8. Lineage 2 9. Clash of Clans 10.Game of War – Fire Age 2016 2017
  • 5.
    Top Grossing Games– Now 1. Monster Strike 2. Clash of Clans 3. Game of War – Fire Age 4. Mobile Strike 5. Pokemon Go 6. Clash Royale 7. Puzzle and Dragons 8. Candy Crush Saga 9. Westward Journey 10.Clash of Kings 1. Monster Strike 2. Honor of Kings 3. Fate/Grand Order 4. Clash Royale 5. Candy Crush Saga 6. Puzzle and Dragons 7. Westward Journey 8. Lineage 2 9. Clash of Clans 10.Game of War – Fire Age 2016 201770% carryover
  • 6.
    What Do ThoseTitles Have in Common? LiveOp s
  • 7.
    “In three yearsour games have generated over $80m revenue. Depending on how you account for it, Live Ops initiatives generated between one and two-thirds of that revenue…. Live Ops underpins everything we do.” -- Simon Hade, COO, SpaceApe “The single greatest predictor of success and of sustainable competitive advantage in this business is live game operations.” -- Owen Mahoney, CEO Nexon
  • 8.
    A Tale ofTwo Games 0 500 1,000 1,500 2,000 2,500 3,000 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 101 105 109 113 117 121 125 129 133 137 141 145 149 153 157 161 165 169 173 177 181 185 DAU Thousands
  • 9.
    What is LiveOps? •Changes to the game after it goes live • Designed to boost monetization, engagement, retention • Focused on how players interact with the game and each other • Preferably without changing game code Live Ops • New cosmetic item • Limited time offer • Weekend event • Paid UA campaign Not Live Ops • New functional item • New crafting mechanic • New tournament mode • New viral mechanic
  • 10.
    LiveOps Requires Gamesto Be Operated as Services Game as Packaged Good • Publishing • Retail • One size fits all • Most work done before launch • Single-player • Content is king Game as Service • Operating • High frequency trading • Personalized • Most work done after launch • Multi-player and social • Data is king
  • 11.
    Server scaling Crash reports Monitoring Payments Anti-fraud / Anti- cheating Social Streaming eSports Mobile PC Console Multiplayer / Matchmaking Chat Trades/Gift s Scripting/ Rules Matchmaking In-Game Commerce Guilds / Social Leaderboards / Tournaments Player Data Player Messaging BI / Analytics Live Events User Acquisition Offers / Promotions Content Mgmt A/B Testing Community / Support Marketing Automation Advertising Live Ops Tools & Analytics Game Services Monetization HostingPlayer Acquisition Distributi App Deployment Unifying LiveOps with a single, highly-scalable backend platform
  • 12.
    Server scaling Crash reports Monitoring Payments Anti-fraud / Anti- cheating Social Streaming eSports Mobile PC Console Multiplayer / Matchmaking Chat Trades/Gift s Scripting/ Rules Matchmaking In-Game Commerce Guilds / Social Leaderboards / Tournaments Player Data Player Messaging BI / Analytics Live Events User Acquisition Offers / Promotions Content Mgmt A/B Testing Community / Support Marketing Automation Advertising Live Ops Tools & Analytics Game Services Monetization HostingPlayer Acquisition Distributi App Deployment Unifying LiveOps with a single, highly-scalable backend platform
  • 13.
    Server scaling Crash reports Monitoring Payments Anti-fraud / Anti- cheating Social Streaming eSports Mobile PC Console Multiplayer / Matchmaking Chat Trades/Gift s Scripting/ Rules Matchmaking In-Game Commerce Guilds / Social Leaderboards / Tournaments Player Data Player Messaging BI / Analytics Live Events User Acquisition Offers / Promotions Content Mgmt A/B Testing Community / Support Marketing Automation Advertising Live Ops Tools & Analytics Game Services Monetization HostingPlayer Acquisition Distributi App Deployment Unifying LiveOps with a single, highly-scalable backend platform
  • 14.
    Server scaling Crash reports Monitoring Payments Anti-fraud / Anti- cheating Social Streaming eSports Mobile PC Console Multiplayer / Matchmaking Chat Trades/Gift s Scripting/ Rules Matchmaking In-Game Commerce Guilds / Social Leaderboards / Tournaments Player Data Player Messaging BI / Analytics Live Events User Acquisition Offers / Promotions Content Mgmt A/B Testing Community / Support Marketing Automation Advertising Live Ops Tools & Analytics Game Services Monetization HostingPlayer Acquisition Distribut App Deployment Unifying LiveOps with a single, highly-scalable backend platform
  • 15.
    Server scaling Crash reports Monitoring Payments Anti-fraud / Anti- cheating Social Streaming eSports Mobile PC Console Multiplayer / Matchmaking Chat Trades/Gift s Scripting/ Rules Matchmaking In-Game Commerce Guilds / Social Leaderboards / Tournaments Player Data Player Messaging BI / Analytics Live Events User Acquisition Offers / Promotions Content Mgmt A/B Testing Community / Support Marketing Automation Advertising Live Ops Tools & Analytics Game Services Monetization HostingPlayer Acquisition Distributi App Deployment Unifying LiveOps with a single, highly-scalable backend platform
  • 16.
    Server scaling Crash reports Monitoring Payments Anti-fraud / Anti- cheating Social Streaming eSports Mobile PC Console Multiplayer / Matchmaking Chat Trades/Gift s Scripting/ Rules Matchmaking In-Game Commerce Guilds / Social Leaderboards / Tournaments Player Data Player Messaging BI / Analytics Live Events User Acquisition Offers / Promotions Content Mgmt A/B Testing Community / Support Marketing Automation Advertising Live Ops Tools & Analytics Game Services Monetization HostingPlayer Acquisition Distributi App Deployment Unifying LiveOps with a single, highly-scalable backend platform
  • 17.
    Server scaling Crash reports Monitoring Payments Anti-fraud / Anti- cheating Social Streaming eSports Mobile PC Console Multiplayer / Matchmaking Chat Trades/Gift s Scripting/ Rules Matchmaking In-Game Commerce Guilds / Social Leaderboards / Tournaments Player Data Player Messaging BI / Analytics Live Events User Acquisition Offers / Promotions Content Mgmt A/B Testing Community / Support Marketing Automation Advertising Live Ops Tools & Analytics Game Services Monetization HostingPlayer Acquisition Distributi App Deployment Unifying LiveOps with a single, highly-scalable backend platform
  • 18.
    PlayFab is asuite of products that complement your existing backend infrastructure. Mix- and-match to meet your needs, or adopt the entire platform as a powerful base for current and future games. The PlayFab Platform
  • 19.
    Fully Cross Platformand Engine Agnostic Targeting: • All major devices and platforms • All popular game engines • All popular cloud services • All popular app stores
  • 20.
    More Than 3,000Developers and Growing
  • 21.
    Building Blocks ofLiveOps • Business Intelligence (actionable analytics) • Live events • Player messaging • Content updates • Store / catalog management • Offers and promotions • Customer service • User acquisition
  • 22.
    Modern Game Successis Driven by Use of Real-Time Data • How well does a game know its players? Across games? • How well can the game act on that information? Player Cross- promotion Tournaments Game tuning Targeted ads Pricing Events Service resolution Acquisition Offers Content
  • 23.
    Dashboards are Justthe Beginning
  • 24.
    “Vanity Metrics” AreNot Actionable Game Events Metrics ?
  • 25.
    Daily Retention CohortsOnly Show the Symptoms Hey, good acquisition! Huh, but weak retention So… Now What?
  • 26.
    Using Metrics MoreEffectively is About Learning Source: Lean Startup Framework Idea Try Data
  • 27.
    “Ideas” Take Work– Explore the Data Idea Try Data 21,340 46,953 982 1,273 484 1,162 4,881 5,495 3,846 3,728 3,651 14,200 -44,257 -91,368 -12,092 -11,355 -3,308 -7,042 -19,343 -9,996 -2,423 -891 -543 -852 Acquired Level 1 Level 2 Level 3 Level 4 Level 5 Level 6+ Level 10+ Level 20+ Level 30+ Level 40+ Level 50+ Population Churned Where are users pooling and dropping out? What is different between users who progress and those who don’t?
  • 28.
    Come Up Witha Hypothesis Idea Try Data 21,340 46,953 982 1,273 484 1,162 4,881 5,495 3,846 3,728 3,651 14,200 -44,257 -91,368 -12,092 -11,355 -3,308 -7,042 -19,343 -9,996 -2,423 -891 -543 -852 Acquired Level 1 Level 2 Level 3 Level 4 Level 5 Level 6+ Level 10+ Level 20+ Level 30+ Level 40+ Level 50+ Population Churned “Users who progress are earning achievements early”
  • 29.
    Test Your IdeaWith Experiments Idea Try Data A/B test to try multiple tests at the same time
  • 30.
    Measure the Resultsof Your Change Idea Try Data 21,340 46,953 1,212 1,354 578 1,239 5,388 6,323 4,231 3,923 4,123 15,300 -44,257 -76,368 -12,092 -10,447 -3,043 -6,479 -17,796 -9,196 -2,229 -820 -500 -784 Acquired Level 1 Level 2 Level 3 Level 4 Level 5 Level 6+ Level 10+ Level 20+ Level 30+ Level 40+ Level 50+ Population Churned Did the change reduce the churn? Is the change focused in a particular A/B treatment (test group)?
  • 31.
    Requirements for ActionableBusiness Intelligence • Rich set of events – you can’t measure what you don’t capture • Deeper access to your data than many analytics tools allow – And support for semi-structured data • Tools to easily run experiments w/o writing new code
  • 32.
    Rules Engine 1. Collectall game data and events in one place Harnessing the Power of Game Data in PlayFab 2. Monitor & visualize data in real- time 3. Exchange data with other tools & services 5. Make real-time decisions and take actions Match started Forum post Crash logge d +10 XP Login (via FB) Win match Buy item Chat msg Support ticket Existing BackendPartner ServicesPlayFab Backend The Game Data warehouse Spent $10 4. Store data for later offline analysis
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
    Export to DataWarehouse • Export your game data into a fully featured data warehouse • Run custom queries • Analyze using Tableau, Looker, or other 3rd party tools
  • 38.
    Live Events • Fun– boost engagement • Monetization – boost revenues, fix economy • Marketing – cross promote another game • Content – promote new update • Tactical – wielded like a scalpel • Hybrid – mix of the above
  • 39.
    Start With aCalendar KPI actuals KPI forecast
  • 40.
    Name Mechanic FictionDuration Ultimate Dragon Rush Special dungeon, available only after normal dungeon “Starlight Sanctuary” is cleared. Legendary Plus difficulty, stamina cost of 99 The skies are ominous and the earth trembles beneath our feet. Behold! The legendary dragons come forth and bring you a challenge to remember. Do you have what it takes to enter their realm? Or will you kneel before their will? 3/3 (Sat), 12:00AM – 3/4 (Sun), 11:59 PM Pal Egg Event Special Pal Egg machine, each pull is 1000 Pal Points. Seven special new monsters are available, including Life Dragon, Cao Cao, and Lu Bu. They may be small, but their stature is great!! Originally just Five Elements Gods, the minis have now reached the Three Kingdoms period. 3/7 (Wed), 12:00AM – 3/13 (Tues), 11:59 PM Descended Challenge Special dungeon with 15 floors, each floor with powerful enemies. No continues are allowed. Each floor gives special rewards. One Descended Dungeon, with the right team, and you think “Okay, not too bad. I can handle this.” You think you’ve won, but instead you keep going. They keep coming. Like meteors from the sky, impacting on the Earth and forcing you to run! 3/2 (Fri), 12:00AM – 3/13 (Tues), 11:59 PM Event Mechanic vs. Fiction
  • 41.
    Live/Recurring Event Examples AdVentureCapitalist, Hyper Hippo One-time Events Recurring Events
  • 42.
  • 43.
  • 44.
  • 45.
  • 46.
  • 47.
  • 48.
    Customer Service 2– Keep it Secret, Keep it Safe
  • 49.
    Customer Service 3– The Revenginating
  • 50.
    User Acquisition • Zerowork to integrate – Don’t need a separate Kochava SDK – No code changes from existing PlayFab login • Automatically generates attribution events • More to come!
  • 51.
  • 52.
    Call to Action •Create an account for free (https://developer.playfab.com) • Consider how your LiveOps strategy can be applied – What metrics do you use now? What continuous learnings? What have you wanted to have, but not yet been able to? – What data do you need to drive good decisions? • Visit our booth, and: – Pick up your free LiveOps guide – Schedule a demo • Give us feedback!
  • 53.

Editor's Notes

  • #11 As you’ve seen w/ Z2, game industry was disrupted as it moved from packaged goods to services. This shift has happened for good reasons – much more valuable to run a service. But much more complex. It’s like running an ecommerce site.
  • #12 Picture even more complex when you consider what’s inside those boxes. Once upon a time, teams had to build all this themselves. Now at least there are third party solutions, but still every one of those lines represents code the team needs to write That’s a huge distraction from the game itself. And the elephant in the room, is that despite all these companies, all all this effort, this is still not a solved problem.
  • #13 Picture even more complex when you consider what’s inside those boxes. Once upon a time, teams had to build all this themselves. Now at least there are third party solutions, but still every one of those lines represents code the team needs to write That’s a huge distraction from the game itself. And the elephant in the room, is that despite all these companies, all all this effort, this is still not a solved problem.
  • #14 Picture even more complex when you consider what’s inside those boxes. Once upon a time, teams had to build all this themselves. Now at least there are third party solutions, but still every one of those lines represents code the team needs to write That’s a huge distraction from the game itself. And the elephant in the room, is that despite all these companies, all all this effort, this is still not a solved problem.
  • #15 Picture even more complex when you consider what’s inside those boxes. Once upon a time, teams had to build all this themselves. Now at least there are third party solutions, but still every one of those lines represents code the team needs to write That’s a huge distraction from the game itself. And the elephant in the room, is that despite all these companies, all all this effort, this is still not a solved problem.
  • #16 Picture even more complex when you consider what’s inside those boxes. Once upon a time, teams had to build all this themselves. Now at least there are third party solutions, but still every one of those lines represents code the team needs to write That’s a huge distraction from the game itself. And the elephant in the room, is that despite all these companies, all all this effort, this is still not a solved problem.
  • #17 Picture even more complex when you consider what’s inside those boxes. Once upon a time, teams had to build all this themselves. Now at least there are third party solutions, but still every one of those lines represents code the team needs to write That’s a huge distraction from the game itself. And the elephant in the room, is that despite all these companies, all all this effort, this is still not a solved problem.
  • #18 Picture even more complex when you consider what’s inside those boxes. Once upon a time, teams had to build all this themselves. Now at least there are third party solutions, but still every one of those lines represents code the team needs to write That’s a huge distraction from the game itself. And the elephant in the room, is that despite all these companies, all all this effort, this is still not a solved problem.
  • #23 For one thing,
  • #25 Registered Users, Downloads, Page Views, etc. – they don’t tell a complete story. Need more detail on usage to get to value.
  • #28 For example…. Let’s look at the sort of data that IS actionable. Let’s say you have a retention problem. You know you are losing players. So you start by asking questions. WHERE am I losing players? HOW are the players who I lose different from the players i keep So you might start with a graph like this... Break down your game by level. And look level by level where players are dropping out. So right away, you can see why this sort of graph is actionable. Based on this data, what might I look at making changes? Clearly something is up with level 1…. Players are pooling there, and dropping out. They’re not continuing on. So how are the players who continue different from players who pool? For that sort of question, it’s useful to go “back in time”. Take two players – one who quit, one who stayed. How are they different early on? Are there patterns in common?
  • #29 Based on that sort of “what if” questions and investigation, you can start to form hypotheses. Try to guess what’s happening in your game. For example, you might decide that earning an achievement quickly is important. Sometimes these hypothesis will be common sense. If the game is too hard, and players keep dying, they’ll quit. Sometimes they’re not at all intuitive. At PopCap, with Plants vs. Zombies, we had a huge problem. New players kept running out of energy and quitting. Experienced players familiar with RTS games never did. When we investigated, we found it was because new players were planting too many shooters and not enough sunflowers. Experienced players knew to first plant lots of sunflowers. We did a lot of brainstorming, and in the end did something very smart. We priced sunflowers cheaper than shooters. So newbies would plant lots of sunflowers since that was the only thing they COULD plant. That was a very clever idea.
  • #31 Then… once you’ve tried your changes, you need to go back and compare the new results to where you started. Did you help? Did you fix things or make them worse?
  • #41 But first, it’s worth noting that alll events have two components… the “fiction” of the event, and the “mechanic” of the event The mechanic is what’s actually changing The fiction is how you position it to the player The same mechanic can be positioned in many different ways – and I’d argue the fiction is more important than the mechanic
  • #54 Pre-seed with one: What does integration look like? Will using PlayFab help me pass XRs? Will PlayFab be using more Azure features? Will PlayFab be using Azure ML? Yes.