LIVE OPS AS A SERVICE
Scott Humphries
Head of Product Development
Scott Humphries
Head of Product Development
Creativity, Professionalism and Data-Driven
Decision Making Are Our Core Values
Full Development
Co-Development
Platform Conversion
Development Services
Live Ops
Game Development & Studio Services Agency
Inside Out: Thought Bubbles (Disney)
Increasing engagement through level design
Gods of Rome (Gameloft)
Overhauling KPI’s through merchandizing and sales
Q&A
Key	Stats
41%
created
20%
optimized
516 levels created 250 levels optimized
From the total of 1250 levels we have
Live Ops Development services for Disney since August 2016
20 new levels every 2 weeks
10 tuned levels every 2 weeks
Daily Challenges
Seasonal Events
Legacy Bug Fixes
Client Updates
Genre: Bubble Shooter
Launched: March 12, 2015
Installs: 20M+
Partner: Disney
Level	Design
When designing new levels we start
from mapping the difficulty of the
levels we’re going to create.
We measure difficulty in average
number of attempts before the user
completes (Wins) a level.
1
Difficulty
16
6
12
2
10
18
2
6
10
3
20
1
5
10
14
5
2
8
10
4
1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092
Target Difficulty
Build difficulty through a series of mountains and valleys. Make sure
to slowly build up towards harder levels, and add easy levels after
hitting a peak to help reduce player stress.
Create an exhaustive list of objectives,
mechanics and features.
A rational design approach will help
create levels that are easily variable
and trackable.
2
Variety
New mechanics create emergent gameplay through level design.
Level
Number
Target
Difficulty
Game
Mode
Ingredients
Target
StarsFrz Blk Hid Can Chg Hrz Vac Bmp Flt Prx Spy Cdt Dua Lck Bag Scr Ins Jan Tdg Brl
1073 16 CLR x x x 3
1074 6 TAR x x x 2
1075 12 FRE x x x x 3
1076 2 DEF x 1
1077 10 CLR x x x 2
1078 18 TAR x x x 3
1079 2 FRE x x x 1
1080 6 CLR x x x x 2
1081 10 TAR x x x x 2
1082 3 FRE x x x 1
1083 20 CLR x x x 3
1084 1 TAR x x x 1
1085 5 DEF x 2
1086 10 FRE x x x 2
1087 14 TAR x x x 3
1088 5 CLR x x x 2
1089 2 TAR x x x 1
1090 8 FRE x x x 2
1091 10 CLR x x x 2
1092 4 TAR x x x 1
Level	Design
We review the performance of our
levels and create a plan based on
what data tells us.
Our main indicators are difficulty,
churn rate, ARPU and level
completion rate on fail
3
Monitor KPIs
16
6
12
2
10
18
2
6
10
3
20
1
5
10
14
5
2
8
10
4
16
14
8
2
10
20
4
8
12
3
10
2
8
12
14
3
4
7
16
4
1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092
Actual vs. Target Difficulty
Levels that are off-target by a significant margin are extracted and
the ARPU, churn and completion rate on fail are examined.
In the chart above, we will take a look at level 1074, 1083 and 1091.
Level	Design
The Product Manager examines the
off-levels in-depth, calculating the
churn vs. ARPU ratio and works closely
with the designers to make
appropriate changes.
4
Optimize
Level
Number
Target
Difficulty
Actual
Difficulty
Churn
Rate
ARPU
Churn	vs.	
ARPU
Decision
1074 6 14 1.20% 0.5 24 Decrease	level	difficulty
1083 20 10 0.20% 0.1 20 Increase	level	difficulty
1091 10 16 0.30% 0.5 6 Leave	as	it	is
16
6
12
2
10
18
2
6
10
3
20
1
5
10
14
5
2
8
10
4
16
14
8
2
10
20
4
8
12
3
10
2
8
12
14
3
4
7
16
4
1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092
Actual vs. Target Difficulty
Level	Design
#1: Interesting gameplay and new mechanics drive the best engagement
and progression.
#2: If there is high churn on easy levels (Success Churn), the game
mechanic or the combination of ingredients used are not appealing for
your users.
#3: Convey the feeling of “I almost won!” a level instead of “I lost badly…”.
Measure completion rate on level fail and tweak it to be over 85%.
#4: Every game is different! Track the skill of your players and create
means of tailoring content based on their skill level.
Other	Considerations	in	Level	Design
Genre: Fighting game
Launched: December 16, 2015
Installs: 27M+
Partner: Gameloft
Live Ops takeover from Gameloft since June 2017,
New Heroes
Store optimizations
Hero Collection
New Story Missions
Daily Activities
Incentivized Ads
SDK Updates
Legacy Bugfixes
Key	Stats
Tell the user what they are getting
BEFORE AFTER
Merchandizing
Not Enough Energy
71%
Rewarded video grants the player energy for watching an
ad and drives additional revenue + ARPDAU
Merchandizing
We A/B test different promotions and bundles with different pricepoints.
We cut underperforming ”deals” and focused on those that were doing
great, providing our players something they consider to be valuable –
items that are otherwise hard to obtain.
High value = quality content at reasonable cost
OPTIMIZE
Conversion, Revenue
ANALYZE
A/B Test for appeal
IMPLEMENT
Bundles, Prices, Segments
Promotions	/	Events
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Revenue Distribution – Without Mid Week Sale
Sales generated revenue spikes during week-
ends
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Revenue Distribution - With Mid Week Sale
Mid-Week time limited events brought additional
revenue without impacting weekend sales
Promotions/Events
We have an engaged audience in Asia.
Taking this into consideration, the
second hero we released: Kumiho (the
nine tails fox) – was inspired by Chinese
legends.
Kumiho generated 20% more revenue
compared to other heroes. First
purchase ever for 51% of the payers
during her promotion.
Know	Your	Audience
Daily Activities
This feature offers players diverse rewards for
completing various, fresh activities daily.
Hero Collection Menu
Show number of heroes missing from player collection. Also the
Coming Soon section lets players know that there is still content to
be added down the road.
Double XP Event
We aim to increase session times with this feature, offering players
bonus experience and discounts for hero upgrades.
These features combined have increased the
average time per session 10%
Engagement
As a service provider:
• Partners can move their teams on projects that are more important from a strategic standpoint
• We gain experience from dealing with diverse projects, genres and audiences
Full Development
Co-Development
Platform Conversion
Development Services
Live Ops
Game Development & Studio Services Agency
Thanks!
Any	questions?

LiveOps as a Service | Scott Humphries

  • 1.
    LIVE OPS ASA SERVICE Scott Humphries Head of Product Development
  • 2.
    Scott Humphries Head ofProduct Development
  • 3.
    Creativity, Professionalism andData-Driven Decision Making Are Our Core Values Full Development Co-Development Platform Conversion Development Services Live Ops Game Development & Studio Services Agency
  • 5.
    Inside Out: ThoughtBubbles (Disney) Increasing engagement through level design Gods of Rome (Gameloft) Overhauling KPI’s through merchandizing and sales Q&A
  • 7.
    Key Stats 41% created 20% optimized 516 levels created250 levels optimized From the total of 1250 levels we have Live Ops Development services for Disney since August 2016 20 new levels every 2 weeks 10 tuned levels every 2 weeks Daily Challenges Seasonal Events Legacy Bug Fixes Client Updates Genre: Bubble Shooter Launched: March 12, 2015 Installs: 20M+ Partner: Disney
  • 8.
    Level Design When designing newlevels we start from mapping the difficulty of the levels we’re going to create. We measure difficulty in average number of attempts before the user completes (Wins) a level. 1 Difficulty 16 6 12 2 10 18 2 6 10 3 20 1 5 10 14 5 2 8 10 4 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 Target Difficulty Build difficulty through a series of mountains and valleys. Make sure to slowly build up towards harder levels, and add easy levels after hitting a peak to help reduce player stress.
  • 9.
    Create an exhaustivelist of objectives, mechanics and features. A rational design approach will help create levels that are easily variable and trackable. 2 Variety New mechanics create emergent gameplay through level design. Level Number Target Difficulty Game Mode Ingredients Target StarsFrz Blk Hid Can Chg Hrz Vac Bmp Flt Prx Spy Cdt Dua Lck Bag Scr Ins Jan Tdg Brl 1073 16 CLR x x x 3 1074 6 TAR x x x 2 1075 12 FRE x x x x 3 1076 2 DEF x 1 1077 10 CLR x x x 2 1078 18 TAR x x x 3 1079 2 FRE x x x 1 1080 6 CLR x x x x 2 1081 10 TAR x x x x 2 1082 3 FRE x x x 1 1083 20 CLR x x x 3 1084 1 TAR x x x 1 1085 5 DEF x 2 1086 10 FRE x x x 2 1087 14 TAR x x x 3 1088 5 CLR x x x 2 1089 2 TAR x x x 1 1090 8 FRE x x x 2 1091 10 CLR x x x 2 1092 4 TAR x x x 1 Level Design
  • 10.
    We review theperformance of our levels and create a plan based on what data tells us. Our main indicators are difficulty, churn rate, ARPU and level completion rate on fail 3 Monitor KPIs 16 6 12 2 10 18 2 6 10 3 20 1 5 10 14 5 2 8 10 4 16 14 8 2 10 20 4 8 12 3 10 2 8 12 14 3 4 7 16 4 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 Actual vs. Target Difficulty Levels that are off-target by a significant margin are extracted and the ARPU, churn and completion rate on fail are examined. In the chart above, we will take a look at level 1074, 1083 and 1091. Level Design
  • 11.
    The Product Managerexamines the off-levels in-depth, calculating the churn vs. ARPU ratio and works closely with the designers to make appropriate changes. 4 Optimize Level Number Target Difficulty Actual Difficulty Churn Rate ARPU Churn vs. ARPU Decision 1074 6 14 1.20% 0.5 24 Decrease level difficulty 1083 20 10 0.20% 0.1 20 Increase level difficulty 1091 10 16 0.30% 0.5 6 Leave as it is 16 6 12 2 10 18 2 6 10 3 20 1 5 10 14 5 2 8 10 4 16 14 8 2 10 20 4 8 12 3 10 2 8 12 14 3 4 7 16 4 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 Actual vs. Target Difficulty Level Design
  • 12.
    #1: Interesting gameplayand new mechanics drive the best engagement and progression. #2: If there is high churn on easy levels (Success Churn), the game mechanic or the combination of ingredients used are not appealing for your users. #3: Convey the feeling of “I almost won!” a level instead of “I lost badly…”. Measure completion rate on level fail and tweak it to be over 85%. #4: Every game is different! Track the skill of your players and create means of tailoring content based on their skill level. Other Considerations in Level Design
  • 14.
    Genre: Fighting game Launched:December 16, 2015 Installs: 27M+ Partner: Gameloft Live Ops takeover from Gameloft since June 2017, New Heroes Store optimizations Hero Collection New Story Missions Daily Activities Incentivized Ads SDK Updates Legacy Bugfixes Key Stats
  • 15.
    Tell the userwhat they are getting BEFORE AFTER Merchandizing
  • 16.
    Not Enough Energy 71% Rewardedvideo grants the player energy for watching an ad and drives additional revenue + ARPDAU Merchandizing
  • 17.
    We A/B testdifferent promotions and bundles with different pricepoints. We cut underperforming ”deals” and focused on those that were doing great, providing our players something they consider to be valuable – items that are otherwise hard to obtain. High value = quality content at reasonable cost OPTIMIZE Conversion, Revenue ANALYZE A/B Test for appeal IMPLEMENT Bundles, Prices, Segments Promotions / Events
  • 18.
    1 2 34 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Revenue Distribution – Without Mid Week Sale Sales generated revenue spikes during week- ends 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Revenue Distribution - With Mid Week Sale Mid-Week time limited events brought additional revenue without impacting weekend sales Promotions/Events
  • 19.
    We have anengaged audience in Asia. Taking this into consideration, the second hero we released: Kumiho (the nine tails fox) – was inspired by Chinese legends. Kumiho generated 20% more revenue compared to other heroes. First purchase ever for 51% of the payers during her promotion. Know Your Audience
  • 20.
    Daily Activities This featureoffers players diverse rewards for completing various, fresh activities daily. Hero Collection Menu Show number of heroes missing from player collection. Also the Coming Soon section lets players know that there is still content to be added down the road. Double XP Event We aim to increase session times with this feature, offering players bonus experience and discounts for hero upgrades. These features combined have increased the average time per session 10% Engagement
  • 22.
    As a serviceprovider: • Partners can move their teams on projects that are more important from a strategic standpoint • We gain experience from dealing with diverse projects, genres and audiences Full Development Co-Development Platform Conversion Development Services Live Ops Game Development & Studio Services Agency
  • 23.