SlideShare a Scribd company logo
Rovio © 2017 Confidential. All rights reserved.
BUILD WITH THE PLAYERS
Sophie Vo
The timed levels
"Timed levels are fun to play!”
“With the time pressure, players
are more likely to convert”
Our assumptions:
"They're not relaxing and I play Angry
Birds Pop to wind down and relax.”
"I love playing AB pop to relax and the timed
levels stress me out.”
"I have multiple sclerosis. Muscles control is problematic I find these levels
very difficult...
...I have to wait until my daughter comes to do it for me. This can be
weeks as she lives a long way from me.”
Top engaged and spenders
What if we actually talk to the
players!
"Gimme more money!”
"The levels are too hard, can we make them
more easy?”
What POP players want...
"More lives”
"I don’t have friends to play with in the game”
"More EVERYTHING!”
"AND FOR FREE :D”
Players don’t know what they want!
In-game surveys
Supervised playtesting
Observations
BEHAVIORAL:
What people do
ATTITUDINAL:
What people say
QUALITATIVE
WHy? HOW TO FIX ?
QUANTITATIVE
HOW MANY / HOW MUCH
Unsupervised playtesting
Game analytics
Behavioral statistics
A/B testing
User interviews
- pair/group interviews
Focus groups
Diary studies
Customer
feedback
Interviews
When do I use qualitative?
1. SELECT FEATURE
2. ITERATE
User tests
Hungry Jelly - new objective
● User tests (LDs, artists, PM)
● Internal playtest
3. VERIFY &
ADJUST
In-game surveys
"Too difficult for the reward
offered."
”I liked it, but I didn't understand the
benefit or purpose.”
Piggy Puzzle Event
Building a new game...
Concept Prototype 1 Prototype 2 PrePrototype 3
If there’s one thing...
“I know that I don’t know”

More Related Content

Recently uploaded

The Unbelievable Tale of Dwayne Johnson Kidnapping: A Riveting Saga
The Unbelievable Tale of Dwayne Johnson Kidnapping: A Riveting SagaThe Unbelievable Tale of Dwayne Johnson Kidnapping: A Riveting Saga
The Unbelievable Tale of Dwayne Johnson Kidnapping: A Riveting Saga
greendigital
 
University of Western Sydney degree offer diploma Transcript
University of Western Sydney degree offer diploma TranscriptUniversity of Western Sydney degree offer diploma Transcript
University of Western Sydney degree offer diploma Transcript
soxrziqu
 
Everything You Need to Know About IPTV Ireland.pdf
Everything You Need to Know About IPTV Ireland.pdfEverything You Need to Know About IPTV Ireland.pdf
Everything You Need to Know About IPTV Ireland.pdf
Xtreame HDTV
 
Divertidamente SLIDE muito lindo e criativo, pptx
Divertidamente SLIDE muito lindo e criativo, pptxDivertidamente SLIDE muito lindo e criativo, pptx
Divertidamente SLIDE muito lindo e criativo, pptx
lunaemel03
 
Top IPTV UK Providers of A Comprehensive Review.pdf
Top IPTV UK Providers of A Comprehensive Review.pdfTop IPTV UK Providers of A Comprehensive Review.pdf
Top IPTV UK Providers of A Comprehensive Review.pdf
Xtreame HDTV
 
高仿(nyu毕业证书)美国纽约大学毕业证文凭毕业证原版一模一样
高仿(nyu毕业证书)美国纽约大学毕业证文凭毕业证原版一模一样高仿(nyu毕业证书)美国纽约大学毕业证文凭毕业证原版一模一样
高仿(nyu毕业证书)美国纽约大学毕业证文凭毕业证原版一模一样
9u08k0x
 
Emcee Profile_ Subbu from Bangalore .pdf
Emcee Profile_ Subbu from Bangalore .pdfEmcee Profile_ Subbu from Bangalore .pdf
Emcee Profile_ Subbu from Bangalore .pdf
subran
 
Orpah Winfrey Dwayne Johnson: Titans of Influence and Inspiration
Orpah Winfrey Dwayne Johnson: Titans of Influence and InspirationOrpah Winfrey Dwayne Johnson: Titans of Influence and Inspiration
Orpah Winfrey Dwayne Johnson: Titans of Influence and Inspiration
greendigital
 
Christian Louboutin: Innovating with Red Soles
Christian Louboutin: Innovating with Red SolesChristian Louboutin: Innovating with Red Soles
Christian Louboutin: Innovating with Red Soles
get joys
 
原版制作(Mercer毕业证书)摩斯大学毕业证在读证明一模一样
原版制作(Mercer毕业证书)摩斯大学毕业证在读证明一模一样原版制作(Mercer毕业证书)摩斯大学毕业证在读证明一模一样
原版制作(Mercer毕业证书)摩斯大学毕业证在读证明一模一样
mul1kv5w
 
Modern Radio Frequency Access Control Systems: The Key to Efficiency and Safety
Modern Radio Frequency Access Control Systems: The Key to Efficiency and SafetyModern Radio Frequency Access Control Systems: The Key to Efficiency and Safety
Modern Radio Frequency Access Control Systems: The Key to Efficiency and Safety
AITIX LLC
 
From Swing Music to Big Band Fame_ 5 Iconic Artists.pptx
From Swing Music to Big Band Fame_ 5 Iconic Artists.pptxFrom Swing Music to Big Band Fame_ 5 Iconic Artists.pptx
From Swing Music to Big Band Fame_ 5 Iconic Artists.pptx
Swing Street Radio
 
The Gallery of Shadows, In the heart of a bustling city
The Gallery of Shadows, In the heart of a bustling cityThe Gallery of Shadows, In the heart of a bustling city
The Gallery of Shadows, In the heart of a bustling city
John Emmett
 
Leonardo DiCaprio House: A Journey Through His Extravagant Real Estate Portfolio
Leonardo DiCaprio House: A Journey Through His Extravagant Real Estate PortfolioLeonardo DiCaprio House: A Journey Through His Extravagant Real Estate Portfolio
Leonardo DiCaprio House: A Journey Through His Extravagant Real Estate Portfolio
greendigital
 
Snoopy boards the big bow wow musical __
Snoopy boards the big bow wow musical __Snoopy boards the big bow wow musical __
Snoopy boards the big bow wow musical __
catcabrera
 
The Enigmatic Portrait, In the heart of a sleepy town
The Enigmatic Portrait, In the heart of a sleepy townThe Enigmatic Portrait, In the heart of a sleepy town
The Enigmatic Portrait, In the heart of a sleepy town
John Emmett
 
From Teacher to OnlyFans: Brianna Coppage's Story at 28
From Teacher to OnlyFans: Brianna Coppage's Story at 28From Teacher to OnlyFans: Brianna Coppage's Story at 28
From Teacher to OnlyFans: Brianna Coppage's Story at 28
get joys
 
Barbie Movie Review - The Astras.pdfffff
Barbie Movie Review - The Astras.pdfffffBarbie Movie Review - The Astras.pdfffff
Barbie Movie Review - The Astras.pdfffff
theastras43
 
DIGIDEVTV A New area of OTT Distribution
DIGIDEVTV  A New area of OTT DistributionDIGIDEVTV  A New area of OTT Distribution
DIGIDEVTV A New area of OTT Distribution
joeqsm
 
Unveiling Paul Haggis Shaping Cinema Through Diversity. .pdf
Unveiling Paul Haggis Shaping Cinema Through Diversity. .pdfUnveiling Paul Haggis Shaping Cinema Through Diversity. .pdf
Unveiling Paul Haggis Shaping Cinema Through Diversity. .pdf
kenid14983
 

Recently uploaded (20)

The Unbelievable Tale of Dwayne Johnson Kidnapping: A Riveting Saga
The Unbelievable Tale of Dwayne Johnson Kidnapping: A Riveting SagaThe Unbelievable Tale of Dwayne Johnson Kidnapping: A Riveting Saga
The Unbelievable Tale of Dwayne Johnson Kidnapping: A Riveting Saga
 
University of Western Sydney degree offer diploma Transcript
University of Western Sydney degree offer diploma TranscriptUniversity of Western Sydney degree offer diploma Transcript
University of Western Sydney degree offer diploma Transcript
 
Everything You Need to Know About IPTV Ireland.pdf
Everything You Need to Know About IPTV Ireland.pdfEverything You Need to Know About IPTV Ireland.pdf
Everything You Need to Know About IPTV Ireland.pdf
 
Divertidamente SLIDE muito lindo e criativo, pptx
Divertidamente SLIDE muito lindo e criativo, pptxDivertidamente SLIDE muito lindo e criativo, pptx
Divertidamente SLIDE muito lindo e criativo, pptx
 
Top IPTV UK Providers of A Comprehensive Review.pdf
Top IPTV UK Providers of A Comprehensive Review.pdfTop IPTV UK Providers of A Comprehensive Review.pdf
Top IPTV UK Providers of A Comprehensive Review.pdf
 
高仿(nyu毕业证书)美国纽约大学毕业证文凭毕业证原版一模一样
高仿(nyu毕业证书)美国纽约大学毕业证文凭毕业证原版一模一样高仿(nyu毕业证书)美国纽约大学毕业证文凭毕业证原版一模一样
高仿(nyu毕业证书)美国纽约大学毕业证文凭毕业证原版一模一样
 
Emcee Profile_ Subbu from Bangalore .pdf
Emcee Profile_ Subbu from Bangalore .pdfEmcee Profile_ Subbu from Bangalore .pdf
Emcee Profile_ Subbu from Bangalore .pdf
 
Orpah Winfrey Dwayne Johnson: Titans of Influence and Inspiration
Orpah Winfrey Dwayne Johnson: Titans of Influence and InspirationOrpah Winfrey Dwayne Johnson: Titans of Influence and Inspiration
Orpah Winfrey Dwayne Johnson: Titans of Influence and Inspiration
 
Christian Louboutin: Innovating with Red Soles
Christian Louboutin: Innovating with Red SolesChristian Louboutin: Innovating with Red Soles
Christian Louboutin: Innovating with Red Soles
 
原版制作(Mercer毕业证书)摩斯大学毕业证在读证明一模一样
原版制作(Mercer毕业证书)摩斯大学毕业证在读证明一模一样原版制作(Mercer毕业证书)摩斯大学毕业证在读证明一模一样
原版制作(Mercer毕业证书)摩斯大学毕业证在读证明一模一样
 
Modern Radio Frequency Access Control Systems: The Key to Efficiency and Safety
Modern Radio Frequency Access Control Systems: The Key to Efficiency and SafetyModern Radio Frequency Access Control Systems: The Key to Efficiency and Safety
Modern Radio Frequency Access Control Systems: The Key to Efficiency and Safety
 
From Swing Music to Big Band Fame_ 5 Iconic Artists.pptx
From Swing Music to Big Band Fame_ 5 Iconic Artists.pptxFrom Swing Music to Big Band Fame_ 5 Iconic Artists.pptx
From Swing Music to Big Band Fame_ 5 Iconic Artists.pptx
 
The Gallery of Shadows, In the heart of a bustling city
The Gallery of Shadows, In the heart of a bustling cityThe Gallery of Shadows, In the heart of a bustling city
The Gallery of Shadows, In the heart of a bustling city
 
Leonardo DiCaprio House: A Journey Through His Extravagant Real Estate Portfolio
Leonardo DiCaprio House: A Journey Through His Extravagant Real Estate PortfolioLeonardo DiCaprio House: A Journey Through His Extravagant Real Estate Portfolio
Leonardo DiCaprio House: A Journey Through His Extravagant Real Estate Portfolio
 
Snoopy boards the big bow wow musical __
Snoopy boards the big bow wow musical __Snoopy boards the big bow wow musical __
Snoopy boards the big bow wow musical __
 
The Enigmatic Portrait, In the heart of a sleepy town
The Enigmatic Portrait, In the heart of a sleepy townThe Enigmatic Portrait, In the heart of a sleepy town
The Enigmatic Portrait, In the heart of a sleepy town
 
From Teacher to OnlyFans: Brianna Coppage's Story at 28
From Teacher to OnlyFans: Brianna Coppage's Story at 28From Teacher to OnlyFans: Brianna Coppage's Story at 28
From Teacher to OnlyFans: Brianna Coppage's Story at 28
 
Barbie Movie Review - The Astras.pdfffff
Barbie Movie Review - The Astras.pdfffffBarbie Movie Review - The Astras.pdfffff
Barbie Movie Review - The Astras.pdfffff
 
DIGIDEVTV A New area of OTT Distribution
DIGIDEVTV  A New area of OTT DistributionDIGIDEVTV  A New area of OTT Distribution
DIGIDEVTV A New area of OTT Distribution
 
Unveiling Paul Haggis Shaping Cinema Through Diversity. .pdf
Unveiling Paul Haggis Shaping Cinema Through Diversity. .pdfUnveiling Paul Haggis Shaping Cinema Through Diversity. .pdf
Unveiling Paul Haggis Shaping Cinema Through Diversity. .pdf
 

Featured

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
Christy Abraham Joy
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
Vit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
MindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
RachelPearson36
 

Featured (20)

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 

Design with the Players

  • 1. Rovio © 2017 Confidential. All rights reserved. BUILD WITH THE PLAYERS Sophie Vo
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11. The timed levels "Timed levels are fun to play!” “With the time pressure, players are more likely to convert” Our assumptions:
  • 12.
  • 13.
  • 14. "They're not relaxing and I play Angry Birds Pop to wind down and relax.” "I love playing AB pop to relax and the timed levels stress me out.” "I have multiple sclerosis. Muscles control is problematic I find these levels very difficult... ...I have to wait until my daughter comes to do it for me. This can be weeks as she lives a long way from me.”
  • 15.
  • 16.
  • 17.
  • 18. Top engaged and spenders
  • 19.
  • 20. What if we actually talk to the players!
  • 21. "Gimme more money!” "The levels are too hard, can we make them more easy?” What POP players want... "More lives” "I don’t have friends to play with in the game” "More EVERYTHING!” "AND FOR FREE :D”
  • 22.
  • 23.
  • 24. Players don’t know what they want!
  • 25. In-game surveys Supervised playtesting Observations BEHAVIORAL: What people do ATTITUDINAL: What people say QUALITATIVE WHy? HOW TO FIX ? QUANTITATIVE HOW MANY / HOW MUCH Unsupervised playtesting Game analytics Behavioral statistics A/B testing User interviews - pair/group interviews Focus groups Diary studies Customer feedback Interviews
  • 26. When do I use qualitative? 1. SELECT FEATURE
  • 27.
  • 29. User tests Hungry Jelly - new objective ● User tests (LDs, artists, PM) ● Internal playtest
  • 31. In-game surveys "Too difficult for the reward offered." ”I liked it, but I didn't understand the benefit or purpose.” Piggy Puzzle Event
  • 32. Building a new game... Concept Prototype 1 Prototype 2 PrePrototype 3
  • 33.
  • 34. If there’s one thing...
  • 35.
  • 36. “I know that I don’t know”

Editor's Notes

  1. The Secret… ... My Secret
  2. Joined Rovio and take over AB POP Live game since now almost 3 years
  3. I was prepared to take that mission, had a lot of xp in my career and was completely feeling prepared for the battlefield
  4. Worked in big companies before, such as GL 8+y xp Live Ops ++
  5. Also Wooga, very analytical driven company where I learnt a lot And which unlocked my potential as a Data freak. I was working with a lot of data over there, ab tests making projections and forecasts etc..
  6. All I had to do with that great experience and knowledge, was to execute my master plan
  7. To give an example Let me illustrate a case I stumbled on AB POP when joining, on with the timed levels in the game
  8. Reaction…. Not great impact on KPI, not a lot of excitement from players either
  9. Mindblowing and shocking revelation. I didn’t know about my players! For a data freak like me, a whole new world of possibility was open to me and that I could explore!
  10. I realized there was something i missed in the picture..
  11. Computing the data, it all made sense now!
  12. In a moment of crazy genius spark, I wondered… What if.. What if we.. what if we talk to the players!
  13. While I wish to fulfill player wishes The problem is that if we listen to all the player wishes without filtering a bit the information…
  14. This is a guarantee for a complete bankruptcy of the game haha
  15. PDCA cycle: I approach a lot the development of new features that way where based on assumption, we get the learning for the next iteration. You use both sources of data in the learning process FRAME ASSUMPTION: you use reviews and Live Ops feedbacks VERIFY: UTs, playtest, surveys
  16. PDCA cycle: I approach a lot the development of new features that way where based on assumption, we get the learning for the next iteration. You use both sources of data in the learning process FRAME ASSUMPTION: you use reviews and Live Ops feedbacks VERIFY: UTs, playtest, surveys
  17. We introduced a new game event where you could play one time limited level called “Piggy Puzzle” It was ab tested and we got the KPI results which were pretty mixed We wanted to understand WHY it didn't perform that well instead of making the next assumptions We got quantitative additional data how players liked it or saw it. Problem: reward not perceived as valuable for the effort. Not a good UX for the event introduction => next iteration Qualitative data on what could have been the problem
  18. Rework the message? Final flow
  19. Crazy random idea…
  20. We are Jon Snow of the game development!
  21. Adjust development process on the fact you don’t know.