How to develop a product leaving all decisions to make by the users? Through to make hypotheses, metrics, ab tests and data analysis, all decisions is on users. That presentation focus on how this process works on GloboPlay.

1. H Y P O T H E S E S , M E T R I C S , A / B T E S T A N D D ATA A N A LY S I S
Data Driven Product Development
J U L Y / 2 0 1 8 @ F I S L
A B

2. Software Engineer @ GloboPlay
MSc in Computer Science @ UFCG
M a r c o R o s n e r
A B

3. Status
Hypotheses
Metrics
A/B test
Data analysis
Conclusions
What is the next step?
Agenda
A B

4. Status

5. Building a culture
A B

6. Growth time
A B

7. ~70 AB tests in 2017
A B

8. Building knowledge
A B

9. Hypotheses

10. What is the problem?

11. Which are the solutions?

12. What is a good sample size?

13. Document it!

14. Metrics

15. One Metric That Matters

16. Supporting metrics

17. How much time is needed?

18. Document it!

19. Keep watching

20. A/B test

21. A B
What is an AB test?

22. AB test structure on Globo.com
A B

23. How to code an A/B test?
A B

24. H Y P O T H E S E S
If we focus on subscription when user
trying to watching videos, we believe that
we got more subscribers.
M E T R I C
Subscriptions and Conversion
D U R AT I O N
1 week
D E V I C E S
Onboarding Subscribers

25. Desktop Controle
S A M P L E : 8 4 %
A LT E R N AT I V E S

26. without-disclaimer
S A M P L E : 8 %
with-disclaimer
S A M P L E : 8 %
A LT E R N AT I V E S

27. H Y P O T H E S E S
If we focus on more information about the
video, we got more engagement with our
users
M E T R I C S
Video flow
D U R AT I O N
1 week
D E V I C E S
Changes on Video Stage

28. vs.
D E F A U L T N E W
A LT E R N AT I V E S

29. H Y P O T H E S E S
If we focus on show more videos on
playlist, we got more engagement with our
users
M E T R I C S
Video flow
D U R AT I O N
1 week
D E V I C E S
Changes on video playlist

30. D E F A U L T N E W
vs.
A LT E R N AT I V E S

31. Data analysis

32. Growth time

33. Notebook

34. Confidence index

35. Conclusions

36. Focus on user
A B

37. Building a Culture
A B

38. Building knowledge
A B

39. Understand your product
A B

40. Understand your users
A B

41. Understand your context
A B

42. What is the next step?

43. Data Driven Rebranding?

44. Questions?

45. talentos.globo.com

46. Marco Rosner
marco.rosner@corp.globo.com
@marcorosner