Your SlideShare is downloading. ×
  • Like
  • Save
Managing Experiment at Spotify
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Managing Experiment at Spotify

  • 7,492 views
Published

Spotify strives for team autonomy and independence. This means that no team should be blocked by others and they should be able to move as fast as they can. The autonomy has is a challenge for …

Spotify strives for team autonomy and independence. This means that no team should be blocked by others and they should be able to move as fast as they can. The autonomy has is a challenge for managing a centralised and coordinated experimentation infrastructure and analysis. This a talk about how we approach experimentation alignment in a fast moving company.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
7,492
On SlideShare
0
From Embeds
0
Number of Embeds
8

Actions

Shares
Downloads
4
Comments
0
Likes
118

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. May 20, 2014 Managing experiments at Spotify Ali Sarrafi Product Owner A/B testing and Experimentation
  • 2. May 20, 2014 The right music for every moment 6
  • 3. May 20, 2014 7 24M active users. 6M paying subscribers. 20Msongs.
  • 4. Tribes Provide fast and reliable access to all the world's music Enable high product development speed while maintaining a highly available service P P P P Squad Squad Squad Squad Chapter Chapter P P P P Squad Squad Squad Squad Chapter Chapter Our team is growing rapidly 250 30 100 10
  • 5. 9
  • 6. 10 Autonomous fast moving teams Feature 1 Feature 3 Feature 2 Feature 4 Feature 5
  • 7. May 20, 2014 Some Statistics Over 40 autonomous teams working on features Over 7 platforms with multiple features Over 3000 source repositories
  • 8. 12 We don’t want a Frankenstein software
  • 9. 13 Well at least a good looking 2014 edition
  • 10. Section name 14 How can we align teams’ optimisation to avoid the frankenstein effect?
  • 11. 15 Align on metrics rather than actual tests
  • 12. 16 DAU (Daily Active Users) MAU (Monthly Active Users)
  • 13. Section name 17 Evolving Metrics
  • 14. May 20, 2014 Focus on functionality as a metric
  • 15. 19 DAU/MAU Function 1 Function 2 Function 3 Function 4 CTR/Team Metrics Importance (High to low) Order of optimisation
  • 16. Section name 20 Now we know what would work how do we make sure everyone uses it?
  • 17. 21 It’s all about automation
  • 18. 22 and of course education
  • 19. Section name 23
  • 20. 24 Make magic happen with desired metrics Vital Sign Group A Group B Functionality 1 +1.5% -0.5% Functionality 2 +2.5% -3% Functionality 3 +0.5% 0% Functionality 4 +1% -7%
  • 21. 25 People/Infrastructure as a service Feature 1 Feature 5 Feature 4 Feature 2 Feature 3
  • 22. Summary • Metric alignment is the most effective way of making sure the overall experience does not suffer • Education and Automation are key for adoption • People as a service is a most effective way of reaching out to teams.
  • 23. May 20, 2014 Copyright stuff Napster pic by bixentro, cc 2.0 http://www.flickr.com/photos/bixentro/ Busy record store pic by gomattolson http://www.flickr.com/photos/gomattolson Empty record store pic by passetti http://www.flickr.com/photos/passetti/ First Frankenstein screenshot form : “Frankenstein” movie by Kenneth Branagh, 1994 Second Frankenstein screenshot form : “I, Frankenstein” movie by Stuart Beattie, 2014
  • 24. May 20, 2014 Thanks @asarrafi