Real time ads personalization @ Spotify

1,951 views

Published on

We talk about how Spotify is trying to solve the problem of ads personalization in near real-time to improve relevancy and engagement of ads.

0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,951
On SlideShare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
75
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Real time ads personalization @ Spotify

  1. 1. May 6th, 2014 Real-Time Personalization with Ad Tech @Spotify Kinshuk Mishra kinshuk@spotify.com @_kinshukmishra 1
  2. 2. About Me • @Spotify since 2011 • Data and Backend engineering • With the Monetization team
  3. 3. May 12, 2014 Spotify in numbers Started in 2006, available in 56 markets 20+ million songs, 20,000 added daily 24+ million active users, 6+ million subscribers 1.5 billion playlists
  4. 4. May 12, 2014 Monetization at Spotify • $1 billion paid to rights holders since launch • 1 in every 4 Spotify users is a paying subscriber • Ad revenue pays for free tier music streaming • Build platform to power artist and music label promotions
  5. 5. What is our mission? • Power the free tier with ads customers love. – Delight users, brands and artists with standout promotional experiences. – Deliver the right message for every moment. 5
  6. 6. Delight users, brands and artists with standout promotional experience 6 • Ad formats need to be - Native - Actionable - Scalable - Engaging
  7. 7. 7 Spotlight
  8. 8. 8 Album promotion
  9. 9. 9 Sponsored Genre
  10. 10. 10 Sponsored Genre
  11. 11. 11 !(Right message for every moment)
  12. 12. Things we had to do • Make ads context aware. • Create measurable ad formats. • Build tools to help brands understand their audience. • Build flexible targeting infrastructure. • Analyze user behavior to improve ad relevancy. 12
  13. 13. Why is real time interesting? • True assessment of the context - current view, last track, etc. • Immediate feedback - ad clicked, followed a playlist, etc. • Increased ad opportunities - hyperlocal, current mood, etc. 13 source : http://nativemobile.com/tag/real-time-bidding
  14. 14. How to serve right message for every moment ? • Fix the WTFs for ads in the music context - Realtime short window activity analysis • Improve ad relevancy based on user behavior - (Realtime + Batch) long window activity analysis 14
  15. 15. Some quick wins • Fix the WTFs for ads in the music context - Fix the “Not Safe For Kids” ads problem. - Fix the relevancy of ads in your music session. • Improve ad relevancy based on user behavior - Fix ad relevancy based on user’s musical taste. 15
  16. 16. Targeting Architecture 16
  17. 17. Desired overall design 17
  18. 18. Our requirements • Scalable log collection • Capability to process logs in batch and realtime mode and aggregate user activity • Capability to store user profiles and enable serve time lookups • Ability to update and add new features to existing user profiles
  19. 19. Bigdata @Spotify and choices • 700 node hadoop cluster • 400 GB service logs daily • 4.5 TB user data daily • 7500 hadoop jobs daily • 64 TB data generated daily • YARN Map-Reduce, Giraph, Storm, Spark, etc. ! 19
  20. 20. Overall design and tech choices 20
  21. 21. Source : http://tfosuccess.com/day-106-107-the-storm-arrived/ What is Storm? • Real time stream processing • Like Hadoop without HDFS • Like Map/Reduce with many reducer steps • Fault tolerant and guaranteed message processing 21
  22. 22. Storm @Spotify • storm-0.8.0 • 22 node cluster • 15+ topologies • 200k tuples/second • ads, recommendation, analytics, monitoring, etc. 22 source: http://storm.incubator.apache.org/
  23. 23. Use-case Use real-time session genre information to control ad serving in real-time 23
  24. 24. Realtime activity analysis • Kafka -> Storm -> Memcached -> Targeting Backend • Soft realtime • Easy to scale 24
  25. 25. Getting data 25
  26. 26. Getting data across the globe 26
  27. 27. Genre Topology 27
  28. 28. May 12, 2014 Measuring effectiveness • Build ad quality scores from lower-level metrics • Positive response, avoidance, etc. • A/B test hypotheses, evaluate against quality scores • Build quality score optimization into real-time system
  29. 29. May 6th, 2014 Want to join the band? https://www.spotify.com/us/jobs/ or https://twitter.com/Spotifyjobs ! Kinshuk Mishra kinshuk@spotify.com @_kinshukmishra

×