0
May 6th, 2014
Real-Time Personalization
with Ad Tech @Spotify
Kinshuk Mishra
kinshuk@spotify.com
@_kinshukmishra
1
About Me
• @Spotify since 2011
• Data and Backend engineering
• With the Monetization team
May 12, 2014
Spotify in numbers
Started in 2006, available in 56 markets
20+ million songs, 20,000 added daily
24+ million...
May 12, 2014
Monetization at Spotify
• $1 billion paid to rights holders since launch
• 1 in every 4 Spotify users is a pa...
What is our mission?
• Power the free tier with ads customers love.
– Delight users, brands and artists with standout
prom...
Delight users, brands and artists with standout
promotional experience
6
• Ad formats need to be
- Native
- Actionable
- S...
7
Spotlight
8
Album promotion
9
Sponsored Genre
10
Sponsored Genre
11
!(Right message for every moment)
Things we had to do
• Make ads context aware.
• Create measurable ad formats.
• Build tools to help brands understand thei...
Why is real time interesting?
• True assessment of the context
- current view, last track, etc.
• Immediate feedback
- ad ...
How to serve right message for every moment ?
• Fix the WTFs for ads in the music context
- Realtime short window activity...
Some quick wins
• Fix the WTFs for ads in the music context
- Fix the “Not Safe For Kids” ads problem.
- Fix the relevancy...
Targeting Architecture
16
Desired overall design
17
Our requirements
• Scalable log collection
• Capability to process logs in batch and
realtime mode and aggregate user acti...
Bigdata @Spotify and choices
• 700 node hadoop cluster
• 400 GB service logs daily
• 4.5 TB user data daily
• 7500 hadoop ...
Overall design and tech choices
20
Source : http://tfosuccess.com/day-106-107-the-storm-arrived/
What is Storm?
• Real time stream processing
• Like Hadoop w...
Storm @Spotify
• storm-0.8.0
• 22 node cluster
• 15+ topologies
• 200k tuples/second
• ads, recommendation,
analytics, mon...
Use-case
Use real-time session genre information to control 
ad serving in real-time
23
Realtime activity analysis
• Kafka -> Storm -> Memcached -> Targeting Backend
• Soft realtime
• Easy to scale
24
Getting data
25
Getting data across the globe
26
Genre Topology
27
May 12, 2014
Measuring effectiveness
• Build ad quality scores from lower-level metrics
• Positive response, avoidance, et...
May 6th, 2014
Want to join the band?
https://www.spotify.com/us/jobs/ or
https://twitter.com/Spotifyjobs
!
Kinshuk Mishra ...
Upcoming SlideShare
Loading in...5
×

Real time ads personalization @ Spotify

1,271

Published on

We talk about how Spotify is trying to solve the problem of ads personalization in near real-time to improve relevancy and engagement of ads.

0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,271
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
66
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "Real time ads personalization @ Spotify"

  1. 1. May 6th, 2014 Real-Time Personalization with Ad Tech @Spotify Kinshuk Mishra kinshuk@spotify.com @_kinshukmishra 1
  2. 2. About Me • @Spotify since 2011 • Data and Backend engineering • With the Monetization team
  3. 3. May 12, 2014 Spotify in numbers Started in 2006, available in 56 markets 20+ million songs, 20,000 added daily 24+ million active users, 6+ million subscribers 1.5 billion playlists
  4. 4. May 12, 2014 Monetization at Spotify • $1 billion paid to rights holders since launch • 1 in every 4 Spotify users is a paying subscriber • Ad revenue pays for free tier music streaming • Build platform to power artist and music label promotions
  5. 5. What is our mission? • Power the free tier with ads customers love. – Delight users, brands and artists with standout promotional experiences. – Deliver the right message for every moment. 5
  6. 6. Delight users, brands and artists with standout promotional experience 6 • Ad formats need to be - Native - Actionable - Scalable - Engaging
  7. 7. 7 Spotlight
  8. 8. 8 Album promotion
  9. 9. 9 Sponsored Genre
  10. 10. 10 Sponsored Genre
  11. 11. 11 !(Right message for every moment)
  12. 12. Things we had to do • Make ads context aware. • Create measurable ad formats. • Build tools to help brands understand their audience. • Build flexible targeting infrastructure. • Analyze user behavior to improve ad relevancy. 12
  13. 13. Why is real time interesting? • True assessment of the context - current view, last track, etc. • Immediate feedback - ad clicked, followed a playlist, etc. • Increased ad opportunities - hyperlocal, current mood, etc. 13 source : http://nativemobile.com/tag/real-time-bidding
  14. 14. How to serve right message for every moment ? • Fix the WTFs for ads in the music context - Realtime short window activity analysis • Improve ad relevancy based on user behavior - (Realtime + Batch) long window activity analysis 14
  15. 15. Some quick wins • Fix the WTFs for ads in the music context - Fix the “Not Safe For Kids” ads problem. - Fix the relevancy of ads in your music session. • Improve ad relevancy based on user behavior - Fix ad relevancy based on user’s musical taste. 15
  16. 16. Targeting Architecture 16
  17. 17. Desired overall design 17
  18. 18. Our requirements • Scalable log collection • Capability to process logs in batch and realtime mode and aggregate user activity • Capability to store user profiles and enable serve time lookups • Ability to update and add new features to existing user profiles
  19. 19. Bigdata @Spotify and choices • 700 node hadoop cluster • 400 GB service logs daily • 4.5 TB user data daily • 7500 hadoop jobs daily • 64 TB data generated daily • YARN Map-Reduce, Giraph, Storm, Spark, etc. ! 19
  20. 20. Overall design and tech choices 20
  21. 21. Source : http://tfosuccess.com/day-106-107-the-storm-arrived/ What is Storm? • Real time stream processing • Like Hadoop without HDFS • Like Map/Reduce with many reducer steps • Fault tolerant and guaranteed message processing 21
  22. 22. Storm @Spotify • storm-0.8.0 • 22 node cluster • 15+ topologies • 200k tuples/second • ads, recommendation, analytics, monitoring, etc. 22 source: http://storm.incubator.apache.org/
  23. 23. Use-case Use real-time session genre information to control ad serving in real-time 23
  24. 24. Realtime activity analysis • Kafka -> Storm -> Memcached -> Targeting Backend • Soft realtime • Easy to scale 24
  25. 25. Getting data 25
  26. 26. Getting data across the globe 26
  27. 27. Genre Topology 27
  28. 28. May 12, 2014 Measuring effectiveness • Build ad quality scores from lower-level metrics • Positive response, avoidance, etc. • A/B test hypotheses, evaluate against quality scores • Build quality score optimization into real-time system
  29. 29. May 6th, 2014 Want to join the band? https://www.spotify.com/us/jobs/ or https://twitter.com/Spotifyjobs ! Kinshuk Mishra kinshuk@spotify.com @_kinshukmishra
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×