Spotify engineers (Kinshuk Mishra and Noel Cody) share their experiences about building personalized ad experiences for users through iterative engineering and product development. The slide explains their process of continuous problem discovery, hypothesis generation, product development and experimentation. They deep dive into the specific ad personalization problems Spotify is solving and explain their data infrastructure technology stack in detail. They also explain how they've experimented various product hypothesis and iteratively evolved their infrastructure to keep up with the product requirements.

1. Ad Personalization
at Spotify
Building personalized ad experiences through iterative
engineering and product development
Kinshuk Mishra
Noel Cody

5. Music…
...is with you throughout the day.
...fits your mood.
...fits your activity.

6. Music…
...is personal.

9. If your day looks like this:
Wake up Work out Commute Focus at Work Relax at Home Sleep

10. Ads should follow.
Wake up Work out Commute Focus at Work Relax at Home Sleep
Ads

11. Ads should follow.
Wake up Work out Commute Focus at Work (Classical) Relax at Home Sleep
Electronic Music ad
Not bad. WTF?

12. Why Personalization?

13. Why Personalization?
“...it works well the advertisements are annoying though I am not a fan of
mainstream music so hearing about pop bands is also driving me crazy”
“Great way to listen to whatever music you want. The ads can be really
annoying though since they don't seem to be targeted. I HATE rap music, yet I
seem to get a lot of ads for it.”

14. Data confirms anecdotal evidence

17. AD PERSONALIZATION

18. User Stories
Hypotheses + Goals
Product MVPs + Experiments

19. User Stories to Hypotheses & Goals:
● Context-aware ads
● Music ads like music recommendations
● Ads that learn

20. Hypotheses to Products:
● Real-time genre targeting
● Historic genre targeting
● Real-time moment targeting

21. (Product MVPs to Experiments)
Control Variation 1 Variation 2

22. INFRASTRUCTURE

23. Ad Targeting Architecture
Feedback Loop

24. Ad Targeting Architecture
OSS Data
Infrastructure
Spotify Backend
Infrastructure

25. Ad Targeting Architecture V1.0
COTS Data
Infrastructure
Real-time Targeting
Spotify Backend
Infrastructure

26. Ad Targeting Architecture V2.0
Real-time + Batch Targeting
(a.k.a. Lambda Architecture)

27. Ad Targeting Architecture V2.5
Transition to Persistent User Profile

28. Ad Targeting Architecture V3.0
Richer Profile Schema with Persistence

29. Tech Choices

30. Kafka
● Kafka is a distributed, partitioned, replicated commit log service.
● Guarantees
● Kafka provides a total order over messages within a partition
● Fault tolerance : handles N-1 failures for replication factor N.

31. Ad Targeting Architecture V1.0
COTS Data
Infrastructure
Real-time Targeting
Spotify Backend
Infrastructure

32. SSttoorrmm
● Real time stream processing
● Like hadoop without HDFS
● Like Map/Reduce with many reducer steps
● Fault tolerant and guaranteed message processing

33. Storm: Testing (since 0.8.1)
Storm

34. Storm: Visualization (since 0.9.2)
Storm

35. Ad Targeting Architecture V2.0
Real-time + Batch Targeting

36. Apache Crunch
● Framework for writing, testing, and running MapReduce pipelines
● Pipelines are composed of user-defined functions and higher-level
abstractions of common MR tasks (filter, join, etc.)

37. Apache Crunch
Data structures:
● PCollection<T>
● PTable<K,V>
● PGroupedTable<K,V>
Functions:
● MapFn<T1,T2>: T1 → T2
● CombineFn<K,V>: (K, Iterable<V>) → (K, V)

38. Apache Crunch
What’s wrong with plain Python Streaming MapReduce?
● Testability
● Optimization
● Performance
● IDE support
● Type Safety
● Lack of higher-level operations (filter/join/aggregate)
From Spotify Presentation: Scalding the Crunchy Pig for Cascading into the Hive

39. Apache Crunch
● About a 5x performance improvement over Python streaming MapReduce
● Readable functional-style API in plain Java
● Great local testing support
● First-class support for Avro records.
From Spotify Presentation: Scalding the Crunchy Pig for Cascading into the Hive

40. Apache Crunch

41. Apache Crunch

42. Ad Targeting Architecture V2.5
Transition to Persistent User Profile

43. CASSANDRA
Rich wide-column
schema support
Solid persistence
and replication
Slower reads
MEMCACHED
K/V only
TTL is default (in-memory
● Rich schema
● Persistence
mgmt)
vs.

44. Ad Targeting Architecture V3.0
Richer Profile Schema with Persistence

45. DATA INGESTION:
CASSANDRA
CRUNCH
STORM
HDFS
KAFKA
LOGS

46. TESTING

48. User Stories
Hypotheses + Goals
Product MVPs + Experiments

49. AAR
Vital Signs
Ad-Specific Metrics

50. AAR
Vital Signs
Higher-level metrics
are hard to move
Ad-Specific Metrics

51. USER EXPERIENCE
TEST ITERATION
IMPACTS AAR

52. AAR
Vital Signs
Our focus Ad-Specific Metrics

53. Test evaluation
● Positive Signals: CTR, Downstream Effects
● Avoidance Signals: Volume, Audio Output
● An “Ad Quality Score”

54. Thanks!
(We’re hiring):
spotify.com/us/jobs/