Can we predict the future popularity of a song, movie or tweet? Recent work suggests that although it may be hard to predict an item’s popularity when it is first introduced, peeking into its early adopters and properties of their social network makes the problem easier. We test the robustness of such claims by using data from social networks spanning music, books, photos, and URLs. We find a stronger result: not only do predictive models with peeking achieve high accuracy on all datasets, they also generalize well, so much so that models trained on any one dataset perform with comparable accuracy on items from other datasets. Though practically useful, our models (and those in other work) are intellectually unsatisfying because common formulations of the problem, which involve peeking at the first small-k adopters and predicting whether items end up in the top half of popular items, are both too sensitive to the speed of early adoption and too easy. Most of the predictive power comes from looking at how quickly items reach their first few adopters, while for other features of early adopters and their networks, even the direction of correlation with popularity is not consistent across domains. Problem formulations that examine items that reach k adopters in about the same amount of time reduce the importance of temporal features, but also overall accuracy, highlighting that we understand little about why items become popular while providing a context in which we might build that understanding.

1. Predictability of popularity:
Gaps between prediction
and understanding
BENJAMIN SHULMAN, YELP
AMIT SHARMA, MICROSOFT RESEARCH
DAN COSLEY, CORNELL UNIVERSITY
1

2. Will it get popular?
2

3. Can we predict popularity?
Even with state-of-the-art features and prediction models, hard to
predict apriori [Bakshy et al. 2011, Martin et al. 2016]
3

4. What if we could peek into
early activity?
Szabo and Huberman Pinto et al. Zhao et al. Cheng et al.
(2010) (2012) (2015) (2014)
Possible to predict future popularity with high accuracy.
4

5. Prediction
Understanding
how items gain
popularity
5

6. Varying conclusions from prior
studies on predicting popularity
FEATURES Content Structural Early Adopters Temporal
Szabo and Huberman (2010) - N - Y
Tsur et al. (2012) Y Y - Y
Pinto et al. (2013) - - - Y
Yu et al. (2015) - N - Y
Romero et al. (2013) - Y - -
Cheng et al. (2014) - Y Y Y
Lerman et al. (2008) - Y Y -
Weng et al. (2013) - Y N -
6

7. Example: Does higher network
density lead to higher popularity?
Lerman and Galstyan, 2008. Lower
network density among early adopters
lead to higher final popularity.
Romero et al., 2013. Both higher
and lower density could lead to
higher final popularity.
7

8. A multi-platform study on accuracy
of popularity prediction and
usefulness of different features.
8

9. Four datasets from different
domains
LOVE SONG
RATE BOOKS
FAVORITE
PHOTOS
SHARE
URLs
437k users
5.8M songs
44M adoptions
737k users
58k tweets
2.7M adoptions
183k users
10.9M tweets
28M adoptions
252k users
1.3M tweets
33M adoptions
9

10. Given a set of items and data about
their early adoptions, which among
them are more likely to become
popular?
• Use data about the first K=5 early adoptions to
predict whether final popularity at T=28 is greater
than median.
[Cheng et al. 2014]
10

11. Features
Temporal: time for each adoption, time between adoptions
Structural: degree of early adopters, network density among early
adopters
Early adopters: age, past activity of early adopters
Preference similarity: mean similarity between early adopters
[Sharma and Cosley, 2015]
11

12. 12

13. A single temporal feature may
be sufficient
time5—time taken for an item to receive 5 adoptions
Even a model with this single temporal feature
- achieves more than 70% accuracy on all datasets
- accounts for nearly 97% of the accuracy of the full model for each
dataset
13

14. 14

15. Other features have inconsistent
effects on popularity
If you fit a single-feature logistic regression:
For 12 of the 25 features, the coefficient term flips between
being positive and negative across models fit on different
datasets.
E.g. Higher density—number of edges in the subgraph of early
adopters
--higher popularity on Flickr (β coefficient=0.04),
-- lower popularity on Last.fm (β coefficient=-0.09).
15

16. What did we learn?
Temporal features dominate other features.
Temporal features also generalize across diverse
item domains, unlike other features.
16

17. Early popularity leads to final
popularity.
If an item quickly gains early adoptions, then it is more likely to be
popular eventually.
17

18. But why?
How do they become popular?
What properties of items, early adopters and
their social networks matter?
18

19. Temporally-matched Balanced
Classification
Control for average rate of early adoption.
Problem:
Among items with exactly k adoptions at the end of a fixed
time period t, which ones would be higher than the median
popularity at a later time T?
Expect temporal features to play a lesser role.
19

20. 20

21. Conclusion
Temporal features can be used for prediction, they even
generalize across item domains.
But prediction tells us little about how items become
popular.
Temporally matched formulation can be a way
forward to study the effect of early adopter
features.
Prediction with
peeking
Understanding how
items gain popularity
21

22. thank you!
Amit Sharma (@amt_shrma)
www.amitsharma.in
Temporally matched formulation can be a way forward to
study the effect of early adopter features.
Prediction with
peeking
Understanding how
items gain popularity
22