Invited talk at @Netsci14 (5 June 2014). Branching-process models of meme popularity.

1. Competing for attention:
branching-process models of meme popularity
James P. Gleeson
MACSI, Department of Mathematics and Statistics,
University of Limerick, Ireland
#branching
www.ul.ie/gleeson james.gleeson@ul.ie
@gleesonj
NetSci14, Berkeley, 5 June 2014

2. Branching processes for meme popularity models
Overview
Φ
Memory
NetworkCompetition 𝑝 𝑘

3. Branching processes for meme popularity models
Part 1
Memory
NetworkCompetition

4. Motivating examples from empirical work on Twitter
Twitter 15M one-year dataset: collaboration with R. Baños and Y. Moreno
𝛼 = 2
fraction of hashtags with
popularity ≥ 𝑛 at age 𝑎

5. Branching processes for meme popularity models
Part 2
Memory
NetworkCompetition

6. Simon’s model
• Simon, “On a class of skew distribution functions”, Biometrica, 1955
• The basis of “cumulative advantage” and “preferential attachment” models;
see Simkin and Roychowdhury, Phys. Rep., 2011
• During each time step, one word is added to an ordered sequence
• With probability 𝜇, the added word is an innovation (a new word)
• With probability 1 − 𝜇, a previously-used word is copied; the copied word is
chosen at random from all words used to date
time

7. • Simulation results at age 𝑎 = 25000: seed time is 𝜏, observation time is
Ω = 𝜏 + 25000
• Early-mover advantage; fixed-age distributions have exponential tails
[Simkin and Roychowdyury, 2007]
Simon’s model
𝜇 = 0.02

8. Simon’s model as a branching process
• During each time step, one word is added to an ordered sequence
• With probability 𝜇, the added word is an innovation (a new word)
• With probability 1 − 𝜇, a previously-used word is copied; the copied word is
chosen at random from all words used to date
𝑡 = 𝜏 𝑡 = Ω

9. A word on probability generating functions (PGFs)
• PGFs are “transforms” of probability distributions:
define PGF 𝑓(𝑥) by
• …but “inverse transform” usually requires numerical methods, e.g. Fast
Fourier Transforms [Cavers, 1978]
• Some properties:
• PGF for the sum of independent random variables is the product of the
PGFs for each of the random variables
e.g., H. S. Wilf, generatingfunctionology, CRC Press, 2005
𝑓 𝑥 = � 𝑝 𝑘 𝑥 𝑘
∞
𝑘=0
𝑓 1 = � 𝑝 𝑘
∞
𝑘=0
= 1 𝑓𝑓 1 = � 𝑘 𝑝 𝑘
∞
𝑘=0
= 𝑧

10. Branching processes solution of Simon’s model
• Define 𝑞 𝑛(𝜏, Ω) as the probability that the word born at time 𝜏 has been
used a total of 𝑛 times by the observation time Ω
• Define 𝐻(𝜏, Ω, 𝑥) as the PGF for the popularity distribution
𝐻 𝜏, Ω, 𝑥 = � 𝑞 𝑛 𝜏, Ω 𝑥 𝑛
∞
𝑛=1
• Define 𝐺 𝜏, Ω, 𝑥 as the PGF for the excess popularity distribution, so that
𝐻 𝜏, Ω, 𝑥 = 𝑥 𝐺 𝜏, Ω, 𝑥
and
𝐺 Ω, Ω, 𝑥 = 1

11. Outcome for seed word Probability Contribution to 𝐺 𝜏, Ω, 𝑥
Copied at 𝜏 + Δ𝜏 (1 − 𝜇)
Δ𝑡
𝜏
𝑥 𝐺 𝜏 + Δ𝑡 2
Not copied 1 − (1 − 𝜇)
Δ𝑡
𝜏
𝐺 𝜏 + Δ𝑡
𝐺 𝜏, Ω, 𝑥 = 1 − 𝜇
𝛥𝑡
𝜏
𝑥 𝐺 𝜏 + 𝛥𝑡, 𝛺, 𝑥 2 + 1 − (1 − 𝜇)
𝛥𝑡
𝜏
𝐺 𝜏 + 𝛥𝑡, 𝛺, 𝑥
⇒ −
𝜕𝜕
𝜕𝜕
≈
1 − 𝜇
𝜏
𝑥 𝐺2 − 𝐺
𝜏 𝜏 + Δ𝑡
Ω ≫ 𝜏 ≫ Δ𝑡when
Branching processes solution of Simon’s model

12. ⇒ 𝐺 𝜏, Ω, 𝑥 =
𝜏
Ω
1−𝜇
1 − 𝑥 1 −
𝜏
Ω
1−𝜇−
𝜕𝜕
𝜕𝜕
=
1 − 𝜇
𝜏
𝑥 𝐺2 − 𝐺
Using 𝐻 = 𝑥 𝐺, the corresponding popularity distribution is
𝑞 𝑛 𝜏, Ω =
𝜏
Ω
1−𝜇
1 −
𝜏
Ω
1−𝜇 𝑛−1
Mean (expected) popularity:
𝑚 𝜏, Ω = � 𝑛 𝑞 𝑛(𝜏, Ω)
∞
𝑛=1
=
𝜕𝜕
𝜕𝜕
𝜏, Ω, 1 =
Ω
𝜏
1−𝜇
“Early-mover advantage”
Branching processes solution of Simon’s model

13. 𝜇 = 0.02
• Simulation results at age 𝑎 = 25000: set Ω = 𝜏 + 25000
• Early-mover advantage; fixed-age distributions have exponential tails
[Simkin and Roychowdyury, 2007]
Branching processes solution of Simon’s model

14. Note 𝛼 ≥ 2
• Power-law distributions arise only after averaging over seed times:
𝑞 𝑛
Ω ≡ � 𝑞 𝑛 𝜏, Ω
1
Ω
𝑑𝑑
Ω
0
=
1
1 − 𝜇
𝐵 𝑛,
2 − 𝜇
1 − 𝜇
∼ 𝑛−𝛼 as 𝑛 → ∞, with 𝛼 =
2−𝜇
1−𝜇
Branching processes solution of Simon’s model

15. A generalization of Simon’s model
Probability that a copying
event at time 𝑡 chooses
the word from time 𝜏
𝜏 𝑡
𝜙 𝜏, 𝑡 Δ𝑡
Simon’s model: 𝜙 𝜏, 𝑡 =
1
𝑡
Copying with memory models:
(e.g. Cattuto et al. 2007,
Bentley et al. 2011)
𝜙 𝜏, 𝑡 = Φ(𝑡 − 𝜏)
𝐺 𝜏, Ω, 𝑥 ≈ exp (1 − 𝜇) � 𝜙 𝜏, 𝑡 𝑥 𝐺 𝑡, Ω, 𝑥 − 1 𝑑𝑑
Ω
𝜏
𝐺 Ω, Ω, 𝑥 = 1withΩ ≫ 𝜏 ≫ Δ𝑡,when

16. A generalization of Simon’s model
Probability that a copying
event at time 𝑡 chooses
the word from time 𝜏
𝜏 𝑡
𝐺 𝜏, Ω, 𝑥 = exp (1 − 𝜇) � 𝜙 𝜏, 𝑡 𝑥 𝐺 𝑡, Ω, 𝑥 − 1 𝑑𝑑
Ω
𝜏
Age of seed at observation
time is 𝑎 = Ω − 𝜏
For 𝜙 𝜏, 𝑡 = Φ(𝑡 − 𝜏), let 𝐺 𝜏, Ω, 𝑥 = 𝐺�(Ω − 𝜏, 𝑥)
⇒ 𝐺� 𝑎, 𝑥 = exp (1 − 𝜇) � Φ(𝑠) 𝑥 𝐺� 𝑎 − 𝑠, 𝑥 − 1 𝑑𝑑
𝑎
0
• In this case, popularity distributions depend only on the age of the seed;
there is no early-mover advantage
𝜙 𝜏, 𝑡 Δ𝑡

17. • Simulation results at age 𝑎 = 25000: set Ω = 𝜏 + 25000
• Memory-time distribution: 𝜙 𝜏, 𝑡 = Φ 𝑡 − 𝜏 =
1
𝑇
𝑒−(𝑡−𝜏)/𝑇, with 𝑇 = 500
A generalization of Simon’s model
• In this case, popularity distributions depend only on the age of the seed;
there is no early-mover advantage
𝜇 = 0.02
𝛼 = 1.5

18. Competition-induced criticality
Simon’s original model, and the copying-with-memory model both have the
following features:
• One word is added in each time step
• Words “compete” for user attention in order to become popular
• The words have equal “fitness” – a type of “neutral model” [Pinto and
Muñoz 2011, Bentley et al. 2004 ]
• … except for the early-mover advantage in Simon’s model…
but only the copying-with-memory model gives critical branching processes.
• Gleeson JP, Cellai D, Onnela J-P, Porter MA, Reed-Tsochas F,
“A simple generative model of collective online behaviour” arXiv :1305.7440v2

19. Branching processes for meme popularity models
Part 3
Memory
NetworkCompetition

20. • Each node (of 𝑁) has a memory screen, which holds the meme of current
interest to that node. Each screen has capacity for only one meme.
• During each time step (Δ𝑡 = 1/𝑁), one node is chosen at random.
• With probability 𝜇, the selected node innovates, i.e., generates a brand-new
meme, that appears on its screen, and is tweeted (broadcast) to all the
node's followers.
• Otherwise (with probability 1 − 𝜇), the selected node (re)tweets the meme
currently on its screen (if there is one) to all its followers, and the screen is
unchanged. If there is no meme on the node's screen, nothing happens.
• When a meme 𝑚 is tweeted, the popularity 𝑛 𝑚 of meme 𝑚 is incremented
by 1 and the memes currently on the followers' screens are overwritten by
meme 𝑚.
The Markovian Twitter model

21. • Network structure: a node has 𝑘 followers (out-degree 𝑘) with probability
𝑝 𝑘.
• In-degree distribution (number of followings) has a Poisson distribution.
• Mean degree 𝑧 = ∑ 𝑘𝑝 𝑘𝑘 .
• A simplified version of the model of Weng, Flammini, Vespignani, Menczer,
Scientific Reports 2, 335 (2012).
• Related to the random-copying “neutral” (Moran-type) models of Bentley
et al. 2004 [Bentley et al. I’ll Have What She’s Having: Mapping Social
Behavior, MIT Press, 2011], where the distribution of popularity
increments can be obtained analytically [Evans and Plato, 2007].
• Our focus is on the distributions of popularity accumulated over long
timescales: when a meme 𝑚 is tweeted, the popularity 𝑛 𝑚 of meme 𝑚 is
incremented by 1.
The Markovian Twitter model

22. • When all screens are non-empty, memes compete for the limited resource
of user attention
• Random fluctuations lead to some memes becoming very popular, while
others languish in obscurity
The Markovian Twitter model

23. • Random fluctuations lead to some memes becoming very popular, while
others languish in obscurity
• The popularity distributions depend on the structure of the network,
through the out-degree distribution 𝑝 𝑘
𝜇 = 0
𝑝 𝑘 = 𝛿 𝑘,10
The Markovian Twitter model

24. • Random fluctuations lead to some memes becoming very popular, while
others languish in obscurity
• The popularity distributions depend on the structure of the network,
through the out-degree distribution 𝑝 𝑘
𝜇 = 0.01
𝑝 𝑘 ∝ 𝑘−𝛾; 𝛾 = 2.5
The Markovian Twitter model

25. overwritten 𝑧 Δ𝑡
𝑡 𝑡 + Δ𝑡
Branching processes solution of Twitter model
Define 𝐺(𝑎, 𝑥) as the PGF for the excess popularity distribution at age 𝑎 of
memes that originate from a single randomly-chosen screen (the root screen)
𝑎 𝑎 − Δ𝑡
Outcome for screen 𝑆1 Probability
𝜕𝜕
𝜕𝜕
= 𝑧 + 𝜇 − 𝑧 + 1 𝐺 + 1 − 𝜇 𝑥𝑥𝑥(𝐺) 𝑓 𝑥 = � 𝑝 𝑘 𝑥 𝑘
∞
𝑘=0
𝐺 0, 𝑥 = 1
selected, innovates 𝜇 Δ𝑡
selected, retweets (1 − 𝜇) Δ𝑡
not selected, survives 1 − (𝑧 + 1) Δ𝑡

26. 𝜕𝜕
𝜕𝜕
= 𝑧 + 𝜇 − 𝑧 + 1 𝐺 + 1 − 𝜇 𝑥𝑥𝑥(𝐺)
𝐻 𝑎, 𝑥 = � 𝑞 𝑛 𝑎 𝑥 𝑛 = 𝑥𝑥 𝑎, 𝑥 𝑓(𝐺 𝑎, 𝑥 )
∞
𝑛=0
Analysis of the branching process equation
Mean popularity of age-𝑎 memes:
𝑚 𝑎 = � 𝑛𝑞 𝑛(𝑎)
∞
𝑛=1
=
𝜕𝜕
𝜕𝜕
𝑎, 1 = 1 + (𝑧 + 1)
𝜕𝐺
𝜕𝜕
𝑎, 1
So:
𝑑𝑑
𝑑𝑑
= (𝑧 + 1)(1 − 𝜇 𝑚)
with 𝑚 0 = 1

27. 𝜕𝜕
𝜕𝜕
= 𝑧 + 𝜇 − 𝑧 + 1 𝐺 + 1 − 𝜇 𝑥𝑥𝑥(𝐺)
𝐻 𝑎, 𝑥 = � 𝑞 𝑛 𝑎 𝑥 𝑛 = 𝑥𝑥 𝑎, 𝑥 𝑓(𝐺 𝑎, 𝑥 )
∞
𝑛=0
Analysis of the branching process equation
Mean popularity of age-𝑎 memes:
𝑚 𝑎 = �
1 + 𝑧 + 1 𝑎 if 𝜇 = 0
1
𝜇
−
1 − 𝜇
𝜇
𝑒−𝜇 𝑧+1 𝑎 if 𝜇 > 0

28. Analysis of the branching process equation
Mean popularity of age-𝑎 memes:
𝑚 𝑎 = �
1 + 𝑧 + 1 𝑎 if 𝜇 = 0
1
𝜇
−
1 − 𝜇
𝜇
𝑒−𝜇 𝑧+1 𝑎 if 𝜇 > 0

29. Long-time (old-age) asymptotics
• If 𝑓′′
1 < ∞ (finite second moment of 𝑝 𝑘),
𝑞 𝑛 ∞ ∼ 𝐴 𝑒−
𝑛
𝜅 𝑛−
3
2 as 𝑛 → ∞
with 𝜅 =
2
𝜇2
𝑓′′ 1 +2𝑧
𝑧+1 2
• If 𝑝 𝑘 ∝ 𝑘−𝛾
for large 𝑘 with 2 < 𝛾 < 3,
𝑞 𝑛 ∞ ∼ � 𝐵 𝑛
−
𝛾
𝛾−1 if 𝜇 = 0
𝐶 𝑛−𝛾
if 𝜇 > 0
as 𝑛 → ∞
𝜕𝜕
𝜕𝜕
= 0
cf. sandpile SOC on networks [Goh et al. 2003]

30. Comparing branching process theory with simulations
𝑝 𝑘 = 𝛿 𝑘,10
𝑝 𝑘 ∝ 𝑘−𝛾
𝛾 = 2.5
𝜇 = 0.01
𝜇 = 0

31. Branching processes for meme popularity models
Part 4
Memory
NetworkCompetition

32. Twitter model with memory
Φ
• During each time step (with time increment Δ𝑡 = 1/𝑁), one node is
chosen at random.
• The selected node may innovate (with probability 𝜇), or it may retweet a
meme from its memory using the memory distribution Φ(𝑡 − 𝜏).

33. • Define 𝐺(𝑎, 𝑥) as the PGF for the excess popularity distribution at age 𝑎 of
memes that originate from a single randomly-chosen seed (the root)
• The mean popularity 𝑚(𝑎) of age-𝑎 memes has Laplace transform:
Branching process analysis
𝐺 𝑎, 𝑥 = � 𝑝 𝑘 � 𝑑𝑡 (𝑧 + 𝜇)𝑒− 𝑧+𝜇 𝑡
×
∞
0𝑘
× exp − 1 − 𝜇 � 𝑑𝑑
min 𝒕,𝑎
0
� 𝑑𝑑
𝑎−𝑟
0
Φ 𝑎 − 𝑟 − 𝜏 1 − 𝑥 𝐺 𝜏, 𝑥 𝑘
𝑚� 𝑠 =
𝑧 + 𝜇 + 𝑠 + 1 − 𝜇 Φ�(𝑠)
𝑠 𝑧 + 𝜇 + 𝑠 − 1 − 𝜇 𝑧 Φ�(𝑠)

34. Φ
Memory
NetworkCompetition 𝑝 𝑘
Comparing the model to data
𝛾 ≈ 2.13

35. Φ 𝜏 = Gamma(𝑘, 𝜃)
=
1
Γ 𝑘 𝜃 𝑘
𝜏 𝑘−1
𝑒−𝜏/𝜃
𝑘 = 0.2; 𝜃 = 355
Φ
𝑚� 𝑠 =
𝑧 + 𝜇 + 𝑠 + 1 − 𝜇 Φ�(𝑠)
𝑠 𝑧 + 𝜇 + 𝑠 − 1 − 𝜇 𝑧 Φ�(𝑠)
Comparing the model to data
𝜇 = 0.02

36. Comparing the model to data
Data Model

37. Conclusions: Branching processes for meme popularity models
Φ
Memory
NetworkCompetition 𝑝 𝑘

38. • Competition between memes for the limited resource of user attention
induces criticality in this model in the 𝜇 → 0 limit
• Criticality gives power-law popularity distributions and epochs of linear-in-
time popularity growth, even for (cf. Weng et al. 2012)
– homogeneous out-degree distributions
– homogeneous user activity levels
• Despite its simplicity, the model matches the empirical popularity
distribution of real memes (hastags on Twitter) remarkably well
• Generalizations of the model are possible, and remain analytically tractable
Conclusions: Competition-induced criticality
⇒ a useful null model to understand how memory, network
structure and competition affect popularity distributions

39.  Davide Cellai, UL
 Mason Porter, Oxford
 J-P Onnela, Harvard
 Felix Reed-Tsochas, Oxford
 Jonathan Ward, Leeds
 Kevin O’Sullivan, UL
 William Lee, UL
 Yamir Moreno, Zaragoza
 Raquel A Baños, Zaragoza
 Kristina Lerman, USC
 Science Foundation Ireland
 FP7 FET Proactive PLEXMATH
 SFI/HEA Irish Centre for High-End
Computing (ICHEC)
Collaborators, funding, references
• “A simple generative model of collective online behaviour” arXiv :1305.7440v2
• Physical Review Letters, 112, 048701 (2014); arXiv:1305.4328

40. Branching processes for meme popularity models
Φ
Memory
NetworkCompetition 𝑝 𝑘
#branchingwww.ul.ie/gleesonj james.gleeson@ul.ie @gleesonj