The presentation covers the problem of the digital badges' impact on social media users and presents the causal framework allowing to validate it. It was presented during The WebConf (WWW2018). The arXiv paper: https://arxiv.org/abs/1707.08160

1. On the
Causal Effect of
Badges
Tomasz Kuśmierczyk Manuel Gomez Rodriguez

2. CONTRIBUTING
ACTIONS
BADGE AWARD
2

3. 3
FIRST ACTION
BADGE AWARD

4. time
tu
edit
action
4
Tag Editor

5. time
su
tu
reputation
1500pts
edit
action
5
Tag Editor

6. time
su
tu
platforminception
su
tu
badgeintroduction
before after
Delayed badge introduction
6

7. Goal: quantify causal effect
beforeafter
7
single user

8. Utility
heterogeneity
Random temporal
changes
8
Challenges
Measuring
progress
?

9. Causal inference
framework 9

10. Basic model
10
time
su
tu
interval
length
probability (tu
-su
) ~ Exp( u
)

11. Basic model
11
time
su
tu
su
tu
badgeintroduction
(tu
-su
) ~ Exp( 1
)(tu
-su
) ~ Exp( 0
)

12. Are intensities the same?
12
interval length
probability
interval length
probability
0
= 1
1
0
vs
H0 : 0
= 1
≥ 0
H1 : 0
≠ 1
, 0
≥ 0, 1
≥ 0

13. Utility
heterogeneity
Random temporal
changes
13
Challenges
Measuring
progress
?

14. Robust model
14
time
badgeintroduction
(tu
-su
) ~ Exp( 1
(u))(tu
-su
) ~ Exp( 0
(u))

15. Are priors the same?
15
(u)
probability
vs
(u)
probability
k1
k0
k0
=k1
H0 : k0
=k1
≥ 0
H1 : k0
≠k1
, k0
≥ 0, k1
≥ 0

16. Comparing H1 vs H0
LLR = log likelihood under H1
16
time
badgeintro
H0
k
^
log likelihood under H0-
time
badgeintro
H1
k1
k0
^
^
k0
k1=

17. No effect: LLR = 0
17
time
badgeintro
H1
k1k0
^ ^
time
badgeintro
H0
k
^

18. Slight change: LLR = ℰ
18
time
badgeintro
H1
k1k0
^ ^
time
badgeintro
H0
k
^

19. Utility
heterogeneity
Random temporal
changes
19
Challenges
Measuring
progress
?

20. Virtual badges: quantifying significance
20
time
badgeintro
virtual
badgeintro
virtual
no badge
effect
badge
effect?
LLRLLRvirtual ?

21. Virtual badges: quantifying significance
21
time
badgeintro
virtual
badgeintro
no badge
effect
badge
effect?
virtual
badgeintro
virtual
badgeintro
LLR
LLR assuming H0
virtual virtualvirtual

22. Measuring
progress
Utility
heterogeneity
Random temporal
changes
22
Challenges
?

23. Results
23

24. StackOverflow First-time Badges
24
● Tag Editor
● Promoter
● Investor

25. ● Tag Editor
● Promoter
● Investor
StackOverflow Badges - Bounties
25

26. Badges significance
26

27. Why?
27
UTILITY FROM THE ACTION
CONTRIBUTING
ACTIONS
BADGE AWARDBADGE UTILITY

28. Why?
28
PROXY
CONTRIBUTING
ACTIONS
BADGE AWARD

29. Understanding user utility
(TagEditor badge)
29

30. Conclusions
● only some badges have
significant impact
● success depends on
action utility structure
● successful badges
improve community
functioning *
30

31. Contact:
tomaszku@ntnu.no
Sources:
learning.mpi-sws.org/badges/
31