Presentation at ECCS tnets satellite, held in Barcelona, September 2013. A study of user engagement in Wikipedia using networks analysis.

1. TEMPORAL VARIATIONS IN
NETWORKED USER ENGAGEMENT
TNETS Satellite - ECCS
In
collabora*on
with:
Mounia
Lalmas,
Ricardo
Baeza-­‐Yates
Jane%e
Lehmann

2. Outline
1. What
is
user
engagement?
Overview,
defini,ons,
examples
2. Why
should
we
combine
it
with
network
analysis?
Networked
user
engagement
Defini,on
of
traffic
networks
3. Do
we
observe
temporal
varia>ons?
Metrics
and
examples
Lights
on
by
JC*+A!

3. Engagement in Wikipedia
JaneHe
Lehmann
Mo,va,on
3
The
ques>on:
How
engaged
are
users
on
Wikipedia?

4. Engagement in Wikipedia
JaneHe
Lehmann
Mo,va,on
4
How
engaged
are
users
on
Wikipedia?
Web
traffic
reports:
Google
Analy>cs,
Alexa.com,
…
Popularity
(#Users)
Ac>vity
(DwellTime)
Loyalty
(Ac,veDays)

5. Can
we
do
more?
leP
by
[
embr
]

6. The Idea
Yes,
we
can…
Users
in
Wikipedia
perform
different
types
of
ac*ons
JaneHe
Lehmann
Mo,va,on
6
Visi,ng
the
main
page
Reading
an
ar,cle
Looking
at
the
revision
history
Visi,ng
categories
Visi,ng
portals
Expor,ng
books
Talking
about
an
ar,cle
Par,cipa,ng
in
the
community
Edi,ng
ar,cles

7. Actions in Wikipedia
Our
assump>on
Ø Engaged
users
perform
different
types
of
ac,ons
(reading,
edi,ng,
par,cipa,ng
in
the
community)
Ø If
Wikipedia
is
engaging,
there
is
traffic
between
ac,ons
JaneHe
Lehmann
Mo,va,on
7

8. 182-­‐365+1
by
meaganmakes
From
site
to
network
engagement

9. Action Networks
Browsing
data
sample
(Sep
2011
–
Oct
2012)
• 48
ac,ons
• 2.3M
pages
• 1.3M
user
• 25.4M
clicks
JaneHe
Lehmann
Mo,va,on
9
portal
main
page
donate
ar,cle
talk
track
community
history
meta
page
edit
sta,s,cs
export
categories
login
search
user
G=(V,
E,
λ)
V:
Ac,ons
on
Wikipedia
E:
Naviga,ons
between
ac,ons
λ(e):
Traffic
volume
(#Clicks)

10. Network-level Metrics
Network-­‐level
metrics
Total
traffic
and
traffic
recircula*on
JaneHe
Lehmann
Networked
User
Engagement
10
TotalNodeTraffic
Total
number
of
users
on
Wikipedia
ac*ons
TotalEdgeTraffic
Total
number
of
users
naviga*ng
between
Wikipedia
ac*ons
TotalTrafficRecircula,on
Propor*on
of
traffic
on
the
network
with
respect
to
the
maximum
possible
(TotalEdgeTraffic
/
TotalNodeTraffic)
20
user
5
user
10
user
30
user

11. TotalNodeTraffic
TotalTrafficRecirculation
Sep 2011 – Oct 2012
Networks over Time
Monthly
traffic
JaneHe
Lehmann
Networked
User
Engagement
11
• General
trend
» Node
traffic
increases
» Traffic
recircula,on
decreases
• Dona,on
campaign
(Nov
–
Dec
2011)
» Increase
of
node
traffic
» Decrease
of
traffic
recircula,on
• SOPA/PIPA
protest
(Jan
2012)
» Peak
of
node
traffic
and
traffic
recircula,on
Opposite
trend
of
node
traffic
and
traffic
recircula7on.
SOPA/PIPA
Dona,on

12. TotalNodeTraffic
Weekdays Weekend TotalTrafficRecirculation
March - April 2012
Networks over Time
Weekdays
and
Weekends
JaneHe
Lehmann
Networked
User
Engagement
12
• Traffic
differs
between
weekdays
and
weekends
• Traffic
is
higher
over
the
weekend,
but
the
traffic
recircula,on
is
lower
There
is
more
traffic
during
the
week,
but
users
are
more
engaged
on
weekends.

13. NodeTraffic
Traffic
on
node
=
Number
of
users
that
perform
this
ac*ons
NodeTrafficRC
Propor*on
of
out-­‐going
traffic
with
respect
to
the
maximum
possible
DiffNodeTrafficT
DiffNodeTrafficRCT
Node-level Metrics
Node-­‐level
metrics
Node
traffic
and
traffic
recircula*on
JaneHe
Lehmann
Networked
User
Engagement
13
10
user
5
user
NodeTrafficT
NodeTrafficT−1
NodeTrafficRCT
NodeTrafficRCT−1
sta,c
dynamic

14. nodeTraffic
TrafficRecirculation
Sep 2011 – Oct 2012
Donation Campaign
How
did
the
dona>on
campaign
affect
the
engagement
in
Wikipedia?
JaneHe
Lehmann
Networked
User
Engagement
14
• We
calculated
the
change
of:
» Node
traffic
(DiffNodeTraffic)
» Outgoing
node
traffic
(DiffNodeTrafficRC)
• October,
November,
December
2011
• We
iden,fied
nodes
(ac,ons)
that
have
received
an
high
increase
in
traffic
and
out-­‐going
traffic
Dona,on
campaign
(Nov
–
Dec)

15. Donation Campaign
JaneHe
Lehmann
Networked
User
Engagement
15
Oct-11 Nov-11 Dec-11
track 1% 0% 7%
account_login 2% 0% 7%
account_creation 0% 18% -6%
mainpage 3% 0% 4%
donate 0% 23% 4%
metapage 2% 11% 18%
wikimedia 0% 7% 2%
participation 0% 27% 0%
style_guides 4% 0% 12%
user 0% 0% 22%
wikipedia -5% 5% -2%
wizard 0% 19% 0%
Oct-11 Nov-11 Dec-11
search_detail 3% 49% 7%
account_creation -7% 27% 26%
article 2% 31% 29%
category 7% 36% 18%
file -2% 36% 31%
donate 0% 38,733% 15%
metapage 1% 17% 28%
wikimedia 100% 6,45% 35%
participation -9% 59% -5%
style_guides 5% 14% 30%
user 0% 39% 37%
wizard 0% 42% 21%
Increase
of
node
traffic
Increase
of
outgoing
node
traffic

16. Donation Campaign
JaneHe
Lehmann
Networked
User
Engagement
16
Ø Less
changes
before
the
dona,on
campaign
starts
Increase
of
node
traffic
Increase
of
outgoing
node
traffic
Oct-11 Nov-11 Dec-11
track 1% 0% 7%
account_login 2% 0% 7%
account_creation 0% 18% -6%
mainpage 3% 0% 4%
donate 0% 23% 4%
metapage 2% 11% 18%
wikimedia 0% 7% 2%
participation 0% 27% 0%
style_guides 4% 0% 12%
user 0% 0% 22%
wikipedia -5% 5% -2%
wizard 0% 19% 0%
Oct-11 Nov-11 Dec-11
search_detail 3% 49% 7%
account_creation -7% 27% 26%
article 2% 31% 29%
category 7% 36% 18%
file -2% 36% 31%
donate 0% 38,733% 15%
metapage 1% 17% 28%
wikimedia 100% 6,45% 35%
participation -9% 59% -5%
style_guides 5% 14% 30%
user 0% 39% 37%
wizard 0% 42% 21%

17. Donation Campaign
JaneHe
Lehmann
Networked
User
Engagement
17
Ø Many
users
donate
money
Increase
of
node
traffic
Increase
of
outgoing
node
traffic
Oct-11 Nov-11 Dec-11
track 1% 0% 7%
account_login 2% 0% 7%
account_creation 0% 18% -6%
mainpage 3% 0% 4%
donate 0% 23% 4%
metapage 2% 11% 18%
wikimedia 0% 7% 2%
participation 0% 27% 0%
style_guides 4% 0% 12%
user 0% 0% 22%
wikipedia -5% 5% -2%
wizard 0% 19% 0%
Oct-11 Nov-11 Dec-11
search_detail 3% 49% 7%
account_creation -7% 27% 26%
article 2% 31% 29%
category 7% 36% 18%
file -2% 36% 31%
donate 0% 38,733% 15%
metapage 1% 17% 28%
wikimedia 100% 6,45% 35%
participation -9% 59% -5%
style_guides 5% 14% 30%
user 0% 39% 37%
wizard 0% 42% 21%

18. Donation Campaign
JaneHe
Lehmann
Networked
User
Engagement
18
Ø Users
inform
themselves
about
the
dona,on
progress
(wikimediafounda,on.org),
but
do
not
return
to
Wikipedia
Increase
of
node
traffic
Increase
of
outgoing
node
traffic
Oct-11 Nov-11 Dec-11
track 1% 0% 7%
account_login 2% 0% 7%
account_creation 0% 18% -6%
mainpage 3% 0% 4%
donate 0% 23% 4%
metapage 2% 11% 18%
wikimedia 0% 7% 2%
participation 0% 27% 0%
style_guides 4% 0% 12%
user 0% 0% 22%
wikipedia -5% 5% -2%
wizard 0% 19% 0%
Oct-11 Nov-11 Dec-11
search_detail 3% 49% 7%
account_creation -7% 27% 26%
article 2% 31% 29%
category 7% 36% 18%
file -2% 36% 31%
donate 0% 38,733% 15%
metapage 1% 17% 28%
wikimedia 100% 6,45% 35%
participation -9% 59% -5%
style_guides 5% 14% 30%
user 0% 39% 37%
wizard 0% 42% 21%

19. Donation Campaign
JaneHe
Lehmann
Networked
User
Engagement
19
Ø More
users
go
to
the
“Account
crea,on”
page
and
also
create
accounts
Increase
of
node
traffic
Increase
of
outgoing
node
traffic
Oct-11 Nov-11 Dec-11
track 1% 0% 7%
account_login 2% 0% 7%
account_creation 0% 18% -6%
mainpage 3% 0% 4%
donate 0% 23% 4%
metapage 2% 11% 18%
wikimedia 0% 7% 2%
participation 0% 27% 0%
style_guides 4% 0% 12%
user 0% 0% 22%
wikipedia -5% 5% -2%
wizard 0% 19% 0%
Oct-11 Nov-11 Dec-11
search_detail 3% 49% 7%
account_creation -7% 27% 26%
article 2% 31% 29%
category 7% 36% 18%
file -2% 36% 31%
donate 0% 38,733% 15%
metapage 1% 17% 28%
wikimedia 100% 6,45% 35%
participation -9% 59% -5%
style_guides 5% 14% 30%
user 0% 39% 37%
wizard 0% 42% 21%

20. Donation Campaign
JaneHe
Lehmann
Networked
User
Engagement
20
Ø More
users
go
to
community
pages
(e.g.
the
community
portal,
reference
desk,
goings-­‐on)
Increase
of
node
traffic
Increase
of
outgoing
node
traffic
Oct-11 Nov-11 Dec-11
track 1% 0% 7%
account_login 2% 0% 7%
account_creation 0% 18% -6%
mainpage 3% 0% 4%
donate 0% 23% 4%
metapage 2% 11% 18%
wikimedia 0% 7% 2%
participation 0% 27% 0%
style_guides 4% 0% 12%
user 0% 0% 22%
wikipedia -5% 5% -2%
wizard 0% 19% 0%
Oct-11 Nov-11 Dec-11
search_detail 3% 49% 7%
account_creation -7% 27% 26%
article 2% 31% 29%
category 7% 36% 18%
file -2% 36% 31%
donate 0% 38,733% 15%
metapage 1% 17% 28%
wikimedia 100% 6,45% 35%
participation -9% 59% -5%
style_guides 5% 14% 30%
user 0% 39% 37%
wizard 0% 42% 21%

21. Discussion and Future Work
JaneHe
Lehmann
Networked
User
Engagement
21
• Network
analysis
enhances
the
understanding
of
user
engagement
• User
engagement
in
Wikipedia
• Depends
on
the
ac,ons
and
interac,ons
between
them
• Changes
over
,me
and
paHerns
can
be
observed
Future
work:
• Defini,on
of
new
metrics
that
combine
network
traffic
and
other
engagement
metrics
(e.g.
dwell
,me)
• Comparing
changes
in
traffic-­‐
and
hyperlink-­‐networks

22. JaneHe
Lehmann
Networked
User
Engagement
22
Ques>ons
and
Discussion…
portal
main
page
ar,cle
track
community
history
meta
page
sta,s,cs
export
categories
login
search
user
Janee
Lehmann
Universitat
Pompeu
Fabra,
Spain
jnt.lehmann@gmail.com
Mounia
Lalmas
Yahoo!
Labs,
Barcelona,
Spain
mounia@acm.org
Ricardo
Baeza-­‐Yates
Yahoo!
Labs,
Barcelona,
Spain
rby@yahoo-­‐inc.com

23. Actions in Wikipedia
JaneHe
Lehmann
Mo,va,on
23
Loc
Context
Ac>on
Descrip>on
en
ar,cle/category/portal/file/talk
VIEW
view
an
ar*cle,
category,
portal,
etc.
en
ar,cle/category/portal/file/talk
VIEW_DETAIL
view
to
history,
revisions,
etc.
en
ar,cle/category/portal/file/talk
EDIT
edi*ng,
rever*ng
an
edit
meta
file
VIEW
view
a
file
in
the
file
database
en
wizard
USE
create
ar*cles,
upload
files
en
*
RANDOM
Visit
a
random
ar*cle
en
*
EXPORT
Export
ar*cles
as
a
pdf
book,
export
en
congress
VIEW
SOPA/PIPA
Strike
en
*
SEARCH
search
for
an
ar*cle
en
*
SEARCH_DETAIL
browse
categories,
see
pages
linking
to
current
one
(whats
link
here),
see
ar*cles
whose
*tle
starts
with
a
given
prefix
en
references
SEARCH
see
links
to
external
sources
about
a
given
book
(given
the
ISBN)
en
mainpage
VIEW
visi*ng
the
main
page
meta
wikimedia
VIEW
visi*ng
the
wikimedia
site
meta
metapage
VIEW
visi*ng
hZp://www.wikipedia.org/
meta
*
DONATE
donate
money
to
wikimedia
content
search
meta

24. Actions in Wikipedia
JaneHe
Lehmann
Mo,va,on
24
Loc
Context
Ac>on
Descrip>on
en
account
LOGIN
login
page
en
account
LOGIN_SUBMIT
login
en
account
LOGOUT
logout
en
account
MANAGE
reset
password,
account
se[ngs,
eMail
verifica*on
en
account
SIGN_UP
create
a
new
account
en
sandbox
USE
sandbox
en
style_guides
SEEK
visi*ng
mediawiki
and
template
namespace
(which
is
used
to
define
css,
provides
templates,
etc.)
en
style_guides
EDIT
edi*ng
style
guides
en
help
SEEK
visi*ng
wikipedia
help
pages
en
wikipedia
VIEW/VIEW_DETAIL/EDIT
visi*ng
wikipedia
project
namespace
en
user/user_talk
VIEW/VIEW_DETAIL/EDIT
Ac*ons
in
user
and
user
talk
namespace
en
maintenance
SEEK
triggers,
to-­‐do
lists,
ongoing
discussions
and
special
cases,
visi*ng
the
special
namespace
en
*
TRACK
track
changes
and
users,
watchlist,
log
en
par,cipants
SEEK
visi*ng
the
community
portal,
reference
desk,
goings-­‐on
en
sta,s,cs
SEEK
visi*ng
sta*s*c
pages
about
Wikipedia
account
management
community