The UserMetrics API
Measuring participation in Wikimedia projects
Dario Taraborelli
Wikimedia Foundation
Wikimania 2013, H...
Overview
Rationale
Background
Introducing the UserMetrics API
The next steps: Wikimetrics
Rationale
How do we determine if a {program, experiment, new feature} we run
to engage new editors and increase participat...
The problem: quasi-experiments
The problem: metric definitions
ACTIVE EDITOR
RETAINED EDITOR
HIGH-QUALITY ARTICLE
NEW WIKIPEDIAN
SURVIVING EDIT
User cohorts
Metric research
[[m:Research:Metrics/
edit sessions]]
Cohort analysis - 1
Define a cohort from a list of editors sharing a meaningful attribute
Examples
• participants in an ex...
Cohort analysis - 2
Measure the {quality, productivity, retention} of these users via a
standard set of metrics
Cohort analysis - 3
Compare a cohort to a (meaningful) baseline using a standard set of
metrics
UserMetrics
UserMetrics
UserMetrics
Formally defined metrics, backed by extensive research and
crash-tested in the WMF Editor Engagement program.
S...
raw results
aggregate results
time series
WikiMetrics
Functionality ported from UserMetrics
Improved UX
Redesigned backend and job management
Access control and use...
Get involved
UserMetrics (private)
end-user documentation
https://www.mediawiki.org/wiki/UserMetrics/Guide
source code
htt...
thank you
Dario Taraborelli
The UserMetrics API: Measuring participation in Wikimedia projects
dario@wikimedia.org
@reader...
The UserMetrics API. Measuring participation in Wikimedia projects
Upcoming SlideShare
Loading in …5
×

The UserMetrics API. Measuring participation in Wikimedia projects

566 views
506 views

Published on

Slides from the presentation of UserMetrics/Wikimetrics at Wikimania 2013 (Hong Kong, 9 August 2013).

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
566
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
1
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

The UserMetrics API. Measuring participation in Wikimedia projects

  1. 1. The UserMetrics API Measuring participation in Wikimedia projects Dario Taraborelli Wikimedia Foundation Wikimania 2013, Hong Kong, 9 August 2013
  2. 2. Overview Rationale Background Introducing the UserMetrics API The next steps: Wikimetrics
  3. 3. Rationale How do we determine if a {program, experiment, new feature} we run to engage new editors and increase participation is actually working? How do we measure the relative success of these interventions in driving participation?
  4. 4. The problem: quasi-experiments
  5. 5. The problem: metric definitions ACTIVE EDITOR RETAINED EDITOR HIGH-QUALITY ARTICLE NEW WIKIPEDIAN SURVIVING EDIT
  6. 6. User cohorts
  7. 7. Metric research [[m:Research:Metrics/ edit sessions]]
  8. 8. Cohort analysis - 1 Define a cohort from a list of editors sharing a meaningful attribute Examples • participants in an experimental condition of an A/B test • students enrolled in a Global Education course • members of a Wikiproject • new editors reaching 10+ edits in their first month since registration • users registered on mobile devices • participants in an editathon
  9. 9. Cohort analysis - 2 Measure the {quality, productivity, retention} of these users via a standard set of metrics
  10. 10. Cohort analysis - 3 Compare a cohort to a (meaningful) baseline using a standard set of metrics
  11. 11. UserMetrics
  12. 12. UserMetrics
  13. 13. UserMetrics Formally defined metrics, backed by extensive research and crash-tested in the WMF Editor Engagement program. Support for cohorts from any Wikimedia project. Fast extraction of computationally intensive metrics for cohorts of thousands of users. Individual responses or aggregate data using various aggregator methods. Supports time series. Extensible and fully parametrized (metrics can be fine- tuned, default parameters overridden as needed)
  14. 14. raw results
  15. 15. aggregate results
  16. 16. time series
  17. 17. WikiMetrics Functionality ported from UserMetrics Improved UX Redesigned backend and job management Access control and user authentication Abstraction
  18. 18. Get involved UserMetrics (private) end-user documentation https://www.mediawiki.org/wiki/UserMetrics/Guide source code http://github.com/wikimedia/analytics-user-metrics WikiMetrics (public, under development) home http://metrics.wmflabs.org source code http://github.com/wikimedia/analytics-wikimetrics mailing list https://lists.wikimedia.org/mailman/listinfo/wikimetrics
  19. 19. thank you Dario Taraborelli The UserMetrics API: Measuring participation in Wikimedia projects dario@wikimedia.org @readermeter

×