• Like
The UserMetrics API. Measuring participation in Wikimedia projects
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

The UserMetrics API. Measuring participation in Wikimedia projects

  • 237 views
Published

Slides from the presentation of UserMetrics/Wikimetrics at Wikimania 2013 (Hong Kong, 9 August 2013).

Slides from the presentation of UserMetrics/Wikimetrics at Wikimania 2013 (Hong Kong, 9 August 2013).

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
237
On SlideShare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
1
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The UserMetrics API Measuring participation in Wikimedia projects Dario Taraborelli Wikimedia Foundation Wikimania 2013, Hong Kong, 9 August 2013
  • 2. Overview Rationale Background Introducing the UserMetrics API The next steps: Wikimetrics
  • 3. Rationale How do we determine if a {program, experiment, new feature} we run to engage new editors and increase participation is actually working? How do we measure the relative success of these interventions in driving participation?
  • 4. The problem: quasi-experiments
  • 5. The problem: metric definitions ACTIVE EDITOR RETAINED EDITOR HIGH-QUALITY ARTICLE NEW WIKIPEDIAN SURVIVING EDIT
  • 6. User cohorts
  • 7. Metric research [[m:Research:Metrics/ edit sessions]]
  • 8. Cohort analysis - 1 Define a cohort from a list of editors sharing a meaningful attribute Examples • participants in an experimental condition of an A/B test • students enrolled in a Global Education course • members of a Wikiproject • new editors reaching 10+ edits in their first month since registration • users registered on mobile devices • participants in an editathon
  • 9. Cohort analysis - 2 Measure the {quality, productivity, retention} of these users via a standard set of metrics
  • 10. Cohort analysis - 3 Compare a cohort to a (meaningful) baseline using a standard set of metrics
  • 11. UserMetrics
  • 12. UserMetrics
  • 13. UserMetrics Formally defined metrics, backed by extensive research and crash-tested in the WMF Editor Engagement program. Support for cohorts from any Wikimedia project. Fast extraction of computationally intensive metrics for cohorts of thousands of users. Individual responses or aggregate data using various aggregator methods. Supports time series. Extensible and fully parametrized (metrics can be fine- tuned, default parameters overridden as needed)
  • 14. raw results
  • 15. aggregate results
  • 16. time series
  • 17. WikiMetrics Functionality ported from UserMetrics Improved UX Redesigned backend and job management Access control and user authentication Abstraction
  • 18. Get involved UserMetrics (private) end-user documentation https://www.mediawiki.org/wiki/UserMetrics/Guide source code http://github.com/wikimedia/analytics-user-metrics WikiMetrics (public, under development) home http://metrics.wmflabs.org source code http://github.com/wikimedia/analytics-wikimetrics mailing list https://lists.wikimedia.org/mailman/listinfo/wikimetrics
  • 19. thank you Dario Taraborelli The UserMetrics API: Measuring participation in Wikimedia projects dario@wikimedia.org @readermeter