Wikipedia Survey
Wikimania Buenos Aires
26 August 2009
Collaborative Creativity Group
United Nations University MERIT
Rishab Ghosh, Ruediger Glott, Philipp Schmidt
http://ccg.merit.unu.edu
schmidt@merit.unu.edu
BACKGROUND

• Wikimedia Foundation &
  United Nations University MERIT
• First official Wikipedia survey (for readers and
  contributors)
• Questionnaire developed with community
  input and building on existing research
• Translated into 22 languages by Wikipedia
  community
BACKGROUND

• Online survey hosted at MERIT. Code reviewed
  by WP technical community
• Survey went live November 2008
• Link to survey was posted in page headers of
  WP sites
• Staggered across different language editions
  to deal with traffic loads
RESEARCH QUESTIONS / OBJECTIVES

• Who is contributing to Wikipedia and how?
• Who is using Wikipedia?
• What are users' and contributors' perceptions
  of quality?
• Pragmatic findings that help the WMF improve
  use and benefits.
• Establish baseline for possible monitoring
  system (panel studies)
SCOPE

• Questionnaire contains 50+ questions (with
  sub-questions) on a broad variety of topics
  and is broken down into sections:
  – General, Contributing, Reading, Non-contributors,
    Ex-contributors
• 310,000 users/contributors accessed the
  survey
• 175,000 valid responses
ANALYSIS

• Extensive data cleaning (removed more than
  3500 cases)
• First sub-reports shared with WMF
  – Survey Overview (available via blog)
  – Non-contributors (for WMF presentation)
  – Quality
ANALYSIS - NEXT STEPS

• August 2009 – Share moderately anonymized
  data with WMF and make available additional
  sub-reports
• November 2009 – Publish comprehensive
  survey report (including all sub-reports)
• Post publication – Open access to all fully
  anonymized data
LANGUAGE EDITION SURVEYS

• 22 languages (incl. 2 surveys for chinese)
• Started with largest language editions
• Added further editions based on interest by
  WMF, availability of volunteer translators, and
  diversity of sample
• Top 5 language editions ~ 80% respondents
• Russian largest group (tested against
  manipulation)
LANGUAGE EDITION SURVEYS
LOCATION

• Responses
  from 231
  countries
USER/ACTIVITY TYPES

• Readers 66%
  Contributors 31%

• Contributors:
  4 hrs / week

• Additional
  categorization
  based on focus
  areas
AGE

• Quartile: 18 yrs - 22 yrs - 30 yrs - 85 yrs

                    Type          Avg Age
          All respondents            25.22
          Readers                    24.79
          Contributors               26.14
          Female                     23.79
          Male                       25.69
GENDER

• Gender by user type
• Female: 30% readers, 12.5% contributors
EDUCATION

• High levels of education (esp. given avg ages)
• Contributors slightly higher than readers
  (~ 50% with tertiary education)
MOTIVATIONS TO CONTRIBUTE

• Ranked motivations (1st - 4th)
REASONS FOR NOT CONTRIBUTING
HOW TO INCREASE CONTRIBUTION

• I would be much likelier to contribute, if …
FOCUS AREAS AND EXPERTISE

• Culture & Arts most popular, Technology &
  Applied Sciences (then History, Geography)
• 70-90% of contributors self-identify as
  “experts”
• Highest shares of experts in technical and
  scientific fields
• Focus areas do not correspond perfectly with
  expertise levels. “Geography & Places”
  attracts high levels of contributors, but
  comparatively low levels of expertise.
FOCUS AREAS AND EXPERTISE
PERCEPTIONS OF QUALITY

• Quality compared to “traditional”
  encyclopedia
  – Reliability (only category where “traditional”
    received higher scores)
  – Broadness
  – Variety
  – Depth
  – Understandability
  – Timeliness
• Compare reader and contributor responses
QUALITY - RELIABILITY

• The information provided is correct
QUALITY - DEPTH

• The information provides deep understanding
  of a topic
QUALITY - VARIETY

• A wide range of topics is dealt with
PERCEPTIONS OF QUALITY

• Contributors are both more critical
  (reliability, understandability) and more
  supportive (all other dimensions) than
  readers.
• Relationship between transparency,
  understanding of the processes and
  mechanisms, and perception of quality.
ANNEX – ADDITIONAL TABLES
MOTIVATIONS TO CONTRIBUTE
REASONS FOR NOT CONTRIBUTING

UNU MERIT Wikipedia Survey

  • 1.
    Wikipedia Survey Wikimania BuenosAires 26 August 2009 Collaborative Creativity Group United Nations University MERIT Rishab Ghosh, Ruediger Glott, Philipp Schmidt http://ccg.merit.unu.edu schmidt@merit.unu.edu
  • 2.
    BACKGROUND • Wikimedia Foundation& United Nations University MERIT • First official Wikipedia survey (for readers and contributors) • Questionnaire developed with community input and building on existing research • Translated into 22 languages by Wikipedia community
  • 3.
    BACKGROUND • Online surveyhosted at MERIT. Code reviewed by WP technical community • Survey went live November 2008 • Link to survey was posted in page headers of WP sites • Staggered across different language editions to deal with traffic loads
  • 4.
    RESEARCH QUESTIONS /OBJECTIVES • Who is contributing to Wikipedia and how? • Who is using Wikipedia? • What are users' and contributors' perceptions of quality? • Pragmatic findings that help the WMF improve use and benefits. • Establish baseline for possible monitoring system (panel studies)
  • 5.
    SCOPE • Questionnaire contains50+ questions (with sub-questions) on a broad variety of topics and is broken down into sections: – General, Contributing, Reading, Non-contributors, Ex-contributors • 310,000 users/contributors accessed the survey • 175,000 valid responses
  • 6.
    ANALYSIS • Extensive datacleaning (removed more than 3500 cases) • First sub-reports shared with WMF – Survey Overview (available via blog) – Non-contributors (for WMF presentation) – Quality
  • 7.
    ANALYSIS - NEXTSTEPS • August 2009 – Share moderately anonymized data with WMF and make available additional sub-reports • November 2009 – Publish comprehensive survey report (including all sub-reports) • Post publication – Open access to all fully anonymized data
  • 8.
    LANGUAGE EDITION SURVEYS •22 languages (incl. 2 surveys for chinese) • Started with largest language editions • Added further editions based on interest by WMF, availability of volunteer translators, and diversity of sample • Top 5 language editions ~ 80% respondents • Russian largest group (tested against manipulation)
  • 9.
  • 10.
    LOCATION • Responses from 231 countries
  • 11.
    USER/ACTIVITY TYPES • Readers66% Contributors 31% • Contributors: 4 hrs / week • Additional categorization based on focus areas
  • 12.
    AGE • Quartile: 18yrs - 22 yrs - 30 yrs - 85 yrs Type Avg Age All respondents 25.22 Readers 24.79 Contributors 26.14 Female 23.79 Male 25.69
  • 13.
    GENDER • Gender byuser type • Female: 30% readers, 12.5% contributors
  • 14.
    EDUCATION • High levelsof education (esp. given avg ages) • Contributors slightly higher than readers (~ 50% with tertiary education)
  • 15.
    MOTIVATIONS TO CONTRIBUTE •Ranked motivations (1st - 4th)
  • 16.
    REASONS FOR NOTCONTRIBUTING
  • 17.
    HOW TO INCREASECONTRIBUTION • I would be much likelier to contribute, if …
  • 18.
    FOCUS AREAS ANDEXPERTISE • Culture & Arts most popular, Technology & Applied Sciences (then History, Geography) • 70-90% of contributors self-identify as “experts” • Highest shares of experts in technical and scientific fields • Focus areas do not correspond perfectly with expertise levels. “Geography & Places” attracts high levels of contributors, but comparatively low levels of expertise.
  • 19.
    FOCUS AREAS ANDEXPERTISE
  • 20.
    PERCEPTIONS OF QUALITY •Quality compared to “traditional” encyclopedia – Reliability (only category where “traditional” received higher scores) – Broadness – Variety – Depth – Understandability – Timeliness • Compare reader and contributor responses
  • 21.
    QUALITY - RELIABILITY •The information provided is correct
  • 22.
    QUALITY - DEPTH •The information provides deep understanding of a topic
  • 23.
    QUALITY - VARIETY •A wide range of topics is dealt with
  • 24.
    PERCEPTIONS OF QUALITY •Contributors are both more critical (reliability, understandability) and more supportive (all other dimensions) than readers. • Relationship between transparency, understanding of the processes and mechanisms, and perception of quality.
  • 25.
  • 26.
  • 27.
    REASONS FOR NOTCONTRIBUTING