• Like
  • Save
Evaluating crowdsourcing websites
Upcoming SlideShare
Loading in...5
×
 

Evaluating crowdsourcing websites

on

  • 417 views

NDF Barcamp, Hamilton City Library, Hamilton, New Zealand

NDF Barcamp, Hamilton City Library, Hamilton, New Zealand

Statistics

Views

Total Views
417
Views on SlideShare
412
Embed Views
5

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 5

https://twitter.com 5

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Evaluating crowdsourcing websites Evaluating crowdsourcing websites Presentation Transcript

    • How effectively are crowdsourcingwebsites supporting volunteerparticipation and quality contribution?Donelle McKinleyPhD candidate, School of Information ManagementVictoria University of WellingtonNDF Barcamp, Hamilton, 21 June 2013www.digitalglam.org @donellemckinley
    • Crowdsourcing: a short versionCrowdsourcing outsources taskstraditionally performed by specificindividuals to a group of people orcommunity through an open call (Howe,2009).
    • Crowdsourcing: a long version“Crowdsourcing is a type of participative online activity in which anindividual, an institution, a non-profit organization, or companyproposes to a group of individuals of varying knowledge,heterogeneity, and number, via a flexible open call, the voluntaryundertaking of a task. The undertaking of the task, of variablecomplexity and modularity, and in which the crowd shouldparticipate bringing their work, money, knowledge and/orexperience, always entails mutual benefit. The user will receive thesatisfaction of a given type of need, be it economic, socialrecognition, self-esteem, or the development of individual skills,while the crowdsourcer will obtain and utilize to their advantagethat what the user has brought to the venture, whose form willdepend on the type of activity undertaken.”(Estellés-Arolas & González-Ladrón-de-Guevara, 2012)
    • Potential benefits of crowdsourcingfor the institutionO Continue tradition of volunteerismO Market and stimulate interest in collectionsO Signal the institutions openess and approachabilityO Achieve goals otherwise too costly and labour-intensiveO Better reflect the diversity of visitorsO Tap into expertise outside the institutionO Engage visitors in new waysO Raise the profile of researchO Demonstrate relevanceO Enable new research questions to be explored
    • Common motivations for volunteer participationO The size of the challengeO The necessity for volunteer contributionO Collaboration with prestigious institutionsO Contribution to researchO EducationO Mental stimulationO Being part of a communityO Personal research interestsO Enhancing a resource from which they will benefit
    • The potential of the crowdDigitalkoot (National Library of Finland)O First 51 days of the projectO 31, 816 visitors to the siteO 15% participatedTranscribe Bentham (University College London)O First 6 months of the projectO 1,207 visitors registered to participateO 21% participated
    • Three common scenariosThe website either:O Follows the structure of the underlyingtechnology or the organisationO Adheres to familiar conventionsO Is the product of personal preferenceGarrett, J. (2011). The Elements of User Experience: User-centereddesign for the web. USA: Aiga/New Riders.
    • How effectively does your website…?O Define its objectivesO Reflect visitor motivationsO Align visitor motivations with relevantincentivesO Minimize sources of frustration andconcern
    • How are they impacting on the effectivenessof your website?O ContentO LanguageO ReadabilityO Website navigationO Arrangement of page elementsO ConsistencyO Visual appearanceO Page load speedO Number and complexity of processes to completethe desired action
    • O Prototype: an early sample or model built to test a conceptor processO Pilot: a small-scale preliminary experiment conducted toevaluate feasibility, time, cost, adverse events, and improvedesign prior to the launch of a full-scale projectO Soft-launch: the release of a website to a limited audience,in order to (beta) test and tweak a design before beinglaunched to a wider audienceO Beta testing: user testing by a limited audience to ensurethe website/software has few faults or bugs, and gatheruser feedbackO Optimization: increasing the percentage of visitors thatfulfill the objective of a webpage or website
    • User evaluation: A different kind of userengagementO Focus groupsO SurveysO Usability testingO Ongoing feedback channels
    • Analytics:How is website design impacting on…?O The number of online visitorsO Time spent on siteO The number of online visitors who register toparticipateO The number of online visitors who actuallyparticipateO The number of abandoned and completed tasksO The number of return visitors
    • OCR text correction: Trove, National Library of Australia
    • Evaluation of the Trove crowdsourcinguser interface involved:O Asking potential volunteers to comment on the prototypeO Sitting potential volunteers in front of the computerscreen and asking them how they would complete thetaskO A soft launch and beta testingO Gathering feedback on the beta version from over 600users over the course of five months, via a survey, anonline contact form, direct observation of user activity,analytics, online comments, and direct contact withusers via email and phone
    • Evaluation of the Transcribe Bentham taskinterface involved:O Beta testingO A user surveyO Website analyticsO Analysis of user interaction statisticsO Comparisons with studies oncrowdsourcing volunteer behaviour
    • UK Reading Experience Database (UK-RED)
    • Evaluation of the UK-RED task interfaceinvolved:O Heuristic evaluationO A survey of current and potentialcontributorsO Comparison with other crowdsourcingtask interfacesO Comparison with recommended practiceas outlined in crowdsourcing and HCIliterature
    • Usability and functionality requirements fora NZ-RED task interfaceO Minimize user effortO Support integration of the task with researchprocessesO Enable new visitors and contributors to understandwhat the task involves quickly and easilyO Support accurate and controlled data entryO Be easy to use for people reasonably confident withthe WebO Support flexible, structured data entryO Support bilingual data entry
    • Challenges impacting on evaluationO FundingO ExpertiseO Organisational culture
    • What could a more effective website meanfor the project?O More online visitors participateO Tasks are completed more efficientlyO Tasks are completed with greateraccuracyO The task is more enjoyableO Volunteers participate more oftenO The project is more cost-effectiveO More volunteers are willing to participatein future projects
    • Thanks!For references and other great reads visithttp://www.digitalglam.org/crowdsourcing/books/Presentation and slides will be available athttp://www.digitalglam.org/crowdsourcing/talks/For crowdsourcing research updates followwww.digitalglam.org @donellemckinley