• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Crowdsourcing Scientific Work: A Comparative Study of Technologies, Processes, and Outcomes in Citizen Science
 

Crowdsourcing Scientific Work: A Comparative Study of Technologies, Processes, and Outcomes in Citizen Science

on

  • 512 views

Slides from my successful dissertation defense. The research focused on the role of technologies in supporting participation and organizing processes in citizen science projects, and the impacts of ...

Slides from my successful dissertation defense. The research focused on the role of technologies in supporting participation and organizing processes in citizen science projects, and the impacts of these processes on scientific outcomes.

Statistics

Views

Total Views
512
Views on SlideShare
512
Embed Views
0

Actions

Likes
4
Downloads
14
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

Crowdsourcing Scientific Work: A Comparative Study of Technologies, Processes, and Outcomes in Citizen Science Crowdsourcing Scientific Work: A Comparative Study of Technologies, Processes, and Outcomes in Citizen Science Presentation Transcript

  • Crowdsourcing   Scientific  Work A  Comparative  Study  of   Technologies,  Processes,  and  Outcomes   in  Citizen  Science Andrea  Wiggins 11  April,  2012 Kevin  Crowston  (Advisor) Geof  Bowker  (External  Reader) Rick  Bonney Murali  Venkatesh  (Internal  Reader) Jian  Qin John  Burdick  (Chair) Steve  SawyerTuesday, May 15, 12
  • Citizen  Science • Projects  involving  the  public  with  scientists  in  collaborative  research.   -­‐ Crowdsourcing  scientific  work  of  data  collection  and  processing -­‐ Increasingly  ICT-­‐mediated • As  citizen  science  gains  in  popularity,  scientists  need  a  better   understanding  of  how  design  and  management  influence  scientific   outcomes,  particularly  for  ICT-­‐enabled  participation.   • Research  goals   -­‐ Describe  the  phenomenon  of  citizen  science. -­‐ Develop  an  empirically-­‐grounded  framework  that  describes  the  conditions,   processes,  and  products  of  citizen  science  projects.Tuesday, May 15, 12
  • Related  Research • Public  participation  in  science -­‐ Purposes  and  forms  of  engagement public participation -­‐ Informal  science  education,  policy,  STS in science cro r tee g •Irwin;  Bonney  et  al;  Cooper  et  al;  Wilderman so wd- urc n n olu itori ing v n mo • Scientific  collaboration * infrastructure online scientific cyber- communities collaboration -­‐ Broader  context  of  practice •Sonnenwald;  Finholt;  Lawrence  et  al • Online  communities * = citizen science -­‐ Participation  in  virtual  environments •Crowston;  Haythornthwaite;  Preece  &  ShneidermanTuesday, May 15, 12
  • Research  Questions How  do  virtuality  and  technology  alter   organizing  in  citizen  science?   How  do  virtuality  and  technology  shape   participation  in  citizen  science?   How  do  organizing  and  participation  influence   scientific  outcomes  in  citizen  science?  Tuesday, May 15, 12
  • CasesTuesday, May 15, 12
  • Mountain  Watch • Monitoring  alpine  climate  change -­‐ Participation  involves: •Finding  monitoring  plots •Identifying  target  plants  and  their  phenophases •Recording  observations  on  paper •Dropping  off  data  sheet  at  facilities  or  entering  online -­‐ Started  in  2004  by  the  Appalachian  Mountain  Club •Primarily  in  White  Mountains  of  New  Hampshire •Combines  citizen  science  with  other  research  efforts •Intensive  study  of  factors  influencing  data  qualityTuesday, May 15, 12
  • Great  Sunflower  Project • Collecting  data  on  pollinator  service  (bees!) -­‐ Participation  involves: •Planting  sunflowers •Creating  garden  description  on  Drupal  website •Recording  15-­‐minute  observation  samples  on  data  sheet •Online  data  entry -­‐ Started  in  2008  by  a  single  academic  researcher •Collects  data  across  North  America •Very  successful  in  attracting  volunteer  interestTuesday, May 15, 12
  • eBird • Collecting  bird  abundance  and  distribution  data -­‐ Participation  involves: •Choosing  observation  methods •Recording  bird  observations •Entering  observations  and  metadata  online -­‐ Launched  in  2002  by  Cornell  Lab  of  Ornithology   (with  National  Audubon  Society) •World’s  largest  biodiversity  data  set •Receives  between  2.5M  -­‐  3M  observations/month •Data  used  in  both  research  and  decision-­‐making  for  policy  and  land  managementTuesday, May 15, 12
  • Comparative  Case  Selection Criterion Mountain  Watch Great  Sunflower eBird Conservation,   Research,  education,   Mission Research,  education education,  recreation conservation Purpose Scientific   Climate  change  effects   Bird  abundance  &   Plant-­‐bee  relationships interests on  alpine  habitats distribution Intended   Hikers Gardeners Birders Community Institutions Single  nonprofit Academic Nonprofit  partnership Environment Resources 1.5  FTE,  $15K 0.5  FTE,  $13K 4.5  FTE,  $300K Paper Structured  data  sheet Structured  data  sheet Variable  &  optional Organization  website   Open  source  CMS   Purpose-­‐built  software   Technologies Digital section website system Data  access Limited Very  limited ExtensiveTuesday, May 15, 12
  • MethodsTuesday, May 15, 12
  • Data  Collection • Semi-­‐structured  interviews  with  project  organizers -­‐ Sampled  for  maximum  diversity  of  roles  and  perspectives,  with  individuals   from  7  organizations -­‐ Some  longitudinal  interviews,  additional  informal  interviews • Participant  observation -­‐ 300+  hours  of  birding,  3  years  of  sunflowers,  6  days  in  the  White  Mountains -­‐ Listservs,  forums,  beta  testing  interfaces  &  mobile  application -­‐ Extensive  involvement  in  citizen  science  organizer  community • Secondary  data,  documents,  &  artifactsTuesday, May 15, 12
  • Analysis • Concurrent  with  data  collection  and  theory  development -­‐ Iterative  deductive  and  inductive  coding Commitment Sustainability Satisfaction -­‐ Rich  process  models Scientific Contributions Interests Individual -­‐ Concept  diagrams Community Development Resources Scientific • Research  Quality Institutions Knowledge Mission Broader Impacts -­‐ Interviewees  reviewed  transcripts Technologies Science Skills -­‐ Key  informants  reviewed  case  chapters Biography Design Organizing Networks -­‐ Expert  and  peer  review  of  findings Personal Interests Participation -­‐ Audit  trail,  ongoing  memos -­‐ Data  triangulationTuesday, May 15, 12
  • FindingsTuesday, May 15, 12
  • Theoretical  Framework • Iteratively  developed Organizational -­‐ Initial  version  based  on   Emergent States Community literature,  used  to  guide  study Sustainability Individual -­‐ 16  versions  over  3  years Emergent States Organizational Commitment Inputs Individual Individual • Inputs-­‐Moderators-­‐Outputs-­‐ Inputs Roles Organizational Outputs Task Design Outputs Demographics Contributions Knowledge Technology Inputs  structure Design Organization Skills Motivation Individual Processes Satisfaction Learning Communication Innovation Design Joining • Example  of  a  relevant  flow: Contributing Organizational -­‐ Design  &  Organizing  -­‐>   Processes Scientific Participation  -­‐>   Research Volunteer Contributions  -­‐>   Management Data Management Scientific  KnowledgeTuesday, May 15, 12
  • Theoretical  Framework Environment • Iteratively  developed -­‐ Initial  version  based  on   Inputs States Products literature,  used  to  guide  study Project Inputs Sustainability Scientific Outcomes Interests Commitment Scientific -­‐ 16  versions  over  3  years Community Satisfaction Knowledge Broader Resources Impacts Institutions • Inputs-­‐Moderators-­‐Outputs-­‐ Mission Technologies Inputs  structure • Example  of  a  relevant  flow: Individual Inputs Outputs Processes Contributions Skills -­‐ Design  &  Organizing  -­‐>   Science Individual Biography Development Networks Design Participation  -­‐>   Personal Interests Organizing Contributions  -­‐>   Participation Scientific  KnowledgeTuesday, May 15, 12
  • Theoretical  Framework Environment • Iteratively  developed -­‐ Initial  version  based  on   Inputs States Products literature,  used  to  guide  study Project Inputs Sustainability Scientific Outcomes Interests Commitment Scientific -­‐ 16  versions  over  3  years Community Satisfaction Knowledge Broader Resources Impacts Institutions • Inputs-­‐Moderators-­‐Outputs-­‐ Mission Technologies Inputs  structure • Example  of  a  relevant  flow: Individual Inputs Outputs Processes Contributions Skills -­‐ Design  &  Organizing  -­‐>   Science Individual Biography Development Networks Design Participation  -­‐>   Personal Interests Organizing Contributions  -­‐>   Participation Scientific  KnowledgeTuesday, May 15, 12
  • Theoretical  Framework Environment • Iteratively  developed -­‐ Initial  version  based  on   Inputs States Products literature,  used  to  guide  study Project Inputs Sustainability Scientific Outcomes Interests Commitment Scientific -­‐ 16  versions  over  3  years Community Satisfaction Knowledge Broader Resources Impacts Institutions • Inputs-­‐Moderators-­‐Outputs-­‐ Mission Technologies Inputs  structure • Example  of  a  relevant  flow: Individual Inputs Outputs Processes Contributions Skills -­‐ Design  &  Organizing  -­‐>   Science Individual Biography Development Networks Design Participation  -­‐>   Personal Interests Organizing Contributions  -­‐>   Participation Scientific  KnowledgeTuesday, May 15, 12
  • Theoretical  Framework Environment • Iteratively  developed -­‐ Initial  version  based  on   Inputs States Products literature,  used  to  guide  study Project Inputs Sustainability Scientific Outcomes Interests Commitment Scientific -­‐ 16  versions  over  3  years Community Satisfaction Knowledge Broader Resources Impacts Institutions • Inputs-­‐Moderators-­‐Outputs-­‐ Mission Technologies Inputs  structure • Example  of  a  relevant  flow: Individual Inputs Outputs Processes Contributions Skills -­‐ Design  &  Organizing  -­‐>   Science Individual Biography Development Networks Design Participation  -­‐>   Personal Interests Organizing Contributions  -­‐>   Participation Scientific  KnowledgeTuesday, May 15, 12
  • Theoretical  Framework Environment • Iteratively  developed -­‐ Initial  version  based  on   Inputs States Products literature,  used  to  guide  study Project Inputs Sustainability Scientific Outcomes Interests Commitment Scientific -­‐ 16  versions  over  3  years Community Satisfaction Knowledge Broader Resources Impacts Institutions • Inputs-­‐Moderators-­‐Outputs-­‐ Mission Technologies Inputs  structure • Example  of  a  relevant  flow: Individual Inputs Outputs Processes Contributions Skills -­‐ Design  &  Organizing  -­‐>   Science Individual Biography Development Networks Design Participation  -­‐>   Personal Interests Organizing Contributions  -­‐>   Participation Scientific  KnowledgeTuesday, May 15, 12
  • Theoretical  Framework Environment • Iteratively  developed -­‐ Initial  version  based  on   Inputs States Products literature,  used  to  guide  study Project Inputs Sustainability Outcomes ? Commitment ? -­‐ 16  versions  over  3  years Community Satisfaction Broader Resources Impacts Institutions • Inputs-­‐Moderators-­‐Outputs-­‐ Mission Technologies Inputs  structure • Example  of  a  relevant  flow: Individual Inputs Outputs Processes Contributions Skills -­‐ Design  &  Organizing  -­‐>   Biography Networks ? Design Individual Development Participation  -­‐>   Personal Interests Organizing Contributions  -­‐>   Participation Scientific  KnowledgeTuesday, May 15, 12
  • Emergent  Themes 1. Project  design  approaches  that  favor  science  versus  hobbies  for   participation  design 2. Design  and  organizing  implications  of  engaging  communities  of   practice 3. Relationships  between  physical  environment,  technologies,   participant  experiences,  and  data  quality 4. Information  technology  tradeoffs:  helpful  for  scale  and   communication,  challenging  for  usability  and  resources 5. Resources  and  sustainability  relate  to  institutions  and  scale  of   participationTuesday, May 15, 12
  • How  do  virtuality  and  technologies   alter  organizing  in  citizen  science? • Virtuality  is  inherent  and  a  key  benefit,  but  leads  to  questions  about   quality -­‐ “People  would  gravitate  towards  the  really  charismatic  species,  which  in  the   White  Mountains  is  diapensia.  So  people  would  go  out  with  these  diapensia-­‐ tinted  glasses,  and  they’d  see  it  everywhere  and  pass  over  the  least  well-­‐known   species.” • Enables  large-­‐scale  research  that  is  more  like  crowdsourcing  than   other  forms  of  scientific  collaboration -­‐ “If  technology  makes  new  things  available,  you  change  your  focus  to  exploit  it.” • Reduces  coordination  costs  and  improves  quality,  but  ICT  often   unsuited  for  use  in  the  field -­‐ “Someone  entered  in  data  that  said  that  they  saw  a  bee  after  130  minutes,  and  I   think  what  they  were  putting  in  is  that  it  was  at  1:30  in  the  afternoon.”Tuesday, May 15, 12
  • How  do  virtuality  and  technologies   shape  participation  in  citizen  science? • Opens  participation  opportunities  to  larger,  more  diverse  population -­‐ “The  skill  base  varies  from  Master  gardeners  and  beekeepers  to  amateur  first-­‐time   gardeners.  ...  Our  audience  skews  a  little  older.  There  are  far  fewer  schoolchildren  who   participate  than  I  thought  there  might  be.” • Importance  of  place:  geographic  biases  and  autonomy,  functional   constraints  of  and  emotional  relationships  to  place -­‐ “Folks  do  have  a  real  connection  to  these  mountains.  So  to  feel  like  they  can  do   something  to  help  out,  and  to  protect,  and  get  a  handle  on  what  is  actually  happening   up  here  in  the  mountains,  it’s  valuable.” • Leads  to  usability  issues  for  some,  but  can  also  be  rewarding  and  more   scalable -­‐ “Some  people  have  difficulty  printing  out  the  data  form,  and  writing  all  this  stuff  in   while  they’re  observing,  and  taking  it  back,  and  then  entering  it  in.” -­‐ “Let’s  give  them  tools  to  do  what  they  want,  and  they’ll  give  us  all  of  their  data.”Tuesday, May 15, 12
  • How  do  organizing  and  participation  influence   scientific  outcomes  in  citizen  science? • Diverse  types  of  scientific  outcomes  suggest  more  holistic  criteria  for   evaluating  project  success -­‐ “[eBird]  is  just  getting  to  the  point  where  we  are  going  to  see  more  and  more   information  come  out  that  will  help  drive  policy  and  decision-­‐making.” • Keep  participants  happy:  greater  quality  and/or  quantity  of   contributions  improve  outcomes -­‐ “The  more  people  enjoy  the  project  and  get  some  reward  then  the  better  off   you’ll  be  for  sustaining  it.  We’ve  seen  significant  growth  that  hasn’t  slowed   down  since  we  turned  the  switch  on  and  changed  the  way  we  think  about  it.”Tuesday, May 15, 12
  • Limitations  &  Future  Work • Limitations -­‐ Depth  rather  than  breadth -­‐ Focused  primarily  on  organizers • Future  work -­‐ Integrate  findings  and  framework  with  participant-­‐oriented  studies -­‐ Compare  to  entirely  online  citizen  science  projects -­‐ Work  with  organizer  community  to  translate  findings  into  recommendations   for  practiceTuesday, May 15, 12
  • Contributions • Theoretical  framework   -­‐ Complements  and  extends  prior  models -­‐ Foundation  for  future  research  with  room  for  expansion  &  refinement • Case  studies -­‐ In-­‐depth  description  and  comparison • New  prospective  best  practices -­‐ Sustainability  planning  in  context  of  organizations  and  resources -­‐ Aligning  scientific  and  personal  interests  as  much  as  possible -­‐ Making  explicit  links  between  individuals,  communities,  and  organizing -­‐ Engaging  non-­‐scientist  community  members  as  organizersTuesday, May 15, 12
  • Thanks • Committee • Case  study  projects -­‐ Mountain  Watch -­‐ Great  Sunflower  Project -­‐ eBird • Writing  group -­‐ Mohammad  Jarrahi  &   Jaime  Snyder • Everett  Wiggins • U.S.  National  Science  Foundation  Grants  09-­‐43049  &  11-­‐11107Tuesday, May 15, 12