Kurt Luther, Georgia Institute of Technology
Scott Counts, Microsoft Research
Kristin B. Stecher, Microsoft Research
Aaron Hoff, Microsoft Research
Paul Johns, Microsoft Research
1

2

3

Comprehension

◦ Preferred reading with
Pathfinder
◦ Felt more confident in
their answers
 But weren‟t faster or
more accurate
* is significant (4 here)
4

Frank M. Chapman
(1864 - 1945)
5

27 participants

90 species

18,500 birds

1900
6

27 participants 59,918 participants
 
2,267 species
90 species 

57,704,250 birds
18,500 birds 

1900 2007
7

Definition

◦ Non-scientist volunteers participating as field
assistants in scientific studies [Cohn 2008]
Many fields

◦ Ornithology, astronomy, oncology, entomology, etc.
Two main goals

◦ Data collection
◦ Engage public in scientific process
8

Christmas Bird Count
9

300+ scientific publications
10

Definition

◦ Non-scientist volunteers participating as field
assistants in scientific studies [Cohn 2008]
Many fields

◦ Ornithology, astronomy, oncology, entomology, etc.
Two main goals


◦ Data collection
?
◦ Engage public in scientific process
11

Scientific Process
Observations Observations 
Data collection Data collection 
Questions Questions ?
Hypotheses Hypotheses ?
Analyses Analyses ?
Discussions Discussions ?
Conclusions Conclusions ?
12

Pathfinder – home page
13

Pathfinder – home page
14

Track (user-generated)
15

Track (scraper-generated)
16

Pathfinder – home page
17

Pathfinder – home page
18

Discussion page – summary section
19

Discussion page – summary section
20

Discussion page – “con” evidence highlighted
21

Discussion page – responses section
22

Discussion page – embedded track expanded
23

Discussion page – editing responses
24

Discussion page – editing responses
25

Can Pathfinder help citizen scientists engage

more deeply with their data?
What would citizen scientists do with

Pathfinder?
◦ Would they be able to use it?
◦ Would they enjoy using it?
How does Pathfinder compare with existing

online collaboration tools?
26

15 participants

◦ Used their own office and computer
2 conditions (within-subjects)

◦ Pathfinder and standard wiki
4 tasks

◦ 2 editing, 2 comprehension
4 topics

Global – Kyoto protocol
◦
Global – Climate change
◦
Local – Seattle traffic and temperature
◦
Local – Seattle traffic and pollution
◦
27

Pathfinder condition Standard wiki condition
28

During each task

◦ Observational notes
◦ Log files
◦ Written responses
After each task

◦ Survey
End of each experiment

◦ General questionnaire
◦ Structured interview
29

Editing

◦ No significant
differences between
systems
* is significant (none here)
30

Comprehension

◦ Preferred reading with
Pathfinder
◦ Felt more confident in
their answers
 But weren‟t faster or
more accurate
* is significant (4 here)
31

Overall

◦ 14 of 15 preferred
Pathfinder to standard
wiki
 1 tie
32

Milestones and data sets

◦ Added all milestone types
 Placeholders/outlining tools
◦ Embedding and exporting data
 Embedding vs. tabbed browsing
◦ Used overview to read the document
“Without the overview it‟s hard to read
at all.” –P7
“I like the inclusion of predictions…
Having them in the template almost
makes you feel free to blue sky. Just
having that little label there.” –P5
33

Accuracy

◦ Argumentation while editing, not writing
Attribution

◦ Wanting to “call out” others but being unable
“If I‟m trying to respond to something, to address that person,
you can‟t really do that without saying, „In the previous
entry…‟” –P8
34

Design considerations for fostering original

research
◦ Attribution
 Scientific community vs. Wikipedia
[Forte & Bruckman 2005]
35

Article (wiki-style) Discussion (forum-style)
[Forte & Bruckman 2005]
36

Design considerations for fostering original

research
◦ Meta-discussions
 Commenting
◦ Social translucence [Erickson & Kellogg 2000]
 Awareness, visibility, accountability
 High level summary  detailed histories
37

Engaging novice and expert citizen scientists

◦ Milestones: the tradeoff for improved
comprehension
 More work to edit, easier to read
◦ Drawing in novices
 Prediction and To-do milestones
 Heterogeneous granularity [Benkler 2002]
 Legitimate peripheral participation [Lave & Wenger 1991]
◦ Supporting expert citizens and professional
scientists
 Rewards and reuse
38

Traditional citizen science rarely goes beyond

data collection.
Pathfinder helps citizen scientists

collaboratively discuss and analyze their data.
Pathfinder users were able to, and enjoyed,

deeper engagement in the scientific process.
Citizen scientists need special tools because

they generate original research.
39

Thank you

◦ Microsoft Research
 Scott Counts, Kristin Stecher, Aaron Hoff, Paul Johns
◦ Georgia Tech
 Craig Tashman, Eugene Medynskiy, Lana Yarosh, Sarita
Yardi, Kelly Caine
◦ Photo credits
 evelynishere on Flickr
Contact me

◦ Kurt Luther: luther@cc.gatech.edu
40