Mutually Linked StudiesBalancing Threats to Internal and Ecological Validity in InfoVis Evaluation<br />Niklas ElmqvistSch...
The Problem of Validity<br />InfoVis evaluation often comes in two flavors:<br />Rigorous + unrealistic<br />Scientific ri...
Example: The ColorLens<br />3<br />[McDonnel & Elmqvist 2009]<br />
Example: Evaluating the ColorLens<br />Realistic photographs:<br />Ecologically valid (+)<br />Learning effects (-)<br />C...
Mutually Linked Studies<br />Why choose one type of study? Do both! <br />Idea: Mutually linked studies<br />1 x toy study...
Questions?<br />Acknowledgments: Color Lens co-authors(coming soon to a journal near you)Pierre DragicevicHM Jean-Daniel F...
Validity Primer<br />Several different types of validity…<br />I will discuss the following here:<br />Internal validity<b...
Upcoming SlideShare
Loading in …5
×

Mutually Linked Studies - Balancing Threats to Internal and Ecological Validity in InfoVis Evaluation.

484 views

Published on

Presenter: Niklas Elmqvist
BELIV 2010 Workshop
http://www.beliv.org/beliv2010/

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
484
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Mutually Linked Studies - Balancing Threats to Internal and Ecological Validity in InfoVis Evaluation.

  1. 1. Mutually Linked StudiesBalancing Threats to Internal and Ecological Validity in InfoVis Evaluation<br />Niklas ElmqvistSchool of Electrical & Computer EngineeringPurdue University<br />
  2. 2. The Problem of Validity<br />InfoVis evaluation often comes in two flavors:<br />Rigorous + unrealistic<br />Scientific rigor<br />Toy dataset<br />=> Lacking in ecological validity<br />Realistic + ad hoc<br />Real dataset<br />Imperfect experimental design<br />=> Lacking in internal validity<br />2<br />
  3. 3. Example: The ColorLens<br />3<br />[McDonnel & Elmqvist 2009]<br />
  4. 4. Example: Evaluating the ColorLens<br />Realistic photographs:<br />Ecologically valid (+)<br />Learning effects (-)<br />Cannot control images (-)<br />Does not “feel” canonical! (-)<br />Perlin Noise:<br />Full control over dataset (+)<br />Eliminate learning (+)<br />Canonical and abstract (+)<br />Not realistic! (-)<br />4<br />?<br />
  5. 5. Mutually Linked Studies<br />Why choose one type of study? Do both! <br />Idea: Mutually linked studies<br />1 x toy study – canonical and exhaustive<br />1 x realistic study – prove it works<br />Complement each other<br />Recycle much of the mechanics for both<br />Experimental design<br />Tasks<br />Analysis<br />Participants (!)<br />5<br />Reduces time and paper space requirements for conducting two studies (they are not separate)!<br />
  6. 6. Questions?<br />Acknowledgments: Color Lens co-authors(coming soon to a journal near you)Pierre DragicevicHM Jean-Daniel Fekete<br />6<br />
  7. 7. Validity Primer<br />Several different types of validity…<br />I will discuss the following here:<br />Internal validity<br />Degree to which the outcome is a function of the controlled parameters of experiment<br />Ecological validity<br />Degree to which the results of the experiment can be applied to realistic situations<br />7<br />

×