CHI 2011 Panel | RepliCHI – Should CHI be Replicating and Validating Results More?		More to Replication than Simply Duplic...
A	  historical	  note	  about	  one	  HCI	  POV	  n    Early	  fundamental	  contributions	  from:	  n    Computer	  sci...
3
Newell’s	  Memo	  n    The	  need	  to	  establish	  HCI	  as	  a	        science	         –  Adopt	  methods	  from	  ps...
Replication	  is	  the	  Norm	  in	  Psychology	  n    Stanford	  Psychology	  Grad	  Students	  	         –  Part	  of	 ...
The	  CHI97	  Browse-­‐Off	                                    Photo:	  Ben	  Shneiderman	                                 ...
x.com               card @parc.xerox.com                    mija@psych.stanford.com                                       ...
0.30, 0.35, and         Question Type factor into just two levels: Retrieval and                         Comparison.throug...
$0.    45,    40.    3S-~.w    15    10     5     o             Low Scent             High ~ e n tFigure 5. Mean performan...
progressed       350                                                    plateau on                                     +  ...
Hyperbolic browser requires only 0.92 sec/level compared to1.75 sec/level or 53% as long.      8"           ............  ...
The	  first	  experiments	  are	  often	  just	  the	  beginning…	  n    Replication	  often	  reveal	  further	  interest...
Upcoming SlideShare
Loading in...5
×

Replication is more than Duplication: Position slides for CHI2011 panel on replication of HCI research

771

Published on

Slides used at the CHI2011 panel on replication of HCI research.
#RepliCHI

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
771
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Replication is more than Duplication: Position slides for CHI2011 panel on replication of HCI research

  1. 1. CHI 2011 Panel | RepliCHI – Should CHI be Replicating and Validating Results More? More to Replication than Simply Duplication Ed H. Chi Google Research 1
  2. 2. A  historical  note  about  one  HCI  POV  n  Early  fundamental  contributions  from:  n  Computer  scientists  interested  in   changing  how  we  interact  with   information  n  Psychologists  interested  in  the   implications  of  these  changes   –  Particularly  Experimental  Cognitive   Psychologists   2
  3. 3. 3
  4. 4. Newell’s  Memo  n  The  need  to  establish  HCI  as  a   science   –  Adopt  methods  from  psychology   –  Dual  purpose:  understand  nature   of  human  behavior  and  build  up   a  science  of  HCI  techniques.  n  Intellectual  Heritage  from  V.   Bush  and  Licklider   –  Augmenting  Cognition   4
  5. 5. Replication  is  the  Norm  in  Psychology  n  Stanford  Psychology  Grad  Students     –  Part  of  their  training  program  is  to  learn  the  experimental   method   –  Find  an  psychological  effect  and  experiment  to  replicate   –  And  write  a  report.   –  Must  pass  to  continue  n  Often  becomes  the  basis  for  later  Ph.D.  research   5
  6. 6. The  CHI97  Browse-­‐Off   Photo:  Ben  Shneiderman   6
  7. 7. x.com card @parc.xerox.com mija@psych.stanford.com laying out the hierarchy in a uniform way on an imaginary hyperbolic plane and then mapping this to the Euclidian space Hyperbolic  Browser  Won!   ualizations have sought to ncreasing the amount of of the display region. The hierarchy can be mouse-dragged through the central display region to bring new parts into theo the user. We study how the focus, or nodes can be mouse-clicked to bring them to the center n End  of  Story?   Hyperbolic Tree browser of the focus. ior in a task similar to the No!  son to a more conventional n  more nodes, searched at a earning. However, the /C, found to be highly affected cues to the value of distal ent made hyperbolic searchwser. Conversely, weak scent antage. There appears to be ing visual attention in these t expands the spotlight ofets in the compressed regionThe results suggest design yperbolic tree, Information F i g u r e 1. The Hyperbolic browseration research is to discover 1.2 Information scent and visual search 7
  8. 8. 0.30, 0.35, and Question Type factor into just two levels: Retrieval and Comparison.through (a) a Table 1. Mean performance times in E x p e r i m e n t 1 b y test phase, and task type and browser.hase participants browsers basic Question Type Browser amiliar with the Explorer Hyperbolic the tasks in the (sec) (sec) Retrieval Tasksomfort with thee practice phase Simple 35.55 34.37actice tasks with rowser. Each of Complex 41.55 42.02ks. Experimental All retrieval 38.55 38.20 rticipants began c browser. Comparison Taskss were tracked, Local 42.78 41.91 system. A brief Global 71.07 73.19 system along a the midpoint of All comparison 56.93 57.55 every set of 14 by having the All questions 47.74 47.87eye-tracking hadted. The n  Individual  differences  was  better  explanation  than  browser  design!   subjecteting each set of Browsers. There was no significant difference in the overall –  Spearman’s  rank  correlation  =  0.78,  and Explorer browsers [F(1, task times between the Hyperbolic p<.01   832) = 0.14, M S E = 0.27]. Our failure to find an overally as the practice difference between the two browsers was somewhat surprising. 8 each were used
  9. 9. $0. 45, 40. 3S-~.w 15 10 5 o Low Scent High ~ e n tFigure 5. Mean performance times for retrieval t a s k sas a function of browser and information scent.Effect of Information Scent on Eye Movements. "&recan see 9
  10. 10. progressed 350 plateau on + Explorer 300 i X Hyperbolic fixations a 25O O ,I 200 x L ~5o E z 1oo 50 Low Scent High Scent (a)Figure 6. Number of fixations in Experiment 1 as afunction of browser and information scent.Scent-finding and scent-following.To understand more about 10how the browser designs affected users eye movements, more
  11. 11. Hyperbolic browser requires only 0.92 sec/level compared to1.75 sec/level or 53% as long. 8" ............ ~T~.E~Bp.~.!C. BRp~ER....................... j . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6. = 0.92 so.Tel ~ m > __ .J 2 ~ " ~ - ~ 1 . 7 5 see/le vel ............... 1 ~ . . - E . X P E O R E R BROWSER 0 0.00 2.00 4.00 6.00 8.00 10.00 Time (See)Figure 12. High-scent search 11
  12. 12. The  first  experiments  are  often  just  the  beginning…  n  Replication  often  reveal  further  interesting  stories.  n  New  context,  new  environments,  new  subjects,  and   other  variables  shed  light  on  the  complexity  of  human   behavior  and  how  it  interacts  with  HCI  systems.   12
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×