Your SlideShare is downloading. ×
Replication is more than Duplication: Position slides for CHI2011 panel on replication of HCI research
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Replication is more than Duplication: Position slides for CHI2011 panel on replication of HCI research


Published on

Slides used at the CHI2011 panel on replication of HCI research. …

Slides used at the CHI2011 panel on replication of HCI research.

Published in: Technology, Business

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. CHI 2011 Panel | RepliCHI – Should CHI be Replicating and Validating Results More? More to Replication than Simply Duplication Ed H. Chi Google Research 1
  • 2. A  historical  note  about  one  HCI  POV  n  Early  fundamental  contributions  from:  n  Computer  scientists  interested  in   changing  how  we  interact  with   information  n  Psychologists  interested  in  the   implications  of  these  changes   –  Particularly  Experimental  Cognitive   Psychologists   2
  • 3. 3
  • 4. Newell’s  Memo  n  The  need  to  establish  HCI  as  a   science   –  Adopt  methods  from  psychology   –  Dual  purpose:  understand  nature   of  human  behavior  and  build  up   a  science  of  HCI  techniques.  n  Intellectual  Heritage  from  V.   Bush  and  Licklider   –  Augmenting  Cognition   4
  • 5. Replication  is  the  Norm  in  Psychology  n  Stanford  Psychology  Grad  Students     –  Part  of  their  training  program  is  to  learn  the  experimental   method   –  Find  an  psychological  effect  and  experiment  to  replicate   –  And  write  a  report.   –  Must  pass  to  continue  n  Often  becomes  the  basis  for  later  Ph.D.  research   5
  • 6. The  CHI97  Browse-­‐Off   Photo:  Ben  Shneiderman   6
  • 7. card laying out the hierarchy in a uniform way on an imaginary hyperbolic plane and then mapping this to the Euclidian space Hyperbolic  Browser  Won!   ualizations have sought to ncreasing the amount of of the display region. The hierarchy can be mouse-dragged through the central display region to bring new parts into theo the user. We study how the focus, or nodes can be mouse-clicked to bring them to the center n End  of  Story?   Hyperbolic Tree browser of the focus. ior in a task similar to the No!  son to a more conventional n  more nodes, searched at a earning. However, the /C, found to be highly affected cues to the value of distal ent made hyperbolic searchwser. Conversely, weak scent antage. There appears to be ing visual attention in these t expands the spotlight ofets in the compressed regionThe results suggest design yperbolic tree, Information F i g u r e 1. The Hyperbolic browseration research is to discover 1.2 Information scent and visual search 7
  • 8. 0.30, 0.35, and Question Type factor into just two levels: Retrieval and Comparison.through (a) a Table 1. Mean performance times in E x p e r i m e n t 1 b y test phase, and task type and browser.hase participants browsers basic Question Type Browser amiliar with the Explorer Hyperbolic the tasks in the (sec) (sec) Retrieval Tasksomfort with thee practice phase Simple 35.55 34.37actice tasks with rowser. Each of Complex 41.55 42.02ks. Experimental All retrieval 38.55 38.20 rticipants began c browser. Comparison Taskss were tracked, Local 42.78 41.91 system. A brief Global 71.07 73.19 system along a the midpoint of All comparison 56.93 57.55 every set of 14 by having the All questions 47.74 47.87eye-tracking hadted. The n  Individual  differences  was  better  explanation  than  browser  design!   subjecteting each set of Browsers. There was no significant difference in the overall –  Spearman’s  rank  correlation  =  0.78,  and Explorer browsers [F(1, task times between the Hyperbolic p<.01   832) = 0.14, M S E = 0.27]. Our failure to find an overally as the practice difference between the two browsers was somewhat surprising. 8 each were used
  • 9. $0. 45, 40. 3S-~.w 15 10 5 o Low Scent High ~ e n tFigure 5. Mean performance times for retrieval t a s k sas a function of browser and information scent.Effect of Information Scent on Eye Movements. "&recan see 9
  • 10. progressed 350 plateau on + Explorer 300 i X Hyperbolic fixations a 25O O ,I 200 x L ~5o E z 1oo 50 Low Scent High Scent (a)Figure 6. Number of fixations in Experiment 1 as afunction of browser and information scent.Scent-finding and scent-following.To understand more about 10how the browser designs affected users eye movements, more
  • 11. Hyperbolic browser requires only 0.92 sec/level compared to1.75 sec/level or 53% as long. 8" ............ ~T~.E~Bp.~.!C. BRp~ER....................... j . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6. = 0.92 so.Tel ~ m > __ .J 2 ~ " ~ - ~ 1 . 7 5 see/le vel ............... 1 ~ . . - E . X P E O R E R BROWSER 0 0.00 2.00 4.00 6.00 8.00 10.00 Time (See)Figure 12. High-scent search 11
  • 12. The  first  experiments  are  often  just  the  beginning…  n  Replication  often  reveal  further  interesting  stories.  n  New  context,  new  environments,  new  subjects,  and   other  variables  shed  light  on  the  complexity  of  human   behavior  and  how  it  interacts  with  HCI  systems.   12