Replication is more than Duplication: Position slides for CHI2011 panel on replication of HCI research
CHI 2011 Panel | RepliCHI – Should CHI be Replicating and Validating Results More? More to Replication than Simply Duplication Ed H. Chi Google Research 1
A historical note about one HCI POV n Early fundamental contributions from: n Computer scientists interested in changing how we interact with information n Psychologists interested in the implications of these changes – Particularly Experimental Cognitive Psychologists 2
Newell’s Memo n The need to establish HCI as a science – Adopt methods from psychology – Dual purpose: understand nature of human behavior and build up a science of HCI techniques. n Intellectual Heritage from V. Bush and Licklider – Augmenting Cognition 4
Replication is the Norm in Psychology n Stanford Psychology Grad Students – Part of their training program is to learn the experimental method – Find an psychological eﬀect and experiment to replicate – And write a report. – Must pass to continue n Often becomes the basis for later Ph.D. research 5
x.com card @parc.xerox.com firstname.lastname@example.org laying out the hierarchy in a uniform way on an imaginary hyperbolic plane and then mapping this to the Euclidian space Hyperbolic Browser Won! ualizations have sought to ncreasing the amount of of the display region. The hierarchy can be mouse-dragged through the central display region to bring new parts into theo the user. We study how the focus, or nodes can be mouse-clicked to bring them to the center n End of Story? Hyperbolic Tree browser of the focus. ior in a task similar to the No! son to a more conventional n more nodes, searched at a earning. However, the /C, found to be highly affected cues to the value of distal ent made hyperbolic searchwser. Conversely, weak scent antage. There appears to be ing visual attention in these t expands the spotlight ofets in the compressed regionThe results suggest design yperbolic tree, Information F i g u r e 1. The Hyperbolic browseration research is to discover 1.2 Information scent and visual search 7
0.30, 0.35, and Question Type factor into just two levels: Retrieval and Comparison.through (a) a Table 1. Mean performance times in E x p e r i m e n t 1 b y test phase, and task type and browser.hase participants browsers basic Question Type Browser amiliar with the Explorer Hyperbolic the tasks in the (sec) (sec) Retrieval Tasksomfort with thee practice phase Simple 35.55 34.37actice tasks with rowser. Each of Complex 41.55 42.02ks. Experimental All retrieval 38.55 38.20 rticipants began c browser. Comparison Taskss were tracked, Local 42.78 41.91 system. A brief Global 71.07 73.19 system along a the midpoint of All comparison 56.93 57.55 every set of 14 by having the All questions 47.74 47.87eye-tracking hadted. The n Individual diﬀerences was better explanation than browser design! subjecteting each set of Browsers. There was no significant difference in the overall – Spearman’s rank correlation = 0.78, and Explorer browsers [F(1, task times between the Hyperbolic p<.01 832) = 0.14, M S E = 0.27]. Our failure to find an overally as the practice difference between the two browsers was somewhat surprising. 8 each were used
$0. 45, 40. 3S-~.w 15 10 5 o Low Scent High ~ e n tFigure 5. Mean performance times for retrieval t a s k sas a function of browser and information scent.Effect of Information Scent on Eye Movements. "&recan see 9
progressed 350 plateau on + Explorer 300 i X Hyperbolic fixations a 25O O ,I 200 x L ~5o E z 1oo 50 Low Scent High Scent (a)Figure 6. Number of fixations in Experiment 1 as afunction of browser and information scent.Scent-finding and scent-following.To understand more about 10how the browser designs affected users eye movements, more
Hyperbolic browser requires only 0.92 sec/level compared to1.75 sec/level or 53% as long. 8" ............ ~T~.E~Bp.~.!C. BRp~ER....................... j . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6. = 0.92 so.Tel ~ m > __ .J 2 ~ " ~ - ~ 1 . 7 5 see/le vel ............... 1 ~ . . - E . X P E O R E R BROWSER 0 0.00 2.00 4.00 6.00 8.00 10.00 Time (See)Figure 12. High-scent search 11
The ﬁrst experiments are often just the beginning… n Replication often reveal further interesting stories. n New context, new environments, new subjects, and other variables shed light on the complexity of human behavior and how it interacts with HCI systems. 12
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.