Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The HCI Perspective on IR (DIR2016 Keynote)

1,035 views

Published on

The Human-Computer Interaction perspective on Information Retrieval, Keynote Speech presented at the 15th Dutch-Belgian Information Retrieval Workshop - Nov 25th 2016

Published in: Technology
  • Be the first to comment

The HCI Perspective on IR (DIR2016 Keynote)

  1. 1. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw The HCI Perspective on IR Max L.Wilson DIR2016 Keynote
  2. 2. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw MyTalk • A brief view of how our fields differ • A brief history of IR in HCI • What CHI wants from IR • The Challenges of giving CHI what it wants from IR • Examples of getting there
  3. 3. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw I admire the fundamentals of
 batch-processing IR Queries Right Answers
  4. 4. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Interesting Differences 
 between HCI & IR • IR is build on top of prior work
 - HCI is explore new ideas (not necessarily on top) • IR is identify a rule to improve system performance
 - HCI is to extract a principle of Human Behaviour
 - or User Interface Design rule • IR Research is temporally close to IR implementation 
 - HCI is 10-15y gap All the papers in between wrestle with these tensions
  5. 5. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Microsoft Surface Dial image: mashable.com Fitzmaurice, Ishii, and Buxton. "Bricks: laying the foundations for graspable user interfaces." CHI’95.ACM
  6. 6. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw A History of IR in HCI (not comprehensive) (or rigorous) (actually: IR papers in CHI) (based on the ACM DL’s ranking) http://bit.ly/IR-HCI-Papers
  7. 7. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  8. 8. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw IR papers at CHI • “Information Retrieval” in CHI in ACM DL
 - 5000+ papers sorted* by ‘relevance’
 - of 450,000+ IR papers in whole DL
 • Export gives me most relevant 1018
 - 373 Full Papers, 65 Short Papers, 580 Extended Abstracts
 - of Full Papers, only first 25 tag themselves with IR • Looked at Peer-Reviewed papers in 200 most relevant (N=84)
 - Rejected a further 12 as not really being about IR *Order keeps changing when press ‘next page’ :-/ http://bit.ly/IR-HCI-Papers
  9. 9. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw author 'tle year Jo W. Tombaugh and Sco1 A. McEwen Comparison of Two Informa>on Retrieval Methods on Videotex: Tree-structure Versus Alphabe>cal Directory 1982 G. Fischer and H. Nieper-Lemke Helgon: Extending the Retrieval by Reformula>on Paradigm 1989 Stuart K. Card and George G. Robertson and Jock D. Mackinlay The Informa>on Visualizer, an Informa>on Workspace 1991 Michael Mills and Jonathan Cohen and Yin Yin Wong A Magnifier Tool for Video Data 1992 H. Ulrich Hoppe and Franz Schiele Towards Task Models for Embedded Informa>on Retrieval 1992 Ben Shneiderman and Christopher Williamson and Christopher Ahlberg Dynamic Queries: Database Searching by Direct Manipula>on 1992 Daniel M. Russell and Mark J. Stefik and Peter Pirolli and Stuart K. Card The Cost Structure of Sensemaking 1993 Vicki L. O'Day and Robin Jeffries Orienteering in an Informa>on Landscape: How Informa>on Seekers Get from Here to There 1993 Gene Golovchinsky and Mark Chignell Queries-R-Links: Graphical Markup for Text Naviga>on 1993 The oldest IR papers in CHI http://bit.ly/IR-HCI-Papers
  10. 10. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  11. 11. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw In general, little evidence for the superiority of one method can be found.There were no significant differences in search times, or in the number of pages accessed for the two methods. [BUT] Users switched twice as often from the tree to the directory than from the directory to the tree.Thus, while users started with the tree and directory an equal number of times, the answer was more often obtained with the directory method.
  12. 12. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  13. 13. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  14. 14. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  15. 15. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw The concept goes beyond the usual notion of an information retrieval system to encompass the cost structure of information from secondary storage to immediate use.
  16. 16. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  17. 17. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  18. 18. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  19. 19. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  20. 20. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  21. 21. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw The direct manipulative qualities of incrementability, reversibility, and smooth graphical feedback encourage the user to explore the database, freely manipulating the properties without fear of syntax errors or of getting lost.
  22. 22. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  23. 23. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  24. 24. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  25. 25. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  26. 26. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  27. 27. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw author 'tle year Cita' ons Kerry Rodden and Kenneth R. Wood How Do People Manage Their Digital Photographs? 2003 4261 Kevin Larson and Mary Czerwinski Web Page Design: Implica>ons of Memory, Structure and Scent for Informa>on Retrieval 1998 3884 Elizabeth Sillence and Pam Briggs and Lesley Fishwick and Peter Harris Trust and Mistrust of Online Health Sites 2004 3572 Ed H. Chi and Peter Pirolli and Kim Chen and James Pitkow Using Informa>on Scent to Model User Informa>on Needs and Ac>ons and the Web 2001 3188 Stuart K. Card and George G. Robertson and Jock D. Mackinlay The Informa>on Visualizer, an Informa>on Workspace 1991 2789 Bill N. Schilit and Gene Golovchinsky and Morgan N. Price Beyond Paper: Suppor>ng Ac>ve Reading with Free Form Digital Ink Annota>ons 1998 2570 Jurgen Koenemann and Nicholas J. Belkin A Case for Interac>on: A Study of Interac>ve Informa>on Retrieval Behavior and Effec>veness 1996 2537 Joseph 'Jofish' Kaye and Janet Vertesi and Shari Avery and Allan Dafoe and Shay To Have and to Hold: Exploring the Personal Archive 2006 2262 Daniel M. Russell and Mark J. Stefik and Peter Pirolli and Stuart K. Card The Cost Structure of Sensemaking 1993 2188 The Most Cited IR Papers in CHI http://bit.ly/IR-HCI-Papers
  28. 28. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  29. 29. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Subjects really "liked" the penetrable version that allowed them to manipulate the list of suggested terms.
  30. 30. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Most Recent IR Papers at CHI author 'tle year Marta E. Cecchinato and Abigail Sellen and Milad Shokouhi and Gavin Smyth Finding Email in a Mul>-Account, Mul>-Device World 2016 Deokgun Park and Simranjit Sachar and Nicholas Diakopoulos and Niklas Elmqvist Suppor>ng Comment Moderators in Iden>fying High Quality Online News Comments 2016 Haizi Yu and Biplab Deka and Jerry O. Talton and Ranjitha Kumar Accoun>ng for Taste: Ranking Curators and Content in Social Networks 2016 Saraschandra Karanam and Herre van Oostendorp Age-related Differences in the Content of Search Queries when Reformula>ng 2016 Benedikt Loepp and Katja Herrmanny and Jürgen Ziegler Blended Recommending: Integra>ng Interac>ve Informa>on Filtering and Algorithmic Recommender 2015 Khalil Klouche and Tuukka Ruotsalo and Diogo Cabral and Salvatore Andolina and Designing for Exploratory Search on Touch Devices 2015 Jaimie Y. Park and Neil O'Hare and Rossano Schifanella and Alejandro Jaimes and Chin- A Large-Scale Study of User Image Search Behavior on the Web 2015 Yanir Kleiman and Joel Lanir and Dov Danon and Yasmin Felberbaum and Daniel Cohen-Or DynamicMaps: Similarity-based Browsing Through a Massive Set of Images 2015 Ben Steichen and Luanne Freund Suppor>ng the Modern Polyglot: A Comparison of Mul>lingual Search Interfaces 2015 http://bit.ly/IR-HCI-Papers
  31. 31. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw A few more IR papers at CHI http://bit.ly/IR-HCI-Papers
  32. 32. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  33. 33. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  34. 34. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  35. 35. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  36. 36. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  37. 37. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  38. 38. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Not all these ideas work all the time we know humans do not ‘do’ relevance feedback explicitly
 - even though it would help them
  39. 39. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Image:Yahoo Research Blog
  40. 40. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw add donato paper?
  41. 41. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Google’s Wonder Wheel image: blogoscoped.com
  42. 42. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw What does CHI want from IR? A principle of human behaviour and when it holds true or A user interface idea and when it will work A clear generalisable problem and context being studied and
  43. 43. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Challenges in HCIR • Effect of Document Collection on Recommendations
 - WWW, Collections, Personal Data, Social Media, Multimedia • Effect ofTask Design - Exploratory is extremely vague
 - subjective, comprehensive, comparative, investigative, etc • >> When should we provide support • What to Measure about it
  44. 44. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw https://ils.unc.edu/searchtasks
  45. 45. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw TaskTaxonomy • Goal: Learn • Topic: Conceptually General • Target: Open ended • Target: Multiple items • Target: Uncertainty
 • Goal:Ambiguous • Process: dynamic • Process: Long • Goal: multi-faceted • Goal: not too easy
  46. 46. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Challenges in HCIR • Focus of Searching - Options 
 - WWW, Collections, Personal Data, Social Media, Multimedia • Task Design - Exploratory is extremely vague
 - subjective, comprehensive, comparative, investigative, etc • >> When should we provide support • What to Measure about it
  47. 47. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw IIiX 2010
  48. 48. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Our findings provide further empirical validation toVakkari’s task- based theory of IR [26], [..] when search interfaces provide support to the search activities associated with the search task, effective interaction ensues helping the user address it efficiently. Otherwise, excessive and unrelated search support can hamper progress and distract the searcher.
  49. 49. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw TeevanThumbnails only when refinding
  50. 50. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  51. 51. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Challenges in HCIR • Focus of Searching - Options 
 - WWW, Collections, Personal Data, Social Media, Multimedia • Task Design - Exploratory is extremely vague
 - subjective, comprehensive, comparative, investigative, etc • >> When should we provide support • What to Measure about it
  52. 52. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Challenges in HCIR • Focus of Searching - Options 
 - WWW, Collections, Personal Data, Social Media, Multimedia • Task Design - Exploratory is extremely vague
 - subjective, comprehensive, comparative, investigative, etc • When should we provide support • >> What to Measure about it
  53. 53. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  54. 54. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw tag cloud papersCHI2011
  55. 55. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw CHI2012
  56. 56. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Wilson & Wilson
  57. 57. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Challenges in HCIR • Focus of Searching - Options 
 - WWW, Collections, Personal Data, Social Media, Multimedia • Task Design - Exploratory is extremely vague
 - subjective, comprehensive, comparative, investigative, etc • When should we provide support • >> What to Measure about it
  58. 58. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  59. 59. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw What is evident is that there is a cost-benefit in Search User Interface Design
  60. 60. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw BCI Evaluation for IR • 2009 - Concluded that we needed to understand the Cognitive Load created by Search User Interfaces • 2011 - EuroHCIR paper on it • 2012 - EuroHCIR paper using EEG • 2012 onwards - focusing on fNIRS
  61. 61. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Neuro IR
  62. 62. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  63. 63. Dr Max L.Wilson http://cs.nott.ac.uk/~mlw/ SUI Design + Brain Response Cognitive LoadTheory Total Mental Capacity Simple UI EasyTask
  64. 64. Dr Max L.Wilson http://cs.nott.ac.uk/~mlw/ SUI Design + Brain Response Cognitive LoadTheory Total Mental Capacity Simple UI HardTask
  65. 65. Dr Max L.Wilson http://cs.nott.ac.uk/~mlw/ SUI Design + Brain Response Cognitive LoadTheory Total Mental Capacity Complex UI HardTask
  66. 66. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Three RelatedTheories • Semantic vs Syntactic Load (HCI / CompSci) • Cognitive LoadTheory (Education) • Mental Workload (Human Factors & Psychology)
  67. 67. Dr Max L.Wilson http://cs.nott.ac.uk/~mlw/ Mental Workload the limited resource model [22] describing the relat between the demands of a task, the resources alloca e task and the impact on performance. re 3: Resources available vs task demands act on performance [22] Megaw,T. (2005)The definition and measurement of mental workload. Evaluation of human work, 525-551.
  68. 68. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Erin’s paper
  69. 69. Dr Max L.Wilson http://cs.nott.ac.uk/~mlw/ Measuring Mental Workload Nasa-TLX fMRI EEG fNIRS
  70. 70. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  71. 71. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  72. 72. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw
  73. 73. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Challenges in HCIR • Focus of Searching - Options 
 - WWW, Collections, Personal Data, Social Media, Multimedia • Task Design - Exploratory is extremely vague
 - subjective, comprehensive, comparative, investigative, etc • When should we provide support • >> What to Measure about it
  74. 74. Dr Max L.Wilson http://cs.nott.ac.uk/~pszmw Questions? http://bit.ly/IR-HCI-Papers

×