Marianne Lykkes presentation at ASIS&T Conference

663 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
663
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Marianne Lykkes presentation at ASIS&T Conference

  1. 1. PERSPECTIVES ON ADAPTIVITY IN INFORMATION RETRIEVAL INTERACTION anASIS&T 2010 annual meeting panel PAIRI
  2. 2. PAIRI Agenda/Outline  Adaptivity in IR Interaction  Four Dimensions  Pecha-Kucha Presentation and Timeline  Audience Interaction!  Wrapping Up
  3. 3. PAIRI PRESENTERS Birger Larsen, Royal School of LIS, Copenhagen, Denmark Marianne Lykke, Aalborg University, Denmark Diane Kelly, University of North Carolina at Chapel Hill, USA Peiling Wang, University of Tennessee, Knoxville, USA ModeratorPeter Ingwersen, Royal School of LIS,Copenhagen, Denmark
  4. 4. PAIRI ADAPTIVITY in IR INTERACTION  … requires that IR systems adapt to users’ situations, and  … the users adapt to the systems.
  5. 5. PAIRI SYSTEM ADAPTION to USERS  … entails dynamic user modeling;  effective information architecture founded in practice, and  enhanced search features, such as  search integration  relevance feedback
  6. 6. PAIRI SEARCHER ADAPTION to SYSTEMs through INTERACTION  … entails mental model building of systems and modification; through  Learning – leading to  Knowledge change, such as  from an anomalous state of knowledge (ASK) towards a coherent state of knowledge (CSK);
  7. 7. PAIRI Adaption - Central DIMENSIONS: through integration of information objects of the Information Retrieval system to the current searcher to context and practice through information architecture of searchers to the Information Retrieval systems
  8. 8. PAIRI PRESENTATION – PECHA-KUCHA  Present Introduction – 7 minutes  First Dimension: 20 slides x 20 seconds (7 min)  Questions – interaction (10 minutes)  Second Dimension: 20 slides x 20 sec. (7 min)  Questions – interaction (10 minutes)  Third Dimension: 20 slides x 20 sec. (7 min)  Questions – interaction (10 minutes)  Fourth Dimension: 20 slides x 20 sec. (7 min)  Questions – interaction (10 minutes) 10 minutes vivid interaction on additional audience questions Wrapping up (5 min)
  9. 9. PAIRI QUESTIONS ON PAPER SLIPS  During presentations you may write questions to the panel (aside from oral questions!!)  ’Collectors’ will collect the slips and hand over to Moderator  Moderator selects which questions to be posed during last 10 minutes of interaction  Moderator knows which questions will be dealt with by ensuing panellists.
  10. 10. PAIRI Adaption through…  …integration of information objects
  11. 11. PAIRI Adaptivity example: me! Wrap it up – this is not your topic…
  12. 12. PAIRI Me – again! Work web page Images Bibliography
  13. 13. PAIRI Different ‘verticals’
  14. 14. PAIRI How do we handle ‘verticals’ in libraries?  As separate silos! User Catalogue . Repositories Databases eJournals JSTOR Publisher 1 Publisher 2 eBooks
  15. 15. PAIRI How to access the silos?  Information literacy!  = educate users how to use each system:  Content, fields, indexing, search operators, interfaces…  But… ‘Why do we want to teach our users to be librarians?’ (Dave Pattern, Library Systems Manager) United Kingdom Serials Group 2009 Annual conference
  16. 16. PAIRI Users don’t want silos! ‘Why is Google so easy and the library so hard?’ (Claire Duddy - student) United Kingdom Serials Group 2009 Annual conference
  17. 17. PAIRI Challenge  Each silo has its own set of metadata and standards developed for different purposes  Very low common denominator…
  18. 18. PAIRI Solution: Federated search? (William H. Mischo, 2005)
  19. 19. PAIRI Solution: Federated search?  “The jury is still out on federated search systems, even though more libraries now have them. There are murmurings that federated search has lower-than-expected use and may not be the magic search bullet we were led to believe” (Tenopir, 2007)
  20. 20. PAIRI Solution: Integrated search?  Harvest all of the relevant data sources, normalize them into a single metadata schema, and index all of them together in one large union index  Add Google-like search box and ranking  (…a kind of federated search 2.0)
  21. 21. PAIRI Solution: Integrated search?  Is being implemented in university and research libraries as we speak  University of Huddersfield , UK  Queens University, Canada  State and Uni library, Denmark  …  Several commercial products  Summon by Serial Solutions  EBSCO Discovery Service  WorldCat Local  ExLibris' Primo Central  III Encore Synergy  …
  22. 22. PAIRI Solution: Integrated search?  But…  Development handed over to large commercial vendors  or giant web search services (Google; Amazon)  Where does that leave us as a field, as practitioners and researchers?
  23. 23. PAIRI Research opportunities  There is sooo much we don’t know:  How to optimise each type of information object?  How to integrate information objects?  How to do relevance feedback in integrated search?  How to best involve users in testing?  etc etc…
  24. 24. PAIRI Research opportunities  So, we can study various elements and details…  But how to generate real knowledge of these colossal integrated systems?  Larger, more complex than ever  How to evaluate these scientifically, and not just rely on vendors?
  25. 25. PAIRI  Information Retrieval (IR) test collection  Purpose: to facilitate studies of integrating different document types iSearch test collection
  26. 26. PAIRI  Three types of documents from physics  18,841 book records  291,244 metadata records, incl. abstract  143,569 full text articles iSearch test collection
  27. 27. PAIRI  65 thoroughly described Information tasks 1. Information need -What are you looking for? 2. Work task context -Why are you looking for this? 3. Knowledge state - Background knowledge of topic? 4. Ideal information -To solve your problem or task? 5. Search terms -Which search term would you use?  + graded relevance assessments iSearch test collection
  28. 28. PAIRI  What can we do with this?  Study these tasks  Optimise each document type  Optimise the integration  Simulate relevance feedback and interfaces?  What we can’t do  Study another domain  Study real user interaction iSearch test collection
  29. 29. PAIRI Adaption through integration of information objects  Pertinent issues  How to design and evaluate systems that successfully integrate genres, media and document types?  Even with shallow data?  How to generate real knowledge and evaluate these scientifically?  Without relying on what vendors offer?
  30. 30. PAIRI Adaptation of …  information retrieval (IR) system to the user
  31. 31. PAIRI TECHNIQUE EXPLICIT-O-SAURUS
  32. 32. PAIRI USER-MODEL-O-SAUR
  33. 33. PAIRI TECHNIQUE IMPLICIT-O-SAURUS
  34. 34. PAIRI
  35. 35. PAIRI Schultz, C. K. (1968). H.P. Luhn: Pioneer of Information Science (p.32). London, UK: American Documentation Institute. 1950s: Luhn’s Selective Dissemination of Information
  36. 36. PAIRI 1960s: Salton, Lesk, Rocchio, Ide and Others Ide, E. (1967). User interaction with an automated information retrieval system. In G. Salton (Ed.) Information Storage and Retrieval: Scientific Report No. ISR-12.
  37. 37. PAIRI Oddy, R. N. (1977). Information retrieval through man-machine dialogue. Journal of Documentation, 33(1), 1-14. 1970s: Oddy’s Thomas
  38. 38. PAIRI 1980s: USER-MODEL-O-SAUR Allen, R. B. (1990). User models:Theory, method, and practice. InternationalJournal of Man-Machine Studies, 32, 511-543. Rich, E. (1983). Users are individuals: Individualizing user models. International Journal of Human-Computer Studies, 51, 323-338. “While the term ‘user model’ emphasizes the information about the person, it is obvious that a great deal of situational, task, or environmental information may be encoded in the model.”
  39. 39. PAIRI Croft,W. B., &Thompson, R. H. (1987). I3R: A new approach to the design of document retrieval systems. Journal of the American Society for Information Science, 38, 389-404. 1980s: Intelligent IR
  40. 40. PAIRI Maes, P. (1994). Agents that reduce work and information overload. CACM, 37(7), 30-40. 1990s: Agents
  41. 41. PAIRI Maes, P. (1994).Agents that reduce work and information overload. CACM, 37(7), 30-40. SYSTEM USER
  42. 42. PAIRI Belkin, N. J., Cool,C., Kelly, D., Lin, S.-J., Park, S.Y., Perez-Carballo,J., & Sikora, C. (2001). Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval. Information Processing & Management 37(3), 404-434. And more relevance feedback
  43. 43. PAIRI What Caused Explicit-O-Saurus’ Demise?  Users are unwilling to put forth the effort required to provide feedback  Users don’t have the additional cognitive resources to engage in explicit feedback
  44. 44. PAIRI 2000s: Implicit-O-Saurus  Click-through  Dwell time  Scrolling  Query Behavior
  45. 45. PAIRI Users are unwilling to put forth the effort required to provide feedback. http://www.pewinternet.org/ http://www.visualeconomics.com/how-the-world-spends-its-time-online_2010-06-16/ REALLY?
  46. 46. PAIRI http://www.movielens.org/rateMore http://www.grouplens.org/ 2010 In My Own Words Interests
  47. 47. PAIRI Users don’t have the additional cognitive resources to engage in explicit feedback.  Well, maybe back then …
  48. 48. PAIRI We need to figure out better ways of eliciting feedback from users:  Better questions  Better measures  More creative  More engaging  More adaptive Saving Explicit-O-Saurus
  49. 49. PAIRI http://hunch.com/
  50. 50. PAIRI Saving Explicit-O-Saurus Questions  How can we create better questions and measures?  How can we make the process creative, engaging and adaptive? Objection  Users are creatures of habit.  Let’s reintroduceTECHNIQUE EXPLICIT-O-SAURUS into the information ecology  Users will adapt …
  51. 51. PAIRI Users’ Adaptability in IT Environment  Adaptivity is a much needed system functionality, yes but  System alone cannot solve all the interaction problems  Humans are evolutionary learners  IT users must / can adapt to the system
  52. 52. PAIRI Biological View
  53. 53. PAIRI Psychological View  Mooers’ Law?  Satisficing principle  Today, majority IT users  Do not really have a choice at one level  Do have choices at another level
  54. 54. PAIRI Cognitive View  Individual differences  Context/situation-based  Knowledge  Skills  Mental models  Tacit
  55. 55. PAIRI Constructivist View  Problem-based learning  Social learning • Observational learning specially relevant to building IT skills • Knowledge sharing • Collective intelligence
  56. 56. PAIRI Adaptability — What is it? AdaptAbility is a key metacompetency which enables individuals quickly, effectively respond to unexpected environmental changes
  57. 57. PAIRI Adaptability in Contexts  In workplace: job adaptability  In multicultural society: cross-cultural adaptability  In information environment: IT adaptability
  58. 58. PAIRI Measuring Adaptability  Job Adaptability Inventory (JAI)  Cross-cultural Adaptability Inventory (CCAI)  Boyatzis-Kolb Adaptive Style Inventory  IT Adaptability – we need instruments  Adaptability – contributing factors?
  59. 59. PAIRI Job Adaptability  Creative problem solving (new …)  Dealing Uncertainty/unpredictable work situations  Learning new tasks, tech, procedures …  Adapting culturally  Adapting interpersonally  Physical factors
  60. 60. PAIRI Cross-Cultural Adaptability Inventory (CCAI) A survey of 50 questions to uncover individuals’ current strengths and weaknesses within four critical skill areas proven necessary for effective cross-cultural communication:  Emotional Resilience  Flexibility/Openness  Perceptual Acuity  Personal Autonomy
  61. 61. PAIRI Adaptive Style Inventory • Acting situation • new • on time • Deciding Situation • two alternatives • an opportunity • Thinking • an idea • analyzing something • Valuing • my feelings • see the world as the another person sees it --Kolb Learning Style Inventory?
  62. 62. PAIRI IT Adaptive Behaviour We need systematic research to understand factors underlying the IT adaptive behavior We need instruments to measure IT adaptability (either adopt existing or develop new relevant one)
  63. 63. PAIRI Where do we start? Adaptive situations  IT system being updated  Need a function never used  New IT system  “I have used it before, but how do I get to it now”?  “The output doesn’t look right”!  “Where are my files I saw all of them just a moment ago?! Or, no, I cannot lose them ...”
  64. 64. PAIRI Adaptive Performance  User’s context  IT Environment  User’s IT competency  User’s mental model  The task situation
  65. 65. PAIRI AdaptAbility vs. LearnAbility  User adaptability calls for the system’s learnAbility  IT system learnability support user’s adaptAbility  learnability is an aspect of usability – rarely studied
  66. 66. PAIRI ALPHA-IT A (Adaptability) with L (Learnability) for P (Personalizing) & H (Humanizing) A (Adaptivity) in IT (InformationTechnology)
  67. 67. PAIRI Personalizing & Humanizing  Personalizing: Beyond user profiling  IT knowledge state  IT competency  Adaptive style  Humanizing : Beyond affordances  Demonstrate/encourage empathy  Affective responses
  68. 68. PAIRI IT Affordance  Specifies the range of possible actions about an object (physical or digital)  Must be visible to the users to be usable  Individual differences in perceived affordances  When affordances fail users: an action does not result in an expected result --?
  69. 69. PAIRI Building IT Adaptability as Social Learning  Sharing learning experience  Transferring knowledge  Collecting user problems  Banking user strategies
  70. 70. PAIRI Study of User Adaptability & IT Design  How do we study users adaptive behavior in adopting a new system, new applications, new functions, new …?  How can personal adaptive behaviors help system design?
  71. 71. PAIRI Adaption to context and practice  Focus on adapting information architecture (IA)  Focus on models and methods for developing tailored information architecture (IA)
  72. 72. PAIRI Quick definition of IA (1) ORGANISATION SYSTEM LABEL SYSTEM Structural design Combination of organisation, labelling, search and navigation
  73. 73. PAIRI Quick definition of IA (2) NAVIGATION SYSTEM SEARCH SYSTEM
  74. 74. PAIRI Adapting IA  Organization system, e.g.  Categories – tailored to user needs  Organization – organized to user perspective  Labels, e.g.  Terminology – jargon, language of youth, basic level
  75. 75. PAIRI Adapting IA  Navigation system, e.g.  Tailored short cuts  Tailored links  Tailored recommending,  Search system, e.g.  Tailored filters  Tailored ranking
  76. 76. PAIRI Adapting IA
  77. 77. PAIRI System development process (Rosenfeld & Morville, 2007, 232)
  78. 78. PAIRI Research process Context UsersContent Methods for data collection: e.g. interview, workshops, questionnaires, content analysis, log files Mission, information behavior, policies, culture, technology, resources (time, economy, competences) Content types, number, genre, usage, relation to work tasks Target groups, information behavior, work tasks, search tasks, discourses, terminology (Rosenfeld & Morville, 2007, 233)
  79. 79. PAIRI Strategy process (Rosenfeld & Morville, 2007, 269)
  80. 80. PAIRI System development models  Plenty of models and approaches:  User-oriented  Domain-oriented  Work-task oriented  Interaction-oriented  Participation-oriented  …  All believe that we can grasp the context
  81. 81. PAIRI System development models  Can we grasp, understand, model the context and practice – and adapt?  Are we “heroes” between “victims” and “tyrants”? (Spinuzzi, 2004)
  82. 82. PAIRI Borger.dk – an example  Development of organisation system for Danish Government portal Borger.dk
  83. 83. PAIRI Borger.dk – an example  Timeline  2008 Analysis, development, implementation  2009 Evaluation: usability tests, online survey and search log analysis  2009 Redesign and implementation of updated version and new features
  84. 84. PAIRI Borger.dk – design methodology  Development phrase  Survey of domain of public digital communication  Personas – user needs and behaviour  Bench marking – related portals and organisations systems  Content analysis  Expert evaluation  Usability tests - 12 citizens completing 4 tasks
  85. 85. PAIRI Borger.dk – design methodology  Evaluation phrase  Usability tests - 40 citizens completing 4 tasks  Online survey - 453 self-selected respondents  Search log analysis - Google Analytics over 11 month
  86. 86. PAIRI Borger.dk - redesign  Information needs • 75% of needs met by categories • Miss personal and factual information  Categories, organization, navigation • Unclear labels • Short cuts - to frequent categories and forms • Cross references • Prioritized ordering • Search • Synonym rings • Best bets Marianne Lykke & Brian KirkegaardLunn
  87. 87. PAIRI Explanation of difficult concepts
  88. 88. PAIRI More and tailored cross references
  89. 89. PAIRI Best bets for important topics
  90. 90. PAIRI ??? Adaption to context ???  Do “heroes” exist within system design and adaption?  What models are feasible to adaption to context and practice?  What tools should be developed?
  91. 91. PAIRI Last round of questions & Interaction!! PAPER SLIP QUESTIONS FROM YOU
  92. 92. PAIRI WRAPPING UP the PANEL  Interesting discussions on:  Design and evaluation of integrated systems?  How to avoid re-engineering of vendor offerings?  Feasibility of explicit relevance feedback for systems adaptivity?
  93. 93. PAIRI WRAPPING UP the PANEL - 2  Interesting discussions on:  How does personal adaption behavior help systems (design)?  Research methdology and tools studying adaptive behavior to new ...?  How to adapt to context and work practice?
  94. 94. PAIRI THANK YOU

×