Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Value of Multistage Search Systems for Book Search

321 views

Published on

Presentation of joint work at the Interactive Social Book Search Lab at the CLEF 2015 conference (Toulouse, France)

Published in: Education
  • Be the first to comment

The Value of Multistage Search Systems for Book Search

  1. 1. Hugo Huurdeman, Jaap Kamps, ! Marijn Koolen and Sanna Kumpulainen! ! University of Amsterdam ! ! ! Social Book Search Workshop, CLEF conference 2015 The Value of Multistage 
 Search Systems for Book Search
  2. 2. 1. Introduction • Interactive Social Book Search track! ! • Amazon / LibraryThing book data ! • focused and open search tasks! • baseline and multistage interfaces! ! • social-book-search.humanities.uva.nl/#/interactive ! • Joint study 2015 iteration: [Gaede-etAl15]: ! • 192 participants! • 97 in baseline interface, 95 in multistage interface
  3. 3. 1. Introduction: baseline interface • ‘standard’ search interface ! • single screen • query box • results list • left column: facet filters • right column: ‘book bag’ for book selections Baseline interface
  4. 4. • The multistage interface contains three screens ! • Aim: support different stages in the search process: browse, search & review ! • Inspired by various models of the information seeking process [Vakkari01,Kuhlthau03] Multistage interface: search Multistage interface: browse Multistage interface: review 1. Introduction: multistage
  5. 5. 2. Our Participation • What is the value of a multistage interface for book search?! ! 1. influence on task duration, book selection and interaction patterns 2. influence on user engagement
  6. 6. 2.1 Task duration • How much time was spent in each task and interface? ! • Participants spend more time in the focused task than the open task ! • Participants spend significantly more time in the multistage interface, regardless of the task time spent (seconds)
  7. 7. 2.2 Bookbag: overview • How many books were selected per task and interface? ! • More books in focused task than open task (significant) ! • Slightly more books were selected in the multistage interface, as compared to the baseline (not significant) number of selected books
  8. 8. 2.2 Bookbag: overlap between the tasks • Only 9 participants have overlap in the book bags for the two tasks 
 (7 out of 9 just one book..) • User may interpret some sub- tasks of the focused task in a similar way as the open task (e.g. hobbies/personal interests or fun). What is the overlap in collected books between the tasks? ! ! ! !
  9. 9. 2.2 Bookbag: overlap between participants • What degree of selected books is the same across participants? ! ! ! ! ! !
  10. 10. 2.2 Bookbag: overlap between participants • What degree of selected books is the same across participants? ! • More overlap in baseline (BL) • Evidence for more varied selections in the multistage interface (MS) 0 0.06 0.12 0.18 0.24 Open task Focused task 14% 7% 23% 7% MSBL MSBL
  11. 11. 2.3 User actions • Paginate used more in multistage • deeper exploration ! ! ! ! ! ! ! ! ! ! ! mean frequencies of actions
  12. 12. 2.3 User actions • Paginate used more in multistage • deeper exploration ! • Few filters and queries in multistage • more options to explore and review results ! ! ! ! ! mean frequencies of actions
  13. 13. 2.3 User actions • Paginate used more in multistage • deeper exploration ! • Few filters and queries in multistage • more options to explore and review results ! • Less/more book metadata views: • triggered to view more in open task? mean frequencies of actions
  14. 14. 2.4 Use of interface panels: transition prob. Focused task Open task
  15. 15. 2.4 Use of interface panels: time spent • Time spent! • Focused task: search screen used for longest duration; • open task: browse ! ! ! ! ! ! Time spent in each panel (sec)
  16. 16. 2.4 Use of interface panels: switches • Time spent! • Focused task: search screen used for longest duration; • open task: browse ! • Switches! • Training task: participants went “in order” through panels” • Other tasks: more varied • 5 subtasks Switches between panels Time spent in each panel (sec)
  17. 17. 2.5 Participants’ Perceptions • Using multidimensional User Engagement Scale [O’Brien & Toms]
  18. 18. 2.5 Participants’ Perceptions • Using multidimensional User Engagement Scale [O’Brien-Toms15] ! • Significant (Mann-Whitney): • Felt Involvement: “how much fun users were having during the interaction and how drawn in” • Endurability: “the assessment of users' perception of success with a task, and their willingness to use an application in future or recommend it to others” p=0.006p=0.041 * engagement did not significantly differ between age groups (18-25, 26-35, 35+)
  19. 19. 3 Conclusion • Longer task duration in multistage interface • Different use of search features • less queries / filters in multistage • Not necessarily more books, but more varied sets of books • More user engagement! ! • Users seem to be willing to switch interface panels and use them for different activity types ! • Positive evidence ‘multistage’ approach
  20. 20. Discussion • Influence personal differences participants • motivation, location, language skills, education, etc. • users who frequently search for books in (online) bookstores? ! • Task properties • open vs. focused task (5 subtasks), • possible to differentiate more easily? • task order ! • From narrative to interactions
  21. 21. References • [Gaede-etAl15] M. Gaede, M. Hall, H. Huurdeman, J. Kamps, M. Koolen, M. Skov, E. Toms, D. Walsh. Overview of the SBS 2015 Interactive Track. CEUR-WS. 2015. • [Hall-Toms15] M. Hall and E. Toms. Building a common framework for iir evaluation. In P. Forner, H. Mueller, R. Paredes, P. Rosso, and B. Stein, editors, Information Access Evaluation. Multilinguality, Multimodality, and Visualization, volume 8138 of Lecture Notes in Computer Science, p. 17–28. Springer Berlin Heidelberg, 2013. • [Huurdeman-etAl15] H. Huurdeman, J. Kamps, M. Koolen, S. Kumpulainen. The Value of Multistage Search Systems for Book Search. CEUR-WS. 2015. • [Koolen-etAl15] M. Koolen, T. Bogers, M. Gaede, M. Hall, H. Huurdeman, J. Kamps, M. Skov, E. Toms, and D. Walsh. Overview of the CLEF 2015 Social Book Search Lab. In CLEF'15: Experimental IR meets Multilinguality, Multimodality, and Interaction, volume 9283 of LNCS. Springer (forthcoming). • [Kuhlthau04] C. Kuhlthau. Seeking Meaning: A Process Approach to Library and Information Services. Libraries Unlimited, Westport, Conn. 2004. • [O’Brien-Toms15] H. O’Brien and E. Toms. The development and evaluation of a survey to measure user engagement. JASIST, 61(1):50–69, 2010. • [Vakkari01] P. Vakkari. A theory of the task-based information retrieval process: a summary and generalisation of a longitudinal study. Journal of Documentation, 57:44– 60, 2001.
  22. 22. Hugo Huurdeman, Jaap Kamps, ! Marijn Koolen and Sanna Kumpulainen! ! University of Amsterdam ! ! ! Social Book Search Workshop, CLEF conference 2015 The Value of Multistage 
 Search Systems for Book Search

×