CoSense: Enhancing Sensemaking for Collaborative Web Search Sharoda A. Paul Meredith Ringel Morris Apr 8, 2009

Getting information off the Internet is like taking a drink from a fire hydrant - Mitchell Kapor

Making sense of information found during a Web search task can be daunting

During collaborative Web search tasks, making sense of information can be even more complex and challenging “ How does sensemaking occur during collaborative Web search task?”

Goals Understand how sensemaking occurs during collaborative Web search Develop a tool to help enhance users’ sensemaking during collaborative Web search tasks

Understand how sensemaking occurs during collaborative Web search

Develop a tool to help enhance users’ sensemaking during collaborative Web search tasks

Background: Sensemaking Sensemaking has been studied in a variety of fields (Weick, 1995; Dervin 2003; Russell, 1993) Challenging problem in HCI Sensemaking is finding meaning or understanding of information Important part of information seeking activities

Sensemaking has been studied in a variety of fields (Weick, 1995; Dervin 2003; Russell, 1993)

Challenging problem in HCI

Important part of information seeking activities

Background: Collaborative Sensemaking Most sensemaking research has been at the individual level Sensemaking is important in collaborative information seeking activities (Hansen & Jarvelin, 2005; Hertzum, 2008) Little understanding of how sensemaking occurs at the group level

Most sensemaking research has been at the individual level

Sensemaking is important in collaborative information seeking activities

(Hansen & Jarvelin, 2005; Hertzum, 2008)

Little understanding of how sensemaking occurs at the group level

Background: Collaborative Web search People collaborate during Web search (Morris, 2008) Tools have been developed to support collaborative Web search (Morris, 2007; Pickens et al., 2008) Lack of understanding of users’ behavior during collaborative Web search, specifically sensemaking behavior

People collaborate during Web search (Morris, 2008)

Tools have been developed to support collaborative Web search (Morris, 2007; Pickens et al., 2008)

Lack of understanding of users’ behavior during collaborative Web search, specifically sensemaking behavior

What we did Formative study Identified sensemaking challenges in using a collaborative Web search tool, SearchTogether Design new tool CoSense to help users overcome sensemaking challenges Evaluation of CoSense to examine how sensemaking takes place during collaborative Web search how tools can be designed to support sensemaking

Formative study

Identified sensemaking challenges in using a collaborative Web search tool, SearchTogether

Design

new tool CoSense to help users overcome sensemaking challenges

Evaluation

of CoSense to examine

how sensemaking takes place during collaborative Web search

how tools can be designed to support sensemaking

SearchTogether (UIST, 2007) Query history Search session Add comments and ratings to Web pages Chat with collaborators

SearchTogether: Features Session summary Group query history Integrated chat Ratings and comments Peek-and-follow browsing Standard, split, and multi-engine search

Session summary

Group query history

Integrated chat

Ratings and comments

Peek-and-follow browsing

Standard, split, and multi-engine search

Formative study: Design Goal: Understand how SearchTogether supports sensemaking Phase 1 Phase 2 Group ST ST ST Interviews Observations Audio and video recording Data collection SearchTogether Tools used 6 groups (3 in each group) Participants Vacation planning 2-phase design Task

Goal: Understand how SearchTogether supports sensemaking

Interviews

Observations

Audio and video recording

Formative study: Findings Sensemaking challenges Awareness Context and action awareness Sensemaking trajectories Steps in the sensemaking process Sensemaking handoffs Understanding information found asynchronously

Sensemaking challenges

Awareness

Context and action awareness

Sensemaking trajectories

Steps in the sensemaking process

Sensemaking handoffs

Understanding information found asynchronously

CoSense Integrated with SearchTogether Four views of information Search strategies view Timeline view Workspace view Chat-centric view Database SearchTogether CoSense

Integrated with SearchTogether

Four views of information

Search strategies view

Timeline view

Workspace view

Chat-centric view

Search strategies view (1)

Search strategies view (2)

Timeline view

Workspace view

Chat-centric view

Evaluation of CoSense Goal: Examine if CoSense improved sensemaking challenges faced in formative study Interviews Observations Online questionnaire Data collection SearchTogether and CoSense Tools used 18 participants (4 in each group) Participants Vacation planning 2-phase design Task

Goal:

Examine if CoSense improved sensemaking challenges faced in formative study

Interviews

Observations

Online questionnaire

Evaluation of CoSense: Data collection Questionnaire tested participants’ understanding of Task-related information Others’ task performance Others’ skills and contributions Task state and progress on goals “ Which group member do you think contributed most to the task and why?” “ Which websites generated a lot of discussion?”

Questionnaire tested participants’ understanding of

Task-related information

Others’ task performance

Others’ skills and contributions

Task state and progress on goals

Evaluation of CoSense: Findings CoSense views supported sensemaking during Synchronous and asynchronous search Initial vs. handoff stages of search Also supported different styles of sensemaking

CoSense views supported sensemaking during

Synchronous and asynchronous search

Initial vs. handoff stages of search

Also supported different styles of sensemaking

CoSense view usage: During task (1) During synchronous search Most used views: search strategies and chat-centric Popular features Viewing tag clouds Clicking on chat messages to see associated Web pages.

During synchronous search

Most used views: search strategies and chat-centric

Popular features

Viewing tag clouds

Clicking on chat messages to see associated Web pages.

CoSense view usage: During task (2) During asynchronous search Most used were timeline and workspace views Popular features Opening Web pages from the timeline Editing ‘scratchpad’ and opening links from summary

During asynchronous search

Most used were timeline and workspace views

Popular features

Opening Web pages from the timeline

Editing ‘scratchpad’ and opening links from summary

CoSense view usage: During questionnaire Search strategies view Skills and strategies of group members Timeline view Connections between different content Workspace view Group members’ contributions and roles Chat-centric view Group members’ contributions and decisions reached

Search strategies view

Skills and strategies of group members

Timeline view

Connections between different content

Workspace view

Group members’ contributions and roles

Chat-centric view

Group members’ contributions and decisions reached

CoSense usage during handoff “ The tag clouds and statistics on how many queries had been done, what were the group’s sites, and what is their popularity, helped me immensely …I was able to plan [the search] better using the websites and keywords ” “ The flow of the search was hard to tell [by looking at SearchTogether]. The timeline in CoSense helped because of the information about order of queries. I identified what had already been done ”

Discussion Synchronous search Focus on ‘ how’ is information being found Asynchronous search Focus on ‘ what’ information was found

Synchronous search

Focus on ‘ how’ is information being found

Asynchronous search

Focus on ‘ what’ information was found

Conclusion CoSense addressed challenges found in the formative study Awareness Temporality and trajectories Handoffs Contributions Gained an understanding of sensemaking in groups Designed features to support sensemaking in collaborative Web search

CoSense addressed challenges found in the formative study

Awareness

Temporality and trajectories

Handoffs

Contributions

Gained an understanding of sensemaking in groups

Designed features to support sensemaking in collaborative Web search

Future work Explore further sensemaking styles during synchronous and asynchronous search and handoff Examine how the products of sensemaking are passed on across time and group members Move towards models of collaborative information seeking

Explore further sensemaking styles during synchronous and asynchronous search and handoff

Examine how the products of sensemaking are passed on across time and group members

Move towards models of collaborative information seeking

Thank you! [email_address] http://www.personal.psu.edu/sap246 Information Processing & Management Special Issue on Collaborative Information Seeking Papers Due: May 8 th http://research.microsoft.com/~merrie for the CFP

[email_address]

http://www.personal.psu.edu/sap246

Information Processing & Management

Special Issue on Collaborative Information Seeking

Papers Due: May 8 th

http://research.microsoft.com/~merrie for the CFP

References Photos courtesy of Flickr users sean dreilinger, Ed Yourdon, quinn.anya, Joseph Robertson, Feuillu, sonson, iamhenry, SonnyandSandy User sketches inspired by www.xkcd.com Weick, K.E.. (1995). Sensemaking in Organizations . Foundations for Organizational Science. Thousand Oaks, CA: Sage Publications, Inc. Russell, D.M., et al. (1993). The cost structure of sensemaking . in Proceedings of CHI ‘93 . Amsterdam, The Netherlands. ACM Press. Dervin, B., L. Foreman-Wernet, and E. Lauterbach, eds. (2003). Sense-Making METHODOLOGY Reader: Selected Writings of Brenda Derwin . Hampton Press, Inc., Cresskill, NJ. Hansen, P. and Jarvelin, K. (2005). Collaborative information retrieval in an information-intensive domain. Information Processing and Management . 41, 1101-1119. Hertzum, M. (2008). Collaborative information seeking: The combined activity of information seeking and collaborative grounding. Information Processing and Management . 44, 957-962. Morris, M. R. and Horvitz, E. (2007). SearchTogether: An interface for collaborative Web search. Proceedings of UIST ‘07 , Newport, RI, USA. ACM Press. Pickens, J., Golovchinsky, G., Shah, C., Qvarfordt, P., and Back, M. (2008). Algorithmic mediation for collaborative exploratory search. In Proc. of SIGIR ’08, Singapore. ACM Press.

Photos courtesy of Flickr users sean dreilinger, Ed Yourdon, quinn.anya, Joseph Robertson, Feuillu, sonson, iamhenry, SonnyandSandy

User sketches inspired by www.xkcd.com

Weick, K.E.. (1995). Sensemaking in Organizations . Foundations for Organizational Science. Thousand Oaks, CA: Sage Publications, Inc.

Russell, D.M., et al. (1993). The cost structure of sensemaking . in Proceedings of CHI ‘93 . Amsterdam, The Netherlands. ACM Press.

Dervin, B., L. Foreman-Wernet, and E. Lauterbach, eds. (2003). Sense-Making METHODOLOGY Reader: Selected Writings of Brenda Derwin . Hampton Press, Inc., Cresskill, NJ.

Hansen, P. and Jarvelin, K. (2005). Collaborative information retrieval in an information-intensive domain. Information Processing and Management . 41, 1101-1119.

Hertzum, M. (2008). Collaborative information seeking: The combined activity of information seeking and collaborative grounding. Information Processing and Management . 44, 957-962.

Morris, M. R. and Horvitz, E. (2007). SearchTogether: An interface for collaborative Web search. Proceedings of UIST ‘07 , Newport, RI, USA. ACM Press.

Pickens, J., Golovchinsky, G., Shah, C., Qvarfordt, P., and Back, M. (2008). Algorithmic mediation for collaborative exploratory search. In Proc. of SIGIR ’08, Singapore. ACM Press.