Jill Freyne - Collecting community wisdom: integrating social search and social navigation

Slide 1: Collecting Community Wisdom: Integrating Social Search & Social Navigation Jill Freyne, Rosta Farzan, Peter Brusilovsky, Barry Smyth & Maurice Coyle University College Dublin & University of Pittsburgh UNIVERSITY COLLEGE DUBLIN

Slide 2: Motivations • Information Overload Problem • Potential in harnessing user activity patterns to drive social information access tools • Independence of social systems – Eurekster, Del.icio.us, Shadows, I-SPY, MySpace, ebay,….. UNIVERSITY COLLEGE DUBLIN

Slide 3: Social Browsing • Form of Social Navigation • Visualizes usage paths in a digital environment • KnowlegeSea II Brusilovsky et al. AH 2004 – Information access system for class mates providing access to open corpus textbooks – Supports information access through visualization, search, annotation and browsing UNIVERSITY COLLEGE DUBLIN

Slide 4: KnowlegeSea II • Footprint based support – Monitors students as they browse the collection • Annotation based support – Allows students to leave specific traces on parts of the system UNIVERSITY COLLEGE DUBLIN

Slide 5: Social Search • Collaborative Web search • Communities of like minded users searching together • I-SPY Smyth et al. UMUAI 2004 – Social search engine where communities of searchers can benefit from each others wisdom in terms of search queries and result selections UNIVERSITY COLLEGE DUBLIN

Slide 6: I-SPY • Monitors users as they search • Post Processing Engine • Re-ranks result lists from underlying search engines to reflect community preferences UNIVERSITY COLLEGE DUBLIN

Slide 7: ACM DL • ACM DL - a vast collection of citations and full text from ACM journal and newsletter articles and conference proceedings. • 2 access strategies – Browsing – through archive lists, tables of contents – Searching – basic and advanced search features UNIVERSITY COLLEGE DUBLIN

Slide 8: Searching UNIVERSITY COLLEGE DUBLIN

Slide 9: Browsing UNIVERSITY COLLEGE DUBLIN

Slide 10: Integrated Social Information Access • “Community Wisdom” collected by the search component should be used to support navigation in the browsing component and vice versa • Social Search and Social Browsing should be seamlessly integrated at the interface level • Social Support in the form of icons/cues UNIVERSITY COLLEGE DUBLIN

Slide 11: Components • Search Component – I-SPY Search technology – Re-rank CACM result-list and add search icons • Browsing Component – Knowledge Sea II social navigation technology – Provide footprint and annotation support and add browse icons UNIVERSITY COLLEGE DUBLIN

Slide 12: Search Component Architecture Hit Matrix RT q q CWS ENGINE RH q` RM CACM SEARCH ENGINE UNIVERSITY COLLEGE DUBLIN

Slide 13: Browse Component Architecture Navigation (Browsing & Annotation) Records KNOWLEDGE SEA ENGINE UNIVERSITY COLLEGE DUBLIN

Slide 14: Integration - Search Users start with search CACM Search Engine Hit Matrix Result Set CWS Engine Re-ranked result With social search icons Navigation (Browsing & Annotation) Records Navigation Engine Re-ranked result With social search and social navigation icons UNIVERSITY COLLEGE DUBLIN

Slide 15: Integration - Browsing Navigation Users start (Browsing & Annotation) with browsing Records Navigation Engine Adding social navigation icons Search Hit Matrix CWS Engine Back to search Query based navigation Adding social search icons UNIVERSITY COLLEGE DUBLIN

Slide 16: Annotations UNIVERSITY COLLEGE DUBLIN

Slide 17: Search Support Annotations: Relevant to previous work by Dourish and Chalmers in Social Related Queries: Navigation Social Navigation Computational Wear Browse Popularity 30% Relevance: 100% Last Selection: 30 mins ago Last Browse: 1 day ago Last Annotation: 1 day ago UNIVERSITY COLLEGE DUBLIN

Slide 18: Browsing Support Annotation: ………… Queries: personalization business customer adaptive web Browse Popularity: 5% UNIVERSITY COLLEGE DUBLIN

Slide 19: Browse – Search Cycle UNIVERSITY COLLEGE DUBLIN

Slide 20: Evaluation • Subjects: 30 students enrolled in “Introduction to Multimedia” course • Task: Produce a literature review on the topic of “the social web” in 1 hour using the ACM social site • Groups: – Control group - no social support (15 users) – Experimental Group - all social support enabled (15 users) UNIVERSITY COLLEGE DUBLIN

Slide 21: Review Relevance • 54 articles returned Relevant 6 Very Relevant 5 • Each examined and 4 assigned relevance score 3 2 • 22.4% relative 1 increase in very 0 relevant papers Ctrl Exp UNIVERSITY COLLEGE DUBLIN

Slide 22: Search and Browsing Effort • Users in the control group had to do more work to return (less relevant) results. Ctrl Exp # queries 13.2 9.13 Query length 2.33 2.37 Ave selected 1.56 1.26 result page # links browsed 12.53 7.0 UNIVERSITY COLLEGE DUBLIN

Slide 23: So? • By adding social support to the CACM DL users found relevant information faster and with less user effort UNIVERSITY COLLEGE DUBLIN

Slide 24: Subjective Data Analysis • Questionnaire – Designed to capture user opinion on the system • 70% of the exp group found social ACM system useful for task • 40% of ctrl group reported the same UNIVERSITY COLLEGE DUBLIN

Slide 25: Searching 4 3 2 1 0 Popularity Recency Related Browse Annotation Queries UNIVERSITY COLLEGE DUBLIN

Slide 26: Browsing 4 3 2 1 0 Query Browse Annotation UNIVERSITY COLLEGE DUBLIN

Slide 27: Conclusions • Different community-based information access technologies can be used in an integrated system to reinforce each other and provide unique added-value to the users • Integration of both technologies was achieved through seamless connection from search to browsing and browsing to search UNIVERSITY COLLEGE DUBLIN

Slide 28: Future work • Larger user study – Larger, longer term study • Domain divergence – Introduce social support to other domains Multimedia, other academic domains UNIVERSITY COLLEGE DUBLIN

Slide 29: Collecting Community Wisdom: Integrating Social Search & Social Navigation Jill Freyne, Rosta Farzan, Peter Brusilovsky, Barry Smyth & Maurice Coyle University College Dublin & University of Pittsburgh UNIVERSITY COLLEGE DUBLIN

Slide 30: User Feedback • All subjects noticed the social icons • All subjects often/always drawn to icons • Users reported that they found it easier to locate information on the ACM DL site with the icons than without. UNIVERSITY COLLEGE DUBLIN

Slide 31: Social Search • Monitors users as they search, recording their queries and result selection pairs • Uses past search interaction to re-rank the results of an underlying search engine to reflect the preferences of a community of searchers UNIVERSITY COLLEGE DUBLIN

Slide 32: Search and Browsing Efforts Average selected • 58% of users did not result page look beyond result 2 page 1 1.75 • 88% did not look 1.5 beyond result page 1.25 3 1 0.75 0.5 0.25 0 Ctrl Exp UNIVERSITY COLLEGE DUBLIN

Slide 33: UNIVERSITY COLLEGE DUBLIN