Information Seeking with Social Signals: Anatomy of a Social Tag-based Exploratory Search Browser


Published on

Presented at IUI2010 conference workshop on Social Recommender Systems

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Informational search – ambiguity in query – where social search has most power
  • Page collection task: data dependent. There are a lot of users who use delicious who are in computer related fields.
  • Information Seeking with Social Signals: Anatomy of a Social Tag-based Exploratory Search Browser

    1. 1. Information Seeking with Social Signals: Anatomy of aSocial Tag-based Exploratory Search Browser Ed H. Chi, Rowan Nairn Palo Alto Research Center Contact: [email_address] Area Manager, Augmented Social Cognition Area
    2. 2. Social Search Survey [Evans & Chi, CSCW2008] <ul><li>150 user surveys </li></ul><ul><li>Help understand the importance of: </li></ul><ul><ul><li>social cues and information exchanges </li></ul></ul><ul><ul><li>vocabulary problems </li></ul></ul><ul><ul><li>distribution and organization </li></ul></ul>
    3. 3. TagSearch Exploratory Focus 3 kinds of search navigational transactional 28% 13% You know what you want and where it is You know what you want to do Existing search engines are OK informational 59% You roughly know what you want but don’t know how to find it Difficult for existing search engines Opportunity
    4. 4. Research Motivation
    5. 6. MapReduce Implementation <ul><li>Spreading Activation in a bigraph </li></ul><ul><li>MapReduce computation over a large data set </li></ul><ul><ul><li>150 Million+ bookmarks </li></ul></ul>Tags URLs P(URL|Tag) P(Tag|URL)
    6. 7. Use Semantic Analysis to Reduce Noise Guide Web Howto Tips Help Tools Tip Tricks Tutorial Tutorials Reference Semantic Similarity Graph
    7. 8. TagSearch Architecture <ul><li>MapReduce: months of computation to a single day </li></ul><ul><li>Development of novel scoring function </li></ul>
    8. 9. Baseline Interface
    9. 10. Exploratory Interface
    10. 11. Experiment Design <ul><li>2 interface x 3 task domain design </li></ul><ul><ul><li>2 Interface (between-subjects) </li></ul></ul><ul><ul><ul><li>Exploratory vs. Baseline </li></ul></ul></ul><ul><ul><li>3 task domains (within-subjects) </li></ul></ul><ul><ul><ul><li>Future Architecture, Global Warming, Web Mashups </li></ul></ul></ul><ul><li>30 Subjects (22 male, 8 female) </li></ul><ul><ul><li>Intermediate or advanced computer and web search skills </li></ul></ul><ul><ul><li>Half assigned Exploratory, half Baseline. </li></ul></ul><ul><li>For each domain, single block with 3 task types: </li></ul><ul><ul><li>Easy and Difficult Page Collection Task [6min each] </li></ul></ul><ul><ul><li>Summarization Task [12min] </li></ul></ul><ul><ul><li>Keyword Generation Task [2min] </li></ul></ul>
    11. 12. Page Collection Tasks [6min each]
    12. 13. Summarization Tasks [12min each]
    13. 14. Procedure [2 hours] <ul><li>Prior Knowledge Test </li></ul><ul><li>1 st Task Domain </li></ul><ul><ul><li>With easy and difficult page collection tasks, summarization and keyword generation task. </li></ul></ul><ul><ul><li>NASA cognitive load questionnaire </li></ul></ul><ul><li>2 nd Task Domain </li></ul><ul><ul><li>Same battery of tasks and cognitive load questionaire </li></ul></ul><ul><li>3 rd Task Domain </li></ul><ul><li>Experimental Survey </li></ul>
    14. 15. Results: Interaction Behaviors <ul><li>Number of Queries </li></ul><ul><ul><li>Effect of Interface on number of queries (p < .01) </li></ul></ul><ul><ul><ul><li>Exploratory (M=7.81) > Baseline (M=3.77) </li></ul></ul></ul><ul><li>Time Taken </li></ul><ul><ul><li>Effect of Interface on time taken (p < .01) </li></ul></ul><ul><ul><ul><li>Exploratory (7.7min) > Baseline (6.6min) </li></ul></ul></ul>
    15. 16. Results: Page Collection Task <ul><ul><li>Effects of Task Domain (p<.01) and Task Difficulty (p<.05) </li></ul></ul><ul><ul><li>Interaction effect of Interface by Task Domain (p<.05), with Exploratory interface performing better in the Web Mashup domain </li></ul></ul><ul><ul><li>For relevance scores, similar patterns. </li></ul></ul><ul><ul><li>Measure of # of pages collected </li></ul></ul>
    16. 17. Results: Summarization Tasks <ul><ul><li>Quality of summarization scored (Cohen’s Kappa=0.7) </li></ul></ul><ul><ul><li>ANCOVA with Prior Knowledge as covariate </li></ul></ul><ul><ul><li>Exploratory Interface scored higher in Future Architecture (p<.05) and Global Warming (p<.05) </li></ul></ul><ul><ul><li>For Web Mashup, Prior Knowledge correlated positively with performance (r=.51) </li></ul></ul>
    17. 18. Results: Keyword Generation Tasks <ul><ul><li>ANCOVA showed Exploratory > Baseline for Future Architecture (p<.05) and Web Mashups (p<.01), but not for Global Warming. </li></ul></ul><ul><ul><li>Linear model between PK and # of keyword generated for Baseline showed mean slope = 0.32 and significant (p<.05) </li></ul></ul>
    18. 19. Results: Cognitive Load <ul><ul><li>Exploratory > Baseline (p<.05) </li></ul></ul>
    19. 20. Discussion <ul><li>Exploratory interface users: </li></ul><ul><ul><li>performed more queries, </li></ul></ul><ul><ul><li>took more time, </li></ul></ul><ul><ul><li>wrote better summaries (in 2/3 domains), </li></ul></ul><ul><ul><li>generated more relevant keywords (in 2/3 domains), and </li></ul></ul><ul><ul><li>had a higher cognitive load. </li></ul></ul><ul><li>Suggestive of deeper engagement and better learning. </li></ul><ul><li>Some evidence of scaffolding for novices in the keyword generation and summarization tasks. </li></ul>
    20. 21. Summary <ul><li>Harnessing user-generated tags to enrich content for social search </li></ul><ul><li>Weaknesses of social tagging systems is Tag Noise and Inconsistency </li></ul><ul><ul><li>Difficult to leverage for search </li></ul></ul><ul><ul><li>Use data mining techniques to normalize and reduce noise </li></ul></ul><ul><ul><li>Apply normalized tag data in new search algorithm </li></ul></ul><ul><li>Study suggest deeper user engagement in exploration and better learning with MrTaggy </li></ul>
    21. 22. Thanks! <ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul><ul><ul><li>Our Blog: </li></ul></ul><ul><ul><li>Contact: </li></ul></ul><ul><ul><li>Ed H. Chi, Ph.D. </li></ul></ul><ul><ul><li>Manager, Augmented Social Cognition Area </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>Kammerer, Y., Nairn, R., Pirolli, P., and Chi, E. H. 2009. Signpost from the masses: learning effects in an exploratory social tag search browser. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 625-634. </li></ul></ul>
    22. 23. Research Vision Augmented Social Cognition <ul><li>Cognition : the ability to remember, think, and reason; the faculty of knowing. </li></ul><ul><li>Social Cognition : the ability of a group to remember, think, and reason; the construction of knowledge structures by a group. </li></ul><ul><ul><li>(not quite the same as in the branch of psychology that studies the cognitive processes involved in social interaction, though included) </li></ul></ul><ul><li>Augmented Social Cognition : Supported by systems, the enhancement of the ability of a group to remember, think, and reason; the system-supported construction of knowledge structures by a group. </li></ul><ul><li>Citation: Ed H. Chi. The Social Web: Opportunities for Research. IEEE Computer, Sept 2008 </li></ul>2008-11-07 Ed H. Chi ASC Overview
    23. 24. Augmented Social Cognition Higher Productivity via Collective Intelligence Intelligence that emerges from the collaboration and competition of many individuals <ul><li>Foundation: </li></ul><ul><li>Understanding of human cognition and behavior </li></ul><ul><li>Data mining of social data </li></ul><ul><li>Generic benefits: </li></ul><ul><li>Greater trust </li></ul><ul><li>Better decision-making </li></ul><ul><li>Useful sharing of info </li></ul><ul><li>Auto-organization thru social data </li></ul>Collective Intelligence search sharing foraging <ul><li>TagSearch: Mining social data for automatic data clustering and organization: </li></ul><ul><ul><li>Better organization via user-assigned tags </li></ul></ul><ul><ul><li>Better UI for browsing interesting contents </li></ul></ul><ul><ul><li>Recommendation instead of just search </li></ul></ul><ul><li>Social Transparency create trust and attribution: </li></ul><ul><ul><li>Increase participation via attribution </li></ul></ul><ul><ul><li>Increase credibility and trust with community feedback </li></ul></ul><ul><ul><li>Reduce wiki risks </li></ul></ul><ul><li> sharing of interesting contents: </li></ul><ul><ul><li>A notebook that automatically organizes your reading </li></ul></ul><ul><ul><li>Social sharing of important and interesting tidbits </li></ul></ul><ul><ul><li>Viral sharing of highlighted and tagged paragraphs </li></ul></ul>