Information Seeking with Social Signals: Anatomy of a Social Tag-based Exploratory Search Browser

Information Seeking with Social Signals: Anatomy of aSocial Tag-based Exploratory Search Browser Ed H. Chi, Rowan Nairn Palo Alto Research Center Contact: [email_address] Area Manager, Augmented Social Cognition Area

Social Search Survey [Evans & Chi, CSCW2008]
150 user surveys
Help understand the importance of:
social cues and information exchanges
vocabulary problems
distribution and organization

TagSearch Exploratory Focus 3 kinds of search navigational transactional 28% 13% You know what you want and where it is You know what you want to do Existing search engines are OK informational 59% You roughly know what you want but don’t know how to find it Difficult for existing search engines Opportunity

Research Motivation

MapReduce Implementation
Spreading Activation in a bigraph
MapReduce computation over a large data set
150 Million+ bookmarks
Tags URLs P(URL|Tag) P(Tag|URL)

Use Semantic Analysis to Reduce Noise Guide Web Howto Tips Help Tools Tip Tricks Tutorial Tutorials Reference Semantic Similarity Graph

TagSearch Architecture
MapReduce: months of computation to a single day
Development of novel scoring function

Baseline Interface

Exploratory Interface

Experiment Design
2 interface x 3 task domain design
2 Interface (between-subjects)
Exploratory vs. Baseline
3 task domains (within-subjects)
Future Architecture, Global Warming, Web Mashups
30 Subjects (22 male, 8 female)
Intermediate or advanced computer and web search skills
Half assigned Exploratory, half Baseline.
For each domain, single block with 3 task types:
Easy and Difficult Page Collection Task [6min each]
Summarization Task [12min]
Keyword Generation Task [2min]

Page Collection Tasks [6min each]

Summarization Tasks [12min each]

Procedure [2 hours]
Prior Knowledge Test
1 st Task Domain
With easy and difficult page collection tasks, summarization and keyword generation task.
NASA cognitive load questionnaire
2 nd Task Domain
Same battery of tasks and cognitive load questionaire
3 rd Task Domain
Experimental Survey

Results: Interaction Behaviors
Number of Queries
Effect of Interface on number of queries (p < .01)
Exploratory (M=7.81) > Baseline (M=3.77)
Time Taken
Effect of Interface on time taken (p < .01)
Exploratory (7.7min) > Baseline (6.6min)

Results: Page Collection Task
Effects of Task Domain (p<.01) and Task Difficulty (p<.05)
Interaction effect of Interface by Task Domain (p<.05), with Exploratory interface performing better in the Web Mashup domain
For relevance scores, similar patterns.
Measure of # of pages collected

Results: Summarization Tasks
Quality of summarization scored (Cohen’s Kappa=0.7)
ANCOVA with Prior Knowledge as covariate
Exploratory Interface scored higher in Future Architecture (p<.05) and Global Warming (p<.05)
For Web Mashup, Prior Knowledge correlated positively with performance (r=.51)

Results: Keyword Generation Tasks
ANCOVA showed Exploratory > Baseline for Future Architecture (p<.05) and Web Mashups (p<.01), but not for Global Warming.
Linear model between PK and # of keyword generated for Baseline showed mean slope = 0.32 and significant (p<.05)

Results: Cognitive Load
Exploratory > Baseline (p<.05)

Discussion
Exploratory interface users:
performed more queries,
took more time,
wrote better summaries (in 2/3 domains),
generated more relevant keywords (in 2/3 domains), and
had a higher cognitive load.
Suggestive of deeper engagement and better learning.
Some evidence of scaffolding for novices in the keyword generation and summarization tasks.

Summary
Harnessing user-generated tags to enrich content for social search
Weaknesses of social tagging systems is Tag Noise and Inconsistency
Difficult to leverage for search
Use data mining techniques to normalize and reduce noise
Apply normalized tag data in new search algorithm
Study suggest deeper user engagement in exploration and better learning with MrTaggy

Thanks!
http://mrtaggy.com
http://spartag.us
http://wikidashboard.parc.com
Our Blog: http://asc-parc.blogspot.com
Contact:
Ed H. Chi, Ph.D.
Manager, Augmented Social Cognition Area
[email_address]
Kammerer, Y., Nairn, R., Pirolli, P., and Chi, E. H. 2009. Signpost from the masses: learning effects in an exploratory social tag search browser. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 625-634.

Research Vision Augmented Social Cognition
Cognition : the ability to remember, think, and reason; the faculty of knowing.
Social Cognition : the ability of a group to remember, think, and reason; the construction of knowledge structures by a group.
(not quite the same as in the branch of psychology that studies the cognitive processes involved in social interaction, though included)
Augmented Social Cognition : Supported by systems, the enhancement of the ability of a group to remember, think, and reason; the system-supported construction of knowledge structures by a group.
Citation: Ed H. Chi. The Social Web: Opportunities for Research. IEEE Computer, Sept 2008
2008-11-07 Ed H. Chi ASC Overview

Augmented Social Cognition Higher Productivity via Collective Intelligence Intelligence that emerges from the collaboration and competition of many individuals
Foundation:
Understanding of human cognition and behavior
Data mining of social data
Generic benefits:
Greater trust
Better decision-making
Useful sharing of info
Auto-organization thru social data
Collective Intelligence search sharing foraging
TagSearch: Mining social data for automatic data clustering and organization:
Better organization via user-assigned tags
Better UI for browsing interesting contents
Recommendation instead of just search
Social Transparency create trust and attribution:
Increase participation via attribution
Increase credibility and trust with community feedback
Reduce wiki risks
SparTag.us: sharing of interesting contents:
A notebook that automatically organizes your reading
Social sharing of important and interesting tidbits
Viral sharing of highlighted and tagged paragraphs

Information Seeking with Social Signals: Anatomy of a Social Tag-based Exploratory Search Browser

by Ed Chi on Feb 08, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 6

Statistics

Information Seeking with Social Signals: Anatomy of a Social Tag-based Exploratory Search Browser — Presentation Transcript

http://www.slideshare.net	4
http://health.medicbd.com	2