The most cited reason of DirectHit failure was low query repetition, which made the social data collected by it too sparse to use frequently and reliably. User diversity was another likely contribution: users with different goals and interests may prefer different results returned by the same query. Finally, the proposed approach to link ranking was too easy to abuse by malicious users who wanted to promote their favorite pages.
Three levels of QQ similarity: direct, through document corpus (all documents returned by a query), through user selections (terms in selected documents)
AntWorld introduced the concept of a quest , which is an information goal pursued by a user over a sequence of queries (Fig. 3). The system successfully encouraged its users to describe their quests in natural language and used this description to determine inter-quest similarity. During their search, the users were able to rank search results by their relevance to the original quest (not a query used to obtain this result!). These innovations allowed the system to address to some extent the sparsity and reliability problems. To determine documents, which are socially relevant for a particular quest, the system looked for positively ranked documents in past similar quests. The system assisted the user by adding socially relevant documents to the list of search results and also adding a small ant icon to socially relevant links returned during each search within the quest
Results with recommendations were shown on over 40% searches. In about 40% of cases the users clicked and 71.6% of these clicks were on recommended links! If only Google results are shown users clicked in only 24.4% of cases - so when social recommendations are provided, chance to click is higher. Also the length of the session is significantly shorter (1.6 vs 2.2) when recommendations are shown. Finally, ratings of the first visited documents are higher if it was recommended (so, appeal and quality both better). In more than 2/3 of cases the users really provided expanded search requests - over 1 sentence! However, regardless of social help, the user rate visited documents only in 2/3 of cases.
Progressor+ significantly outperformed QuizJET. Students spent more time per session in Progressor+ than QuizJET. Students spent more time per session in Progressor+ than Progressor. introducing annotated examples to the open social student modeling visualization did not sacrifice the usage in selfassessment quizzes. Providing personalized guidance in open social student modeling interface (Progressor+) was equivalent efficient as non-social open student modeling interface (JavaGuide). More than that, students did spend more time in studying the annotated examples. Whoever works on 1 type of content, more likely to work on the other type (r=0.81, p<.01) *between subject ANOVA, used Bonferroni adjustment, that is the most conservative method.
- Problem solving importance to knowledge acquisition Why not perfect (100%) - knowledge-based and social-based combination indeed brought added value to the system, where the knowledge-based personalization alone did not.
common: general pattern on exam preparation, especially in final exam period
Social Information Access Peter Brusilovskywith Rosta Farzan, Jaewook Ahn, Sharon Hsiao, Denis Parra, Michael Yudelson,Chirayu Wongchokprasitti, Sherry Sahebi School of Information Sciences University of Pittsburgh http://www.sis.pitt.edu/~peterb
The New Web: the Web of People http://www.veryweb.it/?page_id=27
Web 2.0: Fast Start, Broad Spread • Term was introduced following the first OReilly Media Web 2.0 conference in 2004 • By September 2005, a Google search for Web 2.0 returned more than 9.5 million results • In 2012 similar search returned over 2 billion results http://datamining.typepad.com/data_mining/2005/12/the_rise_and_ri.html
Key Elements • The Users’ Web • User as a first-class • Collective participant, Intelligence: contributor, author Wisdom of Crowds • The power of the user • Applications powered by user community • Stigmergy http://www.masternewmedia.org/news/2006/12/01/social_bookmarking_services_and_tools.ht
The Other Side of the Social Web User contentUser interaction Which wisdom of crowds?
Social Information Access Methods for organizing users’ past interaction with an information system (known as explicit and implicit feedback), in order to provide better access to information to the future users of the system
Critical Questions • What kind of past interaction to take into account? • How to process it to produce “wisdom of crowds” ? • In which context to reveal it to end users? • How to make wisdom of crowds useful in this context?
Social Information Access: Contexts Social Navigation – Social support of user browsing Social Recommendation (Collaborative Filtering) – Proactive information access Social Search – Social support of search Social Visualization – Social support for visualization-based access to information Social Bookmarking – Access to bookmarked/shared information facilitated with tags
Social Navigation: The Motivation • Natural tendency of people to follow each other Making use of “direct” and “indirect cues about the activities of others Following trails Footsteps in sand or snow Worn-out carpet Using dogears and annotations Giving direction or guidance • Navigation driven by the actions from one or more “advice providers”
The Lost Interaction History What is the difference between walking in a real world and browsing the Web? – Footprints – Worn-out carpet – People presence What is the difference between buying and borrowing a book? – Notes in the margins – Highlights & underlines – Dog-eared pages – Opens more easily to more used places
Edit Wear and Read Wear (1992) The pioneer idea of asynchronous indirect social navigation Developed for collaborating writing and editing Indicated read/edited places in a large document
Footprints (1997) Wexelblat & Maes, 1997 Allowing users to create history-rich objects Providing history-rich navigation in complex information space Showing what percentage of users have followed each link
SN in Information Space:The History History-enriched environments – Edit Wear and Read Wear (1992) – Social navigation systems • Footprints, Juggler, Kalas Collaborative filtering – Manual push and pull • Tapestry, LN Recommender – Modern automatic CF recommender systems Social bookmarking – Collaborative tagging systems Social Search
Social Navigation in Information Space Synchronous Direct Communication in real time Direct communication Asynchronous between people Using the Interaction of past Indirect users Relying on user presence and traces of user behavior Synchronous Asynchronous Recommenders Direct Chats Q/A Systems Indirect History-enriched Presence of other people environments
Amazon: Asynchronous, Indirect Traces of viewing and purchasing decisions is a valuable collective wisdom! •Compare with an Amazon review: “the remake of this movie is horrible, I recommend to watch the original version instead”
CourseAgent: Direct, Asynchronous• Adaptive community-based course planning system –Provides social navigation through visual cues http://halley.exp.sis.pitt.edu/courseagent/
Generating Social Navigation Overall workload Averaging over all ratings of the community Overall Relevance Average does not work Irrelevant to many but very relevant to one Goal-centered algorithm 16 rules 29
Trade-offs for Direct Approach • Reasonably reliable • Feedback directly provided • No need to deduce and guess • Explicit feedback is hard to obtain • Takes time to provide and requires commitment • “One out of a hundred” • Social system, which extensively relies on explicit feedback need either large community of users or special approaches to motivate direct contributions
The Intrinsic Motivation Works • Career Planning was not advertised and was not noticed and used by half of the students • Contribution of experimental users who did not use Career planning (experimental group I) is close to control group • Significant increase of all contributions for those who had and used Career planning (experimental group II) 32
More about CourseAgent Farzan, R. and Brusilovsky, P. (2006) Social navigation support in a course recommendation system. In: V. Wade, H. Ashman and B. Smyth (eds.) Proceedings of 4th International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH2006), Dublin, Ireland, June 21-23, 2006, Springer Verlag, pp. 91-100. Farzan, R. and Brusilovsky, P. (2011) Encouraging User Participation in a Course Recommender System: An Impact on User Behavior. Computers in Human Behavior 27 (1), 276-284.
Knowledge Sea II: Indirect, Asynchronous•Social Navigation to support course readings
Trade-off for indirect approach • Feedback is easy to get • Users provide feedback simply by navigating and doing other regular actions • It works quite well • Most useful pages tend to rise as socially important • Social navigation cues attract users • Indirect feedback might not be reliable • A click or other action in the interface is a small commitment, may be a result of error • “Tar pits” • Main challenge of systems based on indirect approach: increase the reliability of indirect feedback • Better processing of unreliable events (time, scrolling) • Use more reliable events (cf. browsing vs. purchase)
Knowledge See II: Beyond clicks • Make better use of existing feedback • Switched from click-based calculation of user traffic to time based • Time and patterns can provide more reliable evidence • Added annotation-based social navigation • Annotations are more reliable • Users are eager to provide annotations and even categorize them into positive/regular
Spatial Annotation InterfaceA Spatial Annotation Interface adds socialnavigation on the page level Staking a space Commenting BooksOnline08 38
Page-level Navigation SupportVisual Cues - annotation background and borderBackground Style•Background fillingOwnership•Background colorOwner’s attitudeBorder style•Border colorPositiveness•Border thickness# of comments•Border strokePublic or personal BooksOnline08 39
Annotation-based SN does work • Usage • With additional navigation support map-based and browsing-based access emerged as the primary access way • Effect on navigation • Significant increase of link following (pro-rated normalized access) • Impact • Annotation leads students to valuable pages
Back to Motivation IssueAnnotations are explicit actions used for implicit feedback and aswith all explicit actions, it come with motivation problems. BooksOnline08
More on KS-II and AnnotatEd Farzan, R. and Brusilovsky, P. (2005) Social navigation support through annotation-based group modeling. In: L. Ardissono, P. Brna and A. Mitrovic (eds.) Proceedings of 10th International User Modeling Conference, Berlin, July 24-29, 2005, Springer Verlag, pp. 463-472 Farzan, R. and Brusilovsky, P. (2008) AnnotatEd: A social navigation and annotation service for web-based educational resources. New Review in Hypermedia and Multimedia 14 (1), 3-32. Brusilovsky, P. and Kim, J. (2009) Enhancing Electronic Books with Spatial Annotation and Social Navigation Support. In: Proceedings of the 5th International Conference on Universal Digital Library (ICUDL 2009), Pittsburgh, PA, November 6-8, 2009
What is Social Search? - Social Information Access in Search context - A set of techniques focusing on: • collecting, processing, and organizing traces of users’ past interactions • applying this “community wisdom” in order to improve search-based access to information
Variables Defining Social Search Which users? • Creators • Consumers What kind of interaction is considered? • Browsing • Searching • Annotation • Tagging What kind of search process improvement? • Off-line performance improvement of search engines • On-line user assistance
The Case of Google PageRank Which users? Which activity? http://www.labnol.org/internet/google-pagerank-drop-stop-worrying/4835/ What is affected? How it is affected? How it improves search?
How Search Could be Changed? Let’s classify potential impact by stages Before search During search After search
Search Engines: Improve Finding Use social data to expand document index (document expansion) What we can get from page authors? Anchor text provided on a link to the page What we can get from searchers? Page selection in response to the query (Scholer, 2002) Query sequences (Amitay, 2005) What we can get from page visitors? Page annotations (Dmitriev et al., 2006) Page tags (Yanbe, 2007)
Search Engines: Improve Ranking What we can get from page authors? Links (Page Rank) What we can get from searchers? Page selection in response to the query (DirectHit) What we can get from page visitors beyond seatch context? Page visit count Page tags (Yanbe, 2007; Bao, 2007) Page annotations Combined approaches PageRate (Zhu, 2001), (Agichtein, 2006)
Using Social Wisdom Before Search Can be done by both search engines and external interfaces Query checking - now standard Suggesting improved/related queries Example: query networks (Glance, 2001) Automatic query refinement and query expansion Using past queries and query sequences - what the user is really looking for (Fitzpatrick, 1997; Billerbeck, 2003; Huang, 2003) Using anchors (Kraft, 2004) Using annotations, tags
Using Social Wisdom After Search Better ranking, link promotion • Link re-ordering using social wisdom (based on the result selection traces by earlier searchers) Suggesting additional results • Suggest results (or sites!) found by earlier searchers Providing social annotations • Link popularity, past link selection by socially connected users
Challenges of Social Search • Matching similar users • Number of page hits is not reliable (DirectHit failure) • Using “everyone” social data is a bad idea – need not good pages overall, but those that match a query • Even matching with users who issue the same query is not reliable enough – same query, very different goals! • Reliability of social feedback • A click on a result link is not a reliable evidence of quality and relevance • Need to do a wise mining of search sessions and sequences • Fusing query relevance and social wisdom • Single ranking is not the best way to express two dimensions of relevance
AntWorld: Quest-Based Approach – Quests establish similarities between users – Relevance between documents and quests is provided by explicit feedback
Quest Approach to Social Search Evaluation of Quest approach: SERF (Jung, 2004) – Results with recommendations were shown on over 40% searches. – In about 40% of cases the users clicked and 71.6% of these clicks were on recommended links! If only Google results are shown users clicked in only 24.4% of cases – The length of the session is significantly shorter (1.6 vs 2.2) when recommendations are shown – Ratings of the first visited document are higher if it was recommended (so, appeal and quality both better)
I-SPY: Mechanism Community-query-hit matrix User similarity defined by communities and queries Result selection provide implicit feedback
Other Ways to Increase Reliability • Moving from single query to query sequences • What the user selected at the end • Moving from page recommendation to site recommendation White, R., Bilenko, M., and Cucerzan, S. (2007) Studying the use of popular destinations to enhance web search interaction. In: SIGIR 07, Amsterdam, The Netherlands, July 23 - 27, 2007, ACM Press, pp. 159-166
Social Search with Visual Cues Query relevance and social relevance shown separately: rank/annotation Similarity score General annotation Question Document with high traffic (higher rank) Praise Negative Document with positive annotation (higher rank) Positive
Annotation-Based Search: Impact Acceptance – Users noticed and applied social visual cues • Frequency of usage - viewed more documents per query with social visual cues – Users agreed with the need for social search • Survey results Performance – Social Visual Cues are taken into account for navigation • Social Navigation cues are twice as more influential in affecting user navigation decision than high rank – Social visual Cues provide higher prediction for page quality that high rank More information – Ahn, J.-w., Farzan, R., and Brusilovsky, P. (2006) Social search in the context of social navigation. Journal of the Korean Society for Information Management 23 (2), 147-165.
SIA Challenges across Contexts • Increasing reliability of indirect sources • Time spent reading vs. simple click • Query sequences vs. simple result access • Adding more reliable evidences of relevance/quality/interests • Annotation vs. browsing • Purchasing/downloading vs. viewing • May add the problem of motivation! • Basis for user similarity (not “all for all”) • Co-rating in recommender systems (sparsity!) • Users with similar goals (CourseAgent) • Single class in Knowledge Sea II (still topic drift!) • Quest or community in AntWorld and iSpy
More Challenges:Merging the Technologies • Different branches of SIA have little connections to each other • Social navigation use navigation data to assist navigation • Social search use search traces to assist future searchers • Many opportunities to merge two or more SIA technologies • Social Web system with broader SIA • Use several kinds of user traces to support a specific SIA technology • Offer several kinds of SIA • Earlier work: Social Navigation + Social Search – ASSIST ACM – ASSIST YouTube • Social Navigation + Recommendation • Adding Social Visualization
ASSIST-ACM: Social Search + Nav Re-ranking result-list Augmenting the links based on search and based on search and browsing history browsing history information information Farzan, R., et al. (2007) ASSIST: adaptive social support for information space traversal. In: Proceedings of 18th conference on Hypertext and hypermedia, HT 07,, pp. 199-208
CoMeT: Social wisdom for talks http://pittcomet.info
Some New Ideas in CoMeT • Broader set of evidences • View, annotate, tag, schedule talks, send to friends, connect to peers • Declare affiliations (similarity!) • Join and post links to a set of communities • Combining in-context (visual cues) and out-of context (ranking) guidance • Exploring the power of “top N” • Powerful, but dangerous!
Conference Navigator Project • Social conference support system – combining social and personalized guidance
Social Visualization in CN3 TalkExplorer: Integrating recommendations and SNS visually
Community vs. Peer-Based NS: E-learning • Progressor and Progressor+ projects • Problem: guide students to most appropriate educational content – examples, problems, etc. • Using reliable indicators of student progress (problem solving success) • Provide visualization to better support guidance • Explore peer-based and community-based SNS