Social information Access2012


Published on

Slides of my invited talk @ WebMEdia 2012

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The most cited reason of DirectHit failure was low query repetition, which made the social data collected by it too sparse to use frequently and reliably. User diversity was another likely contribution: users with different goals and interests may prefer different results returned by the same query. Finally, the proposed approach to link ranking was too easy to abuse by malicious users who wanted to promote their favorite pages.
  • Three levels of QQ similarity: direct, through document corpus (all documents returned by a query), through user selections (terms in selected documents)
  • AntWorld introduced the concept of a quest , which is an information goal pursued by a user over a sequence of queries (Fig. 3). The system successfully encouraged its users to describe their quests in natural language and used this description to determine inter-quest similarity. During their search, the users were able to rank search results by their relevance to the original quest (not a query used to obtain this result!). These innovations allowed the system to address to some extent the sparsity and reliability problems. To determine documents, which are socially relevant for a particular quest, the system looked for positively ranked documents in past similar quests. The system assisted the user by adding socially relevant documents to the list of search results and also adding a small ant icon to socially relevant links returned during each search within the quest
  • Results with recommendations were shown on over 40% searches. In about 40% of cases the users clicked and 71.6% of these clicks were on recommended links! If only Google results are shown users clicked in only 24.4% of cases - so when social recommendations are provided, chance to click is higher. Also the length of the session is significantly shorter (1.6 vs 2.2) when recommendations are shown. Finally, ratings of the first visited documents are higher if it was recommended (so, appeal and quality both better). In more than 2/3 of cases the users really provided expanded search requests - over 1 sentence! However, regardless of social help, the user rate visited documents only in 2/3 of cases.
  • Progressor+ significantly outperformed QuizJET. Students spent more time per session in Progressor+ than QuizJET. Students spent more time per session in Progressor+ than Progressor. introducing annotated examples to the open social student modeling visualization did not sacrifice the usage in selfassessment quizzes. Providing personalized guidance in open social student modeling interface (Progressor+) was equivalent efficient as non-social open student modeling interface (JavaGuide). More than that, students did spend more time in studying the annotated examples. Whoever works on 1 type of content, more likely to work on the other type (r=0.81, p<.01) *between subject ANOVA, used Bonferroni adjustment, that is the most conservative method.
  • - Problem solving importance to knowledge acquisition Why not perfect (100%) - knowledge-based and social-based combination indeed brought added value to the system, where the knowledge-based personalization alone did not.
  • common: general pattern on exam preparation, especially in final exam period
  • Social information Access2012

    1. 1. Social Information Access Peter Brusilovskywith Rosta Farzan, Jaewook Ahn, Sharon Hsiao, Denis Parra, Michael Yudelson,Chirayu Wongchokprasitti, Sherry Sahebi School of Information Sciences University of Pittsburgh
    2. 2. PAWS Lab (at UMAP 2012)
    3. 3. The New Web: the Web of People
    4. 4. Web 2.0: Fast Start, Broad Spread • Term was introduced following the first OReilly Media Web 2.0 conference in 2004 • By September 2005, a Google search for Web 2.0 returned more than 9.5 million results • In 2012 similar search returned over 2 billion results
    5. 5. Social Web of Web 2.0? Social Web Web 2.0
    6. 6. The Social Web
    7. 7. Key Elements • The Users’ Web • User as a first-class • Collective participant, Intelligence: contributor, author Wisdom of Crowds • The power of the user • Applications powered by user community • Stigmergy
    8. 8. Amazon: Reviews and ratings
    9. 9. eBay: Driving a marketplace
    10. 10. Wikipedia: Providing content
    11. 11. Delicious: Sharing + Organization
    12. 12. Social Linking: Identity + Links
    13. 13. Publish Your Self: [Micro]Blogs
    14. 14. The Other Side of the Social Web User contentUser interaction Which wisdom of crowds?
    15. 15. Social Information Access Methods for organizing users’ past interaction with an information system (known as explicit and implicit feedback), in order to provide better access to information to the future users of the system
    16. 16. Critical Questions • What kind of past interaction to take into account? • How to process it to produce “wisdom of crowds” ? • In which context to reveal it to end users? • How to make wisdom of crowds useful in this context?
    17. 17. Social Information Access: Contexts Social Navigation – Social support of user browsing Social Recommendation (Collaborative Filtering) – Proactive information access Social Search – Social support of search Social Visualization – Social support for visualization-based access to information Social Bookmarking – Access to bookmarked/shared information facilitated with tags
    18. 18. Social Navigation: The Motivation • Natural tendency of people to follow each other Making use of “direct” and “indirect cues about the activities of others Following trails Footsteps in sand or snow Worn-out carpet Using dogears and annotations Giving direction or guidance • Navigation driven by the actions from one or more “advice providers”
    19. 19. The Lost Interaction History What is the difference between walking in a real world and browsing the Web? – Footprints – Worn-out carpet – People presence What is the difference between buying and borrowing a book? – Notes in the margins – Highlights & underlines – Dog-eared pages – Opens more easily to more used places
    20. 20. Edit Wear and Read Wear (1992) The pioneer idea of asynchronous indirect social navigation Developed for collaborating writing and editing Indicated read/edited places in a large document
    21. 21. Footprints (1997) Wexelblat & Maes, 1997 Allowing users to create history-rich objects Providing history-rich navigation in complex information space Showing what percentage of users have followed each link
    22. 22. SN in Information Space:The History History-enriched environments – Edit Wear and Read Wear (1992) – Social navigation systems • Footprints, Juggler, Kalas Collaborative filtering – Manual push and pull • Tapestry, LN Recommender – Modern automatic CF recommender systems Social bookmarking – Collaborative tagging systems Social Search
    23. 23. Social Navigation in Information Space Synchronous Direct Communication in real time Direct communication Asynchronous between people Using the Interaction of past Indirect users Relying on user presence and traces of user behavior Synchronous Asynchronous Recommenders Direct Chats Q/A Systems Indirect History-enriched Presence of other people environments
    24. 24. EDUCO: Synchronous, Indirect SN
    25. 25. Amazon: Asynchronous, Indirect Traces of viewing and purchasing decisions is a valuable collective wisdom! •Compare with an Amazon review: “the remake of this movie is horrible, I recommend to watch the original version instead”
    26. 26. CourseAgent: Direct, Asynchronous• Adaptive community-based course planning system –Provides social navigation through visual cues
    27. 27. Ratings: Raw Social Data
    28. 28. Generating Social Navigation Overall workload Averaging over all ratings of the community Overall Relevance Average does not work Irrelevant to many but very relevant to one Goal-centered algorithm 16 rules 29
    29. 29. Trade-offs for Direct Approach • Reasonably reliable • Feedback directly provided • No need to deduce and guess • Explicit feedback is hard to obtain • Takes time to provide and requires commitment • “One out of a hundred” • Social system, which extensively relies on explicit feedback need either large community of users or special approaches to motivate direct contributions
    30. 30. Adding Motivation: Career Planning 31
    31. 31. The Intrinsic Motivation Works • Career Planning was not advertised and was not noticed and used by half of the students • Contribution of experimental users who did not use Career planning (experimental group I) is close to control group • Significant increase of all contributions for those who had and used Career planning (experimental group II) 32
    32. 32. More about CourseAgent Farzan, R. and Brusilovsky, P. (2006) Social navigation support in a course recommendation system. In: V. Wade, H. Ashman and B. Smyth (eds.) Proceedings of 4th International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH2006), Dublin, Ireland, June 21-23, 2006, Springer Verlag, pp. 91-100. Farzan, R. and Brusilovsky, P. (2011) Encouraging User Participation in a Course Recommender System: An Impact on User Behavior. Computers in Human Behavior 27 (1), 276-284.
    33. 33. Knowledge Sea II: Indirect, Asynchronous•Social Navigation to support course readings
    34. 34. Knowledge Sea II (+ AnnotatEd)
    35. 35. Trade-off for indirect approach • Feedback is easy to get • Users provide feedback simply by navigating and doing other regular actions • It works quite well • Most useful pages tend to rise as socially important • Social navigation cues attract users • Indirect feedback might not be reliable • A click or other action in the interface is a small commitment, may be a result of error • “Tar pits” • Main challenge of systems based on indirect approach: increase the reliability of indirect feedback • Better processing of unreliable events (time, scrolling) • Use more reliable events (cf. browsing vs. purchase)
    36. 36. Knowledge See II: Beyond clicks • Make better use of existing feedback • Switched from click-based calculation of user traffic to time based • Time and patterns can provide more reliable evidence • Added annotation-based social navigation • Annotations are more reliable • Users are eager to provide annotations and even categorize them into positive/regular
    37. 37. Spatial Annotation InterfaceA Spatial Annotation Interface adds socialnavigation on the page level Staking a space Commenting BooksOnline08 38
    38. 38. Page-level Navigation SupportVisual Cues - annotation background and borderBackground Style•Background fillingOwnership•Background colorOwner’s attitudeBorder style•Border colorPositiveness•Border thickness# of comments•Border strokePublic or personal BooksOnline08 39
    39. 39. Annotation-based SN does work • Usage • With additional navigation support map-based and browsing-based access emerged as the primary access way • Effect on navigation • Significant increase of link following (pro-rated normalized access) • Impact • Annotation leads students to valuable pages
    40. 40. Back to Motivation IssueAnnotations are explicit actions used for implicit feedback and aswith all explicit actions, it come with motivation problems. BooksOnline08
    41. 41. More on KS-II and AnnotatEd Farzan, R. and Brusilovsky, P. (2005) Social navigation support through annotation-based group modeling. In: L. Ardissono, P. Brna and A. Mitrovic (eds.) Proceedings of 10th International User Modeling Conference, Berlin, July 24-29, 2005, Springer Verlag, pp. 463-472 Farzan, R. and Brusilovsky, P. (2008) AnnotatEd: A social navigation and annotation service for web-based educational resources. New Review in Hypermedia and Multimedia 14 (1), 3-32. Brusilovsky, P. and Kim, J. (2009) Enhancing Electronic Books with Spatial Annotation and Social Navigation Support. In: Proceedings of the 5th International Conference on Universal Digital Library (ICUDL 2009), Pittsburgh, PA, November 6-8, 2009
    42. 42. What is Social Search? - Social Information Access in Search context - A set of techniques focusing on: • collecting, processing, and organizing traces of users’ past interactions • applying this “community wisdom” in order to improve search-based access to information
    43. 43. Variables Defining Social Search Which users? • Creators • Consumers What kind of interaction is considered? • Browsing • Searching • Annotation • Tagging What kind of search process improvement? • Off-line performance improvement of search engines • On-line user assistance
    44. 44. The Case of Google PageRank Which users? Which activity? What is affected? How it is affected? How it improves search?
    45. 45. How Search Could be Changed? Let’s classify potential impact by stages Before search During search After search
    46. 46. Search Engines: Improve Finding Use social data to expand document index (document expansion) What we can get from page authors? Anchor text provided on a link to the page What we can get from searchers? Page selection in response to the query (Scholer, 2002) Query sequences (Amitay, 2005) What we can get from page visitors? Page annotations (Dmitriev et al., 2006) Page tags (Yanbe, 2007)
    47. 47. Search Engines: Improve Ranking What we can get from page authors? Links (Page Rank) What we can get from searchers? Page selection in response to the query (DirectHit) What we can get from page visitors beyond seatch context? Page visit count Page tags (Yanbe, 2007; Bao, 2007) Page annotations Combined approaches PageRate (Zhu, 2001), (Agichtein, 2006)
    48. 48. Using Social Wisdom Before Search Can be done by both search engines and external interfaces Query checking - now standard Suggesting improved/related queries Example: query networks (Glance, 2001) Automatic query refinement and query expansion Using past queries and query sequences - what the user is really looking for (Fitzpatrick, 1997; Billerbeck, 2003; Huang, 2003) Using anchors (Kraft, 2004) Using annotations, tags
    49. 49. Using Social Wisdom After Search Better ranking, link promotion • Link re-ordering using social wisdom (based on the result selection traces by earlier searchers) Suggesting additional results • Suggest results (or sites!) found by earlier searchers Providing social annotations • Link popularity, past link selection by socially connected users
    50. 50. Challenges of Social Search • Matching similar users • Number of page hits is not reliable (DirectHit failure) • Using “everyone” social data is a bad idea – need not good pages overall, but those that match a query • Even matching with users who issue the same query is not reliable enough – same query, very different goals! • Reliability of social feedback • A click on a result link is not a reliable evidence of quality and relevance • Need to do a wise mining of search sessions and sequences • Fusing query relevance and social wisdom • Single ranking is not the best way to express two dimensions of relevance
    51. 51. AntWorld: Quest-Based Approach – Quests establish similarities between users – Relevance between documents and quests is provided by explicit feedback
    52. 52. Quest Approach to Social Search Evaluation of Quest approach: SERF (Jung, 2004) – Results with recommendations were shown on over 40% searches. – In about 40% of cases the users clicked and 71.6% of these clicks were on recommended links! If only Google results are shown users clicked in only 24.4% of cases – The length of the session is significantly shorter (1.6 vs 2.2) when recommendations are shown – Ratings of the first visited document are higher if it was recommended (so, appeal and quality both better)
    53. 53. I-SPY: Community-Based Search
    54. 54. I-SPY: Mechanism Community-query-hit matrix User similarity defined by communities and queries Result selection provide implicit feedback
    55. 55. Other Ways to Increase Reliability • Moving from single query to query sequences • What the user selected at the end • Moving from page recommendation to site recommendation White, R., Bilenko, M., and Cucerzan, S. (2007) Studying the use of popular destinations to enhance web search interaction. In: SIGIR 07, Amsterdam, The Netherlands, July 23 - 27, 2007, ACM Press, pp. 159-166
    56. 56. Social Search with Visual Cues Query relevance and social relevance shown separately: rank/annotation Similarity score General annotation Question Document with high traffic (higher rank) Praise Negative Document with positive annotation (higher rank) Positive
    57. 57. Annotation-Based Search: Impact Acceptance – Users noticed and applied social visual cues • Frequency of usage - viewed more documents per query with social visual cues – Users agreed with the need for social search • Survey results Performance – Social Visual Cues are taken into account for navigation • Social Navigation cues are twice as more influential in affecting user navigation decision than high rank – Social visual Cues provide higher prediction for page quality that high rank More information – Ahn, J.-w., Farzan, R., and Brusilovsky, P. (2006) Social search in the context of social navigation. Journal of the Korean Society for Information Management 23 (2), 147-165.
    58. 58. SIA Challenges across Contexts • Increasing reliability of indirect sources • Time spent reading vs. simple click • Query sequences vs. simple result access • Adding more reliable evidences of relevance/quality/interests • Annotation vs. browsing • Purchasing/downloading vs. viewing • May add the problem of motivation! • Basis for user similarity (not “all for all”) • Co-rating in recommender systems (sparsity!) • Users with similar goals (CourseAgent) • Single class in Knowledge Sea II (still topic drift!) • Quest or community in AntWorld and iSpy
    59. 59. More Challenges:Merging the Technologies • Different branches of SIA have little connections to each other • Social navigation use navigation data to assist navigation • Social search use search traces to assist future searchers • Many opportunities to merge two or more SIA technologies • Social Web system with broader SIA • Use several kinds of user traces to support a specific SIA technology • Offer several kinds of SIA • Earlier work: Social Navigation + Social Search – ASSIST ACM – ASSIST YouTube • Social Navigation + Recommendation • Adding Social Visualization
    60. 60. ASSIST-ACM: Social Search + Nav Re-ranking result-list Augmenting the links based on search and based on search and browsing history browsing history information information Farzan, R., et al. (2007) ASSIST: adaptive social support for information space traversal. In: Proceedings of 18th conference on Hypertext and hypermedia, HT 07,, pp. 199-208
    61. 61. CoMeT: Social wisdom for talks
    62. 62. Some New Ideas in CoMeT • Broader set of evidences • View, annotate, tag, schedule talks, send to friends, connect to peers • Declare affiliations (similarity!) • Join and post links to a set of communities • Combining in-context (visual cues) and out-of context (ranking) guidance • Exploring the power of “top N” • Powerful, but dangerous!
    63. 63. Conference Navigator Project • Social conference support system – combining social and personalized guidance
    64. 64. Social Visualization with VIBE
    65. 65. Social Visualization in CN3 TalkExplorer: Integrating recommendations and SNS visually
    66. 66. Community vs. Peer-Based NS: E-learning • Progressor and Progressor+ projects • Problem: guide students to most appropriate educational content – examples, problems, etc. • Using reliable indicators of student progress (problem solving success) • Provide visualization to better support guidance • Explore peer-based and community-based SNS
    67. 67. Parallel Introspective Views 68
    68. 68. Progressor 69
    69. 69. Progressor+ 70
    70. 70. Students spent more time in Progressor+ Quiz =: 5 hours 71 Example : 5 hours 20 mins
    71. 71. Students achieved higher Success Rate p<.01 72
    72. 72. How Social Guidance WorksNon-adaptive adaptiveSocial, adaptive, single content Progressor+ 73