2010-03-10 PARC Augmented Social Cognition Research Overview

  • 846 views
Uploaded on

This is an overview of the 3-year research works done at the Augmented Social Cognition research group at PARC. …

This is an overview of the 3-year research works done at the Augmented Social Cognition research group at PARC.

See blog at:
http://asc-parc.blogspot.com

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
846
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
17
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Ed  H.  Chi,  Area  Manager   Peter  Pirolli,  Lichan  Hong,  Bongwon  Suh,  Gregorio  Convertino,     Les  Nelson,  Rowan  Nairn   Augmented  Social  Cognition  Area   Palo  Alto  Research  Center   Interns:  Sanjay  Kairam,  Jilin  Chen,  Michael  Bernstein   Alumni:  Raluca  Budiu,  Bryan  Pendleton,  Niki  Kittur,  Todd  Mytkowicz,   Terrell  Russell,  Brynn  Evans,  Bryan  Chan,  KMRC  students   2009-05-01 Ed H. Chi ASC Overview 1 Image from: http://www.flickr.com/photos/ourcommon/480538715/
  • 2. 14 years of work in foraging and sensemaking   Information  Scent   –  WUFIS  /  IUNIS  (Basic  scent  modeling  algorithms)   [CHI2000,2001]   –  Bloodhound  (Simulation  of  web  navigation)  [CHI2003]   –  LumberJack  (Log  analysis  of  user  needs)  [CHI2002]     Information  Foraging   –  ScentTrails  [TOCHI2003]   –  ScentIndex  [CHI2004]   –  ScentHighlight  [IUI2005]   –  Visual  foraging  of  highlighted  text  [HCII]     Sensemaking   –  Visualization  of  Web  Ecologies  [CHI98]   –  Visualization  Spreadsheets  [Infovis97,  Infovis99]   2009-05-01 Ed H. Chi ASC Overview 2
  • 3. Wikipedia is the best thing ever. Anyone in the world can write anything they want about any subject, so you know you’re getting the best possible information.” – Steve Carell, The Office 2009-05-01 Ed H. Chi ASC Overview 3
  • 4. 2009-05-01 Ed H. Chi ASC Overview 4
  • 5.   Groups  utilize  systems  to   make  sense  and  share   complex  topics  and   materials.     Wikipedia  (social  status)     Slashdot  (karma  points)     WikiHow.com     Lostpedia.com   2009-05-01 Ed H. Chi ASC Overview 5
  • 6.   Systems  that  evolve  structures   that  can  be  used  to  organize   information.     Del.icio.us       Flickr       YouTube       Friendster   2009-05-01 Ed H. Chi ASC Overview 6
  • 7.   Counting  votes   –  A  way  to  increase  signal-­‐to-­‐noise  ratio   –  Information  faddishness     Examples:   –  Digg.com   –  Most  bookmarked  items  on  del.icio.us   –  Estimating  the  weight  of  an  ox  or   temperature  of  a  room   –  The  true  value  of  a  stock   –  PageRank  or  Hub  /  Authority  algorithms   2009-05-01 Ed H. Chi ASC Overview 7
  • 8. Voting systems Col. Information Collaborative Structures Co-Creation Digg.com eHow.com IBM dogear Wikipedia PageRank Del.icio.us Flickr Slashdot Naver Heavier collaboration 2009-05-01 Ed H. Chi ASC Overview 8
  • 9. Voting systems Col. Information Collaborative Structures Co-Creation Digg.com Understanding of eHow.com Understanding of info Understanding of micro-economics and social networks IBM dogear Wikipedia conflicts and PageRank coordination •  of foraging [PARC] Del.icio.us Flickr •  Tag network analysis [PARC, Slashdot Naver Golder, Yahoo] •  Wikipedia coordination •  Personal vs. group costs [PARC] [Huberman, Adamic] •  Structural holes (info brokerage) Heavier •  Invisible Colleges [Sandstrom] •  Wisdom of Crowd [Burt] collaboration effects [Pirolli] •  Interference [Surowieki] •  Network constraints and •  Co-laboratories [Olson and •  Information cascades structure [various] Olson] •  Community networks / Col. [Anderson and Holt] •  Semantic of semiotic structures / Problem solving [Carroll] words [IR, LSA] 2009-05-01 Ed H. Chi ASC Overview 9
  • 10.   Cognition:  the  ability  to  remember,  think,  and  reason;  the  faculty  of   knowing.     Social  Cognition:  the  ability  of  a  group  to  remember,  think,  and   reason;  the  construction  of  knowledge  structures  by  a  group.   –  (not  quite  the  same  as  in  the  branch  of  psychology  that  studies  the   cognitive  processes  involved  in  social  interaction,  though  included)     Augmented  Social  Cognition:  Supported  by  systems,  the   enhancement    of  the  ability  of  a  group  to  remember,  think,  and   reason;  the  system-­‐supported  construction  of  knowledge   structures  by  a  group.     Citation:  Chi,  IEEE  Computer,  Sept  2008   2009-05-01 Ed H. Chi ASC Overview 10
  • 11. Characteriza*on   Models   Evalua*ons   Prototypes   2009-05-01 Ed H. Chi ASC Overview 11
  • 12. Characteriza*on   Models   Evalua*ons   Prototypes   2009-05-01 Ed H. Chi ASC Overview 12
  • 13. 100% 95% Maintenance 90% Percentage of total edits Other 85% 80% User Talk 75% User 70% Article Talk 65% Article 60% 2001 2002 2003 2004 2005 2006 2009-05-01 Ed H. Chi ASC Overview 13
  • 14.   Conflict  is  growing  at  the  global  level,  and  we  have   some  idea  about  where  it  is.     But  what  defines  conflict  inside  Wikipedia?     Build  a  characterization  model  of  article  conflict   –  Identify  metrics  relevant  to  conflict   –  Automatically  identify  high-­‐conflict  articles   2009-05-01 Ed H. Chi ASC Overview 14
  • 15.   Controversial”  tag     Use  #  revisions  tagged  controversial   2009-05-01 Ed H. Chi ASC Overview 15
  • 16.   Possible  metrics  for  identifying  conflict  in  articles   Metric type Page Type Revisions (#) Article, talk, article/talk Page length Article, talk, article/talk Unique editors Article, talk, article/talk Unique editors / revisions Article, talk Links from other articles Article, talk Links to other articles Article, talk Anonymous edits (#, %) Article, talk Administrator edits (#, %) Article, talk Minor edits (#, %) Article, talk Reverts (#, by unique Article editors) 2009-05-01 Ed H. Chi ASC Overview 16
  • 17.   5x  cross-­‐validation,  R2  =  0.897   10000 9000 Actual controversial revisions 8000 7000 6000 5000 4000 3000 2000 1000 0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 2009-05-01 Predicted controversial revisions Ed H. Chi ASC Overview 17
  • 18.   5x  cross-­‐validation,  R2  =  0.897   10000 9000 Actual controversial revisions 8000 7000 6000 5000 4000 3000 2000 1000 0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Predicted controversial revisions 2009-05-01 Ed H. Chi ASC Overview 18
  • 19.   Highly weighted features of conflict model:  Revisions  (talk)    Minor  edits  (talk)    Unique  editors  (talk)    Revisions  (article)    Unique  editors  (article)    Anonymous  edits  (talk)    Anonymous  edits  (article)   2009-05-01 Ed H. Chi ASC Overview 19
  • 20.   Revert:  Undoing  one  or  more  edits   –  The  page  being  restored  to  a  version  that   existed  sometime  previously.     –  Often  used  to  fight  vandalism     Revert  ratio  as  resistance  metric   –  #  of  reverted  edits  /  #  of  total  edit   –  This  analysis  excludes  vandalism  to  model   “resistance”  
  • 21.   Research  Goal   –  How  can  we  identify  point  of  views  between  users?   –  Group  people  share  a  common  point  of  view     Using  revert  as  proxy  for  disagreement  between  users   –  Revert  edits:            3,711,638  6.3  %  of  total  edits   –  Due  to  vandalism:  577,643  0.99%  of  total  edits  (15.6%  of  reverts)     Force  directed  layout   –  Node:  user,  Edge:  revert  relationship   2009-05-01 Ed H. Chi ASC Overview 21
  • 22. Group D Group A Group B Group C Number of users in user group A B C Total Users with Korean point of view 10 6 0 16 Users with Japanese point of view 1 8 7 16 Neutral or Unidentified 7 3 6 17 2009-05-01 Ed H. Chi ASC Overview 22
  • 23. Anonymous (vandals/ spammers) Sympathetic to husband Mediators Sympathetic to parents 2009-05-01 Ed H. Chi ASC Overview 23
  • 24. Monthly Ratio of Reverted Edits
  • 25. Characteriza*on   Models   Evalua*ons   Prototypes   2009-05-01 Ed H. Chi ASC Overview 25
  • 26. Encoding   Retrieval   “video    people    talks  technology”     h:p://www.ted.com/index.php/speakers   h:p://edge.org   “science    research  cogni*on”   26   2009-05-01 Ed H. Chi ASC Overview 26
  • 27. Concepts   Topics   Users   Documents   Noise   Tags   Decoding   Encoding   T1…Tn   2009-05-01 Ed H. Chi ASC Overview 27
  • 28. 2009-05-01 Ed H. Chi ASC Overview 28
  • 29. 2009-05-01 Ed H. Chi ASC Overview 29
  • 30. 2009-05-01 Ed H. Chi ASC Overview 30
  • 31. Source: Hypertext 2008 study on del.icio.us (Chi & Mytkowicz) 2009-05-01 Ed H. Chi ASC Overview 31
  • 32. Bongwon  Suh,  Gregorio  Convertino,     Ed  H.  Chi,  Peter  Pirolli   Bongwon Suh, Gregorio Convertino, Ed H. Chi, Peter Pirolli. The Singularity is Not Near: Slowing Growth of Wikipedia. In Proc. of WikiSym 2009. Oct, 2009. Florida, USA 2009-05-01 Ed H. Chi ASC Overview 32
  • 33. Monthly Edits
  • 34. Monthly Active Editors
  • 35.   Edits  beget  edits   –  more  number  of  previous  edits,  more  number  of  new  edits   Growth rate depends on current population size N and r = growth rate of the population N(t) = N 0 ⋅ e rt dN = r⋅ N dt Growth rate Current of population € population €
  • 36.   Ecological  population  growth  model   –  r,  growth  rate  of  the  population   –  K,  carrying  capacity  (due  to  resource  limitation)   4000000 3500000 K 3000000 dN N Population 2500000 = r ⋅ N ⋅ (1− ) 2000000 dt K 1500000 1000000 500000 0 2000 2002 2004 2006 2008 2010 Year
  • 37.   Follows  a  logistic  growth  curve   New Article http://en.wikipedia.org/wiki/Wikipedia:Modelling_Wikipedia’s_growth
  • 38.   Carrying  Capacity  as  a  function  of  time.   K(t) Population 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Year
  • 39. Characteriza*on   Models   Evalua*ons   Prototypes   2009-05-01 Ed H. Chi ASC Overview 39
  • 40. Create  a  Living  Laboratory  as  a  platform  to   develop,  test,  and  market  innovations   [Chi,  HCIC  workshop  2009,  HCII  2009,  IEEE  Computer  Sep/2008]   2009-05-01 Ed H. Chi ASC Overview 40
  • 41. Joint  work  with     Bongwon  Suh,  Aniket  Kittur,  Bryan  Pendleton   Bongwon  Suh,  Ed  H.  Chi,  Aniket  Kittur,  Bryan  A.  Pendleton.  Lifting  the  Veil:   Improving  Accountability  and  Social  Transparency  in  Wikipedia  with   WikiDashboard.  In  Proceedings  of  the  ACM  Conference  on  Human-­‐factors  in   Computing  Systems  (CHI2008).  ACM  Press,  2008.  Florence,  Italy.   2009-05-01 Ed H. Chi ASC Overview 41
  • 42.   Social  translucent  for  effective  communication  and  collaboration     [Erickson  and  Kellogg  2002]   –  Make  socially  significant  information  visible  and  salient   –  Support  awareness  of  the  rules  and  constraints   –  Accountability  for  actions     Wikis  can  be  a  prime  candidate   –  Every  edit  is  logged  and  retrievable   –  WikiScanner.com:  analyze  anonymous  IP  edits   –  WikiRage.com:  top  edits   2009-05-01 Ed H. Chi ASC Overview 42
  • 43. 2009-05-01 Ed H. Chi ASC Overview 43
  • 44. 2009-05-01 Ed H. Chi ASC Overview 44
  • 45. 2009-05-01 Ed H. Chi ASC Overview 45
  • 46.   Surfacing  hidden  social  context  to  users     For  readers   –  Any  incidents  in  the  past  e.g.  A  sudden  burst  of  edits?   –  Who  are  the  top  editors?   –  What  is  their  motivation  /  point  of  views  /  expertise  /  topics  of   interest?   –  Help  them  judging  the  quality/trustworthiness/usefulness  of  an   article.     For  writers   –  Measure  expertise  /  contribution  /  reputation   –  Motivate  them  to  be  more  active  /  responsible  (?)   2009-05-01 Ed H. Chi ASC Overview 46
  • 47.   3  x  2  x  2  design   Controversial Uncontroversial Visualization Abortion Volcano High quality •  High stability George Bush Shark •  Low stability •  Baseline (none) Pro-life feminism Disk defragmenter Low quality Scientology and celebrities Beeswax
  • 48.   Users  recruited  via  Amazon’s  Mechanical  Turk   –  253  participants   –  673  ratings   –  7  cents  per  rating   –  Kittur,  Chi,  &  Suh,  CHI  2008:  Crowdsourcing  user  studies     To  ensure  salience  and  valid  answers,  participants   answered:   –  In  what  time  period  was  this  article  the  least  stable?   –  How  stable  has  this  article  been  for  the  last  month?   –  Who  was  the  last  editor?     –  How  trustworthy  do  you  consider  the  above  editor?  
  • 49. 1.  Significant  effect  of  visualization   –  High  >  low,  p  <  .001   2.  Both  positive  and  negative  effects   –  High  >  baseline,  p  <  .001   –  Low  >  baseline,  p  <  .01   3.  No  effect  of  article  uncertainty   –  No  interaction  of  visualization   with  either  quality  or  controversy   –  Robust  across  conditions  
  • 50. Joint  work  with     Rowan  Nairn,  Lawrence  Lee   Kammerer,  Y.,  Nairn,  R.,  Pirolli,  P.,  and  Chi,  E.  H.  2009.  Signpost  from  the  masses:  learning   effects  in  an  exploratory  social  tag  search  browser.  In  Proceedings  of  the  27th   international  Conference  on  Human  Factors  in  Computing  Systems  (Boston,  MA,  USA,   April  04  -­‐  09,  2009).  CHI  '09.  ACM,  New  York,  NY,  625-­‐634.     2009-05-01 Ed H. Chi ASC Overview 52
  • 51.   Help  understand  the   importance  of:   –  social  cues  and  information   exchanges   –  vocabulary  problems   –  distribution  and  organization   2009-05-01 Ed H. Chi ASC Overview 53
  • 52. 3 kinds of search 59% 28% 13% informational navigational transactional You roughly know what you want You know what you want and where it is You know what you want to do but don’t know how to find it Difficult for existing search engines Existing search engines are OK Opportunity 2009-05-01 Ed H. Chi ASC Overview 54
  • 53. Social Tagging Creates Noise •  Synonyms •  Misspellings •  Morphologies People use different tag words to express similar concepts. 2009-05-01 Ed H. Chi ASC Overview 55
  • 54. 2009-05-01 Ed H. Chi ASC Overview 56
  • 55. Semantic Similarity Graph Web Tools Reference Guide Howto Tutorial Tips Help Tip Tutorials Tricks 2009-05-01 Ed H. Chi ASC Overview 57
  • 56. Tags URLs P(URL|Tag) P(Tag|URL)   Spreading  Activation  in  a  bi-­‐graph     Computation  over  a  very  large  data  set   –  150  Million+  bookmarks   2009-05-01 Ed H. Chi ASC Overview 58
  • 57. Database Lucene • Delicious • P(URL|Tag) • Serve up search • Ma.gnolia • P(Tag|URL) results • Tuples of • Pre-computed • Other social cues bookmarks • Bayesian Network patterns in a fast • Well defined APIs • [User, URL, Tags, Inference index Time] Crawling MapReduce Web Server Web Server UI Search Frontend Results •  MapReduce:  months  of  computa*on  to  a  single  day   •  Development  of  novel  scoring  func*on     2009-05-01 Ed H. Chi ASC Overview 59
  • 58.   Exploratory  interface  users:   –  performed  more  queries,     –  took  more  time,     –  wrote  better  summaries  (in  2/3  domains),     –  generated  more  relevant  keywords  (in  2/3  domains),  and   –  had  a  higher  cognitive  load.     Suggestive  of  deeper  engagement  and  better  learning.     Some  evidence  of  scaffolding  for  novices  in  the  keyword   generation  and  summarization  tasks.   2009-05-01 Ed H. Chi ASC Overview 60
  • 59. Joint  work  with   Lichan  Hong,  Raluca  Budiu,  Les  Nelson,  Peter  Pirolli     Lichan  Hong,  Ed  H.  Chi,  Raluca  Budiu,  Peter  Pirolli,  and  Les  Nelson.  SparTag.us:  A  Low   Cost  Tagging  System  for  Foraging  of  Web  Content.  In  Proceedings  of  the  Advanced   Visual  Interface  (AVI2008),  (to  appear).  ACM  Press,  2008.   2009-05-01 Ed H. Chi ASC Overview 61
  • 60.   Interaction  costs   # People willing to produce for “free” determine  number  of   people  who  participate     Surplus  of  attention  &   motivation  at  small   transaction  costs     Therefore…     Important  to  keep   interaction  costs  low   Cost of participation 2009-05-01 Ed H. Chi ASC Overview 62
  • 61.   In situ tagging while reading –  No new window –  Clicking vs typing   Tagging + highlighting 2009-05-01 Ed H. Chi ASC Overview 63
  • 62.   Intuition:  sub-­‐doc  nuggets  useful   –  Entities,  facts,  concepts,  paragraphs     Annotations  attached  to    paragraphs     Portable  across  pages  and  other  contents  (e.g.   Word  documents)   –  Dynamic  pages   –  Duplicate  content   2009-05-01 Ed H. Chi ASC Overview 64
  • 63. 2009-05-01 Ed H. Chi ASC Overview 65
  • 64. 2009-05-01 Ed H. Chi ASC Overview 66
  • 65. 2009-05-01 Ed H. Chi ASC Overview 67
  • 66. 2009-05-01 Ed H. Chi ASC Overview 68
  • 67. N=18 SparTag.us + Friend superior to both individual conditions No difference between the two controls SparTag.us With A Friend (SF) SF group, M=0.46, SD=0.22 SO group, Without M=0.13, SD=0.32 SparTag.us WS group, (WS) M=0.27, SD=0.23 SparTag.us Only (SO) [Nelson et al., CHI2009] 2009-05-01 Ed H. Chi ASC Overview 69
  • 68. Social Transparency create trust and attribution: •  Increase participation via attribution Collective Intelligence •  Increase credibility and trust with community feedback TagSearch: Mining social •  Reduce wiki risks data for automatic data clustering and organization: •  Better organization via user- assigned tags Higher Productivity via •  Better UI for browsing Collective Intelligence interesting contents sharing Generic benefits: •  Recommendation instead of •  Greater trust just search •  Better decision-making Intelligence that emerges •  Useful sharing of info from the collaboration and •  Auto-organization thru search social data competition of many individuals foraging Foundation: SparTag.us: sharing of •  Understanding of human interesting contents: cognition and behavior •  A notebook that automatically •  Data mining of social data organizes your reading •  Modeling of consensus- •  Social sharing of important and interesting tidbits driven decision-making •  Viral sharing of highlighted and tagged paragraphs 2008-10-28 Ed H. Chi ASC Overview 70
  • 69.   ASC is creating a plug-and-play platform to enable a number of applications in support of the Open Web Applications Social Data Mining Platform Recommendations App Connectors App Connectors Pattern Operators, e.g., Tag Normalization, LDA Clustering, Topic Identification Combine with Summarization, Voting other applications App Connectors Techniques… to create full Expertise Identification products App Connectors Hadoop MapReduce, Pig, MySQL, Django, Java Extracts data in the form of tuples from applications, e.g. Dashboard (user, tag, URL) … (user, activity, object) Core Advantage
  • 70.   Crowdsourcing  [collaborative  co-­‐creation]   –  Is  there  a  wisdom  of  the  crowd  in  Wikipedia?       –  How  does  conflict  drive  content  creation?     Collective  Intelligence  [folksonomy]   –  Are  social  tags  collectively  gathered  useful  for  organization  of  a  large   document  collection?     Collective  Averaging  [social  attention]     –  Does  voting  systems  identify  the  best  quality  and  most  interesting   information  for  that  community?     Participation  Architecture  [interaction]     –  Does  lowering  the  interaction  cost  barrier  increase  participation   productively?     Expertise  finding  [social  networking]     –  Does  getting  experts  through  social  network  gets  you  to  better  quality   information  sooner?   2009-05-01 Ed H. Chi ASC Overview 72
  • 71. 2009-05-01 Ed H. Chi ASC Overview 73
  • 72.   Research  Vision:  Understand  how  social  computing   systems  can  enhance  the  ability  of  a  group  of   people  to  remember,  think,  and  reason.     Living  Laboratory:  Create  applications  that  harness   collective  intelligence  to  improve  knowledge   capture,  transfer,  and  discovery.   http://asc-­‐parc.blogspot.com   http://www.edchi.net   echi@parc.com   2009-05-01 Ed H. Chi ASC Overview 74 Image from: http://www.flickr.com/photos/ourcommon/480538715/