• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Social-Computational Systems: Analysis and Design
 

Social-Computational Systems: Analysis and Design

on

  • 1,609 views

Tutorial given at the Web-Science Summer School 2011 in Galway, Irland

Tutorial given at the Web-Science Summer School 2011 in Galway, Irland

Statistics

Views

Total Views
1,609
Views on SlideShare
1,603
Embed Views
6

Actions

Likes
0
Downloads
30
Comments
0

4 Embeds 6

http://www.twylah.com 2
http://www.linkedin.com 2
http://twitter.com 1
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Social-Computational Systems: Analysis and Design Social-Computational Systems: Analysis and Design Presentation Transcript

    • Knowledge Management Institute Machines Social Computational Systems: Analysis and Design Markus Strohmaier M k St h i Assistant Professor, Graz University of Technology, Austria Visiting Scientist, (XEROX) PARC USA Scientist PARC, Wendy Hall’s talk: "Web Science is the theory and practice of social machines" Markus Strohmaier 2011 1
    • Knowledge Management Institute Tutorial Objectives Provide (some) answers to the following questions: 1. 1 What are social computational systems? How do they social-computational differ from other types of software systems? 2. Why do we need social-computational research? 3. What are examples of social-computational research? 4. How can I make a contribution to social-computational systems research? Markus Strohmaier 2011 2
    • Knowledge Management Institute The socio-economic impact of Social Computing socio economic on the EU Information Society (2009) • 41% of all EU Internet users, and 64% of those aged under 24 were engaged in Social Computing activities Markus Strohmaier 2011 http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=2819 3
    • Knowledge Management Institute Adoption and Use Markus Strohmaier 2011 http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=2819 4
    • Knowledge Management Institute User Engagement Understanding the complex dynamics and emergent properties of such systems requires social-computational research. http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=2819 Markus Strohmaier 2011 5
    • Knowledge Management Institute Reader to Leader Framework Preece, Preece Jennifer and Shneiderman Ben (2009) "The Reader-to-Leader Framework: Motivating Technology- Shneiderman, The Mediated Social Participation," AIS Transactions on Human-Computer Interaction (1) 1, pp. 13-32 Markus Strohmaier 2011 6
    • Knowledge Management Institute Agenda 1. Introduction t S i l C 1 I t d ti to Social-Computational S t t ti l Systems • Motivation, Examples and Definition 2. The Method of Social-Computational Systems Research • Approaches and Evaluation 3. Analysing Social-Computational Systems • E Examples of C l f Current R t Research (Wikipedia, S i l T h (Wiki di Social Tagging S t i Systems) ) 4. Designing Social-Computational Systems Social Computational • Examples of Current Implementations 5. Reflections and C 5 R fl ti d Conclusions l i Markus Strohmaier 2011 7
    • Knowledge Management Institute A Vision of a First Social-Computational System (1945) The Memex [Bush 1945]: A mechanized private library for individual use Mimics Mi i associative memory i ti where users can insert documents navigate documents retrieve documents B build trails through documents (i) (ii) Operated and maintained p individually But trails can be shared socially y e.g. (i) a user A can send trail to user B A C (ii) user B modifies and shares it with user C (iii) ( ) (iii) user C uses the trail for navigation g Cs C‘s interaction with documents is mediated by user A and B Markus Strohmaier 2011 [Bush 1945] V. Bush. As We May Think. Atlantic Monthly, 1945. 8
    • Knowledge Management Institute The World Wide Web (1990-2000) A user‘s interaction with the web is mediated by (a few) editors and publishers Markus Strohmaier 2011 9
    • Knowledge Management Institute Social Computational Systems Today (2010) Interaction between individuals and computational systems is mediated by the aggregate behavior of massive numbers (millions) of users. Markus Strohmaier 2011 [Strohmaier 2011] Emergent Properties of Social-Computational Systems, Habliitation thesis, Graz University of Technology, 2011 10
    • Knowledge Management Institute Social Computation p influences system properties (X) X=Findability X=Utility It is through the process of social computation i e computation, i.e. the combination of social behavior and algorithmic computation, that desired and undesired system properties and functions emerge. X=Navigability X Navigability X=Interestingness X I t ti Markus Strohmaier 2011 [Strohmaier 2011] Emergent Properties of Social-Computational Systems, Habliitation thesis, Graz University of Technology, 2011 11
    • Knowledge Management Institute Unintended Consequences: X = Legal Should the operators be blamed f how a subgroup of for f their visitors use their sites? Markus Strohmaier 2011 13
    • Knowledge Management Institute A Definition: Social-Computational Systems (SoCS) …refer to web-scale systems in which system properties and functions (“X”) are influenced by the behavior of users. Thus, certain system properties and functions are not engineered by a single person, but they are emergent, i.e. the result of aggregating information from a large group of usersusers. In this sense, certain system properties and functions of social- computational systems are b i l beyond the direct control of system d h di l f designers. New approaches for analyzing and designing social-computational systems are needed. Markus Strohmaier 2011 [Strohmaier 2011] Emergent Properties of Social-Computational Systems, Habliitation thesis, Graz University of Technology, 2011 14
    • Knowledge Management Institute Example: X = Connectivity (of the web graph) Questions: • What is X like? • What causes X? bow-tie architecture of the web [Broder et al 2000] Markus Strohmaier 2011 15
    • Knowledge Management Institute Example: X = Connectivity (of the web graph) Questions: • What is X like? • What causes X? • How can we bow-tie architecture Social mechanisms, such as improve X? of the web preferential attachment Preferential attachment: The sum Degree of open of all an vertex i problem vertices‘ e ces degrees Probability of a new vertex attaching to a vertex i with degree k [Broder et al. 2000] [Barabasi and Albert 1999] Markus Strohmaier 2011[Barabasi and Albert 1999] A.-L. Barabási, R. Albert, Emergence of Scaling in Random Networks, Vol. 286. no. 5439, pp. 509 - 512 , Science, 15 October 1999. 16[Broder et al. 2000] A. Broder, R. Kumar, F. Maghoul, P.Raghavan, S. Rajagopalan, R. Stata, A. Tomkins, and J. Wiener. Graph structure on the web. In 9th International WWW Conference, 2000.
    • Knowledge Management Institute Social Computational Systems: What type of questions are we asking? e.g. X = Connectivity of the web graph C ti it f th b h • Description and Classification: • Causality: • What is X like? • Does X cause Y? • What are its properties? • Does X prevent Y? • How can it be categorized? • What causes X? • How can we measure it? • What effect does X have on Y? • Descriptive Process: • Causality - Comparative: • How does X work? • Does X cause more Y than does Z? • What is the process by which X • Is X better at preventing Y than is Z? pp happens? • Does X cause more Y than does Z • How does X evolve? under one condition but not others? • Descriptive Comparative: • How does X differ from Y? • Design • Relationship: • What is an effective way to achieve X? • Are X and Y related? • How can we improve X? • Do occurences of X correlate with Objective: Put (some) control back in occurences of Y? the hand of engineers. Markus Strohmaier 2011[Easterbrook et al. 2007] S. Easterbrook, J. Singer, M.-A. Storey, D. Damian, "Selecting Empirical Methods for Software Engineering Research", Guide to Advanced Empirical Software 17Engineering, 2007
    • Knowledge Management Institute Shaping Social-Computational Systems Studiers Builders What is X like? Improve X? Prevent Y? typically beyond control emergent social-computational system properties and f functions social computation = social behavior + algorithmic computation Markus Strohmaier 2011 18
    • Knowledge Management Institute Agenda 1. Introduction t S i l C 1 I t d ti to Social-Computational S t t ti l Systems • Motivation, Examples and Definition 2. The Method of Social-Computational Systems Research • Approaches and Evaluation 3. Analysing Social-Computational Systems • E Examples of C l f Current R t Research (Wikipedia, S i l T h (Wiki di Social Tagging S t i Systems) ) 4. Designing Social-Computational Systems Social Computational • Examples of Current Implementations 5. Reflections and C 5 R fl ti d Conclusions l i Markus Strohmaier 2011 19
    • Knowledge Management Institute The Fundamental Problem of Social-Computational Systems Research • Lab studies are only capable of telling us behaviors in constrained situations • In-the-wild behavior remains unknown • Algorithms can not be validated theoretically • Actual usefulness is entwined with web scale usage web-scale • Proof-of-Concepts without deployment provide no insights • Usage data inherent aspect of such systems Consequence: Social-Computational Systems can not be stud ed the ab studied in t e lab (by de t o ) definition) For example, it would be impossible to answer questions about aggregate b h i behaviors of millions of Wikipedia or D li i f illi f Wiki di Delicious users Markus Strohmaier 2011 Based on: Ed Chi, Slides: ‘Living Laboratories’: Rethinking Ecological Designs and Experimentation in Human-Computer Interaction 20
    • Knowledge Management Institute So how can we study and engineer social- computational systems rigourously? In a Living Lab: by studying real platforms and services • Not to replace controlled lab studies or simulations • Expand our arsenal to cover new situations p Some principles: • Embedded in the real world • Ecologically valid situations • Embrace the complexity • Rely on big‐data‐science to extract patterns Not first to suggest this: S. Carter, J. Mankoff, S. Klemmer and T. Matthews. Exiting the cleanroom: On ecological validity and ubiquitous computing. HCI Journal, 2008 Markus Strohmaier 2011 Based on: Ed Chi, Slides: ‘Living Laboratories’: Rethinking Ecological Designs and Experimentation in Human-Computer Interaction 21
    • Knowledge Management Institute Social Computational Systems require „Living Lab“ studies Studier Builder Living Labs Markus Strohmaier 2011 Based on: Ed Chi, Slides: ‘Living Laboratories’: Rethinking Ecological Designs and Experimentation in Human-Computer Interaction 22
    • Knowledge Management Institute Agenda 1. Introduction t S i l C 1 I t d ti to Social-Computational S t t ti l Systems • Motivation, Examples and Definition 2. The Method of Social-Computational Systems Research • Approaches and Evaluation 3. Analysing Social-Computational Systems • E Examples of C l f Current R t Research (Wikipedia, S i l T h (Wiki di Social Tagging S t i Systems) ) 4. Designing Social-Computational Systems Social Computational • Examples of Current Implementations 5. Reflections and C 5 R fl ti d Conclusions l i Markus Strohmaier 2011 23
    • Knowledge Management Institute Social Computational Systems: What type of questions are we asking? • Description and Classification: • Causality: • What is X like? • Does X cause Y? • What are its properties? • Does X prevent Y? • How can it be categorized? • What causes X? • How can we measure it? • What effect does X have on Y? • Descriptive Process: • Causality - Comparative: • How does X work? • Does X cause more Y than does Z? • What is the process by which X • Is X better at preventing Y than is Z? pp happens? • Does X cause more Y than does Z • How does X evolve? under one condition but not others? • Descriptive Comparative: • How does X differ from Y? • Relationship: • Are X and Y related? • Do occurences of X correlate with occurences of Y? Markus Strohmaier 2011[Easterbrook et al. 2007] S. Easterbrook, J. Singer, M.-A. Storey, D. Damian, "Selecting Empirical Methods for Software Engineering Research", Guide to Advanced Empirical Software 24Engineering, 2007
    • Knowledge Management Institute A Living Lab What is X like? What does Coordination on Wikipedia look like?Kittur, A., Suh, B., Pendleton, B. A., Chi., E. (2007).Kittur A Suh B Pendleton B A Chi E (2007) He Says She Says: Conflict and Coordination in Wikipedia CHI 2007: Proceedings of the ACM Says, Wikipedia.Conference on Human-factors in Computing Systems. New York: ACM Press Markus Strohmaier 2011 25
    • Knowledge Management Institute A Living Lab: Does X cause Y? Who gets promoted on Wikipedia? Markus Strohmaier 2011 Governance in Social Media: A case study of the Wikipedia promotion process by J. Leskovec, D. Huttenlocher, J. Kleinberg. AAAI International Conference 26 on Weblogs and Social Media (ICWSM), 2010.
    • Knowledge Management Institute From A Social Perspective: Does X cause Y? What causes network creation and evolution? •Theories of self-interest •Theories of contagion •Theories of social and Theories •Theories of balance Theories resource exchange •Theories of homophily •Theories of mutual interest •Theories of proximity p y and collective action •Theories of co-evolutionSources:Contractor, N. S., Wasserman, S. & Faust, K. (2006). Testing multi-theoretical multilevel hypotheses about organizational networks: An analytic framework and empirical example. Academy of Management Review. example ReviewMonge, P. R. & Contractor, N. S. (2003). Theories of Communication Networks. New York: Oxford University Press. Markus Strohmaier 2011 27
    • Knowledge Management Institute From a Social Perspective: Social Theories of Behavior A A B B F + F + - C - E C E D D Theories of Exchange Theories of Structural Holes Theories of Balance A A A B F B F B + - - + F + C E C E C E D D G o v ern m en t Novice D I n d u s tr y Expert E t Theories of Collective Action Theories of Homophily Theories of Cognition Monge, P. R. & Contractor, N. S. (2003). Theories of Communication Networks. New York: Oxford University Press. Markus Strohmaier 2011 28
    • Knowledge Management Institute Does X cause Y? Empirical Analysis of Edge Creation Theories g Markus Strohmaier 2011 Signed Networks in Social Media by J. Leskovec, D. Huttenlocher, J. Kleinberg. ACM SIGCHI Conference on Human Factors in Computing 29 Systems (CHI), 2010.
    • Knowledge Management Institute From a Computational Perspective Markus Strohmaier 2011 30
    • Knowledge Management Institute From A Computational Perspective: Its all about “Relational Metadata” • Technologies that “capture” communities’ relational meta-data (Pingback and trackback in interblog networks, blogrolls, data provenance, retweets) • Technologies to “tag” communities’ relational metadata like – Tagging pictures (Flickr) – Social bookmarking (del.icio.us, LookupThis, BlinkList) – Social citations (CiteULike.org) – Social libraries (discogs.com, LibraryThing.com) – Social shopping (SwagRoll, Kaboodle, thethingsiwant.com) – Social networks (FOAF, XFN, MySpace, Facebook) • Technologies to “manifest” communities’ relational metadata (Tagclouds, Recommender systems, Rating/Reputation systems, ISI’s HistCite, Network Visualization systems) Markus Strohmaier 2011 Source: Noshir Contractor 31
    • Knowledge Management Institute An Examplary Living Lab: Social Tagging Systems Users label and categorize Resources resources with concepts (tags) Tags Users U is a tuple F:= (U, T, R, Y) where • th th the three di j i t fi it sets U T R correspond t disjoint, finite t U, T, d to user 1 – a set of persons or users u ∈ U – a set of tags t ∈ T and – a set of resources or objects r ∈ R tag 1 res. 1 • Y ⊆ U ×T ×R, called set of tag assignments … Markus Strohmaier 2011 32
    • Knowledge Management Institute Emergent Properties and Functions X1 = S Semantics ti X2 = Navigability Analysis X3 = Folksonomies X4 = Navigability (Web) X5 = Navigability (Wikipedia) X6 = Intentionality Design X7 = Abstracting X8 = Image recognition X9 = Persuasion Markus Strohmaier 2011 33
    • Knowledge Management Institute X1=Emergent Semantics g Question: Does Users‘ Motivation for Tagging Influence Emergent Semantics? E tS ti ? Markus Strohmaier 2011 34
    • Knowledge Management Institute Tagging Motivation Is t I tagging motivation amenable t quantitative analysis? i ti ti bl to tit ti l i ? Markus Strohmaier 2011 M. Strohmaier, C. Körner, R. Kern, Why do Users Tag? Detecting Users’ Motivation for Tagging in Social Tagging Systems. (under review) 35
    • Knowledge Management Institute Why Do Users Tag? One ( f O (of many) answers [Strohmaier et al 2010]: ) [St h i t l 2010] To categorize or to describe resources Categorizer (C) Describer (D) Goal later browsing later search Change of vocabulary costly cheap Size f Si of vocabulary b l limited li it d Open O Tags subjective objective Example tag clouds Semantic Assumption: Categorizers produce more precise emergent semantics than Describers. Markus Strohmaier 2011 M. Strohmaier, C. Koerner, R. Kern, Why do Users Tag? Detecting Users Motivation for Tagging in Social Tagging Systems, 4th International AAAI Conference on Weblogs and Social Media 36 (ICWSM2010), Washington, DC, USA, May 23-26, 2010.
    • Knowledge Management Institute Measures for Tagging Pragmatics vs. Tag Semantics Categorizer/Describer: C t i /D ib Semantics: S ti • Size of tag vocabulary • Co-occurrence count [Cattuto et al 2008] • Tags per resource • Cosine similarity (TagCont) • Tags per post • FolkRank [Hotho et al 2006] • Orphaned tags Markus Strohmaier 2011C. Körner, D. Benz, A. Hotho, M. Strohmaier, G. Stumme, Stop Thinking, Start Tagging: Tag Semantics Emerge From Collaborative Verbosity, 19th International World Wide Web Conference(WWW2010), Raleigh, NC, USA, April 26-30, ACM, 2010. 37
    • Knowledge Management Institute Are tags used to categorize or to describe resources? (C) (D) Does tagging motivation influence emerging system properties and functions? at |Ru| = 1000 Markus Strohmaier 2011 M. Strohmaier, C. Koerner, R. Kern, Why do Users Tag? Detecting Users Motivation for Tagging in Social Tagging Systems, 4th International AAAI Conference on Weblogs and Social Media 38 (ICWSM2010), Washington, DC, USA, May 23-26, 2010.
    • Knowledge Management InstituteDescribers Results Tagging motivation influences tag agreement Tag agreement: Example: agreement k = 30% 16/38=42,10%, agreement 6/38=15,79%, no agreement (C) 1http://jquery.com/ categorizer 38javascript 16ajax 6framework 6webdesign 5programming 4 (D) 5http://jquery.com/ describer 68javascript 66ajax 47library 44jquery 31web 25http://browsershots.org/ categorizer 43browser 4webdesign 4webdev 3tools 3development 3http://browsershots.org/ describer 103browser 52webdesign 46testing 38tools 37screenshots 37http://script.aculo.us/ categorizer 35javascript 16ajax 11programming 6web 5design 4http://script.aculo.us/ describer 76ajax 64javascript 60programming 30web2.0 20web 20http://www.colourlovers.com/ categorizer 42color 9design 8css 5inspiration 4webdesign 4http://www.colourlovers.com/http://www colourlovers com/ describer 67color 48design 45webdesign 33colour 24inspiration 23 Table: Most popular URLs and corresponding tags among 445 delicious users Results: Markus Strohmaier 2011 M. Strohmaier, C. Koerner, R. Kern, Why do Users Tag? Detecting Users Motivation for Tagging in Social Tagging Systems, 4th International AAAI Conference on Weblogs and Social Media 39 (ICWSM2010), Washington, DC, USA, May 23-26, 2010.
    • Knowledge Management Institute Semantic Validation of Folksonomies Semantic Networks Semantic Groundingg (Emergent) (Golden Standard) via e.g. hierarchical clustering WordNet: a lexical DB for English computers Map- Synset Hierarchy Programming ping programming distance d1 = 1 distance d2 = 2 Python Design g languages g g patterns abs. difference |d1 - d2| a Semantic simple p y for the q p proxy quality y grounding j java python of emergent semantics Markus Strohmaier 2011 Based on slides by A. Hotho 40
    • Knowledge Management Institute Results Does X cause Y? Tagging motivation influences emergent tag semantics Categorizers perform Describers perform worse than random better than random worse Random Random users users betterCategorizers DescribersC. Körner, D. Benz, A. Hotho, M. Strohmaier, G. Stumme, Stop Thinking, Start Tagging: Tag Semantics Emerge From Collaborative Verbosity, 19th International World Wide Web Conference Markus Strohmaier(WWW2010), Raleigh, NC, USA, April 26-30, ACM, 2010. 2011 41
    • Knowledge Management Institute Results Does X cause Y? Tagging motivation influences social classification accuracy Describers perform worse than Categorizers Supervised multi-class SVM Markus Strohmaier 2011 A. Zubiaga, C. Koerner, and M. Strohmaier. Tags vs. shelves: From social tagging to social classification. In 22nd ACM SIGWEB Conference on Hypertext and Hypermedia 42 (HT 2011), Eindhoven, Netherlands, ACM, 2011.
    • Knowledge Management Institute X2=Navigability g y Question: What Influences Navigability of Social Tagging Systems? T i S t ? Tag clouds as an instrument for g navigation Markus Strohmaier 2011 43
    • Knowledge Management Institute Navigability of Social Tagging Systems Question: Does (i) th size of t clouds and the i f tag l d d (ii) number of resources / tag influence the navigability (X1) of social tagging systems? established systems, many users New system, few users Markus Strohmaier 2011D. Helic, C. Trattner, M. Strohmaier and K. Andrews, On the Navigability of Social Tagging Systems, The 2nd IEEE International Conference on Social Computing (SocialCom 2010), 44Minneapolis, Minnesota, USA, 2010.
    • Knowledge Management Institute Tag Clouds are Supposed to be Efficient Tools for Navigating Tagging Systems The Navigability Assumption: • An implicit assumption among designers of social tagging systems that tag clouds are specifically useful to support navigation. • This has hardly been tested or critically reflected in the past past. Navigating tagging systems via tag clouds: 1) The system presents a tag cloud to the user. ) y p g 2) The user selects a tag from the tag cloud. 3) The system presents a list of resources tagged with the selected tag tag. 4) The user selects a resource from the list of resources. 5) The system transfers the user to the selected resource, and th process potentially starts anew. d the t ti ll t t Markus Strohmaier 2011D. Helic, C. Trattner, M. Strohmaier and K. Andrews, On the Navigability of Social Tagging Systems, The 2nd IEEE International Conference on Social Computing (SocialCom 2010), 45Minneapolis, Minnesota, USA, 2010.
    • Knowledge Management Institute Does X cause Y? How UI constraints effect Navigability Tag Cloud Size Pagination Limiting the tag cloud size n to practically feasible sizes (e.g. 5, 10, or more) does not influence navigability (this is not very surprising). BUT: Limiting the out-degree of high frequency tags k (e.g. through pagination with resources sorted in reverse-chronological order) leaves the network vulnerable to fragmentation. This destroys navigability of prevalent approaches to tag clouds. Markus Strohmaier 2011D. Helic, C. Trattner, M. Strohmaier and K. Andrews, On the Navigability of Social Tagging Systems, The 2nd IEEE International Conference on Social Computing (SocialCom 2010), 46Minneapolis, Minnesota, USA, 2010.
    • Knowledge Management Institute Findings 1. For 1 F certain specific, b t popular, t cloud scenarios, th t i ifi but l tag l d i the so-called Navigability Assumption does not hold. 2. While we can observe that tag-resource networks have g efficient navigational properties in theory, we found that popular user interface decisions significantly impair navigability. navigability These results make a theoretical and an empirical argument against existing approaches to tag cloud construction. How can we improve the navigability of social tagging systems? Markus Strohmaier 2011D. Helic, C. Trattner, M. Strohmaier and K. Andrews, On the Navigability of Social Tagging Systems, The 2nd IEEE International Conference on Social Computing (SocialCom 2010), 47Minneapolis, Minnesota, USA, 2010.
    • Knowledge Management Institute Recovering Navigability in Social Tagging Systems Instead of reverse-chronological ordering of resources, we apply a random ordering. More sophisticated approaches exist . Markus Strohmaier 2011D. Helic, C. Trattner, M. Strohmaier and K. Andrews, On the Navigability of Social Tagging Systems, The 2nd IEEE International Conference on Social Computing (SocialCom 2010), 48Minneapolis, Minnesota, USA, 2010.
    • Knowledge Management Institute X3=Emergent Folksonomies g Question: How can we Acquire and Evaluate Emergent Folksonomies in Social T F lk i i S i l Tagging S t i Systems? ? Markus Strohmaier 2011 49
    • Knowledge Management Institute Intuition: What causes X? Latent Hierachical Structures high centrality: more abstract [Strohmaier et al 2011] low centrality: more specific [Heyman and Garcia-Molina 2006] Markus Strohmaier 2011 M. Strohmaier, D. Helic, D. Benz. Evaluation of Folksonomy Induction Algorithms. Submitted to the ACM Transactions on Intelligent Systems and Technology, (2011). 50
    • Knowledge Management Institute Emergent semantics through hierarchical clustering Approaches: • k-means • Affinity propagation • Tag generality Applications: • user navigation i ti • ontology learning • disambiguation on a delicious dataset Evaluation: • Semantic grounding to Golden Standards (e.g. WordNet) W dN t)also cf. [Plangprasopchoket al. 2010] Semantic and Pragmatic Quality varying greatly. i tl Markus Strohmaier 2011Anon Plangprasopchok, Kristina Lerman, and Lise Getoor (2010), Integrating Structured Metadata via Relational Affinity Propagation, in Proceedings of AAAI Workshop on StatisticalRelational AI 51
    • Knowledge Management Institute What causes X? Tag Hierarchy [Dellschaft, Staab 2006] TP..Taxonomic TR..Taxonomic TF..Taxonomic TO…Taxonomic precision recall F measure overlap Different folksonomy algorithms Holds with other knowledge bases (Yago, Wikitaxonomy) and datasets (bibsonomy, citeulike, lastfm less so) On How to Perform a Gold Standard Based Evaluation of Ontology Learning (2006), by Klaas Dellschaft , Steffen Staab, In Proceedings of the 5th International Semantic Web Conference (ISWC’06) Markus Strohmaier 2011 M. Strohmaier, D. Helic, D. Benz. Evaluation of Folksonomy Induction Algorithms. Submitted to the ACM Transactions on Intelligent Systems and Technology, (2011). 52
    • Knowledge Management Institute Agenda 1. Introduction t S i l C 1 I t d ti to Social-Computational S t t ti l Systems • Motivation, Examples and Definition 2. The Method of Social-Computational Systems Research • Approaches and Evaluation 3. Analysing Social-Computational Systems • E Examples of C l f Current R t Research (Wikipedia, S i l T h (Wiki di Social Tagging S t i Systems) ) 4. Designing Social-Computational Systems Social Computational • Examples of Current Implementations 5. Reflections and C 5 R fl ti d Conclusions l i Markus Strohmaier 2011 53
    • Knowledge Management Institute Emergent Properties and Functions X1 = S Semantics ti X2 = Navigability Analysis X3 = Folksonomies X4 = Navigability (Web) X5 = Navigability (Wikipedia) X6 = Intentionality Design X7 = Abstracting X8 = Image recognition X9 = Persuasion Markus Strohmaier 2011 54
    • Knowledge Management Institute X4 = Navigability of the World Wide Web What th Wh t if the web h b hyperlinked it lf using user *b h i * li k d itself, i *behavior* itself as signals? www.apture.com Markus Strohmaier 2011 55
    • Knowledge Management Institute X5 = Navigability of Wikipedia / Text What th Wh t if the web h b hyperlinked it lf using user *b h i * li k d itself, i *behavior* itself as signals? How it works: David N. Milne, Ian H. Witten: Learning to link with wikipedia. CIKM 2008: 509-518 Markus Strohmaier 2011 56
    • Knowledge Management Institute X6 = User Intentions AOL S Search Q h Query L Log, ~ 20 i queries [P 20mio i [Pass 2006] M. Strohmaier and M. Kroell and C. Koerner. Intentional Query Suggestion: Making user goals more explicit during search. Proceedings of the Workshop on Web Search Click Data WSCD09, in conjunction with WSDM 2009, ACM, Barcelona, Spain, 2009. Markus Strohmaier 2011 57
    • Knowledge Management Institute X7 = Effective Shortening of Text How it works: Markus Strohmaier 2011 Michael S. Bernstein, Greg Little, Robert C. Miller, Björn Hartmann, Mark S. Ackerman, David R. Karger, David Crowell, Katrina Panovich: Soylent: a word 58 processor with a crowd inside. UIST 2010: 313-322
    • Knowledge Management Institute X8 = Image recognition Criticism: minimum wage, exploitation of workers, and more k d Markus Strohmaier 2011 60
    • Knowledge Management Institute X9 = Manipulation/Persuasion of Users According to a spokesperson for US Central Command, Persona Management Software is not being used by the US government inside the United States Markus Strohmaier 2011 61
    • Knowledge Management Institute Agenda 1. Introduction t S i l C 1 I t d ti to Social-Computational S t t ti l Systems • Motivation, Examples and Definition 2. The Method of Social-Computational Systems Research • Approaches and Evaluation 3. Analysing Social-Computational Systems • E Examples of C l f Current R t Research (Wikipedia, S i l T h (Wiki di Social Tagging S t i Systems) ) 4. Designing Social-Computational Systems Social Computational • Examples of Current Implementations 5. Reflections and C 5 R fl ti d Conclusions l i Markus Strohmaier 2011 62
    • Knowledge Management Institute Reflection: The objective of this tutorial was to … provide (some) answers to the following questions: 1. 1 What are social computational systems? How do they social-computational differ from other types of software systems? 2. Why do we need social-computational research? 3. What are examples of social-computational research? 4. How can I make a contribution to social-computational systems research? Markus Strohmaier 2011 63
    • Knowledge Management Institute Social Computation: Revisited purposeful! social computation: the combination of social behavior and algorithmic computation computation. Purposes include but are not limited to: • Linking webpages • Constructing taxonomies of tags • Shortening text • Influencing user behavior • Labeling data • Recommending items • … Markus Strohmaier 2011 64
    • Knowledge Management Institute Research on Social-Computational Systems Objective: To advance our understanding about the analysis and design of a new class of computational systems, which generate emergent properties and functions that arise out of the complex and dynamic interactions among people and computers 1 computers. Background: network theory, data mining, social and semantic systems, and other fields November 2010 Large research programs in the US: • NSF program line: „Social-Computational Systems“ Social Computational Systems Major and emerging conferences: • W3C World Wide Web Conference, ACM Hypertext, IEEE S i l C W ld Wid W b C f H t t Social Computing, and ti d others • ACM SIGWEB Working Group on Social Media • Workshop series to be started in 2012 Markus Strohmaier 2011 1based on: National Science Foundation http://www.nsf.gov/pubs/2010/nsf10600/nsf10600.htm 65
    • Knowledge Management Institute What kind of contribution can I make? If you are a „studier“ • Description and Classification: • Causality: • What is X like? • Does X cause Y? • What are its properties? • Does X prevent Y? • How can it be categorized? • What causes X? • How can we measure it? • What effect does X have on Y? • Descriptive Process: • Causality - Comparative: • How does X work? • Does X cause more Y than does Z? • What is the process by which X • Is X better at preventing Y than is Z? pp happens? • Does X cause more Y than does Z • How does X evolve? under one condition but not others? • Descriptive Comparative: • How does X differ from Y? • Relationship: • Are X and Y related? • Do occurences of X correlate with occurences of Y? Markus Strohmaier 2011[Easterbrook et al. 2007] S. Easterbrook, J. Singer, M.-A. Storey, D. Damian, "Selecting Empirical Methods for Software Engineering Research", Guide to Advanced Empirical Software 66Engineering, 2007
    • Knowledge Management Institute What kind of contribution can I make? If you are a „builder“ • Design es g • What is an effective way to achieve X? • How can we improve X? Markus Strohmaier 2011[Easterbrook et al. 2007] S. Easterbrook, J. Singer, M.-A. Storey, D. Damian, "Selecting Empirical Methods for Software Engineering Research", Guide to Advanced Empirical Software 67Engineering, 2007
    • Knowledge Management Institute How can I make a contribution? Studier Builder Living Labs Markus Strohmaier 2011 Source: Ed Chi, Slides: ‘Living Laboratories’: Rethinking Ecological Designs and Experimentation in Human-Computer Interaction 68
    • Knowledge Management Institute Social Contributions A b ild can change h builder h how people i t l interact. t They enable new social affordances, for example: • New forms of social interaction e.g., friendsourcing. e g friendsourcing • Designs that impact social interactions for example, increasing online participation. • Socially translucent systems interactive systems that allow users to rely on social intuitions. Markus Strohmaier 2011 Source: Ed Chi, Slides: ‘Living Laboratories’: Rethinking Ecological Designs and Experimentation in Human-Computer Interaction 70
    • Knowledge Management Institute Technical Contributions A b ild can come up with novel d i builder ith l designs, algorithms, and l ith d infrastructures. With mechanisms supporting social affordances, e.g. • highly original designs, applications, & visualizations g y g g , pp , to collect and manage social data, or powered by social data • New algorithms that coordinate crowd work Find-Fix-Verify Fi d Fi V if [5] • or derive signal from social data collaborative filtering • Platforms and infrastructures for developing social computing applications Markus Strohmaier 2011 Source: Ed Chi, Slides: ‘Living Laboratories’: Rethinking Ecological Designs and Experimentation in Human-Computer Interaction 71
    • Knowledge Management Institute A Call to Action: Challenges 1. Design and evaluation of social-computational algorithms Devising regulatory algorithms that self-adapt to maintain desirable system properties and functions over time. 2. Analysis of mechanisms to influence (social) behavior Systematic d S t ti development of social-computational and UI design patterns t l t f i l t ti l d d i tt to incentivize desirable user behavior 3. Expansion t other emergent properties 3 E i to th t ti Others emerging properties are likely to be beyond the direct influence of engineers as well (utility, profit, etc). We need means to describe, measure and influence them them. We‘ve barely scratched the surface yet! Markus Strohmaier 2011 72
    • Knowledge Management Institute Thank you! Markus Strohmaier Graz University of Technology, Austria http://kmi.tugraz.at/staff/markus Markus Strohmaier 2011 73