Methods and techniques-comm (3/6)


Published on

ISWC 2010: Tutoria: ten ways to make your semantic app addictive

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Virtual communities have (on average) a huge number of non active users
    Gnutella for instance in the 2000 only 25% pax  98% files and 66% users shared nothing … in the 2005 85% shared nothing
    50-percent of social, hobby, and work mailing lists had no traffic over 4 months

    What are the main characteristics that transform any appl. Into a KILLER one?
    What do you need to know before jumping into design?
  • Both continuous learning and motivations are very relevant to perform better.

    Also environment might affect performance and both motivation (crisis vs. development) and capacity (skills and knowledge)
  • In the real world intrinsic and extrinsic motivation are together and are difficult to analyze as in a theoretical model
  • 2 is ex ante
    3 is ex post
  • Nash and travelling salesman
    Google auction, Dutch auction, etc.
    Prices start at low reserve prices, and rise whenever there is “over-demand” for an ad-slot — i.e. a slot that is currently held by one bidder is desired by another. Such small price increases keep going on until there is no “over-demand”, at which point the auction closes.
  • How to transform individual preferences into social decision
    Google auctions:
    Goal: Efficiency, Revenue of auctioneer, Fairness, Incentive Compatibility: the bid information is given to us by advertisers.
    Complications: google charges TV ads NOT according the number of bidders BUT according the number of TV watchers. Long Term Payments Crowd control (a single ad. (or of the same company) cannot appear twice in the same commercial break, )
    Long vs. short ads. / auctions overlaps / lack of free disposal is not accepted … develop a bunch of filler ads.

  • From literature of psychology, social science, fiel experiment, experimental economics, knowledge management, HRM, organization behaviors, cognitive science….
  • Goal: participation
    Task: variety
    Social structure: hierarchical vs non
    Nature of good: public or private
    Required skills: specific …
  • One of the way to trasfrom into practice mechanism design
  • Methods and techniques-comm (3/6)

    1. 1. Methods and techniques to analyze and design incentivized semantic applications Roberta Cuel, University of Trento, IT and Markus Rohde, University of Siegen, DE ISWC 2010
    2. 2. 1/29/2015 2 Why is my app not as successful as Facebook… 1/29/2015 2 • Motivation and incentives – Reciprocity – Reputation – Competition – Altruism – Self-esteem – Fun – Money • This session is about the methods and techniques you can use – To study your scenario prior to application design – To evaluate your incentives strategy and adjust your application
    3. 3. Overview • Basic notions and definitions • Methods and techniques for analysis, design and evaluation • Case studies 1/29/2015 3
    4. 4. BASIC NOTIONS AND DEFINITIONS Basic notions and definitions Methods and techniques for analysis, design and evaluation Case studies 1/29/2015 4
    5. 5. Starting with motivation… • Basic tenets of organizational behavior – Performance : f(ability*motivation) 1/29/2015 5
    6. 6. Motivation and incentives • Intrinsic vs extrinsic motivations • Incentives are ‘rewards’ assigned to performer to make him/her commit – Can be totally uncorrelated to the nature of the task – Need to be compliant with the values and beliefs of the acting person 1/29/2015 61/29/2015 6
    7. 7. Example: DBPedia The DBpedia community collaborates using the following tools – Mailing lists, Facebook group – Blog, quality assurance and bug tracking – Development • “The framework is easily extended and we offer the possibility to do this yourself, increasing the likelihood of your desired data being included in the next DBpedia release.” – DBpedia consulting (money) 1/29/2015 7
    8. 8. 1/29/2015 8 Typology of motivations Motivations Internal (embedded in structure, e.g., task, tools) External (additional to structure, external re-inforcements) Intrinsic (predispositioned in person, e.g., drives, needs, desires ) Fun, joy, gaming, interest, satisfaction, self-actualization, self-re- inforcement Social appreciation, reputation, love, trust, social capital, community support Extrinsic (additional to personal predispositions, extern re- inforcements ) Usability, sociability, Design-for-fun, curiosity, community-building support Material/financial capital, money, rewards, prices, medals, credit points Structure Person Example: FLOSS software (Ghosh & Prakash, in Lerner & Tirole, 2005)
    9. 9. METHODS AND TECHNIQUES FOR ANALYSIS, DESIGN AND EVALUATION Basic notions and definitions Methods and techniques for analysis, design and evaluation Case studies 1/29/2015 9
    10. 10. Overview of methods and techniques • To analyze your scenario and design your application – Game theory, mechanism design – Participatory design and end-user development • To evaluate an incentives strategy in its context – Interviews and questionnaires and field experiments – Heuristic evaluation, guideline reviews – Cognitive and pluralistic walk-throughs – Participatory/User evaluation 1/29/2015 101/29/2015 10
    11. 11. Game theory • Game theory is a formal way to analyze interaction among a number of rational agents who behave strategically – The rational agents: players involved in the situation (best choice) – A number of players: more than one – Rationality/payoffs: what are the players’ preferences over the outcomes of the game – The interactions: one player’s behavior affects another – The rules: who moves when, what do they know, what can they do – The outcomes: what is the outcome of the game (for each move) 1/29/2015 11
    12. 12. Mechanism design • Mechanism design is about how to translate game theory in effective behavior – To design rules such that a desired set of outcomes happens – Alignment of interests between parties and production of maximum social welfare • Relevant variables – Structural and personal motivations – Goal and nature of good produced – Tasks and skills required – Social structure 1/29/2015 12
    13. 13. Analysis matrix • A multidimensional tool that allows for an analysis of a particular task with respect to the variables relevant for mechanism design 1/29/2015 13 Goal Tasks Social structure Nature of good produced Skill variety/level Communication level Variety of Hierarchy- neutral Private good Skilled ability Participation level Specificity of Hierarchical Public good Competence Clearness level Identification with Common resource Club good
    14. 14. 1/29/2015 14 The matrix in practice: semantic annotation • Goal of the annotation or ontology population exercise • Task, or more typically, an ordered collection of tasks into which the annotation exercise can be broken down • Social structure, a stylized and simplified set of social relationships among the subjects participating in the exercise • Nature of good, a stylized description, in game-theoretical terms, of the relationship between what good is produced and who consumes it • Required skills of the agents to complete the annotation task 1/29/2015 14
    15. 15. The matrix in practice: crowd-sourcing of ontology evolution • Goal: communication and participation level – Coordination (free-open) • Interests are already aligned (free riding) • Private and public benefit coincide • Task: variety vs specificity • Social structure: hierarchy neutral (large vs small groups) • Nature of good: public good non-rival (use does not limit use of others), non-exclusive (open access) • Skill variety/level: Skilled ability 1/29/2015 151/29/2015 15
    16. 16. Participatory Design (PD) • To develop software for the participation of end-users, we propose a participatory way of designing these software tools, integrating potential users by participatory design methods • Participatory Design – Improvement of the participation of workers in software development processes – The cooperation between software developers and end-users – Participate in IT development projects as experts of their own work 1/29/2015 16
    17. 17. 1/29/2015 17 Methods for PD • Participatory-design projects combine – Design-by-doing methods – PD workshops – Scenarios – Different forms of prototyping (mockups, rapid prototypes) – Work organization games – Ethnographic methods
    18. 18. 1/29/2015 18 End-User Development (EUD) • EUD is about designing highly flexible systems that enable users to participate during the use of the system by adapting and modifying the tools according to their needs/preferences • Goal: Empower end-users to develop and adapt systems themselves by designing these systems to be easy to – Understand – Learn – Use – Teach
    19. 19. 1/29/2015 19 How to design an incentivized application • Ideally: field  desk  lab  field • Analyze the domain and find yourselves in the matrixes – Find the relevant point of that situation (goal and tasks) – Focus on a small group of individuals (social structure) – Analyze their motivation (internal/external intrinsic/extrinsic) – Analyze the other relevant variables (nature of good being produced, kill variety/level) • Design a simplest possible model that can effectively support contributors • Test and get feedback • Fine-tune the experiment and add other elements
    20. 20. 1/29/2015 20 Methods • Domain observations (second-hand data) • Ethnography or qualitative face-to-face interviews • Questionnaires • Observations with selected individuals • Quantitative analysis (data collections) • Usability-design methods 1/29/2015 20
    21. 21. CASE STUDIES Basic notions and definitions Methods and techniques for analysis, design and evaluation Case studies 1/29/2015 21
    22. 22. Examples from our project • Telefónica Investigación y Desarrollo (TID - Spain): Knowledge Intranet Platform • Pepper’s Ghost (PGP-UK): Virtual Games/Worlds • Seekda! (Austria): Web Services Search Portal 1/29/2015 22
    23. 23. 1/29/2015 23 Field work: TID • Domain analysis – Site visit, semi-structured, qualitative interviews • Communication processes • Existing usage practices and problems • Existing tools/solutions • Semantic annotation solutions – Tape recording, transcription – Data analysis per ex-post categorization • Focus group discussion – Usability lab tests – Expert walkthroughs
    24. 24. 1/29/2015 24 Field work: TID (2) 1/29/2015 24 • Find TID in the matrix – Goal: communication and participation (contribute to the knowledge portal) – Social structure (various structures co-exist) • Strongly hierarchical organization (control is an issue) • working groups and community of experts – Nature of good: public good vs. private, club goods – Skill variety/level: Skilled ability (knowledge workers) – Motivations: fun, visibility, reputation, promotion, money
    25. 25. 1/29/2015 25 Field work: PGP • Domain analysis – Data analysis (Benchmark: Galaxyzoo, Moonzoo) – Usability lab tests and expert walkthroughs – Collect data on users such as logs files • Find PGP in the matrix – Goal: participation level – Task: specific – Social structure: hierarchy neutral – Nature of good: public good non-rival (use does not limit use of others), non-exclusive (open access) – Skill variety/level: non skilled ability
    26. 26. 1/29/2015 26 Field work: SEEKDA • Domain analysis – Interviews and workshops – Replicating Telefonica model • Find SEEKDA in the matrix: – Goal: communication and participation level – Task: variety vs. specificity – Social structure: hierarchy neutral (mainly) – Nature of good: public good non-rival (use does not limit use of others), exclusive, or club good – Skill variety/level: Skilled ability
    27. 27. Thank you Any question? 1/29/2015 27