INSEMTIVES talk at Semtech2010

301 views

Published on

Slides of the INSEMTIVES talk at Semtech 2010, 2010-06-22

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
301
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Instead of explaining what they are I would present some examples
  • INSEMTIVES talk at Semtech2010

    1. 1. Reputation, community, Web 2.0, and games: incentives for semantics – about getting people involved www.insemtives.eu 1 Katharina Siorpaes and Elena Simperl SemTech 2010
    2. 2. Executive summary • Many tasks related to semantic content authoring cannot be undertaken without human contribution. • User motivation is essential for semantic applications to achieve critical mass and ensure sustainable growth. • How to encourage user participation – Incentives – Technology design – Usability engineering – Games with a purpose www.insemtives.eu 2
    3. 3. About the speakers www.insemtives.eu 3 Katharina Siorpaes Elena Simperl
    4. 4. Agenda • Human-driven semantic content authoring • Incentives and participatory design • Example: Casual games • Example: Virtual worlds • Conclusion www.insemtives.eu 4
    5. 5. HUMAN-DRIVEN SEMANTIC CONTENT AUTHORING Human-driven semantic content authoring Incentives and participatory design Example: Casual games Example: Virtual worlds Conclusions www.insemtives.eu 5
    6. 6. Human vs computational intelligence • Ordered sequence of tasks into which the authoring exercise can be divided • Required skills and expertise www.insemtives.eu 6
    7. 7. Example: semantic annotation www.insemtives.eu 7
    8. 8. Example: ontology evaluation www.insemtives.eu 8
    9. 9. Example: ontology alignment www.insemtives.eu 9
    10. 10. MILLION DOLLAR QUESTION HOW CAN WE MOTIVATE PEOPLE TO DEDICATE THEIR VALUABLE TIME TO ANNOTATE THEIR/OUR DATA? www.insemtives.eu 10
    11. 11. INCENTIVES AND PARTICIPATORY DESIGN Human-driven semantic content authoring Incentives and participatory design Example: Casual games Example: Virtual worlds Conclusions www.insemtives.eu 11
    12. 12. www.insemtives.eu 12 Motivation vs incentives • Incentives are ‘rewards’ assigned by an external ‘judge’ to a performer for undertaking a specific task. • Common belief (among economists): incentives can be translated into a sum of money for all practical purposes. • Incentives can be related to both extrinsic and intrinsic motivations.
    13. 13. www.insemtives.eu 13 Extrinsic vs intrinsic motivation • Extrinsic motivation if task is considered – Boring, dangerous, useless, socially undesirable, dislikable by the performer. • Intrinsic motivation if – The performer likes what he/she is doing – The act is satisfying in itself (it can happen for many different reasons).
    14. 14. Web 2.0 is hot, Semantic Web is not? www.insemtives.eu 14
    15. 15. Web 2.0 motivators www.insemtives.eu 15
    16. 16. User-empowered Web • Comprehensive incentives studies are available (Kuznetsov, 2004; Marlow et al.,2006; Wikipedia, 2009). www.insemtives.eu 1616
    17. 17. Example: Wikipedians • Reciprocity: Altruistic contributors receive a benefit in return. • Community: “Wikipedians […] feel needed”, there is “a sense of common purpose and belonging “. • Reputation: Contributors “develop identities in order to be respected, trusted, and appreciated by peers”. • Autonomy: Contributors enjoy “the freedom of independent decision”. Kuznetsov, 2004 www.insemtives.eu 17
    18. 18. Factors influencing incentives design • Tasks or an ordered collection of tasks into which the authoring exercise can be divided. • Skills required to undertake the task. • Goal of the authoring exercise. • Social structure, i.e. social relationships among the subjects participating in the exercise. • Nature of the good.www.insemtives.eu 18
    19. 19. Example: image sharing on a social platform • Task: attaching concepts and relationships to images (iPhoto). • Skills: identifying objects in images and choosing the right description. • Goal: sharing and retrieval. • Social structure: Friends (in social network). • Nature of the good: annotations accessible to me and my friends. www.insemtives.eu 19
    20. 20. Example: UNSPSC and eCl@ss alignment • Task: ontology alignment • Skills: domain knowledge, modeling skills • Goal: interoperability • Social structure: enterprise, team • Nature of good: for benefit of enterprise www.insemtives.eu 20
    21. 21. Technology and application design • Involve users in the design process. – Design workshops, interviews etc. • Take usability serious. – Design for efficiency, effectiveness and user satisfaction. • Design for sociability. – Support sharing, contributing, collaborating, identity-building. www.insemtives.eu 21
    22. 22. Technology and application design (ii) • Respect trust/security and safety/privacy issues. – Support visibility and awareness. • Design for fun. • Follow an user experience design approach. • Support a user’s individual identity in a (and contribution to a) community with collaborative activities. www.insemtives.eu 22
    23. 23. Example: image annotation in OKEnterprise (i) • Involvement of users in design process – Interviews and user workshops • Usability – Interviews, usability evaluations • Design for sociability – User interaction, visibility of contributions, registered users, collaborative working • Trust and privacy – Trust in enterprise portal, rating – Privacy: users have control over their data www.insemtives.eu 23
    24. 24. Example: image annotation in OKEnterprise (ii) • Design for fun – Annotator of the week, rankings – Casual games • User-experience design approach – Evaluate and revise depending on user experience • Support user’s individual identity – Registration of users, individual user pages www.insemtives.eu 24
    25. 25. NOW: FUN AND COMPETITION: GAMES FOR SEMANTICS! Human-driven semantic content authoring Incentives and participatory design Example: Casual games Example: Virtual worlds Conclusions www.insemtives.eu 25
    26. 26. 1/28/2015 ESWC 2008 26 OntoPronto: Step 1
    27. 27. OntoPronto: Step 2
    28. 28. 1/28/2015 ESWC 2008 28 OntoTube
    29. 29. OntoTube
    30. 30. Results n=2905 rounds n=2234 12.9% 10.2% 76.9% Either skipped or no consensus found in the first task Challenges in which only the first task was completed consensually Challenges in which both tasks were completed consensually 97.6% 0.5% 1.9% Challenges in which at least the first task was completed consensually Wrong judgements of ontological nature Wrong abstractions
    31. 31. MASSACRE – MASive Semantic Annotation Creation Game www.insemtives.eu 31
    32. 32. Phratris – Phrase Analysis TETRIS www.insemtives.eu 32
    33. 33. playence’ Uhani (i) www.insemtives.eu 33 Check www.playence.com!
    34. 34. playence’ Uhani (ii) www.insemtives.eu 34
    35. 35. Casual games 1. Steep learning curve 2. Fast game play: little time effort required 3. Simple implementation (simple interface and graphics) 4. Low hardware efforts (usually browser or mobile app) 5. Low bandwidth requirements 6. Mass audience www.insemtives.eu 35
    36. 36. How to design your own game 1. Specify output 2. Identify input 3. Choose type of game and define game play 4. Based on previous decisions, define game play and adapt underlying game ontology 5. Adapt or define export algorithm 6. Evaluate output www.insemtives.eu 36
    37. 37. Guidelines 1. Timed response 2. Score keeping 3. Player skill level 4. High score lists 5. Randomness 5. Random player pairing 6. Player testing 7. Repetition 8. Taboo outputs www.insemtives.eu 37 Luis von Ahn. Games With A Purpose. IEEE Computer Magazine, June 2006. pp 96-98.
    38. 38. Challenges • Identifying suitable tasks in semantic content creation. • Designing games. • Designing a usable, attractive interface. • Identifying suitable knowledge corpora. • Preventing cheating. • Defusing typical pitfalls of conceptual modeling. • Distribution of labor. • Fostering user participation. • Deriving formal representations. • Scalability and performance. www.insemtives.eu 38
    39. 39. ANNOTATION IN VIRTUAL WORLDS Human-driven semantic content authoring Incentives and participatory design Example: Casual games Example: Virtual worlds Conclusions www.insemtives.eu 39
    40. 40. Tiny Planets • Kids TV property licensed in over 100 countries worldwide • Education • Diverse broadcaster base – from Al Jazeera to American Armed Forces network • Unusual age profile – kids 4 -11 • Brand extensions into web and virtual world www.insemtives.eu 40
    41. 41. Tiny Planets (ii) • Tiny Planets website – 40,000 unique visitors per month • 100,000 page impressions • Average stay 8 minutes • MTP virtual world – active 47,000 user accounts • Books, Fun, TV, Learning, Labs sites as well as virtual world • Real and virtual currencies • Cross-site account creation www.insemtives.eu 41
    42. 42. • Engage kids with annotation games as a means of expression • Combination of ‘rating’, ‘tagging’, and pre-scripted comments • Locate content within virtual world to inform friends, give clues in games, give an ‘emotional’ context to items by expressing opinions. Search for clues and comments left by friends • Reward annotations with virtual currency www.insemtives.eu 42 Virtual world
    43. 43. Real world • Real-world science experiment based on Galaxy Zoo format • Crater marking using data from NASA LRO • Use My Tiny Planets account for sign in and performance tracking • Reward annotations with achievements and virtual currency • And kudos; kids get to know they’re doing science for real, and its going to be used. www.insemtives.eu 43
    44. 44. Incentives • Earn Stars to spend on Virtual Goods, games or videos • Earn awards that can be seen by other players in your passport • Rise in Rank as you perform targeted activities • Join the Pro circuit so that your top scores can be seen by other players • Buy Keys, which unlock new areas and new activities in which to spend your Stars. www.insemtives.eu 45
    45. 45. CONCLUSIONS Human-driven semantic content authoring Incentives and participatory design Example: Casual games Example: Virtual worlds Conclusions www.insemtives.eu 50
    46. 46. Summary • Human contribution is needed for selected tasks in semantic content authoring. • Web 2.0 has impressively demonstrated how strong incentives can foster user participation and create successful applications. • Turning semantic content creation tasks into games is an especially intriguing idea because of the potentially large amounts of human-produced data that can be created. • Challenges are manifold: knowledge corpora, interesting game design, ensuring uptake, difficulty of tasks, etc. www.insemtives.eu 51
    47. 47. 1/28/2015 www.insemtives.eu 52 Realizing the Semantic Web by encouraging millions of end-users to create semantic content.

    ×