Successfully reported this slideshow.

Insemtives cluj meetup


Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Insemtives cluj meetup

  1. 1. Collaborative knowledge creation is not a game Elena Simperl Talk at the Semantic Web meet-up, Cluj Napoca, Romania8/26/2011 1
  2. 2. Insemtives in a nutshell• Many aspects of semantic content authoring naturally rely on human contribution• Motivating users to contribute is essential for semantic technologies to reach critical mass and ensure sustainable growth• Insemtives works on – Best practices and guidelines for incentives-compatible technology design – Enabling technology to realize incentivized semantic applications – Showcased in three case studies: enterprise knowledge management; services marketplace; multimedia management within virtual worlds 2
  3. 3. Incentives and motivators• Motivation is the driving • Incentives can be related force that makes humans to both extrinsic and achieve their goals intrinsic motivations• Incentives are ‘rewards’ • Extrinsic motivation if assigned by an external task is considered boring, ‘judge’ to a performer for dangerous, useless, undertaking a specific socially undesirable, task dislikable by the – Common belief (among performer economists): incentives • Intrinsic motivation is can be translated into a sum of money for all driven by an interest or practical purposes enjoyment in the task itself
  4. 4. Extrinsic vs intrinsic motivations• Successful volunteer crowdsourcing is difficult to predict or replicate – Highly context-specific – Not applicable to arbitrary tasks• Reward models often easier to study and control* – Different models: pay-per-time, pay-per-unit, winner- takes-it-all… – Not always easy to abstract from social aspects (free- riding, social pressure…) – May undermine intrinsic motivation * in cases when performance can be reliably measured
  5. 5. Examples 5
  6. 6. Games with a purpose (GWAP) • „ a human-based computation technique in which a computational process performs its function by outsourcing certain steps to humans in an entertaining way”*Wikipedia
  7. 7. Gamification • “use of game play mechanics for non-game applications […] in order to encourage people to adopt the applications”**WikipediaImage from
  8. 8. Gamification features*• Accelerated feedback cycles. – Annual performance appraisals vs immediate feedback to maintain engagement.• Clear goals and rules of play. – Players feel empowered to achieve goals vs fuzzy, complex system of rules in real-world.• Compelling narrative. – Gamification builds a narrative that engages players to participate and achieve the goals of the activity. *
  9. 9. How to implement gamification*• Cosmetic: adding game-like visual elements or copy (usually visual design or copy driven)• Accessory: wedging in easy-to-add-on game elements, such as badges or adjacent products (usually marketing driven)• Integrated: more subtle, deeply integrated elements like % complete (usually interaction design driven)• Basis: making the entire offering a game (usually product driven) *
  10. 10. What tasks can be gamified?*• Decomposable into simpler tasks.• Nested tasks.• Performance is measurable.• Obvious rewarding scheme.• Skills can be arranged in a smooth learning curve. *
  11. 11. What is different about semantic systems?• Semantic Web tools vs applications – Intelligent (specialized) Web sites (portals) with improved (local) search based on vocabularies and ontologies – X2X integration (often combined with Web services) – Knowledge representation, communication and exchange
  12. 12. What do you want your users to do?• Semantic applications – Context of the actual application – Need to involve users in knowledge acquisition and engineering tasks? • Incentives are related to organizational and social factors • Seamless integration of new features• Semantic tools – Game mechanics – Paid crowdsourcing (integrated)• Using results of games with a purpose
  13. 13. Knowledge engineering tasks• Granularity of ontology engineering activities is too broad; further splitting is needed• Crowdsource very specific tasks that are (highly) divisible – Labeling (in different languages) – Finding relationships – Populating the ontology – Aligning and interlinking – Ontology-based annotation – Validating the results of automatic methods – … 13
  14. 14. Example: ontology alignment 14
  15. 15. Example: relationship finding
  16. 16. Example: video annotation 16
  17. 17. Example: image annotation8/26/2011 17
  18. 18. Example: ontology population8/26/2011 18
  19. 19. Example: ontology evaluation 19
  20. 20. OntoGame API• API that provides several methods that are shared by the OntoGame games, such as – Different agreement types (e.g. selection agreement) – Input matching (e.g. , majority) – Game modes (multi-player, single player) – Player reliability evaluation – Player matching (e.g., finding the optimal partner to play) – Resource (i.e., data needed for games) management – Creating semantic content• eric-gaming-toolkit8/26/2011 20
  21. 21. Lessons learned• Tasks which can be subject to games – Definition of vocabulary – Conceptualization • Based on competency questions • Identifying instances, classes, attributes, relationships – Documentation • Labeling and definitions • Localization – Evaluation and quality assurance • Matching conceptualization to documentation – Alignment – Validating the results of automatic methods• But, the approach is per design less applicable because – Knowledge-intensive tasks that are not easily nestable – Repetitive tasks  players‘ retention? 21
  22. 22. Lessons learned (ii)• Approach is feasible for mainstream domains, where a knowledge corpus is available• Knowledge corpus has to be large-enough to allow for a rich game experience – But you need a critical mass of players to validate the results• Advertisement is essential• Game design vs useful content – Reusing well-kwown game paradigms – Reusing game outcomes and integration in existing workflows and tools• Cost-benefit analysis
  23. 23.