Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Insemtives cluj meetup
1. Collaborative knowledge
creation is not a game
Elena Simperl
Talk at the Semantic Web meet-up, Cluj Napoca, Romania
8/26/2011 www.insemtives.eu 1
2. Insemtives in a nutshell
• Many aspects of semantic content authoring naturally rely on human
contribution
• Motivating users to contribute is essential for semantic technologies to
reach critical mass and ensure sustainable growth
• Insemtives works on
– Best practices and guidelines for incentives-compatible technology design
– Enabling technology to realize incentivized semantic applications
– Showcased in three case studies: enterprise knowledge management;
services marketplace; multimedia management within virtual worlds
www.insemtives.eu 2
3. Incentives and motivators
• Motivation is the driving • Incentives can be related
force that makes humans to both extrinsic and
achieve their goals intrinsic motivations
• Incentives are ‘rewards’ • Extrinsic motivation if
assigned by an external task is considered boring,
‘judge’ to a performer for dangerous, useless,
undertaking a specific socially undesirable,
task dislikable by the
– Common belief (among performer
economists): incentives • Intrinsic motivation is
can be translated into a
sum of money for all
driven by an interest or
practical purposes enjoyment in the task
itself
4. Extrinsic vs intrinsic motivations
• Successful volunteer crowdsourcing is difficult
to predict or replicate
– Highly context-specific
– Not applicable to arbitrary tasks
• Reward models often easier to study and
control*
– Different models: pay-per-time, pay-per-unit, winner-
takes-it-all…
– Not always easy to abstract from social aspects (free-
riding, social pressure…)
– May undermine intrinsic motivation
* in cases when performance can be reliably measured
6. Games with a purpose (GWAP)
• „ a human-based computation technique in
which a computational process performs its
function by outsourcing certain steps to
humans in an entertaining way”
*Wikipedia
7. Gamification
• “use of game play mechanics for non-game
applications […] in order to encourage people
to adopt the applications”*
*Wikipedia
Image from http://gapingvoid.com/2011/06/07/pixie-dust-the-mountain-of-mediocrity/
8. Gamification features*
• Accelerated feedback cycles.
– Annual performance appraisals vs immediate
feedback to maintain engagement.
• Clear goals and rules of play.
– Players feel empowered to achieve goals vs fuzzy,
complex system of rules in real-world.
• Compelling narrative.
– Gamification builds a narrative that engages players to
participate and achieve the goals of the activity.
*http://www.gartner.com/it/page.jsp?id=1629214
9. How to implement gamification*
• Cosmetic: adding game-like visual elements or
copy (usually visual design or copy driven)
• Accessory: wedging in easy-to-add-on game
elements, such as badges or adjacent products
(usually marketing driven)
• Integrated: more subtle, deeply integrated
elements like % complete (usually interaction
design driven)
• Basis: making the entire offering a game (usually
product driven)
* http://uxmag.com/design/a-gamification-framework-for-interaction-designers
10. What tasks can be gamified?*
• Decomposable into simpler tasks.
• Nested tasks.
• Performance is measurable.
• Obvious rewarding scheme.
• Skills can be arranged in a smooth learning
curve.
*http://www.lostgarden.com/2008/06/what-actitivies-that-can-be-turned-into.html
11. What is different about semantic
systems?
• Semantic Web tools
vs applications
– Intelligent (specialized)
Web sites (portals) with
improved (local) search
based on vocabularies
and ontologies
– X2X integration (often
combined with Web
services)
– Knowledge
representation,
communication and
exchange
12. What do you want your
users to do?
• Semantic applications
– Context of the actual application
– Need to involve users in knowledge acquisition and
engineering tasks?
• Incentives are related to organizational and social factors
• Seamless integration of new features
• Semantic tools
– Game mechanics
– Paid crowdsourcing (integrated)
• Using results of games with a purpose
13. Knowledge engineering tasks
• Granularity of ontology
engineering activities is too
broad; further splitting is
needed
• Crowdsource very specific
tasks that are (highly) divisible
– Labeling (in different
languages)
– Finding relationships
– Populating the ontology
– Aligning and interlinking
– Ontology-based annotation
– Validating the results of
automatic methods
– …
www.insemtives.eu 13
20. OntoGame API
• API that provides several methods that are shared by the
OntoGame games, such as
– Different agreement types (e.g. selection agreement)
– Input matching (e.g. , majority)
– Game modes (multi-player, single player)
– Player reliability evaluation
– Player matching (e.g., finding the optimal partner to play)
– Resource (i.e., data needed for games) management
– Creating semantic content
• http://insemtives.svn.sourceforge.net/viewvc/insemtives/gen
eric-gaming-toolkit
8/26/2011 www.insemtives.eu 20
21. Lessons learned
• Tasks which can be subject to games
– Definition of vocabulary
– Conceptualization
• Based on competency questions
• Identifying instances, classes, attributes, relationships
– Documentation
• Labeling and definitions
• Localization
– Evaluation and quality assurance
• Matching conceptualization to documentation
– Alignment
– Validating the results of automatic methods
• But, the approach is per design less applicable because
– Knowledge-intensive tasks that are not easily nestable
– Repetitive tasks players‘ retention?
www.insemtives.eu 21
22. Lessons learned (ii)
• Approach is feasible for mainstream domains, where a
knowledge corpus is available
• Knowledge corpus has to be large-enough to allow for
a rich game experience
– But you need a critical mass of players to validate the
results
• Advertisement is essential
• Game design vs useful content
– Reusing well-kwown game paradigms
– Reusing game outcomes and integration in existing
workflows and tools
• Cost-benefit analysis
