Ten ways to make your semantic
Elena Simperl, University of
• Many aspects of semantic content authoring naturally rely
on human contribution.
• Motivating users to contribute is essential for semantic
technologies to reach critical mass and ensure sustainable
• This tutorial is about
– Methods and techniques to study incentives and motivators
applicable to semantic content authoring scenarios.
– How to implement the results of such studies through
technology design, usability engineering, and game mechanics.
• Typology of semantic content authoring tasks and
the ways they could motivate users to contribute.
• Methodology for analyzing and designing
incentivized semantic applications.
• Pilots, showcases and technology.
Incentives and motivation
• Incentives are ‘rewards’ assigned by an external ‘judge’
to a performer for undertaking a specific task.
• Common belief (among economists): incentives can be
translated into a sum of money for all practical purposes.
• Incentives can be related to both extrinsic and intrinsic
– Extrinsic motivation if task is considered
• Boring, dangerous, useless, socially undesirable, dislikable by
– Intrinsic motivation if
• The performer likes what he/she is doing
• The act is satisfying in itself (for various reasons).
Harnessing human intelligence
Facebook reports 4,000,000,000 minutes (> 7500
person years) are spent on the site every day
Recaptcha users solve 60 million CAPTCHAs
a day, which accounts for around 160,000
human hours (19 person years)
More than 18 000 titles
preserved in 10 years
Harnessing human intelligence
Which tasks can be crowdsourced?
can the task be divided
into smaller chunks (see
casual games, Amazon’s
Mechanical Turk, open
• Skills and expertise:
does the task address a
broad audience (see
– Additive: pulling a rope
(group performs better than
individuals, but each
individual pulls less hard)
– Conjunctive: running in a
pack (performance is that of
the weakest member, group
size reduces group
– Disjunctive: answering a quiz
(group size increases group
performance in term of the
time needed to answer)
Example: video annotation
• Task selection, work breakdown and distribution
• Domain selection and creation of knowledge
• Deriving formal representations from user inputs
• Technology design
• Intrinsic motivations
Factors affecting participation
• Prize is higher
• Participants are more intrinsically motivated ,
have more free time, are non-experts in the field ,
and are not participating due to career concerns,
social motivations, or to beat others.
(Lakhani et al, 2007)
Outline of the tutorial
09:00 – 09:30 Human contributions in semantic content authoring
09:30 – 10:30 Methods and techniques to analyze and design incentivized semantic
10:30 – 11:00 Coffee break
11:00 – 12:30 Guidelines for incentivized technology design
12:30 - 13:30 Lunch break
13:30 – 14:30 Casual games for semantic content creation
14:30 – 15:00 Hands-on (Part I)
15:00 – 15:30 Coffee break
15:30 – 17:00 Hands-on (Part II)
17:00 – 17:30 Wrap-up and closing
1/29/2015 www.insemtives.eu 17
Realizing the Semantic Web by
encouraging millions of end-users
to create semantic content.