• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
An Introduction to Human Computation and Games With A Purpose - Part I

An Introduction to Human Computation and Games With A Purpose - Part I



Crowdsourcing and human computation are novel disciplines that enable the design of computation processes that include humans as actors for task execution. In such a context, Games With a Purpose are ...

Crowdsourcing and human computation are novel disciplines that enable the design of computation processes that include humans as actors for task execution. In such a context, Games With a Purpose are an effective mean to channel, in a constructive manner, the human brainpower required to perform tasks that computers are unable to per- form, through computer games. This tutorial introduces the core research questions in human computation, with a specific focus on the techniques required to manage structured and unstructured data. The second half of the tutorial delves into the field of game design for serious task, with an emphasis on games for human computation purposes. Our goal is to provide participants with a wide, yet complete overview of the research landscape; we aim at giving practitioners a solid understanding of the best practices in designing and running human computation tasks, while providing academics with solid references and, possibly, promising ideas for their future research activities.



Total Views
Views on SlideShare
Embed Views



1 Embed 9

https://twitter.com 9



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Of course this “cheap labor” thing is not something to be proud about…. We will see it later
  • [Blinder 2006, Horton 2013] ???And in fact, Blinder argues that about 20% of current American jobs could be sent down a wire. These include tasks like programming, accounting, marketing, and even machine operators. Recent evidence for crowd work in particular suggests that its volume will be roughly 454 billion dollars per year. That’s 91 billion hours per year, employing about 45 million fulltime workers. What might this mean? Think of current workers having the ability to become fulltime contractors, able to control their jobs and their career as they desire. On the other end of the spectrum, we get far more flexibility in time, like a stay-at-home dad who uses his skills while the baby is sleeping.
  • Alexis Claude de Clairaut found a model that could be solved numerically He recruited two friends for 9 months, divide the calculations of the orbit, mathematically tracing the cometHuman computers soon discovered the benefits of dividing the task and specializing their skills. Adam Smith [1723–1790] the division of labor produce “greatest improvement in productive powers of labor.” Human computers could reduce the cost of computation by either increasing the speed of calculation or by reducing errors in calculationTraditionally, hierarchical control (good in military) =>More visionary: mechanical controlCharles Babbage [1792-1871]'s Difference EngineThe Difference Engine was invented because Babbage was frustrated by limitations of (human) computers.A machine combining the additions and subtractions in order to interpolate a functionFirst World War required large numbers of human computers (map grids, surveying aids, navigation tables and artillery tables)Most computers were women and many were college educatedGreat Depression and Second World War: WPA (Works Progress Administration)  Mathematical Tables Project Requirement: use labor-intensive methods in order to employ the greatest number of workersMost of the computers knew little about arithmeticDeveloped ways of organizing the group and devised mathematical methods that were self-checkingWorkers gladly did the hard labor of research calculation in the hope that they might be part of the scientific community. In the end, they were rewarded by a new electronic machine that took the place and the name of those who were, once, the computers.
  • http://en.wikipedia.org/wiki/Frederick_Winslow_Taylor#Managers_and_workersIt is only through enforced standardization of methods, enforced adoption of the best implements and working conditions, and enforced cooperation that this faster work can be assured. And the duty of enforcing the adoption of standards and enforcing this cooperation rests with management alone.[9]Workers were supposed to be incapable of understanding what they were doing. According to Taylor this was true even for rather simple tasks.
  • Thisis a famous game, called “Where’s Wally”. Identify Wally withinthis image, givenit’sdescriptionprovided by its image
  • It does not take into account the fact that workers can make random guesses or make mistakes and still agree by chance [13]. This is especially problematic if the majority of the workers are novices (who systematically make the same kinds of errors) or spammers (who generate answers at random). Additionally, many of the factors that influence the outcome of the computation are not captured by the simple model. First, each worker may have different biases. F
  • Initialize by aggregating labels for each object (e.g., use majority vote)Estimate error rates for workers (using aggregate labels)Estimate aggregate labels (using error rates, weight worker votes according to quality)Keep labels for “gold data” unchangedGo to Step 2 and iterate until convergence
  • Not that simple: how do you build the data?
  • Mention active learning
  • Writing a news story• Programming software • Composing a symphony
  • the first challenge is to decompose the task:if you were planning a conference, you might split up finding a venue from reviewing papersif you were google, you might split up different parts of the web for different machines to process need to assemble the right teams of people:if you were conference chair you need to find respectable academics or coercable friends to be on the committee if you were google: assign different machines play different roles, like a master node coordinating a mapreduce process finally, you have to execute workflows, which may have multiple stages and decision processesfor a conference we have multistage review processesin computing, we have algorithms: for example, the output of one mapreduce process may get passed to another
  • badges can influence and steer user behavior on a site—leading both to increased participation and to changes in the mix of activities a user pursues on the site.

An Introduction to Human Computation and Games With A Purpose - Part I An Introduction to Human Computation and Games With A Purpose - Part I Presentation Transcript