Successfully reported this slideshow.

CUbRIK Tutorial at ICWE 2013: part 2 - Introduction to Games with a Purpose

1,250 views

Published on

2013, 08 July
Part 2 of the tutorial illustrated at ICWE 2013, by Luca Galli (Politecnico di Milano)

Crowdsourcing and human computation are novel disciplines that enable the design of computation processes that include humans as actors for task execution. In such a context, Games With a Purpose are an effective mean to channel, in a constructive manner, the human brainpower required to perform tasks that computers are unable to perform, through computer games. This tutorial introduces the core research questions in human computation, with a specific focus on the techniques required to manage structured and unstructured data. The second half of the tutorial delves into the field of game design for serious task, with an emphasis on games for human computation purposes. Our goal is to provide participants with a wide, yet complete overview of the research landscape; we aim at giving practitioners a solid understanding of the best practices in designing and running human computation tasks, while providing academics with solid references and, possibly, promising ideas for their future research activities.

  • Be the first to comment

  • Be the first to like this

CUbRIK Tutorial at ICWE 2013: part 2 - Introduction to Games with a Purpose

  1. 1. AN INTRODUCTION TO HUMAN COMPUTATION & GAMES WITH A PURPOSE ALESSANDRO BOZZON DELFT UNIVERSITY OF TECHNOLOGY LUCA GALLI POLITECNICO DI MILANO
  2. 2. ABOUT THE TUTORIAL •  Crowdsourcing, Human Computation, and GWAPs are hot topics •  “Human Computation” => more than 3000 papers •  400 in 2013 •  “Crowd Sourcing” => more than 36000 papers •  4800 in 2013 •  “Games With A Purpose” => more than 1400 papers •  162 in 2013 •  This short tutorial is necessarily shallow, but •  Concrete Examples •  Lot of references and links •  An outlook on the future •  Slides and additional materials available •  http://hcgwap.blogspot.com ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 2
  3. 3. ABOUT THE SPEAKERS ALESSANDRO BOZZON Assistant Professor - TU Delft http://www.alessandrobozzon.com a.bozzon@tudelft.nl LUCA GALLI Ph.D. Student - Politecnico di Milano http://www.lucagalli.me lgalli@elet.polimi.it ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 3 •  RESEARCH BACKGROUND AND INTERESTS •  Web Data Management •  Crowdsourcing and Human Computation •  Game Design •  Web Engineering and Model Driven Development
  4. 4. AGENDA 4 ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
  5. 5. AGENDA •  PART 1 => CrowdSourcing and Human Computation •  Introduction •  Design of Human Computation Tasks •  Frameworks For/With Human Computation •  The Future of Human Computation •  PART 2 => Games With a Purpose •  Play vs Games •  Introduction to Game Design •  Games with a Purpose Design Guidelines ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 5
  6. 6. PART 2 AN INTRODUCTION TO GAMES WITH A PURPOSE 6 ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
  7. 7. 7 ICWE 2013 - An Introduction To Human Computation and Games With a Purpose Something really bad is going to happen…
  8. 8. 8 ICWE 2013 - An Introduction To Human Computation and Games With a Purpose Or not?
  9. 9. 9 ICWE 2013 - An Introduction To Human Computation and Games With a Purpose MAGIC CIRCLE “All play moves and has its being within a playground marked off beforehand materially or ideally, deliberately or as a matter of course… The arena, the card-table, the magic circle, the temple, the stage, the screen, the tennis court, the court of justice, etc., are all in form and function play-grounds, i.e., forbidden spots, isolated, hedged round, hallowed, within which special rules obtain. All are temporary worlds within the ordinary world, dedicated to the performance of an act apart.” Johan Huizinga, “Homo Ludens”, (1938-1950)
  10. 10. WHAT IS PLAY? ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 10 Autonomy:  play  is  a  voluntary  ac2vity.   Safety:  during  play  there  are  radically  reduced  serious                                                                    consequences  in  what  we  do   A<unement:  being  interested  in  the  same  things                                                                    temporarily  while  playing  together   Explora2on:  the  possibility  to  experiment  and  try  out    new                                                                  things   Mastery:  the  will  to  improve  one’s  own  skills      
  11. 11. WHICH ARE THE BENEFITS? ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 11 “Play  is  an  enjoyable  ac/vity  that  allows  us  to  train  and  test  our   skills  in  a  safe  environment.”     Fosters  crea2vity,  flexibility,  and  learning   An2dote  to  loneliness,  isola2on,  anxiety,  and  depression   Teaches  perseverance   Hints  on  how  to  cooperate  with  others   Increases  energy  and  prevents  burnout   Stuart Brown, “Play: How it Shapes the Brain, Opens the Imagination, and Invigorates the Soul”, 2009.
  12. 12. GAMES VS PLAY ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 12    Games  are  dis/nguished  from  play    –  Play  is  free-­‐form    –  Games  are  rule-­‐based   A  game  is  a  closed,  formal  system  that   ·∙  Engages  players  in  structured  conflict  and   ·∙  Resolves  its  uncertainty  in  an  unequal        outcome.   Fullerton, T.; Swain, C. & Hoffman, S. Game Design Workshop: A playcentric approach to creating innovative games, 2008
  13. 13. CLASSIC GAME DEFINITION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 13 1.  Rules    •  Games  are  rule-­‐based.   2.  Variable,  quan/fiable  outcome    •  Games  have  variable,  quan/fiable  outcomes.   3.  Valoriza/on  of  outcome    •  The  different  poten/al  outcomes  of  the  game                                are  assigned  different  values,  some  posi/ve                                and  some  nega/ve  
  14. 14. CLASSIC GAME DEFINITION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 14  4.  Player  effort            •  The  player  exerts  effort  in  order  to  influence  the  outcome     (games  are  challenging).   5.  Player  aVached  to  outcome            •  The  player  is  emo/onally  aVached  to  the  outcome  of  the      game  in  the  sense  that  a  player  will  be  winner  and     “happy”  in  case  of  a  posi/ve  outcome,  but  a  loser  and     “unhappy”  in  case  of  a  nega/ve  outcome.   6.  Nego/able  consequences              •  The  same  game  [set  of  rules]  can  be  played        with  or  without  real-­‐life  consequences   Juul,J.“Half-Real:VideoGamesBetweenReal RulesandFictionalWorlds”,2005
  15. 15. INCENTIVES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 15 Money, Passion, Ease of Participation, Altruism, Appeal to one’s own knowledge, Reputation, Fun Organisciak, Piotr. Why Bother?: Examining the Motivations of Users in Large-scale Crowd-powered Online Initiatives. Diss. University of Alberta, 2010.
  16. 16. INCENTIVES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 16  Intrinsic  mo2va2on  is  defined  as  doing  an  ac2vity  for      its  inherent  sa2sfac2ons  rather  than  for  some  separable      consequence.      Extrinsic  mo2va2on  is  defined  as  doing  an  ac2vity  to  a<ain      some  separable  outcome     Deci, Edward L., and Richard M. Ryan. Self‐Determination. ,1985.
  17. 17. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 17
  18. 18. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 18
  19. 19. GAMES WITH A PURPOSE ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 19    Games  with  a  Purpose  (GWAPs)  are  digital  games  that  generate   useful  data  as  a  by-­‐product  of  play.  [vA06,LvA09]    The  design  of  a  GWAP  requires  to  create  a  game  so  that  its   structure  encourages  computa2on,  correctness  of  the  output  and   players  reten2on.  
  20. 20. WHAT THEY ARE USED FOR ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 20 Krause,M.&Smeddinck,J.(2011b).HumanComputation Games:ASurvey.In:Proceedingsofthe19thEuropean SignalProcessingConference(EUSIPCO-2011). Intui2ve  Decisions   Aesthe2c  Judgment   Contextual  Reasoning   Embodiment  Issues  
  21. 21. ARE THEY WORKING? ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 21 ESP Game Purposes: Let players determine the contents of images by submitting meaningful labels they can agree on. Results: A total of 13,630 people played the game during the test phase, generating 1,271,451 labels for 293,760 different images. Luis von Ahn and Laura Dabbish. 2004. Labeling images with a computer game.
  22. 22. ARE THEY WORKING? ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 22 Foldit Purposes: The objective of the game is to fold the structure of selected proteins to the best of the player's ability, using various tools provided within the game to discover new structural configurations. Results: In 2011, players of Foldit helped to decipher the crystal structure of an AIDS-causing monkey virus. While the puzzle was available to play for a period of three weeks, players produced an accurate 3D model of the enzyme in just ten days. The problem of how to configure the structure of the enzyme had stumped scientists for 15 years. Khatib, F.; Dimaio, F.; Cooper, et al. (2011). "Crystal structure of a monomeric retroviral protease solved by protein folding game players"
  23. 23. GAMIFICATION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 23        “The  use  of  game  design  techniques  and  game                mechanics  to  enhance  non-­‐game  contexts”   S. Deterding, M. Sicart, L. Nacke, K. O’Hara, and D. Dixon, “Gamification. Using game-design elements in non-gaming contexts”        Actually  ques/onable,  we  will  see  why   Sebastian Deterding Miguel Sicart
  24. 24. WHAT IS USED FOR ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 24
  25. 25. IS IT WORKING? ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 25 Samsung Nation Purposes: Samsung Nation, a social loyalty program that lets users earn badges for activities as writing reviews and watching videos and compete for rewards. Results: 500% increase in customers product reviews 66% increase in site visitors 30% increase in comments reduced marketing costs reduced product support costs
  26. 26. IS IT WORKING? ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 26 MTV Italia MyChart Purposes: User generated video chart based on various “game dynamics” like avatars, points and leaderboards to drive users from Facebook to TV and to loyalize existing TV audience. Results: More than 500.000 votes in less then three months More than 150.000 videos viewed
  27. 27. COMMONALITIES AND DIFFERENCES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 27 One  or  more  objec2ves  or  tasks  that  users  have   to  accomplish.   A  gaming  experience,  defining  challenges  to   overcome  and  rewards  for  their  solu2on.   One  or  more  players,  the  users  of  the   applica2on,  who  are  profiled  and  monitored  in   their  ac2vi2es.   But...  
  28. 28. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 28 INTRODUCTION TO GAME DESIGN
  29. 29. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 29 Mancala, 7th century AD
  30. 30. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 30 Unreal Tournament 3, Epic Games, 2007
  31. 31. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 31 Turn  based  boardgame  vs  Real  /me  ac/on  shooter   Handmade  physical  board  vs  Personal  Computer   Public  domain  rules  vs  Copyrighted  
  32. 32. 1 - PLAYERS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 32 Number  of  players   Roles  of  the  players   Interac2ons  among  players  and  the  game:                    Single  Player/Mul2  Single                    Unilateral/Mul2lateral  Compe22on                                        Coopera2ve                    Team  compe22on  
  33. 33. 2 - OBJECTIVES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 33  Objec2ves:  specific  goals  for  the  player  to  accomplish    Capture:  capture  or  destroy  a  resource    Chase:  catch  an  opponent  or  elude  one    Race:  reach  a  goal  before  the  other  players    Alignment:  arrange  game  objects  in  a  spa/al  or  conceptual                                                                            configura/on                                RescueEscape:  bring  a  unit  to  safety                                  Construc2on:  build,  maintain,  manage  objects                                Explora2on:  explore  the  environment  where  the  game  take                                                                                place                                Solu2on:  solve  a  problem  or  puzzle  before  (or  more                                                                      accurately)  the  compe//on  or  certain  constraints                                Outwit:  gain  and  use  knowledge  to  defeat  other  players  
  34. 34. 2 – OBJECTIVES: EXAMPLES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 34 Alignment:  arrange  game  objects  in  a  spa2al  or   conceptual  configura2on   Tetris, Alexey Pajitnov, 1984 Puzzle Bobble, Taito Corporation, 1994
  35. 35. 2 – OBJECTIVES: EXAMPLES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 35 Solu2on:  solve  a  problem  or  puzzle  before  (or   more  accurately)  than  the  compe22on  or   following  certain  constraints   Connect Four, Milton Bradley, 1974 Professor Layton and the Miracle Mask, Level-5, 2011
  36. 36. 3 - PROCEDURES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 36 Procedures:  the  methods  of  play  and  the  ac/ons  that   players  can  take  to  achieve  the  game  objec/ves.  They   are  used  to  define  who  does  what,  where  and  how.     Typically  a  game  is  composed  of:     Star%ng  ac%on:  How  to  put  a  game  into  play.   Progression  of  ac%on:  Ongoing  procedures  a_er                                                                                      the  star/ng  ac/on.   Special  ac%ons:  Available  condi/onal  to  other                                                              elements  or  game  state.   Resolving  ac%ons:  Bring  gameplay  to  an  end.    
  37. 37. 3 – PROCEDURES: EXAMPLE ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 37 Star%ng  ac%on:  Choose  a  player     to  go  first.  Each  player  chooses  a  color:   red  or  black.   Progression  of  ac%on:  On  each     turn,  a  player  drops  one  colored       checker  down  any  of  the  slots  in       the  top  of  the  grid.   Resolving  ac%ons:  The  play  alternates   un/l  one  of  the  players  gets  four   checkers  of  one  color  in  a  row.  The  row   can  be  horizontal,  ver/cal,  or  diagonal.     Connect Four
  38. 38. 4 - RULES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 38 •   Define  Objects   •   Restrict  Ac/ons   •   Determine  Effects  (ECA  rules)       Chess:  A  player  cannot  move  her  king  into  check.     Poker:  A  straight  is  five  consecu8vely  ranked   cards;  a  straight  flush  is  five  consecu8vely  ranked   cards  of  the  same  suit.     WarCrad  II:  To  create  knight  units,  a  player  must  have   upgraded  to  a  keep  and  built  a  stable.  
  39. 39. 5 - RESOURCES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 39 Resources:  valuable  objects  that  can  help  the  players  to  achieve   their  goals  but  are  scarce  in  the  system.   Lives:  number  of  “trials”  that  can  be  aVempted  to  reach  a  goal   Units:  mul/ple  objects  that  has  to  be  managed  instead  of  lives   Health:  represent  the  status  of  loss  or  near  loss  of  lives  and  units   Points:  numerical  value  that  represents  a  measure  of  the  skill  and   progression  of  a  player   Ac2ons:  number  of  possible  dis/nct  choices  that  a  player  can   make  within  a  defined  /meframe   Power-­‐ups:  object  that  gives  a  boost  to  the  player   Items:  used  by  the  player  to  accomplish  an  objec/ve,  made   scarce  by  the  system   Turns:  the  number  of  game  phases  within  which  a  player  must   accomplish  the  objec/ves   Time:  restricts  player  ac/ons  or  phases  in  periods  of  /me.  
  40. 40. 5 – RESOURCES: EXAMPLE ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 40 Card Hunter, Blue Manchu Pty Ltd, TBR Time   Points   Ac/ons  
  41. 41. 6 - CONFLICTS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 41 Conflicts:  emerges  from  the  players  trying  to   accomplish  the  goals  of  the  game  within  its  rules  and   boundaries,  since  procedures  and  rules  tend  to  deter   players  from  accomplishing  goals  directly  or  make   players  work  against  each  other.   The  most  common  conflicts  are  generated  by:   Obstacles,  objects  or  rules  that  limit  the  freedom  of   the  players   Opponents,  since  they  are  usually  trying  to  achieve  an   objec/ve  faster  than  us   Meaningful  Choices,  players  have  to  make  choices   that  will  influence  the  outcome  of  the  game  
  42. 42. 7 - BOUNDARIES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 42 Boundaries are what separate the game from everything that is not the game and defines the physical or virtual scopes in which the game is performed. Example: Football would not be the same game if the boundaries of the football field were not defined.
  43. 43. 8 - OUTCOME ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 43 Outcome:  the  outcome  of  the  game  (if   present)  is  uncertain,  since  it  is  not   possible  to  predict  the  results  ahead.   If  an  outcome  can  be  obtained  it  has  to   be  quan/fiable  with  respect  to  the   defined  goals.     It  is  different  than  the  objec/ve  since  all   the  player  can  reach  the  same  objec/ve.    
  44. 44. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 44 Derived by modeling the guidelines and best practices found in: Fullerton, T.; Swain, C. & Hoffman, S. Game Design Workshop: A playcentric approach to creating innovative games, 2008 Crowford, C. The Art of Computer Game Design, 1984 GAME DEVELOPMENT PROCESS
  45. 45. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 45
  46. 46. GWAPS AS SERIOUS GAMES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 46 BenSawyer,PeterSmith:SeriousGames Taxonomy,2008
  47. 47. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 47 Katharina Siorpaes and Martin Hepp. 2008. OntoGame: weaving the semantic web by online games.
  48. 48. GAMES WITH A PURPOSE: CURRENT ISSUES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 48 ●  “Ad-­‐Hoc”  Design   ●  Engagement  and  reten/on  mechanisms.   ●  Mapping  from  task  to  game  mechanics   ●  Valida/on  techniques  and  strategies   E.G.  Real  user  comments  about  OntoGalaxy  
  49. 49. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 49 1)  PURPOSE   2)  TASK  DEFINITION   3)  GAME            MECHANICS            DESIGN   4)  INSTANCE            MATCHING   5)  OUTPUT              VALIDATION   6)  PERFORMANCE              EVALUATION   7)  ENGAGEMENT            SRATEGY     GWAPS DIMENSIONS
  50. 50. GWAP DEVELOPMENT PROCESS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 50
  51. 51. REQUIREMENTS SPECIFICATION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 51 uTask Task Description Input Objects Output Objects Execution Interface Operations Output Validation
  52. 52. INPUT OBJECTS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 52 Adapted from“Mathematical Modeling of social games, IrwinKing, SIAG2009” An  input  object            is  an  object  with  data  t23e          and  a  set  of   finite  a67ibutes  denoted  as     tex9,  images,  audio  seg=ents,  video  seg=ents,  other   unst7@ct@red  data,  st7@ct@red  data   where  the  data  t23e            is  the  media  t23e  presented  by       Each  a67ibute                        has  a  relationship                                            to  a  set  of   values                                                                                                                                                                                    .   Each                                          is  an  object  with  its  own  data  t23e  that  represent  the   metadata  of    
  53. 53. DEFINING OPERATIONS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 53 uTask Task Description Input Objects Output Objects Execution Interface Operations Output Validation
  54. 54. OPERATION TYPES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 54 They  fall  in  two  broad  categories:    Genera/ve  tasks    Decision  tasks     A  possible  (non-­‐exhaus/ve)  list  of  human  computa/on   tasks  may  include:     •   Object  Recogni/on/Iden/fica/on/Detec/on   •   Sor/ng  (Clustering/Ordering)   •   Natural  Language  Processing     •   State  Space  Explora/on   •   Content  Genera/on/Submission   •   User  preference/opinion  elicita/on  
  55. 55. ENSURE OUTPUT QUALITY ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 55 uTask Task Description Input Objects Output Objects Execution Interface Operations Output Validation
  56. 56. Twofold  purpose:  ensure  output  correctness  and   counter  player  collusion.   OUTPUT VALIDATION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 56 Player  Tes2ng:  compare  the  output  provided  by  the   user  with  known  correct  outputs     Repe22on:collect  more  than  one  solu/on  for  the   same  input  object,  then  aggregate  the  solu/ons   Taboo  Output:  limit  the  possible  solu/ons  that  can   be  submiVed   Random  Pairing:  pairing  players  at  random  
  57. 57. TASK EXECUTION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 57 uTask Task Description Input Objects Output Objects Execution Interface Operations Output Validation
  58. 58. TASK TO GAME MECHANICS MATCHING ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 58 Does  a  game  with  game   mechanics  similar  to  the  task   exists?     If  so,  integrate  the  task  within   the  game     If  not,  custom  game  mechanics   has  to  be  implemented.     Clear  and  Transparent  game   mechanics  
  59. 59. SOLUTION MECHANICS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 59 Pa<ern  Recogni2on         Wikipedia  stands  to  knowledge  as   BoardGameGeek    stands  to  games.   Benng/Wagering     Line-­‐Drawing     Tile-­‐Placement   Pa<ern  Building     Memory     Hand  Management     Ac2on  Programming     Auc2on/Bidding     Partnership   Possible,  meaningful  game  mechanics:   Take  Inspira/on!  
  60. 60. TILE PLACEMENT ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 60 Tile Placement games feature placing a piece to score points, based on adjacent pieces or pieces in the same group/cluster, taking into consideration non-spatial properties like color, "feature completion", cluster size etc.
  61. 61. PATTERN RECOGNITION APPLIED TO GWAPS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 61 Kawrykow A, Roumanis G, Kam A, Kwak D, Leung C, et al. (2012) Phylo: A Citizen Science Approach for Improving Multiple Sequence Alignment Purpose:  Used  to   arrange  the  sequences   of  D.N.A,  R.N.A  or   proteins  to  iden/fy   regions  of  similarity   Game  Mechanic:  align  the  sequence  contained  in  each  row  in   order  to  obtain  the  greatest  number  of  columns  with  matching   colors.    
  62. 62. LINE DRAWING ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 62 Games that make use of this mechanics involve drawing drawing of lines in one way or another.  
  63. 63. LINE DRAWING APPLIED TO GWAPS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 63 Purpose:  Segment  images   Game  Mechanic:  draw  the  shapes  of  objects  in  a  provided  image   in  order  to  make  the  other  players  guess  the  underlying  object.    
  64. 64. MEMORY ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 64 Games that use the Memory mechanic require players to recall previous game events or information in order to reach an objective.
  65. 65. MEMORY APPLIED TO GWAPS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 65 Purpose:  Find  similar  Images   Game  Mechanic:  Exploit  the  visual  memory  of   players  to  find  images  which  are  similar.  
  66. 66. PARTNERSHIP ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 66 Games  with  partnerships  offer  players  a  set  of  rules  for   alliances  and  teams.  Partners  are  o_en  able  to  win  as  a   team,  or  penal/es  are  enforced  for  not  respec/ng  alliances.  
  67. 67. PARTNERSHIP APPLIED TO GWAPS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 67 Purpose:  Collect  “common-­‐sense”  fact  for                                      specific  words.   Game  Mechanic:  provide  hints  to  the  partner   in  order  to  let  him  guess  the  secret  word    
  68. 68. GWAPS VALIDATION MECHANICS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 68 Bad  solu/ons  due  to:    incompetence    mischief  behavior    plain  mistakes     Find  or  adapt  game     mechanics  in  order     to  validate  the  results.    
  69. 69. GWAP FORMAL MODEL ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 69 Based on Mathematical Modeling of social games, Irwin King, SIAG2009 is  the  set  of  uTask  that  are  associated  with  the   problem,  with  the  uTask  defined  as  we  have   previously  seen.   is  the  answer  domain;  it  contains  the  set  of   possible  cer9ain  answers  fi  (based  on  the  input   D)  if  the  out3ut  is  contained  within  a  defined   domain,  or  empt2  if  the  answers  are  provided  by   the  players     is  a  f@nction  that  deter=ines  if  a  par9icular   out3ut  is  a  valid  solution  to  the  problem     is  the  GWAP  Domain,  that  maps  the  input  to   the  out3ut  of  a  GWAP  
  70. 70. MULTIPLAYER: INPUT AGREEMENT ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 70 Same  or  different  input     Based  on  each  other’s  hints,  player   must  agree  if  they  are  dealing  with  the   same  input   Given  a  correct  answer   set  for  the  problem   players  have  a  set  of   poten/al  outputs   which  cardinality  influence  the   probability  of  agreeing  upon  the   input   Results Submission: One Trial Suggested for: Subjective Information Same  or  different   input,  same  roles  
  71. 71. INPUT AGREEMENT: TAG A TUNE ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 71 Same  Input:   Audio  file   Two  different   shared  sets  of   descrip/ons,   one  for  each   player   Result:  same   input  or   different  input   Results   Submission:     One  Trial  
  72. 72. MULTIPLAYER: OUTPUT AGREEMENT ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 72 Same  Input     Agreeing  on  common  output  without   communica/on   Same  Input,     Same  Roles.     Given  a  correct   answer  set  for  the   problem,   players  have  a  set  of   poten/al  outputs   the  probability  to  obtain     an  agreement  depends   on  the  shared  output   Results  Submission:   Mul/ple  Trials    Suggested  for:     Objec/ve  Informa/on  
  73. 73. OUTPUT AGREEMENT: ESP GAME ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 73 Same  Input:   Image   List  of  possible   guesses,  hidden   to  the  other   players   Result: Meaningful tag for the provided image Results   Submission:     Mul/ple   Trials  
  74. 74. MULTIPLAYER: INVERSION PROBLEM ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 74 One  players  knows  the  whole  problem   and  gives  hints  to  the  other  players.   The  others  try  to  unveil  the  secrets   based  on  the  hints.   Input  to  just  one  player   the  output  of  one   player  is  the  input   for  the  others   The  correct  answer  set  for   the  problem  is  unknown   The  probability  of  guessing   depends  on  the  cardinality   of  the  hints   Result:   User  generated  content,   solu/ons  for  complex  data   structures   Results  Submission:   Mul/ple  Trials  
  75. 75. INVERSION PROBLEM: IMAGE LABELING GAME ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 75 Describer: Image Seekers: Textual description given by the describer Results Submission: Multiple trials Results: Meaningful descriptions for the image
  76. 76. INSUFFICIENT PLAYERS: PRERECORDED GAMES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 76 Fake players based on the logs of previous matches Pros: The gwap can be played anytime Cons: The actions in the game has to be modeled and stored Inversion problem games can be difficult to simulate Requires a bootstrapping phase to acquire the initial traces
  77. 77. SINGLE PLAYER: ALGORITHMIC EVALUATION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 77 Automa/c  algorithm  able  to  par2ally  assess  the   quality  of  the  provided  results.     For  each  instance  of  the  game  the  input  is   provided  to  just  one  player   The  correct  answer  set  for  the  problem  is  defined   The  probability  to  obtain  meaningful   results  is  applica/on  specific,based   on  the  algorithm  that  is  used  to   make  the  evalua/on  
  78. 78. ALGORITHMIC EVALUATION: TYPE ATTACK ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 78 Jovian,  L.T.;  Amprimo,  O.,  "OCR  Correc8on  via  Human   Computa8onal  Game,"  System  Sciences  (HICSS),  2011  44th   Hawaii  Interna8onal  Conference  on  ,  vol.,  no.,  pp.1,10,  4-­‐7   Jan.  2011   Improving  the  results   of  OCR  in  digitalizing   ar/cles  old   newspapers.   Human  players  transcribe  snippet  of  text  and  their   performance  is  measured  by  comparing  the  number  of   similar  words  in  the  player’s  output  and  the  text  from   the  library’s  OCR  transcrip/on.    
  79. 79. HYBRID ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 79 Combines  the  previous  approaches   Input  data  based  on  previous  results  or  user  generated  content   Inputs  to  one  or  more  players  with   same  or  different  roles     Answer  set  depending  on  the  nature  of   the  problem  or  provided  by  a  player   The  probability  of  obtaining  a   solu/on  depends  on  the   combina/on  of  the  strategies  
  80. 80. HYBRID: SKETCHNESS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 80 Input:     One  image  to  a  player   Hints  to  the  others   Segments  traced  by   one  player,  possible   tags  submiVed  by   the  others   Result:   Meaningful  tag   +  segmenta/on   Results  Submission:     Mul/ple  Trials  
  81. 81. INSTANCE MATCHING ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 81 How to assign tasks (instances) to players? • Random  Strategy:  assign  a  random  task    Pros:  Easy  to  implement    Cons:  Does  not  take  into  considera/on  player  skills     • Past  History:  assign  a  task  based  on  past  performances    Pros:  Can  improve  the  quality    Cons:  Keeping  track  of  gaming  history                          Performance  measures  must  be  defined     • Selec2ve  Assignment:  assign  a  specific  task  based  on                                                                                          skills  and  past  performances    Pros:  Can  improve  the  quality    Cons:  Same  as  Past  History                          Needs  profiling  of  the  player  to  assess  skills  
  82. 82. PERFORMANCE EVALUTION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 82 Luis von Ahn and Laura Dabbish. 2008. Designing games with a purpose. Commun. ACM 51, 8 Throughput = average number of problem instances solved per human hour ALP = average (across all people who play the game) overall amount of time the game will be played by an individual player Expected contribution = throughput multiplied by ALP.
  83. 83. ENGAGEMENT STRATEGY ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 83 Core activity must be already engaging for the user. Gamification involves defining clear and definitive objectives that last even past the gaming experience, and are either set by, or negotiated between the user and the game along with a progressive path of short and intermediate goals leading to it. Onboarding Retention Virality Social Dimensions Additional Mechanics
  84. 84. POINTS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 84 Points  or  Player  Scores  are  a  numerical  value  that  represents  a   measure  of  the  skill  of  a  player.     •   Immediate  and  las/ng  feedback   •   External  display  of  progression   •   May  determine  the  win  state   • Connec/on  between  progress  in  the  game  and  rewards         Galli, L., Fraternali, P. “Achievement Systems Explained“ SGSC2012, Singapore Werbach, K. & Hunter, D. For the Win: How Game Thinking Can Revolutionize Your Business Wharton Digital Press, 2012
  85. 85. LEADERBOARDS ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 85 A  Leaderboard  is  an  ordered  list  of  players  based  on  the  scores   they  have  obtained  in  a  specific  game  or  system.     •   Relates  the  performance  of  a  player  to  the  others   •   Fosters  compe//on  and  par/cipa/on   •   Risky:  May  be  demo/va/ng.     Galli, L., Fraternali, P. “Achievement Systems Explained“ SGSC2012, Singapore Werbach, K. & Hunter, D. For the Win: How Game Thinking Can Revolutionize Your Business Wharton Digital Press, 2012
  86. 86. ACHIEVEMENTS AND BADGES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 86 An  Achievement  is  a  set  of  tasks,  defined  by  a  designer,  for  the   player  to  fulfill  so  to  achieve  a  milestone  and  track  the   progress  in  a  system.       A  Badge  is  an  ar/fact  associated  to  the  comple/on  of  an   achievement  and  given  to  a  player  a_er  its  comple/on,  or,  in   gaming  terms,  a_er  “unlocking  the  achievement”.         •   Define  goals   •   Onboarding  tool   •   Visual  markers  for  reputa/on,     •   Provide  las/ng  rewards   Galli, L., Fraternali, P. “Achievement Systems Explained“ SGSC2012, Singapore
  87. 87. THIS IS JUST A GLIMPSE OF WHAT GAMIFICATION IS... ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 87
  88. 88. EXAMPLE: ZAMZEE GAMIFICATION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 88 Personalization Rewards Challenges Social Status
  89. 89. CASE STUDY: FASHION TREND MINING ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 89 Problem statement: segment fashion images for mining trends based on visual features of garments (e.g. color and texture) Use case: identifying trends in collections of images of people and garments Applications: retrieving similar garments, inspect clothing trends in image collections, analyzing trends change in the years Color descriptors Texture descriptors coarse (sub-)image similarity
  90. 90. THE FASHION TREND MINING PIPELINE ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 90 Male, 24 Female, 22Female?, ??
  91. 91. REQUIREMENTS SPECIFICATION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 91 Task Description: Recognize if a particular garment is present within a picture or define a new one and outline its contours. Input Objects: A fashion image, an optional tag defining the garment to identify. Aggregation Strategy: assign a value of 0 to each pixel outside the contour and 1 to each pixel contained within the contour, sum all the contribution and apply a threshold based on the number of players. Output Data: For each submitted task the game has to provide the contour of the garment within the image (Polyline) and a tag defining the garment that has been segmented
  92. 92. GWAP DESIGN ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 92 Solution Mechanics: Pattern Recognition Established genre: Draw and Guess Inversion Problem Mechanic PLAYERS:Number >=2 Multilateral Competition Two different roles: Sketcher: has to draw the contours of the stated garment Guesser: has to guess the garment drawn by the sketcher
  93. 93. PLAYER ROLE: SKETCHER ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 93 ●  The only player to see the low confidence image ●  “May” be asked to provide a tag for the image ●  Is asked to draw the contour of the object for which the tag is provided within the allotted time ●  Goal of the Sketcher is to let the other players guess the tag within a time slot without providing any other hints than the contour
  94. 94. PLAYER ROLE: GUESSER ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 94 ●  Any other player in the game ●  His/Her goal is to guess the object for which the Sketcher has provided the contour ●  Not allowed to draw on the whiteboard, just to type in the chat box the probable answer as fast as possible ●  Scoring: ●  Sketcher: 10 pts + 1 for each guesser ●  Guesser: 10 pts to the first, then decreasing down to five
  95. 95. FRAMEWORKS: WEB GAMES ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 95
  96. 96. FRAMEWORKS: GAMIFICATION ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 96 Google Play Game Services Mozilla Open Badges User Infuser
  97. 97. THE CUBRIK PROJECT ICWE 2013 - An Introduction To Human Computation and Games With a Purpose 97 ●  CUbRIK is a research project financed by the European Union ●  Goals: ●  Advance the architecture of multimedia search ●  Exploit the human contribution in multimedia search ●  Use open-source components provided by the community ●  Start up a search business ecosystem ●  http://www.cubrikproject.eu/

×