Introduction to Games with a Purpose design and Playtesting


Published on

Presentation held at 6th Qualinet General Assembly to introduce the current status of research related to Games with a Purpose, with notes regarding the QoE in Gwaps

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Norbert Rosing German Photographer
  • the polar bear returned every night that week to play with the huskys
  • “Adult play”
  • Mancala is not one game, but a family of pit-and-pebble games. Play involves scooping up pebbles from a pit and sowing the pebbles, one at a time, into the other pits. These games were probably created in Africa hundreds (if not thousands) of years ago.A "standard" Mancala board is composed of six pits on each side of the board, and a larger scoring pit on each side. Two players sit across from each other over the board. The large scoring pit to each player's right is "her" scoring pit.
  • One is turn based boardgame, the other is a real time action shooterOne requires a personal computer the other requires a handmade board and some stonesOne is copyrighted and the other has public domain rulesBut...Spettro e caratteristiche differenti, ma di fondo hanno parecchi elementi in comune, quelli elencati dopo, che permettono di rappresentare un gioco.
  • Number of playrsRoles of the playersInteractions among players
  • Objectives: specific goals for the player to accomplish
  • Procedures: actions or methods of play allowed by the rules, they guide player behaviors and interaction with the game.E.G. Walk, run, swim, shot, pick up power ups.Who does what, where and how. Usually they are defined as:Starting actionProgression of ActionSpecial actionResolving action
  • Rules: describe what the objects in the game consists of and clarifies what happens in various situations that might arise, limit players behavior and proscribe reactive events. The rules may be respected by an implicit agreement of the player or by the underlying code of the game in case of digital games.
  • Outcome: the outcome of the game (if present) is uncertain, since it is not possible to predict the results ahead.If an outcome can be obtained it has to be quantifiable with respect to the defined goals.It is different than the objective since all the player can reach the same objective
  • Games with a purpose exploit gamers’ time to perform useful tasks where human judgment is necessaryCombinatorial optimization tasks judging the quality of motion, a sound, or an imageutilizing human interaction with the physical environment to solve real world problems
  • Il messaggio è che, nonostante sia solo un gioco, i giocatori sono stati in grado di organizzarsi in maniera complessa e coordinata in gruppi numerosi per poter raggiungere l'obiettivo di coprire una zona grande come l'australia.
  • Before even thinking about a gwap, it is necessary to define a task. Without a task to be solved, a Gwap would just be a normal game, we want to solve a problem and we will show how it can be done. The requirement specification phase involves the collection of the information necessary for the definition of a task, a unit of work performed by human worker in the process of solving computational problems that cannot be resolved by AI. Cropping the silhouette of the models in a picture, recognizing and identifying the people contained in a set of images, collecting labeled data for training are examples of what a task involving user of a GWAP may look like.So far the GWAPS that have been proposed have never had a complex flow control (fresh results or improvements over the previous results of other players) and the tasks to be performed are considered already as monolithic microtasks.
  • “Typical” playtest Watch people play the game Observe their gameplay/behavior Simulate at‐home experiencePresence of observers can bias results– Salient event can slant interpretation– Behavior requires interpretation
  • People describe their actions as they play Unprompted and uncorrectedInterfere with gameplay/create an artificialexperience/distracting– Inaccurate and biased
  • Record of gameplay behaviors Deaths, level times, friendly fire, … Objective measurements Aggregate perspective Quantify behavior Opportunity for analyses T‐tests Regressions
  • Objective notions of player behavior+ See global trends+ Readily enables comparisons, baselineestablishment, and metric creation+ Track changes over time– Averages hide extreme examples– Miss nuance (lacking context)– Requires rigor– Can see ‘illusory’ patterns
  • More objective measurements of player state+ Quantifiable emotional response+ Analysis/comparison metrics– Expensive– Intrusive– Artificial experience– Requires experimental control
  • Introduction to Games with a Purpose design and Playtesting

    2. 2. ABOUT MYSELF LUCA GALLI Ph.D. Student - Politecnico di Milano 2 • RESEARCH BACKGROUND AND INTERESTS • Robotics & AI • Game Design • Crowdsourcing and Human Computation 6th Qualinet General Meeting
    3. 3. AGENDA 3 6th Qualinet General Meeting
    4. 4. AGENDA • Play vs Games • Pointers to Game Design • Introduction to Games with a Purpose • Gameplay validation through playtesting • A Classic Example: Big Rigs 6th Qualinet General Meeting 4
    5. 5. 5 6th Qualinet General Meeting Something really bad is going to happen…
    6. 6. 6 6th Qualinet General Meeting Or not?
    7. 7. WHAT IS PLAY? 6th Qualinet General Meeting 7 Autonomy: play is a voluntary activity. Safety: during play there are radically reduced serious consequences in what we do Exploration: the possibility to experiment and try out new things Mastery: the will to improve one’s own skills
    8. 8. GAMES VS PLAY 6th Qualinet General Meeting 8 Games are distinguished from play – Play is free-form – Games are rule-based A game is a closed, formal system that · Engages players in structured conflict and · Resolves its uncertainty in an unequal outcome. Fullerton, T.; Swain, C. & Hoffman, S. Game Design Workshop: A playcentric approach to creating innovative games, 2008
    9. 9. 6th Qualinet General Meeting 9
    10. 10. 10 FLASH INTRODUCTION TO GAME DESIGN 6th Qualinet General Meeting
    11. 11. 6th Qualinet General Meeting 11 Mancala, 7th century AD
    12. 12. 12 Unreal Tournament 3, Epic Games, 2007 6th Qualinet General Meeting
    13. 13. 6th Qualinet General Meeting 13 Turn based boardgame vs Real time action shooter Handmade physical board vs Personal Computer Public domain rules vs Copyrighted
    14. 14. 1 - PLAYERS 6th Qualinet General Meeting 14
    15. 15. 2 - OBJECTIVES 6th Qualinet General Meeting 15
    16. 16. OBJECTIVES: EXAMPLES 6th Qualinet General Meeting 16 Solution: solve a problem or puzzle before (or more accurately) than the competition or following certain constraints Connect Four, Milton Bradley, 1974 Professor Layton and the Miracle Mask, Level-5, 2011
    17. 17. 3 - PROCEDURES 6th Qualinet General Meeting 17
    18. 18. 3 – PROCEDURES: EXAMPLE 6th Qualinet General Meeting 18 Starting action: Choose a player to go first. Each player chooses a color: red or yellow. Progression of action: On each turn, a player drops one colored checker down any of the slots in the top of the grid. Resolving actions: The play alternates until one of the players gets four checkers of one color in a row. The row can be horizontal, vertical, or diagonal. Connect Four
    19. 19. 4 - RULES 19 • Define Objects • Restrict Actions • Determine Effects (ECA rules) Chess: A player cannot move her king into check. Poker: A straight is five consecutively ranked cards; a straight flush is five consecutively ranked cards of the same suit. WarCraft II: To create knight units, a player must have upgraded to a keep and built a stable. 6th Qualinet General Meeting
    20. 20. 5 – RESOURCES: EXAMPLE 20 Card Hunter, Blue Manchu Pty Ltd, TBR Time Points Actions 6th Qualinet General Meeting
    21. 21. 6 - OUTCOME 21 6th Qualinet General Meeting
    22. 22. GAMES WITH A PURPOSE 6th Qualinet General Meeting 22 Games with a Purpose (GWAPs) are digital games that generate useful data as a by-product of play. [vA06,LvA09] The design of a GWAP requires to create a game so that its structure encourages computation, correctness of the output and players retention.
    23. 23. EXAMPLE:INGRESS 6th Qualinet General Meeting 23
    24. 24. EXAMPLE: INGRESS 6th Qualinet General Meeting 24
    25. 25. EXAMPLE: AUSTRALIA COVERED 6th Qualinet General Meeting 25
    26. 26. CASE STUDY: FASHION TREND MINING 6th Qualinet General Meeting 26 Problem statement: segment fashion images for mining trends based on visual features of garments (e.g. color and texture) Use case: identifying trends in collections of images of people and garments Applications: retrieving similar garments, inspect clothing trends in image collections, analyzing trends change in the years Color descriptors Texture descriptors coarse (sub-)image similarity
    27. 27. THE FASHION TREND MINING PIPELINE 6th Qualinet General Meeting 27 Male, 24 Female, 22Female?, ??
    28. 28. REQUIREMENTS SPECIFICATION 6th Qualinet General Meeting 28 Task Description: Recognize if a particular garment is present within a picture or define a new one and outline its contours. Input Objects: A fashion image, an optional tag defining the garment to identify. Aggregation Strategy: assign a value of 0 to each pixel outside the contour and 1 to each pixel contained within the contour, sum all the contribution and apply a threshold based on the number of players. Output Data: For each submitted task the game has to provide the contour of the garment within the image (Polyline) and a tag defining the garment that has been segmented
    29. 29. Solution Mechanics: Pattern Recognition Established genre: Draw and Guess Inversion Problem Mechanic PLAYERS:Number >=2 Multilateral Competition Two different roles: Sketcher: has to draw the contours of the stated garment Guesser: has to guess the garment drawn by the sketcher SKETCHNESS 29 6th Qualinet General Meeting
    30. 30. PLAYER ROLE: SKETCHER 30 ● The only player to see the low confidence image ● “May” be asked to provide a tag for the image ● Is asked to draw the contour of the object for which the tag is provided within the allotted time ● Goal of the Sketcher is to let the other players guess the tag within a time slot without providing any other hints than the contour Low Confidence Image Tag of the target object Contour provided by the user 6th Qualinet General Meeting
    31. 31. PLAYER ROLE: GUESSER 31 ● Any other player in the game ● His/Her goal is to guess the object for which the Sketcher has provided the contour ● Not allowed to draw on the whiteboard, just to type in the chat box the probable answer as fast as possible ● Scoring: ● Sketcher: 10 pts + 1 for each guesser ● Guesser: 10 pts to the first, then decreasing down to five Contour of the garment provided by the Sketcher 6th Qualinet General Meeting
    32. 32. RESULTS: 32 6th Qualinet General Meeting
    33. 33. RESULTS: 33 Sketchness Aggregator Kota Yamaguchi, M Hadi Kiapour, Luis E Ortiz, Tamara L Berg, "Parsing Clothing in Fashion Photographs", CVPR 2012 6th Qualinet General Meeting
    34. 34. TASK TO GAME MECHANICS MATCHING 34 Does a game with game mechanics similar to the task exist? If so, integrate the task within the existing game If not, a custom game with custom game mechanics has to be implemented... 6th Qualinet General Meeting
    35. 35. HOW TO EVALUATE A GAME? 6th Qualinet General Meeting 35 The Goal is a fun game… … but also to solve a task! Game designs are hypotheses Playtests are experiments Evaluate designs off playtest results Repeat
    36. 36. REQUIREMENTS SPECIFICATION 36 uTask Task Description Input Objects Output Objects Execution Interface Operations Output Validation 6th Qualinet General Meeting
    37. 37. HOW TO IMPROVE GAME DESIGN 6th Qualinet General Meeting 37 We want to make informed decisions: • Get data early, get data often • Iterate constantly • We don’t know what’s best (players do) • Create a feedback loop between design and playtest
    38. 38. 6th Qualinet General Meeting 38 TRADITIONAL METHODS DIRECT OBSERVATION
    39. 39. 6th Qualinet General Meeting 39 TRADITIONAL METHODS VERBAL REPORTS I don’t know what I am supposed to do... I am bad at drawing...
    40. 40. 6th Qualinet General Meeting 40 TRADITIONAL METHODS Q&A/SURVEYS
    41. 41. 6th Qualinet General Meeting 41 TRADITIONAL METHODS ISSUES Artificial gameplay sessions – Many potential biases – Distorted data (interpreted behavior) – Lack of empiricism – Missing elements of objectivity – Sometimes difficult to establish emotions, baselines, and independence
    42. 42. 6th Qualinet General Meeting 42 TECHNICAL APPROACHES STAT COLLECTION
    43. 43. 6th Qualinet General Meeting 43 TECHNICAL APPROACHES STAT COLLECTION
    44. 44. 6th Qualinet General Meeting 44 TECHNICAL APPROACHES PHYSIOLOGICAL MEASUREMENTS Heartrate Skin Conductance Level Eye Tracking Face Recording EEG
    45. 45. 6th Qualinet General Meeting 45 A CLASSIC EXAMPLE: Just how bad is Big Rigs: Over the Road Racing? It's as bad as your mind will allow you to comprehend. LV68
    46. 46. 6th Qualinet General Meeting 46 CONCLUSIONS •Play is a universal need •Games and videogames are entertaining and controlled means to satisfy it •We can exploit players to accomplish astonishing results... •...if we design a good and tailored gaming experience •Technological improvements cannot fix a gameplay mined by bad design choices
    47. 47. THE CUBRIK PROJECT 47 ● CUbRIK is a research project financed by the European Union ● Goals: ● Advance the architecture of multimedia search ● Exploit the human contribution in multimedia search ● Use open-source components provided by the community ● Start up a search business ecosystem ● 6th Qualinet General Meeting