Chien-Wei Lin, Kuan-Ta Chen, Ling-Jyh Chen
              Academia Sinica

        Irwin King, and Jimmy Lee
   The Chinese...
Motivation
ESP games annotate images on the web.



Can we optimize the goodput of the game?



Service provider can utili...
Idea
Model the performance of the game and optimize it.

A more generalized ESP game.
  The number of players can be more ...
3 Collaborative Quantity
Efficiency
  the rate that labels are matched.


Quality
  the proportion of good labels among al...
Assumption of Model
Round-based play
  Make only one guess in each round.
Independent guess
  Current guess is not affecte...
Parameters in the Model
Number of players
  denoted as n.
Consensus threshold
  denoted as m.
Size of good vocabulary
  de...
Model Validation
Trade-off between       Validate the model by
efficiency & quality.   simulations.
Optimal Stopping Conditions
Optimal stopping condition changes under different
parameter settings.
Benefit of Optimization
We provide twice as much utility as a non-optimized
game.
Contribution
Model for generalized games.

Propose an optimal termination condition to optimize
the system.

Game provider...
Thank You!
Upcoming SlideShare
Loading in …5
×

An Analytical Approach to Optimizing The Utility of ESP Games

528
-1

Published on

In this paper, we propose an analytical model for computing the utility of ESP games, i.e., the throughput rate of appropriate labels for given images. The model targets generalized games, where the number of players, the consensus threshold, and the stopping condition are variable. Via extensive simulations, we show that our model can accurately predict the stopping condition that will yield the optimal utility of an ESP game under a specific game setting. A service provider can therefore utilize the model to ensure that the hosted ESP games produce high-quality labels efficiently, given that the number of players willing to invest time and effort in the game is limited.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
528
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

An Analytical Approach to Optimizing The Utility of ESP Games

  1. 1. Chien-Wei Lin, Kuan-Ta Chen, Ling-Jyh Chen Academia Sinica Irwin King, and Jimmy Lee The Chinese University of Hong Kong
  2. 2. Motivation ESP games annotate images on the web. Can we optimize the goodput of the game? Service provider can utilize our model to improve their system.
  3. 3. Idea Model the performance of the game and optimize it. A more generalized ESP game. The number of players can be more than 2. The consensus threshold can be any positive integer, but not larger than the number of players. The stopping condition can be more than 1.
  4. 4. 3 Collaborative Quantity Efficiency the rate that labels are matched. Quality the proportion of good labels among all matched labels. Utility the throughput rate of good labels. Utility = Efficiency × Quality
  5. 5. Assumption of Model Round-based play Make only one guess in each round. Independent guess Current guess is not affected by previous guesses. Good and bad words The sizes of good and bad words are both limited. Players will do their best to guess good words. Uniform guess The guess is made uniformly.
  6. 6. Parameters in the Model Number of players denoted as n. Consensus threshold denoted as m. Size of good vocabulary denoted as vgood . Probability of guessing good words denoted as probgood . Stopping condition is our main variable.
  7. 7. Model Validation Trade-off between Validate the model by efficiency & quality. simulations.
  8. 8. Optimal Stopping Conditions Optimal stopping condition changes under different parameter settings.
  9. 9. Benefit of Optimization We provide twice as much utility as a non-optimized game.
  10. 10. Contribution Model for generalized games. Propose an optimal termination condition to optimize the system. Game providers can utilize our model to maximize the outcome of games.
  11. 11. Thank You!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×