Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Multi-armed Bandit Approach to Online Spatial Task Assignment


Published on

Presented at UIC 2014

Spatial crowdsourcing uses workers for performing tasks that require travel to different locations in the physical world. This paper considers the online spatial task assignment problem. In this problem, spatial tasks arrive in an online manner and an appropriate worker must be assigned to each task. However, outcome of an assignment is stochastic since the worker can choose to accept or reject the task. Primary goal of the assignment algorithm is to maximize the number of successful assignments over all tasks. This presents an exploration-exploitation challenge; the algorithm must learn the task acceptance behavior of workers while selecting the best worker based on the previous learning. We address this challenge by defining a framework for online spatial task assignment based on the multi-armed bandit formalization of the problem. Furthermore, we adapt a contextual bandit algorithm to assign a worker based on the spatial features
of tasks and workers. The algorithm simultaneously adapts
the worker assignment strategy based on the observed task
acceptance behavior of workers. Finally, we present an evaluation
methodology based on a real world dataset, and evaluate the
performance of the proposed algorithm against the baseline
algorithms. The results demonstrate that the proposed algorithm
performs better in terms of the number of successful assignments.

Published in: Data & Analytics
  • Be the first to comment

A Multi-armed Bandit Approach to Online Spatial Task Assignment

  1. 1. Umair ul Hassan, Edward Curry A Multi-armed Bandit Approach to Online Spatial Task Assignment 11th IEEE International Conference on Ubiquitous Intelligence and Computing December 9-12, 2014 Ayodya Resort, Bali, Indonesia
  2. 2. Agenda  Spatial Crowdsourcing  Task Assignment  Multi-armed Bandit  SpatialUCB  Experiments & Results  Conclusion  Q&A 4 July 20162
  3. 3. Spatial Crowdsourcing Tasks require physical travel to a location. 4 July 20163 Please provide recent photos of this place.
  4. 4. Spatial Crowdsourcing 4 July 20164 Overview of interacting agents WorkersRequesters Spatial Crowdsourcing Platform Worker Assignment & Filtering Response Aggregation & Filtering submit tasks receive results feedback assign tasks submit responses
  5. 5. Task Assignment Pull Method vs Push Method 4 July 20165 Spatial Crowdsourcing Platform Submit tasks Worker selects tasks Spatial Crowdsourcing Platform Submit tasks Server assigns tasks Our focusStarvation & Search Friction
  6. 6. Assumptions A1) Workers do not actively visit the platform to seek tasks. A2) Tasks arrivals are dynamic: task arrival time is unknown A3) The outcome of an assignment is stochastic: worker may accept or reject an assigned task 4 July 20166 Please provide recent photos of this place.
  7. 7. IMIRT Framework Intelligent Models for Iterative Routing of Tasks (A1) 4 July 20167 IMIRT Framework Router ProfilerWorker Models Interface assignment update outcomeexpectation wj ti wj-1 ti+1 wj+1 ti+2 Find The Best Assignment
  8. 8. Offline Task Assignment 4 July 20168 Assignment indicator Acceptance inidcator Optimization Constraints Optimization Objective Number of tasks Number of workers
  9. 9. Online Task Assignment  Either tasks or workers arrive dynamically (A2)  Existing research 4 July 20169
  10. 10. Online Task Assignment  Assignment under uncertainty (A3) Workers may accept or reject an assigned task Exploration-Exploitation Trade-off 4 July 201610 Select worker to learn about acceptance behaviour Select worker that has highest expectation of successful outcome
  11. 11. Multi-armed Bandit Which arm should be played to maximize reward? 4 July 201611 antigavin@flickr “Bandit problems embody in essential form a conflict evident in all human action: choosing actions which yield immediate reward vs. choosing actions (e.g. acquiring information or preparing the ground) whose benefit will come only later.” — P. Whittle (1980)
  12. 12. Multi-armed Bandit Application of multi-armed bandit model 4 July 201612 Clinical Trials Ad Placement Adaptive Routing Stock Investment
  13. 13. Multi-armed Bandit  Assignment under uncertainty (R3) Workers may accept or reject an assigned task 4 July 201613 Exploration Exploitation Trade-off antigavin@flickr What if we knew that reward of each machine is dependent on the time of day (Contextual Bandit)
  14. 14. SpatialUCB  Multi-armed bandit approach to task assignment  Optimistic assignment under uncertainty  Exploits relationship between spatial context and task acceptance 4 July 201614 Start Wait for new task Observe spatial features of task and all workers Calculate expectation of success for all workers Assign worker with highest expectation Observe outcome for the assignment Use ridge regression to update model coefficients for assigned worker
  15. 15. Gowalla Dataset  Location based Social Network  User = Worker  Check-in Spot = Spatial Task  Highlight Spot = Non Spatial Task Characteristics of selected 90 workers 4 July 201615 Users 9,183 Spots 30,367 Highlights 2,767 Check-ins 357,753
  16. 16. Experiment 1  Simulate a worker as Binomial stochastic process  Simulate 90 workers and 5,000 tasks Probability of success based on Gowalla dataset ASR (Assignment Success Rate)  Context free algorithms 4 July 201616
  17. 17. Experiment 2  Assignment with spatial context ASR (Assignment Success Rate) ATD (Average Travel Distance) 4 July 201617 5k tuning tasks 26k testing tasks
  18. 18. Conclusion  Multi-armed bandit is an appropriate tool for modelling the task assignment problem in spatial crowdsourcing  SpatialUCB performs better for learning task acceptance behaviour based on the spatial contextual information  We plan to extend SpatialUCB to combinatorial assignments 4 July 201618
  19. 19. Thank You Umair ul Hassan