Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Gamification of crowdsourcing tasks: What motivates a medical expert?

3,031 views

Published on

Gamification of crowdsourcing tasks: What motivates a medical expert?

Published in: Technology
  • Be the first to comment

Gamification of crowdsourcing tasks: What motivates a medical expert?

  1. 1. Gamification of crowdsourcing tasks: What motivates a medical expert? 18 July 2014 Rens van Honschooten Lora Aroyo (VU) & Robert-Jan Sips (IBM)
  2. 2. What motivates medical experts?
  3. 3. Gamification of crowdsourcing tasks: What motivates a medical expert? Why incentives for a medical expert? ● Train Watson in the medical domain ● To train cognitive systems you need data ○ Lots of it ● To get annotated data:
  4. 4. Gamification of crowdsourcing tasks: What motivates a medical expert? Crowdsourcing games to collect data ● Large amounts of data, fast and cheap [1]
  5. 5. Gamification of crowdsourcing tasks: What motivates a medical expert? Crowdsourcing games to collect data ● One thing they all have in common? ● They motivate their crowd ● The challenge: find out how to motivate busy medical experts
  6. 6. Gamification of crowdsourcing tasks: What motivates a medical expert? Dr. Detective: finding clues for a diagnosis ● Learning features
  7. 7. Gamification of crowdsourcing tasks: What motivates a medical expert? Dr. Detective: finding clues for a diagnosis ● Learning features● Most experts liked playing it ● Downsides: ○ The scoring was not clear ○ No rewards for completing a task ● Incentives ○ Competition ○ Entertainment ○ Learning ● Are these all the incentives? ● Which are the most important?
  8. 8. Gamification of crowdsourcing tasks: What motivates a medical expert? Crowdsourcing incentives (literature) Moon, H., & Baek, Y. (2009). Exploring variables affecting player’s intrinsic motivation in educational games. In Proceedings of the 17th International Conference on Computers in Education (pp. 718-722). Fang, Y., & Neufeld, D. (2009). Understanding sustained participation in open source software projects. Journal of Management Information Systems, 25(4), 9-50. Incentive Example Personal interest Interested in science, medicine, etc. Contributing to the community Helping diabetes patients to eat healthy Status & recognition Being recognized as a good doctor Personal growth Learning new thing in their medical domain Enjoyment Solving challenging medical problems Choice autonomy Choosing what to do and when, at work A monetary reward Receiving money for helping patients
  9. 9. Gamification of crowdsourcing tasks: What motivates a medical expert?
  10. 10. Gamification of crowdsourcing tasks: What motivates a medical expert? Helping patients Improve medical field
  11. 11. Gamification of crowdsourcing tasks: What motivates a medical expert? ● Can we incorporate these incentives to motivate the experts? ● Created a design for next version of Dr. Detective, called Dr. Watson ○ evaluated mockup by interviewing and testing with 6 experts
  12. 12. Gamification of crowdsourcing tasks: What motivates a medical expert? The design of Dr. Watson
  13. 13. Gamification of crowdsourcing tasks: What motivates a medical expert? The design of Dr. Watson ● Personal growth, learning ● Reading most preferred ● Provide overview of literature ● Spark their interest
  14. 14. Gamification of crowdsourcing tasks: What motivates a medical expert? The design of Dr. Watson ● To unlock articles: ○ play games to gather reputation points ● Articles motivate medical experts ○ Except researchers and students ○ Medical books, videos, giveaways and accreditation points should motivate these experts
  15. 15. Gamification of crowdsourcing tasks: What motivates a medical expert?
  16. 16. Gamification of crowdsourcing tasks: What motivates a medical expert? ● Select game mode ● Cannot explore all content ● Make the experts curious ● More difficult tasks, to keep the game challenging
  17. 17. Gamification of crowdsourcing tasks: What motivates a medical expert?
  18. 18. Gamification of crowdsourcing tasks: What motivates a medical expert? ● Make game more challenging ● Motivate the experts to keep playing ○ motivate them to play less popular tasks and annotate articles without annotations
  19. 19. Gamification of crowdsourcing tasks: What motivates a medical expert?
  20. 20. Gamification of crowdsourcing tasks: What motivates a medical expert? ● Goal: beat the score of the crowd ○ average amount of points earned by others ○ matching the score also means you win ● To get points ○ Agreement with other experts ○ Pioneer by suggesting new answer ○ Three or five in a row bonus ● The scoring was clear for the experts ○ streaks motivate them to annotate correctly ● To increase game difficulty use topics the expert is less familiar with
  21. 21. Gamification of crowdsourcing tasks: What motivates a medical expert?
  22. 22. Gamification of crowdsourcing tasks: What motivates a medical expert?
  23. 23. Gamification of crowdsourcing tasks: What motivates a medical expert? ● To support learning goals more topic selection is required ● Medical experts stated to play again ○ To learn more ○ To improve their score ○ Because they enjoyed playing it ● The experts could easily navigate through the UI ○ elements at the bottom of the sidebar ● Experts wanted to see more clearly how Watson is improving
  24. 24. Gamification of crowdsourcing tasks: What motivates a medical expert? Future work - improving Dr. Watson ● Topic selection for tasks ● Personalization of: ○ The difficulty - take specialization in account ○ The homepage - change background image, reorder elements ● Incorporate additional unlockables/rewards ○ Videos ○ Medical books ○ Accreditation points ● Add ‘Watson status’ page ○ track the progress of the crowd and what watson achieved
  25. 25. Gamification of crowdsourcing tasks: What motivates a medical expert? What motivates medical experts? Helping patients Improve medical field
  26. 26. Questions?

×