Strategies for Increasing Engagement in Crowdsourcing
•
2 likes•412 views
1. The document discusses strategies for increasing engagement in crowdsourcing projects through targeted interventions. It presents a machine learning approach to predict when users are at risk of disengaging and then intervenes with personalized messages. 2. The interventions are tested in different conditions on the Galaxy Zoo crowdsourcing platform, varying the message content and timing. Results show that prediction-based interventions can significantly increase user contributions compared to no intervention or random interventions. 3. The authors conclude that increasing engagement requires an intelligent intervention design combined with predictive timing of the interventions. They also discuss the importance of engagement for citizen science projects and make their code for the intervention platform openly available.
Recommended
Recommended
More Related Content
What's hot
Viewers also liked
Viewers also liked (12)
Similar to Strategies for Increasing Engagement in Crowdsourcing
Similar to Strategies for Increasing Engagement in Crowdsourcing (20)
Recently uploaded
Recently uploaded (20)
Strategies for Increasing Engagement in Crowdsourcing
- 1. 1 Intervention Strategies for Increasing Engagement in Crowdsourcing: Platform, Predictions and Experiments Avi Segal, Ben-Gurion University, Israel Ya’akov (Kobi) Gal, Ben-Gurion University, Israel Ece Kamar, Microsoft Research, Redmond WA, USA Eric Horvitz, Microsoft Research, Redmond WA, USA Alex Bowyer, University of Oxford, UK Grant Miller, University of Oxford, UK
- 2. Citizen Science (Zooniverse.org) Wikipedia: scientific research conducted, in whole or in part, by amateur or nonprofessional scientists. 2
- 4. The Disengagement Problem 4 Eveleigh, Alexandra, et al. "Designing for dabblers and deterring drop-outs in citizen science.” CHI 2014 Sauermann, Henry, and Chiara Franzoni. "Crowd science user contribution patterns and their implications." PNAS 2015
- 5. The Approach: Intervention 5 Dimension Description Examples Target Target population for intervention all, newcomers, heavy users Time Intervention timing periodical; as soon as risk factor is identified; one day after disengagement Duration Intervention (interruption) duration for x minutes; until intervention is acknowledged; while user is on webpage Media Type Method of presentation text; graphics; audio; video; mixed Channel Medium of delivery email; web page; modal message Mechanism The method used for this intervention Messages; help system; task routing; competition; achievements; … Content Content of the intervention communication “...If GalaxyZoo didn't suit you, then…”; checkmark image;
- 7. The Intervention Server Platform 7
- 8. Predicting Disengagement 8 Predict Disengagement Intervene Evaluate Supervised Learning Supervised Learning User will disengage within given window Session & History 1M records 13.5K users Andrew Mao, Ece Kamar, and Eric Horvitz. "Why stop now? predicting worker engagement in online crowdsourcing.” HCOPM 2013
- 9. Intervention Message: Helpful 9 Predict Disengagement Intervene Evaluate Please don’t stop just yet. You’ve been extremely helpful so far. Your votes are really helping us to understand deep mysteries about galaxies. Ryan, Richard M., and Edward L. Deci. "Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being." APA(2000)
- 10. Intervention Message: Community 10 Predict Disengagement Intervene Evaluate Thousands of people are taking part in the project every month. Visit Talk at talk.galaxyzoo.org to discuss the images you see with them. Kraut, Robert E., and Paul Resnick. "Encouraging contribution to online communities." Building successful online communities: Evidence-based social design 2011
- 11. Intervention Message: Anxiety 11 Predict Disengagement Intervene Evaluate We use statistical techniques to get the most from every answer; So, you don’t need to worry about being “right”. Just tell us what you see. Segal, Avi, et al. "Improving productivity in citizen science through controlled intervention." WWW 2015
- 13. Conditions 13 Predict Disengagement Intervene Evaluate # Type 1. No Intervention 2. Random Timing, Helpful Message 3. Random Timing, Community Message 4. Random Timing, Anxiety Message 5. Prediction Based Timing, Helpful Message 6. Prediction Based Timing, Community Message 7. Prediction Based Timing, Anxiety Message
- 16. Conclusion Increasing engagement = intervention design + intelligent timing Code is open sourced Sequential decision making with uncertainty Off policy policy evaluation Engagement Importance 16
- 17. Thank You & Questions avise@post.bgu.ac.il
- 18. Human Aware AI? 30 Learn Interven e Evaluate “Intervention” title 1 Papers this morning ~100 Learning ~80 ? 10 Intervention? ~10 Large scale int. ~2 % 2% Is this what we want?Is this what we want?