Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Expert Crowdsourcing with Flash Teams | CrowdConf 2013 poster

  • Be the first to comment

  • Be the first to like this

Expert Crowdsourcing with Flash Teams | CrowdConf 2013 poster

  1. 1. Expert Crowdsourcing with Flash Teams Daniela Retelny, Sébastien Robaszkiewicz, Alexandra To, Michael Bernstein {dretelny, robi, ato1120, msb} Crowdsourcing is Plateauing in Complexity Flash Team Examples Crowd workers struggle to complete open ended and highly Our evaluation of flash teams to date includes the successful completion of eight different flash team examples, utilizing different team structures and workflows. The flash teams varied in size, complexity and expertise and interdependent tasks. they produced outputs that ranged in completion time and quality. The images below depict the team structures, Despite good intentions, teams of crowd experts are workflows and output from three napkin sketch, animation and MOOC flash team examples we completed. typically neither effective nor efficient at these tasks. Expert crowdsourcing lacks generalizable techniques for guiding crowds through complex tasks. Flash Teams We introduce flash teams, modular team structures that computationally guide a team of crowd experts reproducibly and at scale. These modular structures enable crowd expert teams to grow, adapt, and recombine into larger organizations. Defining attributes of flash teams: MODULARITY: self-contained and replicable team structures composed of crowd experts that are computationally guided. ELASTICITY: parameterizable team growth on demand, allowing teams to grow and shrink as expertise are needed. PIPELINING: parallelism enabled by defining interdependencies and handoffs. Methods: To explore the feasibility and scope of flash teams, and understand the effectiveness of modularity, elasticity and pipelining, we gathered thirteen different flash teams to complete complex tasks ranging from design to education. Foundry, a Flash Team Authoring Platform To encapsulate the foundations of the flash teams formalism and lessen We recruit experts through oDesk, a crowdsourcing platform the threshold to creating flash teams, we created Foundry, a web platform that allows requesters to create, reuse and recombine flash team for expert work that allows requesters to post a request under a specific expertise category (e.g., Python development, structures and workflows and allows team members to coordinate tasks and track progress. The image to the right shows a screenshot of Foundry. sound production, or visual design). STANFORD HCI GROUP