Building
   crowdsourcing
    applications
Simon Willison - simonwillison.net - @simonw
          @media - 9th June 2010
Crowdsourcing?
Let me just cop to the fact that
“crowdsourcing” is a stupid buzzword. But
like “blog” before it, sometimes...
Accuracy
                                     Game mechanics
             Psychology
                            Real-time...
Accuracy
                                     Game mechanics
             Psychology
                            Real-time...
Accuracy
                                     Game mechanics
             Psychology
                            Real-time...
Accuracy
                                     Game mechanics
             Psychology
                            Real-time...
Accuracy
                                     Game mechanics
             Psychology
                            Real-time...
Accuracy
                                     Game mechanics
             Psychology
                            Real-time...
Examples
OpenStreetMap
Google Image Labeler
ScenicOrNot
XKCD colour survey
Crowdsourcing at
  the Guardian
The Blair
Rich Project
MP’s expenses v1
http://mps-expenses.guardian.co.uk/
Background

June 2009

450,000 pages of expenses documents released

“Transparency” = dodgy scanned PDFs

One week notice ...
Stuff that worked

The progress bar

Photos of the MPs

Releasing a small group of documents at first

Score boards (once w...
Stuff that didn't

Releasing everything else at once

Asking the wrong questions

  Line items!

Too much time fighting sca...
Contributors
total users




                    date
Votes per user
users




             number of votes cast
MP’s expenses v2
http://mps-expenses2.guardian.co.uk/
Background

December 2009

Smaller number of documents

One weeks notice (again)

Opportunity to learn from our earlier mi...
Goals
Find stuff our journalists cared about

Less boring data entry

Data coming out again from the start

Visible reward...
Lessons learned

Use Redis for random selections, not MySQL

Assignments made a huge improvement

The most important logic...
WildlifeNearYou.com
/dev/fort
Where’s my
nearest llama?
Lessons learned

Be flexible: your users may not share your
precise goals

Optimise for the fat head of your user base

Exp...
Final thoughts

Don’t be afraid: even flawed crowdsourcing
systems produce fascinating results

Think hard about the questi...
Thank you


http://simonwillison.net/

http://twitter.com/simonw

http://simonwillison.net/tags/crowdsourcing/
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Building crowdsourcing applications
Upcoming SlideShare
Loading in...5
×

Building crowdsourcing applications

12,317

Published on

A talk on my experiences building crowdsourcing applications, both at the Guardian newspaper and for my own personal projects. Presented at Web Directions @media 2010 on June 9th.

Published in: Technology
1 Comment
28 Likes
Statistics
Notes
  • Don't be afraid! Take risks! This is the most important lesson in this presentation!

    Thank you for this presentation, Simon! It's really inspirational!

    Kudos! from all of us at crowdsourcing.org
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
12,317
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
320
Comments
1
Likes
28
Embeds 0
No embeds

No notes for slide

Building crowdsourcing applications

  1. 1. Building crowdsourcing applications Simon Willison - simonwillison.net - @simonw @media - 9th June 2010
  2. 2. Crowdsourcing? Let me just cop to the fact that “crowdsourcing” is a stupid buzzword. But like “blog” before it, sometimes it’s the stupid term that sticks. For my purposes, it means collaborating with the people who used to be the silent audience to make something better than you could make alone. - Derek Powazek http://powazek.com/posts/2443
  3. 3. Accuracy Game mechanics Psychology Real-time Copywriting Usability Crowdsourcing Visual design Incentives Moderation Ethics Write-heavy Competition Statistics Legal liability
  4. 4. Accuracy Game mechanics Psychology Real-time Copywriting Usability Crowdsourcing Visual design Incentives Moderation Ethics Write-heavy Competition Statistics Legal liability
  5. 5. Accuracy Game mechanics Psychology Real-time Copywriting Usability Crowdsourcing Visual design Incentives Moderation Ethics Write-heavy Competition Statistics Legal liability
  6. 6. Accuracy Game mechanics Psychology Real-time Copywriting Usability Crowdsourcing Visual design Incentives Moderation Ethics Write-heavy Competition Statistics Legal liability
  7. 7. Accuracy Game mechanics Psychology Real-time Copywriting Usability Crowdsourcing Visual design Incentives Moderation Ethics Write-heavy Competition Statistics Legal liability
  8. 8. Accuracy Game mechanics Psychology Real-time Copywriting Usability Crowdsourcing Visual design Incentives Moderation Ethics Write-heavy Competition Statistics Legal liability
  9. 9. Examples
  10. 10. OpenStreetMap
  11. 11. Google Image Labeler
  12. 12. ScenicOrNot
  13. 13. XKCD colour survey
  14. 14. Crowdsourcing at the Guardian
  15. 15. The Blair Rich Project
  16. 16. MP’s expenses v1 http://mps-expenses.guardian.co.uk/
  17. 17. Background June 2009 450,000 pages of expenses documents released “Transparency” = dodgy scanned PDFs One week notice - so one week to build it!
  18. 18. Stuff that worked The progress bar Photos of the MPs Releasing a small group of documents at first Score boards (once we finally added them) Especially the “top in last 48 hours” one
  19. 19. Stuff that didn't Releasing everything else at once Asking the wrong questions Line items! Too much time fighting scalability fires Reporting tools were 24 hours too late
  20. 20. Contributors total users date
  21. 21. Votes per user users number of votes cast
  22. 22. MP’s expenses v2 http://mps-expenses2.guardian.co.uk/
  23. 23. Background December 2009 Smaller number of documents One weeks notice (again) Opportunity to learn from our earlier mistakes
  24. 24. Goals Find stuff our journalists cared about Less boring data entry Data coming out again from the start Visible rewards for contributors More digestible tasks Better sense of activity by other people
  25. 25. Lessons learned Use Redis for random selections, not MySQL Assignments made a huge improvement The most important logic in a crowdsourcing system is the next thing to review button “Oldest first” pagination is critical
  26. 26. WildlifeNearYou.com
  27. 27. /dev/fort
  28. 28. Where’s my nearest llama?
  29. 29. Lessons learned Be flexible: your users may not share your precise goals Optimise for the fat head of your user base Expose recent activity to site staff Users will do almost anything for a medal!
  30. 30. Final thoughts Don’t be afraid: even flawed crowdsourcing systems produce fascinating results Think hard about the questions you ask Have a minimal barrier to entry Get the next task logic right. Seriously.
  31. 31. Thank you http://simonwillison.net/ http://twitter.com/simonw http://simonwillison.net/tags/crowdsourcing/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×