The document discusses the concept of crowdsourcing, emphasizing collaboration with users to improve outcomes. It highlights various aspects such as game mechanics, usability, and lessons learned from previous crowdsourcing projects like the MP's expenses and Wildlife Near You. Key takeaways include the importance of flexible goals, optimizing user experience, and ensuring minimal barriers for participation.

1. Building
crowdsourcing
applications
Simon Willison - simonwillison.net - @simonw
@media - 9th June 2010

4. Crowdsourcing?
Let me just cop to the fact that
“crowdsourcing” is a stupid buzzword. But
like “blog” before it, sometimes it’s the stupid
term that sticks. For my purposes, it means
collaborating with the people who used to be the
silent audience to make something better than you
could make alone. - Derek Powazek
http://powazek.com/posts/2443

5. Accuracy
Game mechanics
Psychology
Real-time Copywriting
Usability
Crowdsourcing
Visual design Incentives Moderation
Ethics Write-heavy
Competition
Statistics
Legal liability

6. Accuracy
Game mechanics
Psychology
Real-time Copywriting
Usability
Crowdsourcing
Visual design Incentives Moderation
Ethics Write-heavy
Competition
Statistics
Legal liability

7. Accuracy
Game mechanics
Psychology
Real-time Copywriting
Usability
Crowdsourcing
Visual design Incentives Moderation
Ethics Write-heavy
Competition
Statistics
Legal liability

8. Accuracy
Game mechanics
Psychology
Real-time Copywriting
Usability
Crowdsourcing
Visual design Incentives Moderation
Ethics Write-heavy
Competition
Statistics
Legal liability

9. Accuracy
Game mechanics
Psychology
Real-time Copywriting
Usability
Crowdsourcing
Visual design Incentives Moderation
Ethics Write-heavy
Competition
Statistics
Legal liability

10. Accuracy
Game mechanics
Psychology
Real-time Copywriting
Usability
Crowdsourcing
Visual design Incentives Moderation
Ethics Write-heavy
Competition
Statistics
Legal liability

11. Examples

12. OpenStreetMap

17. Google Image Labeler

20. ScenicOrNot

23. XKCD colour survey

29. Crowdsourcing at
the Guardian

30. The Blair
Rich Project

34. MP’s expenses v1
http://mps-expenses.guardian.co.uk/

35. Background
June 2009
450,000 pages of expenses documents released
“Transparency” = dodgy scanned PDFs
One week notice - so one week to build it!

42. Stuff that worked
The progress bar
Photos of the MPs
Releasing a small group of documents at first
Score boards (once we finally added them)
Especially the “top in last 48 hours” one

43. Stuff that didn't
Releasing everything else at once
Asking the wrong questions
Line items!
Too much time fighting scalability fires
Reporting tools were 24 hours too late

44. Contributors
total users
date

45. Votes per user
users
number of votes cast

46. MP’s expenses v2
http://mps-expenses2.guardian.co.uk/

47. Background
December 2009
Smaller number of documents
One weeks notice (again)
Opportunity to learn from our earlier mistakes

48. Goals
Find stuff our journalists cared about
Less boring data entry
Data coming out again from the start
Visible rewards for contributors
More digestible tasks
Better sense of activity by other people

56. Lessons learned
Use Redis for random selections, not MySQL
Assignments made a huge improvement
The most important logic in a crowdsourcing
system is the next thing to review button
“Oldest first” pagination is critical

57. WildlifeNearYou.com

58. /dev/fort

60. Where’s my
nearest llama?

69. Lessons learned
Be flexible: your users may not share your
precise goals
Optimise for the fat head of your user base
Expose recent activity to site staff
Users will do almost anything for a medal!

70. Final thoughts
Don’t be afraid: even flawed crowdsourcing
systems produce fascinating results
Think hard about the questions you ask
Have a minimal barrier to entry
Get the next task logic right. Seriously.

71. Thank you
http://simonwillison.net/
http://twitter.com/simonw
http://simonwillison.net/tags/crowdsourcing/