Houdini                              On Demand Labor API                 Easy Mechanical Turk Integration                 ...
Houdini                           Easy Mechanical Turk Integration with Houdini                       On Demand Labor API ...
Houdini                                       Easy Mechanical Turk Integration with Houdini                       On Deman...
Houdini                          Easy Mechanical Turk Integration with Houdini                       On Demand Labor API  ...
Houdini                        Easy Mechanical Turk Integration with Houdini                       On Demand Labor API    ...
Houdini                           Easy Mechanical Turk Integration with Houdini                       On Demand Labor API ...
Houdini                         Easy Mechanical Turk Integration with Houdini                       On Demand Labor API   ...
Houdini                         Easy Mechanical Turk Integration with Houdini                       On Demand Labor API   ...
Houdini                                      Easy Mechanical Turk Integration with Houdini                       On Demand...
Houdini                          Easy Mechanical Turk Integration with Houdini                       On Demand Labor API  ...
Houdini                             Easy Mechanical Turk Integration with Houdini                       On Demand Labor AP...
Houdini                       Easy Mechanical Turk Integration with Houdini                       On Demand Labor API     ...
Houdini                         Easy Mechanical Turk Integration with Houdini                       On Demand Labor API   ...
Houdini                             Easy Mechanical Turk Integration with Houdini                       On Demand Labor AP...
Houdini                        Easy Mechanical Turk Integration with Houdini                       On Demand Labor API    ...
Houdini                            Easy Mechanical Turk Integration with Houdini                       On Demand Labor API...
Houdini                           Easy Mechanical Turk Integration with Houdini                       On Demand Labor API ...
Houdini                            Easy Mechanical Turk Integration with Houdini                       On Demand Labor API...
Houdini                           Easy Mechanical Turk Integration with Houdini                       On Demand Labor API ...
Houdini                     Easy Mechanical Turk Integration with Houdini                       On Demand Labor API       ...
Houdini               Easy Mechanical Turk Integration with Houdini                       On Demand Labor API             ...
Houdini                                                     Easy Mechanical Turk Integration with Houdini                 ...
Houdini                         Easy Mechanical Turk Integration with Houdini                       On Demand Labor API   ...
Houdini                           Easy Mechanical Turk Integration with Houdini                       On Demand Labor API ...
Houdini                             Easy Mechanical Turk Integration with Houdini                       On Demand Labor AP...
Houdini                           Easy Mechanical Turk Integration with Houdini                       On Demand Labor API ...
Houdini                                                               Easy Mechanical Turk Integration with Houdini       ...
Houdini                                                 Easy Mechanical Turk Integration with Houdini                     ...
Upcoming SlideShare
Loading in...5
×

Easy Mechanical Turk Integration with Houdini

1,424

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,424
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Easy Mechanical Turk Integration with Houdini

  1. 1. Houdini On Demand Labor API Easy Mechanical Turk Integration with Houdini Philly AWS December Meetup Date: December 13, 2010 By: Chris Conley and Jameson Detweiler 12/13/2010Tuesday, December 14, 2010
  2. 2. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Mechanical Turk • Marketplace for work • ~ 500k workers • Accessible via API 12/13/2010Tuesday, December 14, 2010
  3. 3. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Task Examples • Moderation - images, comments, reviews, video • Categorization - products, best image, tagging • Ads - moderation, keywords • Audio Transcription / Video Captioning • Cleanup - proofreading, contact reconciliation, review editing • PDF/Image/Handwriting to Text • Research - contact info, document search 12/13/2010Tuesday, December 14, 2010
  4. 4. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Turk Overview • Workers and Requesters • HITs • Questions • Assignments • Qualifications • Notifications • Blocking Workers 12/13/2010Tuesday, December 14, 2010
  5. 5. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Worker View 12/13/2010Tuesday, December 14, 2010
  6. 6. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Creating HITs WYSIWYG API • Easy • More Flexible • Great for testing • But Complicated the waters • Programmatic • No integration Integration 12/13/2010Tuesday, December 14, 2010
  7. 7. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup WYSIWYG 12/13/2010Tuesday, December 14, 2010
  8. 8. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup WYSIWYG 12/13/2010Tuesday, December 14, 2010
  9. 9. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup API http://docs.amazonwebservices.com/AWSMturkAPI/2008-08-02/index.html 12/13/2010Tuesday, December 14, 2010
  10. 10. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup HIT Questions • XML • External Question 12/13/2010Tuesday, December 14, 2010
  11. 11. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Turk API Libraries • ruby-aws http://rubyforge.org/projects/ruby-aws/ • rTurk https://github.com/mdp/rturk • Turkee https://github.com/aantix/turkee 12/13/2010Tuesday, December 14, 2010
  12. 12. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Questions about Turk? 12/13/2010Tuesday, December 14, 2010
  13. 13. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Houdini • Simple • Qualified results • Automatic • Trusted Workers 12/13/2010Tuesday, December 14, 2010
  14. 14. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Houdini APIs Generic Optimized • Simple • Image Moderation • Exact • Ad Moderation • Multi-answer • Data Reconciliation • Data Extraction • ... 12/13/2010Tuesday, December 14, 2010
  15. 15. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Simple API • One answer per task • Cheap • Image Moderation, Low profile content 12/13/2010Tuesday, December 14, 2010
  16. 16. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Simple API Example 12/13/2010Tuesday, December 14, 2010
  17. 17. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Exact API • Multiple workers • Ensures all answers match • Returns the “final answer” • If not, escalated to trusted worker group • More expensive than Simple • Image Moderation, Data Harvesting, etc 12/13/2010Tuesday, December 14, 2010
  18. 18. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Exact API Example 12/13/2010Tuesday, December 14, 2010
  19. 19. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Multi-answer API • Ask multiple turkers same question • All answers are returned • Surveys, feedback, answer qualification on your end 12/13/2010Tuesday, December 14, 2010
  20. 20. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Image Moderation API • No need to build form_html • Just send the image url • Optionally send custom instructions and possible responses 12/13/2010Tuesday, December 14, 2010
  21. 21. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Image Moderation Example 12/13/2010Tuesday, December 14, 2010
  22. 22. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Image Moderation Demo Source: https://github.com/chrisconley/cat_or_not 12/13/2010Tuesday, December 14, 2010
  23. 23. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Admin Console • Review Work • Perform “Trusted” tasks • Worker Insights: Avg Completion Time, # of workers • Remove bad performing workers from specific task groups 12/13/2010Tuesday, December 14, 2010
  24. 24. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Getting Started • Get an API Key - http://houdiniapi.com/sign-up/ • Tunnlr - http://tunnlr.com • Sandbox - http://houdini-sandbox.heroku.com 12/13/2010Tuesday, December 14, 2010
  25. 25. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Pricing Example Monthly Fee Images Per Month $20 200 $99 1500 12/13/2010Tuesday, December 14, 2010
  26. 26. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Other Solutions • Crowdflower • Microtask • CrowdCloud 12/13/2010Tuesday, December 14, 2010
  27. 27. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Resources Houdini Docs: http://houdiniapi.com/documentation Mturk Docs: http://docs.amazonwebservices.com/AWSMturkAPI/2008-08-02/index.html 12/13/2010Tuesday, December 14, 2010
  28. 28. Houdini Easy Mechanical Turk Integration with Houdini On Demand Labor API Philly AWS December Meetup Thanks! Chris Conley Jameson Detweiler chris@houdiniapi.com jameson@houdiniapi.com @chrisconley @jaymstr 12/13/2010Tuesday, December 14, 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×