Codemotion2012

730 views
661 views

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
730
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Codemotion2012

  1. 1. A novel trilling operator for DBMSs (and other sw systems) Paolo Merialdo (Uniroma3)
  2. 2. A novel trilling operator for DBMSs (and other sw systems) Houyhnhnms* (pron. ˈhwi-məm)* Thanks to Jonathan Swift
  3. 3. A novel trilling operator for DBMSs (and other sw systems) Houyhnhnms* (pron. ˈhwi-məm)* Thanks to Jonathan Swift
  4. 4. An Amazon issue…Categorize products with 1.0 accuracyUnfeasible without introducing human intelligence in the pipeline
  5. 5. An Amazon issue……became a market opportunity!Mechanical Turk!
  6. 6. An Amazon issue……became a market opportunity! Mechanical Turk
  7. 7. Amazon Mechanical TurkA marketplace for HITs (Human Intelligent Tasks)"Requesters" (employers) post tasks"Workers" (on line users) pickup and completetaskPayment (few cents for HIT)
  8. 8. Typical (frequently posted) Tasks Transcription Content generation Content rewriting Object classification Content evaluation and feedback Mediator for other requesters (!)Panagiotis Ipeirotis (NYU) "Analyzing the Amazon Mechanical Turk Marketplace"
  9. 9. 90% of HITS < $0.10 Prices** Data collected from January 2009 till April 2010Panagiotis Ipeirotis (NYU) "Analyzing the Amazon Mechanical Turk Marketplace"
  10. 10. Volume of transactions 90% of days < $2,000 median arrival rate: $1,040 per day median completion rate: $1,155 per day* Data collected from January 2009 till April 2010Panagiotis Ipeirotis (NYU) "Analyzing the Amazon Mechanical Turk Marketplace"
  11. 11. Responsiveness and reward Assignment: 100 HITsMichael Franklin et al (Berkeley and ETH) CrowdDB: Answering Queries with Crowdsourcing
  12. 12. Not only AMT RABJ (FreeBase) 2.3M human judgments 18 sec/judgment Tipical Tasks: reconciliations, typeWriter, geographers, genderizerShailesh Kochhar et al "The Anatomy of a large scale human computation engine"
  13. 13. Not only AMTSamaSource founder Leila Chirayath Janah (26) social business (raised > 5M$ from Google, Rockefeller, Ford Foundation et al) >2,000 worker from dev. countries (India, Kenya) 15$ day per worker To date paid out 1M$ in wages
  14. 14. Not only AMTStartups! Crowdflowers (organize AMT HITS for enterprises) uTest: sw testing (functional, UX, usability) MicroTask: doc processing (feed paper docs into systems)
  15. 15. Houyhnhnms are among us…
  16. 16. Michael Franklin et al (Berkeley and ETH) CrowdDB: Answering Queries with Crowdsourcing
  17. 17. Michael Franklin et al (Berkeley and ETH) CrowdDB: Answering Queries with Crowdsourcing
  18. 18. References• https://www.mturk.com/• http://samasource.org/• http://www.utest.com/• http://crowdflower.com/• http://www.microtask.com/
  19. 19. References• Michael Franklin et al (Berkeley and ETH) CrowdDB: Answering Queries with Crowdsourcing• Shailesh Kochhar et al (Metaweb Tech) "The Anatomy of a large scale human computation engine"• Panagiotis Ipeirotis (NYU) "Analyzing the Amazon Mechanical Turk Marketplace"• Aditya Parameswaran et al (Stanford U) "Answering Queries using Humans, Algorithms and Databases"• http://amplab.cs.berkeley.edu/• Adam Marcus et Al (MIT) Crowdsourced Databases: Query Processing with People

×