Codemotion2012
Upcoming SlideShare
Loading in...5
×
 

Codemotion2012

on

  • 625 views

 

Statistics

Views

Total Views
625
Slideshare-icon Views on SlideShare
625
Embed Views
0

Actions

Likes
0
Downloads
4
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Codemotion2012 Codemotion2012 Presentation Transcript

    • A novel trilling operator for DBMSs (and other sw systems) Paolo Merialdo (Uniroma3)
    • A novel trilling operator for DBMSs (and other sw systems) Houyhnhnms* (pron. ˈhwi-məm)* Thanks to Jonathan Swift
    • A novel trilling operator for DBMSs (and other sw systems) Houyhnhnms* (pron. ˈhwi-məm)* Thanks to Jonathan Swift
    • An Amazon issue…Categorize products with 1.0 accuracyUnfeasible without introducing human intelligence in the pipeline
    • An Amazon issue……became a market opportunity!Mechanical Turk!
    • An Amazon issue……became a market opportunity! Mechanical Turk
    • Amazon Mechanical TurkA marketplace for HITs (Human Intelligent Tasks)"Requesters" (employers) post tasks"Workers" (on line users) pickup and completetaskPayment (few cents for HIT)
    • Typical (frequently posted) Tasks Transcription Content generation Content rewriting Object classification Content evaluation and feedback Mediator for other requesters (!)Panagiotis Ipeirotis (NYU) "Analyzing the Amazon Mechanical Turk Marketplace"
    • 90% of HITS < $0.10 Prices** Data collected from January 2009 till April 2010Panagiotis Ipeirotis (NYU) "Analyzing the Amazon Mechanical Turk Marketplace"
    • Volume of transactions 90% of days < $2,000 median arrival rate: $1,040 per day median completion rate: $1,155 per day* Data collected from January 2009 till April 2010Panagiotis Ipeirotis (NYU) "Analyzing the Amazon Mechanical Turk Marketplace"
    • Responsiveness and reward Assignment: 100 HITsMichael Franklin et al (Berkeley and ETH) CrowdDB: Answering Queries with Crowdsourcing
    • Not only AMT RABJ (FreeBase) 2.3M human judgments 18 sec/judgment Tipical Tasks: reconciliations, typeWriter, geographers, genderizerShailesh Kochhar et al "The Anatomy of a large scale human computation engine"
    • Not only AMTSamaSource founder Leila Chirayath Janah (26) social business (raised > 5M$ from Google, Rockefeller, Ford Foundation et al) >2,000 worker from dev. countries (India, Kenya) 15$ day per worker To date paid out 1M$ in wages
    • Not only AMTStartups! Crowdflowers (organize AMT HITS for enterprises) uTest: sw testing (functional, UX, usability) MicroTask: doc processing (feed paper docs into systems)
    • Houyhnhnms are among us…
    • Michael Franklin et al (Berkeley and ETH) CrowdDB: Answering Queries with Crowdsourcing
    • Michael Franklin et al (Berkeley and ETH) CrowdDB: Answering Queries with Crowdsourcing
    • References• https://www.mturk.com/• http://samasource.org/• http://www.utest.com/• http://crowdflower.com/• http://www.microtask.com/
    • References• Michael Franklin et al (Berkeley and ETH) CrowdDB: Answering Queries with Crowdsourcing• Shailesh Kochhar et al (Metaweb Tech) "The Anatomy of a large scale human computation engine"• Panagiotis Ipeirotis (NYU) "Analyzing the Amazon Mechanical Turk Marketplace"• Aditya Parameswaran et al (Stanford U) "Answering Queries using Humans, Algorithms and Databases"• http://amplab.cs.berkeley.edu/• Adam Marcus et Al (MIT) Crowdsourced Databases: Query Processing with People