Amazon Mechanical Turk: Intro & Overview


Published on

Introduction & overview of Amazon Mechanical Turk

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Amazon Mechanical Turk: Intro & Overview

  1. 1. 1 Mechanical Turk is a marketplace to access a cost- effective, scalable, global workforce of over 500K Workers in 190 countries. The ability to integrate an on-demand, scalable Workforce via API makes Mechanical Turk a solution of choice for all types of businesses worldwide. As a utility model, similar to the electric grid, it delivers what you need, when you need it, on a pay-for-what-you- use basis. As a result, businesses are shifting human- intensive, routine workloads to Mechanical Turk at an unparalleled rate vs. hiring more FTE’s, outsourcing, or staff augmentation. Amazon Mechanical Turk Mechanical Turk allows businesses to get fast, accurate work results without long term contracts or minimum commitments, helping to turn fixed costs into variable (see diagram). The primary use cases include:  data training (i.e. building up sample data to test algorithms)  data collection (i.e. categorization, moderation, time stamping, tagging/annotation, authoring, translation, transcription, research/surveys, sentiment analysis, more). Business Goal(s) Requesters are typically seeking to fix / accomplish / avoid the following business goals via Mechanical Turk:  Reduce cost, transform fixed costs to variable expense  Improve scalability or elasticity (i.e. bursting up & down) in line with workload type (i.e. on-demand)  Accelerate time-to-market  Improve quality / accuracy  Increase revenue Current Situation Requesters are typically in one or both of the following situations: 1) Seeking to supplement, or shift from, using FTE’s, staff augmentation, or traditional outsourcing to perform routine, human-intensive tasks. 2) Seeking to train/test their algorithms by quickly developing large sets of data Potential Benefits Requesters seeking to strategically add Mechanical Turk can accomplish the following:  Turn fixed costs into variable: Transform fixed [human] capital costs into variable expense, by either shifting/supplementing internal FTE or outsourcing models to an on-demand, pay-for-what-you-use model.  Scalability: Like cloud, scale up/down for the human capital needed to avoid waste, meeting your capacity requirements in-line with workload types (on/off, fast growth, backlog, variable & predictable peaks)  Accuracy: Drive fast, accurate work results at competitive prices without long-term contracts or minimum commitments.  Speed & agility: On-demand access to a scalable Workforce and results-based productivity.  Focus on business: Where FTE’s are being utilized for human-intensive tasks, redeploy towards core business. Cost of Ownership A Requester’s cost of ownership in the marketplace is comprised of i) Worker fees, and ii) Amazon fee. Amazon Mechanical Turk collects a 10% commission on top of the reward amount you set for Workers. For example, if a HIT reward is set to $0.20, Amazon Mechanical Turk collects $0.02 for each assignment. The minimum commission charged is $0.005 per assignment. When you grant a bonus, Amazon Mechanical Turk collects 10% of the bonus amount, or a minimum of $0.005 per bonus payment. If you choose to send HITs exclusively to Photo Moderation or Categorization Masters, an additional 20% fee applies.
  2. 2. 2 Thousands of businesses power their solutions through the Mechanical Turk Workforce, either accessing the marketplace directly, and/or via one of our many Partners. See below for your options on how to Connect, Engage and get Support, while following Best Practices to effectively scale. 1. How to Connect:  Manual via WebUI  Automate via API 2. How to Engage:  Self-serve  Partners 3. How to get Support  Documentation  Best Practices  Developer’s Resources  Sandbox  Case studies  Approved Partner Network  Mechanical Turk blog Mechanical Turk ecosystem - equivalent of cloud stack  Begin with a project  Break it into tasks and design your HIT  Publish HITs to the marketplace  Workers accept assignments  Workers submit assignments for review  Approve or reject assignments  Complete your project Mechanical Turk Ecosystem How it Works  Begin with a project…and define the goals & key components of your project. For example, your goal might be to clean your business listing database so that you have accurate information for consumers. The sub-components of your project might be to categorize the businesses by listing type (i.e., restaurant or service) and verify that the related address and phone number are current.  Break it into tasks and design your HIT…so many Workers can work in parallel and faster. For example, if you have 1,000 listings to verify, each listing could be an individual task. Next, design your Human Intelligence Tasks (HITs) by writing crisp and clear instructions, identifying the specific outputs and inputs desired and how much you will pay to have work completed. Calculating reward is a function of defining a competitive effective hourly rate, prorating based on task completion time, competitive marketplace rates, and throttling your cost/accuracy/productivity levers relative to your target performance metrics.  Publish HITs to the marketplace…hundreds, thousands, even millions at a time. For example, each HIT can have multiple assignments so that different Workers can provide answers to the same set of questions and you can compare the results to form an agreed-upon answer.  Workers accept assignments…for special skills, you can qualify the Workforce. For example, if Workers need special skills, specific geography, or specific marketplace rating to complete your tasks, you can require that they pass a Qualification test before they are allowed to work on your HITs.  Workers submit assignments for review. When a Worker completes your HIT, he or she submits an assignment for you to review.  Approve or reject assignments…you pay only for approved work. When your work items have been completed, you can review the results and approve or reject them.  Complete your project…Congratulations; your project has been completed and your Workers paid!
  3. 3. 3 Next Steps We recommend setting up and agreeing to an evaluation roadmap, to help guide us through discussions, to proof- of-concept, to scaling; following is an example: Strategy & Governance •TBD: Initial diagnostics on business + use case(s) objectives • TBD: Map to Mechanical Turk capabilities, deliverable: Updated Goals Summary • TBD: Requesters confirms GO - NO GO to further evaluate Planning & Design • TBD: Requester evaluates & selects WebUI/API/partners • TBD: Discuss and define pilot/POC; review details including scope, use case, deliverables, cost, schedule, success metrics (cost, accuracy, productivity) Implementation • TBD: Launch pilot, measure results against target success metrics • TBD: Requester defines crowdsourcing is a viable solution GO - NO GO • TBD: Requester selects engagement model of choice GO - NO GO • TBD: Reference check/contracts for supplier of choice Launch • TBD: Use case campaign Start • TBD: Monitor performance, iterate & scale