Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Amazon Mechanical Turk Overview

1,149 views

Published on

Amazon Mechanical Turk (https://requester.mturk.com), a crowdsourcing marketplace, adds value by processing work that cannot be computerized.

Typically, these include large-scale [data] projects - think Amazon.com's 500MM product database - which requires some form of human judgment to help digitize, validate, moderate, tag, cleanse, categorize, dedupe, etc...This is accomplished by breaking down the project into micro-tasks, distributing to qualified cloud-based workers, who then deliver the results.

Published in: Technology
  • Be the first to comment

Amazon Mechanical Turk Overview

  1. 1. Introducing AmazonMechanical TurkA Brief Overview of What, Why, and HowJohn Hoskins
  2. 2. AgendaWhat is Mechanical Turk?When would you use it?How are others using it - Case StudiesHow – Implementation options
  3. 3. What isMechanical Turk?
  4. 4. Mechanical Turk is a Marketplace.
  5. 5. Mechanical Turk is a marketplace for work.Mechanical Turk gives businesses and developers accessto an on-demand, scalable workforce. • Flexibility: Scale your workforce up and down quickly • Accuracy: Get high quality, efficient and cost effective results. • Price: Pay only when you are satisfied with the results. • Speed: Start receiving results in minutes
  6. 6. How it works:.“Design & Publish” “Validate, Pay & Go”
  7. 7. Workforce500,000 Workers190+ CountriesWho are the Workers?• Workers are global• Workers represent numerous skill sets• 24X7 Follow the sun availabilityManaging Your Workforce• You can choose who is allowed to work on your assignments• Can narrow to US based (i.e. when you need western culture competence)• Leverage qualifications to find your best Workers
  8. 8. When would you use Mechanical Turk?
  9. 9. Business ChallengesOur Community is generating 10,000 comments per day– how can we eliminate SPAM?We pull information from hundreds of sources – how dowe normalize and cleanse?We have thousands of legacy assets – how do we makethem searchable?Our heuristics need jump started – how can I generatethousands of reference data points fast?We need to add a new attribute to our data?
  10. 10. Business ChallengesRequires Human Judgment: Recognition, Reasoning,Context, Prior KnowledgeCannot be completely solved algorithmicallyHas unpredictable fluctuations in volume
  11. 11. How are businesses using it? Data Management  Categorization  Data Verification  Classification  Data Entry & Collection  Tagging  Data De-duplication  Keywords  Algorithm Training  Sentiment Analysis Content & Media  Business Services  Moderate Photos & Content  Search Relevancy  Content Creation & Editing  Data Gathering  Transcription  Enhancement
  12. 12. How are others using Mechanical Turk?
  13. 13. Search EnhancementMake your assets more discoverable• Categorize them into your search tree• Add descriptions, attributes and keywords• Enhance listing dataHow it works:Item description Workers select is sent to Items are placed Content is more the best categoryMechanical Turk in that branch of easily discovered and sub-category for review by the search tree through search for the item Workers
  14. 14. Data HandlingResolve Data conflicts directly in your business process• Remove duplicates• Normalize data entries• Resolve conflicts and collisions• Enhance entriesHow it works: Business Items flagged forProcess Ingests Workers review Business conflict are sentData Elements item and resolve Process to Mechanical from multiple conflict continues Turk sources
  15. 15. How to apply your task to Mechanical Turk?
  16. 16. Your Project  Tasks“Working Backwards”• What are the business results that you want?• How will you use those results?• What format do the results need to be in?• Which tasks benefits from human intelligence?Break your overall project into smaller “tasks”• Consolidate like items and processesDefine instructions for each task
  17. 17. Example WorkflowContent Moderation with API integration
  18. 18. The Key components of your HITInstruction• Guidelines for how to do a HITData What kind of dress is this?• “batch” concept Cocktail Sun dressQuestion(s) Bridal dress
  19. 19. Work Catalog Workers discover available HITs through the listings catalog Name of Requester, Na me of HIT, Price and Qualifications are displayed
  20. 20. HIT Example Specific instructions on what to do – and how to do it Providing as much information as available
  21. 21. Adjudication StrategiesLeverage different methods of determining the correct answerby designing for accuracy in your HIT:• Multiple Choice • Plurality • Known Answers (“Gold Standards”)• Free-form • Multiple HIT Workflow’s where a new Worker “edits” or “grades” previous Worker submissions• Creative Tasks • Bonus Payments
  22. 22. Your WorkersIdentify and manage your best Workers • All available Workers All Workers • 500,000 + • 24x7, 365 • Any Worker who completes a HIT Your Workers • Evaluate performance on your HITS Your Trusted • Your best Workers with proven quality and Workers availability
  23. 23. Balances & LeversLeverage market dynamics to optimize speed, price and accuracy tomeet your business needs. Speed Price Accuracy
  24. 24. Requester access: Apps & Solution Web UI API & CLT Providers• Easy Access • Programmatic • Experts at• Create HITS in Access Workflow design HTML • Multiple SDKS • They do the• Upload and • Massive scale training and manage data via management of CSV Workers for specific kinds of tasks
  25. 25. Self Service ApplicationsMechanical Turk supports a robust ecosystem of providersthat have built or can build solutions to meet your needs.
  26. 26. Custom Solution Providers
  27. 27. Advantages of Mechanical Turk Gives you access to an on-demand workforce Eliminates 100% EfficiencyStaffing Issues Maintain the flexibility your business demands Use only the capacity you need, when you need it Lowers Converts a fixed cost into a variable cost Costs Significantly reduces ongoing headcount expenses Offers a pay-as-you-go staffing model Eliminates lead time of interviewing and staffing Reduces work Work gets done faster, by working in parallelturnaround time Work continues around the globe, around the clock
  28. 28. Thank you.John Hoskinshoskins@amazon.com206.266.8105 @amazonmturk Facebook.com/amazonmturk mechanicalturk.typepad.com

×