Amazon Mechanical Turk: Intro & Overview
•
3 likes•1,068 views
This document provides an overview and introduction to a workshop on Amazon Mechanical Turk. It discusses crowdsourcing and how Mechanical Turk can be used for microtasking. Common use cases for Mechanical Turk include data management, annotation, content moderation, and analysis. Best practices for using Mechanical Turk such as ensuring worker accuracy, payment strategies, and continuous optimization are also covered. The workshop introduction outlines a process for planning, implementing, and improving projects on Mechanical Turk.
![2
Before we start
][
2
amazon
web services
Daniel Gray, Principal Business Development
Amazon Mechanical Turk, Seattle, WA
djgray@amazon.com
T: 310-800-4832](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (12)
Similar to Amazon Mechanical Turk: Intro & Overview
Similar to Amazon Mechanical Turk: Intro & Overview (20)
Recently uploaded
Recently uploaded (20)
Amazon Mechanical Turk: Intro & Overview
- 1. Overview & Workshop Intro
- 2. 2 Before we start ][ 2 amazon web services Daniel Gray, Principal Business Development Amazon Mechanical Turk, Seattle, WA djgray@amazon.com T: 310-800-4832
- 3. 3 Who is Amazon.com? ][ 3
- 4. 44 ][Amazon.com’s three businesses Consumer Business Tens of millions of active customer accounts Eight countries: US, UK, Germany, Japan, France, Canada, China, Italy Seller Business Sell on Amazon websites Use Amazon technology for your own retail website Leverage Amazon’s massive fulfillment center network IT Infrastructure Business Cloud computing infrastructure for hosting web-scale solutions Hundreds of thousands of registered customers in over 190 countries Marketplace for Work amazon web services
- 5. 5 Our plan for today ][ 5 Section I: Overview 1) Level-setting on ‘crowdsourcing’ 2) What is Amazon Mechanical Turk Section II: Workshop Intro 1) Best Practices 2) Workshop
- 6. 66 More Technical More Business What is Amazon Mechanical Turk? Workshop IntroCrowdsourcing Best Practices How technical is it?[ ]
- 7. 77 1. Crowdsourcing More Technical More Business Crowdsourcing What is Amazon Mechanical Turk? Workshop Intro Best Practices
- 9. 99 Microtasking is…distributed problem solving.
- 10. 10 Common Use Cases[ ] 10 Data Management _ Verification _ Entry & Collection _ De-dupe _ Algorithm Training Annotation _ Tagging _ Classification Content Management _ Moderation (Photos, Content) _ Transcription _ Writing/Editing Analysis/Research _ Sentiment _ Relevance _ Online Research
- 11. 1111 2. Amazon Mechanical Turk More Technical More Business Crowdsourcing What is Amazon Mechanical Turk? Workshop Intro Best Practices
- 12. 1212 [ Mechanical Turk Ecosystem] • Manual via WebUI • Automate via API How to Connect • Self-serve • Assisted i. Partner network ii. Workshops How to Engage • Documentation • Best Practices • Developer’s Resources • Requestor UI Guide (account setup) • Sandbox How to get Support
- 13. 1313 $ Variable expense Transform fixed human capital costs into variable expense $ Economies of scale A lower variable expense than companies can achieve themselves Speed and agility On-demand access to scalable workforce & higher productivity Focus on business Redeploy valuable human resources to core business Global Reach 500K Workers across 190 countriesScalability No need to guess capacity requirements = avoid waste Fast Growth Predictable peaks On and Off Variable peaks Results’ Accuracy Drive fast, accurate work results at competitive prices without long-term contracts or minimum commitments
- 14. 1414 3. Best Practices More Technical More Business Crowdsourcing What is Amazon Mechanical Turk? Workshop Intro Best Practices
- 15. 1515 [ Best Practices ] 1. Worker Accuracy • Blocking / rejecting • Qualification • Curating the workforce • Masters • Plurality • Known answers • Adjudication 2. Worker ergonomics 3. Marketplace reputation 4. Payment & bonusing • Pay-work equalization • Timing • Bundling 5. Optimization
- 16. 1616 4. Workshop Intro More Technical More Business Crowdsourcing What is Amazon Mechanical Turk? Workshop Intro Best Practices
- 17. 1717 [ Workshop Intro] • Strategy • Business issues & objectives: Establish priorities & goals for the core metrics (input & output) for measurement and optimization. • Planning & Design • Establish blueprints for pilot, with scalability in mind, including: • Visioning – Assessment of current workflow(s) and success metrics; disintegrating the stages into Mechanical Turk workflow steps and ensuring they’re optimized for microtasking; integration with and development of Mechanical Turk workflow(s). • Scoping & Planning – API-connectivity, address HIT design, Worker instructions and layout, adjudication strategies, Known Answers, Masters, workflow and best practices implementation. • Implementation & Launch • Implement the best practices and designs established in the planning phase; as Advisors, we help to refine implementation according to the design. • Continuous Improvement • Adjust & optimize the prioritized metrics based on BI reporting; observing results, measuring against goals, root causing gaps, and adjusting: • Cost efficiency • Results’ accuracy • Productivity
- 18. 18 Thank you ][ 18 Daniel Gray, Principal Business Development Amazon Mechanical Turk, Seattle, WA djgray@amazon.com T: 310-800-4832
Editor's Notes
- To understand Amazon Mechanical Turk, firstit’s helpful to understand Amazon is a company of marketplaces - we have three macro and distinct businesses:1. our Consumer/Retail business (with tens of millions of active customers around the world)2. our Seller business (where third parties sell on Amazon websites, using Amazon technology for their own retail websites, leveraging Amazon’s massive fulfillment center network)3. our IT Infrastructure business (where cloud computing infrastructure for hosting web-scale solutions serves hundreds of thousands of registered customers around the world).Amazon Mechanical Turk is a marketplace for work, and is part of Amazon Web Services (AWS). Like our cloud computing infrastructure which offers computing power, Amazon Mechanical Turk is the infrastructure to access a global, scalable, on-demand workforce for human power.
- Data management: This includes data or algorithm training…With Mechanical Turk you can quickly develop large sets of training data to train your algorithmsAnalysis/research: perform sentiment analysis, monitor customer feedbackContent management: moderate your collections for appropriateness/accuracy to market, improve search engine-level discoverability by adding descriptions to your collections in addition to tags/keywordsAnnotation: tag collections with keywords, improving classification/categorization. Again, discoverability is critical to image sales - Mechanical Turk Workers can quickly and accurately add attribute descriptions (or tags) to all types of content so that it can be easily indexed and searched. Example Tasks: Generate keywords for an image, advertisement, or website. Identify objects found in an image for easier searching and advertising targeting. Timestamp and tag sections of a video or audio clip based on content.
- _ Mechanical Turk is a marketplace for work – think of it as the infrastructure layer upon which Partners like Tagasauris offer a technology solution to Requesters (those looking for workers) to access a scalable, global, on-demand Workforce _ Mechanical Turk provides access to the tool required to build workflows and control quality_ Mechanical Turk provides the payment administration (though Amazon Payments)_ …and lastly, it can handle millions of tasks, with web or API-connectivity options_The process includes breaking down a project into tasks, publishing the tasks via HIT’s (human intelligence tasks). Depending on the qualifications set for your tasks, either a defined or undefined group of Workers accept the assignments, submit their results for approval, and the process continues.What are the Key advantages that deliver business benefits: 1. VARIABLE EXPENSE: Lower Costs_ Transform fixed [human capital] costs into variable costs_ Offers a pay-as-you-go model2. GAINECONOMIES OF SCALE: as your collections continue to grow, scale more efficiently3. SPEED AND AGILITY_ On-demand (vs. on the bench) access to a scalable workforce_ Results-focus (vs. time focus) emphasizes productivity; Workers can work in parallel, around the globe, around the clock4. FOCUS ON YOUR BUSINESSWith increased efficiency, productivity, and lower costs—you have more time to focus on critical business objectives, increasing the pace of your innovation._ Experiment Often_ Fail quickly at a low cost= More Innovation5. GLOBAL REACH: 500k Workers across 190 countries6. ACCURACY: drive fast, accurate work results at competitive prices without long term contracts or minimum commitments7. SCALABILITY: avoid waste_ Burst up and down according to the capacity you need, when you need it_ 100% Efficiency_ Independent of your workload type, maintain the flexibility your business demandsThis is the ‘business’ buzz around Mechanical Turk It’s these business benefits that make Mechanical Turk a strategic initiative for companies like Comscore, Twitter, Linkedin…and the difference between the “disruptors” vs. the “disrupted.”