Agency and Exploitation in Amazon Mechanical Turk
Report
Lilly IraniFollow
Nov. 10, 2009•0 likes•2,059 views
Agency and Exploitation in Amazon Mechanical Turk
Nov. 10, 2009•0 likes•2,059 views
Report
Technology
Business
Lilly IraniFollow
More Related Content
Viewers also liked
Bbbee esd and sd presentationMotlalepule "Mapule Mahlulo
ISEM Fashion Business School - Reputación Digital - Francesc PujolMedia, Reputation & Intangibles center - Universidad de Navarra
Chapter 13: Regulatory Compliance for Financial InstitutionsNada G.Youssef
Las razones y los sentimientos que rigen la mente de los stakeholders impacta...Corporate Excellence - Centre for Reputation Leadership
TBWA\Corporate : 2ème baromètre TBWA\Corporate de l’activité parlementaireTBWA\Corporate
Miratech presentation M&A in IT sector in Ukraine Lviv Startup Club
Similar to Agency and Exploitation in Amazon Mechanical Turk
Future of work 2017 part three 29 May 2017 "Artificial Intelligence and Robo...Maureen Boland
Crowdsourcing & ethics: a few thoughts and refences. Matthew Lease
Software Above the Level of a Single Device: The Implications-(Tim O'Reilly, ...Spark Summit
The AIs Are Not Taking Our Jobs...They Are Changing ThemTim O'Reilly
Neha gupta why turk_brightonseoNeha Gupta
Networks and the Next EconomyTim O'Reilly
Recently uploaded
Elevate Your Enterprise with FME 23.1Safe Software
Property Graphs in APEX.pptxssuser923120
Orchestration, Automation and Virtualisation Maturity ModelCSUC - Consorci de Serveis Universitaris de Catalunya
Microsoft Azure New - Sep 2023Daniel Toomey
Easy Salesforce CI/CD with Open Source Only - Dreamforce 23NicolasVuillamy1
Improving Employee Experiences on Cisco RoomOS Devices, Webex, and Microsoft ...ThousandEyes
Agency and Exploitation in Amazon Mechanical Turk
- 1. Agency and Exploitation in Amazon Mechanical Turk Lilly Irani, Six Silberman Laboratory for Ubiquitous Computing and Interaction, Social Code Group Department of Informatics University of California, Irvine lirani@ics.uci.edu, six@wtf.tw
- 2. Peeking into the Machine and Finding it Full of People Pic source: http://www.codinghorror.com/blog/archives/000828.html
- 3. Overview •Methods of Engagement •What is Mechanical Turk? •Who Turks and Why •Occupational Hazards •Asymmetrical Accountabilities •Unresponsive Employers •Wage Theft •Senses of Fairness •Response-ability
- 4. Methods of Engagement •Web forums •Open-ended prompts posted to AMT •HaikuTurk (haikuturk.differenceengines.com/blog) •TurkWork (turkwork.differenceengines.com/blog) •67 Turkers’ Worker Bill of Rights •Developing and supporting Turkopticon (turkopticon.differenceengines.com/blog) •Third-party requester ratings, labor provocations
- 5. What is Amazon Mechanical Turk?
- 6. AMT as Platform Source: http://www.mturk.com
- 7. AMT as Flexible Labor Form Sources: http://www.crowdflower.com Smith, “New Forms of Work” in Annual Review of Sociology (1997) http://www.dol.gov/dol/topic/wages/industrialhomework.htm Barley & Kunda, “Contracting: A New Form of Professional Practice” (2006) in Academy of Management Perspectives
- 8. AMT as Flexible Labor Form Sources: http://www.crowdflower.com Smith, “New Forms of Work” in Annual Review of Sociology (1997) http://www.dol.gov/dol/topic/wages/industrialhomework.htm Barley & Kunda, “Contracting: A New Form of Professional Practice” (2006) in Academy of Management Perspectives
- 9. AMT as Creatives’ Other “We assigned Wamique to manually review the incoming mail... Mindless work, really, and I felt bad about giving it to him, but he did a great job with it. We started calling him the ‘Human API.’” Source: http://www.stighammond.com/watson/ 2005/11/amazons_mechani.html “For the uninitiated, Mechanical Turk allows clients to farm out the kinds of menial clickwork that we all wish computers could do, but can’t...Clickwork is what keeps the internet economy firing with all pistons.” Source: Jeff Howe’s blog: http:// -- Jeff Howe of Wired www.crowdsourcing.com/cs/2008/08/ Source: http://www.crowdsourcing.com/cs/ index.html 2008/08/index.html
- 10. Who Turks and Why?
- 11. Why People Turk? Ipeirotis Demographic Survey Responses to our prompt “What’s it like being a Turker?” “I have a very boring office job that requires so little of me; without MT, I’d be in a padded room right now” “Sometimes interesting, usually tough to make enough money to keep myself afloat.” “If I’m going to spend time doing stuff on the computer this is waaay more productive than just playing a game.” Source: Ipeirotis, Mechanical Turk: The Demographics, http://behind-the-enemy-lines.blogspot.com/2008/03/mechanical-turk- demographics.html “It’s great for those 5 minutes when you need something to do.”
- 12. Turking for Money The money on AMT is... irrelevant to me nice, but doesn’t materially change my circumstances a way for me to pay for nice extras sometimes necessary to make basic ends meet always necessary to make basic ends meet Source: Ross, Zaldivar, Irani, Tomlinson. 2009. “Who are the Turkers? Worker Demographics in Amazon Mechanical Turk.” Technical Report SocialCode-2009-01. http://www.ics.uci.edu/~jwross/pubs/SocialCode-2009-01.pdf
- 13. Making Ends Meet "I realize I have a choice to work or not work on AMT, but that means I would also need to make the choice to eat or not eat, pay bills or not pay bills, etc." - Laura, 37-year-old mom in school while working
- 14. Working Flexibly "We operate as independent contractors...Were we employees, they would set the hours one worked, assign HITs as they saw fit." - Anonymous respondent to the workers’ bill of rights prompt
- 15. Occupational Hazards
- 16. Asymmetrical Accountabilities “Requesters who do “We should be able to not pay out for their comment on people HITs within a week who have posted should be subject to requests.” ratings the way we are.”
- 17. Unresponsive Employers “[We should] allow workers to confront jobs to find out why a job is rejected. There is email but it is usually not answered when questioned. I sometimes wonder if the larger paying jobs ever really plan to pay in the first place.”
- 18. Wage Theft “It’s discouraging to work hard on an article for a mere $4.00 and then to be rejected for some unknown reason. There are posters who do this over and over...I had this happen to me, and when I contacted mturk I was told that there was nothing they could do.”
- 19. Senses of Fairness “I don’t care about the penny I didn’t earn for knowing the difference between an apple and a giraffe, but I’m angry that MT will take requester’s money but not manage, oversee, or mediate the problems and injustices on their site.”
- 20. Response-ability Source: Haraway. When Species Meet. 2008. Suchman. “Agencies in Technology Design: Feminist Reconfigurations.”
- 21. Thank you Lilly Irani, Six Silberman lirani@ics.uci.edu, six@wtf.tw http://turkopticon.differenceengines.com Creative commons: flickr Tobias Higbie
- 22. APPENDIX
- 23. Turkopticon: Band-Aid and Provocation
- 24. Third-Party Turker Systems Sources: http://www.crowdflower.com Smith, “New Forms of Work” in Annual Review of Sociology (1997) http://www.dol.gov/dol/topic/wages/industrialhomework.htm Barley & Kunda, “Contracting: A New Form of Professional Practice” (2006) in Academy of Management Perspectives
- 25. Third-Party Turker Systems Sources: http://www.crowdflower.com Smith, “New Forms of Work” in Annual Review of Sociology (1997) http://www.dol.gov/dol/topic/wages/industrialhomework.htm Barley & Kunda, “Contracting: A New Form of Professional Practice” (2006) in Academy of Management Perspectives
Editor's Notes
- Hi, my name is Lilly Irani. I’m here today to talk about Amazon Mechanical Turk, a system that by its own billing provides “artificial artificial intelligence” by putting tens of thousands of people to work as fuzzy logic gates. In this talk, I’ll talk about what it is like to work as a person whose part-time job is to simulate a computer. The work I’m presenting has been done in collaboration with my colleague Six Silberman from the Social Code Group at UCI.
- Amazon Mechanical Turk, or AMT, is a system that enables employers, called requesters, to put up “human intelligence tasks” that people anywhere with an internet connection can assign to themselves and execute around the clock. The name comes from a wondrous 18th century machine built by Wolfgang von Kempelen. The machine was a wonder of artificial intelligence, beating people at chess across Europe. It was 50 years before it was revealed to be a hoax as inside the automaton’s box sat a human chess master, invisibly directing the supposedly machinic action. The name of AMT and its cheeky tagline calling it “artificial artificial intelligence” is a play on this history. Like the chess-playing Turk, the laborers are kept out of sight and out of mind, contributing to the sense of excitement we see in how many high-tech people talk about AMT. If the Kempelen machine hid the human in a box, AMT hides the labor in the dematerialized cloud, offering AMT alongside its other cloud computing “web services” through which it offers disk storage and processor cycles. Some have even nicknamed AMT as the “human API,” referencing the way massive numbers of people can be called to work tagging, writing, and answering questions through server calls. In this talk, I’ll provide a picture of what it is like to work through AMT.
- Before I get into that, let me give you an overview.
- Discerning figures in this cloud platform isn’t particularly easy. Turkers, as workers sometimes call themselves, work alone where they can get some extra time near a computer, brought together only at the virtual site of exchange. You can only contact workers that you have IDs for and you only get IDs by employing people in the platform. There are also unofficial web forums but many Turkers, as workers are often called, don’t participate in them. Six and I have been talking to Turkers through forums, through open-ended surveys placed on Turk, through more whimsical engagements like Haiku writing tasks placed on the platform, and finally by developing, soliciting feedback on, and supporting a Firefox extension that lets workers review employers.
- I mentioned before that AMT is a place for people to come find microwork. The work typically pays a few cents, though some tasks take longer and might pay a couple of dollars. AMT requesters -- officially employers, but Amazon doesn’t call them that -- place tasks on the site. Tasks include things like labeling a lamp Amazon sells with tags or rewriting a sentence in different words. Requesters receive the work done by workers and can choose to reject the work without giving a reason. If they reject the work, they don’t pay, though they still have the work in a spreadsheet they can use without restriction. Only workers most skilled at clicking efficiently, best at seeking higher paying tasks, and most able to avoid shady requesters will make minimum wage through the platform. Amazon’s terms and conditions carefully position it as a payments processor -- the money flows from requester through them to the worker. The worker is considered a contractor. The site advertises...[[hit button]]
- AMT does not stand alone as a technology. The ideas sold as the “Crowdsourcing advantage” are actually an intensification of outsourcing discourse that can be traced back to 1980s arguments for outsourcing as a means of enhancing a firm’s flexibility. The ability to hire and fire at will allows firms to adapt their human makeup quickly. It also relieves managers of tasks like worrying about “people issues” and worker morale. Two-tiered system in new forms of work where lower tier gets paid less, less stability, and less benefits. 1/3 to 1/4 of US was temping back in 1997. Goes back even further... - pre-industrial revolution turn in the return to the home. piecework still happens, of course, and in the US, it is officially called industrial homework. In the US, laws require that regardless of the piece rate and productivity, industrial homeworkers have to at least earn minimum wage for their time. - Unlike industrial homework, however, AMT is under the same contractor model that applies to people like design freelancers, though Turkers’ wages are radically different than such professionalized contractors that the law usually applies to. The two tiered flexibility I just talked about often take the form of one highly valued flexible labor strata and a lower strata that supports the upper strata by doing labor characterized as menial, repetitive, or mindless (perhaps incorrectly). It’s great lubrication for the knowledge industry but it works by maximizing competition and depressing income and benefits among workers.
- The flexible labor that gets outsourced to AMT is partly about financial and temporal efficiency, but it also seems to have an ideological purpose for high tech workers. New media workers talking about turk on blogs often describe AMT a place to send menial tasks that are necessary for their projects but that they find it hard to wish on other people. READ QUOTES, EXPLAIN GRAPHIC In these posts, “menial work” is posed as the Other of this creative class that does the opposite of menial and mindless labor.
- For many, entertainment, a substitute for games. They get paid, however little, instead of paying to play or getting paid nothing. For others, its a way to fill spare moments and keep themselves mentally engaged as a hobby. But some really needed the money.
- 18% is non-trivial.
- Economic meltdown Woman working and going to school, unable to get enough hours on her job. Gas prices were high. Turk let her work at night for 2-3 $/hr and didn’t cost her gas. Others talked about having lost jobs and doing this while looking for something else.
- A few we encountered were quite assertive about their independence on turk. But the very flexibility that this person valorizes is the flexibility that allows Laura from the prev slide to work the third shift on Turk to make ends meet.
- workers bill of rights
- By accountability, I mean having an aspect of a Turker or employer made visible to others. AMT gives workers a qual rating based on rejection rate. Employers can choose to only employ people who have a certain rating. Many workers were frustrated that employers were not similarly rated. Some, for ex, thought workers should be able to see the employer’s rejection rate so they can avoid ones that seem to reject work.
- employers who did not respond to worker questions, complains, or claims of unfairness. A fifth of bill of rights explicitly mentioned responsivity as an issue on turk. The very design of turk -- the massive scale -- makes being responsive and communicative really difficult. Responding a worker requires either a lot of clicks or API hacking. Some employers have set up websites through tools like Get Satisfaction as a way of dealing with this but those responsive employers are rare. For most, it is precisely the low management cost of having 1000s of people quickly tag your images that makes AMT attractive. Responding to employees increases the employer’s costs.
- The combination of employers who have no reputation to protect -- no accountabilities -- and unresponsive employers gives many workers the sense that their wages are unfairly denied them. Some even identified a recurring pattern of employers offering high wages for a large task as a way of attracting workers, but taking the work and refusing to pay. Workers frequently also mentioned Amazon as someone they expected to arbitrate such disputes and expressed frustration that Amazon did nothing to protect them against unscrupulous employers. Governance on AMT seems thus far to favor the interests of employers and many of our worker respondents had noticed that.
- Asymmetrical accountabilities, unresponsive employers, and wage theft contributed to a broader feeling of unfairness described by many of our respondents. In many cases, it isn’t even the wages but an aesthetic of fair relations that seems to be violated. Only pennies in some cases, workers still rage at the indignity of being ignored and feeling powerless in the face of employers who do not reward them as accepted.
- Continuities of work practices Complexities of objectifying oneself Response-ability I’ve thought about whether AMT is exploitative through a number of lenses: the low wages, the flexible work configurations, the fact that workers have no intellectual property rights over the writing they produce. But when I listen to the people who make the machinic Turk, their sense of exploitation and unfair treatment seems to stem from their objectification. Workers come to Turk willingly, whether for fun or because they are need, but they enter into relations in which the employer does not respond and cannot be held accountable. The worker is a silent producer of value and remains so even when they seek to address their employer. This take locates exploitation not in labor becoming objectified or commodified, but instead in what happens when those objectifications become permanent. Feminist scholars describe how women, for example, voluntarily objectify themselves to, say, get fertility therapy but aren’t permanently then subject to the doctor’s will and knowledge. As Lucy Suchman explains, “alienation is not located in objectification per se” but instead in the breakdown in which parts or reductions come to stand in for the whole. Under this reasoning, we might not want to say people should never participate in the highly abstracted world of work that is AMT. But we’d argue that people should not be irreversibly reduced to an efficient work unit. They should be able to speak, be listened to, and demand action when there is a problem. This is an approach that comes out of the sensibility that old alignments of capital and labor, male and female, or even human and animal can no longer be taken for granted. We see too many of the ways individuals aren’t sortable into those categories. Some of the people who Turk because they need the money are the same ones who buy cheap sweatshop clothes. Who is not exploited in some ways? As Donna Haraway argues, “try as we might to distance ourselves, there is no way of living that is not also a way of someone else dying differentially” (80). If there’s no class of workers that deserves universal rights in relation to a universally exploitative employer, then maybe we need an approach to exploitation that is more case-by-case, more founded on workers speaking and being responded to. In conclusion, Mechanical Turk is a complex site to think about. It brings together leisure, addictive play, making ends meet, and laboring for livelihood together into a single site. My hope is thinking through agencies and occupational hazards of Mechanical Turk, we see some of the complexities of identifying a politics of digital labor on the internet. This is offered not as a conclusion, but a jumping off point. Is there any escape from this exploitation? Relationships between people are always shaped by power one way or another -- there's no escaping that. There isn't some total freedom from being enmeshed in power relations in others. However, it's worth recognizing the repeated patterns of exploitation that are happening, watching for whether they become more severe, and taking each tentative step in a direction that tries to move away from those exploitative conditions. Watch, witness, push back, stay vigilant. While there may be no possibility of getting to a perfect, unexploitative place, believing in something better than an exploitative cop-out is vital. “Utopia has critical power,” Derrida wrote, recognizing while it was unattainable it haunted consciences urged us to strive. To those who demand concrete alternatives to the current state of things -- prescriptions for action as justification of a critique -- Marcuse argued in his "Essay on Liberation" that the institutions have to be built up through trial and error because the alternative is "sufficiently 'abstract' -- the alternative is removed from and incongruous with the established universe to defy any attempt to identify them in terms of this universe.” In other words, I don’t know the utopia I’m directing myself towards but the haunting of exploitation might lead us, step by tentative step, to a different place.
- AMT does not stand alone as a platform. First, an ecosystem of services have sprung up to support employers wishing to get work done through the platform. CrowdFlower, for example, will keep track of which workers are reliable and deny work to those who are not. The AMT ecosystem also is not completely new. The ideas sold as the “Crowdsourcing advantage” are actually an intensification of outsourcing discourse that can be traced back to 1980s arguments for outsourcing as a means of enhancing a firm’s flexibility. The ability to hire and fire at will allows firms to adapt their human makeup quickly. It also relieves managers of tasks like worrying about “people issues” and worker morale. 1/3 to 1/4 of US was temping back in 1997. Two-tiered system in new forms of work where lower tier gets paid less, less stability, and less benefits. Goes back even further... - pre-industrial revolution turn in the return to the home. piecework still happens, of course, and in the US, it is officially called industrial homework. In the US, laws require that regardless of the piece rate and productivity, industrial homeworkers have to at least earn minimum wage for their time. - Unlike industrial homework, however, AMT is under the same contractor model that applies to people like design freelancers, though Turkers’ wages are radically different than such professionalized contractors that the law usually applies to. The two tiered flexibility I just talked about often take the form of a highly valued flexible labor strata that increasingly relies on labor that is characterized menial, repetitive, or mindless (perhaps incorrectly). This is the flexibility that allows young people who don’t need health insurance to make agile startups in their garages. The flexibility is built on an on call, just-in-time, no retainer workforce. It’s great lubrication for the knowledge industry but it works by maximizing competition and depressing income and benefits among workers.