This document discusses the working conditions of Amazon Mechanical Turk workers and issues they face. It describes how Turkers work flexibly but often for low wages and lack protections. Turkers experience occupational hazards like unresponsive employers who don't pay wages owed, lack of oversight of unfair rejections of work, and an inability to rate or confront problematic requesters. While third-party sites have attempted to address some issues, the fundamental imbalances between Turkers and requesters remain.
Midway through a project, a client of ours recently said "One thing I'm learning is that it's ok to give up on the desktop experience once it stops making sense". This wasn't an isolated incident. In fact, i'm beginning to think desktop web sites stopped making sense quite a while ago. We've just had nothing viable to replace them with. Mobile apps have given us a glimpse, but I think they're merely a glimpse into something bigger.
Mobile isn't merely a new stage in the evolution of the web, it's not even merely a new context, it's the very early stages of an entirely new system. A system that has already started to shape our environment, affect the way we live, how we choose to connect with others, and how we're able to spend our time. A system that is also slowly unravelling our assumptions and causing us to question the very reason we build web sites, why people visit them, and where the true value of the web actually lies.
Presented by Stephanie Rieger at Breaking Development in Orlando, Florida on April 17, 2012.
Présentation Gîtes de France Landes - AG association terroir &tourisme - 8...Mediart 360
Un site internet e-commerce et e-tourisme efficace comment ça marche ?
Présentation réalisée pour l'AG de l'Association Terroir et Tourisme dans les Landes avec la Collaboration de Gîte de France.
Responsive Web Design: why is it so crucial?Merixstudio
Let’s face it, there is a new browsing trend that’s growing
exponentially and businesses must adapt.
Stop ignoring your mobile customers!
Learn about and implement Responsive Web Design
Parlons tourisme attentes et comportements des internautes en 2012Mediart 360
Parlons tourisme attentes et comportements des internautes en 2012.
Intervention à Rochefort pour la 7ème journée Parlons Tourisme. Réalisée par Patrice Foresti de Mediart 360 et Alix Howard.
Midway through a project, a client of ours recently said "One thing I'm learning is that it's ok to give up on the desktop experience once it stops making sense". This wasn't an isolated incident. In fact, i'm beginning to think desktop web sites stopped making sense quite a while ago. We've just had nothing viable to replace them with. Mobile apps have given us a glimpse, but I think they're merely a glimpse into something bigger.
Mobile isn't merely a new stage in the evolution of the web, it's not even merely a new context, it's the very early stages of an entirely new system. A system that has already started to shape our environment, affect the way we live, how we choose to connect with others, and how we're able to spend our time. A system that is also slowly unravelling our assumptions and causing us to question the very reason we build web sites, why people visit them, and where the true value of the web actually lies.
Presented by Stephanie Rieger at Breaking Development in Orlando, Florida on April 17, 2012.
Présentation Gîtes de France Landes - AG association terroir &tourisme - 8...Mediart 360
Un site internet e-commerce et e-tourisme efficace comment ça marche ?
Présentation réalisée pour l'AG de l'Association Terroir et Tourisme dans les Landes avec la Collaboration de Gîte de France.
Responsive Web Design: why is it so crucial?Merixstudio
Let’s face it, there is a new browsing trend that’s growing
exponentially and businesses must adapt.
Stop ignoring your mobile customers!
Learn about and implement Responsive Web Design
Parlons tourisme attentes et comportements des internautes en 2012Mediart 360
Parlons tourisme attentes et comportements des internautes en 2012.
Intervention à Rochefort pour la 7ème journée Parlons Tourisme. Réalisée par Patrice Foresti de Mediart 360 et Alix Howard.
Documento elaborado por Corporate Excellence - Centre for Reputation Leadership citando, entre otras fuentes, la obra Brand Psychology
escrita por Jonathan Gabay, consultor y profesor británico experto en Marca, Reputación y Comunicación, y publicada por Kogan Page en 2015.
5th International Disaster and Risk Conference IDRC 2014 Integrative Risk Management - The role of science, technology & practice 24-28 August 2014 in Davos, Switzerland
Future of work 2017 part three 29 May 2017 "Artificial Intelligence and Robo...Maureen Boland
These three lectures were written for a post graduate Australian Project Management course at Curtin University in order to give the students up to date, evidence based information to use in order to make decisions about their futures in a time of rapid change.
Crowdsourcing & ethics: a few thoughts and refences. Matthew Lease
Extracts and addendums from an earlier talk, for those interested in ethics and related issues in regard to crowdsourcing, particularly research uses. Slides updated Sept. 2, 2013.
The AIs Are Not Taking Our Jobs...They Are Changing ThemTim O'Reilly
My talk at the Web Summit in Dublin on November 6, 2014. Reflections on the notion that AI will take away jobs, and our need to recognize and redefine the human role in the applications we build. Covers many of the same ideas as my "Internet of Things and Humans" talk, but from a slightly different angle.
My talk for TechStars at Techweek Kansas City in October 2018. While this is a talk based on my book WTF?, it is fairly different from many of the others that I've posted here, in that it focuses specifically on parts of the book that contain advice for entrepreneurs, rather than on the broader questions of technology and the economy. As always, look at the speaker notes for
Instead of fretting over how easily and soon humans will be replaced, leaders would be better advised to think about the future of automation as interlacing machine strengths with those of humans. Work will need redesigning, but the AI enabled automation – done well – can unlock economic growth, fuel innovation and make work more humanA presentation given at @FutureheadsUK Leaders of Change, at CaptialOne, on 5th December 2018, by Kevin McCullagh.
Towards a New Distributional EconomicsTim O'Reilly
A talk I gave on December 1, 2017 for a workshop on AI and the future of the economy organized by the OECD and the Berkeley Roundtable on the International Economy. In it, I explore implications of AI and internet-scale platforms for the design of markets, with the goal of starting a conversation about what we might call "distributional economics."
Social Effects by the Singularity -Pre-Singularity Era-Hiroshi Nakagawa
Contents:
Stance of scientists community against Pre-Singularity problems
Amplification vs. Replacement
AI takes over jobs
Boarder line between amplification and replacement
Autonomous driver: trolley problem
The right to be forgotten
Towards black box
Responsibility
Vulnerability of financial dealing system made of many AI agent traders connected via internet
AI and weapon
Filter bubble phenomena
Analogy: Selfish gene
AI and privacy
The right to be forgotten, Profiling and Don’t Track
Feeling of friendliness to android
Again self conscious and identity
Documento elaborado por Corporate Excellence - Centre for Reputation Leadership citando, entre otras fuentes, la obra Brand Psychology
escrita por Jonathan Gabay, consultor y profesor británico experto en Marca, Reputación y Comunicación, y publicada por Kogan Page en 2015.
5th International Disaster and Risk Conference IDRC 2014 Integrative Risk Management - The role of science, technology & practice 24-28 August 2014 in Davos, Switzerland
Future of work 2017 part three 29 May 2017 "Artificial Intelligence and Robo...Maureen Boland
These three lectures were written for a post graduate Australian Project Management course at Curtin University in order to give the students up to date, evidence based information to use in order to make decisions about their futures in a time of rapid change.
Crowdsourcing & ethics: a few thoughts and refences. Matthew Lease
Extracts and addendums from an earlier talk, for those interested in ethics and related issues in regard to crowdsourcing, particularly research uses. Slides updated Sept. 2, 2013.
The AIs Are Not Taking Our Jobs...They Are Changing ThemTim O'Reilly
My talk at the Web Summit in Dublin on November 6, 2014. Reflections on the notion that AI will take away jobs, and our need to recognize and redefine the human role in the applications we build. Covers many of the same ideas as my "Internet of Things and Humans" talk, but from a slightly different angle.
My talk for TechStars at Techweek Kansas City in October 2018. While this is a talk based on my book WTF?, it is fairly different from many of the others that I've posted here, in that it focuses specifically on parts of the book that contain advice for entrepreneurs, rather than on the broader questions of technology and the economy. As always, look at the speaker notes for
Instead of fretting over how easily and soon humans will be replaced, leaders would be better advised to think about the future of automation as interlacing machine strengths with those of humans. Work will need redesigning, but the AI enabled automation – done well – can unlock economic growth, fuel innovation and make work more humanA presentation given at @FutureheadsUK Leaders of Change, at CaptialOne, on 5th December 2018, by Kevin McCullagh.
Towards a New Distributional EconomicsTim O'Reilly
A talk I gave on December 1, 2017 for a workshop on AI and the future of the economy organized by the OECD and the Berkeley Roundtable on the International Economy. In it, I explore implications of AI and internet-scale platforms for the design of markets, with the goal of starting a conversation about what we might call "distributional economics."
Social Effects by the Singularity -Pre-Singularity Era-Hiroshi Nakagawa
Contents:
Stance of scientists community against Pre-Singularity problems
Amplification vs. Replacement
AI takes over jobs
Boarder line between amplification and replacement
Autonomous driver: trolley problem
The right to be forgotten
Towards black box
Responsibility
Vulnerability of financial dealing system made of many AI agent traders connected via internet
AI and weapon
Filter bubble phenomena
Analogy: Selfish gene
AI and privacy
The right to be forgotten, Profiling and Don’t Track
Feeling of friendliness to android
Again self conscious and identity
Pragmatic Designer's Guide to Identity on the WebJamie Reffell
This talk was presented at Webvisions 2010 in Portland, Oregon.
When you're designing for the web, you have to think about identity. This includes the nuts and bolts of login fields and passwords, as well as fancy technologies like Facebook Connect, OAuth, and OpenID.
This talk presents a pragmatic approach to identity on the web, focused on best practices and a reality-based understanding of user behavior.
I'll cover:
* How users really handle accounts and passwords, and what that means for your site.
* Best practices for login/logout.
* Shared accounts, shared computers, and other messy realities.
* What designers needs to know about OpenID, OAuth, Facebook Connect, and other identity platforms.
* What might happen next: future-proofing your design without a crystal ball.
BENEFITS OF ARTIFICIAL INTELLIGENCE.pptxAkoloThomas1
Artificial Intelligence (AI) is one of the emerging technologies from the field of computer science that tries to simulate human reasoning in AI systems. John McCarthy invented the term AI in the year 1950. He said, ‘Every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions, and concepts, solve kinds of problems now reserved for humans, and improve themselves
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Monitoring Java Application Security with JDK Tools and JFR Events
Agency and Exploitation in Amazon Mechanical Turk
1. Agency and Exploitation
in Amazon Mechanical Turk
Lilly Irani, Six Silberman
Laboratory for Ubiquitous Computing and Interaction, Social Code Group
Department of Informatics
University of California, Irvine
lirani@ics.uci.edu, six@wtf.tw
2. Peeking into the Machine
and Finding it Full of People
Pic source: http://www.codinghorror.com/blog/archives/000828.html
3. Overview
•Methods of Engagement
•What is Mechanical Turk?
•Who Turks and Why
•Occupational Hazards
•Asymmetrical Accountabilities
•Unresponsive Employers
•Wage Theft
•Senses of Fairness
•Response-ability
4. Methods of Engagement
•Web forums
•Open-ended prompts posted to AMT
•HaikuTurk (haikuturk.differenceengines.com/blog)
•TurkWork (turkwork.differenceengines.com/blog)
•67 Turkers’ Worker Bill of Rights
•Developing and supporting Turkopticon
(turkopticon.differenceengines.com/blog)
•Third-party requester ratings, labor provocations
7. AMT as Flexible
Labor Form
Sources: http://www.crowdflower.com
Smith, “New Forms of Work” in Annual Review of Sociology (1997)
http://www.dol.gov/dol/topic/wages/industrialhomework.htm
Barley & Kunda, “Contracting: A New Form of Professional Practice” (2006) in Academy
of Management Perspectives
8. AMT as Flexible
Labor Form
Sources: http://www.crowdflower.com
Smith, “New Forms of Work” in Annual Review of Sociology (1997)
http://www.dol.gov/dol/topic/wages/industrialhomework.htm
Barley & Kunda, “Contracting: A New Form of Professional Practice” (2006) in Academy
of Management Perspectives
9. AMT as Creatives’
Other
“We assigned Wamique to manually
review the incoming mail... Mindless
work, really, and I felt bad about
giving it to him, but he did a great job
with it. We started calling him the
‘Human API.’”
Source: http://www.stighammond.com/watson/
2005/11/amazons_mechani.html
“For the uninitiated, Mechanical Turk
allows clients to farm out the kinds of
menial clickwork that we all wish
computers could do, but
can’t...Clickwork is what keeps the
internet economy firing with all
pistons.”
Source: Jeff Howe’s blog: http://
-- Jeff Howe of Wired www.crowdsourcing.com/cs/2008/08/
Source: http://www.crowdsourcing.com/cs/ index.html
2008/08/index.html
11. Why People Turk?
Ipeirotis Demographic Survey Responses to our prompt
“What’s it like being a Turker?”
“I have a very boring office job that
requires so little of me; without MT,
I’d be in a padded room right now”
“Sometimes interesting, usually
tough to make enough money to keep
myself afloat.”
“If I’m going to spend time doing
stuff on the computer this is waaay
more productive than just playing a
game.”
Source: Ipeirotis, Mechanical Turk: The Demographics,
http://behind-the-enemy-lines.blogspot.com/2008/03/mechanical-turk-
demographics.html “It’s great for those 5 minutes when
you need something to do.”
12. Turking for Money
The money on AMT is...
irrelevant to me
nice, but doesn’t materially
change my circumstances
a way for me to pay for nice
extras
sometimes necessary to
make basic ends meet
always necessary to make
basic ends meet
Source: Ross, Zaldivar, Irani, Tomlinson. 2009. “Who are the Turkers? Worker
Demographics in Amazon Mechanical Turk.” Technical Report SocialCode-2009-01.
http://www.ics.uci.edu/~jwross/pubs/SocialCode-2009-01.pdf
13. Making Ends Meet
"I realize I have a choice to work or not
work on AMT, but that means I would
also need to make the choice to eat or
not eat, pay bills or not pay bills, etc."
- Laura, 37-year-old mom in school while working
14. Working Flexibly
"We operate as independent contractors...Were
we employees, they would set the hours one
worked, assign HITs as they saw fit."
- Anonymous respondent to the workers’ bill of rights prompt
16. Asymmetrical
Accountabilities
“Requesters who do “We should be able to
not pay out for their comment on people
HITs within a week who have posted
should be subject to requests.”
ratings the way we
are.”
17. Unresponsive Employers
“[We should] allow
workers to confront jobs
to find out why a job is
rejected. There is email
but it is usually not
answered when
questioned. I sometimes
wonder if the larger
paying jobs ever really
plan to pay in the first
place.”
18. Wage Theft
“It’s discouraging to work hard on an article
for a mere $4.00 and then to be rejected for
some unknown reason. There are posters
who do this over and over...I had this happen
to me, and when I contacted mturk I was told
that there was nothing they could do.”
19. Senses of Fairness
“I don’t care about the penny I didn’t earn for
knowing the difference between an apple and a
giraffe, but I’m angry that MT will take requester’s
money but not manage, oversee, or mediate the
problems and injustices on their site.”
24. Third-Party Turker
Systems
Sources: http://www.crowdflower.com
Smith, “New Forms of Work” in Annual Review of Sociology (1997)
http://www.dol.gov/dol/topic/wages/industrialhomework.htm
Barley & Kunda, “Contracting: A New Form of Professional Practice” (2006) in Academy
of Management Perspectives
25. Third-Party Turker
Systems
Sources: http://www.crowdflower.com
Smith, “New Forms of Work” in Annual Review of Sociology (1997)
http://www.dol.gov/dol/topic/wages/industrialhomework.htm
Barley & Kunda, “Contracting: A New Form of Professional Practice” (2006) in Academy
of Management Perspectives
Editor's Notes
Hi, my name is Lilly Irani. I’m here today to talk about Amazon Mechanical Turk, a system that by its own billing provides “artificial artificial intelligence” by putting tens of thousands of people to work as fuzzy logic gates. In this talk, I’ll talk about what it is like to work as a person whose part-time job is to simulate a computer.
The work I’m presenting has been done in collaboration with my colleague Six Silberman from the Social Code Group at UCI.
Amazon Mechanical Turk, or AMT, is a system that enables employers, called requesters, to put up “human intelligence tasks” that people anywhere with an internet connection can assign to themselves and execute around the clock. The name comes from a wondrous 18th century machine built by Wolfgang von Kempelen. The machine was a wonder of artificial intelligence, beating people at chess across Europe. It was 50 years before it was revealed to be a hoax as inside the automaton’s box sat a human chess master, invisibly directing the supposedly machinic action.
The name of AMT and its cheeky tagline calling it “artificial artificial intelligence” is a play on this history. Like the chess-playing Turk, the laborers are kept out of sight and out of mind, contributing to the sense of excitement we see in how many high-tech people talk about AMT. If the Kempelen machine hid the human in a box, AMT hides the labor in the dematerialized cloud, offering AMT alongside its other cloud computing “web services” through which it offers disk storage and processor cycles. Some have even nicknamed AMT as the “human API,” referencing the way massive numbers of people can be called to work tagging, writing, and answering questions through server calls.
In this talk, I’ll provide a picture of what it is like to work through AMT.
Before I get into that, let me give you an overview.
Discerning figures in this cloud platform isn’t particularly easy.
Turkers, as workers sometimes call themselves, work alone where they can get some extra time near a computer, brought together only at the virtual site of exchange. You can only contact workers that you have IDs for and you only get IDs by employing people in the platform. There are also unofficial web forums but many Turkers, as workers are often called, don’t participate in them.
Six and I have been talking to Turkers through forums, through open-ended surveys placed on Turk, through more whimsical engagements like Haiku writing tasks placed on the platform, and finally by developing, soliciting feedback on, and supporting a Firefox extension that lets workers review employers.
I mentioned before that AMT is a place for people to come find microwork. The work typically pays a few cents, though some tasks take longer and might pay a couple of dollars.
AMT requesters -- officially employers, but Amazon doesn’t call them that -- place tasks on the site. Tasks include things like labeling a lamp Amazon sells with tags or rewriting a sentence in different words. Requesters receive the work done by workers and can choose to reject the work without giving a reason. If they reject the work, they don’t pay, though they still have the work in a spreadsheet they can use without restriction.
Only workers most skilled at clicking efficiently, best at seeking higher paying tasks, and most able to avoid shady requesters will make minimum wage through the platform.
Amazon’s terms and conditions carefully position it as a payments processor -- the money flows from requester through them to the worker. The worker is considered a contractor.
The site advertises...[[hit button]]
AMT does not stand alone as a technology. The ideas sold as the “Crowdsourcing advantage” are actually an intensification of outsourcing discourse that can be traced back to 1980s arguments for outsourcing as a means of enhancing a firm’s flexibility. The ability to hire and fire at will allows firms to adapt their human makeup quickly. It also relieves managers of tasks like worrying about “people issues” and worker morale.
Two-tiered system in new forms of work where lower tier gets paid less, less stability, and less benefits.
1/3 to 1/4 of US was temping back in 1997.
Goes back even further...
- pre-industrial revolution turn in the return to the home. piecework still happens, of course, and in the US, it is officially called industrial homework. In the US, laws require that regardless of the piece rate and productivity, industrial homeworkers have to at least earn minimum wage for their time.
- Unlike industrial homework, however, AMT is under the same contractor model that applies to people like design freelancers, though Turkers’ wages are radically different than such professionalized contractors that the law usually applies to.
The two tiered flexibility I just talked about often take the form of one highly valued flexible labor strata and a lower strata that supports the upper strata by doing labor characterized as menial, repetitive, or mindless (perhaps incorrectly). It’s great lubrication for the knowledge industry but it works by maximizing competition and depressing income and benefits among workers.
The flexible labor that gets outsourced to AMT is partly about financial and temporal efficiency, but it also seems to have an ideological purpose for high tech workers. New media workers talking about turk on blogs often describe AMT a place to send menial tasks that are necessary for their projects but that they find it hard to wish on other people.
READ QUOTES, EXPLAIN GRAPHIC
In these posts, “menial work” is posed as the Other of this creative class that does the opposite of menial and mindless labor.
For many, entertainment, a substitute for games. They get paid, however little, instead of paying to play or getting paid nothing. For others, its a way to fill spare moments and keep themselves mentally engaged as a hobby.
But some really needed the money.
18% is non-trivial.
Economic meltdown
Woman working and going to school, unable to get enough hours on her job. Gas prices were high. Turk let her work at night for 2-3 $/hr and didn’t cost her gas.
Others talked about having lost jobs and doing this while looking for something else.
A few we encountered were quite assertive about their independence on turk. But the very flexibility that this person valorizes is the flexibility that allows Laura from the prev slide to work the third shift on Turk to make ends meet.
workers bill of rights
By accountability, I mean having an aspect of a Turker or employer made visible to others.
AMT gives workers a qual rating based on rejection rate. Employers can choose to only employ people who have a certain rating.
Many workers were frustrated that employers were not similarly rated. Some, for ex, thought workers should be able to see the employer’s rejection rate so they can avoid ones that seem to reject work.
employers who did not respond to worker questions, complains, or claims of unfairness. A fifth of bill of rights explicitly mentioned responsivity as an issue on turk.
The very design of turk -- the massive scale -- makes being responsive and communicative really difficult. Responding a worker requires either a lot of clicks or API hacking. Some employers have set up websites through tools like Get Satisfaction as a way of dealing with this but those responsive employers are rare. For most, it is precisely the low management cost of having 1000s of people quickly tag your images that makes AMT attractive. Responding to employees increases the employer’s costs.
The combination of employers who have no reputation to protect -- no accountabilities -- and unresponsive employers gives many workers the sense that their wages are unfairly denied them. Some even identified a recurring pattern of employers offering high wages for a large task as a way of attracting workers, but taking the work and refusing to pay.
Workers frequently also mentioned Amazon as someone they expected to arbitrate such disputes and expressed frustration that Amazon did nothing to protect them against unscrupulous employers. Governance on AMT seems thus far to favor the interests of employers and many of our worker respondents had noticed that.
Asymmetrical accountabilities, unresponsive employers, and wage theft contributed to a broader feeling of unfairness described by many of our respondents.
In many cases, it isn’t even the wages but an aesthetic of fair relations that seems to be violated. Only pennies in some cases, workers still rage at the indignity of being ignored and feeling powerless in the face of employers who do not reward them as accepted.
Continuities of work practices
Complexities of objectifying oneself
Response-ability
I’ve thought about whether AMT is exploitative through a number of lenses: the low wages, the flexible work configurations, the fact that workers have no intellectual property rights over the writing they produce. But when I listen to the people who make the machinic Turk, their sense of exploitation and unfair treatment seems to stem from their objectification. Workers come to Turk willingly, whether for fun or because they are need, but they enter into relations in which the employer does not respond and cannot be held accountable. The worker is a silent producer of value and remains so even when they seek to address their employer.
This take locates exploitation not in labor becoming objectified or commodified, but instead in what happens when those objectifications become permanent. Feminist scholars describe how women, for example, voluntarily objectify themselves to, say, get fertility therapy but aren’t permanently then subject to the doctor’s will and knowledge. As Lucy Suchman explains, “alienation is not located in objectification per se” but instead in the breakdown in which parts or reductions come to stand in for the whole. Under this reasoning, we might not want to say people should never participate in the highly abstracted world of work that is AMT. But we’d argue that people should not be irreversibly reduced to an efficient work unit. They should be able to speak, be listened to, and demand action when there is a problem.
This is an approach that comes out of the sensibility that old alignments of capital and labor, male and female, or even human and animal can no longer be taken for granted. We see too many of the ways individuals aren’t sortable into those categories. Some of the people who Turk because they need the money are the same ones who buy cheap sweatshop clothes. Who is not exploited in some ways? As Donna Haraway argues, “try as we might to distance ourselves, there is no way of living that is not also a way of someone else dying differentially” (80). If there’s no class of workers that deserves universal rights in relation to a universally exploitative employer, then maybe we need an approach to exploitation that is more case-by-case, more founded on workers speaking and being responded to.
In conclusion, Mechanical Turk is a complex site to think about. It brings together leisure, addictive play, making ends meet, and laboring for livelihood together into a single site. My hope is thinking through agencies and occupational hazards of Mechanical Turk, we see some of the complexities of identifying a politics of digital labor on the internet. This is offered not as a conclusion, but a jumping off point.
Is there any escape from this exploitation? Relationships between people are always shaped by power one way or another -- there's no escaping that. There isn't some total freedom from being enmeshed in power relations in others. However, it's worth recognizing the repeated patterns of exploitation that are happening, watching for whether they become more severe, and taking each tentative step in a direction that tries to move away from those exploitative conditions. Watch, witness, push back, stay vigilant.
While there may be no possibility of getting to a perfect, unexploitative place, believing in something better than an exploitative cop-out is vital. “Utopia has critical power,” Derrida wrote, recognizing while it was unattainable it haunted consciences urged us to strive.
To those who demand concrete alternatives to the current state of things -- prescriptions for action as justification of a critique -- Marcuse argued in his "Essay on Liberation" that the institutions have to be built up through trial and error because the alternative is "sufficiently 'abstract' -- the alternative is removed from and incongruous with the established universe to defy any attempt to identify them in terms of this universe.” In other words, I don’t know the utopia I’m directing myself towards but the haunting of exploitation might lead us, step by tentative step, to a different place.
AMT does not stand alone as a platform. First, an ecosystem of services have sprung up to support employers wishing to get work done through the platform. CrowdFlower, for example, will keep track of which workers are reliable and deny work to those who are not.
The AMT ecosystem also is not completely new. The ideas sold as the “Crowdsourcing advantage” are actually an intensification of outsourcing discourse that can be traced back to 1980s arguments for outsourcing as a means of enhancing a firm’s flexibility. The ability to hire and fire at will allows firms to adapt their human makeup quickly. It also relieves managers of tasks like worrying about “people issues” and worker morale.
1/3 to 1/4 of US was temping back in 1997.
Two-tiered system in new forms of work where lower tier gets paid less, less stability, and less benefits.
Goes back even further...
- pre-industrial revolution turn in the return to the home. piecework still happens, of course, and in the US, it is officially called industrial homework. In the US, laws require that regardless of the piece rate and productivity, industrial homeworkers have to at least earn minimum wage for their time.
- Unlike industrial homework, however, AMT is under the same contractor model that applies to people like design freelancers, though Turkers’ wages are radically different than such professionalized contractors that the law usually applies to.
The two tiered flexibility I just talked about often take the form of a highly valued flexible labor strata that increasingly relies on labor that is characterized menial, repetitive, or mindless (perhaps incorrectly).
This is the flexibility that allows young people who don’t need health insurance to make agile startups in their garages. The flexibility is built on an on call, just-in-time, no retainer workforce. It’s great lubrication for the knowledge industry but it works by maximizing competition and depressing income and benefits among workers.