• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Crowdsourcing & ethics: a few thoughts and refences.
 

Crowdsourcing & ethics: a few thoughts and refences.

on

  • 963 views

Extracts and addendums from an earlier talk, for those interested in ethics and related issues in regard to crowdsourcing, particularly research uses. Slides updated Sept. 2, 2013.

Extracts and addendums from an earlier talk, for those interested in ethics and related issues in regard to crowdsourcing, particularly research uses. Slides updated Sept. 2, 2013.

Statistics

Views

Total Views
963
Views on SlideShare
963
Embed Views
0

Actions

Likes
1
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Crowdsourcing & ethics: a few thoughts and refences. Crowdsourcing & ethics: a few thoughts and refences. Presentation Transcript

    • Extracts and Addendums: Ethics & Crowd Computing Matt Lease School of Information @mattlease University of Texas at Austin ml@ischool.utexas.edu
    • What about context? By minimizing context, greater task efficiency can often be achieved – e.g. “Can you name who is in this photo?” 2
    • Context & Informed Consent • But people need context for informed consent • Who will benefit from it and how? Jonathan Zittrain, Harvard 3
    • What about requestor fraud? • Refusing to pay for work, installing malware • Using crowds to defeat online metrics/security 4
    • Robert Sim, MSR Summit’12 5
    • What about regulation? • Wolfson and Lease. Look before you leap: Legal pitfalls of crowdsourcing. Proceedings of the American Society for Information Science and Technology (ASIS&T), 2011. • As usual, technology is ahead of the law – employment law – patent inventorship – data security and the Federal Trade Commission – copyright ownership – securities regulation of crowdfunding • Take-away: don’t panic, but be mindful – Understand risks of “just in-time compliance” 6
    • Who are the workers? • A. Baio, November 2008. The Faces of Mechanical Turk. • P. Ipeirotis. March 2010. The New Demographics of Mechanical Turk • J. Ross, et al. Who are the Crowdworkers? CHI 2010. 7
    • What about ethics? • Six Silberman, Lilly Irani, and Joel Ross. Ethics and tactics of professional crowdwork. XRDS: Crossroads, The ACM Magazine for Students 17.2 (2010): 39-43. – “How should we… conceptualize the role of these people who we ask to power our computing?” – Power dynamics – “Abstraction hides detail” • Fort, Karën, Gilles Adda, and K. Bretonnel Cohen. Amazon mechanical turk: Gold mine or coal mine?. Computational Linguistics 37.2 (2011): 413-420. – “…opportunities for our community to deliberately value ethics above cost savings.” 8
    • Davis et al. (2010) The HPU. HPU 9
    • HPU: “Abstraction hides detail” 10
    • Requester Fraud on MTurk “Do not do any HITs that involve: filling in CAPTCHAs; secret shopping; test our web page; test zip code; free trial; click my link; surveys or quizzes (unless the requester is listed with a smiley in the Hall of Fame/Shame); anything that involves sending a text message; or basically anything that asks for any personal information at all—even your zip code. If you feel in your gut it’s not on the level, IT’S NOT. Why? Because they are scams...” 11
    • Fraud example • Is this really the requester’s captcha? 12
    • Captcha Fraud • Severity? 13
    • • “…not only do malicious crowd-sourcing systems exist, but they are rapidly growing…” 14
    • Digital Dirty Jobs • The Googler who Looked at the Worst of the Internet • Policing the Web’s Lurid Precincts • Facebook content moderation • The dirty job of keeping Facebook clean • Even linguistic annotators report stress & nightmares from reading news articles! 15
    • Crowdsourcing and Vigilantism • http://searchenginewatch.com/article/2221527/Western-Human-Flesh- Search-Chases-Wrong-Guy 16
    • What about freedom? • Crowdsourcing vision: empowering freedom – work whenever you want for whomever you want • Risk: people being compelled to perform work – Chinese prisoners farming gold… – Digital sweat shops? Digital slaves? – We really don’t know (and need to learn more…) – Traction? Human Trafficking at MSR Summit’12 17
    • Mechanical Turk is Not Anonymous Matthew Lease, Jessica Hullman, Jeffrey P. Bigham, Michael S. Bernstein, Juho Kim, Walter S. Lasecki, Saeideh Bakhshi, Tanushree Mitra, and Robert C. Miller. Online: Social Science Research Network, March 6, 2013 ssrn.com/abstract=2190946
    • Worker Privacy Each worker is assigned an alphanumeric ID 19
    • Requesters see only Worker IDs 20
    • Safeguarding Personal Data • “What are the characteristics of MTurk workers?... the MTurk system is set up to strictly protect workers’ anonymity….” 21
    • Broad Perception of Anonymity 22
    • ` Amazon profile page URLs use the same IDs used on MTurk ! Who knew? 23
    • Workers’ Views: Survey & Forums • “... my reviewer profile is linked to my Mturk number! I had no idea...” • “...Amazon needs to separate the Mturk numbers from seller numbers to protect our privacy…” • “I think this is outrageous though. Makes me concerned about trusting privacy agreements.” • “Mine pulled up my Amazon wish list which revealed my identity. It seems to me that so called ”anonymous” tasks on mTurk (like surveys) are not anonymous after all.” 24
    • Risks to Workers • Inadvertent disclosure of PII or private data • Loss of blind hiring practices online • Greater risk of exploitation, reputation damage, loss of income, or even physical harm… 25
    • Risks to Researchers • Exposing participants to undocumented risks • Having disclosed WorkerIDs (e.g., online) • Having not restricted access to the internally – Potential harm to participants – Lack of compliance with Federal/IRB governance of human subjects research – Being required to discard collected data – Delays or inability to conduct future MTurk studies 26
    • Risks to Service Provider • Workers/Requesters abandoning MTurk • The US Federal Trade Commission (FTC) has recently begun to aggressively protect consumers from data breaches by commercial entities, including release of supposedly “anonymous” data – Inadequate protection of customer records: BJWC – De-anonymized customer records: AOL, Netflix – Did workers have a reasonable expectation of privacy in their use of MTurk which has been violated? 27
    • The Future of Crowd Work, CSCW’13 Kittur, Nickerson, Bernstein, Gerber, Shaw, Zimmerman, Lease, and Horton 28
    • Additional References • Paul Hyman. Software Aims to Ensure Fairness in Crowdsourcing Projects. Communications of the ACM, Vol. 56 No. 8, Pages 19-21, August 2013. • Irani, Lilly C. The Ideological Work of Microwork. In preparation, draft available online. • Irani, Lilly C., and Six Silberman. Turkopticon: Interrupting worker invisibility in amazon mechanical turk. ACM SIGCHI Conference, 2013. • Adda, Gilles, et al. Crowdsourcing for language resource development: Critical analysis of amazon mechanical turk overpowering use. Proceedings of the 5th Language and Technology Conference (LTC). 2011. • Adda, Gilles, and Joseph J. Mariani. Economic, Legal and Ethical analysis of Crowdsourcing for Speech Processing. (2013). • Harris, Christopher G., and Padmini Srinivasan. Crowdsourcing and Ethics. Security and Privacy in Social Networks. 67-83. 2013. • Harris, Christopher G. Dirty Deeds Done Dirt Cheap: A Darker Side to Crowdsourcing. IEEE 3rd conference on social computing (socialcom). 2011. • Horton, John J. The condition of the Turking class: Are online employers fair and honest?. Economics Letters 111.1 (2011): 10-12. 29
    • A Few Legal References • Felstiner, Alek. Working the Crowd: Employment and Labor Law in the Crowdsourcing Industry. Berkeley Journal of Employment and Labor Law 32.1 (2011). • Felstiner, Alek. Sweatshop or Paper Route?: Child Labor Laws and In-Game Work. Proceedings of CrowdConf (2010). • Zittrain, Jonathan. Ubiquitous human computing. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 366.1881 (2008): 3813-3821. 30
    • Join the conversation online • Crowdwork-ethics: “an informal and occasional newsletter for researchers interested in ethical issues in crowd work.” – Email Six Silberman to subscribe (manual) 31