Andreessen’s Corollary
Ethical Dilemmas in Software Engineering
CTO
bryan@joyent.com
Bryan Cantrill
@bcantrill
Ethics in software engineering, ca. 1997
Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)
Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)
Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)
Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)
Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)
Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)
Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)
Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)
Ethics in software engineering, ca. 1997
• There is northing wrong per se with the 1990s code of software
ethics — it just represents a precambrian era of software
• Notably, the guiding context surrounding the code of ethics itself
remains timeless…
• …but the code itself is quaint, and serves primarily to remind
how much software has changed in the last two decades
• With the rise of ubiquitous internet in the late 1990s came the
first real foreshocks of the ethical dilemmas to come…
1999: Napster
2003: Friendster
2003: Friendster
Source: Gary Rivlin, “Wallflower at the Web Party” (New York Times, October 15th, 2006)
2011: Andreessen’s prophesy
— Marc Andreessen, “Why Software Is Eating The World” (2011)
Software after Andreessen
• As software has indeed come into broader domains, the
internet-era challenges have multiplied and compounded — and
with increasingly serious ramifications
• There have been many clear ethical transgressions, common to
any era with a frenzied rush for mammon…
• …but much more common are true ethical dilemmas, laden
with complexity and ambiguity
• These are not entirely unrelated! Those least burdened by
ethics seem most likely to find themselves on the ethical
frontier, facing the greatest dilemmas
2012: Facebook emotional manipulation
Source: Kashmir Hill, “Facebook Manipulated 689,003 Users’ Emotions for Science” (Forbes, June 28th, 2014)
2012: Facebook emotional manipulation
Source: Kramer et al., “Experimental evidence of massive scale emotional contagion through social networks”
2013: Zenefits “Macro”
Source: Findings of fact in SEC Administrative Proceedings against Parker Conrad et al. (file 3-18263)
2014: Uber Greyball
Source: Greyball Audit Report, Portland Bureau of Transportation, April, 2017
2014: Uber Greyball
Source: letter from Uber to Portland City Attorney, as quoted in Greyball Audit Report, Portland Bureau of Transportation, April, 2017
2015: Google Photos mis-labelling
Source: https://twitter.com/jackyalcine/status/615329515909156865
2016: First Tesla “Autopilot” fatality
Source: National Transportation Safety Board Highway Accident Report (NTSB/HAR-17/02)
2016: First Tesla “Autopilot” fatality
Source: National Transportation Safety Board Highway Accident Report (NTSB/HAR-17/02)
2017: Facebook and Anti-Rohingya violence
Source: Report of the independent international fact-finding mission on Myanmar, United Nations Human Rights Council
2017: Facebook and Anti-Rohingya violence
2018: Joyent provides infrastructure to Gab
2019: 737 MAX MCAS and JT610/ET302 crashes
Source: Aircraft Accident Investigation Bureau Preliminary Report, Ethiopian Airlines Group, B737-8 ET-AVJ, March 10, 2019
Source: Boeing Company, April 17th 2019 MCAS update
2019: 737 MAX MCAS and JT610/ET302 crashes
Software in the post-Andreessen world
• This has been but a tiny sampling of the ethical dilemmas faced
by software in the post-Andreessen age
• It is clear that what is right for software is not necessarily right
for society: we must address our ethical dilemmas directly!
• Software is on the cusp of yet broader societal impact — and at
a time when our society is increasingly divided and fractured
• We have a greater burden to society than ever before —
how can we process these dilemmas?
Post-Andreessen ethics
Software in the post-Andreessen world
• Finished in the summer of 2018, the ACM’s new Code of Ethics
and Professional Conduct has arrived at the right time
• A radical overhaul of the 1990s-era code of ethics, it is much
more principles based, e.g.:
• Contribute to society and to human well-being
• Avoid harm
• Be honest and trustworthy
• Entire code is at https://ethics.acm.org
Beyond a Code of Ethics
• The ACM has gone beyond merely providing a Code of Ethics
by kicking off its Integrity Project
• Includes case studies and an “Ask an Ethicist” feature
• The ACM’s efforts show that to face ethical dilemmas, we need
to be able to ask tough questions
• These will often not have crisp answers — but the resulting
discussion is most likely to yield behavior consistent with the
code of ethics
• Organizations must initiate and support this discussion!
Andreessen’s Corollary
• Organizations that support frank discussion fo ethical dilemmas
will enjoy ethical differentiation
• They will attract like-minded individuals who can help
perpetuate a culture of consideration of ethical consequences
• We must recognize that as software’s footprint continues to
grow, out ethical dilemmas will only expand
• We must brace ourselves for future dilemmas
• Ethical quandary is Andreessen’s corollary — and it is our
collective responsibility to address It directly!

Andreessen's Corollary: Ethical Dilemmas in Software Engineering

  • 1.
    Andreessen’s Corollary Ethical Dilemmasin Software Engineering CTO bryan@joyent.com Bryan Cantrill @bcantrill
  • 2.
    Ethics in softwareengineering, ca. 1997
  • 3.
    Ethics in softwareengineering, ca. 1997 — Gottbaum et al., “Software Engineering Code of Ethics” (1997)
  • 4.
    Ethics in softwareengineering, ca. 1997 — Gottbaum et al., “Software Engineering Code of Ethics” (1997)
  • 5.
    Ethics in softwareengineering, ca. 1997 — Gottbaum et al., “Software Engineering Code of Ethics” (1997)
  • 6.
    Ethics in softwareengineering, ca. 1997 — Gottbaum et al., “Software Engineering Code of Ethics” (1997)
  • 7.
    Ethics in softwareengineering, ca. 1997 — Gottbaum et al., “Software Engineering Code of Ethics” (1997)
  • 8.
    Ethics in softwareengineering, ca. 1997 — Gottbaum et al., “Software Engineering Code of Ethics” (1997)
  • 9.
    Ethics in softwareengineering, ca. 1997 — Gottbaum et al., “Software Engineering Code of Ethics” (1997)
  • 10.
    Ethics in softwareengineering, ca. 1997 — Gottbaum et al., “Software Engineering Code of Ethics” (1997)
  • 11.
    Ethics in softwareengineering, ca. 1997 • There is northing wrong per se with the 1990s code of software ethics — it just represents a precambrian era of software • Notably, the guiding context surrounding the code of ethics itself remains timeless… • …but the code itself is quaint, and serves primarily to remind how much software has changed in the last two decades • With the rise of ubiquitous internet in the late 1990s came the first real foreshocks of the ethical dilemmas to come…
  • 12.
  • 13.
  • 14.
    2003: Friendster Source: GaryRivlin, “Wallflower at the Web Party” (New York Times, October 15th, 2006)
  • 15.
    2011: Andreessen’s prophesy —Marc Andreessen, “Why Software Is Eating The World” (2011)
  • 16.
    Software after Andreessen •As software has indeed come into broader domains, the internet-era challenges have multiplied and compounded — and with increasingly serious ramifications • There have been many clear ethical transgressions, common to any era with a frenzied rush for mammon… • …but much more common are true ethical dilemmas, laden with complexity and ambiguity • These are not entirely unrelated! Those least burdened by ethics seem most likely to find themselves on the ethical frontier, facing the greatest dilemmas
  • 17.
    2012: Facebook emotionalmanipulation Source: Kashmir Hill, “Facebook Manipulated 689,003 Users’ Emotions for Science” (Forbes, June 28th, 2014)
  • 18.
    2012: Facebook emotionalmanipulation Source: Kramer et al., “Experimental evidence of massive scale emotional contagion through social networks”
  • 19.
    2013: Zenefits “Macro” Source:Findings of fact in SEC Administrative Proceedings against Parker Conrad et al. (file 3-18263)
  • 20.
    2014: Uber Greyball Source:Greyball Audit Report, Portland Bureau of Transportation, April, 2017
  • 21.
    2014: Uber Greyball Source:letter from Uber to Portland City Attorney, as quoted in Greyball Audit Report, Portland Bureau of Transportation, April, 2017
  • 22.
    2015: Google Photosmis-labelling Source: https://twitter.com/jackyalcine/status/615329515909156865
  • 23.
    2016: First Tesla“Autopilot” fatality Source: National Transportation Safety Board Highway Accident Report (NTSB/HAR-17/02)
  • 24.
    2016: First Tesla“Autopilot” fatality Source: National Transportation Safety Board Highway Accident Report (NTSB/HAR-17/02)
  • 25.
    2017: Facebook andAnti-Rohingya violence Source: Report of the independent international fact-finding mission on Myanmar, United Nations Human Rights Council
  • 26.
    2017: Facebook andAnti-Rohingya violence
  • 27.
    2018: Joyent providesinfrastructure to Gab
  • 28.
    2019: 737 MAXMCAS and JT610/ET302 crashes Source: Aircraft Accident Investigation Bureau Preliminary Report, Ethiopian Airlines Group, B737-8 ET-AVJ, March 10, 2019
  • 29.
    Source: Boeing Company,April 17th 2019 MCAS update 2019: 737 MAX MCAS and JT610/ET302 crashes
  • 30.
    Software in thepost-Andreessen world • This has been but a tiny sampling of the ethical dilemmas faced by software in the post-Andreessen age • It is clear that what is right for software is not necessarily right for society: we must address our ethical dilemmas directly! • Software is on the cusp of yet broader societal impact — and at a time when our society is increasingly divided and fractured • We have a greater burden to society than ever before — how can we process these dilemmas?
  • 31.
  • 32.
    Software in thepost-Andreessen world • Finished in the summer of 2018, the ACM’s new Code of Ethics and Professional Conduct has arrived at the right time • A radical overhaul of the 1990s-era code of ethics, it is much more principles based, e.g.: • Contribute to society and to human well-being • Avoid harm • Be honest and trustworthy • Entire code is at https://ethics.acm.org
  • 33.
    Beyond a Codeof Ethics • The ACM has gone beyond merely providing a Code of Ethics by kicking off its Integrity Project • Includes case studies and an “Ask an Ethicist” feature • The ACM’s efforts show that to face ethical dilemmas, we need to be able to ask tough questions • These will often not have crisp answers — but the resulting discussion is most likely to yield behavior consistent with the code of ethics • Organizations must initiate and support this discussion!
  • 34.
    Andreessen’s Corollary • Organizationsthat support frank discussion fo ethical dilemmas will enjoy ethical differentiation • They will attract like-minded individuals who can help perpetuate a culture of consideration of ethical consequences • We must recognize that as software’s footprint continues to grow, out ethical dilemmas will only expand • We must brace ourselves for future dilemmas • Ethical quandary is Andreessen’s corollary — and it is our collective responsibility to address It directly!