Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

XR Ethics Manifesto (UPDATED Nov 2, 2019)


Published on

UPDATED. The immersive industry needs an ethical framework to navigate moral dilemmas. This is my XR Ethical Manifesto presented at Greenlight's XR Strategy Conference on Friday, October 18, 2019 starting at 1:33p PT in San Francisco, CA.

Published in: Design
  • Login to see the comments

XR Ethics Manifesto (UPDATED Nov 2, 2019)

  1. 1. An XR Ethics Manifesto @KentBye
  2. 2. A brief chronology & evolution of my thinking about ethics in XR
  3. 3. What is the Ultimate Potential of VR?
  4. 4. Photo by
  5. 5. Photo by
  6. 6. Photo by
  7. 7. Photo by
  8. 8. Photo by
  9. 9. A Manifesto for Ethical XR
  10. 10. Expression of Identity
  11. 11. Consciousness Hacking
  12. 12. Sensory Substitution
  13. 13. Sensory Addition
  14. 14. Self-Sovereign Identity
  15. 15. ● Biometric data privacy implications ● XR modulates human perception ● Diversity of avatar representation ● Which biometric streams are personally identifiable information? ● Cybersickness ● Correlation of behavior with genetics ● We should NOT have companies harvesting our emotions I. Self / Biometric Data / Identity Dilemmas
  16. 16. Biometric data is ephemeral & context dependent. Do real-time processing on it when possible. Don’t hoard it. Imagine 10 years of biometric data in the wrong hands. An XR Ethics Manifesto - I
  17. 17. Offer a diversity of avatar representations to increase the fidelity of identity expression. An XR Ethics Manifesto - I
  18. 18. Modulating perception & consciousness is a super power. Do no harm. Ensure it’s consensual. Use this ability wisely & responsibly. An XR Ethics Manifesto - I
  19. 19. Conduct studies to assess the long-term impacts of immersion. An XR Ethics Manifesto - I
  20. 20. Nudging user behavior could have unintended & negative long-term impacts. An XR Ethics Manifesto - I
  21. 21. Enhance the control & power of people in an experience while respecting how we’re interdependent. We should be sovereign individuals within the context of a collective. An XR Ethics Manifesto - I
  22. 22. Yang Yin Competition Walled Garden Curated Content Vertical-Integration Proprietary App Store Cooperation Open Ecosystem No Gatekeepers Horizontal Platform Shared Source Code Open Access
  23. 23. Virtual Gift Economies
  24. 24. ● Ethics of surveillance capitalism ● Corporations have a fiduciary responsibility to shareholder profit, which can conflict with user autonomy & privacy ● Do we have to pay for privacy? ● Can we own & sell our own data? ● Asymmetry of power in attention economy ● Can we share/loan virtual objects? ● Rights to ownership in virtual space ● Can differential privacy or homomorphic encryption help privacy architectures? II. Resources / Money / Values Dilemmas
  25. 25. Security & privacy are essential. An XR Ethics Manifesto - II
  26. 26. You should own your data. You should have a right to export, sell, or permanently delete your data. An XR Ethics Manifesto - II
  27. 27. Don’t aggregate financial transactions across multiple contexts. We should have the right to exchange value without being tracked. An XR Ethics Manifesto - II
  28. 28. Surveillance capitalism is fundamentally unethical because it changes the cultural & legal definition of a “reasonable expectation of privacy.” This is due to the Third Party Doctrine, which established that data given to third parties is not private. An XR Ethics Manifesto - II
  29. 29. Assume any de-identified data may have a biometric fingerprint that could unlock someone’s identity if correlated with other Personally Identifiable Information (PII), observed behavior, or through machine learning. An XR Ethics Manifesto - II
  30. 30. Assume any data that’s tracked can be leaked onto the Dark Web & into the hands of tyrannical governments, authoritarian adversaries, or malicious hackers. The best practice is not to record it in the first place. An XR Ethics Manifesto - II
  31. 31. We need new business models that don’t create power asymmetries. Fiduciary responsibility to profit can’t be prioritized above the privacy & autonomy of users. An XR Ethics Manifesto - II
  32. 32. There is a digital divide preventing equal access to these technologies. We have a moral imperative to design business models & political systems that serve underrepresented minorities across all classes. An XR Ethics Manifesto - II
  33. 33. What we see as valuable changes over time & algorithmic inference will always be incomplete. Be careful how these models create assumptions about us that may be completely wrong. Offer ways to provide feedback or mitigate the recording of information when possible. An XR Ethics Manifesto - II
  34. 34. Early Education
  35. 35. Memory Palaces
  36. 36. Telepresence
  37. 37. Data Visualization
  38. 38. Spatial Language
  39. 39. Metaphoric Communication
  40. 40. ● Brain control interfaces will be able to read our thoughts. Where’s that data go? ● Risks of negative transfer in education ● Design systems to mitigate hate speech ● Enable users to mute interactions ● Surveillance of private communication ● There’s also metadata surveillance tracking who we’re talking to and when ● Implications of sharing biometric data? ● Consent for accessibility autotranscripts? III. Early Education / Communication Dilemmas
  41. 41. Technology should enable freedom of expression. An XR Ethics Manifesto - III
  42. 42. We have a right to keep who we’re communicating with & what we say private. We also need our privacy to be balanced with security. An XR Ethics Manifesto - III
  43. 43. Offer end-to-end encryption when possible. Don’t track who we’re talking to and when we talk to them. We need to balance freedom with security, & continue to question which takes priority based upon context. An XR Ethics Manifesto - III
  44. 44. There is loss in all acts of communication. We must design systems that minimize that loss & account for a spectrum of abilities of communicative expression & reception. It’s important that everyone has a voice regardless of ability. An XR Ethics Manifesto - III
  45. 45. We’re moving from explicit data entry to implicit data entry based on embodied movements & biometric data. We need standardized ways to port individual profiles of trained input gestures, movements, & voices to work across different virtual worlds & systems. An XR Ethics Manifesto - III
  46. 46. Design systems that effectively handle harassment and trolling. Educate users of your prevention tools as a part of your onboarding process. Offer code of conduct orientations or interactive training experiences to help cultivate an inclusive culture. An XR Ethics Manifesto - III
  47. 47. Volumetric Memories
  48. 48. ● Who owns an AR space? Is it a new public commons? Or determined by private property ownership? ● Be aware of spatial doxxing threats ● What’s the ecological impact of tech? ● Threats of memory holing official history ● Tracking movements can reveal where you or loved ones live ● Try to have a balance of private & public spaces in your spaces. IV. Home & Family / Private Property Dilemmas
  49. 49. It is everyone’s responsibility to ensure technology is being produced & materials sourced in an ecologically-sustainable manner. An XR Ethics Manifesto - IV
  50. 50. Virtual worlds can have public and private spaces. Design private virtual spaces to enable social experiences that allow users more control, autonomy, & identity than in public spaces. An XR Ethics Manifesto - IV
  51. 51. A home address is very sensitive to doxxing. Educate users about the risks of spatial doxxing in AR & how to mitigate exposure of environmental information that could used to geo-locate them. An XR Ethics Manifesto - IV
  52. 52. Immersive media can alter and enrich our sense of the history and culture of a place. Never secretly alter historical records in an attempt to misinform or memory hole events. An XR Ethics Manifesto - IV
  53. 53. Users must be educated on the risks of allowing private spaces to be volumetrically captured as it reveals intimate personal & mental health information. Otherwise they will almost certainly be violating the privacy of non-consenting friends & family inadvertently. An XR Ethics Manifesto - IV
  54. 54. Immersive Storytelling
  55. 55. The Holodeck
  56. 56. Flow States
  57. 57. Creativity & Art
  58. 58. ● Addiction & dopamine economy ● Conscious vs unconscious behaviors ● Consenting to violent content ● Unintended consequences of XR porn? ● Systems that limit creative expression ● Accidentally revealing sexual preferences through eye tracking data ● Tracking of our entertainment options ● Mitigating sharing of explicit sexual content V. Entertainment / Hobbies / Sex Dilemmas
  59. 59. Don’t hijack our attention. An XR Ethics Manifesto - V
  60. 60. Don’t use addictive gameplay mechanics to create dependencies that disempower users or degrade their quality of life. An XR Ethics Manifesto - V
  61. 61. Use content warnings, but be aware some people may not know what they’re consenting to. There is no way to take an immersive experience back & there may be unintended consequences to experiencing violence in XR.* *See Section VIII for more details. An XR Ethics Manifesto - V
  62. 62. Emulate the view source principle of the open web, which makes it transparent & remixable. Support WebXR & OpenXR to enable freedom of creative expression & innovation with open ecosystems rather than walled gardens. An XR Ethics Manifesto - V
  63. 63. Real-time biofeedback & implicit input should not undermine users’ conscious agency. Passive informed consent must not be used to hijack autonomy against the better interests of the user. An XR Ethics Manifesto - V
  64. 64. Ensure XR porn is ethically produced according to porn industry standards, consent policies, & performer codes of conduct. Reduce shame around this topic by engaging in open dialogue, & look for ways to mitigate harm sustained from porn addiction. An XR Ethics Manifesto - V
  65. 65. Eye tracking & other biometric data can be used to determine sexual preferences, which can be life-threatening for people in the wrong country. Design for the most marginalized people in the most authoritarian scenarios. An XR Ethics Manifesto - V
  66. 66. Implement content moderation tools to prevent the sharing of sexually-explicit content on systems frequented by minors. An XR Ethics Manifesto - V
  67. 67. Neuroplasticity
  68. 68. Ownership of Health
  69. 69. Healing Trauma
  70. 70. Greenleaf’s Medical Sectors
  71. 71. ● Do no harm ● Should you disclose medical conditions that you discover in users? ● Flashing lights can trigger seizures ● Are people with specific mental health diagnoses more susceptible to harm? ● Privacy for telemedicine ● Content trigger warnings & map cartography of PTSD & trauma ● A rehabilitation path for banned users? ● Should HIPAA regulate biometric data? VI. Medicine / Healing Dilemmas
  72. 72. All biometric data contains health & medical information, & needs to be protected as such. The users’ data must be guarded against malicious actors seeking to capitalize by profiling through behavioral inferences. An XR Ethics Manifesto - VI
  73. 73. The storage of biometric data should be regulated through something like HIPAA or new regulatory frameworks will need to be implemented. An XR Ethics Manifesto - VI
  74. 74. Design technology to empower people to monitor & assess health conditions safely & effectively, thereby providing more autonomy & control over their own healing processes. An XR Ethics Manifesto - VI
  75. 75. Design experiences for people on a wide spectrum of physical conditions and/or mental diversity. An XR Ethics Manifesto - VI
  76. 76. Virtual Dates
  77. 77. Empathy + Truth & Reconciliation
  78. 78. ● New boundaries for intimate relationships ● Designing for safety from harassment ● Mitigating deep fakes & forged identity ● Retributive vs Restorative Justice ● The downsides of empathy machine or trauma tourism as tokenized spectacle. ● Dangers of anthropomophic AI influence VII. Other / Partnerships Dilemmas
  79. 79. Provide tools for users to block and mute people who harass them. An XR Ethics Manifesto - VII
  80. 80. Implement methods for users to protect the autonomy & safety of their personal spaces (i.e. through personal space bubbles or other innovations). An XR Ethics Manifesto - VII
  81. 81. Improve upon self-sovereign identity standards and invent new methods to validate people’s identity and mitigate harm caused by identity theft, deep fakes, and bots. An XR Ethics Manifesto - VII
  82. 82. Promote systems of restorative justice within immersive technological communities. Cultivate practices of owning harm done, authentic apologies, & paths towards redemption. An XR Ethics Manifesto - VII
  83. 83. WebXR content should work across all of the major platforms & devices.
  84. 84. Grief Rituals
  85. 85. ● Rights to identity after you die ● Implications of killing people in VR ● The right for you to be forgotten ● Using VR for torture ● Experiential warfare ● Long-term implications of exiling & permanently banning people ● Sexual assault in XR ● Human rights violations ● Filtering violent or terrorist content VIII. Death Dilemmas
  86. 86. We have the right to be forgotten. Architect systems that allow people to export their data or permanently delete all traces of their digital footprint. An XR Ethics Manifesto - VIII
  87. 87. We must design trauma-aware interfaces & annotate experiences with content ratings to mitigate harm. We have a right to produce content exploring traumatic events, & also a responsibility to do our best not to cause harm. An XR Ethics Manifesto - VIII
  88. 88. Consider the unknown ethical implications of content you design. Immersive violence may have unintended consequences we cannot predict by comparing it to 2D technologies. Treat this new medium with the respect unknowable & unpredictable technology requires. An XR Ethics Manifesto - VIII
  89. 89. Living & dying is a process we all go through. Honor how immersive experiences may help with the grieving process, & be encouraged to create new rituals around death & loss. An XR Ethics Manifesto - VIII
  90. 90. Breaking Down Academic Silos
  91. 91. Embodied Cognition
  92. 92. Comprehend Complexity
  93. 93. Sci-Fi World Building
  94. 94. Wonder & Awe
  95. 95. ● Many philosophical implications of XR ● What is truth? What is reality? ● Mitigating filter bubbles of reality ● Ethics of biohacking ● Need a comprehensive ethics framework ● Informed consent & progressive permissions ● What content is illegal? ● Implications of Third Party Doctrine ● Need new economic business models ● Future dream both protopias & cautionary tales ● Embrace paradox, plurality, process thinking IX. Dilemmas: Philosophy / Higher Education / Law
  96. 96. Participate & initiate discussions about ethical frameworks for XR. Consider this XR Ethics Manifesto a template & catalyst for the creation of your own ethical frameworks. An XR Ethics Manifesto - IX
  97. 97. Be aware of how the stories we tell & worlds we build shape the future. Dystopic narrative tropes can either be cautionary tales or self-fulfilling prophecies. XR is a powerful medium to prototype worlds designed to cultivate protopian cultures. Create new worlds with deliberate intention. An XR Ethics Manifesto - IX
  98. 98. Spatial Design
  99. 99. Virtual Screens for Productivity
  100. 100. ● Mitigating governmental overreach, surveillance, & tyrannical control ● XR used as governmental loyalty tests ● Conflicts of interest with industry & academia ● Using biometric data from XR for hiring ● Augmenting public spaces for good ● Implications of remote work ● Algorithmic transparency for AI ● Collective right to augment public space X. Career / Government / Institutions Dilemmas
  101. 101. Don’t rely solely on biometric data assessment for hiring. Modelling is incomplete & can’t handle the full complexity of a human individual. Don’t relegate human judgment to a computer when judging humans. An XR Ethics Manifesto - X
  102. 102. Enable algorithmic transparency so the public can see how the architecture of code is influencing their lives. An XR Ethics Manifesto - X
  103. 103. Create public virtual spaces that are open, vibrant, & encourage the freedom of expression. An XR Ethics Manifesto - X
  104. 104. Think critically about the implications of technology across different geopolitical contexts. Authoritarian governments misuse & abuse data for mass surveillance & human rights violations. Don’t enable turnkey tyranny within your software architectures. An XR Ethics Manifesto - X
  105. 105. WebXR content should be deliverable to the web around the world independent of country boundaries. An XR Ethics Manifesto - X
  106. 106. Hanging Out with Friends
  107. 107. ● Danger of explicit & implicit social scores ● How to support the cultivation of communities ● Code of conducts & enforcement ● Violating normative standards (playing Pokémon Go at The Holocaust Museum) ● Freedom of assembly & association implications ● Principles of diversity & inclusion ● Preventing algorithmic bias ● Biometrics data radiated to community XI. Friends / Community / Cultural Dilemmas
  108. 108. Don’t cause harm to society. An XR Ethics Manifesto - XI
  109. 109. Cultivate cultures, don’t engineer them. Driving collective behaviors requires many ethical considerations, don’t take shortcuts. When it doubt, choose to benefit the many over the few. An XR Ethics Manifesto - XI
  110. 110. Diversity & inclusion is a foundational principle, & should be encouraged at all levels because it benefits the collective. An XR Ethics Manifesto - XI
  111. 111. Promote diversity of input & feedback as a way to mitigate algorithmic bias. Raise the accuracy of your algorithms by including data sets & stakeholders from diverse backgrounds. An XR Ethics Manifesto - XI
  112. 112. Have a code of conduct for your community, experience, or product. Behavior is modelled both from the top down as well as the bottom up. Encourage users to cultivate the cultures they want. An XR Ethics Manifesto - XI
  113. 113. WebXR content should work across all of the major XR platforms & devices. An XR Ethics Manifesto - XI
  114. 114.
  115. 115. Combating Isolation
  116. 116. Accessibility
  117. 117. ● Inclusive design & accessibility ● Is it possible to be truly anonymous? ● Dark Spatial Web ● Biometric polygraph ● Re-identifying de-identified PII data ● Bad inferences from incomplete context ● Utility vs Downsides of exile or banning XII. Hidden / Exiled / Accessibility Dilemmas
  118. 118. Make XR content available to all people regardless of physical ability or fidelity of input. Making content accessible improves the experience for everyone. Solicit feedback from people with disabilities for how to make your experience more inclusive. An XR Ethics Manifesto - XII
  119. 119. Enable consumers of WebXR content to modify how content is rendered in similar way to how they can with the 2D web. An XR Ethics Manifesto - XII
  120. 120. Conclusion
  121. 121. ������ �� Domains of Human Experience (Contexts)�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Self Biometric Data Identity Resources Money Values Early Education Communication Home & Family Private Property Entertainment Hobbies Sex Medical Heath Other Partnerships Law Death Collective Resources Philosophy Higher Education Career Government Institutions Friends Community Collective Culture Hidden Exiled Accessibility
  122. 122. This is the start of the conversation. We need everyone’s insights & input.
  123. 123. Appendix
  124. 124. Slides from My “Ethical & Moral Dilemmas of Mixed Reality Talk at Augmented World Expo
  125. 125. ������ �� Photo by Domains of Human Experience (Contexts)�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Self Biometric Data Identity Resources Money Values Early Education Communication Home & Family Private Property Entertainment Hobbies Sex Medical Heath Other Partnerships Law Death Collective Resources Philosophy Higher Education Career Government Institutions Friends Community Collective Culture Hidden Exiled Accessibility
  126. 126. ������ �� Domains of Human Experience (Contexts)�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Self Biometric Data Identity Resources Money Values Early Education Communication Home & Family Private Property Entertainment Hobbies Sex Medical Heath Other Partnerships Law Death Collective Resources Philosophy Higher Education Career Government Institutions Friends Community Collective Culture Hidden Exiled Accessibility
  127. 127. ������ �� Photo by Self Biometric Data Identity�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Biometric fingerprint? Modulate perception Diversity of avatar options Self-sovereign identity Movements as PII? Who owns your data? Sell access to your data? Privacy as a luxury good BCI reading thoughts Correlate genetics to behavior Biofeedback entertainment Unconscious vs Conscious Agency Ethics of detecting & reporting medical conditions Informed Consent Progressive Permissions Deep Fakes of Your Identity Autonomy of data sovereignty vs Utilitarian public good Ethics of biohacking Biometrics used in interviews Government loyalty tests Sharing biometrics with friends Implicit bias Biometrics as polygraph Anonymity vs biometric security False positives from missing context Can biometrics be de-identified? �� �� Mortgaging your privacy Others profiting on your data Harvesting your emotions
  128. 128. ������ �� Photo by Communication Early Education�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Eye tracking & emotions Physiological development of children What age is safe? How long? Free Educational Resources End-to-end encryption Peer-to-peer connections Being able to work from home` People sharing inappropriate sexual content Privacy of Telemedicine consults 1st Amendment Free Speech Hate Speech Education for EveryoneRemote Telepresence Code of Conduct Banning people Anonymous communication �� �� Negative Transference
  129. 129. ������ �� Photo by Home Family Private Property�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Scan your home & the objects in it Burglers scoping out your valuables Protecting home address Giving consent for others to use your property for augmentation 4th Amendment and the “Reasonable expectation of privacy” Ecological sustainability of XRCollective right to augment Public spaces Pokemon Go at Holocaust Museum Spatial doxxing �� �� Rehabilitation & exercise at home
  130. 130. ������ �� Photo by Entertainment Sex Hobbies�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Biometric-responsive entertainment Sexually explicit avatars Virtual violence in your home Causing PTSD in XR Content Trigger Warnings Hijacking your attention IP ownership & use Remix culture Content Ratings Addiction to virtual worlds �� ��🏽 Physical danger from AR games Filtering terrorist & violent content Filtering Virtual & Augmented Pornography
  131. 131. ������ �� Photo by Medical Health�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Cybersickness Individualized medicine Become your own healer Secure communications Own your own medical data Ethics of detecting & reporting medical conditions Should HIPAA govern biometric data? New models of science from big data to do individual assessments and personalized medicine Public research benefit from sharing biometric data Community healing rituals Privacy in telemedicine Homomorphic encryption Differential Privacy �� ��
  132. 132. ������ �� Photo by Other Partnerships Law�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Violation of personal space Avatar discrimination Sharing virtual resources with your friends Mute others Hate speech Having private spaces to gather in Augmenting other people New boundaries for intimacy Anthropomorphizing AI agents Empathy Accountability Comprehensive frameworks For privacy XR Ethical Frameworks Restorative justice Moderators enforce the Rules & code of conduct Implicit trust & reputation scores Cultural norms Block harassment & trolls Possible to be an anonymous user? ��♀ �� Illegal content Sexual assault in XR Blocking others
  133. 133. ������ �� Photo by Death Collective Resources�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Right to have your data deleted Ephemerality of biometric data Rights to your likeness after you die Don’t record metadata of communications Killing people in virtual worlds Rehabilitation of blocked users Implications of permanent blocking? Taxes for public services? Taboos around death ⚰ �� Virtual funeral rituals
  134. 134. ������ �� Photo by Philosophy Higher Education�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Siloed academic disciplines Conflicts of interest of academic collaboration with industry & IP Disclose what’s “real” and what’s “not real” Filter bubbles of reality Is direct experience basis of “reality” for harassment What is Reality? Definition of consciousness Ethical frameworks World build “Protopias” More interdisciplinary collaborations between academia and industry Diversity of data for training AI �� �� Technical debt of machine learning Visualizing dystopian futures
  135. 135. ������ �� Photo by Career Government Institutions�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Using biometric data as a Polygraph test Legally-mandated, profit-driven maximization of shareholder value Memory holing history Augment historical monuments Third party doctrine NDAs Experiential Warfare GDPR for USA? TransparencyGovernment surveillance Open source algorithms Social credit scores impact Access to physical services Undermining democracy via leaked psychographic profiles �� �� VR Torture Government getting secret access to our behavioral & biometric data streams Government getting access to what we say online
  136. 136. ������ �� Photo by Friends Community�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� We can identify how friends move P2P communications Broadcast biometrics to friends? Private hangouts for friends Social network analysis via metadata Social scores Metadata of our connections �� ��
  137. 137. ������ �� Photo by Hidden Exiled Accessibility�� �� 🏠 👪 ����🏽 �� �� �� ��♀ ⚰�������������� �� �� Design for non-abled bodies Reading thoughts Eye tracking accessibility apps Paralyzed Blind Deaf What we value = what we look at Autotranscripts for deaf Living permanently in Virtual worlds Rehabilitation of trolls Restorative justice for banned Illegal content in XR (child porn?) Map the unconscious psyche Unknown long-term effects Dark spatial web �� �� Harvesting your emotions
  138. 138. Support this work at