Citation: Bye, K. (2023, October 19). The Promises & Perils of Extended Reality. [Opening Keynote]. Broadcast Education Association On-Location @ University of South Carolina, Lexington Room at the Courtyard by Marriott Columbia Downtown at USC, Columbia, South Carolina.
Voices of VR podcast host Kent Bye has been asking thousands of virtual reality creators about the ultimate potential of XR for over 9 years now, and he will be sharing some of the best answers that map out the future of spatial computing. VR also presents unique tech ethics challenges around privacy, accessibility, conduct moderation, and moving from 2D into real-time, 3D spaces that will require new tech architectures and potentially new laws. This session will set a broad context for some of the promises and perils of virtual reality, augmented reality, and mixed reality while also providing a glimmer for what's to come.
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Promises & Perils of Extended Reality, Broadcast Education Association Keynote
1. @KentBye
Bye, K. (2023, October 19). The Promises & Perils of Extended Reality. [Opening Keynote]. Broadcast Education Association On-Location @ University
of South Carolina, Lexington Room at the Courtyard by Marriott Columbia Downtown at USC, Columbia, South Carolina.
The Promises & Perils
of Extended Reality
3. ● Experiential Design &
Immersive Storytelling Framework
○ Quality
○ Story
○ Character
○ Context
● Promises and Perils of XR
4. ● Experiential Design &
Immersive Storytelling Framework
○ Quality
○ Story
○ Character
○ Context
● Promises and Perils of XR
5. “True character is revealed in the choices
a human being makes under pressure -
the greater the pressure, the deeper the
revelation, the truer the choice to the
character's essential nature.”
― Robert McKee
McKee, R. (1997). Story: Substance, structure, style, and principles of screenwriting. (pp. 101) ReganBooks.
6. Placed in a
Context with
Pressure
Make
Choices &
Take Action
Essential
Character is
Revealed
+ =
Unfolding Process Over Time
9. Meta Quest. (2014, October 1). Oculus Connect Keynote: Brendan Iribe and Nate Mitchell. [Presented on September 20, 2014 at Oculus Connect, Loews Hollywood Hotel, Los
Angeles, CA]. YouTube. Retrieved April 21, 2022, from https://www.youtube.com/watch?v=1xeK8zUXAvQ
10. Slater’s Place Illusion & Plausibility Illusion (2009)
● Slater, M. (2009, Dec 14). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society
B: Biological Sciences, 364(1535), 3549–3557. doi:10.1098/rstb.2009.0138.
● Images via Toast (2019, July 21). Richie's Plank Experience Oculus Quest Trailer. YouTube Retrieved on April 25, 2022 from
https://www.youtube.com/watch?v=faNsP7ExSt0.
Plausibility Illusion
“the illusion that what is apparently
happening is really happening (even
though you know for sure that it is
not).”
Place Illusion
“It is the strong illusion of being in a
place in spite of the sure knowledge
that you are not there.”
16. Jung on Archetypes: Can’t be “reduced to a simple formula”
Jung, C. G. (1968). The Psychology of the Child Archetype. In R. F. C. Hull (Trans.), The Collected Works of C. G Jung | Complete Digital Edition (Vol. 9: Part 4 of The Archetypes
and the Collective Unconscious, 2nd Edition, pg. 179, paragraph 301). Princeton, NJ: Princeton University Press. (Original work published 1940).
17. Jung on Archetypes: Can’t be “reduced to a simple formula”
Jung, C. G. (1968). The Psychology of the Child Archetype. In R. F. C. Hull (Trans.), The Collected Works of C. G Jung | Complete Digital Edition (Vol. 9: Part 4 of The Archetypes
and the Collective Unconscious, 2nd Edition, pg. 179, paragraph 301). Princeton, NJ: Princeton University Press. (Original work published 1940).
“Clear-cut distinctions and strict formulations are quite
impossible in this field, seeing that a kind of fluid
interpenetration belongs to the very nature of all
archetypes. They can only be roughly circumscribed at
best. Their living meaning comes out more from their
presentation as a whole than from a single formulation.
Every attempt to focus them more sharply is immediately
punished by the intangible core of meaning losing its
luminosity. No archetype can be reduced to a simple
formula. It is a vessel which we can never empty, and
never fill. It has a potential existence only, and when it
takes shape in matter it is no longer what it was. It persists
throughout the ages and requires interpreting ever anew.
The archetypes are the imperishable elements of the
unconscious, but they change their shape continually.”
19. Active Presence
Agency, Interactions, Dynamics,
Action, Locomotion, Exploration,
Spontaneity, Creativity, Will,
Engagement, Gameplay,
Intention, & Live Performance.
Emotional Presence
Emotional Immersion, Feelings,
Mood, Vibe, Color, Lighting,
Consonance & Dissonance
Cycles, Music, Story, Time,
Flow, Narrative Presence,
Empathy, & Compassion
Mental & Social Presence
Thoughts, Language, Choices,
Expectations & Novelty, Mental
Models, Plausibility, Coherence,
Attention, Suspension of Disbelief,
Puzzles, Mental Friction, Rules,
Social Presence, & Virtual Beings
Embodied Presence
Self-Presence, Body, Embodiment,
Sensory Experience, Biometric &
Physiological Data, BCIs & Neural
Inputs, Haptics, Immersive Sound,
Spatial Presence, Environmental
Immersion, & Architecture
Bye, K. (2022, May 5) "A Primer on Presence, Immersive Storytelling, & Experiential Design." [Presentation]
Storycon:A collision of (he)art and technology. BOZAR. Brussels, Belgium.
20. Slater’s Place Illusion & Plausibility Illusion (2009)
Slater, M. (2009, Dec 14). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B:
Biological Sciences, 364(1535), 3549–3557. https://doi.org/10.1098/rstb.2009.0138.
22. Slater et al (2022) Update to Presence Illusions adds
Co-presence & Body Ownership Illusion (Embodiment)
Slater, M., Banakou, D., Beacco, A., Gallego, J., Macia-Varela, F., & Oliva, R. (2022). A separate reality: An update on place illusion and plausibility in virtual reality. Frontiers in
Virtual Reality, 3. https://doi.org/10.3389/frvir.2022.914392
25. “The process of using these dimensions to create such an
experience is known as experiential design” - Chertoff (2010)
● Chertoff, Dustin, "Exploring Additional Factors Of Presence" (2009). Electronic Theses and Dissertations. 3910. https://stars.library.ucf.edu/etd/3910
● Chertoff, D. B., Goldiez, B., & LaViola, J. J. (2010). Virtual Experience Test: A virtual environment evaluation questionnaire. 2010 IEEE Virtual Reality Conference (VR).
doi.org/10.1109/vr.2010.5444804
● Kent, B., & Dustin, C. (2017, July 19). #557: Using Experiential Design to Expand VR Presence Theory. Voices of VR Podcast.
https://voicesofvr.com/557-using-experiential-design-to-expand-vr-presence-theory/
Experiential Design "originated from the marketing field
where it was used to encourage people to create meaningful
emotional and social connections to a product."
26. Cognitive &
Relational
Active
Affective
Sensory
Chertoff, D. B., Goldiez, B., & LaViola, J. J. (2010). Virtual Experience Test: A virtual environment evaluation questionnaire. 2010 IEEE Virtual Reality Conference (VR).
doi.org/10.1109/vr.2010.5444804
27. Schmitt, B. (1999). Experiential marketing. Journal of marketing management, 15(1-3), 53-67.
Experiential Marketing: “Strategic Experiential Modules” (1999)
● Sense
● Feel
● Think
● Act
● Relate
29. Brakus, J. J., Schmitt, B. H., & Zarantonello, L. (2009). Brand experience: what is it? How is it measured? Does it affect loyalty?. Journal of marketing, 73(3), 52-68.
https://doi.org/10.1509/jmkg.73.3.052
Measuring Brand Experience (2009)
● Sensory
● Affective
● Behavioral
● Intellectual
30. Intellectual
Behavioral
Affective
Sensory
Brakus, J. J., Schmitt, B. H., & Zarantonello, L. (2009). Brand experience: what is it? How is it measured? Does it affect loyalty?. Journal of marketing, 73(3), 52-68.
https://doi.org/10.1509/jmkg.73.3.052
31. Learning domains. CBL. (n.d.). https://casebasedlearning.weebly.com/learning-domains.html
Four Learning Domains
33. International Joint Conference on
Artificial Intelligence (IJCAI) Topics
● Agent-Based & Multi-Agent
Systems
● AI Ethics, Trust, Fairness
● AI4Good
● AI ARTS
● Computer Vision
● Constraint Satisfaction and
Optimization
● Humans and AI
● Knowledge Representation &
Reasoning
● Machine Learning
● Multidisciplinary Topics &
Applications
● Natural Language Processing
● Planning and Scheduling
● Search
● Uncertainty
IJCAI (2022, July). International Joint Conference on Artificial Intelligence Schedule. IJCAI 2022. Retrieved on April 20, 2023 from
https://ijcai-22.org/wp-content/uploads/2022/07/Main_Conference_Program_Printing_V.pdf
34. ● Knowledge Representation &
Reasoning
● Natural Language Processing
● Planning and Scheduling
● Constraint Satisfaction & Optimization
● Multidisciplinary Topics & Applications
● Humans and AI
● Search
● Uncertainty
● Agent-Based and
Multi-Agent Systems
● AI Ethics, Trust, Fairness
● AI4Good
● AI Arts
● Emotional Detection
● Computer Vision
● Machine Learning
● Robotics
IJCAI (2022, July). International Joint Conference on Artificial Intelligence Schedule. IJCAI 2022. Retrieved on April 20, 2023 from
https://ijcai-22.org/wp-content/uploads/2022/07/Main_Conference_Program_Printing_V.pdf
35. Skarbez, R., Brooks, Jr., F. P., & Whitton, M. C. (2017). A Survey of Presence and Related Concepts. ACM Computing Surveys, 50(6), 1–39. https://doi.org/10.1145/3134301
Survey of Presence Concepts (Skarbez et al, 2017)
36. Mental & Social Presence
Active Presence
● Agency
● Modify Environment
● Engagement
● Participation
● Control
● Plausibility Illusion [Interactivity]
● Action Fidelity
Emotional Presence
● Involvement & Engagement
● Flow
● Absorption
● Engrossment
● Transportation & Transportability
● Total Immersion
● Narration & Dramatic Presence
● Meaningfulness
Embodied Presence
● Sensory Presence & Sensory Modality
● Embodiment, Body Ownership, Avatar,
Self-Presence, & Subjective Personal Presence
● Perceptual Immersion
● Spatial Presence, Place Illusion
● Realism, Fidelity, Scene Realism,
Experiential Fidelity
● Multimodal Presentation, Telepresence
● Separation Anxiety Bye, K. (2021, June 2) "State of Privacy in XR & Neuro-Tech: Conceptual Frames" Presentation at the
VRARA Global Summit (Online). Recording available at https://www.youtube.com/watch?v=pIpD4-gYImU
● Telepresence Mental Model
● Co-Presence with Others
● Anticipation of Events
● Consistency of
Information
● Interface Awareness
● Communicative
Immersion
● Plausibility Illusion
● Coherence
● Reality Judgment
● Cognitive Absorption
● Social Presence
● Selective Attention
● Psychological
Immersion
38. Bye, K. (2022, May 5). A Primer on Presence, Immersive Storytelling, & Experiential Design. [Presentation] Storycon: A collision of (he)art and technology. Brussels, Belgium;
BOZAR - Centre for Fine Arts. Video available at https://www.youtube.com/watch?v=n_2_qvIymt4. Photo by Amir Khidojatov
A Primer on Presence,
Immersive Storytelling,
& Experiential Design
39.
40. Bye, K. (2023, September 26). #1272: Kickoff of Venice Immersive 2023 coverage with winner “Songs by a Passerby” and Atmospheric Storytelling. Voices of VR Podcast.
https://voicesofvr.com/1272-kickoff-of-venice-immersive-2023-coverage-with-winner-songs-by-a-passerby-and-atmospheric-storytelling/
Using Presence to Understand Immersive Stories
See my 30+ hours of Venice Immersive Coverage
41. Bye, K. (2023, September 26). #1272: Kickoff of Venice Immersive 2023 coverage with winner “Songs by a Passerby” and Atmospheric Storytelling. Voices of VR Podcast.
https://voicesofvr.com/1272-kickoff-of-venice-immersive-2023-coverage-with-winner-songs-by-a-passerby-and-atmospheric-storytelling/
Visualizing Presence Priority for Venice Immersive 2023 Experiences
42. Kent, B., Pola, W., Mikhaleva, A., & Pietrobon, A. (2023, September 27). #1305: Critics roundtable for Venice immersive 2023 breaking down all 43 projects. Voices of VR Podcast.
https://voicesofvr.com/1305-critics-roundtable-for-venice-immersive-2023-breaking-down-all-43-projects/
3.5-hour Breakdown of All 43 Stories & Characters
from Venice Immersive 2023 XR Experiences
46. Image retrieved from Forsyth, K (2013, January) on November 13, 2021:
https://web.archive.org/web/20130420041820/http://www.kateforsyth.com.au/images/Blog%20January%202013/ThreeActStructureFlat.jpg
Fully Traditional Stories
Authored
Narrative
Generative
Narrative
48. Authored
Narrative
Generative
Narrative
Laskow, S. (2017, June 13).These maps reveal the hidden structures of 'Choose your own adventure' books.Atlas Obscura. Retrieved November 14, 2021, from
https://www.atlasobscura.com/articles/cyoa-choose-your-own-adventure-maps.
Branching Path Stories
52. Participatory Experience
with Dynamic
Waves of Potential
Abouhaib,A. (2014). Retrieved on November 13, 2021 from
http://ffden-2.phys.uaf.edu/webproj/212_spring_2014/Abdel_Abouhaib/101559128653564b3256663/uploads/2/8/6/1/28617611/1047737_orig.jpg
Passive Experience
with Static
Particle-like Actuals
57. Are You a Ghost or an Embodied Character?
Does You Have Impact on the Story?
Bye, K., & Dolan, D. (2016, February 4) #292:The Four DifferentTypes of Stories inVR.Voices ofVR Podcast. Retrieved November 14, 2021 from
https://voicesofvr.com/292-the-four-different-types-of-stories-in-vr/.
58. Passi, E. (2017, January 10). Bringing the Westworld attribute matrix to life. Medium. Retrieved November 14, 2021, from
https://epassi.medium.com/recreating-the-westworld-attribute-matrix-3e72d9d419df.
59. Virtue Continuum
Lanctot, J. D., Irving, J. A. (2010). Character and Leadership: Situating Servant Leadership in a Proposed Virtues Framework. International Journal of Leadership Studies, Vol. 6 Iss. 1,
2010. Retrieved November 14, 2021 from https://www.regent.edu/acad/global/publications/ijls/new/vol6iss1/2_Final%20Edited%20Lanctot%20and%20Irving_pp%2028-50.pdf via
https://www.regent.edu/acad/global/publications/ijls/new/vol6iss1/home.htm
60. Big Five Personality Characteristics
Barrick, M. R., & Mount, M. K. (1991). The Big Five Personality Dimensions and Job Performance: a Meta-analysis. Personnel Psychology, 44(1), 1–26.
doi:10.1111/j.1744-6570.1991.tb00688.x. Image via https://commons.wikimedia.org/wiki/File:Wiki-grafik_peats-de_big_five_ENG.png
61. ● Sacharin, V., Schlegel, K., & Scherer, K. R. (2012). Geneva Emotion Wheel rating study (Report). Geneva, Switzerland: University of Geneva, Swiss Center for Affective
Sciences. Retrieved on November 13, 2021 from https://www.unige.ch/cisa/files/4514/6720/4016/Geneva_Emotion_Wheel_Rating_Study_Report_2012_08_11_2.0.pdf.
● Tschannen-Moran, B. (2012, February). Appreciative Empathy: New Frameworks for New Conversations. International Journal of Appreciative Inquiry: AI Practitioner.
Volume 14 Number 1. Retrieved November 14, 2021 from http://www.schooltransformation.com/wp-content/uploads/2012/06/Appreciative_Empathy.pdf
Feelings Needs
63. Modified Milgram and Kishino's Continuum
[aka Extended Reality (XR) Spectrum]
Concepts via Milgram, P. and Kishino, F. (1994) A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information and Systems,
E77-D, 1321-1329. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.102.4646
Layout via Bye, K. (2017, May 18). Google's @claybavor presents the "Immersive Computing Spectrum" graphic at Google I/O 2017. Twitter. Retrieved October 20, 2021, from
https://twitter.com/kentbye/status/865244388057694208.
Physical Environment
Virtual Environment
Physical
Reality
Augmented
Reality (AR)
Virtual Reality
(VR)
Mixed Reality (MR)
viaVR Passthrough
64. Microsoft (2020, August 26). What is mixed reality? Retrieved April 02, 2021, from https://docs.microsoft.com/en-us/windows/mixed-reality/discover/mixed-reality
AR Modifies Your Existing Context
VR Completely Switches Your Existing Context
65. Microsoft (2020, August 26). What is mixed reality? Retrieved April 02, 2021, from https://docs.microsoft.com/en-us/windows/mixed-reality/discover/mixed-reality
AR Modifies Your Existing Context
VR Completely Switches Your Existing Context
66. Microsoft (2020, August 26). What is mixed reality? Retrieved April 02, 2021, from https://docs.microsoft.com/en-us/windows/mixed-reality/discover/mixed-reality
AR Modifies Your Existing Context
VR Completely Switches Your Existing Context
67. Meta is Working Towards “Contextually-Aware AI” in AR
Teaching AI to perceive theWorldThroughYour Eyes. Meta AI. (2021, October 14). Retrieved January 4, 2023, from
https://ai.facebook.com/blog/teaching-ai-to-perceive-the-world-through-your-eyes/
68. “Where did I put grandma’s watch?”
Meta’s Speculative Example of AR + Episodic Memory AI
Teaching AI to perceive theWorldThroughYour Eyes. Meta AI. (2021, October 14). Retrieved January 4, 2023, from
https://ai.facebook.com/blog/teaching-ai-to-perceive-the-world-through-your-eyes/
69. Egocentric 4D perception (EGO4D). (2021, October 14). Retrieved January 5, 2023, from https://ego4d-data.org/#challenges
Meta’s Ego4D Challenges Push the Limits of What
Egocentric POV, Contextually-Aware AI Can Do
70. ● Egocentric 4D perception (EGO4D). (2021, October 14). Retrieved January 5, 2023, from https://ego4d-data.org/#challenges
● Teaching AI to perceive the World ThroughYour Eyes. Meta AI. (2021, October 14). Retrieved January 4, 2023, from
https://ai.facebook.com/blog/teaching-ai-to-perceive-the-world-through-your-eyes/
Meta’s Ego4D Challenges Push the Limits of What
Egocentric POV, Contextually-Aware AI Can Do
What
happened
when?
What
will I do
next?
What am
I doing
& how?
Who said
what
when?
How are we
interacting?
71. “The future of HCI demands an exceptionally
easy-to-use, reliable, and private interface that
lets us remain completely present in the real
world at all times. That interface will require
many innovations in order to become the
primary way we interact with the digital world.
Two of the most critical elements are
contextually-aware AI that understands your
commands and actions as well as the context
and environment around you, and technology to
let you communicate with the system
effortlessly — an approach we call
ultra-low-friction input. The AI will make deep
inferences about what information you might
need or things you might want to do in various
contexts, based on an understanding of you
and your surroundings, and will present you
with a tailored set of choices.”
Inside Facebook Reality Labs:Wrist-based interaction for the next computing platform.Tech at Meta. (2021, March 18). Retrieved January 4, 2023, from
https://tech.facebook.com/reality-labs/2021/3/inside-facebook-reality-labs-wrist-based-interaction-for-the-next-computing-platform/
Meta Sees “Contextually-Aware AI” as Future of HCI
72. Image viaYan Shvartzshnaider (April 15, 2020) https://twitter.com/ynotez/status/1250578500588879873
Nissenbaum, H. (2010). Privacy in Context:Technology, Policy, and the Integrity of Social Life. Stanford University Press.
73. HumanCenteredDesignE. (2021, March 9). 2021 Distinguished Lecture: Helen Nissenbaum, "Contextual Integrity".YouTube. https://www.youtube.com/watch?v=VPwmC0Sfe50.
74. Role of Context in Dr. Anita Allen’s Four Needs for a
Comprehensive Philosophy of Privacy
Allen,A. L. (2019, October 31). Presidential Address,“The Philosophy of Privacy and Digital Life,” 93 Proceedings of the American Philosophical Association 21-38 (2019).,
Available at SSRN: https://ssrn.com/abstract=4022657
“A comprehensive philosophy of privacy needs
1) an account of definition and meaning;
2) an account of value, ethical and other
values;
3) an account of political dimensions of privacy;
and
4) context-specific practical applications and
implications that demonstrate the
explanatory power and import of the
accounts defended.”
75. Solove, D. J. (2010). Understanding Privacy. Harvard University Press.
“In terms of generality, I argue that
privacy should be conceptualized from
the bottom up rather than the top down,
from particular contexts rather than in
the abstract…”
“In focusing on privacy problems, my
approach seeks to be contextual
without being overly tied to specific
contexts, flexible enough to
accommodate changing attitudes
toward privacy, yet firm enough to
remain stable and useful.”
Solove’s Approach to Privacy & Context
76. ● Bye, K., & Segall, M. (2020, December 10). #965: Primer onWhitehead's Process Philosophy as a Paradigm Shift & Foundation for Experiential Design. Voices ofVR Podcast. Retrieved
January 5, 2023, from https://voicesofvr.com/primer-on-whiteheads-process-philosophy-as-a-paradigm-shift-foundation-for-experiential-design/
● Bye, K., & Maxwell, G. (2022, October 27). #1147:Thirteen Philosophers on the Problem of Opposites: Grant Maxwell's Integration & Difference Book & Archetypal Approaches to
Character.Voices ofVR Podcast. Retrieved January 5, 2023, from
https://voicesofvr.com/1147-thirteen-philosophers-on-the-problem-of-opposites-grant-maxwells-integration-difference-book-archetypal-approaches-to-character/
● Kent, B., & Segall, M. (2023, March 9). #1183: From Kant to an organic view of reality: Scaffolding a process-relational paradigm shift withWhitehead scholar Matt Segall.Voices ofVR
Podcast. https://voicesofvr.com/1183-from-kant-to-an-organic-view-of-reality-scaffolding-a-process-relational-paradigm-shift-with-whitehead-scholar-matt-segall/
Process-Relational Thinking Helps to Frame
the Contextual & Relational Nature of XR Privacy
77. “Process philosophy is based on the premise that being is
dynamic and that the dynamic nature of being should be
the primary focus of any comprehensive philosophical
account of reality and our place within it. Even though we
experience our world and ourselves as continuously
changing, Western metaphysics has long been obsessed
with describing reality as an assembly of static individuals
whose dynamic features are either taken to be mere
appearances or ontologically secondary and derivative.”
“Process Philosophy” as explained by
Stanford Encyclopedia of Philosophy
Seibt, J. (Fall 2021 Edition) "Process Philosophy", The Stanford Encyclopedia of Philosophy, Edward N. Zalta (ed.). Retrieved on December 10, 2021 from
https://plato.stanford.edu/archives/fall2021/entries/process-philosophy/.
78. Process-Relational Metaphysics =
Building Blocks of Process,
Potential, & Relationships
Bye, K., & Segall, M. (2023, March 9). #1183: From Kant to an organic view of reality: Scaffolding a process-relational paradigm shift with Whitehead scholar Matt Segall. Voices of VR. Retrieved March
10, 2023, from https://voicesofvr.com/1183-from-kant-to-an-organic-view-of-reality-scaffolding-a-process-relational-paradigm-shift-with-whitehead-scholar-matt-segall/
Substance Metaphysics =
Building Blocks of Materiality
& stuff that’s been actualized
79. Seibt: “Processes have mereological structure”
Seibt, J. (2015). Non-Transitive Parthood, Leveled Mereology, and the Representation of Emergent Parts of Processes. Grazer Philosophische Studien, 91(1), 165-190.
https://doi.org/10.1163/9789004302273_008
80. ● Varzi, Achille, (Spring 2019 Edition) "Mereology", The Stanford Encyclopedia of Philosophy, Edward N. Zalta (ed.), Retrieved on December 10, 2021 from
https://plato.stanford.edu/archives/spr2019/entries/mereology/
● Calosi, C., Fano, V., & Tarozzi, G. (2011). Quantum Ontology and Extensional Mereology. Foundations of Physics, 41(11), 1740–1755. doi:10.1007/s10701-011-9590-z
● Burkhardt, H., Seibt, J., Imaguire, G., & Gerogiorgakis, S. (2017). Handbook of Mereology. Philosophia Verlag GmbH.
● Cotnoir, A. J., & Varzi, A. C. (2021). Mereology. Oxford University Press.
Mereology: Wholes & Parts Context
“Mereology (from the Greek μερος,
‘part’) is the theory of parthood
relations: of the relations of part to
whole and the relations of part to
part within a whole.”
- Achille
82. Eastman, T. E. (2020). Untying the Gordian Knot: Process, reality and context. (pp. 3) Rowman & Littlefield.
Eastman: “Real-world interactions always involve context”
“Real-world interactions always involve context, and
some local-contextual framing inevitably occurs.
Whereas scientific description and approximations
very often strip away context and so enable discrete
measurement results and simple input-output
pairings (dyads), real-world interactions always
require triads, cycles of input-output-context, or, at
least, some inevitable reference to a third.”
-Tim Eastman
83. Identity is Socially Constructed & Contextual
Young, K., & The Stoa. (2020, October 12). Internet identity and Our Digital Selves with Kaliya Young (Identity Woman). Retrieved April 02, 2021, from
https://www.youtube.com/watch?v=q-9h7v4N39s
84. Juarrero, A. (2023). Context Changes Everything: How constraints create coherence. The MIT Press.
“Identity and individuation must be reframed as fluid
yet metastable and persistent dynamic
interdependencies forged by constraints in particular
contexts and historical moments.The collective
properties that irreversibly emerge from such
interactions among individuals are inherently
relational, reflecting the contextually-constrained
interactions that generated them.”
Juarrero’s Context Changes Everything
85. Diller, A., Ayim, M., Pauly Morgan, K., & Houston, B. (1996). Figure 8.1 (p. 107). In The Gender Question in Education: Theory, Pedagogy, & Politics. Routledge, Taylor et Francis Group.
Intersecting Axes of Privilege, Domination, & Oppression
86. Alex McDowell’s World Building Mandala
McDowell, Alex. (2020, January 25). Storytelling shapes the future. Retrieved March 03, 2021, from https://www.theantiagency.org/storytelling-shapes-the-future/
87. Lessig’s Pathetic Dot Theory
Market
Law
Cultural
Norms
Technology Architecture
& Code
User
Lessig, Lawrence, (2006). Code 2.0, Chapter: What Things Regulate ( pp. 120–137). New York : Basic Books
88. Lessig’s Pathetic
Dot Theory
Nested Contexts
vs
Market
Law
Cultural
Norms
Technology
Architecture
& Code
User
Lessig, Lawrence, (2006). Code 2.0, Chapter: What Things Regulate ( pp. 120–137). New York : Basic Books
96. Network Architecture
XR Hardware
Operating System & Game Engine
Apps, Experiences, & Context
User Experience
(Qualities of Presence, Character, Story, etc.)
Design Guidelines
Economy
Laws
Culture
Earth
OS Code
App Code
User
Experience
Economy
Laws
Culture
Design
Guidelines
Network
Architecture
Earth
XR Hardware
97. Bye, K. (2016, April 28). The Human Experience of Virtual Reality: A Model of the VR Landscape [Presentation],
Silicon Valley Virtual Reality Conference, April 28, 2016 (San Jose, CA). Published on YouTube, May 3, 2016. Retrieved on November 10, 2021 from
https://www.youtube.com/watch?v=acp0Ul0QlmI. Also Published on Voices of VR Podcast on May 4, 2016. Retrieved on November 10, 2021 from
https://voicesofvr.com/355-the-human-experience-of-virtual-reality-a-model-of-the-vr-landscape/.
99. Bye, K., Ashmore, L., Niles, S. Rizzo, S., Richir, S., Luo, V., Bouchet, A., LeBlanc., M., Pallot., M. (2019, March 19). "Future Dreaming: Designing for New Realities &
Mapping the Ethical Landscape of VR/AR." Laval Virtual Visionaries' Think Tank 2019. Presented at Laval Virtual Conference. Retrieved on November 10, 2021 from
https://twitter.com/kentbye/status/1108116834186919937 (2019, March 19).
100. AWE - Augmented World Expo (Presented on 2019, May 31, 2019. Published 2019, June 20). Kent Bye (Voices of VR Podcast): The Ethical & Moral Dilemmas of Mixed
Reality. Presented at Augmented World Expo USA, Santa Clara, CA. YouTube Retrieved on November 9, 2021 from https://www.youtube.com/watch?v=BNgXKX8IwcU.
101. AWE - Augmented World Expo (Presented on 2019, May 31, 2019. Published 2019, June 20). Kent Bye (Voices of VR Podcast): The Ethical & Moral Dilemmas of Mixed
Reality. Presented at Augmented World Expo USA, Santa Clara, CA. YouTube Retrieved on November 9, 2021 from https://www.youtube.com/watch?v=BNgXKX8IwcU.
102. XR Ethics Manifesto
Bye, K. (2019, October 19). XR Ethics Manifesto [Presentation], Greenlight XR Strategy Conference, October 18, 2019 (San Francisco, CA). Available at
https://www.slideshare.net/kentbye/xr-ethics-manifesto-updated-nov-2-2019 (Accessed November 9, 2021) and https://www.youtube.com/watch?v=CXgY3YXxqJ8
103.
104. Diversity,
Inclusion &
Accessibility
Virtual Clones
&
Identity
Business, Finance,
& Economics
Education
Metaverse Governance
Social Spaces:
Trolling,
Harassment,
& Online
Safety
Medicine
Erosion of Anonymity
& XR Privacy
IEEE Global Initiative on the Ethics of Extended Reality
● IEEE Standards Association (2022, June 6). IEEE Global Initiative on the Ethics of Extended Reality, Institute of Electrical and Electronics Engineers Standards Association.
Retrieved on July 6, 2022 from https://standards.ieee.org/industry-connections/ethics-extended-reality/.
● Bye, K. (2022, June 6). #1090: IEEE XR Ethics: Diversity, Inclusion, & Accessibility. Voices of VR Podcast. Retrieved July 6, 2022, from
https://voicesofvr.com/1090-ieee-xr-ethics-diversity-inclusion-accessibility/.
105. ● Experiential Design &
Immersive Storytelling Framework
○ Quality
○ Story
○ Character
○ Context
● Promises and Perils of XR
113. Artists: Android Jones, Marshmallow Laser Feast, Marshmallow Laser Feast, & Kevin Mack
Artists Blazing New Neural Pathways
114. Creation Tools
Doronichev, A. (2016, April 5). Tilt Brush: Painting from a new perspective. Google. Retrieved March 10, 2023, from https://blog.google/products/google-ar-vr/tilt-brush/
115. Fowler, E. (2005, November 28). Reality 1024x768 by EranFowler on DeviantArt. DeviantArt. Retrieved March 10, 2023, from
https://www.deviantart.com/eranfowler/art/Reality-1024x768-25788560
Escapism & Addiction
116. Adult Content & No Age Verification
YTMINI. (2021, March 2). Going to the strip club in VRChat. YouTube. Retrieved March 10, 2023, from https://www.youtube.com/watch?v=iI2hSbECokU
Bye, K., & Darling, E. (2018, March 19). #634: VR Porn Livestreaming & emotional intimacy with CAM4VR. Voices of VR. Retrieved March 10, 2023, from
https://voicesofvr.com/634-vr-porn-livestreaming-emotional-intimacy-with-cam4vr/
117.
118. VR Curing Lazy Eye
Concept of Neuroplasticity
Bye, K., & Blaha, J. (2014, June 2). #11: James Blaha's Diplopia VR game helped him see in 3D in the real world for the first time of his life. Neuroplasticity & VR therapy for Lazy
Eye. Voices of VR. Retrieved March 10, 2023, from
https://voicesofvr.com/11-james-blahas-diplopia-vr-game-helped-him-see-in-3d-in-the-real-world-for-the-first-time-of-his-life-neuroplasticity-vr-therapy-for-lazy-eye/
119. Greenleaf, W. (2023, March 6). Virtual Reality, Machine Learning, Biosensing - Converging to Transform Healthcare. [Presentation] Slideshare. (Slide 55) Retrieved March 9, 2023,
from https://www.slideshare.net/waltergreenleaf/virtual-reality-machine-learning-biosensing-converging-to-transform-healthcare
Walter Greenleaf: “More than 300
Emerging Medical VR/AR Companies”
120. Greenleaf, W. (2023, March 6). Virtual Reality, Machine Learning, Biosensing - Converging to Transform Healthcare. [Presentation] Slideshare. Retrieved March 9, 2023, from
https://www.slideshare.net/waltergreenleaf/virtual-reality-machine-learning-biosensing-converging-to-transform-healthcare
Walter Greenleaf on Medical XR Use Cases
121. Training for High-Stakes Scenarios
BioFlightVR. (2017, July 7). VR's Healthcare Revolution: Transforming Medical Training at [Children’s Hospital Los Angeles]. YouTube. Retrieved March 9, 2023, from
https://www.youtube.com/watch?v=4om8g0u9a4M
132. Combining NeuroTech with VR
Bye, K., Russomanno, C., & Artuso, J. (2022, July 7). #1112: OpenBCI's project galea hands-on & fusion of biometric & physiological data inVR.Voices ofVR. Retrieved March 10, 2023, from
https://voicesofvr.com/1112-openbcis-project-galea-hands-on-fusion-of-biometric-physiological-data-in-vr/
Bernal, G. (2021,April 15). Developing Galea:An open source tool at the intersection ofVR and neuroscience. MIT Media Lab. https://www.media.mit.edu/posts/galea/.
134. Kröger, J.L., Lutz, O.HM., Müller, F. (2020). What Does Your Gaze Reveal About You? On the Privacy Implications of Eye Tracking. In: Friedewald, M., Önen, M., Lievens, E.,
Krenn, S., Fricker, S. (eds) Privacy and Identity Management. Data for Better Living: AI and Privacy. Privacy and Identity 2019. IFIP Advances in Information and Communication
Technology, vol 576. Springer, Cham. https://doi.org/10.1007/978-3-030-42504-3_15
Psychographic Inferences from Eye Gaze Data
135. ● Bye, K., & Reardon, T. (2021, March 30). #987: The Neuroscience of Neuromotor Interfaces + Privacy Implications with Facebook Reality Labs' Thomas Reardon. Voices
of VR Podcast. Retrieved January 4, 2023, from
https://voicesofvr.com/987-the-neuroscience-of-neuromotor-interfaces-privacy-implications-with-facebook-reality-labs-thomas-reardon-2/
● Bye, K., & Wetmore, D. (2019, September 12). #814: Neuroscience & VR: Using Muscles & EMG for neural interfaces with CTRL-Labs. Voices of VR Podcast. Retrieved
January 4, 2023, from https://voicesofvr.com/814-neuroscience-vr-using-muscles-emg-for-neural-interfaces-with-ctrl-labs/
Meta’s Non-Invasive Neural Interface Can Detect
Intention to Move Down to a Single Motor Neuron
136. Mental & Social Presence
Mental Thoughts
Cognitive Processes
Cognitive Load
Social Presence
Active Presence
Behaviors
Intention
Actions
Movement
Emotional Presence
Affective State
Emotional Sentiment
Facial Expression
Microexpressions
Embodied Presence
Stress / Arousal
Physiological Reactions
Eye Gaze /Attention
Body Language
Muscle Fatigue Bye, K. (2021, June 2) "State of Privacy in XR & Neuro-Tech: Conceptual Frames" Presentation at the
VRARA Global Summit (Online). Recording available at https://www.youtube.com/watch?v=pIpD4-gYImU
137. Joseph O'Hagan, Pejman Saeghe, Jan Gugenheimer, Daniel Medeiros, Karola Marky, Mohamed Khamis, and Mark McGill. (2022, December) Privacy-Enhancing Technology and
Everyday Augmented Reality: Understanding Bystanders’ Varying Needs for Awareness and Consent. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous
Technologies. 6, 4, Article 177, 48 pages. https://doi.org/10.1145/3569501. [pre-print image courtesy of author]
Bystander Privacy Spectrum of AR Data Processing
139. ● Farahany, N. A. (2023). The battle for your brain: Defending the right to think freely in the age of Neurotechnology. St. Martin's Press.
● Bye, K., & Farahany, N. (2023, March 9). #1184: "Battle for Your Brain" author Nita Farahany on Establishing Cognitive Liberty as a Human Right for Limits on Neurotechnologies & XR. Voices of
VR. Retrieved March 10, 2023, from https://voicesofvr.com/1184-battle-for-the-brain-author-nita-farahany-on-establishing-cognitive-liberty-as-a-human-right-for-limits-on-neurotechnologies-xr/
141. (1) The right to mental privacy
(2) The right to identity
(3) The right to agency
(4) The right to fair access to
mental augmentation
(5) The right to protection from
algorithmic bias
Proposed Neuro-Rights
● Yuste, R.,Genser, J. & Herrmann, S. "It's Time for Neuro-Rights." Horizons: Journal of International Relations and Sustainable Development, no. 18, 2021. pp 154-164. JSTOR,
https://www.cirsd.org/en/horizons/horizons-winter-2021-issue-no-18/its-time-for-neuro--rights. Accessed 31 Mar. 2021.
● Bye, K., & Yuste, R. (2021, June 4). #994: Neuro-Rights Initiative: A human rights approach to preserving mental privacy with Rafael Yuste. Voices of VR Podcast.
https://voicesofvr.com/994-neuro-rights-initiative-a-human-rights-approach-to-preserving-mental-privacy-with-rafael-yuste/
145. Photorealistic Codec Avatars (2022)
Meta. (2022, October 11). Meta Connect Keynote 2022. YouTube. Retrieved March 9, 2023, from https://www.youtube.com/watch?v=hvfV-iGwYX8
147. Snapchat Dysmorphia from Facial Filters
Migala, J. (2018, October 16). What is 'snapchat dysmorphia'? A detailed look at the trend. EverydayHealth.com. Retrieved March 10, 2023, from
https://www.everydayhealth.com/wellness/united-states-of-stress/what-snapchat-dysmorphia-detailed-look-trend/
148. Consciousness Hacking
Free trial of Healium. Healium. (n.d.). Retrieved March 10, 2023, from https://www.tryhealium.com/free-trial/
149. Sensory Substitution
Bye, K., & Eagleman, D. (2017,April 18). #527:Turning your torso into an ear: Sensory substitution & addition with Neosensory. Voices ofVR. Retrieved March 10, 2023, from
https://voicesofvr.com/turning-your-torso-into-an-ear-sensory-substitution-addition-with-neosensory/
150. Sensory Addition
Bye, K., & Matzner , E. (2017, May 5). #532:The extremes of mental presence: Cognitive enhancement, biohacking, psychedelics, & transhumanism. Voices ofVR. Retrieved March 10, 2023, from
https://voicesofvr.com/532-the-extremes-of-mental-presence-cognitive-enhancement-biohacking-psychedelics-transhumanism/
154. Early Education
Bye, K., & Frazier, K. (2020, January 24). #882VR for good: Kai XR's inclusive & Accessible Education Kits.Voices ofVR. Retrieved March 10, 2023, from
https://voicesofvr.com/882-vr-for-good-kai-xrs-inclusive-accessible-education-kits/
155. VR Minimum Age of 13
https://pixexid.com/image/a-child-wearing-a-virtual-reality-headset-co8tjkoa
156.
157. Volumetric Memories
Bye, K., & Sherrill , R. (2016, July 24). #405: Memory Capture &Virtual tourism for people with physical disabilities.Voices ofVR. Retrieved March 10, 2023, from
https://voicesofvr.com/405-memory-capture-virtual-tourism-for-people-with-physical-disabilities/
158. Connect to the Earth
Bye, K., Podwal, M., & Kaeser, D. (2016, November 16). #475: Designing google earthVR:The Overview Effect & Finding Common Ground. Voices ofVR. Retrieved March 10, 2023, from
https://voicesofvr.com/designing-google-earth-vr-the-overview-effect-finding-common-ground/
159.
160. ● Egocentric 4D perception (EGO4D). (2021, October 14). Retrieved January 5, 2023, from https://ego4d-data.org/#challenges
● Teaching AI to perceive the World ThroughYour Eyes. Meta AI. (2021, October 14). Retrieved January 4, 2023, from
https://ai.facebook.com/blog/teaching-ai-to-perceive-the-world-through-your-eyes/
Meta’s Ego4D Challenges Push the Limits of What
Egocentric POV, Contextually-Aware AI Can Do
What
happened
when?
What
will I do
next?
What am
I doing
& how?
Who said
what
when?
How are we
interacting?
161.
162. Virtual Dates
Bye, K., & Hunting, J. (2022, January 21). #1045: Process of shooting a feature-length documentary in VRChat: Director Joe Hunting on "We Met in Virtual Reality" premiering at Sundance. Voices of
VR. Retrieved March 10, 2023, from https://voicesofvr.com/1045-process-of-shooting-a-feature-length-documentary-in-vrchat-director-joe-hunting-on-we-met-in-virtual-reality-premiering-at-sundance/
163. VR as Empathy Machine?
● Milk, C. (2015, April 23). How virtual reality can create the ultimate empathy machine. TED Talk. Retrieved March 10, 2023, from
https://www.ted.com/talks/chris_milk_how_virtual_reality_can_create_the_ultimate_empathy_machine?language=en
● Bye, K., & Arora, G. (2017, January 31). #499: VR as the Ultimate Empathy Machine with Gabo Arora. Voices of VR. Retrieved March 10, 2023, from
https://voicesofvr.com/vr-as-the-ultimate-empathy-machine-with-the-uns-gabo-arora/
164. Bye, K., Outlaw, J., & Cortese, M. (2022, June 4). #1089: IEEE Global Initiative on the Ethics of Extended Reality: Trolling, harassment, and online safety. Voices of VR Podcast.
https://voicesofvr.com/1089-ieee-global-initiative-on-the-ethics-of-extended-reality-trolling-harassment-and-online-safety/
165. Diller, A., Ayim, M., Pauly Morgan, K., & Houston, B. (1996). Figure 8.1 (p. 107). In The Gender Question in Education: Theory, Pedagogy, & Politics. Routledge, Taylor et Francis Group.
Marginalized Communities are Disproportionately Harrassed
166.
167. Grief Rituals
Bye, K., & Smith, P. (2018, November 8). #713: Moving through grief of suicide through a spatial story in VR with "homestay". Voices of VR. Retrieved March 10, 2023, from
https://voicesofvr.com/713-moving-through-grief-of-suicde-through-a-spatial-story-in-vr-with-homestay/
168. Funerals
Bye, K., & Demos, A. (2023, April 3). #1192: The last moments of AltSpaceVR, Athena Demos’ Eulogy &; Retrospective Journey Into Social VR. Voices of VR Podcast.
https://voicesofvr.com/1192-the-last-moments-of-altspacevr-athena-demos-eulogy-retrospective-journey-into-social-vr/
169.
170.
171. Training is One of VR’s Killer Apps
Bye, K., & Casale, M. (2016, September 1). #429: Revolutionizing VR Sports Training with STRIVR Labs. Voices of VR. Retrieved March 10, 2023, from
https://voicesofvr.com/429-revolutionizing-vr-sports-training-with-strivr-labs/
Bye, K., & Casale, M. (2019, September 12). #819: Neuroscience & VR: STRIVR is Leading the VR Training Revolution from Elite NFL Athletes to Walmart Employees. Retrieved March 10, 2023, from
https://voicesofvr.com/819-neuroscience-vr-strivr-is-leading-the-vr-training-revolution-from-elite-nfl-athletes-to-walmart-employees/
174. Bye, K., & Heller, B. (2021, April 8). #988: Defining “biometric psychography” to fill gaps in privacy law to cover XR Data: Brittan Heller’s Human Rights Perspectives. Voices of
VR Podcast. https://voicesofvr.com/988-defining-biometric-psychography-to-fill-gaps-in-privacy-law-to-cover-xr-data-brittan-hellers-human-rights-perspectives/
175. Speculative World Building
Bye, K., & Young , L. (2022, July 7). #1109: Tribeca XR: "Planet City VR" is a speculative architecture provocation on who we want to be in the future. Voices of VR. Retrieved
March 10, 2023, from https://voicesofvr.com/1109-tribeca-xr-planet-city-vr-is-a-speculative-architecture-provocation-on-who-we-want-to-be-in-the-future/
176. Indigenous Futurism
Bye, K., Van Loon (Ahnahktsipiitaa), C., & Monkman, J. (2022, January 28). #1050: Embedding Indigenous Relationality & Practice of bearing witness in "this is not a ceremony" 360 video. Voices of
VR Podcast. Retrieved March 10, 2023, from https://voicesofvr.com/1050-embedding-indigenous-relationality-practice-of-bearing-witness-in-this-is-not-a-ceremony-360-video/
177. Afrofuturism
Bye, K., et al. (2017, March 3). #511: Neurospeculative Afrofeminism: Building the future you want to live into. Voices of VR. Retrieved March 10, 2023, from
https://voicesofvr.com/511-neurospeculative-afrofeminism-building-the-future-you-want-to-live-into/
179. Wonder & Awe
Bye, K., & Mack, K. (2019, August 12). #798: VR artist Kevin Mack: Architecting Awe with VR-native, cooperative AI agents. Voices of VR. Retrieved March 10, 2023, from
https://voicesofvr.com/798-vr-artist-kevin-mack-architecting-awe-with-vr-native-cooperative-ai-agents/
180. David J. Chalmers on VR as a “Genuine Reality” (2022)
Bye, K., & Chalmers, D. (2022, January 11). #1043: Philosopher David Chalmers' book "reality+" may change how you see reality: VR is a genuine reality & we can't prove we're
not in a simulation. Voices of VR. Retrieved March 10, 2023, from
https://voicesofvr.com/1043-philosopher-david-chalmers-book-reality-may-change-how-you-see-reality-vr-is-a-genuine-reality-we-cant-prove-were-not-in-a-simulation/
181. Bye, K., & Segall, M. (2023, March 9). #1183: From Kant to an organic view of reality: Scaffolding a process-relational paradigm shift with Whitehead scholar Matt Segall. Voices of VR. Retrieved March
10, 2023, from https://voicesofvr.com/1183-from-kant-to-an-organic-view-of-reality-scaffolding-a-process-relational-paradigm-shift-with-whitehead-scholar-matt-segall/
Philosophical Paradigm Shift
Into a Process-Relational Philosophy
182. What is Truth? What is Reality?
Misinformation, Disinformation,
Fake News, and Malinformation
Bye, K., Panetta, F., & Burgund, H. (2019, December 12). #846 Doclab: Deep fake of a synthesized Nixon speech that never happened. Voices of VR Podcast.
https://voicesofvr.com/846-doclab-deep-fake-of-a-synthesized-nixon-speech-that-never-happened/
186. Workplace Attention Monitoring
Farahany, N. A. (2023). Chapter 2: Your Brain at Work. In The Battle for Your Brain: Defending the right to think freely in the age of Neurotechnology, St. Martin's Press.
187. Governmental Mass Surveillance
Data Leviathan: China's burgeoning Surveillance State. Human Rights Watch. (2020, October 28). Retrieved March 10, 2023, from
https://www.hrw.org/news/2019/08/16/data-leviathan-chinas-burgeoning-surveillance-state
192. Self-Regulation
Bye, K. & Leufer, D. (2023, March 7) #1177: How the EU’s AI Act Could Impact Biometric Data Definitions & XR Privacy. Voices of VR Podcast. Retrieved April 20, 2023 from
https://voicesofvr.com/1177-how-the-eus-ai-act-could-impact-biometric-data-definitions-xr-privacy/
AI Act Takes a Human Rights Approach & Not a Utilitarian One
193. ● Comonline. (2022, December 16). A fit for purpose and borderless European Artificial Intelligence Regulation. Telefónica. Retrieved April 20, 2023, from
https://www.telefonica.com/en/communication-room/blog/a-fit-for-purpose-and-borderless-european-artificial-intelligence-regulation/
● Bye, K. & Leufer, D. (2023, March 7) #1177: How the EU’s AI Act Could Impact Biometric Data Definitions & XR Privacy. Voices of VR Podcast. Retrieved April 20, 2023
from https://voicesofvr.com/1177-how-the-eus-ai-act-could-impact-biometric-data-definitions-xr-privacy/
AI Act Lays Out a Tiered Set of Risks
196. Accessibility
Bye, K., & Azenkot, S. (2023, July 12). #1222: Kickoff of XR Accessibility Series with XR Access Co-Founder Shiri Azenkot. Voices of VR Podcast.
https://voicesofvr.com/1222-kickoff-of-xr-accessibility-series-with-xr-access-co-founder-shiri-azenkot/