Virtual Architecture as Process: Visualizing the Geometry of Music

Virtual Architecture as Process:
Visualizing the Geometry of Music
& Dynamic Data Over Time
@KentBye

● Fusion of Affordances & Design
● Linear vs Iterative Design
● Process Philosophy & Potentiae
● Whole/Part Mereology + Context
● Perception as Process
● Immersive Data Visualization Trends
● Geometry of Music & Data Over Time

Mental & Social
Presence
Active
Presence
Emotional
Presence
Embodied &
Environmental
Presence

Making
Choices
Taking
Action
Emotional
Immersion
Sensory
Experience

VR & AR
Books
Film
Radio
Video Games
Internet
TV
Theatre
Mobile Phone
Oral
Storytelling

Alex McDowell’s Brief History of Storytelling
McDowell, Alex. (2020, January 25). Storytelling shapes the future. Retrieved March 03, 2021, from https://www.theantiagency.org/storytelling-shapes-the-future/

Participatory Experience
with Dynamic
Waves of Potential
http://ffden-2.phys.uaf.edu/webproj/212_spring_2014/Abdel_Abouhaib/101559128653564b3256663/uploads/2/8/6/1/28617611/1047737_orig.jpg
Passive Experience
with Static
Particle-like Actuals

Authored
Narrative
Generative
Narrative
Spectrum of Story Authorship
No Agency or
Impact on Story
Maximized Agency &
Expression of Will

Edition Distribution
Bernard Miège’s Definition of a Medium
Kaplan-Rakowski, R. , & Meseberg, K. (2019). Immersive media and their future. In R.M. Branch et al. (Eds.), Educational Media and Technology Yearbook (Vol. 42, pp. 143-153).
Springer. https://doi.org/10.1007/978-3-030-27986-8_13 citing Miège, B. (2003). La contribution des industries de la culture, de l'informationet de la communication à
l'informationnalisation et à la globalisation. Questions de communication, (3), 211-229 cited in
“adaptation of content into the
medium through editing,
filming, post-production, etc.”
“spread of the edited content to
make it accessible for the
audience”

Communications Medium as Process
Emerging Tech
Provides New
Aﬀordances

Emerging Tech
Provides New
Aﬀordances
Creators
Explore New
Aﬀordances

Emerging Tech
Provides New
Aﬀordances
Creators
Explore New
Aﬀordances
Distributors
Provide Channels
For Audiences to
Access New Work

Audiences
Learn How to
Watch New Work &
Provide Feedback
Emerging Tech
Provides New
Aﬀordances
Creators
Explore New
Aﬀordances
Distributors
Provide Channels
For Audiences to
Access New Work

Authored
Design
Generative
Structure

Joint Mathematics Meetings. (Recorded 2018, January 10, Published 2018, April 4). Gunnar Carlsson: "Topological Modeling of Complex Data." YouTube. Retrieved on November 9,
2021 from https://www.youtube.com/watch?v=8nUBqawu41k
Topological Data Analysis as a way to architect spatial relationships

Joint Mathematics Meetings. (Recorded 2018, January 10, Published 2018, April 4). Gunnar Carlsson: "Topological Modeling of Complex Data." YouTube. Retrieved on November 9,
2021 from https://www.youtube.com/watch?v=8nUBqawu41k

Title from Kagiyama, N., Shrestha, S., Farjo, P. D., & Sengupta, P. P. (2019). Artiﬁcial Intelligence: Practical Primer for Clinical Research in Cardiovascular Disease. Journal of the
American Heart Association, 8(17). doi:10.1161/jaha.119.012788
Image from Casaclang-Verzosa G, Shrestha S, Khalil MJ, Cho JS, Tokodi M, Balla S, Alkhouli M, Badhwar V, Narula J, Miller JD, Sengupta PP. Network tomography for understanding
phenotypic presentations in aortic stenosis. JACC Cardiovasc Imaging. 2019;12:236–248.
“Topological data analysis enables integration of multidimensional
complex data and visualization of hidden patterns in the data.”

Silver, A. (2019, June 10). Organizing complexity: How architects collaborate to build the world around us. Retrieved March 03, 2021,
from https://medium.com/swlh/organizing-complexity-how-architects-collaborate-to-build-the-world-around-us-2a508aeb83a1

Design phases - hmh modern architecture + interiors - boulder, co. (2017, November 17). Retrieved March 03, 2021, from https://hmhai.com/design-phases/

Alex
McDowell’s
Visualization
of the Film
Production
Process
+
Different
Roles.
McDowell, Alex. (2020, January 25). Storytelling shapes the future. Retrieved March 03, 2021, from https://www.theantiagency.org/storytelling-shapes-the-future/

Game Design
is a
continuously
Iterative
Process
Bromley, M. (2012, July 30). From the Virtual Teaching Frontlines: Game Design Summer Program. Joan Ganz Cooney Center. Retrieved November 9, 2021, from
https://joanganzcooneycenter.org/2012/07/30/from-the-virtual-teaching-frontlines-game-design-summer-program/.

Zahra, Z. (2017, October 07). Scrum methodology. Retrieved March 03, 2021, from https://zaynabzahrablog.wordpress.com/2017/10/07/scrum-methodology/

World Leaders in Research-Based User Experience. (n.d.). Design thinking 101. Retrieved March 03, 2021, from https://www.nngroup.com/articles/design-thinking/

Holcolmb, S., Klebahn, P., Segovia, K., & Utley, J. (2017, September 17). d.bootcamp 25 Take Oﬀ Presentation. Retrieved March 03, 2021 from
https://static1.squarespace.com/static/57c6b79629687fde090a0fdd/t/59bc54422aeba555ﬀ0041fe/1505514587533/MASTER+DECK_2017+Sept+25+Bootcamp.pdf

Gray, D. (2017, July 14). Empathy map. Retrieved March 03, 2021, from https://gamestorming.com/empathy-mapping/

What do they THINK?
What do they SAY?
What do they need to DO?
What do they DO?
What do they FEEL?
What do they SEE?
What do they HEAR?

Process Thought at a New Threshold - October 31, 2020
Cobb Institute. (Recorded 2020, October 31, Published 2020, November 28). Process Thought at a New Threshold [Video ﬁle]. YouTube. Retrieved on November 8, 2021
from https://www.youtube.com/watch?v=wl1jUJhw_zI

Interview with Whitehead Scholar Matt Segall on Process Philosophy
Bye, K., & Segall, M. D. (2020, December 10). #965: Primer on Whitehead’s Process Philosophy as a Paradigm Shift & Foundation for Experiential Design. Voices of VR podcast.
Retrieved November 8, 2021, from https://voicesofvr.com/primer-on-whiteheads-process-philosophy-as-a-paradigm-shift-foundation-for-experiential-design/.

Whitehead’s Process Philosophy
Whitehead, A. N. (1978). Process and reality. (pp. 59) Free Press.
"Every actual entity is what it is, and is with
its deﬁnite status in the universe,
determined by its internal relations to other
actual entities. 'Change' is the description of
the adventures of eternal objects in the
evolving universe of actual things.”
-Alfred North Whitehead

Chemistry
Biology
Psychology
Physics
Material Reality
Substance
Metaphysics
Whole/Part
Mereological
Relations of
Event Sequences
(Philosophy of Organism)
Process & Potential
Process-Relational
Metaphysics
vs

Whitehead’s Relational Event Ontology
Desmet, Ronald and Andrew David Irvine, "Alfred North Whitehead", The Stanford Encyclopedia of Philosophy (Fall 2018 Edition), Edward N. Zalta (ed.), Retrieved on November
8, 2021 from https://plato.stanford.edu/archives/fall2018/entries/whitehead/.
“The relational event ontology that
Whitehead developed in his London period
might serve to develop a relational
interpretation of quantum mechanics, such
as Rovelli’s [1996] or one of the many
proposed by Whitehead scholars (cf. Stapp
1993 and 2007, Malin 2001, Hättich 2004,
Epperson 2004, Epperson & Zaﬁris 2013).”
-Desmet et al.

● Rovelli, C. (1996). Relational quantum mechanics. International Journal of
Theoretical Physics, 35(8), 1637–1678. doi:10.1007/bf02302261
Epperson, Michael & Elias Zaﬁris, 2013, Foundations of Relational Realism,
Lanham, MD: Lexington Books.
● Epperson, Michael, 2004, Quantum Mechanics and the Philosophy of Alfred
North Whitehead, New York: Fordham University Press.
● Stapp, Henry, 1993, Mind, Matter, and Quantum Mechanics, Berlin: Springer
Verlag.
● Stapp, Henry, 2007, Mindful Universe: Quantum Mechanics and the Participating
Observer, Berlin: Springer Verlag.
● Malin, Shimon, 2001, Nature Loves to Hide: Quantum Physics and the Nature of
Reality, a Western Perspective, Oxford: Oxford University Press.
● Hättich, Frank, 2004, Quantum Processes: A Whiteheadian Interpretation of
Quantum Field Theory, Münster: Agenda Verlag.
Relational Interpretations of Quantum Mechanics

Eastman, T. E. (2020). Untying the Gordian Knot: Process, reality and context. Rowman & Littleﬁeld.
Cobb Institute. (Recorded 2021, August 21, Published 2021, August 29). Tim Eastman Unties the Gordian Knot - Session 3 [Video ﬁle]. YouTube. Retrieved on November 8,
2021 from https://www.youtube.com/watch?v=mcujSZojJ6Y.
Tim Eastman’s Untying the Gordian Knot: Process, Reality, & Context

Block Model of the Universe is Deterministic
with No Novelty, Creativity, or Participation as
Past, Present, Future have all already happened
Tegmark, M. (2015, October 1). Life is a braid in spacetime - Issue 29: Scaling. Nautilus. Retrieved November 8, 2021, from
https://nautil.us/issue/29/scaling/life-is-a-braid-in-spacetime-rp.

Michael Epperson "with Elias Zafiris, has formalized a
similar argument against the time-symmetric, actualist
classical block world ontology, proposing a topological
interpretation of quantum mechanics whereby
spatiotemporal extensiveness and its metrical structure
is emergent from dynamical topological quantum event
structures."
-Kastner et al.
Relational Realism argues against “classical block world”
Space is Emergent from “dynamical topological quantum event structures”
Kastner, R. E., Kauﬀman, S., Epperson, M. (2018, March 27) Taking Heisenberg's Potentia Seriously, International Journal of Quantum Foundations, Volume 4, Issue 2, pages
158-172. Retrieved on November 7, 2021 from https://ijqf.org/archives/4643.
Epperson, M., & Zaﬁris, E. (2013). Foundations of Relational Realism: A topological approach to quantum mechanics and the philosophy of nature. Lexington Books.

Eastman, T. E. (2020). Untying the Gordian Knot: Process, reality and context. (pp. 72) Rowman & Littleﬁeld.
“Physical things are the outcome of real histories of quantum events”
“The central thesis of the Relational Reality model is
that the classical, conventional conception of the
relationship between physical objects and their
presumed direct mapping to discrete facts must be
reconceived such that physical things are understood
as the outcome of real histories of quantum events.
This requires a reconceptualization of ontological and
contextual properties as mutually implicative features
of every quantum event (thus, inevitably, the triad of
input-output-context). In this Relational Realist
framework, quantum events are identified as
measurement outcomes that refer to corresponding
physical observables.The theory then provides the
means of relating these events.”
-Timothy Eastman

Why Quantum Potentiae should be considered as having ontological reality
Kastner, R. E., Kauﬀman, S., Epperson, M. (2018, March 27) "Taking Heisenberg's Potentia Seriously," International Journal of Quantum Foundations, Volume 4, Issue 2,
pages 158-172. Retrieved on November 7, 2021 from https://ijqf.org/archives/4643.

"I wish to view as physically real the possible
quantum events that might be, or might have
been, experienced. So, in this approach, those
possible events are real, but not actual; they exist,
but not in spacetime. The actual event is the one
that is experienced and that can be said to exist
as a component of spacetime."
-Ruth Kastner
Kastner, R. E. (2013). The transactional interpretation of quantum mechanics: The reality of possibility. (pp. 2) Cambridge University Press.
Images: Science Animated (2019, May 21) Quantumland and Transactions [Video ﬁle]. YouTube. Retrieved on November 8, 2021 from
https://www.youtube.com/watch?v=3J9JeJMAOBw.
“Possible events are real, but not actual; they exist, but not in spacetime”

Science Animated (2019, May 21) Quantumland and Transactions [Video ﬁle]. YouTube. Retrieved on November 8, 2021 from
Spacetime emergent from the quantum substrate & Potentiae
Actualized Reality
Quantum Potentiae

Actualizations = Boolean Logic
Quantum Potentiae = Non-Boolean Logic
“The Relational Reality model requires a
distinction between the logic of actualizations
(standard Aristotelian or Boolean logic) and a
non-Boolean logic for potentiae; real-world
application of these logics necessarily requires
three components; these are input-output-context
with context most often designated as
“environment” in quantum physics applications.”
-Tim Eastman

Science Animated (2019, May 21) Quantumland and Transactions [Video ﬁle]. YouTube. Retrieved on November 8, 2021 from
A partial Boolean algebra is
"locally Boolean, yet globally non-Boolean"
Actualized Reality = Boolean Logic
Quantum Potentiae = Non-Boolean Logic

Non-Boolean Logic with
Waves of Potential
http://ffden-2.phys.uaf.edu/webproj/212_spring_2014/Abdel_Abouhaib/101559128653564b3256663/uploads/2/8/6/1/28617611/1047737_orig.jpg
Boolean Logic with
Particle-like Actuals

Lebowitz, Josiah and Klug, Chris. (2011) "Interactive Storytelling forVideo Games." Focal Press.
Authored
Narrative
Generative
Narrative
Lebowitz & Klug’s Interactive Storytelling Spectrum

Authored
Narrative
Generative
Narrative
https://web.archive.org/web/20130420041820/http://www.kateforsyth.com.au/images/Blog%20January%202013/ThreeActStructureFlat.jpg
Fully Traditional Stories

Authored
Narrative
Generative
Narrative
Riedl, Mark. "Incorporating Authorial Intent into Generative Narrative Systems," Association for the Advancement of Artificial Intelligence, 2009.
Interactive Traditional Story

Authored
Narrative
Generative
Narrative
Chooseco LLC, Journey Under the Sea. Retrieved from http://www.cs.cmu.edu/afs/cs/project/oz/web/papers/CMU-CS-02-206.pdf
Branching Path Stories

Authored
Narrative
Generative
Narrative
https://i0.wp.com/www.raphkoster.com/wp-content/uploads/2013/04/CASlide7.jpg
Open-Ended Stories

Authored
Narrative
Generative
Narrative
Emily Short, Spirit AI, Game Developers Conference “Narrative Innovation Showcase” 2017.
Fully Player-Driven Stories

Primas, H. (2017). Knowledge and time. (pp. 3) Sprinter International PU.
Spacetime is Locally Boolean, but emergent from Globally Non-Boolean Quantum Potential
“A partial Boolean algebra is a family of Boolean
algebras whose operations coincide on overlaps
so that it is locally Boolean, yet globally
non-Boolean…
The family of Boolean contexts plays a privileged
role since a sound theory should be able to
describe intersubjectively communicable empirical
propositions.Therefore the overall non-Boolean
framework must have a locally Boolean logical
structure. This guarantees that aspects of reality
can be perceived by projections onto empirically
accessible Boolean reference frames.”
-Hans Primas

Whitehead’s “Eternal Objects” as Pure Potentials
“The eternal objects are the pure
potentials of the universe, and the actual
entities diﬀer from each other in their
realization of potentials.”
The mathematical (i.e. Platonic) world is real, physical world is a shadow of that pure, idea world. Mathematics and science. (n.d.). Retrieved November 9, 2021, from
http://abyss.uoregon.edu/~js/21st_century_science/lectures/lec02.html.

Quantum
Potentiae

Aristotle’s Formal Causation
● Math Structure
● Blueprint
● Design
● Shape
Quantum
Potentiae

Primas, H. (2010, May 1). Complementary Time Concepts. The Forgotten Present: A Quest for a Richer Concept of Time, Second Circular for the Parmenides Workshop.
Munich-Pullach; Germany. Retrieved on November 7, 2021 from https://www.yumpu.com/en/document/view/25159165/pdf-ﬁle-parmenides-foundation
“Full understanding of time is not possible within a Boolean Logic framework”

Primas, H. (2017). Knowledge and time. (pp. 6-7) Sprinter International PU.
Chronos vs Kairos Time

Monochronic vs Polychronic
https://web.archive.org/web/20130420041820/http://www.kateforsyth.com.au/images/Blog%20January%202013/ThreeActStructureFlat.jpg
Encyclopaedia Britannica, 1st ed., s.v. “Astronomy.” Chicago: Encyclopaedia Britannica, 1771.
Martineau, J. (2011). Quadrivium: The four classical liberal arts of number, geometry, Music, & Cosmology. Walker & Co.

Experience as streams of “concresence” processes
Desmet, Ronald and Andrew David Irvine, "Alfred North Whitehead", The Stanford Encyclopedia of Philosophy (Fall 2018 Edition), Edward N. Zalta (ed.), Retrieved on November
8, 2021 from https://plato.stanford.edu/archives/fall2018/entries/whitehead/.
“His theory of feelings claims that not only our
perception, but our experience in general is a
stream of elementary processes of concrescence
(growing together) of many feelings into one—“the
many become one, and are increased with one”
(1929c [1985: 21])—and that the process of
concrescence is not primarily driven by the
objective content of the feelings involved (their
factuality), but by their subjective form (their
valuation, cf. 1929c [1985: 240]).
-Desmet et al.

Unconscious analysis with high temporal resolution
then rendered conscious at discrete “percept” intervals?
Herzog, M. H., Kammer, T., & Scharnowski, F. (2016, April 12). Time Slices: What Is the Duration of a Percept? PLOS Biology, 14(4), e1002433.
doi:10.1371/journal.pbio.1002433
(a) Our sensory systems differ remarkably in terms of the temporal scale along which they can detect
the difference between two stimuli. Whereas two clicks can already be separated if they are only 1–3
ms apart [33], two taps need to be about 10 ms apart [33], and two flashes about 25 ms [34]. However,
for trains of stimuli, the presentation rate at which the sensation of flicker ceases is similar for the
visual and auditory systems, i.e., at around 16 ms inter-stimulus-interval (ISI) [35,36].
(b) Temporal order judgement within a sensory system is in the range of 20–50 ms ISI and has been
found not to differ substantially between the modalities [37–39]. However, under particular
circumstances the auditory system yields faster discrimination compared to the visual system [40].
(c) Comparing the order of events across different sensory systems has approximately the same
temporal resolution as within a sensory system [37]. This has been taken as evidence for a
supra-sensory timing mechanism that is involved in the detection of onsets across all modalities [41].
However, the lack of transfer of learned temporal order judgements between modalities questions the
existence of a supra-sensory timing mechanism [40].

Image: Mejia, K. W. (2015, September 7). Synchronicity and the story of Our lives: Time is art. Time is Art | a documentary series. Retrieved November 9, 2021, from
https://thesyncmovie.com/2015/09/07/synchronicity-and-the-story-of-our-lives/.
Text: Jung, C. G. (1979). Collected Works of C.G. Jung, volume 9 (part 2): Aion: Researches into the phenomenology of the self. Para 126. Princeton University Press.
Jungian theories about the unconscious
“The psychological rule says that
when an inner situation is not made
conscious, it happens outside, as fate.
That is to say, when the individual
remains undivided and does not
become conscious of his inner
opposite, the world must perforce act
out the conflict and be torn into
opposing halves.”
-C.G. Jung

The Yin & Yang of Right Brain & Left Brain Functions
McGilchrist, I. (2019). The master and his emissary: The Divided Brain and the making of the Western World. (pp. xii) Yale University Press.

Steinicke, F. (2021, March 31). B(l)ending Realities. Keynote lecture presented at The IEEE Conference on Virtual Reality and 3D User Interfaces in Spain (virtual),
Libson. Retrieved from https://www.youtube.com/watch?v=Ds-h1J4MFMI&t=13496s

Jacob, E. (2020, May 18). Pic.twitter.com/o5uh4edi5x. Twitter. Retrieved November 9, 2021, from https://twitter.com/i/status/1262452945573916673.
Architecture as Process

Chemistry
Biology
Psychology
Physics
Material Reality
Substance
Metaphysics
Whole/Part
Mereological,
Relations of
Event Sequences
(Philosophy of Organism)
Process & Potential
Process-Relational
Metaphysics
vs

Mereology: Wholes & Parts Context

Lessig’s Pathetic Dot Theory
Market
Law
Cultural
Norms
Technology
Architecture
& Code
User
Lessig, Lawrence, (2006). Code 2.0, Chapter: What Things Regulate ( pp. 120–137). New York : Basic Books

Lessig’s Pathetic
Dot Theory
Mereological
Nested Contexts
vs
Market
Law
Cultural
Norms
Technology
Architecture
& Code
User
Lessig, Lawrence, (2006). Code 2.0, Chapter: What Things Regulate ( pp. 120–137). New York : Basic Books

Economy
Laws
Culture
Design
Guidelines

User
Experience
Economy
Laws
Culture
Design
Guidelines

App Code
User
Experience
Economy
Laws
Culture
Design
Guidelines

OS Code
App Code
User
Experience
Economy
Laws
Culture
Design
Guidelines

Nested Contexts & Mereological Whole/Parts

“Real-world interactions always involve context”
“Real-world interactions always involve context, and
some local-contextual framing inevitably occurs.
Whereas scientific description and approximations
very often strip away context and so enable discrete
measurement results and simple input-output
pairings (dyads), real-world interactions always
require triads, cycles of input-output-context, or, at
least, some inevitable reference to a third.”
-Tim Eastman

Image viaYan Shvartzshnaider (April 15, 2020) https://twitter.com/ynotez/status/1250578500588879873
Nissenbaum, H. (2010). Privacy in Context:Technology, Policy, and the Integrity of Social Life. Stanford University Press.

ThingLink. (n.d.). Universal Declaration of Human Rights by TERMCAT.ThingLink.
https://www.thinglink.com/scene/885518633164341250.
FDR Presidential Library & Museum. (1949, November). Eleanor Roosevelt
holding poster of the Universal Declaration of Human Rights (in English), Lake
Success, NewYork. November 1949. Flickr.
https://www.flickr.com/photos/fdrlibrary/27758131387/.

United Nations. (2015, December 30). Sustainable Development Goals kick off with start of new year – United Nations Sustainable Development. United Nations.
https://www.un.org/sustainabledevelopment/blog/2015/12/sustainable-development-goals-kick-off-with-start-of-new-year/.

Stewart, M. (2021, June 17). Breadth & depth: Why I'm optimistic about Facebook's Responsible Innovation efforts. Facebook Technology.
https://tech.fb.com/responsible-innovation/.

Diller, A., Ayim, M., Pauly Morgan, K., & Houston, B. (1996). Figure 8.1 (p. 107). In The Gender Question in Education: Theory, Pedagogy, & Politics. Routledge, Taylor et Francis Group.
Intersecting Axes of Privilege, Domination, & Oppression

Klein, T. J., & Lewis, M. A. (2012). A physical model of sensorimotor interactions during locomotion.
Journal of Neural Engineering, 9(4), 046011. doi:10.1088/1741-2560/9/4/046011
Embodied Cognition as Nested Contexts

Different Models of Perception & Cognition

Attention
Active
Engagement
Error
Feedback
Consolidation
Dehaene, Stanislas. How We Learn: Why Brains Learn Better than Any Machine ... for Now. Viking, 2020.

Hinton, A. (2014, December). Understanding Context. O'Reilly Online Learning. Retrieved October 19, 2021, from
https://www.safaribooksonline.com/library/view/understanding-context/9781449326531/ch04.html.

Roth, C. (2019). Video Game Violence from the Perspective of Cognitive Psychology. Role Identification and Role Distancing in A WAY OUT. Violence | Perception | Video
Games, 53–62. doi:10.14361/9783839450512-006

Jain, A., Horowitz, A. H., Schoeller, F., Leigh, S., Maes, P., & Sra, M. (2020). Designing interactions beyond conscious control. Proceedings of the ACM on Interactive,
Mobile, Wearable and Ubiquitous Technologies, 4(3), 1-23. doi:10.1145/3411829
Cognitive processing stages and pathways for
external stimuli from the environment to the mind

Stefanics, G., Kremlácek, J., & Czigler, I. (2014). Visual mismatch negativity: a predictive coding view. Frontiers in Human Neuroscience, 8. doi:10.3389/fnhum.2014.00666
Retrieved Feb 20, 2020 from: https://www.researchgate.net/figure/Simplified-scheme-of-the-hierarchical-
predictive-coding-framework-Friston-2005-2008_fig1_266401430.
Alexander, W.H., & Brown, J.W. (2018). Frontal cortex function as derived from hierarchical predictive coding. Scientific Reports.
Predictive Coding Theory
of Neuroscience

Embodied
Experience
Test
Hypothesis
Adapt
Model
Synthesize
& Sleep

Klein, T. J., & Lewis, M. A. (2012). A physical model of sensorimotor interactions during locomotion. Journal of Neural Engineering, 9(4), 046011.
doi:10.1088/1741-2560/9/4/046011
Embodied Cognition as Nested Contexts

Lundy, M., Sutton, D., Ashton, A., Martineau, J. (2010) Quadrivium: The Four Classical Liberal Arts of Number, Geometry, Music, & Cosmology. Walker & Co.

Number in
Space & Time
Number
Number in
Time
Geometry Music
Mathematics
Astronomy
The Quadrivium
Number in
Space

Objects Moving
Through Space
Abstract Patterns
& Forms
Consonance
Dissonance Cycles
In Time
Geometry Music
Mathematics
Astronomy
The Quadrivium
Architecture &
Embodiment
in Space

Mental & Social
Presence
Active
Presence
Emotional
Presence
Embodied
Presence

Number
Math
Number in
Space & Time
Astronomy
Number in Time
Music
Number in Space
Geometry

Abstract
Patterns &
Forms
Objects Moving
Through Space
Consonance
Dissonance
Cycles
Architecture &
Embodiment

Geometry
Architecture &
Embodiment
in Space

Oliver Kreylos Geospatial Visualizations
Kreylos, O. (2013, August 5). Vrui on Oculus rift with Razer Hydra and Kinect - YouTube. Retrieved November 7, 2021, from
https://www.youtube.com/watch?v=IERHs7yYsWI.

Dubel, S., Rohlig, M., Schumann, H., & Trapp, M. (2014). 2D and 3D presentation of spatial data: A systematic review. 2014 IEEE VIS International Workshop on 3DVis (3DVis).
doi:10.1109/3dvis.2014.7160094 Retrieved November 7, 2021, from
http://blogs.evergreen.edu/vistas/ﬁles/2015/02/dubel-topost-2dvs3d-visieeevis2014_submission_3.pdf.
Tradeoffs of 2D vs 3D
● occlusion
● clutter
● distortion
● scalability

Dubel, S., Rohlig, M., Schumann, H., & Trapp, M. (2014). 2D and 3D presentation of spatial data: A systematic review. 2014 IEEE VIS International Workshop on 3DVis (3DVis).
doi:10.1109/3dvis.2014.7160094 Retrieved November 7, 2021, from
http://blogs.evergreen.edu/vistas/ﬁles/2015/02/dubel-topost-2dvs3d-visieeevis2014_submission_3.pdf.
Geospatial Focus on 3D Data Visualization

Bye, K., & Herzog, L. (2019, December 12). #851 DocLab: Using VR to Visualize Language Extinction with Lena Herzog’s “Last Whispers.” Voices of VR podcast. Retrieved
November 7, 2021, from https://voicesofvr.com/851-doclab-using-vr-to-visualize-language-extinction-with-lena-herzogs-last-whispers/.
Visualizing Languages Going to Sleep

Immersive Data Visualization Research Lagging:
Some Using 2D Panels in Virtual Environments
● Lee, B. (2020, October 30). Full Paper Session on Theory IEEE Visualization Conference 2020 . Retrieved November 7, 2021, from https://www.youtube.com/watch?v=5cWWPBE_DTQ.
● Lee, B., Brown, D., Lee, B., Hurter, C., Drucker, S., & Dwyer, T. (2021). Data visceralization: Enabling deeper understanding of data using virtual reality. IEEE Transactions on Visualization
and Computer Graphics, 27(2), 1095–1105. https://doi.org/10.1109/tvcg.2020.3030435, Retrieved November 7, 2021 from https://arxiv.org/abs/2009.00059
● Bastiras, J., & Thomas, B. H. (2017). Combining Virtual Reality and Narrative Visualisation to Persuade. 2017 International Symposium on Big Data Visual Analytics (BDVA).
doi:10.1109/bdva.2017.8114623
● D. Ren, B. Lee, and T. Hollerer. Xrcreator: Interactive construction of immersive data-driven stories. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and
Technology, VRST '18, pp. 136:1–136:2. ACM, New York, NY, USA, 2018. doi: 10.1145/3281505.3283400

Lee, B., Brown, D., Lee, B., Hurter, C., Drucker, S., & Dwyer, T. (2021). Data visceralization: Enabling deeper understanding of data using virtual reality.
IEEE Transactions on Visualization and Computer Graphics, 27(2), 1095–1105. https://doi.org/10.1109/tvcg.2020.3030435

Data Visceralization: Speed, Distance, Scale

Molecular Science Sonification
Bye, K., & Herzog, L. (2019, December 12). #851 DocLab: Using VR to Visualize Language Extinction with Lena Herzog’s “Last Whispers.” Voices of VR podcast. Retrieved
November 7, 2021, from https://voicesofvr.com/851-doclab-using-vr-to-visualize-language-extinction-with-lena-herzogs-last-whispers/.

Vi Hart, Andrea Hawksley, Elisabetta A. Matsumoto, and Henry Segerman. Non-euclidean virtual reality I: explorations of H3. In Proceedings of Bridges 2017: Mathematics,
Music, Art, Architecture, Culture.Tessellations Publishing, 2017.
Non-Euclidean VR & Hyperbolic Spaces

Non-Euclidean VR Ray Marching
Henry Segerman. (2019, March 1). Non-euclidean virtual reality using ray marching. YouTube Retrieved November 9, 2021, from
https://www.youtube.com/watch?v=ivHG4AOkhYA.

Lundy, M., Sutton, D., Ashton, A., Martineau, J. (2010) Quadrivium: The Four Classical Liberal Arts of Number, Geometry, Music, & Cosmology. Walker & Co.
Spatial Representations of Time

Abstract Patterns
& Forms
Mathematics

Data Visceralization: Extreme Scale,
Discrete Quantities, Abstract Measures

QuantVR: Visualizing Market Cap as Buildings
Bye, K., & Greenbaum, E. (2015, July 30). #179: Visualizing Stock Market Data & Bringing VR to Wall Street with QuantVR. Voices of VR podcast. Retrieved November 7, 2021,
from https://voicesofvr.com/179-visualizing-stock-market-data-bringing-vr-to-wall-street-with-quantvr/.

Bye, K., & North, C. (2016, June 4). #375: Immersive Analytics & Embodied Cognition. Voices of VR podcast. Retrieved November 9, 2021, from
https://voicesofvr.com/375-immersive-analytics-embodied-cognition/
Chris North: Big Visualization, Big Cognition,
Big Interaction, and Big Algorithms.

Li, T., Belghith, Y., North, C., & Luther, K. (2020). CrowdTrace: Visualizing Provenance in Distributed Sensemaking. 2020 IEEE Visualization Conference (VIS).
doi:10.1109/vis47514.2020.00045

Examination of Grouping and Spatial Organization
Tasks for High-Dimensional Data Exploration
Wenskovitch, J., & North, C. (2021). An Examination of Grouping and Spatial Organization Tasks for High-Dimensional Data Exploration. IEEE Transactions on Visualization and
Computer Graphics, 27(2), 1742–1752. doi:10.1109/tvcg.2020.302889

Flow Immersive
Flow Immersive - Data storytelling with immersive visualizations. Visually compelling data stories. (n.d.). Retrieved November 9, 2021, from https://ﬂowimmersive.com/.

BadVR
BadVR. (n.d.). Retrieved November 9, 2021, from https://badvr.com/.

4D Objects
4D Toys. An interactive toy for 4D children. (n.d.). Retrieved November 9, 2021, from https://4dtoys.com/.

Treehouse in the Shade in VRChat using Shaders
1001. (2018, July 12). Treehouse in the Shade. VR Chat. Retrieved November 9, 2021, from
https://vrchat.com/home/world/wrld_551c30cb-0e56-4c69-9176-10e259a89e2a.

Objects Moving
Through Space
Astronomy

Indiecade Games as Systems
Bye, K., & Roberts, S. (2018, October 23). #708: Indiecade’s Sam Roberts on the Future of Play as Immersive & Experiential. Voices of VR podcast. Retrieved November 9, 2021,
from https://voicesofvr.com/708-indiecades-sam-roberts-on-the-future-of-play-as-immersive-experiential/.

Consonance
Dissonance Cycles
In Time
Music

Bye, K. (2018, April 6) Geometry of Music: Visualizing Beethoven's Fifth Symphony. YouTube. Retrieved November 9, 2021, from
https://www.youtube.com/watch?v=XTv8g_DEH5Q

Primas, H. (2010, May 1). Complementary Time Concepts. The Forgotten Present: A Quest for a Richer Concept of Time, Second Circular for the Parmenides Workshop.
Primas’ Triply correlated systems of Material, Mental, & Temporal Domains

Hans, P. (2010, May 1). Complementary Time Concepts. The Forgotten Present: A Quest for a Richer Concept of Time, Second Circular for the Parmenides Workshop.
Music Perception grouped by time, frequency, and scaling

Beat Saber only focused on the Rhythm of the Music, Not Chords
JoshDub. (2018, December 3). THIS BEAT SABER LEVEL IS INSANE (3000+ NOTES). YouTube Retrieved November 9, 2021, from
https://www.youtube.com/watch?v=3ehSPtWoiuc

The Geometry of Music
Jensen, J. (n.d.). Fun with the Circle Of 5ths and Identifying Chords. Circle of Fifths. Retrieved November 9, 2021, from https://jjensen.org/CircleOf5thsFun.html.

Bye, K. (2018, December 17). The Geometry of Music featuring 82 Harmonies of C. Retrieved November 9, 2021, from https://www.instagram.com/p/BrfIoxZHUXn/.
Geometry of Music featuring 82 Harmonies of C
These are plotted on wheels
that cluster the notes into
polarity points (minor 3rds of
Ab/D, E/Bb, C/Gb on top half &
A/Eb, Db/G, & F/B on bottom
half), augmented chords in
quadrants (C/E/Ab in upper left,
Gb/Bb/D in upper right, A/Db/F
in lower left, and Eb/G/B in
lower right ).
Each chord forms a unique
shape, and each relationship
between notes is colored with
brown = b9/vii, purple = 9/bvii,
red = biii/vi, blue = iii/#v, green
= IV/V and ﬁnally orange = bv.

Bye, K. (2021, August 21) [unlisted]. Beethoven Circle of Fifths Amibisonic Arrangement: A Geometry of Music 360 Video Visualization. YouTube. Retrieved November 9, 2021,
from https://www.youtube.com/watch?v=wJsJXV2xQR8

Jacob, E. (2020, May 18). Pic.twitter.com/o5uh4edi5x. Twitter. Retrieved November 9, 2021, from https://twitter.com/i/status/1262452945573916673.

Virtual Architecture as Process: Visualizing the Geometry of Music

Recommended

Recommended

More Related Content

What's hot

What's hot (14)

Similar to Virtual Architecture as Process: Visualizing the Geometry of Music

Similar to Virtual Architecture as Process: Visualizing the Geometry of Music (20)

More from Kent Bye

More from Kent Bye (18)

Recently uploaded

Recently uploaded (11)

Virtual Architecture as Process: Visualizing the Geometry of Music