Recommended
Recommended
More Related Content
More from LoriLanday
More from LoriLanday (11)
Recently uploaded
Recently uploaded (20)
Ready for the Metaverse: Immersive and Interactive Experiences in Virtual Worlds/ Game Audio Boston
- 6. @LoirL
- 7. Twitter Garden, Vesuvius, Second Life, 2010, scripted by Jeroen Frans @Fr @LoirL
- 8. @LoirL
- 10. Second Life, 20 Lori Landay, Transformation: Virtual Art on the Brink (2011) @LoirL
- 13. IMMERSIVE VIRTUAL WORLDS (VR) Rec Room @LoirL
- 17. IMMERSIVE VIRTUAL WORLDS (VR) VRChat @LoirL
- 18. IMMERSIVE VIRTUAL WORLDS (VR) JanusVR @LoirL
- 20. Blockchain in VR @LoirL
- 21. Near-field audio (sound close to you) works in shared VR without adding latency @LoirL
- 23. Ready Player One trailer, 2018. @LoirL
- 24. READY FOR THE METAVE @LoirL
Editor's Notes
- What’s happening in virtual worlds now that consumer VR is available? How is spatialized audio part of the new immersive virtual worlds? Just returned from speaking at GDC (and recent visits to High Fidelity), Lori Landay will give an overview of the past, present, and possible futures of virtual worlds and shared VR. Topics will include: social interaction, avatars, first- and third-person perspectives, environment, interacting with objects, quests, games, and “scripting hijinks”—when people goof around with the affordances of the virtual world. We’ll take a spoiler-free glimpse at Ernest Cline’s book Ready Player One and the about-to-be-released movie based on it. Are you ready for the metaverse??
- The portrayal of the Metaverse that has influenced many people involved in virtual worlds, including the founders of Second Life such as Philip Rosedale (now one of the founders of the immersive shared VR platform High Fidelity) is Neal Stephenson’s novel Snowcrash. If you haven’t read it, read it! If you read it a while ago, it’s worth reading again. Another source of visual and narrative inspiration is Star Trek’s Holodeck. In terms of the structure of the metaverse, one of the issues in Ready Player One by Ernest Cline (2011) is who will control the metaverse, a corporation locking it down for profit or people who will keep it open and, as Sean White suggests, empowering positive endeavors like learning, creativity, questioning, and increased empathy and connection across geographical and cultural divides.
- The metaverse is comprised of virtual worlds, spaces, portals, etc., like the universe is made up of planets, moons, stars, and other astronomical objects. Mark Bell’s definition 2008 definition is still , for how it puts people at the center. A virtual world is made up of a network of people, not of computers or builds or scripts or servers. The people are represented by avatars, it is in real-time, and as we have established, persistent. If a virtual space is too empty, it doesn’t feel “real” (see Bell, Mark. "Toward a Definition of “Virtual Worlds”" Journal For Virtual Worlds Research [Online], Volume 1 Number 1 (1 July 2008) The second definition comes from my work on Minecraft, which I argue is the first breakthrough virtual world. As experienced by what the New York Times and others call the “Minecraft Generation,” it provides a model of virtual worlds and a metaverse that they can make theirs, making mods to flavor the Vanilla version with whatever IP or style or content they desire. There are servers for games, and robust community made narratives, forum participation, tutorials (in fact, no official tutorials or instructions during the Notch/early Mojang era), vidoes, songs, poetry, comics, ebooks, etc. Minecraft has given the Minecraft generation high expectations for virtual worlds, and the metaverse, and we better deliver.
- Generations of Virtual Worlds Persistence is the most important new formal element of virtual worlds: the world exists and continues when you turn off your instance of it. We are now at a 4th generation, the immersive social VR that will work across a variety of devices and platforms, with graduated immersion. Jeremy Bailenson, director of the Stanford Virtual Human Interaction Lab, insists that people only stay in a headset for 20 minutes at a time; immersive VR is not for extended use, he says, but for shorter intense experiences. I was lucky to hear him speak at the Unity Educators Meeting at GDC, and you can read about his ideas in his book, Experience on Demand. If that is the case, the 3D web/metaverse may be best experienced with mixed reality/XR: sometimes with headsets, and also through augmented reality, tablets, mobile VR that looks more like 360 video on a phone than avatar-embodied immersive and desktop.
- The second definition is based on my work on Minecraft, which I argue is the first breakthrough virtual world. As experienced by what the New York Times and others call the “Minecraft Generation,” it provides a model of virtual worlds and a metaverse that is in part created by and with the network of participants. it provides a model of virtual worlds and a metaverse that they can make theirs, making mods to flavor the Vanilla version with whatever IP or style or content they desire. There are servers for games, and robust community made narratives, forum participation, tutorials (in fact, no official tutorials or instructions during the Notch/early Mojang era), vidoes, songs, poetry, comics, ebooks, etc. Minecraft has given the Minecraft generation high expectations for virtual worlds, and the metaverse, and we better deliver. Minecraft is not one unified world, but a confederation of single player and multiplayer experiences, some of which are persistent because they are hosted on servers, and some of which are client-side. Combining elements from those two definitions, we could say that there is a network of people represented by avatars, participating in the wider Minecraft phenomenon, that is synchronous and persistent, albeit outside the game worlds generated when they login. Minecraft as a virtual world takes place in the imaginary manifestation of Minecraft as a place that is constituted by people’s varied experiences in the crafted spaces inside computers. The game sold by Microsoft might not be a virtual world, but the wider transmedial imaginary world constructed through audience-led production is. Minecraft is an emergent virtual world, created from a ludic and creative spirit similar to improvisation, in which performers build on each other’s contributions, saying “yes, and” in an ever- expanding validation and encouragement of inspiration. Minecraft is a new kind of virtual world that is a transmedial portal experienced seamlessly across computers, consoles, mobile devices, and now XR. Lori Landay, “Persson’s Minecraft.” In The Routledge Companion to Imaginary Worlds. Edited by Mark J.P. Wolf. Routledge, 2017. Lori Landay, “Minecraft: Transitional Objects and Transformational Experiences in an Imaginary World.” In Revisiting Imaginary Worlds. Edited by Mark J.P. Wolf. New York and London: Routledge, 2017, 127-147.
- In 2010, we had a ”mixed reality” event, with speakers in Second Life, and in Boston. In Boston, we could see the virtual space on a big screen, and in Second Life, the avatars could see the Boston audience via live video stream.
- My favorite thing ever in Second Life was this Twitter garden, because of the way it takes a boring twitter hashtag feed and makes it into a lovely immersive space. I hope this is how AR and mixed reality is, and not just a floating window with a twitter feed. And yes, that is my Second Life avatar, a centaur. You can laugh. I think it’s endlessly entertaining. I wanted to be both human and not only human as an avatar, so it was a good choice for me, and also the centaur has mythical associations with teachers.
- Ten years ago, when I was making interactive art installations in the virtual world Second Life, I was fascinated by the idea that I could be logged into the virtual world, with my avatar and another person could also be logged into the same place, but see the physical environment in a radically different way. For me it could be day, but for them, night. See, there’s the eyeball with the capital letter I that is also like a cursor, showing the fusion of the physical point of view, visual perspective, and sense of self. I’d like to think I’m more subtle 10 years later, but probably not. It’s not only about day and night settings, or other graphics, but about a radical subjectivity based on the digitization and virtualization of the shared environment, which can be customized without the other person being involved. On the one hand, this is terrific for the VR design for autism project I’m working on, for example. But on the other hand, there’s a loss of what’s shared, or what we can assume is shared. Then again, from the point of view of neurodiversity, maybe we always overestimate what we share in how we perceive, understand, and experience the world.
- During my sabbatical project in 2008-2009, I formulated ideas about virtual subjectivity, wrote articles about them, made virtual art installations and machinima videos based on them, and gave various talks on virtual subjectivity. It’s been a central concept in my work since, extending to the sense of self in games and XR.
- I made videos using machinima—animation captured in real-time in a 3D graphical environment such as a video game or a virtual world, which I sometimes combined with real life action footage shot in front of a green screen (or in this case, a green fleece blanket from Walgreens). This is my favorite frame grab because actual me, who is explaining virtual art, ends up in a scene from one of my machinima videos, where my characters shake their heads in disgust and pity. https://www.youtube.com/watch?v=6mCrjkzzDJk I have always found it funny to use avatars like ventriloquist’s dummies who upstage and mock me, or chat bots who play a similar role to avatars.
- Not exhaustive, but a list of some current shared/social VR. My comments on each are based on my own experiences and research. About audio in them: All have spatial audio. Spatial audio attempts to construct a soundfield with the listener’s head at the center, in 360 degrees (x, y, z axes), that uses real-world acoustics. If it is based on how we hear, it is auditory realism. If it is based on how we record, it is PHONOREALISTIC. The important aspect is to implement spatial audio both technically and artfully as spatialized sound, the sum of reverb, occlusion, ambisonic soundfield, sound effects, how we hear speech from avatars, other source sound, all as the listener and other sources of sound are moving. Spatialized sound, however, (and this is my distinction and use of the term spatialized sound when we are doing the spatializing), is sound that we design and implement to enhance and create particular expressionistic effects in an immersive environment or experience. The Oculus Audio SDK enabled spatialized sound in Altspace VR in 2015. With the recent implementation of near-field audio in High Fidelity, an avatar behind my avatar called my name and I turned to see who it was as instinctively and naturally as if I were in the actual world.
- Maybe more of a proto-world or a virtual communications hub than a robust virtual world, but it is populated by your actual friends and family (if they have headsets). It should be interesting to see what Facebook does with it. Will it merge with Oculus Home or take on a more domestic metaphor? Or is Facebook already imagining itself as the metaverse?
- Very fun and friendly
- Linden Lab (Second Life) owned virtual world, bringing in many of the social structures of Second Life. Some impressive builds the last time I looked, but it all felt static to me. During the free Creator Beta, I tried building a sound installation (Plato’s Cave!), but it crashed so fast I couldn’t fix it. I’ll try again, but I didn’t enjoy the building interface and I was not interested in paying for a subscription. And after I see the movie Ready Player One, I’m looking forward to visiting Aech’s Basement: https://atlas.sansar.com/experiences/sansar-studios/ready-player-one-aech-s-basement
- Build in Unity (a special version of Unity) and upload to SineSpace, with a delay for verification. Great tech support. Good audio.
- I’ve had a great time each time I’ve been in AltSpaceVR. The first time I went in, I went to a place where the avatars were watching an old science fiction movie and commenting on it Mystery Science Theater style. It was entertaining and friendly and never inappropriate or uncomfortable. Another time, I gave a presentation in it at a meetup of Women in the Next Realities, and it was fantastic for that. The hearts and smiley emoticons are surprisingly useful at quickly indicating emotion and affect, so sorely missing from avatar faces right now in all virtual worlds. One time my avatar fell over a cliff when I wandered away from friends and although I could hear their conversation above me, they couldn’t hear me and it took me a long time to get back up to where they were. That kind of thing happens to me all the time in virtual worlds, and I view it as hilarious. It’s the I Love Lucy experience in the new medium. It works nicely with Gear VR and will likely be good on the forthcoming standalone $199 Oculus Go.
- Exponential growth, challenges of moderating the community. I did hear offensive conversation almost immediately and lost interest, having had mostly positive experiences in other shared VR spaces. I will give it another try at some point because people I respect use it and like it.
- More of a portal – decentralized and on the web. Very very interesting, something to keep an eye on. WebXR is likely to be the platform for the metaverse.
- I really like High Fidelity. The people I’ve met in the company and the world are engaged in building and scripting the world in a way that is smart and fun, based in an expansive vision of the metaverse but also with attention to the practical details that make it viable. Basing the currency and marketplace in blockchain is, in my opinion, the right move, and I am always in support of open source. HiFi has always considered audio as an integral part of immersion, and its sound is better developed than in the other worlds. But the other virtual worlds have their strengths, too. The idea of the metaverse is that someone can move between whichever virtual world they think of as a home base, with their identity and digital stuff, like going from website to website on the web, or “place” to “place” in a virtual world, so whichever shared VR experiences you prefer could, in the metaverse, be connected to the others. For example, JanusVR and High Fidelity just announced the Virtual Reality Blockchain Alliance to facilitate that through a currency acceptance, digital asset recognition, and identity recognition and privacy settings.
- Blockchain provides the economic and intellectual property foundations for a robust metaverse. https://www.youtube.com/watch?v=eAtCCNkcPN0&t=3s
- https://www.youtube.com/watch?v=95ZqCvQn_4w “When there are 150 people in a room, our server does a lot of work; creating a unique version of audio for each source and listener means 150 x 150, or 22,500 different sender/receiver pairs to process at about 100hz.” https://blog.highfidelity.com/whisper-in-my-virtual-ear-c30c8b6cb06c With the recent implementation of near-field audio in High Fidelity, an avatar right behind my avatar said my name and I turned to see who it was as instinctively and naturally as if I were in the actual world.
- Portrayals of the metaverse in fiction and film – nostalgia for the 80s?
- Trailer(s) for Ready Player One. How is the virtual world portrayed? How is the actual world portrayed? These representations are going to be incredibly influential, like Minority Report was for imagining what interfaces and control could be like. For many people, it will be their introduction to virtual reality and virtual worlds. If you don’t like or agree with the how the future looks in these fictions, offer up some alternatives! This is an exciting time in which we can contribute to how the metaverse will look, feel, sound, and behave. Get your music in there, help figure out how to create audio that isn’t simply phono-realistic, but is expressive and supports the created environment and experience. https://www.youtube.com/watch?v=cSp1dM2Vj48&t=6s
- You might as well get ready for the metaverse, and get the metaverse ready for whatever vision you have of the future. This is the time to get work in spatialized audio. It’s in demand and right now there aren’t enough people working at a high level with ambisonics, binaural audio, and spatialized soundscape environments. We are only at the very beginning of what people will be able to do for sound design, performance, audience participation, new instruments, creatively and artfully visualized DAWs, games, and more. When the popular discourse inevitably turns to the pathologizing of VR and immersive experiences, you can counter with the beneficial possibilities of clinical and educational VR, as well as the affordances of a virtual world that extends not replaces the actual one.