CodeMash – VR/AR talk
•Download as PPTX, PDF•
0 likes•137 views
This document discusses virtual reality, mixed reality, and physical reality. It provides information on VR headsets, Windows Mixed Reality headsets, and what HoloLens is. HoloLens is described as the first fully untethered holographic computer that redefines personal computing using holograms to enhance the real world. Links are provided for Unity, Oculus, and other VR and mixed reality resources.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to CodeMash – VR/AR talk
Similar to CodeMash – VR/AR talk (20)
Recently uploaded
Recently uploaded (20)
CodeMash – VR/AR talk
- 6. PHYSICAL REALITY VIRTUAL REALITY
- 7. MIXED REALITY VIRTUAL REALITYPHYSICAL REALITY
- 8. PHYSICAL REALITY VIRTUAL REALITY DESKTOP PERFORMANCE MOBILE PERFORMANCE T E T H E R E D O P A Q U E S E E - T H R O U G H P O R T A B L E
- 9. VR Headsets
- 10. Windows Mixed Reality headsets
- 11. What is HoloLens? Holograms enhance the real world Microsoft HoloLens is the first, fully untethered holographic computer that redefines personal computing and empowers you in new ways. New ways to connect, create, and explore A more natural way to interact Advanced technology from Microsoft
- 17. Spatial Sound
- 18. https://www.roadtovr.com/b-reel-simulating-object-weight-mass-virtual-reality-motion-controllers/
- 32. Complete the survey for a chance to win a GoPro Hero 6! aka.ms/codemash
Editor's Notes
- In this slide, I explain some of my background and the places I have worked, and currently am working.
- What is a hologram? HoloLens lets you create holograms, objects made of light and sound that appear in the world around you, just as if they are real objects. Holograms respond to your gaze, gestures and voice commands, and can interact with real-world surfaces around you. With holograms, you can create digital objects that are part of your world. A hologram is made of light and sound - The holograms that HoloLens renders appear in the holographic frame directly in front of the user's eyes. Holograms add light to your world, which means that you see both the light from the display and the light from your surroundings. HoloLens doesn't remove light from your eyes, so holograms can't be rendered with the color black. Instead, black content appears as transparent. Holograms can have many different appearances and behaviors. Some are realistic and solid, and others are cartoonish and ethereal. Holograms can highlight features in your surroundings, and they can be elements in your app's user interface. Holograms can also make sounds, which will appear to come from a specific place in your surroundings. On HoloLens, sound comes from two speakers that are located directly above your ears, without covering them. Similar to the displays, the speakers are additive, introducing new sounds without blocking the sounds from your environment. A hologram can be placed in the world or tag along with you - When you have a particular location where you want a hologram, you can place it precisely there in the world. As you walk around that hologram, it will appear stable relative to the world around you. If you use a spatial anchor to pin that object firmly to the world, the system can even remember where you left it when you come back later. Some holograms follow the user instead. These tag-along holograms position themselves relative to the user, no matter where they walk. You may even choose to bring a hologram with you for a while and then place it on the wall once you get to another room. A hologram interacts with you and your world - Holograms aren't only about light and sound; they're also an active part of your world. Gaze at a hologram and gesture with your hand, and a hologram can start to follow you. Give a voice command to a hologram, and it can reply. Holograms enable personal interactions that aren't possible elsewhere. Because the HoloLens knows where it is in the world, a holographic character can look you directly in the eyes as you walk around the room. A hologram can also interact with your surroundings. For example, you can place a holographic bouncing ball above a table. Then, with an air tap, watch the ball bounce and make sound when it hits the table. Holograms can also be occluded by real-world objects. For example, a holographic character might walk through a door and behind a wall, out of your sight. A hologram is whatever you dream up - As a holographic developer, you have the power to break your creativity out of 2D screens and into the world around you. What will you build?
- TALKING POINTS FOR NEXT FOUR SLIDES HoloLens embraces virtual reality and augmented reality to create a new reality—mixed reality. Virtual reality immerses you in a simulated world. Augmented reality overlays digital information on top of your real world. By understanding your environment, mixed reality enables holograms to look and sound like they’re part of your world. How does HoloLens augment the real world with information? It allows you to place holograms in your real world and on real objects. Lock them in a physical location or let them travel with you in your holographic frame. How does HoloLens blend holograms in your real world? Like physical objects, you can see and hear holograms wherever they are in a room around you. Even if you turn around, you can still pinpoint a hologram’s location by it’s sound. How does HoloLens transport you to a virtual world? You can move freely and interact with holograms, people, and other tools with the added benefit of see-through lenses that preserve your peripheral vision.
- Same points as above
- Same points as above
- It’s the magic behind the experiences available across our OEM and HoloLens devices. It’s the platform that allows for rich user experiences that merge our real world and our digital world (FTE note: similar to Windows Hello – we could imagine Windows Mixed Reality running across a variety of mobile or desktop devices). Mixed Reality is the future of computing and Windows makes jumping in easier than ever before Head-Mounted Display Hardware: Microsoft HoloLens is the first mixed reality device to track user movement across six degrees of freedom User Experience: Windows 10 is the first operating system designed to operate the way people do Consumer Applications: The world’s largest catalog of apps grows with new franchise titles and experiences on Xbox and Windows Commercial Applications: A brand new platform for global institutions and government to transform the way we work and achieve more 3D Images and 360 Video: Windows 10 is the world’s leading platform for beautiful image capture and viewing in three dimensions
- Pre-read for speaker: https://developer.microsoft.com/en-us/windows/mixed-reality/spatial_sound When objects are out of our line of sight, one of the ways that we can perceive what's going on around us is through sound. On HoloLens, the audio engine provides the aural component of the mixed-reality experience by simulating 3D sound using direction, distance, and environmental simulations. Using spatial sound in an application allows developers to convincingly place sounds in a 3 dimensional space (sphere) all around the user. Those sounds will then seem as if they were coming from real physical objects or the mixed reality holograms in the user's surroundings. Given that holograms are objects made of light and sometimes sound, the sound component helps ground holograms making them more believable and creating a more immersive holographic experience. Although holograms can only appear visually where the user's gaze is pointing, your app's sound can come from all directions; above, below, behind, to the side, etc. You can use this feature to draw attention to an object that might not currently be in the user's view. A user can perceive sounds to be emanating from a source in the mixed-reality world. For example, as the user gets closer to an object or the object gets closer to them, the volume increases. Similarly, as objects move around a user, or vice versa, spatial sounds give the illusion that sounds are coming directly from the object. Simulating the perceived location and distance of sounds By analyzing how sound reaches both our ears, our brain determines the distance and direction of the object emitting the sound. HRTF, or Head Related Transfer Functions, simulates this interaction by modeling the spectral response that characterizes how an ear receives sound from a point in space. The spatial audio engine uses personalized HRTFs to expand the mixed-world holographic experience, and simulate sounds that are coming from various directions and distances. Integrating spatial sound Because the general principle of HoloLens is to ground holograms into the real world, most sounds from holograms should be spatialized. Naturally, there are CPU and memory budget considerations, but you can use 10-12 spatial sound voices while using less than ~12% of the CPU (~70% of one of the four cores). Recommended use for spatial sound voices include: Gaze Mixing (highlighting objects, particularly when out of view). When a hologram needs a user's attention, play a sound on that hologram (e.g. have a virtual dog bark). This helps the user to find the hologram when it is not in view. Audio Haptics (reactive audio for touchless interactions). For example, play a sound when the user's hand enters and exits the gesture frame. Or play a sound when the user selects a hologram Immersion (ambient sounds surrounding the user). It is also important to note that while blending standard stereo sounds with spatial sound can be effective in creating realistic environments, the stereo sounds should be relatively quiet to leave room for the subtle aspects of spatial sound, such as reflections (distance cues) that can be difficult to hear in a noisy environment. Window's spatial sound engine only supports 48k samplerate for playback. Most middleware programs, like Unity, will automatically convert sound files into the supported format, but when using Windows Audio APIs directly please match the format of the content to the format supported by the effect.
- Keep horizon level -