Immersive Computing
•
3 likes•1,115 views
HMD shipments are forecast to grow rapidly over the next few years, reaching around 76 million units by 2020. Immersive computing technologies like virtual reality, augmented reality and mixed reality are poised for growth as they blend physical and digital worlds and allow for natural language and gesture-based interactions. Developers can create immersive applications for these platforms across entertainment, training, manufacturing and other areas using tools like Unity, Windows Mixed Reality and Azure cognitive services.
More Related Content
What's hot
What's hot (20)
Similar to Immersive Computing
Similar to Immersive Computing (20)
More from David Chou
More from David Chou (20)
Recently uploaded
Recently uploaded (20)
Immersive Computing
- 3. The future: Immersive Computing • Natural language and interactions between people and technology • Bots and agents • VR, AR and MR 2000s: Mobile • Social • User download apps from App Stores 1990s: Internet • Search • User “visits” websites 1980s: PC • Desktop
- 4. These new experiences are poised for rapid growth HMD shipments to reach ~76M units by 2020 growing at 65% CAGR Source: IDC, Worldwide Quarterly Augmented and Virtual Reality Headset Tracker, 3Q16 196,722 1,937,954 10,252,907 22,672,192 37,674,548 54,003,645 75,972,265 0 10,000,000 20,000,000 30,000,000 40,000,000 50,000,000 60,000,000 70,000,000 80,000,000 2014 2015 2016 2017 2018 2019 2020 7xgrowth of market forecasted through 2020
- 6. The state of things as they “actually exist” through our human senses without any technology Artificially created sensory experiences of people, environments and objects, which can include sight, touch, hearing, and smell PHYSICAL REALITY DIGITAL REALITY
- 7. DIGITAL REALITYPHYSICAL REALITY MIXED REALITY The blending of physical and digital worlds
- 8. Perception
- 10. Environment
- 12. Virtual Reality
- 13. Virtual Reality Virtual Reality immerses the wearer of the HMD within virtual worlds Virtual Reality HMD’s typically require the use of a PC or phone and in some cases, extensive hardware within the room to provide the wearer the illusion of presence within the virtual world
- 14. Virtual Reality PremiumMiddle TierEntry Level
- 15. GIBLIB – Live Stream Surgeries in VR
- 17. Augmented Reality Where Virtual Reality (VR) replaces the physical reality, Augmented Reality (AR) augments or modifies the physical reality Often Augmented Reality is intended to enhance the perception of the physical reality, such as within an aircrafts Heads Up Display (HUD)
- 20. Azure Computer Vision API Tag images based on content Categorize images Identify the type and quality of images Detect human faces and return their coordinates Recognize domain-specific content. Generate descriptions of the content Use optical character recognition to identify text found in images Distinguish color schemes Flag adult content Crop photos to be used as thumbnails Animal_dog People People_crowd Outdoor_mountainFood_bread Distill actionable information from images
- 21. Mixed Reality
- 22. Mixed Reality
- 23. Windows Mixed Reality headsets Head mounted displays (HMDs) that provide mixed reality experiences by leveraging the native Windows Mixed Reality experiences in Windows 10
- 25. Holographic headsets Untethered computer Advanced sensors Transparent lenses Holographic Processing Unit Spatial Sound
- 26. Immersive headsets So easy to set up: Works out-of-box More freedom to place the system where you want Wide range of affordable headsets and PCs The largest apps library in the category 20k+
- 27. Thyssenkrupp Mixed Reality case study
- 29. Types of mixed reality apps Enhanced environment apps (HoloLens only) Blended environment apps Immersive environment apps
- 30. Holograms
- 32. Shared Experiences Sharing scenario Group size Location(s) of participants Sharing mode Physical environments Devices
- 33. Developing Windows Mixed Reality Apps Basic interaction model Environmental understanding features 2D apps Immersive and holographic apps Windows Store
- 34. Areas of opportunity Creation & Design Training & Development Assembly & Manufacturing Communication & Understanding Entertainment & Engagement
- 35. Partners make it real
- 37. Immersive Solution Immersive App (Unity UWP) Immersive App (Unity UWP) 2D App (cross-platform) Dashboards & Visualizations IntelligenceInformation Management Big Data Stores Machine Learning & Analytics Power BI Bot Service Data Catalog SQL Data Warehouse Data Lake Analytics Cognitive Services Data Factory Data Lake Store Machine Learning CortanaEvent Hubs HDInsight Stream Analytics Cosmos DB Cloud (Azure)
- 38. Q&A
- 39. © Copyright Microsoft Corporation. All rights reserved. Thank you!
Editor's Notes
- Rick (1-9)
- This slide focus on HCI – human computer interaction, that is, how humans have interacted with computers throughout the last 3 decades. This slide sets the stage for the presentation, illustrating the journey we have been on since the 1980’s with the introduction of the PC, the 1990’s with the introduction of the Internet and into the 2000’s with Mobile computing. The present and future is all about Immersive Computing and related technologies. In the following sections we’ll briefly explore VR and AR before exploring Mixed Reality and the opportunity it represents for our ISV partners.
- Rick (22) Classic chicken and egg problem e.g. no devices, no apps More HMDs are being sold, experiences are getting better Get in the game and get ahead of your competition
- Reza Before we jump into our session, I wanted to take a moment to give a high-level overview of this program. The session were in today is an Envisioning session, where we talk about the latest trends and how to capitalize on these trends. We highly encourage you all to follow through to the other two phases as well though. The design sessions are led by Microsoft technical experts and dive into architectural considerations and design best practices for incorporating these technologies and finally the Accelerate phase is where you have an opportunity to work side by side with Microsoft technical experts on building out your specific solution And something unique about this program is that we have allocated a special offer for companies going through this program to help you kickstart your projects, which I’ll dive deeper into that at the end
- “Immersive technology refers to technology that blurs the line between the physical world and digital or simulated world”
- “Immersive technology refers to technology that blurs the line between the physical world and digital or simulated world”
- Immersive experiences consist of 3 key pillars – perception, interaction, and environment. Perception (human senses; but of course current technology doesn’t do much with smell and taste, yet; but there are some examples) vision (3D display, holography, head-mounted display, full-dome) auditory (3D audio, surround sound, etc.) tactile (haptic) olfaction (smell) gustation (taste)
- Interaction – gestures, speech recognition, omnidirectional treadmill, brain-machine interface
- Environment – creation and management of the immersive space for the immersive experience, whether it is a single user, private experience, or one that is shared among many users. Just like mobile applications today, this often means integration with server-side resources and services.
- This is dangerous for now
- Transition statement: "That gets us today and lets talk about some of the areas we see immersive computing happening today and more specifically the devices we see those experiences on." Ryan (10-21)
- Virtual Reality – “An artificial, computer-generated world that can be experienced and interacted with” Eye balls are covered up; completely immersed; you have been transported elsewhere; you are taking out of the physical world Sega Master System 3D glasses 2012 Oculus with a kick start campaign; 4 years ; later we had a consumer ready device. HTC Vive Gear VR – head rotation Playstation VR Virtual Reality typically requires a PC, which is OK, but it also requires the use of external sensors (cameras and base stations) to track presence in a space Track light and dark spots; highly details math http://event.on24.com/eventRegistration/console/EventConsoleApollo.jsp?uimode=nextgeneration&eventid=1402899&sessionid=1&key=81802DAB649756CFCCB96A7067D4E07F&contenttype=A&eventuserid=305999&playerwidth=1000&playerheight=650&caller=previewLobby&text_language_id=en&format=fhaudio Examples virtual reality
- Discuss the wide range of devices and experiences they deliver. Make note that these VR HMD’s are all fully occluded, so are 100% digital reality
- RICK Why they are relevant… and perhaps industry antidotes Use case – GIBLIB is company that specializes in surgical videos. They recently launched a VR service that allows medical students, physicians, etc to live stream a surgery in full 360 virtual reality Video stream Connect to video stream Media Services Next slide – why msft; lightweight non-commercial; linux is not; osx is not the go to;
- You stay in your physical world but your physical world is augemented; it has data layed over the top of ypur physical reality; A phone up ; a tablet up magical window; data overlayed; they are still in their physical world; Pokemon;
- AR is more accessible because it doesn’t necessarily require dedicated specialized hardware – there are many experiences that can be built into a simply smartphone – but it can also go all the way up to providing real time information about the world around you through object recognition and computer vision Cog services
- RYAN - Sprinkle Azure; IKEA inventory; graphics; RESTUL; database no-sql / SQL Identify an object; layering cloud data
- David Beyond the case studies some other things you might want to do with computer vision include working out what’s in the photo right? That way you can write some code that does intelligent things based on the content of the image. This could be for moderation where you don’t want to see certain types of images, or perhaps you want to intelligently crop photos? The Vision API is a great place to start solving these problems.
- You stay in your physical world but your physical world is augemented; it has data layed over the top of ypur physical reality; A phone up ; a tablet up magical window; data overlayed; they are still in their physical world; Pokemon;
- It’s the magic behind the experiences available across our OEM and HoloLens devices. It’s the platform that allows for rich user experiences that merge our real world and our digital world (FTE note: similar to Windows Hello – we could imagine Windows Mixed Reality running across a variety of mobile or desktop devices). Mixed Reality is the future of computing and Windows makes jumping in easier than ever before Head-Mounted Display Hardware: Microsoft HoloLens is the first mixed reality device to track user movement across six degrees of freedom User Experience: Windows 10 is the first operating system designed to operate the way people do Consumer Applications: The world’s largest catalog of apps grows with new franchise titles and experiences on Xbox and Windows Commercial Applications: A brand new platform for global institutions and government to transform the way we work and achieve more 3D Images and 360 Video: Windows 10 is the world’s leading platform for beautiful image capture and viewing in three dimensions
- Fixed color on header Animate + gradient to show hardware mix and blend
- RICK - Sprinkle Azure; IKEA inventory; graphics; RESTUL; database no-sql / SQL Identify an object; layering cloud data
- Transition: "Ryan, no MSFT presentation would be complete with talking about the cloud, Azure."
- Tethered; inside / outside tracking; no base stations and external cameras 3 DOF – head rotation; x,y,z 6 DOF – body movement – lateral movement; forward, back, left and right, up, down Degrees of Freedom Freedom of movement: Experience virtual reality in any room of your house. Experience complete freedom with 6DOF tracking. Effortless Setup: Works right out of the box, no need to drill holes or setup extraneous hardware. Windows does all the work. Single, Consistent User Interface: Windows provides a consistent interface across varied Mixed Reality devices. One SDK for many devices: Build apps using the familiar Windows development tools you already know and love, and have them run across multiple HMDs. More consumer choice: With HoloLens and multiple OEM partners, Windows Mixed Reality is available on a wide range of devices, giving the consumer more freedom. More sales channels: Monetize and promote your app from day one in the Windows Store, merchandising in a Store optimized for VR and holographic apps. More customers: By targeting a larger variety of PC specs, Windows Mixed Reality opens the door to reach more consumers than ever before. Transition: "Rick, and we are seeing rapid growth in the market aren't we."
- One of the advantages of developing apps for Windows Mixed Reality is that there is a spectrum of experiences that the platform can support. From fully immersive, virtual environments, to light information layering over a user’s current environment, Windows Mixed Reality provides a robust set of tools to bring any experience to life. It is important for an app maker to understand early in their development process as to where along this spectrum their experience lies. This decision will ultimately impact both the app design makeup and the technological path for development. Enhanced environment apps (HoloLens only) One of the most powerful ways that mixed reality can bring value to users is by facilitating the placement of digital information or content in a user’s current environment. This is an enhanced environment app. This approach is popular for apps where the contextual placement of digital content in the real world is paramount and/or keeping the user’s real world environment “present” during their experience is key. This approach also allows users to easily move from real world tasks to digital tasks and back easily, lending even more credence to promise that the mixed reality apps the user sees are truly a part of their environment. Blended environment apps Given Windows Mixed Reality’s ability to recognize and map the user's environment, it is capable of creating a digital layer that can be completely overlaid on the user’s space. Thin layer respects the shape and boundaries of the user’s environment, but the app may choose to transform certain elements best suited to immerse the user in the app. This is called a blended environment app. Unlike an enhanced environment app, blended environment apps may only care enough about the environment to best use its makeup for encouraging specific user behavior (like encouraging movement or exploration) or by replacing elements with changes (a kitchen counter is virtually skinned to show a different tile pattern). This type of experience may even transform an element into an entirely different object, but still retain the rough dimensions of the object as its base (a kitchen island is transformed into a dumpster for a crime thriller game). Immersive environment apps Immersive environment apps are centered around an environment that completely changes the user’s world and can place them in a different time and space. These environments can feel very real, creating immersive and thrilling experiences that are only limited by the app creator’s imagination. Unlike blended environment apps, once Windows Mixed Reality identifies the user’s space, an immersive environment app may totally disregard the user’s current environment and replace it whole stock with one of its own. These experiences may also completely separate time and space, meaning a user could walk the streets of Rome in an immersive experience, while remaining relatively still in their real world space. Context of the real world environment may not be important to an immersive environment app.
- Spatial mapping provides a detailed representation of real-world surfaces in the environment around the HoloLens, allowing developers to create a convincing mixed reality experience. By merging the real world with the virtual world, an application can make holograms seem real. Applications can also more naturally align with user expectations by providing familiar real-world behaviors and interactions.
- Holograms don't need to stay private to just one user. Holographic apps may share spatial anchors from one HoloLens to another, enabling users to render a hologram at the same place in the real world across multiple devices. Six questions to define shared scenarios Before you begin designing for shared experiences, it’s important to define the target scenarios. These scenarios help clarify what you’re designing and establish a common vocabulary to help compare and contrast features required in your experience. Understanding the core problem, and the different avenues for solutions, is key to uncovering opportunities inherent in this new medium. Through internal prototypes and explorations from our HoloLens partner agencies, we created six questions to help you define shared scenarios. These questions form a framework, not intended to be exhaustive, to help distill the important attributes of your scenarios.
- Mixed reality experiences are enabled by new Windows features for environmental understanding. These enable developers to place a hologram in the real world, and allow users to move through digital worlds by literally walking about. The basic interaction model for HoloLens is gaze, gesture and voice, sometimes referred to as GGV. All mixed reality devices benefit from the input ecosystem available to Windows, including mouse, keyboard, gamepads, and more. With HoloLens, hardware accessories are connected via Bluetooth. With immersive devices, accessories connect to PCs via Bluetooth, USB, and other supported protocols. The environmental understanding features like coordinates, spatial sound and spatial mapping provide the necessary capabilities for mixing reality. Spatial mapping enables holograms to interact with both the user and the world around them. Coordinate systems allow the user's movement to affect movement in the digital world. Holograms are made of light and sound, which rely on rendering. Understanding the experience of placement and persistence, as demonstrated in the mixed reality shell is a great way ground yourself in the user experience.
- From scientists and technicians to designers and clients, five areas of opportunity have emerged where Microsoft partners are find value with mixed reality. These areas are already providing massive insight into the future needs of platforms like Windows Mixed Reality and can help you understand how these new experiences will impact the ways we learn, collaborate, communicate, and create.
- Craig and Reza