Azure Day Rome Reloaded 2019 - Getting Started with Unity, AR/VR and Azure Cognitive Services for the .NET Developer

•

1 like•62 views

We all see the cool demos of how augmented and virtual reality are going to be the future of our interaction with devices, but it often feels like there’s a significant barrier to getting started in building these types of apps. This session is here to help. If you’ve ever thought about developing for Oculus Rift S, HTC VIVE or the Windows Mixed Reality headsets, but haven’t known where to start, this is the session for you. At Azure day, I will present what I learned building a small game, set in a mysterious dungeon using Unity, the SteamVR plugin and Azure Cognitive Services. We'll explore how to import assets from the Unity store, how to implement locomotion techniques and interactions using the motion/touch controllers, including some tips for optimising performance and avoid motion sickness. Come along, there’ll be .NET code, demos and we’ll have some fun looking at how you can get started on your first VR/AR project enhanced by Azure and the intelligent cloud.

Getting started with Unity,
AR/VR and Azure Cognitive
Services for the .NET
developer
Davide Zordan
B: https://www.davide.dev
E: mail@davide.dev
T: @DavideZordan

Davide Zordan
Senior Software Engineer
AR/VR Mixed Reality Developer
Udacity VR Nanodegree graduate
MCSD, MCSA, MS, MCP
Microsoft MVP (2009-2014)

The state of things as they “actually
exist” through our human senses
without any technology.
Artificially created sensory experiences of
people, environments and objects, which can
include sight, touch, hearing, and smell.
PHYSICAL REALITY VIRTUAL REALITY

MIXED REALITY
VIRTUAL REALITYPHYSICAL REALITY

PHYSICAL
REALITY
DIGITAL
REALITY
MIXED REALITY (MR)
WINDOWS MIXED REALITY HOLOGRAPHIC DEVICES (SEE-THROUGH) /
MAGIC LEAP ONE / Nreal Light / RealWare HMT-1
OCULUS RIFT S - QUEST / HTC VIVE - COSMOS / VALVE INDEX /
WINDOWS MIXED REALITY IMMERSIVE DEVICES (OPAQUE) / Varjo XR-1
AR
TODAY
VR
TODAY
Physical - Digital Reality Spectrum
*The term mixed reality was originally introduced in a 1994 paper by Paul Milgram and Fumio Kishino, "A Taxonomy of Mixed Reality Visual Displays."

VR/AR as next computing platform
Estimated $80bn revenue by 2025
MR is revolutionizing many processes

Gaming
Video entertainment
Healthcare
Real Estate
Retail
Education
Engineering
Military

Designing great VR experiences
Place illusion
Embodiment illusion
Plausibility illusion
Source: https://www.coursera.org/lecture/introduction-virtual-reality/introduction-to-plausibility-illusion-psi-K5PGj

C#/.NET
C++
JavaScript
Game Engines (Unity, Unreal Engine, …)
OpenGL and Microsoft DirectX
WebVR / WebXR

VR Locomotion
Eye gazing
Teleportation
Dash

VR interactions
Pointing
Selecting
Grabbing

Integrating Azure Cognitive Services
Language Understanding (LUIS)

Key takeaways:
Check performance early and often to avoid motion sickness
Test frequently using the target device
Design properly the experience for maximising the user immersion
Use cloud services to enhance the user experience (LUIS, speech, vision)

Summary
What is Mixed Reality
Why Mixed Reality
How to develop for AR/VR
VR Interactions
VR Locomotion
Adding vocal commands
Using Natural Language Understanding (LUIS)
Performance tips

“Any sufficiently advanced technology is
indistinguishable from magic”
Arthur C. Clark, Profiles of the future, 1961

Thanks
Davide Zordan
B: https://davide.dev
E: mail@davide.dev
T: @DavideZordan

References
AR / VR / Mixed Reality samples
https://github.com/davidezordan
SteamVR Unity plugin
https://github.com/ValveSoftware/steamvr_unity_plugin
Mixed Reality Toolkit
https://github.com/Microsoft/MixedRealityToolkit-Unity
Microsoft Ignite
https://8gportalvhdsf9v440s15hrt.blob.core.windows.net/ignite2017/session-presentations/BRK2404.PPTX
Goldman Sachs VR/AR report
https://www.goldmansachs.com/insights/pages/virtual-and-augmented-reality-report.html

References
Coursera “Introduction to Virtual Reality”
https://www.coursera.org/lecture/introduction-virtual-reality/introduction-to-plausibility-illusion-psi-K5PGj
Cognitive Services
https://azure.microsoft.com/en-us/services/cognitive-services
Language Understanding (LUIS) documentation
https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/
Oculus VR performance optimization guide
https://developer.oculus.com/documentation/pcsdk/latest/concepts/dg-performance-opt-guide
Mixed Reality Academy and Azure Services tutorial
https://docs.microsoft.com/en-us/windows/mixed-reality/tutorials

Slides from my presentation at DDD conference 2019 https://www.developerdeveloperdeveloper.com. Abstract: We all see the cool demos of how augmented and virtual reality are going to be the future of our interaction with devices, but it often feels like there’s a significant barrier to getting started in building these types of apps. This session is here to help. If you’ve ever thought about developing for Oculus Rift, HTC Vive or the Windows Mixed Reality headsets, but haven’t known where to start, this is the session for you. At DDD, I will present what I learned building a small game, set in a mysterious dungeon using Unity and the SteamVR plugin. We'll explore how to import assets from the Unity store, how to implement locomotion techniques and interactions using the motion/touch controllers, including some tips for optimising performance and avoid motion sickness. Come along, there’ll be .NET code, demos and we’ll have some fun looking at how you can get started on your first VR/AR project.

Getting started with Unity and AR/VR for the .NET developer... Reloaded!

Davide Zordan

Getting started with Unity and AR/VR for the .NET developer - October 2020

Davide Zordan

Getting started with unity and AR/VR for the .net developer - DDD North 2020

Davide Zordan

3D in Android

Wen Jiun Yap

Mobile Augmented Reality Development tools

Thiwanka Makumburage

426 lecture2: AR Technology

Mark Billinghurst

Etna dev 2016 - Introduction to Mixed Reality with HoloLens

Matteo Valoriani

Virtual Reality refers to a high-end user interface that involves real-time simulation and interactions through multiple sensorial channels. Virtual Reality is often used to describe a wide variety of applications, commonly associated with its immersive, highly visual, 3D environments. The development of CAD software, graphics hardware acceleration, head mounted displays, database gloves and miniaturization have helped popularize the concept. Jaron Lanier coined the term Virtual Reality in 1987. Today Virtual Reality plays a big part in the everyday lives of the world’s population.

TalkUX - UX in VR - UNIT9

Laura Cortes

Augmented reality intro for mobile apps

Heather Downing

Market Based Analysis of Virtual Reality Products - IT Strategy

MohitMhapuskar

Virtual Reality is a rapidly advancing technology that initially began it's life as an entertainment product but has rapidly grown into other areas such as healthcare, education, live events and military. While there is massive scope for VR, the technology is still in it's infancy and there are avenues for improvements in the future which could allow it to expand and capture larger segments of the market. Our team selected the VR company Oculus for our case study and conducted various market based analyses such as SWOT Analysis, Scenario Analysis, Porter’s 5 Forces for Strategy, PESTEL Analysis, BCG Matrix, Ansoff Matrix and Fishbone analysis to propose a strategic roadmap for future VR products and services.

Augmented World Expo 2014 Wearable SDK Overview

Patrick O'Shaughnessey

Augmented realityAbhisek Padhi

2013 Lecture 8: Mobile AR

Mark Billinghurst

Handout: 'Future of Augmented & Virtual Reality' (BDPA Cincinnati, Oct 2015)

BDPA Education and Technology Foundation

Technology is improving at a rapid pace, as many things are possible today that were not possible 10 years ago even if we tried our best to make it happen. Today, some of the impossible things are rising to the occasion in the form of Augmented Reality and Virtual Reality. Both virtual reality and augmented reality are similar in the goal of immersing the user, though both systems to this in different ways. With AR, users continue to be in touch with the real world while interacting with virtual objects around them. With VR, the user is isolated from the real world while immersed in a world that is completely fabricated. Our guest speaker, Vince Kilian, is product manager for Marxent. Marxent is the leader in Virtual Reality and Augmented Reality product visualization for sales and marketing. Vince will share examples of successful AR/VR solutions and what's next for the future of this rapidly emerging technology.

AR and VR development tools and platforms

Sushmita Chatterjee

Augmented Reality

Akash J Nair

ElipseAD Portoflio 2013

ElipseAD

Augmented reality ppt

Sourav Rout

2013 426 Lecture 2: Augmented Reality Technology

Mark Billinghurst

Augmented Reality PresentationSJSU

Mini workshop on ar vr using unity3 d

AkshetPatel

Getting Started with Point Clouds Visualisation and AR/VR

Davide Zordan

Slides from my session at the Global XR Conference. Nov 2022. Abstract: Augmented and Virtual Reality technologies allow advanced interactions in three-dimensional space by projecting digital content into users' field of view using dedicated opaque and see-through head-mounted displays. At the Global XR Conference 2022, I will show what I have learned by building a software prototype combining AR and VR to transmit in real-time a three-dimensional video. Point clouds were captured using a mobile depth camera and transmitted to a Virtual Reality device (Meta Quest 2) using a WebSocket server hosted on the Microsoft Azure platform. Come along, there will be Unity demos, and we will have some fun looking at building a distributed system integrating holograms in a Virtual Reality experience.

Building Holographic & VR Experiences Using the Mixed Reality Toolkit for Unity

Nick Landry

Mixed Reality blends people, places and things across a full spectrum ranging from the real-world to digital virtual worlds. This session is your introduction to building 3D applications and games for Windows Mixed Reality using C#, Visual Studio and Unity. We’ll start with a quick recap of Mixed Reality, the HoloLens device, immersive headsets and the Universal Windows Platform (UWP), and talk about interacting with holograms using gaze, gestures and voice. Next, we’ll dive into building 3D apps for Windows Mixed Reality in Unity with C#, using the Mixed Reality Toolkit for Unity, exporting our project to Visual Studio and running it on either a HoloLens device, an immersive Mixed Reality headset or the HoloLens Emulator. Science fiction becomes science fact, and Unity & UWP developers are at the front of this revolution. Come learn how to get started and build your journey into a world of holograms you create.

Virtual RealityNaresh Pokhriyal

What's hot

Making Augmented Reality Applications with Android NDK

Evren Coşkun

Augmented Reality

Apurva Hyanki

Creating a Virtual Reality in Unity - by Unity Evangelist Kelvin Lo

智傑楊

Augmented World Expo 2013 Mobile AR SDK Comparison and Tutorial

Patrick O'Shaughnessey

Virtual Reality

Akshay Patole

TalkUX - UX in VR - UNIT9

Laura Cortes

Augmented reality intro for mobile apps

Heather Downing

Market Based Analysis of Virtual Reality Products - IT Strategy

MohitMhapuskar

Augmented World Expo 2014 Wearable SDK Overview

Patrick O'Shaughnessey

Augmented realityAbhisek Padhi

2013 Lecture 8: Mobile AR

Mark Billinghurst

Handout: 'Future of Augmented & Virtual Reality' (BDPA Cincinnati, Oct 2015)

BDPA Education and Technology Foundation

AR and VR development tools and platforms

Sushmita Chatterjee

Augmented Reality

Akash J Nair

ElipseAD Portoflio 2013

ElipseAD

Augmented reality ppt

Sourav Rout

2013 426 Lecture 2: Augmented Reality Technology

Mark Billinghurst

Augmented Reality PresentationSJSU

Mini workshop on ar vr using unity3 d

AkshetPatel

What's hot (19)

Making Augmented Reality Applications with Android NDK

Augmented Reality

Creating a Virtual Reality in Unity - by Unity Evangelist Kelvin Lo

Augmented World Expo 2013 Mobile AR SDK Comparison and Tutorial

Virtual Reality

TalkUX - UX in VR - UNIT9

Augmented reality intro for mobile apps

Market Based Analysis of Virtual Reality Products - IT Strategy

Augmented World Expo 2014 Wearable SDK Overview

Augmented reality

2013 Lecture 8: Mobile AR

Handout: 'Future of Augmented & Virtual Reality' (BDPA Cincinnati, Oct 2015)

AR and VR development tools and platforms

Augmented Reality

ElipseAD Portoflio 2013

Augmented reality ppt

2013 426 Lecture 2: Augmented Reality Technology

Augmented Reality Presentation

Mini workshop on ar vr using unity3 d

Similar to Azure Day Rome Reloaded 2019 - Getting Started with Unity, AR/VR and Azure Cognitive Services for the .NET Developer

Getting Started with Point Clouds Visualisation and AR/VR

Davide Zordan

Building Holographic & VR Experiences Using the Mixed Reality Toolkit for Unity

Nick Landry

Virtual RealityNaresh Pokhriyal

Mixed Reality Development Overview

Nick Landry

Mixed reality

Musafar Sakeen

Virtual reality in hcijeet patalia

Metaverse Development Services

Mobiloitte Technologies

The metaverse may provide a way for people to explore the world, either as an alternative to traveling in person or as a way to prepare or as a way to prepare for an upcoming journey. In Metaverse Virtual reality makes certain activities more accessible to individuals who are physically separated from events or places, as well as providing users with unique personalized experiences that will enhance their travel experience.

Virtual reality

majid asgharizadeh

Virtual reality

Harikrishna Surisetti

Paweł Ruszlewski - First steps in Mixed Reality design

Paweł Ruszlewski

Virtual / Artificial Reality-Augmented Reality

JasminJaman1

VR/AR introduction & state-of-the-art VR/AR prototyping

Valentijn Destoop

Sam Frish: MR molecular game for scientists

Lviv Startup Club

Building Mixed Reality Experiences with the HoloToolkit for Unity

Nick Landry

Mixed reality blends 3D holographic content into your physical world, giving your holograms real-world context and scale, allowing you to interact with both digital content and the world around you. Nick Landry offers an overview of the HoloToolkit, an open source project led by Microsoft and the HoloLens community aimed at simplifying the development of mixed reality experiences. The HoloToolkit consists of a collection of scripts and components intended to accelerate the development of holographic applications targeting Windows Holographic. Nick leads a technical dive into the HoloToolkit for Unity, exploring advanced holographic features, such as gaze and gesture input, spatial mapping, plane finding, spatial understanding, microphone input, text to speech, and creating shared holographic experiences for multiple users. All demos will be shown on the Microsoft HoloLens, the world’s first self-contained, untethered head-mounted holographic computing device for mixed reality. Join Nick to learn how this open source toolkit—combined with Unity—can get you started on your development journey into a world of holograms.

How ABB shapes the future of industry with Microsoft HoloLens and Unity - Uni...

Unity Technologies

It's high time for augmented reality to be brought to a wider audience. In ABB, we know that it is not just a gimmick any more. However, with every innovative technology comes new challenges. In these slides, we show how to overcome them and deliver valuable products with Hololens and Unity. Speakers: Maciej Włodarczyk - ABB Rafał Kielar - ABB Watch the session on YouTube: https://youtu.be/QFsj8Pi_3Ho

2014 COSC 426 Lecture 2: Augmented Reality Technology

Mark Billinghurst

Matt Zeller (Microsoft): Windows Mixed Reality: Holograms to VR, and Everythi...

AugmentedWorldExpo

A talk from the Life Track at AWE USA 2017 - the largest conference for AR+VR in Santa Clara, California May 31- June 2, 2017. Matt Zeller (Microsoft): Windows Mixed Reality: Holograms to VR, and Everything in Between Mixed reality covers a spectrum of experiences from traditional AR to VR. Come learn about Microsoft’s mixed reality strategy, and how you can create experiences that cover the entire spectrum using one platform and one set of tools. http://AugmentedWorldExpo.com

eXtended Reality(XR) Basic introductions

Elanthirayan Madhavan

Extended Reality (XR) has transformed the way we interact with digital content, blending the physical and virtual worlds seamlessly. This presentation delves into the evolution of XR technologies, including Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR), and their integration into the web through WebXR, as well as the underlying graphics technologies such as WebGPU and WebGL. Additionally, we will explore the importance of Zone certification in ensuring safety and reliability within XR experiences.

Developing Immersive Experiences With Windows Mixed Reality

FITC

Presented at FITC's Spotlight AR/VR event on Dec 2, 2017. More info at www.fitc.ca Margaryta Ostapchuk Microsoft Overview Windows Mixed Reality opens a window into the virtual world, right from Windows Desktop. This new platform enables developers to create rich, immersive experiences, using a variety of supported headsets. In this talk, you will learn about how you can get started developing for Windows Mixed Reality. Margaryta will discuss the device and platform technologies. She will walk through the emerging development trends and look at some resources that can help kick-start your development journey. Finally, Margaryta will wrap it all up with a look at the development workflow, with a device demo that will inspire you to take the leap into Windows Mixed Reality. Objective Learn more about Windows Mixed Reality and how you can leverage frameworks to create immersive experiences Target Audience General VR/AR/MR developers Assumed Audience Knowledge Some VR development experience Five Things Audience Members Will Learn What Windows Mixed Reality is and its importance Device technologies and specification Emerging trends and techniques Challenges in development and how to avoid them Development workflow

Virtual Reality

Akash Bhokare

Similar to Azure Day Rome Reloaded 2019 - Getting Started with Unity, AR/VR and Azure Cognitive Services for the .NET Developer (20)

Getting Started with Point Clouds Visualisation and AR/VR

Building Holographic & VR Experiences Using the Mixed Reality Toolkit for Unity

Virtual Reality

Mixed Reality Development Overview

Mixed reality

Virtual reality in hci

Metaverse Development Services

Virtual reality

Paweł Ruszlewski - First steps in Mixed Reality design

Virtual / Artificial Reality-Augmented Reality

VR/AR introduction & state-of-the-art VR/AR prototyping

Sam Frish: MR molecular game for scientists

Building Mixed Reality Experiences with the HoloToolkit for Unity

How ABB shapes the future of industry with Microsoft HoloLens and Unity - Uni...

2014 COSC 426 Lecture 2: Augmented Reality Technology

Matt Zeller (Microsoft): Windows Mixed Reality: Holograms to VR, and Everythi...

eXtended Reality(XR) Basic introductions

Developing Immersive Experiences With Windows Mixed Reality

Virtual Reality

More from azuredayit

Azure Day Rome Reloaded 2019 - ML.NET Model Lifecycle with Azure DevOps

azuredayit

Azure Day Rome Reloaded 2019 - Reactive Systems with Event Grid

azuredayit

Azure Day Rome Reloaded 2019 - Building serverless microservices in azure

azuredayit

Azure Day Rome Reloaded 2019 - Cloud Journey – FastTrack for Azure

azuredayit

Starting from a scenario with obsolete legacy infrastructure services, we moved the entire .NET application landscape to Azure Platform as a Service. The migration strategy was based on Azure Database Migration and was completed in a month and a half using Azure PaaS (Database as a Service, App Service and Application Gateway). With minimal application changes, we were able to provide workload optimization, management optimization and Infrastructure as Code, which resulted in a reduction in the number of servers (from 30 to 5), in management effort (-70%) and in deployment time for the client.

Azure Day Rome Reloaded 2019 - Azure: a cloud with a purpose

azuredayit

Azure Day Rome Reloaded 2019 - Azure Sentinel: set up automated threat respon...

azuredayit

Azure Day Rome Reloaded 2019 - Azure Application Insights Overview

azuredayit

Azure Day Rome Reloaded 2019 - Deconstructing Kubernetes using AKS

azuredayit

Azure Day Rome Reloaded 2019 - Python, Azure Cosmos DB, Docker and Azure Cont...

azuredayit

Azure Day Rome 2019 Reloaded - Strangle(r pattern) your legacy application ru...

azuredayit

In questa sessione verrà presentato un Business Case che vede la conversione di un applicativo monolitico legacy ASP.NET Framework che gira su Virtual Machines Windows On-Premises, in un docker container e il suo rilascio in un pool di nodi Windows orchestrati da Kubernetes (AKS). Solo successivamente, attraverso lo Strangle Pattern, verrà dimostrato come in modo graduale sia possibile direzionare il traffico verso i nuovi Micro-Servizi rilasciati in parallelo su di un Pool di nodi Linux.

Azure Day Rome 2019 Reloaded - Effettuare il provisioning su Azure utilizzand...

azuredayit

Azure Day Rome 2019 Reloaded - Utilizzare Azure Kubernetes Service per i nost...

azuredayit

Azure Day Rome Reloaded 2019 - Ingestion nel datalake passando tramite API Ma...

azuredayit

Azure Day Rome Reloaded 2019 - Azure Cognitive Search Deep Dive

azuredayit

More from azuredayit (14)