Brisbane GameTech - Kicking off Development & Object Recognition with HoloLens

Brisbane GameTech – May 2017
Kicking Off
Development & Object
Recognition with
HoloLens
Join the Conversation #HoloLens @StavrosTheGeek

Stephen Carter
@StavrosTheGeek
https://au.linkedin.com/in/stephenmarkcarter
https://www.flickr.com/photos/imagegratification
VIC State Mgr | Senior Software Architect

HoloLens’ Tech Specs
Commercial and Gaming Applications
HoloLens’ Fundamental Building Blocks
A Brief History of Holograms
Agenda
Kicking Off HoloLens Development
Object Recognition and Other Techniques

Agenda

Mixed Reality (MR) - HoloLens
● Augmented reality on steroids
● Seamless merging of real and virtual objects
● Mix of reality and virtual reality
● Analyses the environment – spatial mapping
● Virtual objects can be positioned on real objects

OS Windows 10
Memory 64GB Flash
2GB RAM
Weight 579 grams
Battery 2-3 hours

Pros Cons
Hands free operation Price
Holograph clarity Weight
Holograph stability View portal size
Spatial mapping
Multiple input possibility

Agenda

Applications in
● Engineering and Design
● Asset Management & Maintenance
● Showcasing design before construction

Agenda

Fundamental building blocks
● Gaze input
● Gesture input
● Voice input
● World coordinates
● Spatial sound
● Spatial mapping

Gaze input
● First form of input
● Used for targeting
● Tells you where the user is looking
● Lets you determine user’s intent
● Think of it as a mouse pointer

Gesture input
● Allow users to interact
with holograms
● Using hands or optional
clicker

Voice input
● Direct commands to hologram, without gestures
● Gaze at a hologram and speak command
● Natural way to communicate intent
● Traverse complex interfaces easily

World coordinates
● Real-world coordinates
● Provides stationery frame of reference
● Keeps objects near user stable
● Translates to Unity’s World Origin
● Persistable spatial anchors

Spatial sound
● Highlight out of view objects
● Audio haptics
● 12 spatial voices consume approx. 12% CPU

Spatial mapping
● Place holograms in the real world
● Use visualisation techniques like shadows
● Use occlusion techniques
● Physics interactions with real environment
● Navigate holograms around real objects

Other forms of input
● Gamepads like Xbox controller
● Bluetooth keyboard and mouse
● Smartphone

Agenda

What you’ll need
● Unity 5.5 or newer
● HoloToolkit for Unity
● Visual Studio 2015 Update 3 or VS 2017
● HoloLens Emulator

HoloLens Emulator

HoloToolkit
● Apply Unity Settings
● Various Inputs
● Share Multiple HoloLens
● Spatial Sound
● Spatial Mapping
● Spatial Understanding
● UI Elements
● Utilities

Agenda

Vuforia Object Recognition
• Images
• Objects
• Boxes
• Cylinders
• Text
• User Defined Images

Creating Vuforia Objects
● Register for a Vuforia Developer Account
● Create a Project License Key
● Add a Target Manager Database
● Add a Target
● Download Database

Setting Up a Vuforia Unity Project
● Import Unity HoloToolkit Package
● Apply HoloLens Settings
● Import Vuforia Unity Package
● Import Vuforia Target Database
● Add Target Prefabs to Scene

HoloToolkit – Spatial Understanding
● Spatially maps environment
● Recognise walls, floors, ceilings, surfaces,
chairs etc
● Query recognised objects to determine
holographic object placement

Agenda

Monetising HoloLens
Applications
info@ssw.com.au
www.ssw.com.au
Sydney | Melbourne | Brisbane

https://vimeo.com/182656596

Thank you!
info@ssw.com.au
www.ssw.com.au

Q & A
info@ssw.com.au
Microsoft HoloLens: https://www.microsoft.com/microsoft-hololens/en-us
Dev Community: https://developer.microsoft.com/en-us/windows/holographic/community
HoloLens Academy: https://developer.microsoft.com/en-us/windows/holographic/academy
HoloToolkit for Unity: https://github.com/Microsoft/HoloToolkit-Unity
Unity 3D: https://unity3d.com/
Unity Tutorials: https://unity3d.com/learn/tutorials
Vuforia: https://www.vuforia.com/
SSW HoloLens Consulting: https://www.ssw.com.au/ssw/Consulting/HoloLens.aspx
SSW TV Videos: http://tv.ssw.com.au

Brisbane GameTech - Kicking off Development & Object Recognition with HoloLens

Recommended

Recommended

More Related Content

Similar to Brisbane GameTech - Kicking off Development & Object Recognition with HoloLens

Similar to Brisbane GameTech - Kicking off Development & Object Recognition with HoloLens (20)

Recently uploaded

Recently uploaded (20)

Brisbane GameTech - Kicking off Development & Object Recognition with HoloLens