This document provides an overview of building business applications with Microsoft HoloLens and cloud technologies. It discusses the different types of realities, HoloLens capabilities including gestures, spatial sound, and spatial understanding. It also covers connecting HoloLens to the cloud and considerations for data, connectivity, user experience, and security when building applications. Examples of HoloLens applications are demonstrated.

2. BUILDING BUSINESS APPLICATIONS WITH HOLOLENS
AND CLOUD

3. “HoloLens Evangelist”
I am responsible for architecture related business for our project services
department at ETTU. I’m involved as architect at different companies like Jumbo
Maritime, PGGM, Gemeente Amersfoort and others
Alexander Meijers
ABOUT ME
 Solutions Architect for ETTU
 Founder of the Mixed Reality User Group
 MixUG: http://www.mixug.nl
 Email: a.meijers@ettu.nl or alexander@appzinside.com
 Twitter: @ameijers
 Blog: http://www.appzinside.com

4. 5INTRO ETTU
TODAY’S TALK
AGENDA
 Realities explained
 Microsoft HoloLens
 Fields of Application
 HoloLens capabilities
 Gestures
 Sound
 Spatial understanding
 Local storage
 Cloud
 Data
 Connectivity
 User Experience
 Security
Slider
Content Ads

5. REALITIES EXPLAINED

6. HOLOLENS & CLOUD 7
REALITIES
EXPLAINED
Merging of real and virtual worlds to
produce new environments and
visualizations where physical and
digital objects co-exist and interact
in real time.
An overlay of synthetic content on
the real world that is anchored to
and interacts with the real world
MIXED REALITY
Direct or indirect view of a physical,
real-world environment whose
elements are augmented by
computer-generated sensory input
such as sound, video, graphics
or GPS data
An overlay of content on the real
world where that content is not
anchored to or part of it
AUGMENTED REALITY
Generation of realistic images, sounds
and other sensations that replicate a
real environment or create an
imaginary setting
An immersive experience created
entirely from computer-generated
Content. Also similar to 360 degree
video
VIRTUAL REALITY

7. HOLOLENS & CLOUD 8
EVERYTHING
BECOMES MIXED REALITY
 Virtual Reality devices
 SteamVR
 AltSpaceVR
 October/November 2017
IMMERSIVE HEADSETS
 Mixed Reality Device
 Developer and Commercial Suite
version
 Since October 2016
HOLOLENS
2019
HoloLens v3

8. MICROSOFT HOLOLENS

9. HOLOLENS & CLOUD 11
MICROSOFT HOLOLENS
 First Holographic computer running on Windows 10
 Contains a CPU, GPU and HPU
 Human understanding
 Gesture and voice input
 Gaze tracking
 Spatial sound
 Uses wi-fi to connect and has no wires attached
 Design for comfort

10. HOLOLENS & CLOUD 12
SENSORS, OPTICS AND
SPEAKERS
Environment
camera
Depth
camera
Video
camera
Spatial Sound
With
speakers
High definition
lenses

11. “DEMONSTRATION OF HOLOLENS
APPLICATIONS”
“Holograms”

12. HOLOLENS & CLOUD 15
MICROSOFT HOLOLENS
EDITIONS
 Kiosk mode
 Limiting running apps to enable demo or showcase experiences
 MDM
 Manage Multiple HoloLens devices through solutions like Microsoft
Intune
 Business Store Portal
 Private enterprise store for your companies apps
 Secure boot
 Only boot software which is trusted for your organization
 Windows Update for Business
 Controlled OS updates to device

13. FIELDS OF APPLICATION

14. HOLOLENS & CLOUD 17
FIELDS OF APPLICATION
HoloLens
 Enables the user to visualize
surrounding which is not
directly visible
 Get visual insight of an
object or model
 Modelling of non-existing
objects
VISUALIZATION
 Enrich real-world
environment objects with
information
 Show information as part of
the real-world on a wall or
table
 Show additional information
to objects, persons or
activities
PROVISIONING
OF INFORMATION
 Support through linked
view, voice and gestures
 Meetings around
Holographic objects
 Working together on
projects whereby
visualization is an important
part
 Working together when
distance is involved
WORKING
TOGETHER
 Holographic media
 Enrich media with
additional holographic
objects
MEDIA AND
ENTERTAINMENT
 Experience learning
 Getting more realistic
teaching by moving 2D
presentations in books to
3D holographic models
EDUCATION
 Play games as part of your
real-world environment
 Game interaction with your
walls, ceiling and floor
GAMING

15. HOLOLENS & CLOUD 18
HOLOLENS EXAMPLES

16. HOLOLENS & CLOUD 19
HOLOLENS EXAMPLES

17. HOLOLENS & CLOUD 20
HOLOLENS EXAMPLES

18. HOLOLENS & CLOUD 21
HOLOLENS EXAMPLES

19. HOLOLENS & CLOUD 22
HOLOLENS EXAMPLES

20. HOLOLENS & CLOUD 23
HOLOLENS EXAMPLES

21. “DEMONSTRATION OF SPATIAL DESIGN
CONCEPT”
“HoloLens app for creating neighbourhood facilities”

22. INPUT CONTROLS

23. HOLOLENS & CLOUD 26
INPUT CONTROLS
GAZE
 The direction of the HoloLens pointing is called the gaze
 In principle the direction in which you look
 Hits an object or part of the environment
 Used for selecting or pointing

24. HOLOLENS & CLOUD 27
INPUT CONTROLS
GESTURES
 Recognize input by tracking the position of either or
both hands within the gesture frame
 Bloom gesture
 Click gesture
 Press and release
 Click gesture with hold
 Manipulation
 Navigation

25. HOLOLENS & CLOUD 28
INPUT CONTROLS
SPEECH
 Use words or sentences to control your environment
 English language only at the moment
 Hey Cortana

26. SPATIAL SOUND

27. HOLOLENS & CLOUD 30
SPATIAL SOUND
POSSIBILITIES
 Simulates 3D sound using direction, distance and environmental
simulations
 Above, below, behind, to the side, etc.
How?
Gaze Mixing
 Attach sound to holographic objects
 Works also when object is not in line of sight
 Used to draw attention
Audio Haptics
 Selecting an object
 Hands within the gesture frame
Immersion
 Ambient sounds surrounding the user
 Background music

28. SPATIAL UNDERSTANDING

29. HOLOLENS & CLOUD 32
SPATIAL UNDERSTANDING
PERCEPTION
 Spatial perception and mapping
 Rooms
 Different ways of detecting and working with spatial
perception
 Meshes & planes
Spatial perception is the
ability to be aware of
your relationships with
the environment around
you and with yourself

30. HOLOLENS & CLOUD 33
SPATIAL UNDERSTANDING
PERCEPTION
 Device scans your environment and builds a digital model in
real time
 It allows HoloLens to see different surfaces, walls and ceiling
 Possible to simulate a physical space
 E.g. Projecting a terain over your floor

31. “DEMONSTRATION OF SPATIAL MAPPING
HOLOGRAM”
“Spatial Understanding”

32. OTHER TECHNOLOGIES

33. HOLOLENS & CLOUD 36
TECHNOLOGIES
VUFORIA
 Allows you to develop holographic apps which recognize
specific things in the environment and attach experiences to
them
 Recognizes specific images and objects in the environment
 Connects holograms to targets
 Vuforia engine
 Extend the capabilities of your holographic app
 Targets can be stored on Device or in the Cloud

34. HOLOLENS & CLOUD 37
TECHNOLOGIES
SHARING EXPERIENCES
 Two or more people are experiencing the same holographic
objects
 Accomplished by anchor sharing
 Use cloud to send anchors over to other devices

35. CLOUD & DATA

36. HOLOLENS & CLOUD 39
CLOUD & DATA
The workforce in your application is defined by the
amount and type of data which is transferred from
and to the HoloLens application
DATA
Use of certain data, latency when connecting
services and security will influence the user
experience
USER EXPERIENCE
Location and preferences of your customer defines
the ability of your HoloLens application to connect
to services and is divided into different scenarios
CONNECTIVITY
Data transferred need to be secured depending on
the type of data. Also authentication differs from
online services and available customer security.
SECURITY
APPLICATION REQUIREMENTS
COSTS

37. HOLOLENS & CLOUD 40
APPLICATION
REQUIREMENTS
 What is the goal of your application?
 Are you storing any data?
 Storage on device
 Storage in the Cloud
 What type of data are you storing?
 Common application data
 Holograph objects
 Sensory data
 Are you depending on any (external) data?
 Need the data to be exposed to other services or
applications?
 Where is the device used?
 What kind of access do we have with external sources?
 Continuously
 Hybrid
 Offline
 What is the impact on user experience of HoloLens users?
 Security
 Latency with connected sources

38. HOLOLENS & CLOUD 41
DATA
LOCAL STORAGE
 Only allowed to store files to a number of local folders
 Save or open from another App like OneDrive
 HoloLens does not support connecting external hard drives or
SD Cards
 There is no File Explorer application
 You could use the Windows Device Portal

39. HOLOLENS & CLOUD 42
DATA SERVICES
CONNECTING DATA TO HOLOLENS
 Online Services
 Bing
 Other third-party services
 Azure Services
 Custom services
 Bots / AI
 Azure Cognitive Services
 Office 365
 Microsoft Graph
 SharePoint Online
 Special
 Vuforia Cloud Recognition Service

40. “DEMONSTRATION OF HOLOMARKER”
“HoloLens app using SharePoint Online as
registration service for tagging improvements”

41. HOLOLENS & CLOUD 44
CONNECTIVITY
CONNECTION SCENARIOS
 Preloading data to device and
post saving data to Cloud
 No internet whatsoever
 Keep application small by
getting only relevant data
OFFLINE
 Connection not always present
or very low bandwidth
 Application runs in different
environments with different
connectivity
 Caching mechanism for
availability of data and
performance
HYBRID
 Continuously connected
 Get only data when needed
 Caching mechanism for
performance
ONLINE

42. HOLOLENS & CLOUD 45
CLOUD & DATA
SECURITY CONSIDERATIONS
 Do you need to secure your content?
 Building services
 Is your content available for other applications?
 Is your content anonymous available for the big world?
 E.g. a service which delivers certain data from your organization or type
of services
 GDPR is not an illusion, data need to be secured depending
on the content

43. HOLOLENS & CLOUD 46
CLOUD & DATA
IDENTITY & AUTHENTICATION
Anonymous External Providers
(Facebook, Twitter, Google, etc.)
Azure Active Directory Custom Providers Keys
Online services X X X X X
Office 365
Microsoft Graph
X
Azure Services
(AI/Bots/Custom)
X X X
Office 365
SharePoint Online
X
Azure Cognitive Services X
Vuforia Cloud Recognition
Service
X

44. “DEMONSTRATION OF OFFICE
SKETCH”
“HoloLens app accessing Microsoft Graph”

45. HOLOLENS & CLOUD 48
USER EXPERIENCE
USER PERSPECTIVE
 Ensure the application is comfortable to the user
 Avoid abrupt movements
 Allow them to see the world in the background
 Animation from down, left or right or fade in
 Move content slowly and smoothly toward the user
 Keep a minimal distance of 3 meters
 Design your application from an user’s unique point of view
 Determine if the user moves around, user is sitting down or standing
 The user is the camera!
 Let them control the movement

46. HOLOLENS & CLOUD 49
USER EXPERIENCE
HOLOGRAPHIC FRAME
 Use the center of the holographic frame for your holographic
objects
 Help the user through guideness
 Arrows pointing to a specific location
 Light trails
 Pointers
 Spatial sound
 Voice prompts
 Make sure your holograms fit the frame and be part of the
real-life environment
 They will feel natural

47. HOLOLENS & CLOUD 50
USER EXPERIENCE
APPLICATION
 Make sure that your application uses background processes
to prevent latency
 Only load data which you actually need
 Keep traffic to a minimum when accessing services
 Keep framerates up to 60 fps
 Use optimized assets specifically build for HoloLens

48. “DEMONSTRATION OF HOLOMAPS”
“HoloLens app using Bing as a service”

49. HOLOLENS & CLOUD 52
CLOUD & DATA
COSTS
 Devices
 Number of devices
 Development or Commercial Suite edition
 Additional hardware / software
 Sensors to enhance application or collect data
 Embedded software/hardware
 Additional cost for solution development to support hybrid and offline scenarios
 Holograms
 External services
 Data storage
 Retrieval of data
 Use of a services

50. WRAP-UP

51. HOLOLENS & CLOUD 54
WRAP-UP
TAKEAWAYS
 HoloLens is at the moment the best mixed reality device for
building applications for small to large business
 The inbuild gestures, speech, spatial sound and spatial
mapping give the user that true immersive experience for
building business applications
 Consider application requirements and the cost
 When building applications keep the four in mind; Data,
Connectivity, User Experience and Security

52. THANKS FOR WATCHING!
SharePoint Unite
Kampenringweg 45b, Gouda
0182-686 000
a.meijers@ettu.nl / alexander@appzinside.com
Contact me:
facebook.com/alexandermeijers.5
@ameijers
Follow me on: