O365 and SharePoint Connect - Create an immersive experience with office 365 data and mixed reality using hololens

Alexander Meijers
Solutions Architect / Mixed Reality Evangelist
Create an immersive experience with
Office 365 data and Mixed Reality
using HoloLens
#O365Connect

#O365CONNECT
SOLUTIONS ARCHITECT
MIXED REALITY EVANGELIST
MIXED REALITY TEAM @ ETTU
MICROSOFT WINDOWS DEVELOPMENT MVP
FOR MIXED REALITY
EMAIL: A.MEIJERS@ETTU.NL / ALEXANDER@APPZINSIDE.COM
TWITTER: @AMEIJERS
BLOG: HTTP://WWW.APPZINSIDE.COM
ABOUT ME

#O365CONNECT
FOUNDER OF THE MIXED REALITY USER GROUP
CO-ORGANIZER OF GLOBAL AI/MR BOOTCAMP
CO-ORGANIZER OF SP&C
PROGRAM COMMITTEE @ FUTURETECH 2019
THE COMMUNITY

#O365CONNECT
DIFFERENT REALITIES
Merging of real and virtu
al worlds to produce new
environments and visuali
zations where physical a
nd digital objects co-exist
and interact in real time.
An overlay of synthetic c
ontent on the real world t
hat is anchored to and int
eracts with the real world
MIXED REALITY
Direct or indirect view of a
physical, real-world enviro
nment whose elements ar
e augmented by computer
-generated sensory input
such as sound, video, gra
phics or GPS data
An overlay of content on t
he real world where that c
ontent is not anchored to
or part of it
AUGMENTED REALITY
Generation of realistic ima
ges, sounds and other sen
sations that replicate a real
environment or create an i
maginary setting
An immersive experience c
reated entirely from compu
ter-generated Content. Als
o similar to 360 degree vid
eo
VIRTUAL REALITY

#O365CONNECT
XR (X Reality)
VR
Virtual Reality
MR
Mixed Reality
AR
Augmented Reality

#O365CONNECT
Controls
Sound Spatial understanding
HoloLens as device

#O365CONNECT
COLLABORATION USING MIXED REALITY
Capabilities
Intelligent Cloud
Employees
Skype
Cognitive
Services
Cortana
Collaborative Services
Azure
StorageLogic App
Simulation & Visualization
Services
CRM Online
Experience
Application
Service
Azure
Active Directory
Intune
Teams
Store for
BusinessAzure Compute
Remote assistanceSimulation training CollaborationData visualization
Mixed Reality Experiences
Cloud

#O365CONNECT
BUILDING A MIXED REALITY APP FOR HOLOLENS
GETTING PEOPLE AND RELATED PEOPLE DATA FROM OFFICE 365
GETTING TRENDS AROUND PEOPLE
VISUALIZE THE PEOPLE AND THEIR RELATIONS IN 3D
WATCH THE MODEL AND INTERACT!
HOW ARE WE GOING TO BUILD IT?

#O365CONNECT
TOOLS FOR BUILDING MR

#O365CONNECT
APPLICATION LIFECYCLE
 Create Unity Project
 Configure HoloLens settings
 Create scene
Visual Studio
Unity HoloLens
 Configure build settings
 Build and generate Visual Studio
project
 Open project with Visual Studio
 Pair with HoloLens
 Build & deploy Visual Studio project
 Start Application
 Test & debug
 Monitor
Unity

#O365CONNECT
UNITY EDITOR EXPLAINED IN A (VERY) SHORT WAY
Main
Camera
Directional
Light
Game
Objects
Assets
Components
Script
Scene

#O365CONNECT
API DELIVERS V1 AND BETA INTERFACES
GRAPH EXPLORER
THERE IS A TON OF DOCUMENTATION
FIND EVERYTHING AT HTTPS://DEVELOPER.MICROSOFT.COM/EN-US/GRAPH
MICROSOFT GRAPH API

#O365CONNECT
USE THE GRAPH API
Register your
App
Authenticate Do API calls
Handle
returned data

#O365CONNECT
REGISTRATION OF AN APP IS DONE IN THE FOLLOWING PORTAL
HTTPS://APPS.DEV.MICROSOFT.COM
HTTPS://PORTAL.AZURE.COM (RECOMMENDED)
USE A PERSONAL OR
ORGANIZATIONAL ACCOUNT FOR REGISTRATION
USE ORGANIZATIONAL ACCOUNT WHEN BUILDING APP
FOR YOUR ORGANIZATION
SELECT ONE OR MORE PLATFORMS
REGISTER YOUR APP

#O365CONNECT
HELPS YOU TO REGISTER THE APP
GENERATES CODE FOR YOU TO AUTHENTICATE
AGAINST FOR EXAMPLE THE MICROSOFT GRAPH
AUTHENTICATION USES MICROSOFT
AUTHENTICATION LIBRARY (MSAL)
REFERENCES THE MICROSOFT.IDENTITY.CLIENT
NUGET PACKAGE
GUIDED SETUP

PEOPLE, THEIR RELATIONS AND TRENDS

#O365CONNECT
RELEVANCE IS DETERMINED BY USER’S COMMUNICATION AND COLLABORATION
PATTERNS AND BUSINESS RELATIONSHIPS
RELEVANCE SCORING TAKES PLACE ON EMAIL ADDRESS IDENTITY
GET RELATED PEOPLE FOR LOGGED ON USER
HTTPS://GRAPH.MICROSOFT.COM/V1.0/ME/PEOPLE
GET RELATED PEOPLE FOR A SPECIFIC USER
HTTPS://GRAPH.MICROSOFT.COM/V1.0/USERS/{GUID OF USER ID}/PEOPLE
MICROSOFT GRAPH: RELEVANCE

#O365CONNECT
RELATIONSHIPS CALCULATED USING ADVANCED ANALYTICS AND ML
TECHNIQUES
INSIGHTS ON TRENDNG, USED, SHARED
TRENDING RETURNS
GET INSIGHTS AROUND TRENDS FOR A SPECIFIC USER
HTTPS://GRAPH.MICROSOFT.COM/BETA/USERS/{GUID OF USER ID}/INSIGHTS/TRENDING
MICROSOFT GRAPH: INSIGHTS

GET PEOPLE, RELATIONS AND TRENDS
IN OFFICE 365 VIA GRAPH EXPLORER

#O365CONNECT
INTEGRATES THE MICROSOFT GRAPH API IN YOUR .NET PROJECT
HTTPS://GITHUB.COM/MICROSOFTGRAPH/MSGRAPH-SDK-DOTNET
MICROSOFT.GRAPH.CLIENT
HTTP CALL
• Get the user or app token through
authentication
• Setup Http client
• Add Content Type header
• Use the bearer token
• Call the url
https://graph.microsoft.com/v1.0/me
• Create data classes
• Use convert to json to generate classes
Microsoft.Graph.Client
• Get the authenticated client
GraphServiceClient graphClient
• Call the method
graphClient.Me.Request().GetAsync()
• Handle the collection of objects based
on predefined data classes
Developers Are Lazy, And That's (Usually) A Good thing!

#O365CONNECT
A NUMBER OF ALGORITHMS WHICH ALLOWS YOU TO DRAW DATA AND THEIR
UNDERLAYING RELATIONSHIPS IN A TWO- OR THREE-DIMENSIONAL WAY
FORCE DETECT GRAPH

#O365CONNECT
DATA IS POSITIONED BY NODES AND RELATIONS ARE POSITIONED BY EDGES
ALGORITHM ASSIGN FORCES BETWEEN NODES AND BETWEEN EDGES
EDGES ARE MOSTLY THE SAME LENGTH AND
DO HAVE THE LEAST NUMBER OF CROSSINGS
ADDING NEW NODES AND EDGES WILL REPOSITION THE MODEL
WOONG GYU LA – DEVELOPER FROM CANADA
WROTE A 2D/3D FORCE DIRECTED GRAPH ALGORITHM IN C#
HTTPS://GITHUB.COM/JUHGIYO/EPFORCEDIRECTEDGRAPH.CS
FORCE DETECT GRAPH

SHOW FDG MODEL IN UNITY USING
RANDOM NODES AND EDGES

#O365CONNECT
STEPS & CHALLENGES
Setup a project in Unity
for building a Mixed
Reality experience
Create UWP DLL to
access Microsoft Graph
Integrate Force Detect
Graph code
Add controller code to
access DLL and
use FDG to
handle the data
Use GameObjects and
Components to create
experience
Implement Gaze and
Gestures for interaction

#O365CONNECT
STEPS & CHALLENGES
Setup a project in
Unity for building
a Mixed
Reality experienc
e
Create UWP DLL
to
access Microsoft
Graph
Integrate Force D
etect Graph code
Add controller co
de to access DLL
and
use FDG to
handle the data
Use GameObject
s and Componen
ts to create exper
ience
Implement Gaze
and
Gestures for inter
action
SETUP PROJECT USING UNITY
IMPORT MRTK PACKAGE
INCLUDE HELPER FRAMEWORKS
ADJUST SETTINGS FOR HOLOLENS
APP
BUILD AND GENERATE VISUAL STUDIO
SOLUTION

#O365CONNECT
STEPS & CHALLENGES
Setup a project in
Unity for building
a Mixed
Reality experienc
e
Create UWP DLL
to
access Microsoft
Graph
Integrate Force D
etect Graph code
Add controller co
de to access DLL
and
use FDG to
handle the data
Use GameObject
s and Componen
ts to create exper
ience
Implement Gaze
and
Gestures for inter
action
CREATE SEPARATE UWP DLL
INCLUDE NUGET PACKAGES FOR
AUTHENTICATION AND MICROSOFT G
RAPH LIBRARIES
IMPLEMENT TRUE ASYNCHRONOUS
CALLS

#O365CONNECT
STEPS & CHALLENGES
Setup a project in
Unity for building
a Mixed
Reality experienc
e
Create UWP DLL
to
access Microsoft
Graph
Integrate Force D
etect Graph code
Add controller co
de to access DLL
and
use FDG to
handle the data
Use GameObject
s and Componen
ts to create exper
ience
Implement Gaze
and
Gestures for inter
action
REWRITE CODE FROM OUR
DEVELOPER FROM CANADA TO GET
IT TO WORK IN UNITY 3D
CREATE PREFABS FOR NODES AND
EDGES

#O365CONNECT
STEPS & CHALLENGES
Setup a project in
Unity for building
a Mixed
Reality experienc
e
Create UWP DLL
to
access Microsoft
Graph
Integrate Force D
etect Graph code
Add controller co
de to access DLL
and
use FDG to
handle the data
Use GameObject
s and Componen
ts to create exper
ience
Implement Gaze
and
Gestures for inter
action
DEFINE THE GRAPHCONTROLLER
COMPONENT
USE !UNITY_EDITOR PRECOMPILER DIRE
CTIVE TO ACCESS DLL (NO
DUMMY DLL USED)
USE QUEUE FOR HANDLING DATA.
UNITY CALLS ARE NOT ALLOWED FROM
ANOTHER THREAD

#O365CONNECT
STEPS & CHALLENGES
Setup a project in
Unity for building
a Mixed
Reality experienc
e
Create UWP DLL
to
access Microsoft
Graph
Integrate Force D
etect Graph code
Add controller co
de to access DLL
and
use FDG to
handle the data
Use GameObject
s and Componen
ts to create exper
ience
Implement Gaze
and
Gestures for inter
action
INCLUDE UNITY PACKAGES LIKE
LEANTWEEN AND OFFICE PROPS
PACKS
CREATE A STATE MACHINE BASED ON
THE STATE DESIGN PATTERN

#O365CONNECT
STEPS & CHALLENGES
Setup a project in
Unity for building
a Mixed
Reality experienc
e
Create UWP DLL
to
access Microsoft
Graph
Integrate Force D
etect Graph code
Add controller co
de to access DLL
and
use FDG to
handle the data
Use GameObject
s and Componen
ts to create exper
ience
Implement Gaze
and
Gestures for inter
action
SPATIAL MAPPING AND SPATIAL
CURSOR IMPLEMENTATION FOR
POSITIONING MODEL
APPCURSOR FOR GAZING NODES
APPLOADER FOR “LONG” WAITING
ON SERVER CALLS TO RETURN
TAP AND DOUBLE TAP GESTURES FOR
CONTROLLING THE APP

FINAL RESULT IN A MIXED REALITY
DEVICE

#O365CONNECT
MICROSOFT GRAPH API ALLOWS YOU TO ACCES ALMOST ANY TYPE OF DATA
FROM OFFICE 365 THROUGH AN UNIFIED SYSTEM.
THE MICROSOFT.GRAPH INTERFACE ALLOWS YOU TO EASILY AUTHENTICATE AND
RETRIEVE DATA
USE MATHEMATICAL ALGORITHMS TO VISUALIZE YOUR DATA IN SPECIFIC WAYS
HOLOLENS DEVICE IS A GREAT DEVICE WHICH CAN EXTEND YOUR BUSINESS
PROCESS INTO THE 3D WORLD TO VISUALIZE DATA
TAKEAWAYS

O365 and SharePoint Connect - Create an immersive experience with office 365 data and mixed reality using hololens

More Related Content

Similar to O365 and SharePoint Connect - Create an immersive experience with office 365 data and mixed reality using hololens

More from Alexander Meijers

Recently uploaded

O365 and SharePoint Connect - Create an immersive experience with office 365 data and mixed reality using hololens

Editor's Notes