1) The document discusses how augmented reality and Azure are an unbeatable combination. It provides examples of how AR can be used for wayfinding, collaborative design reviews using persistent multi-user virtual content, and empowering firstline workers and those with low vision. 2) The cloud plays a role in AR by providing secure storage and scalable processing for a digital copy of the real world that can be accessed by any user from any device for shared AR content. 3) The document then discusses an example solution architecture for shared AR notes using Azure Spatial Anchors, an Azure App Service, Azure Cosmos DB and a sharing service.

1. #azuresatpn
Azure Saturday 2019
Realtà aumentata ed Azure, un binomio
imbattibile
Alessio Iafrate
Microsoft MVP, Freelance developer
Founder DotNetAbruzzo

2. By 2020, 100 million consumers
will shop in AR online and in-store,
according to Gartner, Inc.
DatafromSeptember2018by
https://pages.arm.com/arvr-report.html

3. Why does the cloud play a role here?
The cloud provides the secure storage
and scalable processing for a digital copy
of the real world which can be accessed
by any user at any time on any device
for shared AR content.

4. x
y
z

5. x
y
z

6. x
y
z
x
y
z x
y
z

7. x
y
z
x
y
z

9. Persistent multi-user virtual content
Collaborative design reviews

10. Persistent multi-user virtual content
Collaborative design reviews
Real-time IoT digital twin data on
actual equipment

11. Collaborative design reviews
Real-time IoT digital twin data on
actual equipment
Training guidance
Persistent multi-user virtual content

12. Wayfinding
Empowering firstline workers
to navigate large facilities

13. Wayfinding
Empowering firstline workers
to navigate large facilities
Guiding the way to IoT sensors
or failing equipment

14. Wayfinding
Empowering firstline workers
to navigate large facilities
Guiding the way to IoT sensors
or failing equipment
Navigating warehouses

16. Wayfinding
Guiding the way through a store
following a spatial grocery list

17. Guiding the way through a store
following a spatial grocery list
Empowering people who are blind or
low vision with spatial sound anchors
Wayfinding

18. Wayfinding
Guiding the way through a store
following a spatial grocery list
Empowering people who are blind or
low vision with spatial sound anchors
Navigating large areas like theme
parks, museums, festivals

19. Persistent multi-user virtual content
Urban murals and digital art

20. Persistent multi-user virtual content
Urban murals and digital art
Architecture visualization

21. Urban murals and digital art
Architecture visualization
Marketing and ads
Points of interest
Games

25. 1) Collaboration 2) Wayfinding 3) UX Considerations

26. #azuresatpn
HoloLens iOS (ARKit) Android (ARCore)
Unity
C++/WinRT
Xamarin
Unity
Objective-C
Swift
Xamarin
Unity
Java
C++/NDK

27. http://bit.ly/2VlLcOA

28. #azuresatpn

29. Mobile
HoloLens
Client
Place notes in the real world
Persistent virtual content
SOLUTION ARCHITECTURE
Azure Spatial
Anchors
Azure
App Service
Azure
Cosmos DB
Sharing Service
iOS – Swift, ARKit, SceneKit
Android – Java, ARCore, SceneForm

30. API Key Concepts
• CloudSpatialAnchorSession
• Provides core services (Ex. Create, locate, update, delete CloudSpatialAnchor’s)
• Captures data about the environment
• CloudSpatialAnchor
• Links to the underlying AR platform Anchor (WorldAnchor on HoloLens, ARAnchor on iOS, Anchor on Android)
• Holds an ID, Expiration, and Properties (Dictionary<String, String>)
• CreateAnchorAsync(CloudSpatialAnchor anchor) { … }
• Save a CloudSpatialAnchor to Azure Spatial Anchors
• Returns the CloudSpatialAnchor with an ID assigned
• CreateWatcher(AnchorLocateCriteria criteria)
• Begins to watch for anchors that meet the specified criteria
• Returns located CloudSpatialAnchor’s through a delegate
• AnchorLocateCriteria
• Assign it an array of CloudSpatialAnchor IDs

31. Mobile
HoloLens
Client
Place notes in the real world
Persistent virtual content
SOLUTION ARCHITECTURE
Azure Spatial
Anchors
Azure
App Service
Azure
Cosmos DB
Sharing Service

32. Mobile
HoloLens
Client
Place notes in the real world
Persistent virtual content
SOLUTION ARCHITECTURE
Azure Spatial
Anchors
Sharing
Service
1) Initialize session
2) Create anchor
3) Locate anchors
1. Save anchor
2. Get anchor ID back
3. Save anchor ID in database
4. Get IDs from database
5. Locate anchors by ID
6. Get anchors back

33. protected void init() {
// Set up ARCore & SceneForm.
this.sceneView = ...
Scene scene = ...
...
// Initialize the CloudSpatialAnchorSession.
this.spatialAnchorsSession = new CloudSpatialAnchorSession();
this.spatialAnchorsSession.Configuration.AccountKey = “xxxxxxxxx”;
this.spatialAnchorsSession.Configuration.AccountId = “xxxxxxxxx”;
...
// Give frames to the CloudSpatialAnchorSession.
scene.Update += (_, args) =>
{
// Pass frames to Spatial Anchors for processing.
this.spatialAnchorsSession?.ProcessFrame(this.sceneView.ArFrame);
};
// Add a callback that tells us how much information about the environment we have.
this.spatialAnchorsSession.OnSessionUpdated += (_, sessionUpdateArgs) =>{
this.sessionUpdated = sessionUpdateArgs.Args.Status.RecommendedForCreateProgress;
}
// Register handleTap() as a callback to be invoked when the user taps to place a sphere.
this.arFragment.TapArPlane += ArFragment_TapArPlane;
...
}

34. Private async void ArFragment_TapArPlane (HitResult hitResult, Plane plane, MotionEvent motionEvent) {
// Create the ARCore Anchor.
Anchor localAnchor = hitResult.createAnchor();
// Render a white sphere at the localAnchor position.
...
// Set the previously created ARCore Anchor as the localAnchor of the CloudSpatialAnchor.
CloudSpatialAnchor cloudAnchor = new CloudSpatialAnchor();
cloudAnchor.LocalAnchor=localAnchor;
// Prompt the user to input a note, set it as a property on the CloudSpatialAnchor.
...
// Wait until we have enough data about the environment
while (this.sessionUpdated < 1) { ... wait ... }
// Save the CloudSpatialAnchor to Azure Spatial Anchors.
this.spatialAnchorsSession.CreateAnchorAsync(cloudAnchor)
.ContinueWith(async cloudAnchorTask =>{
// Save CloudSpatialAnchor ID to the Sharing Service.
PostAnchor(cloudAnchor.getIdentifier());
}
});
Azure Spatial
Anchors
Sharing
Service

35. #azuresatpn
public void LocateAnchors(String[] identifiers) {
AnchorLocateCriteria criteria = new AnchorLocateCriteria();
criteria.setIdentifiers(identifiers);
this.spatialAnchorsSession.createWatcher(criteria);
}
String[] identifiers = GET https://<foo>.azurewebsites.net/api/anchors
LocateAnchors(identifiers);
Azure Spatial
Anchors
Sharing
Service

36. #azuresatpn
private void initializeSession() {
// Initialize the CloudSpatialAnchorSession.
this.cloudSession = new CloudSpatialAnchorSession();
...
this.cloudAnchorManager.OnAnchorLocated += CloudAnchorManager_OnAnchorLocated;
}
private void CloudAnchorManager_OnAnchorLocated(object sender, AnchorLocatedEventArgs args)
{
if (args.Args.Status == LocateAnchorStatus.Located) {
// Get the ARCore Anchor from the CloudSpatialAnchor.
Anchor localAnchor = args.Args.Anchor.LocalAnchor;
// Render object at the ARCore Anchor location.
...
}
});
Azure Spatial
Anchors

38. #azuresatpn
Connected anchors

39. Wayfinding
public void LocateAnchors(String[] identifiers) {
AnchorLocateCriteria criteria = new AnchorLocateCriteria();
criteria.setIdentifiers(identifiers);
cloudSession.createWatcher(criteria);
}
public void LocateNearbyAnchors(CloudSpatialAnchor anchor) {
NearAnchorCriteria nearAnchorCriteria = new NearAnchorCriteria();
nearAnchorCriteria.setSourceAnchor(anchor);
AnchorLocateCriteria criteria = new AnchorLocateCriteria();
criteria.setNearAnchor(nearAnchorCriteria);
cloudSession.createWatcher(criteria);
}

40. #azuresatpn
Connected anchors
DCA
B

42. Placing anchors
Target interesting visual features
• Avoid blank surfaces or surfaces
without details.
Record from perspectives where you
want the next person to discover
from
• Think about those who will come
after you.

43. Finding anchors
Target Scenario Room Scenario

44. #azuresatpn

45. #azuresatpn
Azure Saturday 2019
Grazie a tutti!!!

46. #azuresatpn
Azure Saturday 2019
Question time