A talk from the Develop Track at AWE USA 2017 - the largest conference for AR+VR in Santa Clara, California May 31- June 2, 2017. Alessandro Terenzi (InGlobe Technologies): Object Tracking with ARMedia SDK The talk presents the main features of the latest version of the ARMedia SDK with a focus on the 3D object tracking capabilities from a developer's perspective. The whole process involved in the creation of 3D trackable data will be discussed using both CAD-based and photogrammetry-based approaches, showing the strengths and weaknesses of each with reference to real-world use-cases. Finally, the new multiple-objects tracking feature will be demonstrated with reference to a car-tracking scenario. http://AugmentedWorldExpo.com

1. Object Tracking with the
ARMedia SDK
ALESSANDRO TERENZI
CTO @ INGLOBE TECHNOLOGIES SRL
#AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)

2. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
INTRODUCTION

3. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
INTRODUCTION
• ARMedia: Company’s Brand for…
• Augmented Reality
• Virtual Reality
• Perceptual Computing

4. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
INTRODUCTION
• The Offer:
• ARMedia Plugins: 3ds Max, Maya, Cinema 4D, SketchUp, …
• HyperSpaces: General Purpose Authoring Platform
• HyperIndustry: Authoring Platform for the Industry
• HyperPublisher: Authoring Platform for Publishing
• ARMedia SDK: Software Developers
• Custom Solutions…

5. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
INTRODUCTION
• The Market:
• Architecture, Engineering & Construction (AEC)
• Healthcare
• Manufacturing
• Aerospace
• Automotive
• Cultural Heritage

6. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
INTRODUCTION
• The Clients:
• 40 Countries
• 2.000+ Customers
• 50.000+ Registered Users

7. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG

8. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
• Version 0.0 (June 2013): Object Tracking
• iOS/Android/Windows/MacOS: Not a SDK!

9. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
• Version 1.0 (June 2014): Object Tracking
• iOS/Android/Unity3D

10. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
• Version 1.1.0 (June 2015):
• + 360 Panorama Viewer, 3D Viewer, “Motion” Tracking
• + more “ways” to load 3D models…

11. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
• Version 2.0.0 (March 2016):
• + Planar Tracking
• + Location Tracking

12. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: CHANGE LOG
• Version 2.1.0 (October 2016):
• + Performances Improvements
• + APIs Documentation
• + Multiple Object Tracking
• + Improved & Simplified Trackable Data Creation

13. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: OVERVIEW

14. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: OVERVIEW
• Tools
• Target Creation
• ARMedia Toolset (iOS/Android)
• ARMedia Developer Portal
• 3rd Party Components (Optional)
• Content Related
• Conversion Utilities

15. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: OVERVIEW
• Tracking Technologies —> Tracking Module
• Planar (2D)
• Object (3D)
• Location (GPS)
• Motion (IMUs)

16. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: OVERVIEW
• Content & Visualisation (Rendering)
• 3rd Party Engines or Native APIs
• SDK APIs —> Rendering Module
• Unity3D Plugin

17. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: OVERVIEW
• Licensing & Subscriptions
• 1, 6, 12 Months…
• Application Keys…
• Targets Creation & Management…
• …the Developer Portal, Helpdesk & Support

18. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?

19. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Multiple Object Tracking
• Object?
• Multiple?

20. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
1 (*.JPG)

21. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
2 (PSZ)

22. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
Have CAD model or reconstructed mesh?
(OBJ, MTL, PNG/JPG)

23. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
3

24. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
4

25. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
5

26. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
6

27. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
7

28. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
8

29. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
9 (ZIP)

30. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: WHAT’S NEW?
• Trackable Data Creation Process (Objects):
10
(ZIP) ARMedia Toolset

31. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs…

32. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• Simplicity: as few APIs as possible (Objective-C, Java) …

33. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• Supported Platforms

34. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• License/Application Keys
• Every single app must have a valid license key, generated on
the Developer Portal
• The role of the bundle identifier…
• Set using the SDK APIs…

35. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• Tracking Module
• Just tracking APIs (and related events)…
• Camera capturing (not mandatory)
• Just need an OpenCV::Mat object
• Modularity:
• Capturing (Input)
• Rendering (Output)

36. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

37. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

38. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

39. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

40. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• Rendering Module
• Simple graphics APIs (few high level methods)
• Loading models (“as is”)
• Managing animations (rigid body + bones systems)
• Components-based (ViewControllers/Activities)
• Not only AR… VR as well…

41. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

42. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

43. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
NEW!

44. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

45. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs
• Unity3D Plugin
• Simple APIs (C#)
• Ready-to-use components (prefabs)
• Examples for each tracking method
• Interactions with ‘real’ objects: easy!

46. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

47. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

48. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

49. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: APIs

50. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: HANDS-ON…

51. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: HANDS-ON…
• Unity3D Plugin:
• Import SDK Package
• Import Trackable Data
• Setup Scene & App Logic
• Configure Player Settings
• Build & Run

52. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: COMING SOON…

53. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
SDK: COMING SOON…
• GearVR
• Smart Glasses & Headsets…

54. #AWE2017
Augmented World Expo (USA) - June 1, 2017 - Santa Clara (CA)
CONTACTS
Alessandro Terenzi
a.terenzi@inglobetechnologies.com
Skype: a.terenzi