Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Philipp Nagele (Wikitude) Wikitude SDK Tutorial

Get to know the Wikitude SDK in its details. Learn what the Wikitude SDK is offering and how it is structured. See the advantages of the Native API vs. the JavaScript API.

Augmented World Expo (AWE) is back for its seventh year in our largest conference and expo featuring technologies giving us superpowers: augmented reality (AR), virtual reality (VR) and wearable tech. Join over 4,000 attendees from all over the world including a mix of CEOs, CTOs, designers, developers, creative agencies, futurists, analysts, investors, and top press in a fantastic opportunity to learn, inspire, partner, and experience first hand the most exciting industry of our times. See more at http://AugmentedWorldExpo.com

Philipp Nagele (Wikitude) Wikitude SDK Tutorial

  1. 1. Wikitude SDK Advanced
  2. 2. Agenda for today ● Who is Wikitude ● What has the Wikitude SDK to offer -Architecture and APIs -Plugins API ● Latest Additions to the Wikitude SDK – 3D Tracking
  3. 3. 3
  4. 4. v v v Wikitude AR at a glance. First AR app in the world in 2008 10,000 apps powered by wikitude SDK 3D tracking technology v REFERENCES
  5. 5. Wikitude’s path to success 20142013 2015 Business Model: B2C Focus: Wikitude AR App 2016201220112009 Founder stage in Salzburg garage: First AR Browser launched. 2010 VCs brought in., Management hired: Martin Herdina, Andy Gstoll Established Wikitude Inc. in USA Business Model: B2B Focus: AR technology and tools Achieved 25 million downloads and app installs gloabally Launch of Wikitude SDK for 2D image recognition Launch of Wikitude Studio, the AR CMS Launch of Wikitude Cloud Recognition service Launch of 3D tracking SLAM
  6. 6. Wikitude offices wikitude GmbH (HQ) in Salzburg Austriawikitude Inc. in San Francisco, USA wikitude Inc. wikitude GmbH.
  7. 7. AR creator and CMSCloud Recognition Wikitude AR Products & Tools Connect your own CMS
  8. 8. Native API JavaScript API Easy access to AR mobile development Xamarin PhoneGap Cordova titaniumunity Android iOS
  9. 9. GEO / sensor based AR Pioneering products and technologies
  10. 10. Maturing products and technologies 2D Image Recognition & Tracking
  11. 11. Tailored SDKs for Smart Glasses Partnership since 2014, optimized SDK for BT-200, full documentation →BT-300 in pipeline Partnership since 2016 optimized SDK for R-7 Partnership since 2014, SDK for M100, full documentation →M300 in pipeline
  12. 12. Wikitude SDK Architecture 2D On-Device Reco & Tracking 2D Cloud Reco 3D Reco & Tracking Unity3D Unity Plugin Plugins API Custom CV plugin Custom camera plugins Your App Your App Cordova Plugin Titanium Module Xamarin Component Your App Cordova Titanium Xamarin Your App Your App JavaScript APINative API Your App 3D Rendering Augmen- tations LBS Plugin ManagerOpenGL ES2 Rendering ARMv7 NEON optimizations ARMv8 optimizations Camera GPU optimizations OpenCL RenderScript Metal API IMU Hardware Abstraction Hardware Optimizations C++ Layer
  13. 13. Plugins API - Modularization 13 Description The Plugins APIs allows to extend the Wikitude SDK by 3rd party functionality. Plugins for the Wikitude SDK have access to the camera information, recognized images. This is perfect for additional functionality that also requires camera access. Plugins are written in C++ and can communicate both with the JavaScript API and the Native API.
  14. 14. Plugins API in detail 14 Custom Frame Provider Image Frame Standard Frame Provider Wikitude CV Engine Rendering Custom CV Plugin Plugins API class Plugin { public: … void cameraFrameAvailable(const Frame&; cameraFrame_); void update(const vector<RecognizedTarget> recognizedTargets_); void startRender(); void endRender … }; 1 2 3 4 5 6 7 class RecognizedTarget { public: const string&amp; getIdentifier() const; const Mat4&amp; getModelViewMatrix() const; const Mat4&amp; getProjectionMatrix() const; const float getDistanceToCamera() const; };
  15. 15. Use-Cases for Plugins API ● Run own computer-vision related code in parallel to Wikitude SDK -OCR Engine -Barcode Reader -Face Recognition ● Own the camera stream and the camera rendering -Custom effects and shaders -Remote camera stream with local processing
  16. 16. JavaScript Augmented Reality Experience 16 Architect Worlds consist of: • .html file(s) • .js file(s) • .css file(s) • Defines your augmented reality experience • Use the Wikitude JS API provided within the Architect view context <script src=“architect://architect.js"></script> • Loaded by the Wikitude SDKs Architect view
  17. 17. Architect world 17 Javascript execution triggers actions in the underlying C++ layer
  18. 18. Architect view composition 18 OpenGL ES 2 view * Renders camera stream * Renders augmentations Web view * Loads main .html file * Transparent background
  19. 19. GeoObjects & HTML content Menu bar jQuery support Geo-located POIs Multiple Drawables - 3 images - 3 text labels Web View Augmented Reality View
  20. 20. Support for jQuery and web APIs
  21. 21. Image Recognition Augment target images using JavaScript API capabilities (Drawables, Sounds, Animations,Videos, 3D Models) var kitchenTracker = new AR.Tracker("Kitchen.wtc"); var overlay = new AR.VideoDrawable("howdens.mp4", 0.65); var trackable2DObject = new AR.Trackable2DObject( kitchenTracker, "Kitchen", { drawables: { cam: overlay }});
  22. 22. Types of augmentations 22 Augmentations are represented by AR.Drawable subclasses • Images - AR.ImageResource / AR.ImageDrawable • Videos (also with alpha channel) - AR.VideoDrawable • 3D models - AR.Model • Web views - AR.HTMLDrawable • Labels - AR.Label
  23. 23. ….architecture, real estate….. ….gaming, filming industry….. ….indoor navigation, retail….. New algorithms and concepts 3D Tracking / SLAM Object Reco & Tracking
  24. 24. Recognition & tracking (core engine) 3D One core engine with many “flavors” Small Scale Scenes / Spaces / Objects - SLAM - Point cloud - Smaller scenes - desktop,small rooms and environments Object Recognition Edge Detection / CAD models - pre-registeredCAD models and displayed silhouettes - Focus: shapes & lines Positional Tracking Indoor Navigation” - Computer vision plusIMU / sensor fusion - Malls, airports, buildings - Untethered indoor gaming in apartment,houseor anywhere Large Scale Scenes Spaces - SLAM based - Point cloud - buildings / spaces - indoors, outdoors
  25. 25. 3D Tracking Flavors 1. Object Recognition Edge Detection / CAD models Object Recognition Edge Detection / CAD models - pre-registeredCAD models and displayed silhouettes - Focus: shapes & lines
  26. 26. Wikitude Lab – RnD Sneak Preview
  27. 27. 3D Tracking Flavors 2. Large Scale Scenes / Spaces Large Scale Scenes Spaces - SLAM based - Point cloud - buildings / spaces - indoors, outdoors Watch here: https://www.youtube.com/watch?v=Zqb6Fo4uRkM
  28. 28. Positional Tracking – Demo Video
  29. 29. Philipp Nagele, CTO Wikitude See more.

    Be the first to comment

    Login to see the comments

  • kaushalkishore14

    Aug. 25, 2016
  • SIKANDARMOHAMMAD

    Nov. 10, 2016
  • johnOpe

    Feb. 14, 2017
  • daisukenishioka

    May. 19, 2020

Get to know the Wikitude SDK in its details. Learn what the Wikitude SDK is offering and how it is structured. See the advantages of the Native API vs. the JavaScript API. Augmented World Expo (AWE) is back for its seventh year in our largest conference and expo featuring technologies giving us superpowers: augmented reality (AR), virtual reality (VR) and wearable tech. Join over 4,000 attendees from all over the world including a mix of CEOs, CTOs, designers, developers, creative agencies, futurists, analysts, investors, and top press in a fantastic opportunity to learn, inspire, partner, and experience first hand the most exciting industry of our times. See more at http://AugmentedWorldExpo.com

Views

Total views

1,370

On Slideshare

0

From embeds

0

Number of embeds

8

Actions

Downloads

17

Shares

0

Comments

0

Likes

4

×