Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Kean Walmsley
Platform Architect & Evangelist, Autodesk Research
Collaborative VR using Google Cardboard and the
Forge Vie...
@AutodeskForge
#ForgeDevCon
About me
• 20 years at Autodesk in 4 countries
• 10 years blogging at http://autode.sk/ttif
• 9 years tweeting at @keanw
•...
To participate in the experiment…
Your mobile phone will need…
Or search for “QR Code Reader”
VR comes of age
© 2016 Autodesk
Gartner Hype Cycle 1995
© 2016 Autodesk
Gartner Hype Cycle 2015
© 2016 Autodesk
© 2016 Autodesk
Using the Autodesk Viewer to implement VR
The original application
• Web-page containing two viewer instances
• Camera-changed event maintains stereo effect
• Memor...
Navigation choices
• “Look around” or walkthrough viewer
• Sub-optimal for systems without parallax
• Really requires head...
VR has serious challenges around user input
• Opportunities around gesture-based input
• Leap Motion has an Oculus mount &...
Adding speech recognition
• Google Chrome has a Speech Recognition API
• Requires an internet connection
• Handy JavaScrip...
Using mobile VR SDKs with our web-based viewer
Android SDK for Google Cardboard
• Provides additional capabilities for Android apps
• Magnet-based trigger, uses the phon...
Oculus Mobile SDK for the Samsung Gear VR
• Great optics – wider field of view
• Low latency/more responsive due to additi...
Calling into our code
• Native apps embedding WebViews can call JavaScript
mWebView.loadUrl("javascript:openSelected()");
...
Daydream
• Latest entrant into the mobile VR space
• Announced at Google I/O 2016
• Based on Android N, requires high-end ...
Implementing collaborative features in web-based VR
User input in VR is tricky, remember?
• Another approach is to let someone else guide you…
• Your teacher, professor, arch...
The Internet has a key role to play
• The web enables collaboration and inter-device comms
• Existing infrastructure makes...
The “VR Party” team’s project
• Allow a master web-page to control a number of clients
• Master page embeds the full “GUI”...
How does it work?
• Unique session created when master page loads
• Custom QR code presented for clients to scan
• GUI vie...
Technologies
• HTML5, JavaScript, CSS3, jQuery
• Forge Viewer (Three.js & WebGL)
• Node.js
• Socket.io
• GitHub for source...
Vrok-It!
“Best Web-based VR Project” at the
SF VR Hackathon in May 2015
50% will fail
http://vrok.it
If you’re interested in going further…
• Try uploading your own models
• http://www.vrok.it
• http://www.vrok.it/v2 (WIP w...
Merten Stroetzel, from our Office of
the CTO, has taken this approach
and integrated it with VRED. He’s
showing it here in...
Future directions
• Tune the viewer for VR
• Complete the single viewer work
• Focus on supporting larger models
• Improve...
Forge - DevCon 2016: Collaborative VR using Google Cardboard & the View & Data API
Forge - DevCon 2016: Collaborative VR using Google Cardboard & the View & Data API
Forge - DevCon 2016: Collaborative VR using Google Cardboard & the View & Data API
Forge - DevCon 2016: Collaborative VR using Google Cardboard & the View & Data API
Forge - DevCon 2016: Collaborative VR using Google Cardboard & the View & Data API
Forge - DevCon 2016: Collaborative VR using Google Cardboard & the View & Data API
Forge - DevCon 2016: Collaborative VR using Google Cardboard & the View & Data API
Upcoming SlideShare
Loading in …5
×

Forge - DevCon 2016: Collaborative VR using Google Cardboard & the View & Data API

738 views

Published on

Kean Walmsley, Autodesk Research

One of the issues with VR today relates to the largely solitary experience it delivers: the user is by definition immersed and isolated from the people around them. But this doesn’t have to be the case. Web technology allows us to enable collaborative design reviews in VR, where one person guides others through a virtual experience.

Google Expeditions allows teachers to take their students on virtual field trips to the Great Wall of China or to the bottom of the ocean. We’ll be showcasing a comparable solution – using Forge – for the design industry, whereby architects and engineers can lead a group of participants – each using Google Cardboard – through a design review in a virtual space.

This session will demonstrate a prototype, collaborative VR solution created by a cross-divisional Autodesk team as part of the 2015 VR Hackathon in San Francisco.

Published in: Technology

Forge - DevCon 2016: Collaborative VR using Google Cardboard & the View & Data API

  1. 1. Kean Walmsley Platform Architect & Evangelist, Autodesk Research Collaborative VR using Google Cardboard and the Forge Viewer
  2. 2. @AutodeskForge #ForgeDevCon
  3. 3. About me • 20 years at Autodesk in 4 countries • 10 years blogging at http://autode.sk/ttif • 9 years tweeting at @keanw • Current role is in Autodesk Research • Focused on IoT and VR
  4. 4. To participate in the experiment…
  5. 5. Your mobile phone will need… Or search for “QR Code Reader”
  6. 6. VR comes of age
  7. 7. © 2016 Autodesk Gartner Hype Cycle 1995
  8. 8. © 2016 Autodesk Gartner Hype Cycle 2015
  9. 9. © 2016 Autodesk
  10. 10. © 2016 Autodesk
  11. 11. Using the Autodesk Viewer to implement VR
  12. 12. The original application • Web-page containing two viewer instances • Camera-changed event maintains stereo effect • Memory inefficient: better to use two viewports on a single viewer/scene • View adjusts with device tilting • HTML5’s deviceorientation event • Gives handy roll, pitch & yaw • http://autode.sk/gcb
  13. 13. Navigation choices • “Look around” or walkthrough viewer • Sub-optimal for systems without parallax • Really requires head-tracking • Cybersickness: “vergence-accommodation conflict” • “Widget viewer” / turntable navigation • Sensible option for smaller models • Made sense for this prototype
  14. 14. VR has serious challenges around user input • Opportunities around gesture-based input • Leap Motion has an Oculus mount & Android SDK • Webcam-based hand tracking may also be good enough • Voice is often greatly overlooked • In many ways a natural way to interact within VR • No need to worry about displaying hands virtually
  15. 15. Adding speech recognition • Google Chrome has a Speech Recognition API • Requires an internet connection • Handy JavaScript wrapper called Annyang • Easy to register commands for common operations • EXPLODE, COMBINE, zoom IN and OUT • FRONT, BACK, LEFT, RIGHT, TOP, BOTTOM • http://autode.sk/gcbv (https://youtu.be/0csk5bK8Iyk)
  16. 16. Using mobile VR SDKs with our web-based viewer
  17. 17. Android SDK for Google Cardboard • Provides additional capabilities for Android apps • Magnet-based trigger, uses the phone’s magnetometer • Disables the display’s auto-off feature • Also does view management (which we don’t need) • Simple to use with Android Studio • NFC can also be used to launch a Cardboard app
  18. 18. Oculus Mobile SDK for the Samsung Gear VR • Great optics – wider field of view • Low latency/more responsive due to additional IMU • It doesn’t use the phone’s • Adds more UI capabilities • Touchpad on the side + Bluetooth gamepad • No longer a walled garden • Samsung has a WebVR-capable browser • Works with Samsung Galaxy Note 5 or S6/S7 • A bargain at $99 (plus phone)
  19. 19. Calling into our code • Native apps embedding WebViews can call JavaScript mWebView.loadUrl("javascript:openSelected()"); • Same approach works for Cardboard or Gear VR apps • Allows us to wire-up “hardware” events to our code • http://autode.sk/gvr
  20. 20. Daydream • Latest entrant into the mobile VR space • Announced at Google I/O 2016 • Based on Android N, requires high-end phone hardware • Nexus 6P is the closest currently available • A new Wiimote-like controller is mandated • Could be a major step in democratizing VR
  21. 21. Implementing collaborative features in web-based VR
  22. 22. User input in VR is tricky, remember? • Another approach is to let someone else guide you… • Your teacher, professor, architect, engineer or doctor • What if they could lead you/a group through a virtual environment? • VR becomes more social, more personal • Building walkthroughs, site flythroughs • Google has “Expeditions” for virtual fieldtrips
  23. 23. The Internet has a key role to play • The web enables collaboration and inter-device comms • Existing infrastructure makes this easy • Forge Viewer is a great place to implement this • Web-based, 3D visualization technology • Pre-integrated with Google Cardboard (as we’ve just seen) • The basis for Autodesk’s entry in last year’s VR Hackathon in San Francisco
  24. 24. The “VR Party” team’s project • Allow a master web-page to control a number of clients • Master page embeds the full “GUI” viewer • Clients have a stereo view on the same model • The master propagates certain operations • Isolate, Show, Hide, Explode, Section and Zoom • Importantly not camera/target position changes
  25. 25. How does it work? • Unique session created when master page loads • Custom QR code presented for clients to scan • GUI viewer events capture interesting operations • Transmitted to the back-end via Web Sockets • Passed on to connected clients • Applied in the client’s viewer(s) • Page close/disconnect cleans up session state • Informs clients of disconnection
  26. 26. Technologies • HTML5, JavaScript, CSS3, jQuery • Forge Viewer (Three.js & WebGL) • Node.js • Socket.io • GitHub for source control • Heroku for hosting • CloudFlare for SSL, DNS, caching
  27. 27. Vrok-It! “Best Web-based VR Project” at the SF VR Hackathon in May 2015
  28. 28. 50% will fail http://vrok.it
  29. 29. If you’re interested in going further… • Try uploading your own models • http://www.vrok.it • http://www.vrok.it/v2 (WIP with single viewer) • Clone the source from GitHub • https://github.com/KeanW/vr-party • Send feedback • kean.walmsley@autodesk.com or @keanw
  30. 30. Merten Stroetzel, from our Office of the CTO, has taken this approach and integrated it with VRED. He’s showing it here in the Exhibit Hall.
  31. 31. Future directions • Tune the viewer for VR • Complete the single viewer work • Focus on supporting larger models • Improve WebVR support • Now uses pose info, not deviceorientation • Test support with tethered headsets • Oculus Rift CR, HTC Vive Pre

×