2. What are we talking about ??
What is WebXR : From real to « yet »
undefined Reality
From 360° to Augmented reality :
−Devices
−Experiences
−Interactions
Can we anticipate:
−Technologies
−Experiences
−Interactions
Session UX/UI - Toward WebXR
thomas@blumenlab.com
3. What is XR ?
Superset of various types of “Reality”, with “X”
as the variable.
Milgram’s Reality-Virtuality Continuum
− Tangible User Interface (TUI) : Interaction with real objects.
− Augmented Reality (AR) : Adding information on the real worl.
− Augmented Virtuality (AV) : Real information to computer environment.
− Virtual Reality (VR) : Immersion in complete computer environment.
R
X
Session UX/UI - Toward WebXR
thomas@blumenlab.com
4. What is XR ?
Reality not limited to one sense :
− Sound (Binaural, ambisonic, etc.)
− Touch (haptic, proprioception, etc.)
− Smell (Odor synthesizers, etc.)
− Etc.
We are just at the beginning !!
− Technology is becoming mature
− Network and Cloud offers computing
− Almost XReady!!
Session UX/UI - Toward WebXR
thomas@blumenlab.com
5. What is the Web ?
The most accessible medium worldwide
−Multi-Canal, Multi-Support, Multi-Content, etc.
−Built on top of the Internet (IP-Network)
Session UX/UI - Toward WebXR
thomas@blumenlab.com
6. What is the Web ?
Built on open standards:
−Allows Interoperability among devices.
−Specification are open (W3C), only browsers have
to integrate them.
Session UX/UI - Toward WebXR
thomas@blumenlab.com
7. What should be webXR ?
Open Standard:
−New form of “Reality” should be accessible:
●For consumers: Through web interfaces and APIs.
●For developers: Through web languages and
specifications (Next HTML X, Javascript releases).
Interoperability:
−Cross devices deployment
−Avoiding One SDK = One App = One experience
Scalability:
−XR should be accessible to everybody worldwide.
−XR should be integrated in most browsers.
Session UX/UI - Toward WebXR
thomas@blumenlab.com
8. Panoramic and 360°
Expose space at 360°
−Can turn around an object.
−Can look around an environment.
−Can interact with elements (click/keyboard)
Interesting for experiences like:
−Virtual tour/visit.
−Visualization of an object
Not really interactive!
Session UX/UI - Toward WebXR
thomas@blumenlab.com
9. Web360° : Requirements
No need of a special browser.
Addition of simple lightweight javascript library
Panoramic pictures:
−Need a special camera and/or a “stitching”
framework to create a 360 content.
−Picture exposure can be tricky (360°)
−Pictures may have different format and resolution:
equirectangular, cubic, cylindrical, etc.
Session UX/UI - Toward WebXR
thomas@blumenlab.com
12. Web360° : Interaction
Click/Keyboard: Select objects/div, explore
360°
Mobile Swipe: Let turn around a 360 scene.
Mostly based on captured images with overlay
Hotspots: Add interactivity in the 360° page:
○ Launch new panoramic/hyperlinks
○ Select feature to interact with the content.
○ Access to additional information (expanding
a div)
Session UX/UI - Toward WebXR
thomas@blumenlab.com
14. 3D
3D elements are extensively used in:
○Game/Special effects/Representation
3D modeling is quite expensive
(time/knowledge/computation):
○Rendering/Meshes/polygons.
○Needs computation power to render.
When 3D environment/Characters done:
○Easy to change and adapt.
○Can move the “virtual camera”.
○Physical engines to simulate real environment.
Session UX/UI - Toward WebXR
thomas@blumenlab.com
15. Web3D: Requirements
Today, WebGL allows to render 2D and 3D
objects in the browser.
WebGL allows to use OpenGL ES 2.0 API.
−Inside an HTML <canvas> element
Most modern browser including in smartphone
have webGL enabled.
Will use the GPU of graphical cards.
Session UX/UI - Toward WebXR
thomas@blumenlab.com
18. Web3D : Interactivity
Mouse/keyboard: Easy desktop control
Joypad/Gamepad: Propose extended
functionalities and buttons.
Other devices: Myo, Magic Leap, etc.
Ability to enter in the 3D scene:
−Create 3D world that you can explore.
−Physic engine to render gravity, physical
boundaries, etc.
Embedding real time content (webRTC)
Session UX/UI - Toward WebXR
thomas@blumenlab.com
19. Web3D : Open Source
Javascript 3D Libraries:
− three.js: https://threejs.org/
− babylon.js: https://www.babylonjs.com/
Creating 3D assets:
− https://www.blender.org/
− https://3dc.io/
Share 3D content:
− Sketchfab: https://sketchfab.com/
− Archive3D: https://archive3d.net/
WebGL: Khronos: https://www.khronos.org/
Session UX/UI - Toward WebXR
thomas@blumenlab.com
20. Virtual Reality (VR)
Immersive medium:
−Need a special headset to enter in VR.
Technology:
−Not new: Based on stereoscopic display.
−New: Gyroscope, displays, smartphone.
Can look all around inside the immersive
content: stereoscopic images/video/3D
−Stereoscopic video/images/3D.
Session UX/UI - Toward WebXR
thomas@blumenlab.com
22. VR : Devices
A large choice of devices/headsets:
−Fully integrated: Oculus, HTC Vive,
−With smart phones: Google Cardboard, daydream,
Samsung Gear VR.
Google: Daydream and cardboard
Session UX/UI - Toward WebXR
thomas@blumenlab.com
23. WebVR : Devices
Each headset has it’s own SDK/Store/Feature
Need a high level interface for developers:
−WebVR.
Session UX/UI - Toward WebXR
thomas@blumenlab.com
26. WebVR : Interaction
Through the eyes:
− Gaze: An element is activated when fixing it during a certain
duration.
− Hover: Same as with mouse, can activate element when
looking over.
Through standard controllers:
− Buttons, remotes, gamepads, etc
− Activate an action when pressed.
New haptic controllers:
− Gesture: Myo - https://www.myo.com/
− Virtual hands: Leap Motion - https://www.leapmotion.com/
Session UX/UI - Toward WebXR
thomas@blumenlab.com
27. WebVR : open source
Need to have a compliant browser:
−See available browser: https://webvr.rocks/
−A-frame: https://aframe.io/
Session UX/UI - Toward WebXR
thomas@blumenlab.com
28. Mixed Reality (MR)
Immersive and “experiencing” environment.
Immersion with the addition of:
○Tracking movements in space.
○Evolve, manipulate, Explore the
environment.
Additional feature can be added:
Session UX/UI - Toward WebXR
thomas@blumenlab.com
32. WebMR : Interaction
Traditional controllers:
○Remotes, gamepads, wireless pads.
Need additional hardware for interaction:
○Gesture Space sensors
○Depth camera/sensors
Can be mix with other evolved controllers:
○Gesture (Myo), LeapMotion (handtracking).
○Specific installation (Birdy simulator, etc.)
Session UX/UI - Toward WebXR
thomas@blumenlab.com
33. WebMR : Open source
Basically the same requirements than for
webVR:
−See available browser: https://webvr.rocks/
−A-frame: https://aframe.io/
−BabylonJS: https://www.babylonjs.com/
Session UX/UI - Toward WebXR
thomas@blumenlab.com
34. Augmented Reality (AR)
Add a layer on top of what you see:
○Can be a 3D assets/scene, a picture,
animation, etc.
Need to see through a device:
○Can be a very specific hardware
○Commodity hardware for light experience
AR can be applyed to other senses:
○Mainly Sound and connected objects.
Session UX/UI - Toward WebXR
thomas@blumenlab.com
37. webAR Devices.
Microsoft Hololens
(no web support yet:
HoloJS)
Google Project Tango
ARCore
iPhone 11
ARKit
Session UX/UI - Toward WebXR
thomas@blumenlab.com
40. WebAR : Interactivity
Need a camera to see the environment
○ Can change the projected image.
Manipulation of “superposed” objects:
○ Swipe, Hover, Finger, etc.
Need a device to communicate:
○ IoT devices, sensors, actuators, etc.
Other Augmented senses to be added.
Session UX/UI - Toward WebXR
thomas@blumenlab.com
42. WebXR : UI/UX ?
Can we anticipate next Interfaces and usage:
○Holograms
○Tactile interfaces
○Smell interfaces
○Speech control, etc.
Can we merge existing interfaces with new one?
Can we integrate it with the Web?
What are going to be ne next button?
Is it the end of smart phones?
Session UX/UI - Toward WebXR
thomas@blumenlab.com