Rob Manson, CEO and co-founder of awe.media, presents on the evolution of mixed reality (MR) using open web standards, emphasizing the significance of augmented and virtual reality. He discusses various interaction modes for MR and outlines future needs for technology development, including improved computer vision capabilities. The document also highlights the current state of VR market growth and anticipates mainstream adoption by 2020.

1. Rob Manson, CEO & co-founder - https://awe.media - follow me on twitter @nambor
Augmenting, Mixing & Extending
Reality on the web

2. Who am I?
Rob Manson, CEO & co-founder
https://awe.media

3. Who am I?
Rob Manson, CEO & co-founder
https://awe.media
The original evangelist for Mixed Reality using open Web Standards

4. Who am I?
Rob Manson, CEO & co-founder
https://awe.media
The original evangelist for Mixed Reality using open Web Standards
Invited Expert

5. Who am I?
Rob Manson, CEO & co-founder
https://awe.media
The original evangelist for Mixed Reality using open Web Standards
Invited Expert
W3C

6. Who am I?
Rob Manson, CEO & co-founder
https://awe.media
The original evangelist for Mixed Reality using open Web Standards
Invited Expert
W3C
Khronos Group

7. Who am I?
Rob Manson, CEO & co-founder
https://awe.media
The original evangelist for Mixed Reality using open Web Standards
Invited Expert
W3C
Khronos Group
ISO

8. A brief history…

9. How did we get here?

10. How did we get here?

11. How did we get here?

12. How did we get here?

13. How did we get here?

14. How did we get here?

15. How did we get here?

16. How did we get here?

17. How did we get here?
2017
The web gets
Computer Vision!

18. What predictions did we get right?
Mixed Reality will be possible in the web

19. What predictions did we get right?
Mixed Reality will be possible in the web
ARStandards Workshop Seoul, 2010

20. What predictions did we get right?
Mixed Reality Web Standards test harness

21. What predictions did we get right?
Mixed Reality Web Standards test harness
isweb3here.com, 2011

22. What predictions did we get right?
OS-level spatial tracking would be released

23. What predictions did we get right?
OS-level spatial tracking would be released
W3C WebVR Workshop, 2016

24. But getting the timing right
is really hard…

26. But Tim is only right
if you give up too soon!

27. Here are the different ways
people can interact with Mixed Reality

28. Mixed Reality Interaction Mode – Public
- Whole body in fov
- More spectators
- Limited distribution

29. Mixed Reality Interaction Mode – Personal
- Upper/whole body in fov
- Fewer spectators
- Wider distribution

30. Mixed Reality Interaction Mode – Private
- Extremeties or head in fov
- 1-2 direct participants
- Wider distribution & locations

31. Mixed Reality Interaction Mode – Intimate
- Only extremeties in fov
- 1 direct participant
- Pervasive but currently limited distribution

32. Where are we up to now?

33. What is Mixed Reality?
Image Credit
Paul Milgram - 1994

34. What is Augmented Reality?
Augmented Reality allows the user to see the real world,
with virtual objects superimposed upon
or composited with the real world.
Ron Azuma - 1997
Augmented Reality…
1. Combines the real and virtual
2. Is interactive in real time
3. Is registered in 3D

35. What is Extended Reality?
Image Credit

36. What is Extended Reality?
Image Credit

37. What is Extended Reality?
Image Credit

39. Image Credit

41. How do you “View” VR & AR?
Have a suitable HMD
v
1
2Have a valid account
with the HMD vendor's
app store
3
Complete the HMD
vendor's first use
4
v
Search for & find the app
in their app store
5 Download/install
the app through
their app store
6Often requires system
or app updates also
v
7 Select/play the
downloaded app
!
MOBILE ONLY APP
Have a valid account
with the Mobile OS
vendor's app store
v
1
2
Find the app in
their app store3
Download/install the
app through their
app store
4
v
Often requires
system or app
updates also
5
Download/install
the app through
their app store
6
Select/play the
downloaded app
v
7 !
View promo for app on
their website
WEB APP
1 Click on a web link to
open it in your
browser on any
device
HMD ONLY APP

42. How do you “Create” VR & AR?
MOBILE ONLY APPs WEB APPSHMD ONLY APPS
Hire or be a developer
v
1
2
Download & get to know (&
deal with the various
issues/quirks) the developer
SDK for the target platform
3 Develop your app/s
4
v
Test it on the target HMD
vendor platform/s
5
Convert media and
3D assets to the
app/s format
6
Sign & submit your
app/s to the app store/s
& wait for approval/s
v
7
Publish & distribute
links to one or more
app stores
!
Download & get to know (&
deal with the various
issues/quirks) the developer
SDK for the target mobile OS
v
1
2
Develop your app/s
3
Test it on the target HMD
vendor platform/s 4
v
Convert media and
3D assets to the
app/s format
5
Sign & submit your app/s
to the app store/s & wait
for approval/s
6
Publish & distribute
links to one or more
app stores
v
7
Hire or be a developer
8
Create promotional web
content to drive people
to the app/s ! 8
Create promotional web
content to drive people
to the app/s
Get an account with
awe.media
*blatant plug J
v
1
2
Upload any types of
common media you
need (photos, videos,
3D & more)
3
Manipulate these objects
and add rich interactions
to create your VR & AR
experiences
4
Open your browser on
any type of device
v
5 Share a single link that
points directly to your web-
based VR/AR experience

43. What’s evolving as we speak?

44. Depth sensing
Depth Sensing is starting to become commonplace
Image Credit

45. OS-level pose tracking
Image Credit

46. WebVR/WebAR
Image Credit

47. WebVR/WebXR
Image Credit

48. Deep/Machine Learning
Image Credit

49. More cameras!
Image Credit

50. What’s needed in the future?
(The missing pieces)

51. Camera intrinsics web API
Image Credit
A more accurate underlying geometry for Computer Vision

52. Optimised pixel access
Zero copy texture transfer & faster workers
Image Credit

53. Lets look at how this is all
playing out in the market

54. Why has VR exploded recently?
VR has things to sell
Image Credit

55. Why has VR exploded recently?
The power of the s-curve
Image Credit

56. Where are we now?
In reality, VR is a tiny tiny market right now

57. Where are we now?
In reality, VR is a tiny tiny market right now
More iPhone 8s than HMDs by the end of the year!

58. What’s happening in the global mobile market?
Image Credit

59. The good news for iOS is the majority are now iOS11
Image Credit

60. Where are we now?
Installed web browsers
20 MILLION (END 2017)
Includes both the mobile and
HMD users above
2 BILLION
3 BILLION
Mobile devices
HMD only

61. Where are we now?
Installed web browsers
20 MILLION (END 2017)
Includes both the mobile and
HMD users above
2 BILLION
3 BILLION
Mobile devices
HMD only

62. Where are we now?
Installed web browsers
20 MILLION (END 2017)
Includes both the mobile and
HMD users above
2 BILLION
3 BILLION
Mobile devices
HMD only

63. Where are we now?
Installed web browsers
20 MILLION (END 2017)
Includes both the mobile and
HMD users above
2 BILLION
3 BILLION
Mobile devices
HMD only

64. Immersive Computing will become Pervasive
Image Credit
What is likely in the future?

65. What are the real market trends?
This will take more than 3-5 years
Image Credit

66. Mainstream adoption won’t “start” until at least 2020
Image Credit
How long until we get our magic glasses?

67. What will drive market changes between now & then?
Convenient multi-use mobile form factors are the key today
Image Credit

69. Questions? / Discussion
Rob Manson, CEO & co-founder - https://awe.media - follow me on twitter @nambor