1. VIRTUAL REALITY:
SENSING THE POSSIBILITIES
Professor Mark Billinghurst
Director Empathic Computing Lab
mark.billinghurst@unisa.edu.au
Dr. Ross Smith
Co-Director: Wearable Computer Laboratory
ross.smith@unisa.edu.au

2. Outline
5:50 – What is VR (Mark)
6:00 – History and Business of VR (Mark)
6:15 - Applications of VR (Ross)
6:25 - Tools for Developing VR (Ross)
6:35 - VR Research Directions (Mark)
6:45 – Demos (Mark and Ross)

3. Downloads to Try
• Search on iOS/Play Store for:
• Google Cardboard
• Cardboard Camera
• DiveCityCoaster
• Streetview
• Within

4. WHAT IS VIRTUAL REALITY?

5. Ivan Sutherland (1963)
• Sketchpad – first interactive graphics program

6. The Ultimate Display
“The ultimate display would, of course, be a room
within which the computer can control the
existence of matter. A chair displayed in such a
room would be good enough to sit in. Handcuffs
displayed in such a room would be confining, and
a bullet displayed in such a room would be fatal”.
Ivan Sutherland, 1965

7. An Invisible Interface

8. Virtual Reality
Computer generated multi-sensory simulation of an
artificial environment that is interactive and immersive.

10. What is Virtual Reality?
Virtual reality is..
a computer technology that replicates an
environment, real or imagined, and simulates a
user's physical presence and environment to
allow for user interaction. (Wikipedia)
• Defining Characteristics
• Environment simulation
• Presence
• Interaction

11. Defined in Terms of Presence
• Presence is the key to defining VR in terms of experience
• Presence is defined as the sense of being in an environment
• Telepresence is defined as the experience of presence in an
environment by means of a communication medium.
• A “virtual reality” is defined as a real or simulated environment
in which a perceiver experiences telepresence.

12. David Zeltzer’s AIP Cube
! Autonomy – User can to
react to events and stimuli.
! Interaction – User can
interact with objects and
environment.
! Presence – User feels
immersed through sensory
input and output channels
Interaction
Autonomy
Presence
VR
Zeltzer, D. (1992). Autonomy, interaction, and presence. Presence: Teleoperators
& Virtual Environments, 1(1), 127-132.

13. First VR Experience
• “This is so real..”
• https://www.youtube.com/watch?v=pAC5SeNH8jw

14. Key Technologies
• Autonomy
• Head tracking, body input
• Intelligent systems
• Interaction
• User input devices, HCI
• Presence
• Graphics/audio/multisensory output
• Multisensory displays
• Visual, audio, haptic, olfactory, etc

15. Types of VR
1
5

16. HISTORY OF VR

17. Early History (30,000 BC - )
The history of VR is rooted in human’s first
attempts to reproduce the world around them

18. 1800’s – Capturing Reality
• Panoramas (1790s)
• Immersive paintings
• Photography (1820-30s)
• Oldest surviving photo (Niépce, 1826)
• Stereo imagery (1830s)
• Wheatstone (1832)
• Brewster (1851)
• Movies (1870s)
• Muybridge (1878)
• Roundhay Garden Scene (1888)

19. Stereo Viewers
Wheatstone (1832)
Brewster (1860)

20. Viewmaster (1939)

21. 1900s – Interactive Experiences
• Early Simulators (<1960s)
• Flight simulation
• Sensorama (1955)
• Early HMDs (1960s)
• Philco, Ivan Sutherland
• Military + University Research (1970-80s)
• US Airforce, NASA, MIT, UNC
• First Commercial Wave (1980-90s)
• VPL, Virtual i-O, Division, Virtuality
• VR Arcades, Virtual Boy

22. Ivan Sutherland HMD (1968)
https://www.youtube.com/watch?v=NtwZXGprxag

23. Early Experimenters (1950’s – 80’s)
Helig 1956
Sutherland 1965
Furness 1970’s

24. The First Wave (1980’s – 90’s)
NASA 1989
VPL 1990’s
Virtuality 1990’s

25. Desktop VR - 1995
• Expensive - $150,000+
• 2 million polys/sec
• VGA HMD – 30 Hz
• Magnetic tracking

26. Rise of Commercial VR Companies
• W Industries/Virtuality (1985 - 97)
• Location based entertainment
• Virtuality VR Arcades
• Division (1989 – 1998)
• Turn key VR systems
• Visual programming tools
• Virtual i-O (1993 -1997)
• Inexpensive gamer HMDs
• Sense8 (1990 - 1998)
• WorldToolKit, WorldUp
• VR authoring tools

27. • April 2007 Computer World
• VR Voted 7th on of 21 biggest flops
• MS Bob #1

28. Second Wave (2010 - )
• Palmer Luckey
• HMD hacker
• Mixed Reality Lab (MxR)
• Oculus Rift (2011 - )
• 2012 - $2.4 million kickstarter
• 2014 - $2B acquisition FaceBook
• $350 USD, 110o FOV

29. • sddg

30. Oculus Rift
Sony Morpheus
HTC/Valve Vive
2016 - Rise of Consumer HMDs

31. Desktop VR 2016
• Graphics Desktop
• $1,500 USD
• >4 Billion poly/sec
• $600 HMD
• 1080x1200, 90Hz
• Optical tracking
• Room scale

32. HTC Vive
• Room scale tracking
• Gesture input devices

33. Example Vive App – Tilt Brush
https://www.youtube.com/watch?v=ijukZmYFX-0

34. Computer Based vs. Mobile VR

35. Mobile VR: Google Cardboard
• Released 2014 (Google 20% project)
• >5 million shipped/given away
• Easy to use developer tools
+ =

36. Multiple Mobile VR Viewers Available

37. THE BUSINESS OF VR

38. Gartner Hype Cycle (2016)

39. Gartner Hype Cycle (2016)

40. Companies
• sdfa

41. AR/VR Investments
• > $1 Billion USD from VCs, > 120 deals in US alone

42. HMD Sales
• asdf

43. Market Size

44. AR / VR Market Size
• sadf

45. • In 2016 – 46m possible desktop VR users vs. 400 m mobile VR users
• https://thoughts.ishuman.co/vr-will-be-mobile-11529fabf87c#.vfcjzy1vf

46. • zxcvz

47. Potential for Disruption (BDMI)
• asD

48. Why 2016 won’t be like 1996
• It’s not just VR anymore
• Huge amount of investment
• Inexpensive hardware platforms
• Easy to use content creation tools
• New devices for input and output
• Proven use cases – no more Hype!
• Most important: Focus on User Experience

49. Conclusion
• Virtual Reality has a long history
• > 50 years of HMDs, simulators
• Key elements for VR were in place by early 1990’s
• Displays, tracking, input, graphics
• Strong support from military, government, universities
• First commercial wave failed in late 1990’s
• Too expensive, bad user experience, poor technology, etc
• We are now in second commercial wave
• Better experience, Affordable hardware
• Large commercial investment, Significant installed user base
• Will Virtual Reality be a commercial success this time?

50. APPLICATIONS OF VR

51. Types of VR Experiences
• Immersive Spaces
• 360 Panorama’s/Movies
• High visual quality
• Limited interactivity
• Changing viewpoint orientation
• Immersive Experiences
• 3D graphics
• Lower visual quality
• High interactivity
• Movement in space
• Interact with objects

52. Immersive Panorama
• High quality 360 image or video surrounding user
• User can turn head to see different views
• Fixed position

53. Example Applications
• VRSE – Storytelling for VR
• http://vrse.com/
• High quality 360 VR content
• New York Times VR Experience
• NYTVR application
• Documentary experiences
• Vrideo
• http://vrideo.com/
• Streamed immersive movies

54. Applications: Virtual Tours
• Visualise architectural diagrams
• Tools such as Autodesk, Revit supporting VR
• Metricon, 3D tours

55. Demo: Metricon VR Tours
• https://www.youtube.com/watch?v=vN16M2RZ8k4

56. Applications: Games and Entertainment
• 100s of games

57. Star Wars Trials On Tatooine VR Game
• https://www.youtube.com/watch?v=zb2HTngc8xE

58. 100’s of Google Play Cardboard apps

59. Applications: Sports and Entertainment
• www.Nexvr.com
• Live streaming events
• NBA Basketball VR
• (1 game per week)
• US Open Tennis VR
• Live Nation concerts

60. Virtual Campus Tour
• Current undergraduate student project
• Captured hundereds 360 photos
• Custom menus and navigation
• Allowing remote experience of facilities

61. Applications: Architectural Visualizations
• Combined virtual and real imagery
• Revit architectual diagrams
• Incorporated Street photography
• Available today from app store

62. Applications: Medical
• Virtual Reality in Clinical Applications
• Acute and Chronic Pain Modification
• Phobias, Post-traumatic stress disorder
• Eating disorders
• Benefits
• Low cost
• Repeatable

63. How do these systems work?
• Phobias
• Repeated experience reduces fear and anxiety
• Wound dressing
• Distraction
• Lower pain self report compared to opiates
• Chronic pain
• Illusion based systems

64. Application: Chronic Neck Pain
•Injuries happen to everyone, for some pain persists
after injury healing
•1 in 5 Australians suffer chronic pain
•Current Treatments provide no long term benefit,
AND often cause further complications.
•Science suggests these methods don’t target the
main contributor: the brain.
•Pain with movement depends on perceived, not
actual movement (Harvie, Smith et al. 2015).
•Shows real time manipulation of pain using MoOVi

65. Application: Virtual Training
• Virtual Dissection
• Cadaver lab
• OH&S Training tools (Forklift)
• Learning benefits
• Easily repeatable
• No harm to specimen
• No exposure to chemicals
• http://www.tactustech.com

66. TOOLS FOR DEVELOPING
VR EXPERIENCES

67. Capturing Panoramas
• Stitching photos together
• Image Composite Editor (Microsoft)
• AutoPano (Kolor)
• Using 360 camera
• Ricoh Theta-S
• Fly360

68. Capturing 360 images
Kodac 360 Fly 360 Gear 360 Theta S Nikon
LG 360 Pointgrey Ladybug Panono 360
Bublcam

69. 360 Images with Smartphone
• Cardboard Camera
• Capture 360 panoramas
• S3tch together images on phone
• View in VR on Cardboard

70. What You Need
• Cardboard Viewer/VR Viewer
• https://www.google.com/get/cardboard/
• Smart phone
• Android/iOS
• Authoring Tools/SDK
• Google VR SDK
• Unity/Unreal game engine
• Non programming tools
• Content
• 3D models, video, images, sounds

71. Software Tools
• Low level SDKs
• Need programming ability
• Java, C#, C++, etc
• Example: Google VR SDK (iOS, Android)
• https://developers.google.com/vr/
• Game Engines
• Powerful, need scripting ability
• Unity - https://unity3d.com/
• Unreal - https://www.unrealengine.com/vr
• Combine with VR plugins (HMDs, input devices)
• Google VR Unity plugin

72. Unity Interface

73. Developing Software
• Unity Game Engine
• Unreal Game Engine
• Asstore access (not just for developers)
• Comprehensive tools to support
• Physics
• Lighting effects
• Animations
• Colliders
• Scripts
• Audio
• Particle systems (smoke, fog..)
•

74. Tools for Non-Programmers
• Focus on Design, ease of use
• Visual Programming, content arrangement
• Examples
• Insta-VR – 360 panoramas
• http://www.instavr.co/
• Vizor – VR on the Web
• http://vizor.io/
• A-frame – HTML based
• https://aframe.io/
• ENTiTi – Both AR and VR authoring
• http://www.wakingapp.com/
• Eon Creator – Drag and drop tool for AR/VR
• http://www.eonreality.com/eon-creator/

75. InstaVR (Web Based)
• http://www.instavr.co/
• Development flow
• Capture 360 photos
• Upload to web
• Add content flow
• Add links, hotspots
• Publish to platform:
• Android, IOS, Gear VR, Web,
• Daydream, Oculus Rift, HTC Vive, Playstation VR

76. Demo: InstaVR
• https://www.youtube.com/watch?v=qk7CxZ0yB2s

77. VR RESEARCH DIRECTIONS

78. Areas for VR Research
• Display
• Wide field of view, new display technologies
• Tracking
• Precise tracking, wide area
• Interaction
• Natural gesture interaction, human factors
• Authoring Tools
• Content capture, authoring in VR
• Applications
• Collaboration, large scale VR

79. Bare Hands
• Using computer vision to track bare hand input
• Creates compelling sense of Presence, natural interaction
• Challenges need to be solved
• Not having sense of touch
• Line of sight required to sensor
• Fatigue from holding hands in front of sensor

80. Example: Leap Motion
• https://www.youtube.com/watch?v=QD4qQBL0X80

81. Eye Tracking
• Technology
• Shine IR light into eye and look for reflections
• Advantages
• Provides natural hands-free input
• Gaze provides cues as to user attention
• Can be combined with other input technologies

82. Example: FOVE VR Headset
• Eye tracker integrated into VR HMD
• Gaze driven user interface, foveated rendering
• https://www.youtube.com/watch?v=8dwdzPaqsDY

83. Pedestrian Devices
• Pedestrian input in VR
• Walking/running in VR
• Virtuix Omni
• Special shoes
• http://www.virtuix.com
• Cyberith Virtualizer
• Socks + slippery surface
• http://cyberith.com

84. Cyberith Virtualizer Demo
• https://www.youtube.com/watch?v=R8lmf3OFrms

85. Social VR
• Facebook Social Virtual Reality, AltspaceVR
• Bringing Avatars into VR space
• Natural social interaction

86. Demo: Facebook Social VR
• https://www.youtube.com/watch?v=W71o4RbckNA

87. CONCLUSIONS

88. Conclusions
• Virtual Reality has long history
• 60 years of experimentation
• Recent VR explosion
• Desktop VR, mobile VR
• Rapidly growing market
• Many areas for commercial application
• Significant Research Opportunities
• Collaboration, applications, display, interaction

89. DEMONSTRATIONS

90. Hands on Demonstrations
• HTC Vive
• PC Based VR
• Wide area tracking
• Two handed input
• Google Cardboard
• Mobile VR
• iOS/Android applications
• Gear VR
• Mobile VR
• Improved Tracking
• Touch pad input

91. Google Cardboard Demonstration
Step 2:
- Search for a location
- I chose “Brighton Beach”
- Click “red dot”
Step 3:
- Select from available
- 360 images and click
Step 4:
- Click on cardboard icon
- Place in viewer
Step 1: Download “Street View” from Apple or Play store

92. @marknb00
mark.billinghurst@unisa.edu.au
ross.smith@unisa.edu.au