SlideShare a Scribd company logo
1 of 63
Download to read offline
LECTURE 7: EXAMPLE
VR APPLICATIONS
COMP 4026 – Advanced HCI
Semester 5 - 2019
Bruce Thomas, Mark Billinghurst, Gun Lee
University of South Australia
September 10th 2019
Lecture 6: Recap
• Interaction Design for VR
• Iterative method for designing VR experiences
• Applying well known ID techniques to VR
• Interaction Design Process
• Needs analysis
• Experience Design
• System Prototyping
• Evaluation
The Interaction Design Process
Evaluate
(Re)Design
Identify needs/
establish
requirements
Build an
interactive
version
Final Product
Develop alternative prototypes/concepts and compare them
And iterate, iterate, iterate....
Methods for Identifying User Needs
Learn from
people
Learn from
analogous
settings
Learn from
Experts
Immersive
yourself in
context
VR Design Considerations
• Use UI Best Practices
• Adapt know UI guidelines to VR
• Use of Interface Metaphors/Affordances
• Decide best metaphor for VR application
• Design for Humans
• Use Human Information Processing model
• Design for Different User Groups
• Different users may have unique needs
• Design for the Whole User
• Social, cultural, emotional, physical cognitive
Typical Development Steps
▪ Sketching
▪ Storyboards
▪ UI Mockups
▪ Interaction Flows
▪ Video Prototypes
▪ Interactive Prototypes
▪ Final Native Application
Increased
Fidelity &
Interactivity
VR Prototyping Tools
• Low Fidelity
• Sketched Paper Interfaces – pen/paper, non-interactive
• Onride Photoshop tool – digital, non-interactive
• InstaVR - 360 web based tool, simple interactivity
• SketchBox – create VR interface inside VR
• Microsoft Marquette – mock up VR interfaces in VR
• High Fidelity
• EditorVR – Unity wrapper inside VR
• Unity/Unreal Game Engine – programming needed
Four Evaluation Paradigms
•‘quick and dirty’
•usability testing (lab studies)
•field studies
•predictive evaluation
Examples Mentioned
EXAMPLE VR APPLICATIONS
Virtual Reality Applications
• Ideal applications for VR should:
• Be strongly visual, have 3D spatial elements
• Benefit from first person immersion
• Benefit from 3D manipulation/navigation
• Support Autonomy, Interaction and Presence (AIP Cube)
• Etc..
Not Everything Should be Done in VR
Dr Fun - 1990
Many Possible Types of VR Applications
From https://www.slideshare.net/ampnewventures/virtual-reality-vr-continuum-amp-new-ventures
Potential Disruption for Existing Domains
https://www.slideshare.net/BDMIFund/the-emerging-virtual-reality-landscape-a-primer
Example VR Applications
• Education
• Google Expeditions
• Medicine
• Virtual Characters
• Entertainment
• The Void, Zero Latency
• Art + Design
• Tilt Brush
• Collaboration
• Facebook Spaces
EDUCATION
Google Expeditions
• https://edu.google.com/expeditions/
• Mobile VR Educational application (Android, iOS)
• Designed for classroom experiences
Google Expeditions
• Goal: Provide low cost educational VR experience
• Based on Google Cardboard VR platform
• Different roles:
• Guide— person leading an expedition on a tablet
• Explorer— person following an expedition on a phone.
• Usage
• Used by millions of students
• Over 1000 educational experiences developed
• Royal Collection Trust, American Museum of Natural History, etc.
Teacher Led VR Experiences
• Teacher/Guide uses tablet to control the experience
• Selects the virtual tour experience
• Guide sees tour script, can select immersive scenes to view
• Guide sees focus point and where individual students are looking
• Students connect as followers, look at what guides highlight
Guide Interface
System
• Hardware
• Google Cardboard mobile viewer
• Smart phones + tablet (class set)
• Wireless router
• Software
• Viewer and Guide applications (iOS/Android)
• 360 image/video VR experiences
Class set for 30 students
Example Experiences
• Over 1000 locations/experiences
• Great barrier reef, Great Wall of China, Grand Canyon, etc.
Demonstration
• https://www.youtube.com/watch?v=3MQ9yG_QfDA
Feedback
• Teacher/student survey (100 people)
• 65% experienced a “Wow” moment during Google expedition
• Noted the variety of educator styles and approaches possible
• People enjoyed “The feeling of ‘being’ there”
From https://www.slideshare.net/zoesujon/google-expeditions-virtual-reality-and-the-classroom
Limitations
• But 53% of participants identified some problems:
• Difficult for some people who wore glasses
• Some complained of eye strain, headaches or nausea
• Some staff were reluctant/resistant to use the leader tablet
• Issues of disabilities and inclusion
Key Findings
• Low cost VR/mobile VR can provide a valuable
educational experience
• Visit different locations, different times, etc.
• Teach interaction key
• Acting as guide, providing educational context
• VR requires more work
• Address simulator sickness, ergonomic issues, etc.
• Immersion/Presence creates learning
• Immersion creates memorable educational experience
Challenges/Solutions
• Making VR accessible
• Designing for phones, tablets, low cost viewers
• Synchronizing content with all viewers
• Teacher controlled viewing
• Teacher can guide experiences
• Engaging interaction on simple viewers
• Head pointing based interaction, button input
• Supporting Educational goals
• Providing compelling educational content
MEDICINE
Virtual Patients
• Problem
• Many doctors have poor doctor/patient skills
• Have limited opportunity during training to learn skills
• Solution
• Virtual patients that doctors can communicate with naturally
• Artificial agents with speech understanding
Typical System Setup
• Trainee in front of projection screen
• Speech and gesture recognition
• Intelligent agent on screen
Johnsen, K., Raij, A., Stevens, A., Lind, D. S., & Lok, B. (2007, April). The validity of a virtual
human experience for interpersonal skills education. In Proceedings of the SIGCHI conference
on Human factors in computing systems (pp. 1049-1058). ACM.
Demo:
• https://www.youtube.com/watch?v=xC70_tRGOOk
Key Findings
• Virtual Humans can replace actors in training
• interaction skills used with a virtual human translate to
the interaction skills used with a real human
• Students feel a strong sense of co-presence
• Having character respond to speech and gesture
increases immersion
• VR is capable of creating realistic characters
• Life size, intelligent backend, speech recognition
• Skills learnt transfer to real world
Challenges/Solutions
• Training in medical environment
• Design for training in medical exam room
• Use projected VR not HMDs
• Natural interaction
• Support speech and gesture interaction
• Tactile/haptic feedback
• Use prosthetics to add support for palpation and other
tactile interaction between doctor and virtual patient
• Supporting Educational goals
• Give virtual character domain knowledge
ENTERTAINMENT
Large Scale VR Gaming
• Provide multi-player VR gaming in warehouse space
• Examples
• The Void - https://www.thevoid.com/
• Zero Latency - https://zerolatencyvr.com/
Typical System
• Wide Area Tracking
• Computer vision, lights/reflective balls
• > 120 cameras for 300 m2 space
• Backpack VR system
• Haptic feedback, wireless HMD
• Real Props
• Tracked objects, walls
Tracking cameras
Backpack system
The Void Demo
• https://www.youtube.com/watch?v=XgetffuOgBA
Key Findings
• Wide area tracking possible
• vision based systems can create large scale wide areas
tracking, fast enough for game play
• Shared gameplay improves experience
• Focus on collaborative experiences, using avatar
representations and roll division
• Haptic feedback significantly increases presence
• Use of physical props (objects, walls)
• Content is king
• Systems need compelling content/game place
Challenges/Solutions
• Wide area tracking
• Computer vision tracking of people
• Over 100 cameras + multiple servers
• Freedom of movement
• Custom wireless VR backpacks
• Ruggedized HMDs, weapon props
• Natural interaction
• Redirected walking, tangible props
• Compelling content
• Multi-sensory feedback, custom game platform
ART + DESIGN
Tilt Brush
• Intuitive 3D immersive drawing/sculpting program
• Developed by Patrick Hackett and Drew Skillman 2014
• Acquired by Google in 2015
• https://www.tiltbrush.com/
Functionality
• Goal: Extremely natural 3D painting/sculpting
• User Interface
• Two handed interface designed for two controllers (Vive, Rift)
• Brush in dominant hand, tool palette in non-dominant
• Typical drawing functionality – color, brush width, undo/redo, etc..
• Content sharing
• Created content can be exported/shared in 2D/3D formats
Demo
• https://www.youtube.com/watch?v=TckqNdrdbgk
Artist Feedback
• https://www.youtube.com/watch?v=91J8pLHdDB0
Example Tilt Brush Sketches
• https://poly.google.com/
• Explore in desktop VR
Key Findings
• Use familiar tools
• Tilt brush interface has familiar sculpting/painting tools –
e.g. brush size, colour pallet, etc
• Use intuitive interface
• Two handed tools with natural metaphor – one hand for
pallet/menu, one hand for painting/sculpting
• Provide Magical experience
• Provide experience not possible in real world, e.g.
changing body scale, painting in 3D, etc.
• Create a community
• Provide ways for people to share content
Challenges/Solutions
• Intuitive Interface
• Very natural metaphor – painting in space
• Two handed interface – map to VR controllers
• Familiar menu objects from paint programs
• Need for limited training
• Provide in app training, tool tips
• Content sharing
• Enable content to be exported in variety of formats
• Video, animated GIFs, 2D images, 3D files
• Engaging Experience
• Provides novel immersive artistic experience
COLLABORATION
Facebook Spaces
• Collaborative VR environment
• VR meeting and interaction space (up to 4 people)
• Focus on communication
• Speech and gesture based
• https://www.facebook.com/spaces
System Interaction
• Designed for Oculus Rift/HTC Vive
• Upper body tracking, touch controllers
• Simple interaction
• Loading scenes, direct object manipulation
• Content creation
• Selfie pictures, simple sketching
Demo
• https://www.youtube.com/watch?v=PVf3m7e7OKU
Key Findings
• Minimal social cues okay
• Even simple avatars can provide rich social experience
• Create shared social context
• Important to place users in same shared Virtual Reality
environment/shared social context
• Audio is key
• Provide low latency audio, spatial audio cues
• Create a reason for communicating
• Why should people want to connect? Create shared
activity/reason for people to conference
Challenges/Solutions
• Create shared sense of Presence
• Use common background, shared objects
• Natural communication
• Support non-verbal behaviour, speech/gesture input
• Intuitive interaction
• Map real body motion onto Avatars
• Limited ability to navigate/move through environment
• Engaging Experience
• Shared content creation, experience capture
Other Examples
• Many other examples of collaborative VR
• Rec Room, High Fidelity, AltspaceVR
• Sansar, VR chat, etc..
Example: High Fidelity Worlds
• https://www.youtube.com/watch?v=-ivL1DDwUK4
OTHER APPLICATIONS
Collisions – Australian VR Film
• http://www.collisionsvr.com/
• https://www.youtube.com/watch?v=-NZHLtmNi_s
Best VR Apps of 2019 (Digital Trends)
• ALLUMETTE – VR Stop motion film
• Google Earth – Travel/geography
• Kingspray Graffiti – Art/content creation
• The FOO Show - VR Talk show
• Virtual Desktop – Use desktop in VR
• www.digitaltrends.com/virtual-reality/best-virtual-reality-apps/
Google Earth
• https://www.youtube.com/watch?v=SCrkZOx5Q1M
Allumette
• https://www.youtube.com/watch?v=AkzdxgMBDi8
KingSpray Graffiti
• https://www.youtube.com/watch?v=3ygZBR_WPmI
www.empathiccomputing.org
@marknb00
mark.billinghurst@unisa.edu.au

More Related Content

What's hot

Comp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research DirectionsComp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research DirectionsMark Billinghurst
 
Virtual Reality and its impact
Virtual Reality and its impactVirtual Reality and its impact
Virtual Reality and its impactShahriar Parvez
 
VR - Virtual reality
VR - Virtual realityVR - Virtual reality
VR - Virtual realityRohit Jain
 
Augmented Reality
Augmented Reality Augmented Reality
Augmented Reality slire
 
Augmented reality ppt
Augmented reality pptAugmented reality ppt
Augmented reality pptDark Side
 
AUGMENTED REALITY PPT's
AUGMENTED REALITY PPT'sAUGMENTED REALITY PPT's
AUGMENTED REALITY PPT'sVenu Gopal
 
Comp4010 Lecture5 Interaction and Prototyping
Comp4010 Lecture5 Interaction and PrototypingComp4010 Lecture5 Interaction and Prototyping
Comp4010 Lecture5 Interaction and PrototypingMark Billinghurst
 
2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: Perception2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: PerceptionMark Billinghurst
 
virtual reality Barkha manral seminar on augmented reality.ppt
virtual reality Barkha manral seminar on augmented reality.pptvirtual reality Barkha manral seminar on augmented reality.ppt
virtual reality Barkha manral seminar on augmented reality.pptBarkha Manral
 
Advanced Methods for User Evaluation in AR/VR Studies
Advanced Methods for User Evaluation in AR/VR StudiesAdvanced Methods for User Evaluation in AR/VR Studies
Advanced Methods for User Evaluation in AR/VR StudiesMark Billinghurst
 
Augmented Reality
Augmented RealityAugmented Reality
Augmented RealityKaviyaraj R
 
2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR SystemsMark Billinghurst
 
Augmented reality
Augmented reality Augmented reality
Augmented reality pneumonia
 
2022 COMP4010 Lecture1: Introduction to XR
2022 COMP4010 Lecture1: Introduction to XR2022 COMP4010 Lecture1: Introduction to XR
2022 COMP4010 Lecture1: Introduction to XRMark Billinghurst
 

What's hot (20)

Augmented Reality
Augmented RealityAugmented Reality
Augmented Reality
 
Comp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research DirectionsComp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research Directions
 
Virtual Reality and its impact
Virtual Reality and its impactVirtual Reality and its impact
Virtual Reality and its impact
 
Virtual Reality
Virtual RealityVirtual Reality
Virtual Reality
 
VR - Virtual reality
VR - Virtual realityVR - Virtual reality
VR - Virtual reality
 
ISS2022 Keynote
ISS2022 KeynoteISS2022 Keynote
ISS2022 Keynote
 
Augmented Reality
Augmented Reality Augmented Reality
Augmented Reality
 
Augmented reality ppt
Augmented reality pptAugmented reality ppt
Augmented reality ppt
 
AUGMENTED REALITY PPT's
AUGMENTED REALITY PPT'sAUGMENTED REALITY PPT's
AUGMENTED REALITY PPT's
 
Comp4010 Lecture5 Interaction and Prototyping
Comp4010 Lecture5 Interaction and PrototypingComp4010 Lecture5 Interaction and Prototyping
Comp4010 Lecture5 Interaction and Prototyping
 
Virtual reality
Virtual realityVirtual reality
Virtual reality
 
2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: Perception2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: Perception
 
virtual reality Barkha manral seminar on augmented reality.ppt
virtual reality Barkha manral seminar on augmented reality.pptvirtual reality Barkha manral seminar on augmented reality.ppt
virtual reality Barkha manral seminar on augmented reality.ppt
 
Advanced Methods for User Evaluation in AR/VR Studies
Advanced Methods for User Evaluation in AR/VR StudiesAdvanced Methods for User Evaluation in AR/VR Studies
Advanced Methods for User Evaluation in AR/VR Studies
 
Augmented Reality
Augmented RealityAugmented Reality
Augmented Reality
 
2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems
 
Virtual Reality
Virtual RealityVirtual Reality
Virtual Reality
 
Augmented reality
Augmented reality Augmented reality
Augmented reality
 
2022 COMP4010 Lecture1: Introduction to XR
2022 COMP4010 Lecture1: Introduction to XR2022 COMP4010 Lecture1: Introduction to XR
2022 COMP4010 Lecture1: Introduction to XR
 
Virtual reality
Virtual realityVirtual reality
Virtual reality
 

Similar to Lecture7 Example VR Applications

COMP 4010 Lecture 6: VR Applications
COMP 4010 Lecture 6: VR ApplicationsCOMP 4010 Lecture 6: VR Applications
COMP 4010 Lecture 6: VR ApplicationsMark Billinghurst
 
COMP 4010: Lecture 6 Example VR Applications
COMP 4010: Lecture 6 Example VR ApplicationsCOMP 4010: Lecture 6 Example VR Applications
COMP 4010: Lecture 6 Example VR ApplicationsMark Billinghurst
 
Using Interaction Design Methods for Creating AR and VR Interfaces
Using Interaction Design Methods for Creating AR and VR InterfacesUsing Interaction Design Methods for Creating AR and VR Interfaces
Using Interaction Design Methods for Creating AR and VR InterfacesMark Billinghurst
 
De diepte in met virtual reality
De diepte in met virtual realityDe diepte in met virtual reality
De diepte in met virtual realitySURF Events
 
Lecture 6 Interaction Design for VR
Lecture 6 Interaction Design for VRLecture 6 Interaction Design for VR
Lecture 6 Interaction Design for VRMark Billinghurst
 
COMP 4010: Lecture 5 - Interaction Design for Virtual Reality
COMP 4010: Lecture 5 - Interaction Design for Virtual RealityCOMP 4010: Lecture 5 - Interaction Design for Virtual Reality
COMP 4010: Lecture 5 - Interaction Design for Virtual RealityMark Billinghurst
 
COMP 4010 - Lecture 5: Interaction Design for Virtual Reality
COMP 4010 - Lecture 5: Interaction Design for Virtual RealityCOMP 4010 - Lecture 5: Interaction Design for Virtual Reality
COMP 4010 - Lecture 5: Interaction Design for Virtual RealityMark Billinghurst
 
Applying virtual environments in distance learning of product development
Applying virtual environments in distance learning of product developmentApplying virtual environments in distance learning of product development
Applying virtual environments in distance learning of product developmentHAMK Design Factory
 
Tech Tools to Support Communication Competency
Tech Tools to Support Communication CompetencyTech Tools to Support Communication Competency
Tech Tools to Support Communication Competencyloater
 
Coil2012 torcivia calixv4
Coil2012 torcivia calixv4Coil2012 torcivia calixv4
Coil2012 torcivia calixv4LPCalix
 
Digital toolbox for 21st Century Learning
Digital toolbox for 21st Century LearningDigital toolbox for 21st Century Learning
Digital toolbox for 21st Century LearningHawkesdale P12 College
 
Website Usability & User Experience: Veel bezoekers, weinig klanten?
Website Usability & User Experience: Veel bezoekers, weinig klanten?Website Usability & User Experience: Veel bezoekers, weinig klanten?
Website Usability & User Experience: Veel bezoekers, weinig klanten?Johan Verhaegen
 
Designing and Evaluating Virtual Reality for Learning
Designing and Evaluating Virtual Reality for LearningDesigning and Evaluating Virtual Reality for Learning
Designing and Evaluating Virtual Reality for LearningUniversity of Sheffield
 
SL formal_education
SL formal_educationSL formal_education
SL formal_educationJason Murray
 
Ideas for using Web 2.0 tools in the classroom - updated September 2016
Ideas for using Web 2.0 tools in the classroom - updated September 2016Ideas for using Web 2.0 tools in the classroom - updated September 2016
Ideas for using Web 2.0 tools in the classroom - updated September 2016talkPrimaryICT
 
3D Virtual Worlds for Professional Development and Lifelong Learning
3D Virtual Worlds for Professional Development and Lifelong Learning3D Virtual Worlds for Professional Development and Lifelong Learning
3D Virtual Worlds for Professional Development and Lifelong LearningSheila Webber
 
Using your projector in your classroom
Using your projector in your classroomUsing your projector in your classroom
Using your projector in your classroomHeather Lewis
 

Similar to Lecture7 Example VR Applications (20)

COMP 4010 Lecture 6: VR Applications
COMP 4010 Lecture 6: VR ApplicationsCOMP 4010 Lecture 6: VR Applications
COMP 4010 Lecture 6: VR Applications
 
COMP 4010: Lecture 6 Example VR Applications
COMP 4010: Lecture 6 Example VR ApplicationsCOMP 4010: Lecture 6 Example VR Applications
COMP 4010: Lecture 6 Example VR Applications
 
Using Interaction Design Methods for Creating AR and VR Interfaces
Using Interaction Design Methods for Creating AR and VR InterfacesUsing Interaction Design Methods for Creating AR and VR Interfaces
Using Interaction Design Methods for Creating AR and VR Interfaces
 
De diepte in met virtual reality
De diepte in met virtual realityDe diepte in met virtual reality
De diepte in met virtual reality
 
Lecture 6 Interaction Design for VR
Lecture 6 Interaction Design for VRLecture 6 Interaction Design for VR
Lecture 6 Interaction Design for VR
 
COMP 4010: Lecture 5 - Interaction Design for Virtual Reality
COMP 4010: Lecture 5 - Interaction Design for Virtual RealityCOMP 4010: Lecture 5 - Interaction Design for Virtual Reality
COMP 4010: Lecture 5 - Interaction Design for Virtual Reality
 
COMP 4010 - Lecture 5: Interaction Design for Virtual Reality
COMP 4010 - Lecture 5: Interaction Design for Virtual RealityCOMP 4010 - Lecture 5: Interaction Design for Virtual Reality
COMP 4010 - Lecture 5: Interaction Design for Virtual Reality
 
Storytelling
StorytellingStorytelling
Storytelling
 
Applying virtual environments in distance learning of product development
Applying virtual environments in distance learning of product developmentApplying virtual environments in distance learning of product development
Applying virtual environments in distance learning of product development
 
Tech Tools to Support Communication Competency
Tech Tools to Support Communication CompetencyTech Tools to Support Communication Competency
Tech Tools to Support Communication Competency
 
Coil2012 torcivia calixv4
Coil2012 torcivia calixv4Coil2012 torcivia calixv4
Coil2012 torcivia calixv4
 
Tell me what you want and I’ll show you what you can have: who drives design ...
Tell me what you want and I’ll show you what you can have: who drives design ...Tell me what you want and I’ll show you what you can have: who drives design ...
Tell me what you want and I’ll show you what you can have: who drives design ...
 
Digital toolbox for 21st Century Learning
Digital toolbox for 21st Century LearningDigital toolbox for 21st Century Learning
Digital toolbox for 21st Century Learning
 
Website Usability & User Experience: Veel bezoekers, weinig klanten?
Website Usability & User Experience: Veel bezoekers, weinig klanten?Website Usability & User Experience: Veel bezoekers, weinig klanten?
Website Usability & User Experience: Veel bezoekers, weinig klanten?
 
Designing and Evaluating Virtual Reality for Learning
Designing and Evaluating Virtual Reality for LearningDesigning and Evaluating Virtual Reality for Learning
Designing and Evaluating Virtual Reality for Learning
 
SL formal_education
SL formal_educationSL formal_education
SL formal_education
 
Ideas for using Web 2.0 tools in the classroom - updated September 2016
Ideas for using Web 2.0 tools in the classroom - updated September 2016Ideas for using Web 2.0 tools in the classroom - updated September 2016
Ideas for using Web 2.0 tools in the classroom - updated September 2016
 
3D Virtual Worlds for Professional Development and Lifelong Learning
3D Virtual Worlds for Professional Development and Lifelong Learning3D Virtual Worlds for Professional Development and Lifelong Learning
3D Virtual Worlds for Professional Development and Lifelong Learning
 
VIGC Academy
VIGC AcademyVIGC Academy
VIGC Academy
 
Using your projector in your classroom
Using your projector in your classroomUsing your projector in your classroom
Using your projector in your classroom
 

More from Mark Billinghurst

Future Research Directions for Augmented Reality
Future Research Directions for Augmented RealityFuture Research Directions for Augmented Reality
Future Research Directions for Augmented RealityMark Billinghurst
 
Evaluation Methods for Social XR Experiences
Evaluation Methods for Social XR ExperiencesEvaluation Methods for Social XR Experiences
Evaluation Methods for Social XR ExperiencesMark Billinghurst
 
Empathic Computing: Delivering the Potential of the Metaverse
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the MetaverseMark Billinghurst
 
Empathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the MetaverseEmpathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the MetaverseMark Billinghurst
 
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationTalk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationMark Billinghurst
 
Empathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader MetaverseEmpathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader MetaverseMark Billinghurst
 
Novel Interfaces for AR Systems
Novel Interfaces for AR SystemsNovel Interfaces for AR Systems
Novel Interfaces for AR SystemsMark Billinghurst
 
2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR PrototypingMark Billinghurst
 
2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR InteractionMark Billinghurst
 
2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR TechnologyMark Billinghurst
 
Empathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive AnalyticsEmpathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive AnalyticsMark Billinghurst
 
Empathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole MetaverseEmpathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole MetaverseMark Billinghurst
 
Research Directions in Transitional Interfaces
Research Directions in Transitional InterfacesResearch Directions in Transitional Interfaces
Research Directions in Transitional InterfacesMark Billinghurst
 
Comp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research DirectionsComp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research DirectionsMark Billinghurst
 
Comp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsComp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsMark Billinghurst
 
Comp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsComp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsMark Billinghurst
 
Grand Challenges for Mixed Reality
Grand Challenges for Mixed Reality Grand Challenges for Mixed Reality
Grand Challenges for Mixed Reality Mark Billinghurst
 
Comp4010 Lecture10 VR Interface Design
Comp4010 Lecture10 VR Interface DesignComp4010 Lecture10 VR Interface Design
Comp4010 Lecture10 VR Interface DesignMark Billinghurst
 
Comp4010 Lecture9 VR Input and Systems
Comp4010 Lecture9 VR Input and SystemsComp4010 Lecture9 VR Input and Systems
Comp4010 Lecture9 VR Input and SystemsMark Billinghurst
 

More from Mark Billinghurst (20)

Future Research Directions for Augmented Reality
Future Research Directions for Augmented RealityFuture Research Directions for Augmented Reality
Future Research Directions for Augmented Reality
 
Evaluation Methods for Social XR Experiences
Evaluation Methods for Social XR ExperiencesEvaluation Methods for Social XR Experiences
Evaluation Methods for Social XR Experiences
 
Empathic Computing: Delivering the Potential of the Metaverse
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the Metaverse
 
Empathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the MetaverseEmpathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the Metaverse
 
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationTalk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
 
Empathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader MetaverseEmpathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader Metaverse
 
Novel Interfaces for AR Systems
Novel Interfaces for AR SystemsNovel Interfaces for AR Systems
Novel Interfaces for AR Systems
 
2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping
 
2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction
 
2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology
 
Empathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive AnalyticsEmpathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive Analytics
 
Metaverse Learning
Metaverse LearningMetaverse Learning
Metaverse Learning
 
Empathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole MetaverseEmpathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole Metaverse
 
Research Directions in Transitional Interfaces
Research Directions in Transitional InterfacesResearch Directions in Transitional Interfaces
Research Directions in Transitional Interfaces
 
Comp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research DirectionsComp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research Directions
 
Comp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsComp4010 lecture11 VR Applications
Comp4010 lecture11 VR Applications
 
Comp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsComp4010 lecture11 VR Applications
Comp4010 lecture11 VR Applications
 
Grand Challenges for Mixed Reality
Grand Challenges for Mixed Reality Grand Challenges for Mixed Reality
Grand Challenges for Mixed Reality
 
Comp4010 Lecture10 VR Interface Design
Comp4010 Lecture10 VR Interface DesignComp4010 Lecture10 VR Interface Design
Comp4010 Lecture10 VR Interface Design
 
Comp4010 Lecture9 VR Input and Systems
Comp4010 Lecture9 VR Input and SystemsComp4010 Lecture9 VR Input and Systems
Comp4010 Lecture9 VR Input and Systems
 

Recently uploaded

Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialJoão Esperancinha
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructureitnewsafrica
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfROWELL MARQUINA
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 

Recently uploaded (20)

Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorial
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdf
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 

Lecture7 Example VR Applications

  • 1. LECTURE 7: EXAMPLE VR APPLICATIONS COMP 4026 – Advanced HCI Semester 5 - 2019 Bruce Thomas, Mark Billinghurst, Gun Lee University of South Australia September 10th 2019
  • 2. Lecture 6: Recap • Interaction Design for VR • Iterative method for designing VR experiences • Applying well known ID techniques to VR • Interaction Design Process • Needs analysis • Experience Design • System Prototyping • Evaluation
  • 3. The Interaction Design Process Evaluate (Re)Design Identify needs/ establish requirements Build an interactive version Final Product Develop alternative prototypes/concepts and compare them And iterate, iterate, iterate....
  • 4. Methods for Identifying User Needs Learn from people Learn from analogous settings Learn from Experts Immersive yourself in context
  • 5. VR Design Considerations • Use UI Best Practices • Adapt know UI guidelines to VR • Use of Interface Metaphors/Affordances • Decide best metaphor for VR application • Design for Humans • Use Human Information Processing model • Design for Different User Groups • Different users may have unique needs • Design for the Whole User • Social, cultural, emotional, physical cognitive
  • 6. Typical Development Steps ▪ Sketching ▪ Storyboards ▪ UI Mockups ▪ Interaction Flows ▪ Video Prototypes ▪ Interactive Prototypes ▪ Final Native Application Increased Fidelity & Interactivity
  • 7. VR Prototyping Tools • Low Fidelity • Sketched Paper Interfaces – pen/paper, non-interactive • Onride Photoshop tool – digital, non-interactive • InstaVR - 360 web based tool, simple interactivity • SketchBox – create VR interface inside VR • Microsoft Marquette – mock up VR interfaces in VR • High Fidelity • EditorVR – Unity wrapper inside VR • Unity/Unreal Game Engine – programming needed
  • 8. Four Evaluation Paradigms •‘quick and dirty’ •usability testing (lab studies) •field studies •predictive evaluation
  • 11. Virtual Reality Applications • Ideal applications for VR should: • Be strongly visual, have 3D spatial elements • Benefit from first person immersion • Benefit from 3D manipulation/navigation • Support Autonomy, Interaction and Presence (AIP Cube) • Etc..
  • 12. Not Everything Should be Done in VR Dr Fun - 1990
  • 13. Many Possible Types of VR Applications From https://www.slideshare.net/ampnewventures/virtual-reality-vr-continuum-amp-new-ventures
  • 14.
  • 15. Potential Disruption for Existing Domains https://www.slideshare.net/BDMIFund/the-emerging-virtual-reality-landscape-a-primer
  • 16. Example VR Applications • Education • Google Expeditions • Medicine • Virtual Characters • Entertainment • The Void, Zero Latency • Art + Design • Tilt Brush • Collaboration • Facebook Spaces
  • 18. Google Expeditions • https://edu.google.com/expeditions/ • Mobile VR Educational application (Android, iOS) • Designed for classroom experiences
  • 19. Google Expeditions • Goal: Provide low cost educational VR experience • Based on Google Cardboard VR platform • Different roles: • Guide— person leading an expedition on a tablet • Explorer— person following an expedition on a phone. • Usage • Used by millions of students • Over 1000 educational experiences developed • Royal Collection Trust, American Museum of Natural History, etc.
  • 20. Teacher Led VR Experiences • Teacher/Guide uses tablet to control the experience • Selects the virtual tour experience • Guide sees tour script, can select immersive scenes to view • Guide sees focus point and where individual students are looking • Students connect as followers, look at what guides highlight Guide Interface
  • 21. System • Hardware • Google Cardboard mobile viewer • Smart phones + tablet (class set) • Wireless router • Software • Viewer and Guide applications (iOS/Android) • 360 image/video VR experiences Class set for 30 students
  • 22. Example Experiences • Over 1000 locations/experiences • Great barrier reef, Great Wall of China, Grand Canyon, etc.
  • 24. Feedback • Teacher/student survey (100 people) • 65% experienced a “Wow” moment during Google expedition • Noted the variety of educator styles and approaches possible • People enjoyed “The feeling of ‘being’ there” From https://www.slideshare.net/zoesujon/google-expeditions-virtual-reality-and-the-classroom
  • 25. Limitations • But 53% of participants identified some problems: • Difficult for some people who wore glasses • Some complained of eye strain, headaches or nausea • Some staff were reluctant/resistant to use the leader tablet • Issues of disabilities and inclusion
  • 26. Key Findings • Low cost VR/mobile VR can provide a valuable educational experience • Visit different locations, different times, etc. • Teach interaction key • Acting as guide, providing educational context • VR requires more work • Address simulator sickness, ergonomic issues, etc. • Immersion/Presence creates learning • Immersion creates memorable educational experience
  • 27. Challenges/Solutions • Making VR accessible • Designing for phones, tablets, low cost viewers • Synchronizing content with all viewers • Teacher controlled viewing • Teacher can guide experiences • Engaging interaction on simple viewers • Head pointing based interaction, button input • Supporting Educational goals • Providing compelling educational content
  • 29. Virtual Patients • Problem • Many doctors have poor doctor/patient skills • Have limited opportunity during training to learn skills • Solution • Virtual patients that doctors can communicate with naturally • Artificial agents with speech understanding
  • 30. Typical System Setup • Trainee in front of projection screen • Speech and gesture recognition • Intelligent agent on screen Johnsen, K., Raij, A., Stevens, A., Lind, D. S., & Lok, B. (2007, April). The validity of a virtual human experience for interpersonal skills education. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 1049-1058). ACM.
  • 32. Key Findings • Virtual Humans can replace actors in training • interaction skills used with a virtual human translate to the interaction skills used with a real human • Students feel a strong sense of co-presence • Having character respond to speech and gesture increases immersion • VR is capable of creating realistic characters • Life size, intelligent backend, speech recognition • Skills learnt transfer to real world
  • 33. Challenges/Solutions • Training in medical environment • Design for training in medical exam room • Use projected VR not HMDs • Natural interaction • Support speech and gesture interaction • Tactile/haptic feedback • Use prosthetics to add support for palpation and other tactile interaction between doctor and virtual patient • Supporting Educational goals • Give virtual character domain knowledge
  • 35. Large Scale VR Gaming • Provide multi-player VR gaming in warehouse space • Examples • The Void - https://www.thevoid.com/ • Zero Latency - https://zerolatencyvr.com/
  • 36. Typical System • Wide Area Tracking • Computer vision, lights/reflective balls • > 120 cameras for 300 m2 space • Backpack VR system • Haptic feedback, wireless HMD • Real Props • Tracked objects, walls Tracking cameras Backpack system
  • 37. The Void Demo • https://www.youtube.com/watch?v=XgetffuOgBA
  • 38. Key Findings • Wide area tracking possible • vision based systems can create large scale wide areas tracking, fast enough for game play • Shared gameplay improves experience • Focus on collaborative experiences, using avatar representations and roll division • Haptic feedback significantly increases presence • Use of physical props (objects, walls) • Content is king • Systems need compelling content/game place
  • 39. Challenges/Solutions • Wide area tracking • Computer vision tracking of people • Over 100 cameras + multiple servers • Freedom of movement • Custom wireless VR backpacks • Ruggedized HMDs, weapon props • Natural interaction • Redirected walking, tangible props • Compelling content • Multi-sensory feedback, custom game platform
  • 41. Tilt Brush • Intuitive 3D immersive drawing/sculpting program • Developed by Patrick Hackett and Drew Skillman 2014 • Acquired by Google in 2015 • https://www.tiltbrush.com/
  • 42. Functionality • Goal: Extremely natural 3D painting/sculpting • User Interface • Two handed interface designed for two controllers (Vive, Rift) • Brush in dominant hand, tool palette in non-dominant • Typical drawing functionality – color, brush width, undo/redo, etc.. • Content sharing • Created content can be exported/shared in 2D/3D formats
  • 45. Example Tilt Brush Sketches • https://poly.google.com/ • Explore in desktop VR
  • 46. Key Findings • Use familiar tools • Tilt brush interface has familiar sculpting/painting tools – e.g. brush size, colour pallet, etc • Use intuitive interface • Two handed tools with natural metaphor – one hand for pallet/menu, one hand for painting/sculpting • Provide Magical experience • Provide experience not possible in real world, e.g. changing body scale, painting in 3D, etc. • Create a community • Provide ways for people to share content
  • 47. Challenges/Solutions • Intuitive Interface • Very natural metaphor – painting in space • Two handed interface – map to VR controllers • Familiar menu objects from paint programs • Need for limited training • Provide in app training, tool tips • Content sharing • Enable content to be exported in variety of formats • Video, animated GIFs, 2D images, 3D files • Engaging Experience • Provides novel immersive artistic experience
  • 49. Facebook Spaces • Collaborative VR environment • VR meeting and interaction space (up to 4 people) • Focus on communication • Speech and gesture based • https://www.facebook.com/spaces
  • 50. System Interaction • Designed for Oculus Rift/HTC Vive • Upper body tracking, touch controllers • Simple interaction • Loading scenes, direct object manipulation • Content creation • Selfie pictures, simple sketching
  • 52. Key Findings • Minimal social cues okay • Even simple avatars can provide rich social experience • Create shared social context • Important to place users in same shared Virtual Reality environment/shared social context • Audio is key • Provide low latency audio, spatial audio cues • Create a reason for communicating • Why should people want to connect? Create shared activity/reason for people to conference
  • 53. Challenges/Solutions • Create shared sense of Presence • Use common background, shared objects • Natural communication • Support non-verbal behaviour, speech/gesture input • Intuitive interaction • Map real body motion onto Avatars • Limited ability to navigate/move through environment • Engaging Experience • Shared content creation, experience capture
  • 54. Other Examples • Many other examples of collaborative VR • Rec Room, High Fidelity, AltspaceVR • Sansar, VR chat, etc..
  • 55. Example: High Fidelity Worlds • https://www.youtube.com/watch?v=-ivL1DDwUK4
  • 57. Collisions – Australian VR Film • http://www.collisionsvr.com/
  • 59. Best VR Apps of 2019 (Digital Trends) • ALLUMETTE – VR Stop motion film • Google Earth – Travel/geography • Kingspray Graffiti – Art/content creation • The FOO Show - VR Talk show • Virtual Desktop – Use desktop in VR • www.digitaltrends.com/virtual-reality/best-virtual-reality-apps/