Downloaded 110 times
Uploaded byMark Billinghurst
ISS2022 Keynote
The document discusses the growth and potential of the metaverse, highlighting its definition, key components, and research opportunities. It emphasizes how technological advancements are transforming virtual and augmented realities, leading to new collaborative experiences and interactions. The author calls for a broader understanding of the metaverse, focusing on empathic computing and its implications for future research.
More Related Content
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
ISS2022 Keynote
- 1. METAVERSE Mark Billinghurst mark.billinghurst@auckland.ac.nz November 22nd2022 Opportunities for Research Schmetaverse!!
- 2. The Metaverse isHot!
- 3. Google Searches onthe Metaverse • 5 year search volume to October 2022 • Metaverse (blue) vs. Virtual Reality (red)
- 4. Publications about theMetaverse • Going vertical !
- 6. A Definition • Real-time,3D, Interactive, Social, Persistent AWE 2022 John Riccitiello CEO, Unity Technologies
- 7.
- 8.
- 9. Back to 1994:Young Me in Seattle
- 10. Virtual Reality wasHOT! .. In 1995..
- 11. My First VRExperience 1500 Polygons! $250,000
- 12. My First VRExperience … https://www.youtube.com/watch?v=pAC5SeNH8jw
- 13. In Five toTen years .. • VR becoming commonplace • VR devices less than $1000 • Large scale networked worlds • Billion-dollar VR industry
- 15. • April 2007Computer World • VRVoted 7th on list of 21 biggest technology flops
- 16. What Happened AfterThat • Growth of internet • Natural user interfaces • Increase in graphics performance • Explosion in mobile phones • Pokemon Go!
- 17. Increase in VRResearch • Papers grew over fivefold..
- 19. Social VR Today RecRoom - 30 million users Fortnite - 24 million DAU - 28 million @ travis scott
- 21. A Better Definition •Neal Stephenson’s “Snow Crash” (1992) • The Metaverse is the convergence of: • 1) virtually enhanced physical reality • 2) physically persistent virtual space A persistent virtual environment around The Street 15 million avatars on The Street at any one time
- 22. Metaverse Taxonomy • FourKey Components • Virtual Worlds • Augmented Reality • Mirror Worlds • Lifelogging • Metaverse Roadmap • http://metaverseroadmap.org/
- 23. Mirror Worlds • Simulationsof external space/content • Capturing and sharing surroundings • Photorealistic content • Digital twins Matterport Deep Mirror Google Street View Soul Machines
- 24. Lifelogging • Measuring user’sinternal state • Capturing physiological cues • Recording everyday life • Augmenting humans Apple Fitbit Shimmer OpenBCI
- 25.
- 26.
- 27.
- 28. Possible Research Directions •Lifelogging to VR • Bringing real world actions into VR, VR to experience lifelogging data • AR to Lifelogging • Using AR to view lifelogging data in everyday life, Sharing physiological data • Mirror Worlds to VR • VR copy of the real world, Mirroring real world collaboration in VR • AR to Mirror Worlds • Visualizing the past in place, Asymmetric collaboration • And more..
- 29.
- 30. Communication Trends • 1.Experience Capture • Move from sharing faces to sharing places • 2. Natural Collaboration • Faster networks support more natural collaboration • 3. Implicit Understanding • Systems that recognize behaviour and emotion
- 31.
- 32.
- 33. Empathic Computing Vision Canwe develop systems that allow us to share what we are seeing, hearing and feeling with others?
- 34. Example: Sharing CommunicationCues • Measuring non-verbal cues • Gaze, face expression, heart rate • Sharing in Augmented Reality • Collaborative AR experiences
- 35. Empathy Glasses • Combinetogether eye-tracking, display, face expression • Implicit cues – eye gaze, face expression + + Pupil Labs Epson BT-200 AffectiveWear Masai, K., Sugimoto, M., Kunze, K., & Billinghurst, M. (2016, May). Empathy Glasses. In Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM.
- 36. Remote Collaboration • Eyegaze pointer and remote pointing • Face expression display • Implicit cues for remote collaboration
- 37.
- 38. Example: AR/VR Collaboration •Augmented Reality • Bringing remote people into your real space • Virtual Reality • Bringing elements of the real world into VR • AR/VR for sharing communication cues • Sharing non-verbal communication cues
- 39. Sharing Virtual CommunicationCues • AR/VR displays • Gesture input (Leap Motion) • Room scale tracking • Conditions • Baseline, FoV, Head-gaze, Eye-gaze
- 40.
- 41. Technology Trends • Advanceddisplays • Wide FOV, high resolution • Real time space capture • 3D scanning, stitching, segmentation • Natural gesture interaction • Hand tracking, pose recognition • Robust eye-tracking • Gaze points, focus depth • Emotion sensing/sharing • Physiological sensing, emotion mapping
- 42. Sensor Enhanced HMDs Eyetracking, heart rate, pupillometry, and face camera HP Omnicept Project Galea EEG, EMG, EDA, PPG, EOG, eye gaze, etc.
- 43. • Advanced displays •Real time space capture • Natural gesture interaction • Robust eye-tracking • Emotion sensing/sharing Empathic Tele-Existence
- 44. Empathic Tele-Existence • Basedon Empathic Computing • Creating shared understanding • Covering the entire Metaverse • AR, VR, Lifelogging, Mirror Worlds • Transforming collaboration • Observer to participant • Feeling of doing things together • Supporting Implicit collaboration
- 45. NeuralDrum • Using brainsynchronicity to increase connection • Collaborative VR drumming experience • Measure brain activity using 3 EEG electrodes • Use PLV to calculate synchronization • More synchronization increases graphics effects/immersion Pai, Y. S., Hajika, R., Gupta, K., Sasikumar, P., & Billinghurst, M. (2020). NeuralDrum: Perceiving Brain Synchronicity in XR Drumming. In SIGGRAPH Asia 2020 Technical Communications (pp. 1-4).
- 46. Set Up • HTCVive HMD • OpenBCI • 3 EEG electrodes
- 47.
- 48. Results "It’s quite interesting,I actually felt like my body was exchanged with my partner." Poor Player Good Player
- 49. Conclusions • The Metaverseis overhyped • Long-term impact could be more significant than imagined • We need a broader Metaverse definition and taxonomy • Many research opportunities in crossing boundaries • Need to focus on high-level visions • Empathic Computing encompasses the Metaverse
- 50. Delivering the EntireMetaverse
- 51.