This document proposes developing an adaptive virtual reality system called "meta-space" to promote well-being for astronauts and others in isolated environments. It would collect physiological and behavioral data to detect psychological states and adapt VR content accordingly, such as virtual escapes of Earth or interactive games. A proposed development plan includes exploring signals, combining them into an adaptive layer, generating the virtual world, and optimizing the headset through testing.

1. Meta-space
Adaptive Virtual Reality
to Promote Wellbeing
in Space
Authors
Alex Aroyo, John Muñoz, Michael Gaebler
Collaborators
Kostas Konstantinidis, Andres Käosaar,
David Keatley, Alan Pope

2. About US
John Muñoz
VR/Physiological Computing
Technologist @ UW
STRATEGY
PLAN
ABOUT US
Kostas Konstantinidis
Space Systems Engineer,
former ESA ACT, GR
Michael Gaebler
Cognitive and Neuroscientist @ Max
Planck Institute, GE
Andres Käosaar
Extreme Environments
Psychologist @ University of
Central Florida, US
David Keatley
Senior Lecturer and Chair of
Crime Science @ Murdoch
University, AU
Alan Pope
Distinguished NASA Research
Associate, US
Alex Aroyo
Human Robot Interaction
Researcher @ UW
idea

3. Extreme environment isolation
Confined spaces
Hazardous environment
Limited social interaction
Earth remoteness
● Stress
● Home-sickness
● Anxiety
● Depression
● Fatigue
● Irritability
● Emotional liability
● Loneliness
● concentration difficulties
● Sleeping problems
● Interpersonal conflicts
● Post-mission issues
● ...
STRATEGY
PLAN
ABOUT US
idea

4. Mental health continuum model
Focus
STRATEGY
PLAN
ABOUT US
idea

5. Create a tool
Escape the feeling of isolation
Promote interpersonal relationships
STRATEGY
PLAN
ABOUT US
idea
Result in positive wellbeing
Increased team cohesion
Astronauts, ICE people
People on Earth

6. STRATEGY
PLAN
ABOUT US
idea
Open Space Innovation Platform
meta-space
A collaborative virtual space where
crew members are transported to their
favorite Earth experiences and the
virtual content adapts to promote
positive wellbeing and team cohesion

7. SYSTEM COMPONENTS
COLLECT AND INTEGRATE
Multimodal data from:
i) psychological profiles,
ii) self-reported
questionnaires,
iii) physiological signals,
iv) behavioral.
PROCESS AND ANALYZE
An adaptive layer will
process the information
to detect specific
psychological states
and suggest VR content
VR ADAPTIVE CONTENT
VR experiences:
i) virtual escape 360º
video of Earth,
ii) interactive games for
entertainment or training.
STRATEGY
PLAN
ABOUT US
idea

8. SYSTEM ARCHITECTURE
STRATEGY
PLAN
ABOUT US
idea

9. HARDWARE AND ADAPTIVE LAYER
STRATEGY
PLAN
ABOUT US
idea
Hardware
VR Headset
Physiology
LooxidLabs
-EEG
-Eye Tracking
-Heart Rate
EmteqLabs
-fEMG
-Eye Tracking
-Heart Rate
Software (collect, analyze, translate)
Physiological computing tools

10. CONTENT DESIGN, USER CENTERED
STRATEGY
PLAN
ABOUT US
idea
Virtual escape
Meditation, 360 videos family/friends, adaptive
based on measured states
Interactive games
Exergaming, cognitive training, entertainment

11. Multiplayer experiences
● Shared virtual environments
● Cooperative gameplay
● Multiplayer is optional
● Interactions can be used to
infer/promote team
working.
STRATEGY
PLAN
ABOUT US
idea

12. plan
STRATEGY
PLAN
ABOUT US
idea

13. plan
STRATEGY
PLAN
ABOUT US
idea
● M0 - Previous work.
● M1-3: Signal exploration
● M2-6: Combining signals into an adaptive layer.
● M3-9: Generating the virtual world.
● M10-18: Headset optimization and demonstration.

14. plan
STRATEGY
PLAN
ABOUT US
idea
● M0 - Previous work.
● M1-3: Signal exploration
● M2-6: Combining signals into an adaptive layer.
● M3-9: Generating the virtual world.
● M10-18: Headset optimization and demonstration.

15. PREVIOUS WORK ON VR/PHYSIO/MULTIPLAYER
STRATEGY
PLAN
ABOUT US
idea

16. Social psychology of polar exploration
STRATEGY
PLAN
ABOUT US
idea ● Developing behavioral models of crew
members during past polar expeditions*
● Ability to detect patterns between certain
behaviors that could result in a negative
outcome like team breakdown.
* Käosaar, A., Keatley, D., Smith, N., Konstantinidis, K., von Looz, M., & Ambhorkar, P.

17. plan
STRATEGY
PLAN
ABOUT US
idea
● M0 - Previous work.
● M1-3: Signal exploration
● M2-6: Combining signals into an adaptive layer.
● M3-9: Generating the virtual world.
● M10-18: Headset optimization and demonstration.

18. Signal exploration
STRATEGY
PLAN
ABOUT US
idea
- User’s profiling: personality, group
dynamics, stress management…
- Self-reported questionnaires.
- In-game cognitive assessment.
Psychological
- EEG/fEMG
- Heart Rate
- Eye tracking
Physiological - Auditory
- In game actions (single
or multiplayer)
- Joystick control
feedback
- Time & duration
behavioral
Public
Datasets
Human participants

19. plan
STRATEGY
PLAN
ABOUT US
idea
● M0 - Previous work.
● M1-3: Signal exploration
● M2-6: Combining signals into an adaptive layer.
● M3-9: Generating the virtual world.
● M10-18: Headset optimization and demonstration.

20. PROCESS AND ANALYZE
STRATEGY
PLAN
ABOUT US
idea
PROCESS MODEL
● Model of personal/social psychology in extreme
environments
OUTPUTS
● Intervention needed alert
● Intervention type suggestion
● ...
(SOME) OPTIONS FOR IMPLEMENTATION
● Rules-based: too simplistic
● Neural network: black box, hard to implement
● Sequence analysis: a happy middle ground?

21. Driving under the influence of alcohol
STRATEGY
PLAN
ABOUT US
idea
Behavioural sequence
analysis
● What transitions occur at
levels above pure chance?

22. Behavioural sequence analysis
STRATEGY
PLAN
ABOUT US
idea
BUT
● Is this the best approach?
● Hybrid approach of ML + BSA?
● Compare results of individual models?
● Leave it for the discussion
● Robust for complex social interactions
● Temporal behavior
● Group dynamics
● Lightweight

23. plan
STRATEGY
PLAN
ABOUT US
idea
● M0 - Previous work.
● M1-3: Signal exploration
● M2-6: Combining signals into an adaptive layer.
● M3-9: Generating the virtual world.
● M10-18: Headset optimization and demonstration.

24. plan
STRATEGY
PLAN
ABOUT US
idea
● M3-9: Generating the virtual world (& tailoring the adaptive layer).
○ Ideally a co-design with:
■ Astronauts (active/retired)
■ Analogue Field Testing and Exploration experts
■ Human Research experts
■ Astronaut psychologists
○ And/or Earth population:
■ Students/adults
■ Older adults
■ Children
● M10-18: Headset optimization and demonstration.

25. plan
STRATEGY
PLAN
ABOUT US
idea
● M3-9: Generating the virtual world.
● M10-18: Headset testing, optimization, and demonstration.
○ Earth population: Students/adults, older adults, kids...
○ Ideally testing with:
■ ICE (isolated confined extreme environment)
■ Analog environments: antarctic bases, caves...
■ Space
○ Experiments, tuning up, setbacks..
Extra: Potential for individual enhancement
● Tailoring the virtual environment
● Deep psychological profile
● Physiological and auditory baseline

26. THANKS!
Do you have any questions?
Contact
john.munoz.hci@uwaterloo.ca
alexander.aroyo@uwaterloo.ca
CREDITS: This presentation template was created by Slidesgo, including icons by Flaticon, infographics & images by Freepik.
STRATEGY
PLAN
ABOUT US
idea