The document discusses intelligent user interfaces in the context of virtual, augmented, and mixed realities, emphasizing the distinctions between immersion and presence and guidelines for effective design. It explores the challenges of creating engaging VR experiences using AI and IoT, along with applications in various industries such as healthcare, education, and gaming. Key design considerations include focusing on user experience, minimizing uncanny valley effects, and optimizing for interactivity and contextual relevance.

1. Intelligent User
Interfaces
ICS2208
Dr Vanessa Camilleri
Department of AI,
University of Malta

2. Topic 7: Overview
• VR, AR & MR
• AI & VR
• Design guidelines for Applications
• Perceptual Models & Modalities
• IoT design for Mixed Realities
2

3. Immersion, Presence &
Reality Trade-offs
• Immersion is the objective degree to which a VR
system and application projects stimuli onto the
sensory receptors of users in a way that is
extensive, matching, surrounding, vivid, interactive
and plot informing.

4. Immersion, Presence &
Reality Trade-offs
• Presence is a sense of being there inside a space,
even when physically located in a different location.
• Whereas immersion is about the characteristics of
a technology, presence is about the internal
psychological and physiological state of the user.
• Immersion is capable of producing a sense of
presence but immersion does not always induce
presence.

5. Illusions of Presence
• Illusion of being in a stable spatial place;
• Illusion of self-embodiment;
• Illusion of physical interaction;
• Illusion of social communication

6. Reality Trade-offs
• When does reality in VR become too much?
• When do the characters, if they do, become too
creepy with their realism?
This is called the uncanny valley – one has to
attribute power to simplicity.

8. Design Guidelines
How to design for VR Communication:
• Focus on the user experience rather than the technology;
• Simplify and harmonise the communication between user
and technology
• Focus on making the technical intermediary between the
user and content transparent so users feel they have direct
access to the virtual world and its entities.
• Design for visceral communication to induce presence
and inspire awe in users

9. Design Guidelines
How to design for various Realities:
• Choose what form of reality you wish to create. Where
does it fall on the virtuality continuum?
• Choose what type of input and output hardware to use
• VR is more than just the hardware and technology.
Create a strong conceptual story, an interesting
design or layout of the environment and engaging
characters.

10. Design Guidelines
How to design for Immersion & Presence:
• Minimise break-in presence
• For maximum presence, focus first on world stability and
depth cues. Then consider adding physical user
interactions, cues of one’s own body and social
communication
• Avoid the uncanny valley by not trying to make characters
appear too close to the way real humans look
• Choose the level of fidelity you want to create.

11. Perceptual Models &
Processes

12. • VR creators should use signifying cues, feedback
and constraints to help the user form quality mental
models and to make assumptions explicit.
• Learned helplessness is the decision that
something cannot be done due to a perceived
absence of control.
• It has been difficult for VR to gain wide adoption
most often due to poor interfaces. Bad interaction
design for VR can lead to learned helplessness.

13. IoT & Mixed Reality
• IoT design to connect people to smart devices
• Computer Interfaces changing to more immersive
and engaging
• AI & ML used to create engaging experiences
• BUT… deluge of information demand much attention
• SO… focus on displays that support changes in user
situational context and minimise attention costs

14. • Challenge: how to evoke a consistent user
experience across multiple devices
• Challenge: moving from a traditional dashboard to
a more immersive one with a contextually relevant
interface
• Challenge: learning from user behaviour
• Challenge: as personal spaces become more
embedded with technology, users need to visualise
the overall space

15. IoT & Mixed Reality
• Virtual Perspective
• Ambient Perspective
• Informational Context Perspective
• Practical applications: Debug & repair IoT devices,
smart city scenario, Games (which feature real time
data)

16. AI & VR
• Enhancing realism
• Processing large amounts of data related to
physics, lighting and user behaviour
• AI-driven hand tracking and gesture recognition
• Predictive modelling capabilities to reduce
motion sickness
• Dynamic user adaptation

17. AI & VR - which Industries?
• Applications across Industries:
• Healthcare
• Education
• Gaming
• Architecture and Design
• Future Trends

18. Applications of AI in VR
• Game development
• Generative AI in VR
• Realistic Environments
• Interactive and Dynamic Experiences
• Simulating Real World Scenarios
• Infinite Creativity and Exploration
• AI-Generated Characters and Avatars

19. Online Writing Task
Write about AI & VR, investigating how AI can enhance VR experiences and how VR can
provide valuable training data for AI algorithms. Cite at least 3 references from academic
sources. Choose one of these questions and answer in about 400 words:
1. AI-driven Personalisation in VR: Imagine a VR experience tailored to an individual's
preferences and emotional state. Describe a specific VR scenario (e.g., historical simulation,
phobia therapy, language learning) where AI personalises the experience in real-time. Explain
how AI could achieve this level of personalisation using user data, emotion recognition, and
adaptive algorithms.
2. VR for AI Training: Discuss the challenges of training complex AI algorithms, particularly in
tasks requiring real-world experience. Explain how VR simulations can be leveraged to create
safe, controlled environments for AI training. Provide examples of VR applications that could be
used to train AI for tasks like:
•Robot navigation in dynamic environments
•Autonomous vehicle decision-making in various road scenarios
•Social AI agents for improved human-computer interaction