UNIT III-UX-UI.pptx

KONGU ENGINEERIG COLLEGE (AUTONOMOUS)
DEPARTMENT OF COMPUTER SCIENCE AND DESIGN
20CDH01-HONOR DEGREE-IMMERSIVE DESIGN THEORY
UNIT III - UX/UI for Immersive Design
P.GOWSIKRAJA M.E., (Ph.D.,)
Assistant Professor
Department of Computer Science and Design

UX/UI for
Immersive
Design
The UX
of XR
The UI
of XR

The UX of XR:
3.1
Approachable design
★ H
★ H
★ H
Seamless user flow
● K
● K
● K
3.2
Know thy audience
● J
● J
● J
3.3
Making reality
accessible
● W
● W
● W
3.4
UX challenge
● R
● R
● R
3.5

The UI of XR
3.6 The z-axis
3.10 Inspiration is all around you.
3.7 3D interface metaphors
3.9 Microinteractions
3.8 Time and space
The UI
of XR

The UX of XR: Approachable design
To create an optimal user experience (UX) within VR, MR, and AR. We will
discuss:
APPROACHABLE DESIGN With new technology, many users may feel
uncomfortable or even less confident in their initial uses of the
technology.
It’s important to acknowledge and to design for this and then empower
users to try the experience.
SEAMLESS USER FLOW Users have goals, and to help them achieve
those goals you need to create a visual that maps out the full flow of the
interaction.

KNOW THY AUDIENCE To make the best design decisions, you need to
know who you are designing for.
This requires user research and data collection to make the most effective
design decisions for real users.
MAKING REALITY ACCESSIBLE While you can’t design a unique
experience to accommodate the needs of every user, you can allow for
things like personalization to enable users to benefit from the many ways
XR can make the world around us more accessible every day or even in
certain situations.

3.1 Approachable design
❏ Creating an XR experience is personal. Unlike playing a game on a
computer screen or a TV screen, when you use a HMD (head-mounted
display) the screens are attached physically to you and covering your
eyes.
❏ Having control of someone’s sight is extremely personal.
❏ In life, before you enter into someone’s personal space you need to
establish a certain amount of trust with them if you want to avoid a
strong negative reaction.
❏ It likely will take some time before they are willing to let you come
within close proximity to establish a physical connection.

3.1 Approachable design
The greeting
Positive user experiences:
1. Personal space
2. Agency
3. Social signifiers
4. Feedback
5. Affordances
6. Interactions
7. Safety
8. Guides
9. Cohesion
The greeting
Pandemic time How you
are greet your friends?

Positive user experiences
First impressions matter, and in XR they are essential to creating a positive
user experience (UX) for the user.
The UX is the design that happens behind all the visuals.
The role of the designer is to create an environment where the user
understands the space and is able to distinguish the physical and digital
worlds.
This goal can be achieved using both diegetic and non-diegetic elements.
Diegetic Non-diegetic
Diegetic The placement of an element to appear inside the
virtual environment.
Non-diegetic Overlaying an element on top of a virtual
environment, making it clear it is not part of the
environment itself.

INTERACTION More than just a user and an object, but rather the result
and impressions created by the user from interacting with that object, as a
result of how it is designed. These impressions are emotional.
CONTEXT The situation where the interaction occurs in collaboration with
the user and the object (or service) provides the context.
FUSION The user experience is multifaceted. It is constructed by a
combination of all experiences, both the single experiences and also the
combination of all those experiences together.

● The foundation for this is the consideration of how interaction, context,
and the fusion of experiences all work together to aid the user toward their
goal.
● Working in an immersive, 3D environment adds complexity to creating a
good user experience.
● Some additional elements are unique to this industry and should be
considered for an optimal user experience in XR, for example:

Personal space:
XR is personal. With devices covering our eyes and experiences happening within our homes,
technology is invited into our most personal spaces.
♣ As a designer, your role is to acknowledge the need for trust a user has to have to even
start an experience. Your users are trusting what they see and feel to you. As such, you
don’t want to do anything to break that trust.
♣ Before the user begins the experience, they should have an understanding of what they
are about to experience. This can be through an onboarding experience as we discussed, a
demo, or part of the media that promoted the experience in the first place.
♣ Even the inclusion of a static image of what the experience will look like or a concise
paragraph of text to set up the user’s expectations will improve the experience.

♣ Emotions of anxiety and fear will take over, and the user will likely remove
the headset and not want to try it again anytime soon.
♣ Being thoughtful of how the positioning of augmented objects can work to
further generate trust the user has with the experience, as long as you respect
their personal space.
♣ You can easily make a user uncomfortable with the proximity and motion of
elements around them. To recap, in the initial scene:
♠ Reduce cognitive load by limiting the number of elements
♠ Avoid extremes
♠ Don’t place content too close to user
♠ Avoid fast-moving (potentially alarming) elements

Agency: The ability for a user to control or change something in an
experience versus just observing.
This ability to choose their own story will have an effect on the full
experience.
Agency can be global, meaning the user can impact the entire experience, or
local, meaning the user can impact a specific scene or object.
Understanding this, it is important to realize that agency is about control,
which is really power.
The more agency you allow, the more power the user has within the
experience.

Social signifiers:- A visual cue that carries
meaning that is based on a social or cultural
agreement.
If someone has a red flag up on their mailbox,
what does that mean? What about lines
painted on a street?
Knowing that a dashed yellow line in the middle
of the road means it is safe to pass someone in
that area is a social understanding.
Social signifiers can communicate critical
information for the user to understand an
experience and can also influence their
behavior and actions.

Feedback:
In a conversation, it is important to know if the other person has heard you.
There are multiple ways to receive feedback from the other person to confirm
that the message you sent was received.
They might respond verbally or through the use of body language, such as a
nod of understanding. Without this response you would be left unsure
whether or not you were understood.
The same is true when a human interacts with a computer of any kind: They
are is in a conversation with the device or experience.
Every time the user communicates something, whether by voice or action, it
is important that they receive some type of response so they feel confident
that their communication was heard.

Affordances:
A property or properties of an object that inform a user how they can interact
with it to carry out an action. Within the first minute of an experience a user
should have a good understanding of:
Where they are and how they fit into that space
Why they are there (what is their goal)
How they can interact with the elements within the experience to move toward
achieving their goal (perhaps the most important to know)
Think about it, how do you know how to interact with an object? For example,
how do you know how to pick up a coffee mug? The shape of the handle is a
property of the object that directs you to notice it.

The beauty of using affordances within XR is that you can design interactions to
directly reflect the real physical world and what we already know and understand
about physics.
Interactions
Once you understand that you can interact with an object, how do you actually
interact with it? There are three types of interactions:
Direct interactions in which you physically touch the object or screen you
are using, such as using a touch screen
Indirect interactions in which you use a physical object to control a digital
one, such as typing on a keyboard or scrolling with a mouse or trackpad
Semi-direct actions, which are a mix between direct and indirect interactions
such as scaling an object by pinching directly on the object and then tapping a
button to save the size property

The GRASP Taxonomy of Human
Grasp Types. Hand grasps are
classified in columns according to
their assignment into the categories:
power, intermediate, and precision
grasp; and then the opposition type
and the VF assignment. The two
rows are based on the position of the
thumb—which can be in an
abducted (thumb flexed) or
adducted position.

It is important to consider what technology you are designing for to make
sure to create the best safe experience.
There are different considerations that need to be taken into account for
VR versus AR and MR. Based on the amount of transparency you have in
your screen and your experience, you need to embed different safety
precautions.
One great example of this is the Guardian boundary visualization
option on the Oculus. The first step is setting up your play space for your
experience. Then when you get close to or walk out of bounds the view
changes to a mixed reality experience switching to a camera view of the
space around you.

This step helps you make sure
the user can select the space
that is available—avoiding
furniture and walls for
example—before going into a
fully virtual experience.
In all XR experiences, even in
experiences where there is a
transparent view, you should
avoid creating a situation where
someone would have to walk
backwards.
This may be an obvious safety concern, but without being able to turn and see what is behind you,
or on the ground behind you, this could cause injury. It is also advisable to include some reminders
for users to continue to pay attention to their physical environment while using your application. It
is important that they continue to be aware of their surroundings.

Guides: When you are visiting a new place for the first time, a tour guide can
be helpful to help you find all the must-see places.
The same is true when you first explore an XR experience. Use visual cues
to help direct the user to discover the space and to show them where they
can explore all the must-see places and experiences.
Depending on the field of view, it may be hard to see an experience in full,
so directors that help guide the user to needed places are an essential visual
tool.
These can be anything that captures attention, such as arrows, lights,
colors, motion, sound, path, lines, or anything else you come up with.
There are many useful AR applications that use these guides as the main
function of the experience.

Navigational AR, for example, is all about
creating these visual directors to help find
something as you are driving, walking, or even
in a grocery store.
If you want to make sure that the user finds
something, guide them to it
While exploring deeper into a 3D space, users
can easily lose their way or be unsure what
they are looking at.
Guides can play an important role in keeping
the user oriented in these situations, too.
In addition, guides can also help the user
navigate the digital and physical worlds in
more effective ways.

Cohesion
During a new experience, we all receive a sense of comfort from feeling that we gain
more understanding with each thing we do.
Once you learn how to do something, think about how good it feels the next time you
get to use that new skill again.
As a designer, you need to take these feelings into account and consider building
cohesion and consistency within the experience.
This is not only important in the user interface, which will be discussed, but also in
the experience as a whole.
If you maintain a systematic approach to the design and the interactions within an
experience, users will grow increasingly confident with each minute they are
immersed.

● This includes the position of key menu items, but also how you use
color, shapes, sizes, type, style of imagery, motion, sound, and the rules
of spatial awareness, such as occlusion.
● In an augmented experience, having a consistent style for the objects
overlaid in a world is a helpful way for the user to easily find and explore
the information as they scan a space.
● To take that to the next step to help you start to broaden your
perspective, it helps if objects that look visually cohesive react and relate
to the real-world objects in a similar way.
● if some objects are able to occlude within the space (be hidden under or
behind physical objects), then it would make sense to have all objects
occlude.

● Unless, of course, you identify some that will act differently in a
systematic way to fulfill a goal within the experience.
● If you group similar things together and keep a consistent look and
feel to all the design elements, the user will have an easier time
connecting the parts experience to form a whole.
● With so much that can overwhelm a new user, it is important to make
the first impression comfortable and approachable.
● In the same way that it can be challenging for the user to experience
too much at one time, it is also hard to design for an entire experience
at once. So, next we are going to break it up step-by-step to make it
easier to design.

UNIT III
UX/UI for Immersive Design
The UX
of XR:
Approachable design
Seamless user flow
Know thy audience
Making reality accessible - UX challenge
The UI
of XR:
The z-axis
3D interface metaphors
Time and space
Microinteractions
Inspiration is all around you

Seamless user flow
♣ User flow
♣ Mental Model
♠ Introduce user agency
♠ Guide them to start interacting
♠ Identify any needed transitions between
spaces
♠ Allow for mistakes
♠ Allow for a refresh
♠ Provide an exit
♠ Provide a clear way to save the experience

⮚ The XR experience is a story. If you approach this part of the process
as you would tell a story, then it becomes easier and logical to break
out each step of the process into a sequence.
⮚ There is a beginning, a middle, and an end.
⮚ There will be conflict points, characters, plots, and scenes. As the
designer, you get to create this story.
⮚ To get started, draw the story out as a chart that shows the
movements of the user and how they will engage with your
experience, step-by-step.

♣ This step-by-step process is important as it breaks this large idea
and concept into smaller, more manageable pieces.
♣ if you map out each step, you can just focus on it step-by-step, and
then you can zoom out to a 100-foot view and evaluate it from a
broader perspective. This chart is called the user flow.
♣ User flow A diagram of the step-by-step journey a user takes through
an experience, evaluating and highlighting the display, decisions, and
actions that are possible along the way.

♣ it does become more complex when you add the addition of space
and dimension to the experience.
♣ User Flow Design. User flow sketch for “SimplyStudy,” an AR
study experience focusing on the use of spatial UI.
♣ Designed for UI Prototype Project.
♣ Starts with the entry point and breaks down interactions using
shapes to represent the different parts of the flow.
♣ Displays are shown within rectangles, decisions within
diamonds, and actions within circles

♣ The first 60 seconds of an experience are critical to the retention of the
user. As you start your user flow, consider the entry point where the
user will enter the experience.
♣ What options will be immediately available to the user? As mentioned
earlier, you don’t want to overwhelm the user, so instead of providing a
large number of options and customizations, the first priority should be
to help them orient into the environment.
♣ A user needs to develop a mental model of the experience to
understand the space and how they relate to it.

♣ Mental model A map formulated internally to help a person
understand the environment they are in and the various relationships
between space and any objects, including themselves.
♣ This can be more complex in VR, as the user will be leaving one
environment and entering a completely different one.
♣ It is important at the start of the scene to allow the user to have a wide
angle on the space so they can process the space.
♣ Understanding space is absolutely essential in fully immersive setups;
however, some orientation will be needed for MR and AR experiences,
as well.

♣ MR and AR experiences may require the user to align their digital
objects and scenes with their physical ones.
♣ This often can require the user to select a vertical or horizontal plane to
start their experience or to move around their space to map it out.
♣ For someone who has never had to connect their digital and physical
worlds, this initial setup will not be intuitive.
♣ Providing a visual tool tip or instruction before the user is asked to
perform the action will help the user more successfully set up the
experience.

♣ you can have an illustration of a
grid appearing on a surface and an
icon of a mobile device moving back
and forth supported with the text,
“Surface found. Tap to create
surface anchor.”
♣ Screenshot from Adobe Aero
showing the process of selecting a
surface within your physical space
to anchor your 3D content.

INTRODUCE USER AGENCY. What can the user control or customize?
How much power should they have based on the goal?
Understand that your role as the designer is to still maintain comfort for the
user.
This requires balancing the amount of power that you give with retaining
control of the base line of the experience.
GUIDE THEM TO START INTERACTING. The first step is sometimes the
hardest one, so anything you can do to get users over this initial road block
is a good idea.
IDENTIFY ANY NEEDED TRANSITIONS BETWEEN SPACES. Where are
the main scenes, and how does the user get from one to the next?

ALLOW FOR MISTAKES. How can a user get out of each area and each
action? Let them feel that they can perform actions without consequences.
Build in an edit, undo.
ALLOW FOR A REFRESH. With new technology, especially using complex
3D imagery that can have large file sizes, there is a chance that the
experience will glitch or freeze. Providing an option to refresh or reload
more complex parts of the experience will allow the user to have the ability
to try to resolve individual issues that may be a result of their device.

PROVIDE AN EXIT. This should be easy to find and easy to perform by the
user.
❖ Little effort should be needed. This will help them feel comfortable and
will instill trust.
❖ Each time we enter a new space, the brain tries to understand the full
space, including how to get out of the space, just as a backup.
❖ If your users do not find an exit or a way out, you will create unneeded
anxiety for them. Always build in a way out.
PROVIDE A CLEAR WAY TO SAVE THE EXPERIENCE. The user must be
able to save and resume again later, or even launch it where they left off in
an alternate location.

✔ Knowing that this safety net exists, users will be more willing to try the
experience in different physical locations or to come back to play a game
from the beginning.
✔ Think about how helpful it is when you can pause a video from playing
and can return to it later knowing you can resume from where you left
off.
✔ This is the same idea in XR. Let the user pause and resume later.
✔ As you consider these points and walk through the full experience
creating your user flow, understand that it should be a working
document.

❑ As you work though an idea, it is helpful to then go back over the full
experience to see how that idea might shift the overall flow.
❑ For example, if you realize that the user needs to be able to search a
specific thing at some point in the experience, it might be worthwhile to
add a global search feature that users can access at any point during the
experience. Make note of the global actions that need to be available at all
times, as that will make it easier as you design and develop your
navigation items.
❑ Once you have a solid user flow that addresses all the main functions
and affordances as planned, double-check to make sure that you can
answer the following questions:

❖ What is the goal the user is trying to accomplish? (How do they
learn about this goal?)
❖ What information is needed for the user to accomplish this goal?
(Where is the information presented to them?)
❖ What pain points, or challenges, can prevent the user from achieving
this goal? (What can help them overcome these challenges?)
❖ You and your team can answer these questions, but to get the best
insight and feedback it’s wise to ask your users directly about their needs
or observe their behaviors as they interact within the experience.

User research and usability testing is critical to identify the needs of your
users.
It should be part of your design process consistently, starting with the user
flow, all the way through the launch, updates, and into the full life of the
product.
This next step is where the user journey truly begins

Seamless user flow
♣ User flow
♣ Mental Model
♠ Introduce user agency
♠ Guide them to start interacting
♠ Identify any needed transitions between spaces
♠ Allow for mistakes
♠ Allow for a refresh
♠ Provide an exit
♠ Provide a clear way to save the experience

UNIT III- UX/UI for Immersive Design
The
UX of
XR:
Approachable design
Seamless user flow
Know thy audience
Making reality accessible - UX challenge
The UI
of XR:
The z-axis
Time and space
Microinteractions
Inspiration is all around you

♣ Develop user personas
○ Augmented Learning
○ user personas
○ User Persona Template
♣ Walk through with persona
○ User stories
Know thy audience

Know thy audience:
Have you ever been walking along a
sidewalk or paved path and saw a
footpath shortcut or secondary path
veering another direction through the
grass?
● Did it draw you?
● These desire lines provide powerful
user feedback that tell city planners
and architects where people would
prefer to walk, as opposed to the
current path.
● Paying attention to these signals,
what users are requesting, provides

In XR, a main goal should be the comfort of your user. Comfort can take
many forms:
● Physical comfort
● Environmental comfort
● Physiological comfort
The best way to learn about your users is to talk to them.
Talk to people who have used XR technology, and talk to people who
haven’t.
Learn about their hesitations and any barriers.
Learn about what they enjoy most about it and what they hope to experience
with the technology in the future.

Develop user personas
● It is important to include that audience as part of the design process as
well. They can provide feedback and insights that only a user may
consider.
● To start, identify three to five diverse user personas that you can use to
test your user flow.
● To create these, it is best if you can use real data about actual people
that can be collected from interviews, surveys, or analytics.
● A combination of qualitative and quantitative research should be
completed and analyzed.

For each persona created, it is helpful
to identify their goal as a user, their
demographic information, and any
additional information that is relevant
within the context of the experience.
User Persona Template. Create a
template to learn more about the
humanity behind your users by
learning more about their personality
and motivations.

♣ If you could add into your everyday world one piece of information
that you could always have available,
♣ what would it be?
♣ A closed-captioning experience?
♣ A dictionary? A clock? A map? A voice assistant?
♣ benefits of augmented and mixed realities is that they open a world
of possibilities of supplemental information that can be provided
where you need it—in context.
Know thy audience

Know thy audience
♣ The power of this is really a game changer.
♣ In a world where we rely on a variety of devices, screens, and speakers
for an array of different activities, AR provides an opportunity to align
these into one space: the real world.
♣ I have been part of too many conversations and projects where these
considerations are an afterthought, where Band-Aids were then applied
to try to fix issues, instead of accessibility being an intentional aspect of
how the experience works

Walk through with personas
Once you create some user personas, you can take a more specific user
journey, walking through your user flow with the goals and needs of
one of those personas in mind.
Imagine each user persona as they walk through each step of the process,
and create user stories for them. At the outset these stories must:
● Identify the type of user
● Identify what their goal is
● Identify the motivations driving their journey

User Stories. Comparison of two
main kinds of users of the website
for the organization HUGs from
Cara (Help Understanding Grief),
the Griever and the Hugger.
The organization has a mission to
provide a place for grieving children,
families, and friends to find comfort
and support.
User stories The narrative of goals,
needs, and motivations written from
the user’s perspective to evoke
empathy.

Making reality accessible
The overall accessibility benefits that XR
provides are:
● Focus
● Multiple modalities
● Personalization
● Context

Augmented Learning:
An augmented photo prototype
exploring the use of an augmented
closed captioning experience for
students.
This was created to assist students
who are hard of hearing, deaf, have a
specific learning disability, or are
learning in a school outside of their
native language.

● There are many types of challenges that a variety of users can face in
their own unique ways.
● They range in sensory, motor, and cognitive disabilities or
challenges. In her book Mismatch: How Inclusion Shapes Design, Kat
Holmes explained the importance of inclusive design and how designers
must first consider who we are excluding.
● If you are designing an experience that requires the use of both hands
as the only way to interact within an experience,
● for example: arm mobility: those with only one hand, those with
limited functions, or those with missing or broken digits.

● Holmes identifies a “spectrum of
permanent, temporary, and
situational mismatches that
people experience based on their
abilities and disabilities.”
● Someone may have permanent
blindness or low vision, or
temporarily have low vision because
of cataracts or situational factors,
such as the sun shining in their
eyes or bright light washing out a
display.

As you focus on your target audience, understand that each has a unique
range of abilities.
There is no normal, despite what the term disability may imply.
Instead, designers should look at this spectrum approach to create inclusive
experiences.
That empathetic experience changed my whole perspective of how the
accessibility of buildings can alter so many components of daily life.
Take time to see the world from other points of view or, even better, talk and
engage with people who have different challenges than your own.

Focus
● Could you imagine what it would be like to learn and create or perform
in the same space?
● Many areas of education, training, and medicine are early adaptors of
this concept where they can clearly see its benefits.
● With so many screens grabbing our attention it is hard to stay
focused on one area.
● This can be even harder for those with cognitive processing disorders
and learning disabilities.
● However, learning disabilities don’t end when we leave a classroom, as
they extend far into our everyday lives and careers.

● This can be even harder when there is a specific format that is required,
such as a date.
● Was it 8/14/20 or 08/04/20 or 08/14/2020?
● It can be frustrating to the user to not have this example to view at the
same time they are entering the information.
● This may save space, but it is not as accessible to the users.
Example: Frames Direct to produce compatible prescription inserts for
HMDs that are anti-reflective, are light managed, and even have eye-tracking
features.
varieties of vision needs—from lazy eye, astigmatism, cataracts, or
increased sensitivity to light to eye surgery complications to the low vision
that is increasingly common with age—there is not a one-size-fits-all design.

Multimodal
There is a difference between requiring two hands to engage with an
experience, which is likely to exclude, versus designing it to work in
alternative ways such as with two hands, one hand, or with speech.
Offering alternatives allows opportunity.
providing multiple kinds of information, such as an icon and type or audio
and a visual, expands the reach of who can access that information.

Personalization
You can’t design a different experience for the needs of every person.
However, you can allow opportunities for the user to have control of
their experience.
To do this effectively, as always, it is important to learn more about the
needs of the user.
Then you can offer what is most helpful versus allowing the user full agency
to control the entire experience; we have discussed how too much control
can create confusion.
Settings such as volume, brightness, type size, and speech control allow
personalization without changing the overall experience.

It is important that a user be able to
access the options without having to quit
or leave an experience.
Just as you always have access to a
volume and brightness control on a
personal computer,
it is the expectation that these options
will also be available in an XR
experience.

UX challenge
● As you learn more about your users and their needs, I challenge you to
look for who you might be excluding from your experience—from the
beginning of the design process.
● Find willing users who you feel could be most excluded from
experiences you are designing and learn from them, as they will have
the most valuable insight for how an experience can be improved to be
inclusive.
● Observe and talk with people who have a variety of comfort levels,
preferences, abilities, challenges, height, age, identities, socio-
economic backgrounds, and experience using XR technology.

The overall accessibility benefits that XR
provides are:
● Focus
● Multiple modalities
● Personalization
● Context
● The best way to create accessible and inclusive environments is
to always be willing to learn from a diverse group of users.

The Z Axis:
● UI Interface, 2D and 3D interface.
● six degrees of freedom: Heaving
○ Swaying
○ Surging
○ Pitch
○ Yaw
○ Roll
● screen space versus world space
● UI tips for XR:- Direct and Indirect Manipulation.

The UI of XR:The z-axis
● How to design a user interface (UI) for extended reality.
● Many of the foundational elements of usability for UI design
remain, but with immersive interactions there are some new
challenges and considerations that deserve extra attention.
● Working with depth in UI may be new to you.
The z-axis
● The mind is the interface between ourselves and the world.
● To design an interface, you first need to understand and
contemplate this relationship.
● Intentionally keeping this in mind is essential to keeping your
thought process human-centered.

A user interface (UI) is the connection point between two systems; that
connection can be between a mind and an environment or even a
human and a computer.
There are interfaces all around us; some we may be aware of and some
we may not.
Interfaces connect us to things like laptops, phones, memory cards,
vending machines, key fobs, ATMs, and even the chips on credit
cards.
Take a few minutes and count how many computers you can find in the
room where you are right now.
Don’t forget to check places like your bags, wallets, pockets, and walls.
As you find each computer, look at the interface or the place where you
interact with that computer.

This could be a simple button or a complex system made up of
buttons, screens, and plugs.
Focus on the interfaces that are 3D versus flat (as with a touch
screen).
Notice physical interface components like light switches, locks, or
remote controls.
To explore how to design 3D interfaces, it is important to be inspired by
the interfaces that are already around us.
For the moment, however, just pay special attention to all the different
kinds of interfaces in your space and how you interact with them every
day:

● How do you know how to interact with them?
● Which ones are the easiest use?
● Which ones are the most confusing?
There is a lot to learn from both positive and negative interactions as
you explore designing a successful UI.
2D interfaces are actually harder to design for, as many of them are
trying to replicate a 3D interaction in a reimagined flat format.
Designing with depth is actually a benefit of the XR medium.
Instead of trying to draw a folder that looks like a file folder—but is really
a flat icon—you can have a 3D model of a file folder.

You can even place the file folder object in the environment where people
might expect to see it, such a file cabinet.
This is a challenge that is harder to overcome in a flat desktop format.
However, this depth does add more complexity to the interface, which
raises some additional considerations.
As we start to talk about depth, we need to understand what is meant.
When working with a 2D object we deal with the x-axis (left and right)
and the y-axis (up and down).

With the addition of the z-axis we now have depth, or the ability to
move objects forward and back (close and far).
Knowing that a point of view may vary depending on the experience,
if you start with this approach, it will eliminate some of the complexities
that may arise based on the kind of interactions you are testing.
Consider how you interact with a vending machine, an ATM, or even a
household appliance as just a few examples to get you thinking.

With the x, y, and z axes we can now work
with the six degrees of freedom, or 6DoF.
This freedom allows the user to move within
the three dimensions as well as change the
orientation along each of those axes, which
accounts for the six degrees: up/down
(+y, –y), left/right (+x, –x),
forward/back (+z, –z), and
rotation around x, y, and z.

Six degrees of freedom Directions that an object or user is able to move
about freely within three-dimensional space, sometimes written as 6DoF.
The six degrees are heave, surge, sway, pitch, yaw, and roll.
As an object or user moves linearly along different the axes, these
movements are the degrees of translation.
● Heaving: Moving up and down along the y-axis
● Swaying: Moving left and right along the x-axis
● Surging: Moving forward and backward along the z-axis
In comparison, if you turn to change an orientation within the three
axes, the degrees of rotation.

● Pitch: Rotating between
the x- and y-axis, up and
down
● Yaw: Rotating between
the x- and z-axis, left and
right.
● Roll: Rotating between
the z- and y-axis, front to
back

The UI design should reflect data and other information discovered by
any UX research and journey mapping.
The UI consists of the visual elements that make interacting with an
application positive and even enjoyable.
It adds appeal. However, in XR you don’t want it to overpower the core
experience. Because AR and MR experiences often have a limited field of
view, you want to optimize the available space to show as much of the
physical world as possible.
This requires keeping the interface elements minimal to avoid
cluttering the view and overwhelming the viewer.

● Because the interactions you are designing for are dependent on
technological limitation and allowances, it is important to design
specifically for the system on which the experience will be used.
● At this point, we are not yet able to create a fully responsive or
adjustable system that will adapt based on the technology.
● So, you will need to design the UI for each different system that the
experience will be used for. (The exception is mobile devices:
● Although Android and iOS require different programming
languages and AR frameworks, and capabilities will vary, there are
enough similarities that you may not need to design a fully different
interface for each.)

● For HMDs, the field of view (FOV) varies with different models. For
example, the HTC Vive has a FOV of 110 degrees; by comparison the
Microsoft HoloLens 2 has a FOV of 52 degrees.
● This difference has a huge impact on how you can use the space. It
would be hard to implement the same design into both platforms.
● Designing for each environment will allow you to design for the
specific viewing space available.
● Even with a variation of 52–110 degrees, you can see how the design
will need to be adjusted to create the best user experience.

Screen space World space
The screen space is the 2D space
within the application that is at the
same position and orientation as the
display (the frame visible to the viewer).
This space is determined by the device
you use and will vary by product.
The world space is the augmented or
virtual space that shows the 3D
environment surrounding an object or
user. This includes objects that are out
of the screen space.
Screen space The 2D space defined by
the screen or viewable area. This is
reliant on the screen size, position, and
resolution of the device.
World space The 3D x, y, and z
coordinates of any object, defined by
the environment.

UI tips for XR
Keep these points in mind as you make design decisions.
● Create a stationary UI in screen space for main navigation items
that do not move regardless of what is happening elsewhere on the
screen. This will help users feel safe and secure and avoid feeling
like they are lost.
● For any items that are within the world space, place them in the
center third of the viewing area for optimal viewing.
● Allow direct manipulation for objects whenever possible, as opposed
to indirect.

● Place any UI elements that are
spatial at a viewing distance far
enough from the user to avoid eye
strain from vergence-
accommodation conflict, ideally
between 1.3–3 meters (4–9 feet).
● Direct and Indirect Manipulation. Direct manipulation allows the
user to interact with object itself, versus relying on selecting on an icon
or button.

● Use solid areas of bright color and pure white sparingly. These will
appear even brighter when wearing an HMD and can increase eye
fatigue.
● Avoid transitions between dark and bright scenes. Instead, you can
allow the user to select a day or night mode so the scene lighting
will be consistent, which will reduce the amount of adjustments for
the eyes.
● Let the user have the control to hide or show certain UI elements.
● Use minimal text, only what is needed and nothing more.

● Use real or perceived affordances to allow users to anticipate the
action of an icon. A hamburger icon, for example, is understood to
bring out a navigation panel.
● Avoid flashing or blinking elements.
● Make minimal use of small detailed textures and patterns. These
create unnecessary clutter and can cause vibrations and
unintentional motion.
● Have tools feel available to the user, without being distracting.

Metaphors
Interface Metaphors
How real should you go?
3D interaction techniques
NAVIGATION
SELECTION
MANIPULATION
SYSTEM CONTROL
Virtual reality
RAYCASTING
METAPHOR
VIRTUAL HAND
METAPHOR
Ultraleap Hand
Tracking
PORTAL METAPHOR
Augmented reality
TRIGGER SPACE

Metaphors:
♣ versatile and powerful visual communication tools. It is true in motion
design where literal interpretations are often seen as cliché and are less
successful as a result.
♣ Leaning into metaphoric concepts can often help elevate the meaning of the
message.
♣ Visual metaphors can be seen in many different areas of design.
♣ A metaphor is a way to refer an idea, object, or concept, using a parallel
concept as a symbol.
♣ This can help reveal similarities between the two or help create a
relationship between them.
♣ This same idea is true in UI design and can be seen in many forms in our
current computer systems.

♠ Interface metaphor A commonly understood method or language,
based on a cultural connection that informs a user how to interact with
the user interface.
♠ These icons represent objects in the physical world.
♠ These examples, while great, are 2D concepts that have been used
within our flat screens to replicate a 3D action.
♠ The exciting part of designing immersive interfaces is that you can be
inspired by a 3D action and then create a metaphor, also in 3D.
♠ XR currently lacks accepted user interface metaphors, meaning that
the language of how users interact within this space has not been
firmly established.

As you design your XR
experiences, you can be inspired
by your physical environment.
Some great possibilities to jump
start your thinking:
♣ Shelves
♣ Tabs
♣ Doors
♣ Windows
♣ Books
♣ Libraries
♣ Kitchens
♣ Workshops
♣ Bags or luggage
♣ Clocks
you can be inspired by the actions
you take and the way you interact
with objects and elements in your
physical space to then re-create
them in an immersive way. To get
you thinking, here are a few:
♣ Moving, Deleting, Viewing
♣ Searching, Organizing, Creating
♣ Throwing, Picking up, Opening
♣ Closing, Pinching, Tapping
♣ Turning
♣ Speaking
♣ Gesturing
♣ Eye gazing

There needs to be a balance between how real versus playful you
make these metaphoric interactions.
♣ There are different opinions on how close to real life you should make your
actions. Remember these are metaphors, not exact replications.
♣ As part of this balance, also understand that you need to consider that
users interacting in a virtual world may not be very precise with their
actions.
♣ Design for that. This power of a metaphor works so well, in fact, that it can
have an adverse reaction when an object doesn’t work the way the user
expects.
♣ If a user perceives a 3D object to be just like it is their physical world, they
will expect it to act the same in the digital world.
♣ If that that isn’t possible, then it will create frustration.
♣ When we throw, we usually choose our dominant hand, pull back, and then
release in an arc motion. Knowing this, a user would want to be able to:

1. Select their preferred hand.
2. Receive some type of feedback that they have the ball successfully in
their hand.
3. See that they can pull back and not lose the ball from their hand
4. See and/or feel that the ball has left their hand as they release.
5. Receive the visual feedback of seeing the ball flying through the air in
the direction they released it.
Creating interactive prototypes also allocates space for user feedback
early on. This enables you to make adjustments in the design phase
before any additional time and energy is spent bringing these ideas into
3D space, and before the programming stage.

3D interaction techniques:- Research about 3D interaction techniques
has been conducted for decades.
This provides beneficial guidance. Especially notable is the work of Mark
Mine, dating back to 1995, where research was conducted using an
immersive simulated program called ISSAC.
This early research led him to jobs at NASA, and even Disney as an
imagineer, and as the director of the Technology Innovation Group for the
Walt Disney Studios.
His early research surfaced four main user behaviors that need to be
included as part of a 3D interface.
NAVIGATION
SELECTION
MANIPULATION
SYSTEM CONTROL

NAVIGATION:- It includes understanding the location of the user within
the overall environment, as well as an understanding of the 3D space, and
how to wayfind through it.
Essentially, provide the user with a way to understand:
What is this space?
Where am I in relation to it?
How can I move through it?
SELECTION:- The user needs a way to select objects within the
environment. This can be through direct and in-direct interactions.

MANIPULATION:- Once an object is selected, the user needs a way to modify
it from its original state.
Example, they might rotate an object to view it in its entirety.
A modification could go as far as being able to change the properties of the
object: size, texture, transparency, or color.
SYSTEM CONTROL:- This includes the overall functions and options that an
application has.
This is the direct way a user can communicate with the application as a
whole.

Virtual reality:- Within VR, where we are not able to see anything
except what is displayed within the digital environment, the user
completely loses the ability to rely on seeing any kind of physical input
devices such as a keyboard or a mouse, unless they are replicated in the
digital space.
They are also unable to see their hands for any gestures or related
interactions.
Having the ability to use your hands, controllers, and any other gestures
becomes harder without having any visual feedback or spatial
relationships.
New paradigms are needed for this challenge. Here are some examples
that can be used:

RAYCASTING METAPHOR Creates a virtual
ray into the environment that is defined by the
position and orientation of the user’s hand.
This feature allows the user to select or
manipulate an object while receiving feedback
of how it relates to the environment.
Raycasting. A photo illustration demonstrating
what a raycasting interface looks like: creating a
light ray that is intended to help the user
properly select a digital object.

VIRTUAL HAND METAPHOR Employs a
digital representation of the user’s physical
hand within the virtual space to give them
clear context, in terms of the location of
their hand, to improve interactions.
These are usually reliant on the
user holding controllers or reliant on
outside-in tracking.
Ultraleap has proven itself as a
leader in the hand-tracking space allowing
for unique control using touchless hand
tracking within VR and in other immersive
and interactive spaces

Ultraleap Hand Tracking:-
Ultraleap’s world-leading hand tracking enables direct interaction with
digital content.
The technology captures the subtlety and complexity of natural hand
movements by tracking a user’s hand in 3D space.
Hand-tracking technology can be used as an alternative for a computer
mouse or cursor, but it can also be used to interact with any digital object,
hologram, or experience in 3D space.

PORTAL METAPHOR Provides a
visual way to transition between
spaces using rectilinear shapes
such as doors, windows, and
display screens.
Because users already
understand how doors serve as
transition points, users can
connect these visuals to the
action, making the transition
more natural.

Portal. Doors and windows, either digital or physical, can be used to help
signify to the user the change from one environment to another.
This AR portal brings the user from their living room into a digital forest
captured in 360-degree video.
Augmented reality
Within the AR space, users are able to see and use their hands, so there isn’t
as much need to create visuals to represent hands.
However, there are other challenges that need consideration. Users still need
to a way to interact with both physical and digital objects.
Interactions for mobile AR will be the most familiar, as they use the same
touch interfaces that people are used to.
The Apple Human Interface Guidelines offer some guidance on best practices
for these cases. Some highlights of these include:

TRIGGER SPACE With such a small screen on a mobile device, space is
limited, which can make it challenging for the user to be precise with
touch points.
If digital objects are far away, the touch points will be even smaller.
Apple suggests that you include the space around your object in its
trigger space. This way, if a gesture is detected near the object, it will
select the object itself.
SCALE Do not allow scaling by the user if the object needs to be
represented at scale. If the user is using the AR experience to see how a
new appliance or furniture will fit within their space, then you wouldn’t
want to allow it to change size.

GESTURES Choose gestures that will not be confused with other very similar
ones.
Example, pinching and rotating can be perceived as similar if they both
require two fingers.
Be sure to use gestures that offer enough difference in the motion to produce
more accurate results. Of course, it is best to confirm this during user testing.
INTERACTION Consider other interaction methods beyond gestures.
Drag and drop, tap, slide, and pinch and zoom, are all reliant on gestures.
You can expand these through the use of voice, tilting of the phone (like a
steering wheel mechanism), and proximity.
As the user approaches an object it could animate or move as an example.
You could also instigate some interactions at the beginning of a session, so
they automatically start.

As head-mounted displays are
expanded for AR, and advancements
continue to explore interaction in MR,
the need for more interface metaphors
will arise. Currently, the technology
has been creative in terms of adding
touchable trackpads along the sides of
glasses, using hand controllers to
allow for more precise selections, and
using a smartphone to assist as a
controller.

One advantage of using AR or MR is that you are not tethered to an
immobile computer system, as you often are in VR. This freedom allows
users to move around.
Take note of the unique potential within this space to push the
boundaries of what can be done in 3D interactions, because there are
fewer limits.
Imagine how interacting with the UI of a thermometer in a digital
environment could cause it to start snowing or melting objects all around
you. The possibilities are really limitless.

Metaphors
Interface Metaphors
3D interaction techniques
NAVIGATION
SELECTION
MANIPULATION
SYSTEM CONTROL
Virtual reality
RAYCASTING METAPHOR
VIRTUAL HAND METAPHOR
Ultraleap Hand Tracking
PORTAL METAPHOR
Augmented reality
TRIGGER SPACE
GESTURES
INTERACTION

TIME AND SPACE
● Diegetic
● Non-Diegetic
● Meta UI
● Spatial UI

Time and space
As we explore the best practices of creating a user interface for XR, it is
important to point out that the needs of the user will change throughout
an experience as they become more versed in using your application.
Some controls that are needed at the very start of an experience will not
be needed during the entire experience.
As people explore and move around their physical or digital environments,
they may have a need for different controls and the ability to interact with
different elements.
As you are designing the UI, have your user flow with you as a guide to
note the different needs of the user at each given point during the
experience.
You only want to have the needed UI elements available for the given
moment, based on the time and space of the user in the experience.

The world space is the augmented or virtual space that shows the 3D
environment.
This designing of space is really about looking at the relationship of the
elements and determining the layering of each object.
Within XR you have many options and relationships to consider.
If you overlay an element right on top of the world view, this is non-diegetic.
This could be a single element or a box with multiple elements; however, this
style is typically represented two dimensionally to reinforce that is a separate
from the physical space.
The opposite of that is diegetic, which as you would expect means that the
element is displayed directly within the world space.

Non-diegetic The technique of overlaying an element on top of a virtual
environment, making it clear it is not part of the environment itself.
Diegetic The technique of placing an element to appear inside the virtual
environment.
you can also create meta UIs, which are similar to non-diegetic elements,
but are often temporary or situational.
This could be a notification when the user adds something to a shopping
cart for example, or a visual notification that updates the user based on
an interaction.
These are often 2D to provide a distinction.
If the UI is displayed three dimensionally, it would be considered spatial
UI. This could be a path to show which way to go or something that
adapts to the physical depth and space of the environment. contrasts the
two UI approaches.

Spatial vs. Meta UI. Turn-by-turn navigation,
using augmented reality via a mobile device, to
help customers find all the items on their
shopping list.
The top view shows spatial UI with the design
adapting to the physical space.
The bottom half of the view is a meta UI window,
which shows the map view and shopping list on a
2D window.
Meta UI:- A temporary or situational 2D window
that appears on top of a virtual environment.
Spatial UI:- A temporary or situational 3D
element that appears on top of a virtual
environment.

MICROINTERACTIONS
Frame-by-Frame Breakdown.
TRIGGER
RULES
FEEDBACK
LOOPS AND MODES
3D INTERACTIONS

Interaction is power from seeing something in context to understand how
it works. Tool tips have become a helpful way for users who need
additional instruction to learn more about how to use a digital tool.
These started as just text boxes that appeared as you hovered
over a tool, but have since grown into short motion clips that demonstrate
how the tool works.
This is much more useful to a user as it is faster and can
communicate universally.
Seeing an animation before the user interacts with an element
can effectively communicate the gesture.
Seeing a subtle motion as you are about to select something offers
you feedback as to what object you are selecting before you commit.
Seeing an icon jumping or shaking alerts you to focus on it. These
detail-oriented animations are called microinteractions.

Frame-by-Frame Breakdown:- A subtle shape change of a circle cursor will
help the user understand that they need to scroll upwards, just as the motion
of growing taller shows.
Although the motion is small, the interaction it communicates is powerful.
Microinteraction:- The small and subtle details of an interaction with a
product or experience. As broken down by Dan Saffer in the book,
Microinteractions, there are four parts of this detail-focused interaction:
1. TRIGGER The start of the interaction. These can be started by the user,
manually, or they can be automatically initiated if a condition is met. If you
flip a light switch, this is the trigger to turn on a light.
2. RULES Guidelines on what can and should happen in a specific order.
When you turn a light on, it will stay on until it is turned off. This is the
rule.

3.FEEDBACK Communication to the user about what is happening.
When you see a light come on, this is the feedback that your action, or
trigger, of moving the light switch was successful, and now the light is on.
This feedback can, and should, be multimodal.
Feedback can be provided through visuals, sound, and touch, such as
haptic.
4. LOOPS AND MODES These determine how long the interaction will last
and are the final part of the microinteraction.
The light will stay on until it is turned off.
Turning the light off will close the interaction loop.
These can determine whether the action repeats or changes over time as
well.

1. Bells ringing to show a notification
2. Volume adjustment bars moving up and down
3. Sound icons with and without diagonal lines (sound on and off)
4. An icon scaling up and down
5. The annoyance of a sound or vibration eventually going off after you
hit snooze for your alarm

3D Microinteraction. Providing interaction tips in
context, and with appropriate motion, makes
directions more intuitive.
Using 3D microinteractions will limit the need for
written-out directions, making the interaction fully
visual.
Example the user can see that they can rotate the
present object.
With so much complexity in immersive environments
and experiences, anything you can do to improve the
design and to simplify and assist the user will help
make the overall experience better

Inspiration is all around you:
the mind is the interface between ourselves and the world.
Think about way we understand how to interact with the
world around us—that is the way we need to think when we
design elements that users can interact with in XR.
3D designs deserve 3D interfaces, or at least spatially aware
interfaces that acknowledge and respect the time and space
that the user is part of.
As you design the user interfaces for your XR experiences,
find inspiration in the relationships and interactions we
have in our physical world.

DESIGN CHALLENGE
ON/OFF SWITCH DESIGNS
Sketch 20 kinds of on/off switches.
Think of some switches you use daily, but also expand your thinking to
some 3D switches that you can dream up.
How can you turn something on and off?
Consider thinking through various indicators that might signify how a
user will know when it is switched on versus off.
These sketches can be rough, but you should try to show dimension
when it is part of your design concept.

DESIGN CHALLENGE
If you come up with 20 different switches relatively quickly, push
yourself further and create 10 more.
Chances are the last 10, when you push beyond your initial ideas, will be
some of the most interesting and creative ideas you have.
If you get stuck, one solution is to use a random word generator, such as
the site www.randomwordgenerator.com, to help you generate additional
context for your on/off switch.

DESIGN CHALLENGE
The word that was provided on my first click was network.
With that, you could sketch out a network on/off switch.
What would that look like?
How would it function? Sketch it out.
There is one. Now, 19 more.

UNIT III-UX-UI.pptx

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to UNIT III-UX-UI.pptx

Similar to UNIT III-UX-UI.pptx (20)

Recently uploaded

Recently uploaded (20)

UNIT III-UX-UI.pptx