UNIT-II Immersive Design for 3D

KONGU ENGINEERIG COLLEGE (AUTONOMOUS)
DEPARTMENT OF COMPUTER SCIENCE AND DESIGN
20CDH01-HONOR DEGREE-IMMERSIVE DESIGN THEORY
UNIT II IMMERSIVE DESIGN FOR 3D
P.GOWSIKRAJA M.E., (Ph.D.,)
Assistant Professor
Department of Computer Science and Design

I. Introduction to Extended Reality
II. Immersive Design for 3D
III. UX/UI for Immersive Design
IV. Human Factors and Background of Immersive Design
V. Extended Reality(XR) Development

1) The Immersive Experience: The world in 3D
2) Affordances - Multimodal experiences
3) Experience design
4) Ideation: Identify the why
5) Innovation and practicality - Outside influence
6) Keep it human - The answer to your questions
7) Creating The Prototype: Fake it 'til you make it
8) Sketch transparently - The power of prototypes
9) Use what you know - Process work.

2.1 The Immersive Experience: The world in 3D
The Immersive Experience is that how to make 3D shapes, shapes user
friendly and easy to interact with full sensory system.
THE WORLD IN 3D The complex down into smaller pieces and looking at
additive and subtractive methods for creating 3D models.
AFFORDANCES How we interact with physical objects in our real world
can and should inspire how we interact with digital objects in a virtual
world.
MULTIMODAL EXPERIENCES With so many different ways for a user to
experience XR, it makes sense to engage the user in as many modes as
they are used to in their physical world.

2.1 The Immersive Experience: The world in 3D
The world in 3D:- To have a deeper understanding of 3D than you do of 2D.
● Most designers have to train themselves in how to make a flat icon, how to take
something that is complex and simplify it to simple geometric shapes.
● Our understanding of 3D space through experience will allow us to know if we
are designing for it correctly.
● To start, shapes remain the essence of objects, and in 3D we work with the
same shapes you are used to, but they become volumetric.
○ A square becomes a cube.
○ A circle becomes a sphere.
○ A line becomes a cylinder.

Primitives Three-
dimensional geometric
shapes that can be
added, subtracted, and
combined to make more
complex shapes.
The use of additive and
subtractive techniques
can help you make more
complex shapes more
easily.

The table was created out of cubes, and only cubes.
How many cubes do you think it used? 5? 6? 280?

Coordinate axes
● Primitive shapes are defined using a three-dimensional coordinate
system that represents width, height, and depth.
● Locations in the system are represented through x, y, and z
coordinates, both positive and negative.
● The x-axis is horizontal with coordinate values increasing from left
to right.
● The y-axis is vertical with coordinates moving up (increasing) and
down (decreasing).
● Two-dimensional images include those two axes only, but in 3D the
addition of the z-axis adds coordinates indicating depth: forward and
back. This also includes the use of volume as it extrudes a shape in
all directions, including z-space. All together the three axes are
called the coordinate axes.

Coordinate axes The three number lines, all crossing at a central origin,
used to reference the position of every point in a 3D space.
♣ The x-axis, y-axis, and z-axis define the three coordinates, x, y, and z,
used to identify the position of a point.
♣ To fill and stroke of a shape, the attributes expand to designing an entire
scene.
♣ This process has similarities to photography in many ways.
♣ You need a scene, you need a camera that you can move around to
capture the shot, you need light, you need to consider the overall
composition.

♣ After capturing a photograph, you need to process it—often by moving the image file from
the memory card inside the camera to your computer.
♣ The equivalent in 3D is the rendering process, which creates the final file for use in an
illustration, animation, or an XR experience.
♣ Because some of the terms in this process might be new, let’s look at each of them one by
one.
♠ Splines
♠ Mesh
♠ Material
♠ Camera
♠ Light
♠ Scene
♠ Render
♠ 3D files

♠ Splines:
Spline A 3D curve made up of at least two points. With these you are able to
create the skeleton structure of a form using any combination of curved and
straight shapes.
Extrusion A method of expanding a 2D path or spline into 3D space by adding
vertices and edges along the z-axis.
♥ to create organic shapes and then make them volumetric. Most 3D programs
create curves using splines.
♥ If you have worked in Adobe After Effects, these are the equivalent of paths.
The process starts with creating the spline and adding extrusion to generate
geometry. This results in custom 3D shapes.
♥ some 3D programs, such as Cinema 4D (Maxon) allow you to import paths
directly from Adobe Illustrator.

Mesh
Working with 3D shapes might be a trip back to your elementary geometry
classes.
3D shapes are made up of points, edges, and faces. It is the relationship of
all those elements collectively that make a mesh. This mesh is the visible
representation of the 3D form. For example, you can create a 3D mesh of a
cube or a sphere.
Mesh Collection of points, edges, and faces that make up an object.
The relationship between the elements of the mesh help define the shape and
the overall form. The point where two or more edges meet is called a vertex.

Vertex Point where two or more line segments meet to
create, in essence, a corner.
When multiple points and edges close a complete shape,
they create a polygon. Triangles, squares, rectangles, and
pentagons are all examples of polygons. Multiple
polygons help form the full, complete 3D shape.

Material
• Once you have a mesh created, it is like gray shapes, looking like clay.
• It will not look too exciting yet—not until you start to create more physical qualities
to it. The first place to start is with materials.
• These are all the physical properties that you add over the full mesh. If you think of
your mesh as the structure, or the body, then the materials are like the clothes or
upholstery(place) that you drape over it.
• To take on the same shape as the structure.
• Materials are essentially the properties that you can add on top.
• These digital materials determine:
how light interacts with the different surfaces,
how the color looks, and what the texture looks like.
• Materials can be transparent, opaque, or reflective, for example.

Material Physical properties added to an object to
determine how it will appear in an environment.
♣ Materials that include a texture are a bit
different from your standard material, because
they use an image wrapped around the skin of a
3D object.
♣ You can apply and layer multiple materials such
as wood, brick, or concrete, just to name a few.
♣ The user can apply only one texture.
♣ Textures work only when they are added to a
material. So, you can’t have a texture without
defining how that texture will be seen on an
object.

Texture A flat 2D image that is wrapped on
an object using a material.
Another way to add an image to the surface of
your model is through the use of a UV map.
♠ This flattens a 3D model’s x, y, and z
coordinates into a 2D tile.
♠ The name UV doesn’t stand for anything,
but is instead used methodically. The x-
axis translates to the U, the y-axis
translates to the V, and then some maps
also use the z-axis, or depth,
represented as W.

♥ To take more control of the appearance of your model, you can also add
shader information to a material.
♥ This holds data about the object’s appearance. As the name implies,
shaders can change the appearance of darks and lights using gradients, or
even by creating custom shading and make the object look glitchy, electric
powered, or even change colors.
♥ Once you start getting more comfortable working with your 3D models,
materials, and textures, you can venture into the use of shaders to get a
more custom or stylized look to your models.

Camera: Your eyes are a camera looking at the scenes in your physical world.
♠ If you start to think like that, then the idea of the camera view is in essence where
you are looking within a provided space.
♠ In video or film you have control where each person is looking when. This is not
true when working with an individual 3D model, but you can set up a fixed
viewpoint that determines how people will see an object.
♠ You set the perspective, which will be how people will first see the model when they
place it onto a physical object using AR.
♠ There is an option to create a fixed view using a camera to ensure that users
always see the object a certain way no matter where the user looks or moves.
♠ If you don’t have a fixed view, then a user may have full control to view the object
from all angles as they choose.
♠ You can think of the camera as the viewpoint to an object or a scene.

Light
Light is the essence of how you visually experience the world. To see any object or scene you
create, you need to have some light on it. You can determine the qualities and characteristics of
the light you add. As you need light to make a photograph, you need light to see and
understand a 3D scene.
☻In XR, light becomes an essential part of the scene. It isn’t just that the light can brighten
some areas and darken others, but rather that it feels like it belongs in that space.
☻what direction they go, how dark they are. If you were to add another object into that same
space, it would need to have a shadow that mimicked the others to appear realistic.
☻Just as with a photo illustration, where you make selections from multiple photographs and
composite them into one image, you need to create a universal light source.
☻Each element should have shadows in similar directions based on the placement of the
perceived placement of the light.

Tip
● Remove all materials from your object when you are designing your lighting.
● The materials can change the way you perceive the light and even the color of
the light. Once your lighting is set, then you can reapply your materials.
● The good news is that there is some great computer vision technology that you
can use to assist with this.
● However, you still need to know how to create multiple lighting setups. This
includes understanding environmental versus directional lights.
● Sunlight creates stronger and deeper shadows than an artificial directional light
in a scene. We know this from experiencing light; the next challenge is re-
creating it in a digital environment.

Scene
➔ When you create for 3D, you are designing a full scene. This may feel like a lot
during your transition from designing for a single paper or screen to designing
an entire scene.
➔ Work on creating a single primitive first. Then work on creating more complex
objects using only primitives, after which you can explore creating more organic
shapes.
➔ From there, you can add lighting to mimic the kind of space you are envisioning.
Take it to the next step by adding materials and textures. Once you have all of
these set, you can start looking at the bigger picture. This includes the camera
and the environment.

➔ The environment is the space that surrounds a
3D object and contains the lighting and ground
plane.
➔ In the case of VR, the environment will be a key
component to creating the digital space people
will walk through.
➔ When you think of the environment, think of
creating a diorama. As you may recall from your
childhood school projects, a diorama is a
miniature replica of a larger 3D scene.

• its benefits for the design process. As you think of designing the environment,
• it may help you to picture a small box scene, as you would make for a diorama.
• At a minimum you need to have a floor. This is where all the objects will be
anchored.
• Even if you are creating only one 3D model, you still need to identify the location of
the floor, which is shown by the value of the x-axis, the horizontal coordinate.
• If you would like something to touch the ground, then you need to position it
accordingly. If you would like it to float above the ground, then you need to adjust
your y-axis coordinate to lift the object up.
• The ground location will help set the relationship of the object itself to its
environment.

The visual feedback that confirms
this will be the placement of the
shadows created by the lighting and
where the shadows fall. All the
different elements are coming
together.
Just to recap, your full 3D scene includes:
•Objects with materials and textures
•Environment
•Lighting
•Camera(s)

Render
When you are working within 3D software, it is important to know that what you are
looking at as you work may not exactly match the final version.
Once you create your full model and scene, then you need to render it. 3D rendering
is the process of interpreting your 3D content to a form you can share with your
audience.
While many programs have render previews, these often are not high quality and as a
result don’t show an accurate visual of the materials and lights.
It isn’t until you do a full render, that you, in essence, export your final model or
scene.
Some programs offer an option to do a lower resolution render, which I would
recommend doing to check the result of major edits and changes in lights, materials,
and environmental decisions.

3D rendering Conversion of a 3D model into a 2D image that can be displayed on a screen.
With AR and MR, the need for rendering has greatly increased. This has pushed programs to use a
real-time rendering engine. Unity and Blender both offer this option.
• Blender is free and open-source 3D software that uses an EEVEE real-time renderer. Unity
offers a Sprite Render that allows you optimize for different software. For a higher quality, there
is high-definition render pipeline (HRDP), and specifically for mobile use there is a lightweight
render pipeline (LWRP).
• Vectary provides a real-time rendering process within a web-based program. This is called
physically based rendering (PBR), which computes lights, shadows, and reflections in real
time. It is reliant on your internet speed and connection to work, as it is a web program.
• Other popular 3D modeling programs for designers are Cinema 4D and Adobe Dimension.
• The one nice thing about Adobe Dimension is that you can work with Adobe Photoshop and
Illustrator to create your own custom models from 2D shapes.
• Rendering from Adobe Dimension also allows you to select a Photoshop export instead of just
the 3D model.

3D files
The purpose of a 3D file is to store all the information about the 3D models, including
the overall shape geometry. Some 3D file types include only the data as it relates to the
shape itself, while others also include properties, such as materials, scene elements,
and animations. Because each of these properties adds to the complexity of the model,
they also increase the file size.
With XR, file size is an important issue that you need to pay attention to. Depending on
the device you are designing for, the file size limitations may change. For mobile AR and
WebAR, you need to optimize the object to reduce how much processing of the image is
needed on a mobile phone. We have talked about how more complexity needs more
processing, which needs more power, which creates heat. Because you can’t have an
HMD that produces a lot of heat while it’s worn on the face, you must be thoughtful of
how much processing is needed to display your designed models.

3D files
The best way to reduce file size is to reduce your polygon count. You are trying to achieve a balance
between the appearance of the model and the performance.
More complex shapes will affect the load time and possibly slow down the experience for the user. If
your computer can’t process the full file,
it will often glitch or even crash the program, which can be frustrating to you and if you are
working with a developer.
This can be even worse for the users. So, as you build, you need to plan for the technology you
want to use.
It is fun to play around with materials and shaders, but you don’t want them to interfere with the
overall goal of the experience by increasing file size and reducing performance.
VR experiences that require tethering to a high-powered PC allow you to off-load this processing,
but as VR becomes more mobile as well, the same considerations will apply.

There are a number of file formats that you can use in the 3D world. The kind of format you end
up using is reliant on what software you work with.
It is important to know which file format you will need to add a model to your XR experience
and then work backward from there.
Each program will render different file formats, so you could choose your 3D program
accordingly, or you could use a file convertor.
If your file isn’t too large, you could try a web-based format converter, such as
meshconvert.com.
To convert larger, more complex files, you will need a desktop application, such as Spin 3D
(NCH Software).
File conversion will add more time to your process, so account for that if you plan on needing to
convert files based on the software you choose.

One way to speed up your creation of a 3D assets is to purchase a license to use one that is already made.
These are some places you can purchase and download 3D models and textures:
•TurboSquid
•Sketchfab
•Google Poly
•Adobe Creative Cloud and Mixamo
•Unity Asset Store

2.2 Affordances - Multimodal experiences
Affordance(a use or purpose):-
Physical properties of an
object or environment that
indicate its functionality.
When the design of the object
interferes with the way we
interact with it, then it causes
frustration.
1. Real
2. Perceived
3. What can it do?
Multimodal experiences:-
It is a design for XR, you are designing a sensory
experience, with information and feedback available
from multiple modalities, engaging multiple senses
at once. If the balance between the different sensory
inputs is off, then the user will likely react to it.
1. Visual
2. Auditory
3. Olfactory
4. Haptic
5. Proprioceptic and kinesthesic
6. It all makes sense
7. Accessibility

Affordance:- Physical properties of an object or environment that indicate its
functionality. When the design of the object interferes with the way we interact with
it, then it causes frustration.
1. Real
2. Perceived
3. What can it do?
♣ The use of physical world as inspiration for how certain senses are engaged and
mimic these in an immersive experience, that balance will provide comfort to the
user.
♣ The key is making an emotional connection, one that the user may not even realize
is happening. Creating something with these visceral(emotional) qualities is easier
said than done.

★ This is an area where the roles of
industrial designers, interaction designers,
and experience designers overlap.
★ We strive to create objects that, how we
look at computer-human interactions.
★ The concept has a deep connection to
psychology, as well as the understanding
of how humans perceive physical objects.
★ This understanding is a core fundamental
needed for designing 3D in digital space,
especially as we work toward designing
worlds where our digital designs will be
augmented to our physical world.

Real:-
❖ If you are creating an object that you expect people to rotate or turn, how
will you indicate that behavior in the overall design?
❖ You should consider how the design will afford the best interaction. The best
interactions happen when they are successful.
❖ Norman describes these as real affordances, as they are actual properties
of the object or environment itself.
❖ These don’t require a previous understanding or knowledge of how
something works, because the design indicates the proper interaction.
Examples of this are mugs, teapots, and scissors. These don’t require any
instruction or labels for people to use them correctly. They lend themselves to,
and are designed for, just one function.

● If you look at a mug, there is only
one handle, which is shaped for
your hand;
● without any other option of where to
grasp, what do you do?
● You wrap your hand around the
handle and pick it up.
● The physical property of the handle
means that it is real.

Perceived
↔In contrast to real affordances, perceived
affordances may be more ambiguous.
↔The affordance is not communicated through
a physical property but rather through a
user’s previous experience and interactions.
↔Perceived affordance is use in digital
interactions and computer interface design.
↔When you arrive on a website, you look for a
menu, and possibly even a hamburger menu
icon.

♣“The UI of XR.” Shadow can add an understanding of depth.
♣Guides inside or outside the object help show it can be selected.
♣Icons can indicate the motion, such as move, rotate, scale, select.
♣Feedback can provide communicates the interaction to the user through
a variety of different senses, including sound, touch, or visual change.
What can it do?
♥To get a better sense of affordances and how impactful your design
choices are in assisting a user, what kind of interactions are
associated to each element?
♥What it can do? and how people will interact with it should greatly
influence the design? For each element, identify its possible actions.

The categories are:
Static: An object or element that does not move and cannot be interacted with,
although could possibly be viewed from different perspectives.
Animated: An object or element that moves in a sequential motion over a specific
amount of time.
Dynamic: An element that can change based on interactions or inputs from a user or
over time.
Interactive: An element that the user can control, manipulate, and interact with.
Metric: An element that changes based on a mathematical calculation or algorithm.

Affordance(a use or purpose):- Physical properties of an object or
environment that indicate its functionality. When the design of the
object interferes with the way we interact with it, then it causes
frustration.
1. Real
2. Perceived
3. What can it do?

Multimodal experiences:-
It is a design for XR, you are designing a sensory experience, with information and
feedback available from multiple modalities, engaging multiple senses at once. If
the balance between the different sensory inputs is off, then the user will likely react
to it.
Multimodal
experiences
Visual Auditory Olfactory Haptic
Proprioceptic
and kinesthesic
It all makes
sense
Accessibility

1.Visual
♣ The first sense that you may think of, especially as a designer, is our sense of
sight.
♣ This is our visual ability to interpret the surrounding environment and is reliant
on perceiving the waves of light that reflect off objects in order to see them.
♣ color, the visual spectrum also referred to as the human color space and visual
perception influences our immersive experiences.
♣ If you want someone to be able to walk through a historical place, that may be
best experienced from a second-person viewpoint, where the story can be told to
you. You want to choose the point of view that makes the most sense for the
content.

Visual Perception and Immersive Experiences
Human Color Space
Presence and Immersion
User Point of View (POV)
Consistency in Visuals
Narrative and Storytelling
Content-Driven POV Selection
User Engagement and Interaction
Visual

1.Visual Perception and Immersive Experiences: it is used for designing
environments and interactions that feel authentic and compelling.
2.Human Color Space: The visual spectrum, or human color space, is the range of
colors that our eyes can perceive.
• Designers need to consider color theory and how different colors evoke emotions,
convey information, and influence user behavior within XR experiences.
3.Presence and Immersion: Achieving a sense of presence is vital in XR design.
• Presence is the feeling that users are truly present within the virtual environment
they're interacting with. This can be enhanced through realistic visuals, accurate
scaling, and seamless interaction mechanics.

4.User Point of View (POV): The chosen point of view significantly impacts how users perceive
and interact with an XR experience.
• Different POVs, such as first-person, second-person, or third-person, offer unique
perspectives and cater to different types of content and narratives.
5.Consistency in Visuals: To maintain the illusion of immersion, the visuals in VR should be
coherent and consistent. The virtual world has its own rules and physics,
• It has reflect and adhere to those rules to create a believable environment.
6.Narrative and Storytelling: The choice of point of view affects how storytelling is conveyed
within an XR experience.
• First-person viewpoints can provide personal immersion, while third-person viewpoints may
allow users to observe a story unfold from an external perspective.

7.Content-Driven POV Selection: The point of view should align with the content
and the experience's objectives.
• For example: Hybrid classroom host, presenter, simulator…
8.User Engagement and Interaction: Visual cues and elements play a significant
role in guiding users' interactions within XR environments.
Clear visual feedback helps users understand their actions and the consequences
within the digital space.
It's a multidimensional process that requires a deep understanding of both design
principles and human perception.

2.Auditory
Auditory
The sound design is a vital
skill in modern design,
especially within the realm of
Extended Reality (XR).
Understanding how to
effectively utilize sound and
optimize audio experiences
can greatly enhance various
design disciplines.
Importance of Sound Design
Perception of Sound
Sound's Role in Environment
Spatial Audio
Surround Sound and Immersion
Ambisonics for XR
Chapter on Sound Design

1. Importance of Sound Design: Sound design is becoming increasingly crucial for
designers across different domains.
Whether it's interaction design, motion design, or XR, sound plays a significant role
in creating immersive and impactful experiences.
It contributes to storytelling, emotional engagement, and user perception.
2.Perception of Sound: Our ability to perceive sound is complex and relies on the
cochlea(inner ear- organ of Corti-which response to sound vibration) in the ear,
which converts vibrations into neural signals that our brain interprets as sound.
Understanding this process helps designers create audio experiences that resonate
with users.
3. Sound's Role in Environment: The way we hear sounds changes as we move
through a space, and our proximity to a sound source affects our experience of it.

4. Spatial Audio: In XR experiences, replicating the spatial aspect of audio is crucial
for creating a realistic and immersive environment.
Similar to visual cues, sound cues should be directional and change based on the
user's position in the virtual or augmented space.
5.Surround Sound and Immersion: The difference between traditional stereo sound
and surround sound is akin(similar) to the difference between two-dimensional and
three-dimensional experiences.
Surround sound can significantly enhance emotional engagement and make users feel
like they're truly present in the XR environment.

6.Ambisonics for XR: Ambisonics is a technique used in XR to create 3D
soundscapes.
It involves capturing audio from all directions and reproducing it in a way that
maintains spatial accuracy, enhancing the immersive quality of the experience.
7.Chapter on Sound Design: Given the depth and importance of sound design,
dedicating a chapter to it in your learning material is a great approach.
This topic encompasses various aspects, from creating realistic auditory environments
to establishing emotional connections through sound.

3. Olfactory
The sense of smell, or olfaction, holds a unique power to evoke memories and emotions.
While it might not be the first sense that comes to mind in the realm of digital
experiences, it's gaining attention as researchers explore its potential in enhancing
virtual and augmented reality (VR and AR) experiences.
1.Smell and Memory: Smells have a remarkable ability to trigger memories and
emotions.
Certain scents can instantly transport us back to specific moments or places, creating a
strong and lasting connection between scent and experience.
2.Olfactory Association: We often associate particular scents with people, places, or
even seasons.
The scent of spring rain, someone's perfume, or the distinct smell of a familiar location
can have a profound impact on our perceptions.

3.Digital Scent Research: The Imagineering Institute's work in creating digital scents is an
exciting area of research.
They've explored the creation of digital fragrances like fruity, woody, and minty scents, potentially
allowing users to experience aromas within digital environments.
4.Enhancing Immersion: Just as visuals and audio contribute to immersion in VR and AR, the
addition of olfactory stimuli can further enhance the feeling of being present within a virtual
world.
5.Future of Digital Scents: While the technology to incorporate digital scents into VR and AR
experiences is still in its infancy, the potential is promising. Imagine being able to not only see and
hear a virtual environment but also smell it, creating a multi-sensory and truly immersive
experience.

6.Use Cases: One potential application is within the culinary world. In the future, AR might
enable users to virtually smell and see dishes from a restaurant before deciding to order takeout,
adding a new layer of sensory exploration to the decision-making process.
6.Challenges: Creating and delivering digital scents in a consistent and controlled manner
presents challenges. Developing technologies that can accurately replicate and emit scents on
demand is complex.
The integration of smell into XR experiences adds another layer of sensory engagement,
potentially making digital worlds feel more tangible and real. As research in this area progresses,
the possibilities for enhancing the way we interact with digital content and environments are
exciting to consider.

Haptic technology is a critical aspect of designing immersive and interactive experiences,
particularly in the context of Extended Reality (XR).
It focuses on recreating tactile sensations through vibrations or motion feedback, enhancing
user engagement and providing a more intuitive interaction. Haptic
Haptic Feedback and Touch Screens
Haptic Sensations
Feedback Reinforcement
Designing Haptic Feedback
Proprioception
XR and Haptics
Considerations and Challenges
Communication and Meaning
Instant Feedback
Haptic

Haptic Feedback and Touch Screens: Touch screens lack tactile feedback, which can
sometimes result in a less immersive interaction.
Haptic technology addresses this by providing users with physical sensations in response to
their actions, enhancing the sense of touch in digital environments.
Haptic Sensations: Haptic feedback can range from subtle vibrations to more noticeable
motions, depending on the action being performed by the user.
Feedback Reinforcement: Haptic feedback reinforces user actions by providing a response to
their interactions.
It helps users understand that their actions are registered by the system, which can be
particularly useful in situations where visual or auditory cues might be insufficient.

Designing Haptic Feedback: matching the intensity of the feedback to the action.
Consistency in feedback across similar interactions ensures that users can anticipate and
understand the actions they're performing.
Proprioception: Haptics contribute to proprioception – our awareness of our body's position
and movements. users can better perceive their virtual interactions as they align with their
natural proprioceptive sense.
XR and Haptics: Haptic feedback is commonly integrated into hand controllers in XR
experiences, offering users a physical connection to virtual objects and actions.
In mobile AR, built-in haptic technology can provide feedback to users without requiring
additional hardware.

Considerations and Challenges:
Unwanted vibrations or excessive feedback can disrupt an experience, particularly in scenarios
involving video capture or sound recording.
Communication and Meaning: Haptics communicate messages to users nonverbally.
Consistent use of haptics helps users associate certain sensations with specific actions, creating
a form of communication that aids in understanding and engagement.
Instant Feedback: Haptic technology offers instant and tangible feedback to users, providing
them with information about their actions in real-time.
This immediate response helps users make informed decisions and navigate digital
environments effectively.
Incorporating haptic feedback into XR experiences can bridge the gap between the physical
and digital worlds, making interactions more intuitive and immersive.

Proprioceptic and kinesthesic: Close your eyes, and try to touch your index
finger to your thumb. Try it on both hands. If you don’t see your fingers, how
can they find one another?
They can find each other because of proprioception, the sense from within. Often
referred to as the sixth sense, the perception of the user’s position is an essential
component for immersive experiences.
Proprioception The perception and awareness from within of the position and
movement of the body.
As you put on an HMD, you will be relying on your internal senses to help you use
a hand controller, make hand gestures, and move—even while you can’t see your
physical body.

Kinesthetic:The sense that enables
movement of the body in conjunction with
the awareness of the position of muscles
and joints.
It is important to know that our bodies are
using more than just our five main senses
in an immersive experience.
Advantage as you design to ensure that
people can actually interact with an
environment and move through it, even if
they are not able to see their hands and
feet.

It all makes sense: In XR, multimodal, or multi-sensory, is the combination of
visual, auditory, tactile(touch sense/feel), sometimes even smell, and
proprioceptors to create a fully immersive experience.
♣Multiple modalities are used effectively within many different industries, from
transportation to education.
♣If you listen to a podcast, you can hear someone talking, a single modality, but
you don’t see their body language as you would in an in-person lecture or even a
virtual webinar.
♣If an in-person speaker uses visuals as they speak, then they are using a
variety of modes to communicate the message.
♣The words that they speak, the tone of their voice, their body language, and the
visuals they show all work together to help you, as the viewer, comprehend a
message.

♣The wider the variety of modes that are used, the more ways that you can
connect and engage with the information being shared, both verbally and
nonverbally, and the more engaged and connected you will become with the
experience.
♣If someone talks about a story while showing you an image that you find
disturbing in any way, your whole body will react to it. The level of connection to
the experience will change the user’s response too.
♣Our somatosensory perception translates all these stimuli and combines them
to define what the body senses, and therefore knows, on a multidimensional
level.

Somatosensory perception The cohesive
awareness of what is going on inside you
and all around you through sensory
stimuli.
As we create XR experiences, we look to our
real-world experiences to inform the virtual
and augmented worlds. In our physical
spaces, communication is multimodal.
You can read an emotion of someone from
across the room just from looking at their
facial expressions.
You can show someone where something is
by pointing in a direction or can reinforce
instructions you are speaking by using
gestures.

Accessibility
❖ Just as shortsighted as it would be to have the office for disabilities services on
the second floor of a building without an elevator (yes, this has actually
happened), it would be equally crazy to not use opportunities to engage a variety
of senses in your experience to make it more accessible.
❖ When it comes to accessibility in XR experiences, The subtle vibration of a device
can provide great feedback to the user without needing to make any sound.
❖ This benefits those who are hard of hearing or deaf, but also those who have the
sound turned off because they are in a public space.
❖ It is important to understand that people will need different accommodations for
different reasons throughout the day, at different locations, and even throughout
different times of their life.

• Those with any of the many varieties of cognitive, physical, and psychological
challenges can benefit from an experience that engages more than sense.
• Our needs may vary from a permanent disability to a temporary challenge
created by a specific environment.
• If there is any part of the experience that can only be understood through
hearing, then you could be excluding part of your audience.
• One way to help make the experience individualized to accommodate the
needs of each user is to offer a setting that can be adjusted at any point in the
experience. There may be times when a user may be in a louder physical
space and might elect to have closed captioning turned on during that time.
Or if a person would like to always have captioning on, they can do that too.
• But if you don’t include this as an option, you are excluding people from using
the experience.

As you work to create multimodal experiences, consider how the user experiences each part of
it.
If there are any areas that are too reliant on only one input type, then that is an opportunity to
expand the sensory experience to make it a more immersive experience and more accessible to
the wide needs of users.
Haptic Feedback and Touch Screens
Haptic Sensations
Feedback Reinforcement
Designing Haptic Feedback
Proprioception
XR and Haptics
Considerations and Challenges
Communication and Meaning
Instant Feedback

Multimodal
experiences
Visual Auditory Olfactory Haptic
Proprioceptic
and kinesthesic
It all makes
sense
Accessibility

Experience design
Throughout this chapter we have been looking at foundational elements
of the immersive experience.
Starting with some of the new vocabulary and workflow of working within
a 3D rendering program, moving on to understanding how to create the
best interactions with those elements, and then looking at how much our
senses are a part of the experience. All of these elements need to be
designed in harmony to create a positive experience for the user.

Now that we have set the foundation of
what the capabilities of technology are
as well as explored some of the
introductory concepts to designing for
XR, it is time to start going through
each step of the design process,
starting with coming up with your next
big idea.
Design Challenge

3D MODEL
For this challenge you will need a couple supplies:
• Marshmallows (small ones work best, but you can use whatever you have)
• Toothpicks
If you don’t have one or both of these, then you can be creative with substituting
other options. Instead of marshmallows, you could use clay, modeling dough, dried
or fresh fruit, gummy bears—anything soft. Instead of toothpicks, you could use
cotton swabs, craft sticks, pencils, branches—anything that you have a lot of and
that is long and thin.
Once you collect your supplies, it is time to get to work. To start, select a primitive
shape—cube, prism, cone, or cylinder—to create. You will use your marshmallows
as the vertices to hold together your toothpicks, which will act as the edges. Start by
completing one face (one polygon) of the shape.

For example, if you are building a cube, then create a
square as the first face. From there, build out your full
cube using your supplies. This first round is just to
help your brain understand how to put a 3D shape
together. For a bigger challenge, find three objects
around you and create three separate marshmallow
models to represent them.
Be creative. The only rules are that you use only your
marshmallows and toothpicks and that the objects
must be full 3D models. See if you can challenge
yourself to use as few polygons as possible to create
the shapes.
(Oh, and don’t forget to save some extra
marshmallows for a snack after all that hard work!)

4. Ideation: Identify the why
We will be identifying those as well as expanding them by
providing techniques for brainstorming and exploration to assist in coming
up with a concept for your XR and 3D projects.
❖IDENTIFY THE WHY
❖BALANCE INNOVATION AND PRACTICALITY
❖CONSIDER OUTSIDE INFLUENCE
❖KEEP IT HUMAN
❑ Solve a problem
❑ Step back and observe
❑ Identified problem
❑ Medium
❑ Advantages
❑ Merging realities

4. IDENTIFY THE WHY(you need to identify the why)
Why? Why are you creating this experience?
This is the most important question to consider when you are at the
start of any project. If you cannot answer this question, then you are designing
backwards.
Starting this process, you need to have a clear understanding of the purpose of
your experience.
What problem will it solve?
♥Will it be educational? Social? Navigational?
♥Will it offer people a new solution to a common problem?
♥Will it serve as a tool to help improve a daily task?

Solve a problem:- “I want to create an XR experience,”
To see dinosaurs made of paper walking down the street in front of
you, but what problem does that solve?
What do we gain from this experience? Consider why people would want to
experience this. Perhaps they want to learn more about dinosaurs. it is to
show the size proportions of a dinosaur to our modern spaces. There are
likely many other reasons you could brainstorm to determine why a
dinosaur’s AR experience could be beneficial, but the one you choose will
influence all the other design decisions that come next in the process.
Why are there dinosaurs?
Why are they made of paper?
Why are they walking down the street in front of you?

• Without this foundation, people don’t have a reason to come back to the
experience over and over again.
• Once your user has experienced it, they move on to the next thing—that
is, unless you give them a reason to return.
• If you are in search of a purpose for your project, two approaches can
be helpful.
STEP BACK AND OBSERVE: step back—literally, away from your
computer, your screens, the rectangles that confine our views—and be
inspired by actual reality.
Bring a pen and something to sketch on with you wherever you go, and
just observe.

IDENTIFY WHERE PEOPLE FACE CHALLENGES IN THEIR EVERYDAY LIFE.
• It could be reading a menu from far away, finding something in a grocery store, or trying to
watch the road and traffic while simultaneously watching the turn-by-turn directions on a
GPS.
• Think about a common frustration in your life or that you have observed others around you
experience to see if there is an opportunity for an augmented experience to improve it or
reduce the dissatisfaction.
• Remember when you are interacting with an immersive application, you are experiencing the
final product that came to fruition only because of the many rounds of evolutions,
exploration, and brainstorming.
Every product started off at the same place you are now. If you focus on the end product.

Form follows function
In consulting with a student on which technology they should use for their
project, the first question I ask always is, why? Why are you creating this
experience?
What do you want people to learn, feel, or experience while
interacting with your project?
The answer guides the choice of specific medium. In one case, the student
project was an art installation that was created to make people more aware
of the light in their environments.
With light as a primary inspiration for the project, it was clear that the
medium selected should celebrate light and make people even more aware
of it.

Projection mapping was the
direction they selected as a
result, which re-created
moments of light into the
different spaces they in stalled
the project.
Never forget a name again:
As a professor, I meet 100 or more new students every semester, and I am always struggling to
remember their names. When coming up with the concept for tagAR, I was looking to create a
tool that helped people helped solve this problem. Think of how impactful it is when someone
you have met only once comes up and addresses you by name. That was the scenario I wanted
to help facilitate.

IDENTIFIED PROBLEM. Forgetting people’s names, as well as the need for
multi-modal introductions, offering auditory and visual cues to assist those
with dyslexia and other specific learning disabilities.
MEDIUM. Augmented reality mobile application for initial launch, but
ultimately created for a hands-free experience using AR glasses with built-in
speakers.
ADVANTAGES. AR allows the user to see name tags right where needed,
above a person’s head. Using AR glasses, the user can look the person in the
eye and address them by name.
With spatial speakers that are part of AR glasses, users can tap on a name
tag to hear the correct name pronunciation in-ear before having to say it out
loud.

Merging realities:-
To further expand your ideas for the potential of what AR can provide in this ideation stage,
consider how you can merge realities. How can the experience be improved by combining things
that will already be in the environment with the digital objects that your XR solution will provide?
● Who your project is for?
● When will it be used?
● What length of time if is used?
● Where it will be used?
● How best is it used?
● How will it influence the user?

5. Innovation and practicality - Outside influence
Innovation and
practicality
You don’t have to be innovative
Make thoughtful choices
The power of deadlines
Outside influence
Promote divergent thinking
Random influence
Narrow with convergent thinking
Look for themes and patterns
Reimagine how we see

♣ Innovation without constraint is the definition of creativity. Within the XR field there is often a
parallel between technology and a need or push for innovation.
♣ There is a duality of pressure between trying to create something new with the technology
while making it innovative.
♣ Example:
♠ The first person to add augmented stickers in their app was innovative,
♠ the second person to add this became a copycat.
♠ However, adding animation or even better animations that track with moving objects, such as
a face, to these stickers made them innovative once again.
♠ The sense of innovation comes from this sense of new or a new idea. When you combine a new
way to do something with new technology, the result rises above the ranks.

You don’t have to be innovative
♥ The risk of this rush to be innovative has its downfalls.
♥ Many businesses and companies release products before they are ready, quite possibly so
they can claim to be the first to do so.
♥ They also may not spend as much time, energy, or focus on the design of the experience
as needed in order to get the working product out to market to claim the spot as first.
♥ why design is such an important part of the process and why it needs to be considered from
the very beginning.
♥ Early adopters of technology can often see the potential of what their ideas can become with
time and a focused vision.
♥ To me, this is much more innovative. Don’t create an idea to simply use the technology and
figure it out from there.
♥ Focus your creativity toward an end goal that can be achieved at a higher standard through
the use of emerging technology.

♥ Remembering your goal, your purpose—understanding why you are creating the experience—
will keep you focused during the process.
♥ As Steve Jobs said, “I’m as proud of many of the things we haven’t done as the things we
have done. Innovation is saying no to a thousand things.”
♥ You can see how this can be true in other scenarios, such as if you have ever gone grocery
shopping without a list. With so many options and things you could buy, you may come
home with bags full of food, but nothing to eat for dinner or a complete meal.
♥ Focus. This may feel intuitive, but when searching for innovative solutions, there is more
distraction
♥ caused by seeing and trying all that is “shiny and new.”
♥ Technology is evolving and improving at such a rapid rate, it is phenomenal and also
overwhelming. Trying to create content for multiple platforms may not be possible with these
changes happening so rapidly; instead, try focusing on one, for the moment, getting that
experience to where it becomes successful and impactful to the user, and then looking to
expand from there.

Make thoughtful choices
♥ If you were about to go for a run, you would have to select a pair of shoes to wear.
♥ It is more practical to wear a pair that is comfortable and that will help you as you run.
♥ In fact, there are shoes made just for the sole purpose of being worn for running.
♥ You can even be so selective as picking a pair based on your terrain preference.
♥ This decision improves your overall running experience. The same should be true for an
immersive experience. You want it to be practical for the users.
♥ It’s vital to have answers to these essential questions:
♥ Where will users be when they interact with your experience? Indoors? Outdoors? In public? In
private?
♥ For how long will they typically be interacting?
♥ Will they be sitting, standing, walking around?

The power of deadlines
⁂ Stress and time constraints confine innovation. As does lack of sleep,
taking the same route each time you go somewhere, and working
toward perfectionism.
⁂ Knowing these variables will alter your ideation process, you want to
try to find optimal times to work and focus on your ideas.
⁂ If you are like me, having a deadline is helpful to completion of a
project or task, but at the same time, it can add stress if you haven’t
planned ahead of time and managed your time.
⁂ Having a deadline creates a need for definitiveness which magically
allows you to make final decisions and to get things done.

♥ However, use the time before a deadline for this purpose—to make final decisions on anything
you have been deciding about—versus to create new ideas, as they will be strained with the
pressure.
♥ With new technology, or really any technology, you need to be realistic about the possibility of
change.
♥ Ideas on their own don’t make a difference; things do.
♥ You have to continue to work ideas into a real product to start to measure their impact.
♥ Working within this field there is great potential to make a difference and a change in people’s
lives.
♥ Do this, and you will create experiences with purpose.

5. Innovation and practicality - Outside influence
Outside influence
Random influence

⁂ People have about 60,000 thoughts each day, according to research completed by the
Cleveland Clinic.
⁂ Most of those thoughts, about 95%, are the same thoughts as the day before, and the day
before that.
⁂ Our habits and patterns influence our thoughts, and this repetition will also have a strong
impact on the dialogue in our minds.
⁂ This may not seem possible until you think about how watching a movie, reading a book,
or having a conversation with someone else all can change your perspective of
thinking and help divert your thoughts in another direction.
⁂ If you stay within your own head and your own thoughts, you may be limiting yourself to the
same recurring thoughts that you know and are comfortable with.
⁂ Imagine the potential when you take just one of those thoughts and filter it through an
unexpected juxtaposition from an outside source.
Outside influence

•Identify the biggest pain points for the user.
•Narrow your focus, and find parts of the process where you would have
the most impact.
•Identify the priority for the user. Compare this to the current priority of
the process.
•If these don’t align, then you could easily find a place to get started with
confidence with the data to support your decision.

Divergent thinking The ability to explore multiple solutions to a problem.
What are you thinking of right now? As you read, perhaps you have an
example or use case in your mind. As you digest each new sentence, you
think how it could apply to your project. Now, whatever you do, don’t think
about a red apple. In fact, stop reading for a moment, and focus—close your
eyes if it helps—but think of anything but a red apple.
• If you hand a child a basket full of blocks, they can start building towers, cities, roads, houses, towns, anything
they can imagine.
• Each time they sit down with the same basket of blocks they likely will come up with a new design. This idea of
divergent thinking is an essential building block (pun intended) to the ideation process.
• To find the best solution, or even gain confidence to choose a specific direction, you first need to explore
multiple solutions.

☼How did it go? Chances are, no matter what you did, you thought of a red apple.
☼Maybe you tried to consider a green apple instead just to follow instructions surrendering
to the fact that apple was now in your head.
☼This illustrates how our thoughts are highly influenced by stimuli around us.
☼In this case, it was the object I placed into your mind, an object that was likely not there
until I introduced it.
☼Knowing this, think about how going for a walk or taking a different route home can
suddenly inspire new thoughts and ideas.
☼When you surround yourself with a different environment, filled with different stimuli than
you are used to, the energy changes, and your perspective changes with the unfamiliarity of
your route.

Random influence
If you embrace the idea that our thoughts can be influenced by an external source to introduce new,
different, and additional thoughts, then whenever you are stuck or feeling uninspired in your
brainstorming sessions, introduce this notion into your ideation flow.
This is in essence the fluid process of coming up with an idea or concept.
The expression train of thought provides a visual of what this connected thought process looks like.
It starts the moment you are faced with a problem to solve.
To start this flow of ideas you can try a few different techniques. One technique is to add a random object
or word into your thought. This simple step can help diverge your overall thought flow. The idea is to
change how you were thinking about the original idea.
Tip
www.randomwordgenerator.com is my go-to site, but a quick search will provide a quite few random
word generator sites

● Challenges of high-traffic times
● Differences at low-traffic times
● Optimal placement of the menu
● Location of pick-up windows
● Distance of the drive-thru in relationship to the main street to
accommodate traffic

• This is the beauty of divergent thinking and outside thinking.
• You can open up the possibilities to explore for more than one solution.
• Every idea that you think of will not be viable. So, once you have thought wide and far, you
will then begin the process of convergent thinking, where you narrow your ideas from the
large pool of options to one or just a few of the top solutions.
• These can then be refined in preparation of the creation and testing stages that will follow.
• It is much easier to narrow in your options in this way of thinking after you have spent the time
exploring a wide and vast array of options.
• If you choose one direction from a list of three possible ones, it could be okay, but with so few
options it is hard to really understand rank.

However, if you choose one direction from 150 options, then you know that that idea ranked higher
than 149 others, giving you more confidence in your solution.
Convergent thinking The pursuit of one single solution to a problem.
If we go back to our drive-thru example,
after you have explored a number of different ideas, you would then narrow to just one
approach to explore as a solution to just one problem.
If you determined from your brainstorming session that your ideas about health in the drive-
thru experience was the most important direction, then you would deep dive into that one
topic.
How can health and nutrition be communicated to consumers as a part of the drive-thru
experience?

As you explore this idea be sure to spend time just
looking. In fact, think of it more as listening with
your eyes. Instead of just coming up with ideas about
how to make the drive-thru process more immersive,
go to the drive-thru. Watch others order at a drive-
thru. Now, as you start to explore your idea with a
more narrow focus, you need to consider the where.
● Where do you anticipate people will be using
the product or experience you are creating?
● What is there at that location?
● What do people do there?
● Where are the problems that exist currently?

☺If you consider designing an augmented drive-thru experience,
☺you have a very clear and focused location to answer the question of where you are
designing for: a vehicle. However, it is even more important to identify priorities for the
design.
☺In this case, it is the driver who needs to be able to access the menu information easily
as they will have to place the order while also keeping their hands on the wheel and
keeping their eye on the cars and other barriers around them.
☺By exploring what you know about the driver’s needs, you can start to come up with ideas
for how this experience could be improved through AR.

❖ Even if you have driven through a drive-thru before, you want to make sure that you do a test-drive using and
trying out the different designs to identify your own pain points through the lens of considering how AR
could improve the experience.
❖ Having this hands-on experience will offer one point of view. Then, you need to start the process of looking
and really listening to what you see by watching others as they go through the full process.
❖ To actively be looking, you need to just make observations. Try to avoid assumptions. The goal is to identify
themes and patterns.
❖ Make sure to observe a diverse group of people. Mothers with a car full of kids might have different needs
than a businessperson on their lunch break. Through observation of a wide variety of people, the patterns that
arise will become clearer when you see them as a constant need among quantities of people all with
different goals and needs.

Designing for XR is about reimagining the way we see. But to change the way we see, we
first have to look.
✔ To come up with unique ideas it is helpful to know how we create these images or visual
ideas in our minds.
✔ We have discussed how to listen with our eyes in order to really look,
✔ How our minds perceive what we are observing Activity in the primary visual cortex,
commonly referred to as V1, is where the conscious processing of visual stimuli occurs in
the brain.
✔ The primary visual cortex works as a system that starts with your eyes, specifically the
retina, and is transmitted using the optic nerve to both sides of the brain, or cerebral
hemispheres.

Cerebral hemispheres The parts of your brain
that hold different cognitive processing.
• The left side of the brain is where
processing and analysis occurs, and the
right side of the brain houses creativity
and emotions.
• Activity in all these different areas come
together to coherently comprehend the
visual scene, including depth perception.
• to transfer all the visual signals we see from
the retina to the visual cortex so we can
orient ourselves in new spaces.

♣create a 3D model of the space including trying to identify which direction
is up.
♣Once we can create that in our minds, we must then understand our
position within it.
♣This is an essential part of the creation process to understand so that we
can then mimic the way the mind orients itself in space into the virtual
and digital spaces
♣we are creating Have you ever walked into a space and had to stop to take
a minute to look around to create a better understanding of where you are
in context of the space?

♣ If you open a door expecting to walk into a hallway and you see desks and people sitting
around, you will be taken back and likely will retreat for fear of embarrassment.
♣ This is part of understanding how humans comprehend space and their environment.
♣ Start to pay attention to the thoughts you have as you enter an entirely new space;
♣ these steps are essential to your orientation. This orientation process is the foundation of the
launch of every XR experience.

6. Keep it human - The answer to your questions
Keep it human
Human-Centered Design Toolkit.
Google Glass example
Perfection is not human
The answer to your questions
Design challenge
MIND MAP

Keep it human-Human-Centered Design Toolkit.
• Knowing who you are creating for is critical for it to be successful.
• Your thought process should and will be completely different if you are
creating something for a young child versus an elderly dementia
patient.
• Although it is impossible to design for everyone, it is important to have
a good understanding of some of the challenges and abilities of your
audience.
• This idea keeps the whole experience and design human focused. To do
this, you need to empathize with your audience.

• Empathy is explained as a “deep understanding of the problems and realities of the
people you are designing for” in IDEO’s Human-Centered Design Toolkit.
• To really embrace this human-centered idea, you have to let go of feeling like you have to
know the answers at the start of your design process.
• The intent is that you will find the answers along the way. You have to be okay not knowing
and letting go of expectations or assumptions of what might happen to open up your mind to
be accepting of what information you will learn during the process.
• The process that IDEO explains in their design kit (designkit.org) brings you through three
main phases including inspiration, ideation, and implementation.
• Through these steps you build a deeper empathy for those you are designing for.

Human-centered design is a practical, repeatable
approach to arriving at innovative solutions. Think of
these Methods as a step-by-step guide to unleashing
your creativity, putting the people you serve at the
center of your design process to come up with new
answers to difficult problems.

Google Glass example
In 2014, Google released the Google Glass, which was Google’s first wearable product.
This is a classic of example of why empathy and understanding your user are necessary parts of the design process,
both in traditional design and even more so when exploring new territories such as the XR space.
Google made assumptions about the users and did not clearly define what problem the product was solving for the
users. It was as if they expected people to buy the product based on the hype;
Google put being first before being the best. They lacked a clear understanding and communication about the best
use cases for the product and how the product would fit in to culture and daily activities

Perfection is not human
⁕ It is safe to say that humans are not perfect. That is good news because
perfectionism is boring and mundane(old), so striving for perfection should not be
the goal.
⁕ Knowing this, and that we will make mistakes along the way in the design process,
allows us to own up to them, learn from them, and then continue creating.
⁕ You may have to continually adjust your focus and make changes as you go
through the design process.
⁕ you also have to accommodate any possible technology restrictions.
⁕ It is good to embrace some sense of imperfection in your work to accommodate
this. As we start to explore how to merge physical and digital worlds.
⁕ we must bring some humanity into the digital environments and spaces that we
create; this will make them more approachable and inviting.

The answer to your questions
To create an idea is to really see the
idea, both in your environment and
also in your mind.
To have a fully comprehensive idea,
1. you need to be able to know
2. why you are creating
3. what you are
4. how it will work
5. where it is intended to be used,
6. who will be using it.
• Then make sure you take all of
these variables into account to
determine that your idea will
work for the technology that
you plan to test your design on.
• Once you have your answers to
these questions and you have
converged your direction, you
can move on to the next stage
of the design process: the
prototype.

Design challenge-MIND MAP
1. Create a mind map using only circles, lines, and words.
2. Choose one main theme and place it in the center of the page, and then set a timer for
one minute and write as many connecting words as you can from that center shape.
These should each be within their own circle.
3. Next, set the timer for 30 seconds and add additional thoughts that come to mind
from each of the secondary circles.
4. Continue this one for more level.
5. Once you have three levels, take a step back, and highlight intriguing ideas.
6. During the timed intervals, it is important not to judge how good or bad your
thoughts are.
7. Save this until the end, and then pull out words that seem to pop out at you.

https://mm.tt/app/map/2946149842?t=KN1YygqKTO
https://lucid.app/documents#/documents?folder_id=recent

7. Creating The Prototype: Fake it 'til you make it
FAKE IT ’TIL YOU MAKE IT Overcome imposter syndrome—through creation and
experimentation—to foster a more optimistic mindset that enables you to try something
new or unknown.
SKETCH TRANSPARENTLY When brainstorming for augmented and mixed reality, it
is important to test your ideas in changing environments and backgrounds without the
mechanism interfering.
THE POWER OF PROTOTYPES When you are trying to figure out the experience and
flow of your experiences, you need to create rapid tests using basic materials.
USE WHAT YOU KNOW When creating with new technology, lean on the familiar to
take away some of the intimidation.

FAKE IT ’TIL YOU MAKE IT Overcome imposter syndrome—through creation and
experimentation—to foster a more optimistic mindset that enables you to try something new or
unknown.
It is said that if you change the way you feel or think about something it will provide the
confidence you need to continue to feel and think in that new way.
This approach can be useful in overcoming the lack of confidence you may have when you are
learning something new.
If you tell yourself over and over that you can do something, then you start to believe it and
you will eventually be able do that thing.

This optimistic mindset is especially
important when working with something
new and unfamiliar to you.
The truth is that for many who start to work
in the XR field, much of it will be in this
unknown territory. Here’s where you will
have to fake it at first, because so much of
it is new—not mention constantly changing
and evolving. The only way to keep up is to
dive in and create something. Experience
the full process.

This positive thinking approach can also help overcome imposter syndrome, a psychological pattern
of self-doubt caused by feeling that you aren’t actually qualified for the work you are doing.
If that doubt takes over, it can stop a great idea before it ever has a chance to get off the ground, or
worse, it could alter an entire career path.
Imposter syndrome Doubting yourself, specifically your skills and accomplishments, in fear that it
will someday be discovered that you are an imposter or fraud and that you aren’t as talented as once
thought.
If you had only ever worked in a field where you felt like you had enough experience,
how would you ever have gained that experience? In fact, in XR, there are so few people who have
the experience that everyone is learning together. So, if you feel like you have a lot to learn, then
you are far from alone.

However, without faking it at first, they would never have pushed through their own self-doubt to give
their idea a chance.
Imposter syndrome is completely normal; in fact, psychological research shows why this behavior forms
this pattern of self-doubt. By changing the message, to adopt the glass half-full perspective, you can:
1. Tell yourself you can do it.
2. Go do it.

Sketch transparently
Think about your experience reading this book: How much are you noticing the type itself compared to the ideas
conveyed in the type? If designed well, the answer should be “not much at all.” The design of the typography should
not interfere with the delivery of the message. As Beatrice Warde, a twentieth-century scholar of typography, wrote in
her essay “The Crystal Goblet,”
The vessel that transfers information should be as transparent as a clear glass; the less you notice about the typography
of a book, the more it aids in the process of communicating the book’s ideas.
Warde’s thoughts on typography extend to other areas of graphic design, as well. You don’t want the vehicle that you
are using to transmit a message get in the way of the message itself.
This concept is intriguing to explore as it relates to augmented and mixed reality. The intent is to design for a
transparent interface that will allow for information to be experienced within the context of the environment that you
are in.
Taking this concept literally, what then could happen if you did, in fact, design for the transparent page on a
transparent page?

The power of prototypes
● Make it fast
● The walk-through
● Picture this
● User flow

The power of prototypes:
Grab some supplies: paper, scissors, glue, tape, sticky notes, foam core, Play-Doh, and an open
mind.
You may wonder how I could be suggesting using such basic materials to start creating for such a
highly technical medium, but that is what makes quick prototyping so great. Sketching is such an
important part of this process.
You can really focus on exploring and pushing your ideas when just working with your hands. Once
you bring in a computer into the process, you start focusing on the technology of making it work
instead of the idea itself.
Designing for 3D on a 2D device can be difficult to wrap your head around. Really, what better way
is there to work through a three-dimensional idea than in 3D? Another prototyping challenge is
being able to view the environment or model in its entirety.

When you brainstorm physically with whatever materials you have and work without the restraint
of any frames or screens, you will find that you are able to focus on form and relationships.
Make it fast
With paper and a few other supplies, you can create something fast; this is called rapid
prototyping. To get started in this process, you should first select materials based on what you are
creating. By this point you should have answered the why, who, what, and how.
“Ideation.” If you are exploring an environmental space, then you could create a diorama to create
a small model of the space you are trying to design for. If you would prefer to sketch more at
scale, which can be helpful for spatial awareness and depth, then you might want to think bigger.
My vision of a perfect way to sketch for XR involves wrapping a small room with paper and
drawing out the full 360-degree design viewpoints.

Rapid prototyping A technique that involves creating a 3D model using any num
ber of materials to test out an overall idea quickly.
You can even surround yourself with large sheets of paper on stands that are as wide as your
intended field of view. This will guide you to think about the user and their direction of gaze.
Where will they be looking throughout the experience? Their direction of gaze may change
throughout the experience, but for the initial drawing, just select a key moment of the experience
to illustrate. This could be the first interaction of the experience or the key interaction of the
concept, depending on what makes the most sense for the project.

Field of view (FOV) The size, big or small, of the viewing space for an augmented experience.
If you will be working with a 3D model, then you can use some Play-Doh or clay to quickly explore
shape and form.
These can be used as separate explorations, or even better, you can combine these into your spatial sketch
so you can see how the 3D form will relate to the environment.
The best place to be inspired about how to create for any virtual or augmented reality is in reality itself.
Be sure to immerse yourself in the type of environment you are trying to create digitally.
If you are using AR, then test your sketches in the space for which you’re creating the experience. If you
are creating an experience to be used within a museum, example, then you should spend a lot of time in
museums to develop ideas and test your sketches in those spaces.

The walk-through
A creative way for you to get the most out of these immersive and 3D sketches is to
physically act out the experience. Role play a user going through the experience, and be sure
to move your body as the user will.
● If you intend for the user to be sitting the whole time, then you should test the idea
while sitting.
● If you want them to be walking around, then physically walk around as they would.

Picture this: In addition to the time benefits,
rapid prototyping affords the opportunity to
effectively share your vision with team
members.
A quickly created panoramic sketch can
allow others to see the vision of the
experience, offer feedback, or ask questions
as it relates to their role in the project.
This works best for those who have had
experience with the specific technology you
are sketching for, as they can make the leap
from the flat sketch to the immersive
experience.

The best way to get started thinking in
this way is to use a grid that will help set
you up for success. By starting with
paper that has the x-, y-, and z-axes
drawn out, you can sketch freely without
worrying as much about the keeping a
consistent perspective. This is called
axonometric drawing and is a common
practice for architects as they are
sketching out initial ideas

Use what you know
Process work
Desgin challenge

Use what you know
Before becoming an AR designer, I spent a lot of time
working in motion design.
This dated back to the early Flash days, but my interest
really grew when I opened Adobe After Effects for the
first time and saw the potential of how this could change
the way we experienced design.
The exploration of how time-based design and 3D space
could impact the way information was communicated
ignited a new part of my creative process and storytelling
potential.
In addition to motion, I also have a background as a UI
designer and UX researcher.

This journey ultimately led me to XR design, which is really an advanced version of motion design and UI/UX,
but without the confines of the frame.
Realizing this allowed me to rely on many aspects of the design process that I was already using in those fields
for my XR design process.
FROM THE UX PROCESS you can rely on skills such as user research, observation, and wireframing. You
can then just adjust some of these ideas into this immersive framework, such as using the transparency sketching
method mentioned earlier.
FROM THE UI PROCESS you can use a design systems and pattern approach. A goal of a UI designer is to
make the design scalable, and so in this case it just extends the UI into 3D space. The interactions you will
sketch need to account for depth and that the environment is the new interface; however, the basis of interface
design has not changed.

FROM THE MOTION DESIGN
PROCESS you can use storyboard
drawings to show the sequence.
Instead of just your typical
storyboards, extend these to be a
field-of-view storyboard taking into
consideration all that would be in the
user’s peripheral viewpoints as well
as the central focus. This takes into
consideration the fact that the user is
in control of where they look and
what they see within the XR
environment.

Process work:
● Your focused concept
● Your ideas on paper (transparent or otherwise)
● A user flow diagram
● Field of view storyboards
● All these assets brought into the computer and combined in a 360-degree video or as an augmented video in
order to test the full experience
The point is to create a low-fidelity representation of an idea and concept so that it can be more clearly explained.
Your goal is to embrace the design thinking process and work through a lot of the initial questions needed to refine
your ideas before you start working on a high-fidelity experience.
This final output is the XR experience wireframe. Chances are when you bring it into this view, you will need to make
adjustments to the design, which means you are doing it right. This is considered a living prototype that may be
adjusted at least a few more times throughout the next steps as you refine your initial layout and design.

MAKE A PAPER AIRPLANE
To do this you will need the following supplies:
● 10 sheets of letter-size paper
● Scissors (if you want to try trimming the paper to change proportions)
● Space to fly your planes
Notice that “looking up directions” is not in the instructions. That is intentional; in fact, do whatever you need to resist
seeking any instructions on how to make the best paper airplane. It will be more fun this way, I promise. If you have a
go-to paper airplane design, also do everything you can to convince yourself to try something new.
1. Using five pieces of paper, create five unique plane designs using only one sheet of paper per design. You can
use the scissors if you want to experiment with the paper proportion ratio.

1. Choose a consistent launch point, and fly all five airplanes. Observe which plane goes the farthest distance.
2. Using the winner of the first round, notice and pay attention to its overall form. Using three more sheets of
paper, create three new designs.
3. From a consistent launch point, fly all four of these airplanes (three new designs plus the winner of round 1).
Observe which plane goes the farthest distance.
4. Using the winner of the second round, notice and pay attention to its overall form. Using the last two sheets of
paper, create two new designs.
5. From a consistent launch point, fly all three of these airplanes (two new designs plus the winner of round 2).
Observe which plane goes the farthest distance.
6. Reflect on what just happened. What did you learn? What did you observe? How did your approach change
through the creation of these paper prototypes? How long did this take overall? That is the beauty of rapid
prototyping.

UNIT-II Immersive Design for 3D

More Related Content

What's hot

Similar to UNIT-II Immersive Design for 3D

More from GOWSIKRAJA PALANISAMY

Recently uploaded

In this document

UNIT-II Immersive Design for 3D