1.
THE EFFECTS OF VISUAL REALISM ON SEARCH
TASKS IN MIXED REALITY SIMULATION
Un d e r t h e g u i d a n c e o f RAJI R. PILLAI (Asst. Professor)
Yadhu Kiran
Roll No. 63
Seminar on :
1

2.
INTRODUCTION
• Experiments in AR Domain are difficult.
- Difference in Display Systems used.
(Result are not generalizable)
- Difficult to conduct Controlled Experiments
in Unpredictable Environment Conditions.
2

3.
DISPLAY SYSTEMS
Head Mounted
Binocular Omni-
Orientation Motor
Data Glove Display ( H. M. D. )
Cave
Automatic
Virtual
Environment
( CAVE )
3

4.
SOLUTION
Hint : Real Display systems can be used only in Real
Environment.
Hence use, Mixed Reality Simulations.
• A concept of using High Fidelity Virtual Reality systems to
simulate environment from MR Continuum.
• Different display systems are simulated by varying the level of
immersion.
4

5.
FIDELITY OF MR SYSTEMS
Display Interaction Simulation
Fidelity Fidelity Fidelity
(Depends on (Depends on (Depends on
Sensory Fidelity) Interactions) Environment &
Fidelity of Objects)
5

6.
VISUAL REALISM
• The degree to which images on simulated environment is
perceived by user in the real world.
• The concept of visual realism is very important for
understanding and verifying Fidelity of Simulations.
• In this paper, the aim is to conduct experiments in
different Simulated Environments rather than Real World
Environment by varying the level of Visual Realism in
each Simulation.
6

7.
What level of Visual Realism is required for
replicating AR Experiments?
• Search Experiments were conducted and
performance was analyzed in 3 MR Simulations
and Real World.
Level 1 Level 2 Level 3 Real World
Increasing Order of Visual Realism
7

8.
VISUAL REALISM FACTORS
• Shadow Softness
• Surface Softness
• Lighting
• Geometry
Analyzing these, it was
found that Soft Shadows
and Surface Type affects
perception of Visual
Realism.
8

9.
CREATING VIRTUAL MODELS
• Different formats and techniques are used for
creating Virtual Models, each resulting in different
levels of Visual Realism.
Model Geometry Color Information Lighting Techniques
• Image Based
• Point Based
• Stored Polygonal
Data
• Simple Colors
• Coloring Material
Properties
• Textures
• Ray Casting
• Ray Tracing
9

10.
THE EXPERIMENT
There were 2 goals to be achieved :
1. Verify the validity of MR Simulations by checking
whether Visual Realism has any impact on Search
Task Performance. Or Simply, Does Visual Realism
impacts results?
2. If so, How?
10

11.
Steps :
• Design outdoor AR experiment involving
common tasks.
• Choose a Real World location for
experiment.
• Use 3 Models and Simulate them for
conduction experiment indoors.
11

12.
TASK & ENVIRONMENT
 Virtual Models were designed using 3D Modeling Techniques.
Low Fidelity Medium Fidelity High Fidelity
• Polygons for major
scenes only.
• Neglected minor
features.
• Plain color was
used but no
textures.
• Increased number
of polygons for
Geometry.
• Simple low
resolution Textures.
• No change in
lighting.
• High polygon
count.
• High resolution
images instead of
textures.
• Baked Lighting.
12

13.
THE TASK
• To search for both ‘Virtual’ and ‘Physical’
information based on both ‘Virtual’ and ‘Physical’
Information Criteria.
( The most recognizable AR Experiments which
require very little training. )
They created Virtual Information on the scene such that
it must be suitable for the current environment where a
particular user might want to do it.
13

14.
The high-fidelity model and the real environment, both with virtual annotations.
Miscellaneous Information
Classroom
Information about Office
Laboratory
14

15.
The Method :
• Created 16 questions.
• There were mix of questions that required the user to find
both virtual information and physical information based
on both virtual and physical information context.
• For each task question the participant was required to
find certain pieces of information and verbally report that
information.
Target information refers to the exact nature of the response
required by the task question.
Physical information is inherent in the real world while virtual
information needs to be provided by the virtual icons and the text
they contain.
Criteria information refers to what information the user is using to
search the scene
15

16.
DISPLAY SYSTEM USED
NVis SX111 HMD
• Pointgrey USB3 Flea camera
• 102° Horizontal FOV
• 64° Vertical FOV
• 1600 x 1024 pixels resolution
• 60 FPS
+ InterSense IS900 Tracking System.
16

17.
Additional Hardwares and Softwares used :
• Windows 7 PC with a Quadro 5600
• Intel Core2 2.4 GHz Duo-Core CPU
• 2 GB Memory
• WorldViz’s VR Toolkit Vizard 4.0
17

18.
EXPERIMENT DESIGN
Dependent Variable for each task was the Search Time.
The 3-Time Question Sessions :
• Q1 – Q5 : Single item search
• Q6 – Q11 : Multiple item Search
• Q12 – Q16 : Comprehensive
Avoided ‘Learning Effect’ by giving 2 Minute breaks after each session.
18

19.
THE PROCEDURE
• Color Vision Test for Participants.
• Training with Icons using HMD.
• Screening Participants.
19

20.
THE QUESTIONS :
20

21.
ISSUES WITH REPLICATING AN
OUTDOOR ENVIRONMENT
• Accuracy of Models
• Real Location undergone changes.
• Geometrical difference between CAD plan and Real
Buildings.
• Weather change affected lighting.
21

22.
RESULTS
1. Difference in Task Times :
• For the analysis of the time results from our experiment,
ANOVA test was used to determine if the level of realism had
a significant effect on task time.
• In the second stage a post-hoc analysis of all task pairs was
performed.
22

23.
Result of ANOVA :
A plot of the mean task times, 95 % confidence intervals, and significance P
values for each level of realism for all 16 task questions
23

24.
P - Value :
• These result were reported as P-Values.
• P-Value is a measure how likely this spot will be
obtained if no real difference existed. A smaller P-Value
indicates more significant is the difference between
groups. (Small usually means 0.05).
Only four of the 16 task questions revealed a significant difference
between the realism conditions.
 Q3 (a door under bridge)
 Q6 (a tree) For these tasks, A Tukey post-hoc
analysis was used to determine
 Q8 (trash bin) pairwise difference.
 Q12 (a comprehensive search)
24

25.
A Tukey
post-hoc
analysis of
Q3, Q6, Q8
and Q12.
Pair-wise
significant
differences
between
each level
of visual
realism are
highlighted
in red
25

26.
Observations from Tukey Post-hoc test :
• For Q3, Q6, Q8 the difference in significant only in Real
Conditions.
• Camera artifacts, vegetation, and lighting conditions,
affected the visibility of the real-world objects much
greater than the virtual objects.
• Cluttered environment might have increased task time
when the search criteria was relatively small.
26

27.
2. Equivalence in Task Times :
• All other tasks did not reveal significant difference for level of
realism.
• For further investigation, the Two-one-sided t-test (TOST) analysis
was used to look for equivalent task time performance within the
groups.
• All other task questions produced at least one instance where
two of the realism conditions were equivalent except Q12 and
Q13.
What type of courses are generally
taught in North Hall ?
Which professor has the office which is
located the furthest from their lab ?
Comprehensive
27

28.
3. Notable Task Questions :
• Q8 - the only task with results indicating any obvious
performance trend with respect to task time.
• Q12 - by far the most difficult one. the results here
reflect the difficulty of the task and the effects from
strategy more than from level of realism.
28

29.
4. Task type and Visual Realism :
• Participants were quicker to respond when the
search target type was virtual information.
• Search criteria type is shown to be significant in all
four conditions while search target type has a
similar effect on the high and the real conditions.
This suggests that the high level of realism performs
similarly to the real conditions.
• The similarity of these effects support the validity of
MR Simulation.
29

30.
FUTURE WORK
• Clarify the exact cause of the differences between the AR
case and the simulated AR cases.
• Improving the actual AR experience for better ways to present
AR imagery to users.
• Exploring Mixed Reality Simulation for different task types (not
just search, but also browsing and annotation and interaction
tasks) and for different training environments to simulate.
30

31.
REFERENCES
• D. A. Bowman, C. North, J. Chen, N. F. Polys, P. S. Pyla, and U. Yilmaz.
Information-rich virtual environments: theory, tools, and research
agenda. In Proceedings of the ACM symposium on Virtual reality
software and technology, VRST ’03, pages 81–90, New York, NY, USA,
2003. ACM.
• M. Elhelw, M. Nicolaou, A. Chung, G.-Z. Yang, and M. S. Atkins. A gaze-based
study for investigating the perception of visual realism in
simulated scenes. ACM Trans. Appl. Percept., 5(1):3:1–3:20, Jan. 2008.
• J. L. Gabbard, J. E. Swan, II, and D. Hix. The effects of text drawing
styles, background textures, and natural lighting on text legibility in
outdoor augmented reality. Presence: Teleoper. Virtual Environ.,
15(1):16–32, 2006.
31