The paper introduces the Visual Attention Convergence Index (VRC) for analyzing eye tracking in virtual reality (VR), which measures how users focus on the same or different virtual objects. It presents mathematical tools for calculating both aggregated and individual variations of VRC, enabling insights into the effectiveness of narrative cues in VR experiences. The methodology has potential applications in behavioral psychology, VR design, and research involving augmented reality.

1. Kobylinski P., Pochwatko G. (2020) Visual Attention Convergence Index for Virtual Reality
Experiences. In: Ahram T., Taiar R., Colson S., Choplin A. (eds) Human Interaction
and Emerging Technologies. IHIET 2019. Advances in Intelligent Systems and Computing,
vol 1018. Springer, Cham
Visual Attention
Convergence Index
for Virtual Reality Experiences

2. Grzegorz Pochwatko, PhD
Virtual Reality and Psychophysiology Lab, Institute of Psychology, Polish Academy of Sciences
Warsaw, Poland
Visual Attention
Convergence Index
for Virtual Reality Experiences
Pawel Kobylinski, PhD
Laboratory of Interactive Technologies, National Information Processing Institute
pawel.kobylinski@opi.org.pl IHIET 2019, Nice, France

3. Introduction
(1) The paper introduces a novel quantitative method in the domain of eye tracking (ET)
for virtual reality (VR): visual attention convergence index (denoted as VRC).
(2) The method might be of interest to researchers on the human factor in VR,
behavioral psychologists, and designers of VR experiences.
(3) Research involving augmented reality (AR), standalone, and mobile 2D ET may benefit
from usage of the methodology described in the paper. It is also conceivable to apply
VRC to positional tracking data in virtual, augmented and real-space setups.
(4) VRC is based on recently developed distance variance (dVar),
a function of distances between observations in metric spaces. Szekely & Rizzo 2009

4. Motivation
(1) Users love good narratives. Graphic and cinematic VR environments, to be effective
and/or attractive, should be immersive, create the illusion of spacial and social
presence, and “tell” a good story. Napiorkowski 2018; Danilicheva et al 2009
(2) Effective narration keeps users’ attention focused during a virtual experience, helps
following the storyline and improves performance. Hsu & Lu 2004; Gorini et al 2011
(3) Presence and good narrative lead to motivation, engagement, better memory, and/or
persuasion. Lin et al 2002; Baylor 2009; Tussyadiah et al 2018
(4) Effective narration is required equally in games, training and therapeutic
environments, as well as in cinematic VR experiences.

5. Motivation
(5) Cinematic VR, a new kind of VR experiences, has recently emerged and is growing
in popularity.
(6) Cinematic VR creates new possibilities of applications other than art or entertainment.
Persuasive or training (educational or vocational) cinematic VR experiences are the
best examples.
(7) It also brings new challenges for creators, who need to narrate in a different way to
keep the viewer’s attention in the right place of the scene.

6. Motivation
(8) If a designer of a VR experience employs any system of attentional cues (e.g. spatial
sounds) to guide participants’ attention along an intended narration line, they might
be interested in measuring the system’s effectiveness. Kobylinski, Pochwatko, & Biele 2019
(9) This is why we have devised and introduce in the paper an aggregated version
of the visual attention convergence index (denoted as VRCa).
(10) VRCa measures the effectiveness of narration in VR.
(11) In simple words, VRCa tells us if several participants looked at
the same or rather different virtual objects/areas during a VR experience.

7. Motivation
(12) In the paper we also put stress on individual differences in responses to
VR experiences.
(13) Some users look around a lot to explore the virtual space, whereas
other do not execute visual exploration at all.
(14) In order to measure such differences in the visual exploration, we have devised
an individual version of the visual attention convergence index (denoted as VRCi).

8. Mathematical Method
(1) Let us consider a set of n gaze fixations F = {f1,f2,…,fn} detected from
eye tracking data recorded during a VR experience for a chosen time interval.
(2) F may originate from either a number of participants of the VR experience
(in this case, F is a union of many individual subsets)
or a single participant (in this case, F equals a single subset).
(3) We will discuss both the aggregated and individual variants later on.
(4) Three coordinates xi, yi, and zi are identified for every fixation fi.
Thus, for the chosen time interval of interest, we obtain
a three-dimensional discrete variable with n values.
(5) This variable can be represented as a matrix T with n rows and three columns.

9. Mathematical Method
Distance Variance Coefficient (dVar)
(1) In the first step, T is transformed into matrix of Euclidean distances between rows
treated as three-dimensional vectors of observations
(square matrix D with n rows and n columns).
(2) Let D’ denote a n ︎x n matrix obtained by double-centering the distance matrix D
(i.e. the row and column means of D are subtracted and the grand mean added).
(3) The distance variance dVar is defined as follows: Szekely & Rizzo 2009
! (1)dVar =
1
n
n
∑
i,j=1
(D′ij)2

10. Mathematical Method
Distance Variance Coefficient (dVar)
(4) dVar constitutes the basis for further calculations.
(5) dVar is nonnegative.
(6) dVarmin = 0 represents a situation in which
the spatial coordinates for all fixations fi ∈ F are equal:
! (2)
(7) The more the coordinate vectors differ from each other,
the greater the value of dVar.
(8) dVar can be easily calculated with the usage of the R package energy. Rizzo & Szekely 2018
[
x1
y1
z1
]
=
[
x2
y2
z2
]
= ⋅ ⋅ ⋅ =
xn
yn
zn

11. Mathematical Method
Visual Attention Convergence Index (VRC)
(1) Let VRC denote the most basic form of the visual attention convergence index:
! (3)
(2) VRCmax = 1 represents a situation in which
the spatial coordinates for all fixations fi ∈ F are equal
and implies maximal possible level of the visual attention convergence.
(3) The more the coordinate vectors differ from each other, the smaller the value of VRC.
(4) Though neither VRC = 0 nor VRC < 0 imply any convenient interpretation,
the basic form of the index suffices for experimentally controlled comparisons of
visual attention convergence levels measured with regard to the same virtual reality
environment.
VRC = 1 − dVar

12. Mathematical Method
Scaled Visual Attention Convergence Index (sVRC)
(1) Let sVRC denote a scaled version of the visual attention convergence index:
! (4)
(2) sVRC ∈ [0,1], so interpretation of its values becomes convenient.
(3) sVRCmax = 1 implies maximal possible level of the visual attention convergence.
(4) Depending on the choice between theoretical and empirical value of dVarmax,
sVRCmin = 0 implies either minimal theoretically possible
or minimal empirically observed level of the visual attention convergence.
(5) In the first and preferred case scenario, sVRC can be used to compare
visual attention convergence levels also between virtual reality environments.
sVRC = 1 −
dVar
dVarmax

13. Mathematical Method
Scaled Visual Attention Convergence Index (sVRC)
(6) The basic form of the visual attention convergence index (VRC)
requires double-centering the distance matrix D, which is a way to
deal with the central tendency in the set of distances between the gaze fixations.
(7) This complication is however unnecessary when
we choose to explicitly scale the values of the index into the [0,1]︎ interval.
Thus, in the case of the scaled index, the dVar formula takes the following form:
! (5)dVar =
1
n
n
∑
i,j=1
(Dij)2

14. Mathematical Method
Scaled Visual Attention Convergence Index (sVRC)
(8) As soon as a theoretical value of a maximal possible distance dmax between positions
of gaze fixations in a virtual reality environment of choice is established,
it is possible to calculate the theoretical value of dVarmax, based on Eq. (5):
! (6)
(9) Based on Eqs. (4) and (6), the scaled version of
the visual attention convergence index is calculated as follows:
! (7)
dVarmax =
1
n
(n2
− n)d2
max =
n − 1
n
dmax ≈ dmax
sVRC = 1 −
dVar
dmax

15. Mathematical Method
Scaled Visual Attention Convergence Index (sVRC)
(10) In the case of the cinematic VR, a 360-degree video is displayed on a virtual sphere
(or twin spheres in the case of stereoscopic cinematic experience).
(11) Let r denote the radius of the sphere. Then:
! (8)
(12) Thus, in the case of cinematic VR experiences,
the sVRC formula takes the following form:
! (9)
dVarmax = 2r
sVRC = 1 −
dVar
2r

16. Mathematical Method
Aggregated Visual Attention Convergence Index (VRCa, sVRCa)
(1) For reasonably short time intervals, if the set F contains
fixations originating from more than one participant of a VR experience, VRC conveys
information about similarity in the visual attention distribution across participants.
(2) To simplify the interpretation: a value of the aggregated visual attention convergence
index (let us denote it as VRCa and sVRCa) tells us
if several people looked at the same or rather different virtual objects/areas
during a chosen, short time interval.
(3) VRCa and sVRCa allow us to measure
the effectiveness of any system of attentional cues employed by a designer
to guide the attention of VR experience participants along an intended narration line.

17. Mathematical Method
Aggregated Visual Attention Convergence Index (VRCa, sVRCa)
(4) The figure below originates from an empirical study and illustrates
changes in VRCa calculated for half-second intervals
over 96 participants of the same educational cinematic VR experience.
(5) At the beginning of the 360-degree video subjects clearly differed as to the areas
of interest (AOIs). During the first 30 s of the projection the AOIs steadily converged
and then over the next 30 s, subjects steadily followed the intended narrative.

18. Mathematical Method
Individual Visual Attention Convergence Index (VRCi, sVRCi)
(1) While VRCa tells us something about the narration in a VR experience, we also might
want to capture individual differences in the visual attention convergence.
(2) In order to achieve it, VRC must be calculated separately for each participant
of a VR experience, for any chosen time interval.
(3) Let us denote the individual visual attention convergence index as VCRi
(and sVRCi for the scaled version).

19. Mathematical Method
Individual Visual Attention Convergence Index (VRCi, sVRCi)
(4) The figure below illustrates data originating from the empirical study
already mentioned. For the sake of the example, we calculated VCRi
for six ten-second intervals, for each viewer of the educational 360-degree video.
The figure shows VCRi values averaged for two groups of subjects.
(5) Viewers belonging to the “red group” exhibited less visual exploration (more
visual convergence) during the first 30 s of the educational cinematic VR experience.

20. Conclusion
(1) We believe that the novel method described in the paper might be
of interest to researchers on the human factor in VR, behavioral psychologists,
and – last but not least – designers of VR experiences.
(2) Effectiveness of a system of attentional cues employed in a VR experience,
especially in cinematic VR, can be examined ex post with the usage of
the aggregated version of the visual attention convergence index.
This possibility may help creators of VR content to better understand how to build
cohesive narration.

21. Conclusion
(3) On the other hand, the individual visual attention convergence index
may be employed both in real-time and offline.
(4) Real-time application may be linked to
a feedback system that reacts to lack of visual focus.
(5) Ex post application of VRCi may give behavioral psychologists
insight into individual differences in the way people react to VR experiences.
(6) VRCi can be used in combination with
other psychological (questionnaire) or psychophysiological measures (e.g. heart rate)
to dive into mechanisms of attention, presence, and immersion.

22. Conclusion
(7) The scope of research in which the method may find its applications
is not limited to the ET for VR field.
(8) Research involving augmented reality (AR), standalone, and mobile 2D ET
may benefit from usage of the methodology described in the paper.
It is also conceivable to apply VRC to positional tracking data
in virtual, augmented and real-space setups. Kobylinski, Pochwatko, & Biele 2019

23. Thank you for your (visual) attention
pawel.kobylinski@opi.org.pl
Kobylinski P., Pochwatko G. (2020) Visual Attention Convergence Index for Virtual Reality
Experiences. In: Ahram T., Taiar R., Colson S., Choplin A. (eds) Human Interaction
and Emerging Technologies. IHIET 2019. Advances in Intelligent Systems and Computing,
vol 1018. Springer, Cham
https://link.springer.com/chapter/10.1007/978-3-030-25629-6_48