The body of knowledge accumulated by gesture elicitation studies (GES), although useful, large, and extensive, is also heterogeneous, scattered in the scientific literature across different venues and fields of research, and difficult to generalize to other contexts of use represented by different gesture types, sensing devices, applications, and user categories. To address such aspects, we introduce RepliGES, a conceptual space that supports (1) replications of gesture elicitation studies to confirm, extend, and complete previous findings, (2) reuse of previously elicited gesture sets to enable new discoveries, and (3) extension and generalization of previous findings with new methods of analysis and for new user populations towards consolidated knowledge of user-defined gestures. Based on RepliGES, we introduce GEStory, an interactive design space and visual tool, to structure, visualize and identify user-defined gestures from a number of 216 published gesture elicitation studies

1. new data
same
method
new
method
same participants new participants
Vatavu &Wobbrock
(2015, 2016),
Tsandilas (2018)
5
Nacenta et al.
(2013), Schipor &
Vatavu (2019)
3
6
Nebeling et al.
(2014), Hoff et al.
(2016)
n/a
(and unlikely)
4
Repeatability
Reproducibility
Replicability
not yet done
new data
1
2
by the virtue of the
method itself
same data
Repurposability
10
9
Gheran et al. (2018)b,
Soni et al. (2020)
same data
REPL
ICA
TE REPURPOSE
new population
same population
checking of analysis
reanalysis of data
EX
TEND&GENERA
L
IZ
E
new goal
new
population
conceptual extension
exact replication,
same method
Replicability
Generalizability
Extensibility
Leiva et al.
(2017)
exact replication,
new method
same
population
exact replication,
same method
new data
7
8
Rust et al. (2014)
empirical
generalization
generalization and
extension
repurposing and
extension
empirical
repurposing
Soni et al. (2019)
RepliGES and GEStory: Visual Tools for Systematizing and
Consolidating Knowledge on User-Defined Gestures

2. RepliGES and GEStory: Visual Tools for Systematizing and
Consolidating Knowledge on User-Defined Gestures
Jean Vanderdonckt
Université catholique
de Louvain (UCLouvain),
Belgium
Radu-Daniel Vatavu
Machine Intelligence &
Information Visualization Lab
University of Suceava, Romania
Santiago Villarreal-Narvaez
Université catholique
de Louvain (UCLouvain),
Belgium
Bogdan-Florin Gheran
Machine Intelligence &
Information Visualization Lab
University of Suceava,
Romania

3. Introduction
A Critical aspect of designing gesture user interfaces
• Mapping between gestures
• System functions
Scientific knowledge
I. Findings from experiments examining the factors that impact the user experience
of gesture-based interaction (social acceptability, memorability, perceived
difficulty, and naturalness)
II. Insights into users' preferences for gestures elicited in gesture elicitation studies.

4. Introduction
Gesture Elicitation Studies (GESs).
• A powerful method in the toolbox of HCI researchers and practitioners to unveil
users' preferences for and mental models of gesture-based interaction
• Participants propose gestures they would like to use to effect the system functions of
interactive systems
Zooming in on a map with touch input
Turning on/off the TV set with a smart ring
• The practitioner analyzes the elicited gestures to develop an understanding of users'
preferences for gesture commands that are perceived:
Intuitive
Low effort
Memorable
• Consequently, likely to reflect the preferences of the larger user population

5. Introduction
Gesture Elicitation Studies (GESs).
• Several shortcomings about the body of knowledge accumulated on user-defined
gestures in the scientific literature:
S1. Scattered literature: GESs are published at various venues
S2. Heterogeneous findings: GESs employ different methods of analysis
S3. Partial coverage of context: Some GESs lack a full description of the context of use
S4. Partial evidence: Some GESs do not provide any indication whether the user-defined gestures
they report are representative
S5. Difficulty of cross-checking findings: Findings from different GESs are difficult to structure and
put in correspondence

6. Introduction
Contributions of this paper
1) We formalize replication, extension, generalization, and repurposing of
GES findings with RepliGES
2) Based on RepliGES, we introduced GEStory, an interactive design space
meant to structure, visualize, and identify user-defined gestures (216
published GESs)
3) Using GEStory, we identify a set of twelve GES replications in the scientific
literature, which we characterize using the dimensions of our RepliGES
conceptual space
4) We propose a set of replicability criteria to foster replications, extensions,
generalizations, and repurposing of findings about user-defined gestures

7. RepliGES: A conceptual space
RepliGES specifies five types of replications, three types of extensions and
generalizations, and two types of repurposing studies that reuse previous GES
findings for new discoveries
new data
same
method
new
method
same participants new participants
Vatavu &Wobbrock
(2015, 2016),
Tsandilas (2018)
5
Nacenta et al.
(2013), Schipor &
Vatavu (2019)
3
6
Nebeling et al.
(2014), Hoff et al.
(2016)
n/a
(and unlikely)
4
Repeatability
Reproducibility
Replicability
not yet done
new data
1
2
by the virtue of the
method itself
same data
Repurposability
10
9
Gheran et al. (2018)b,
Soni et al. (2020)
same data
REPL
ICA
TE REPURPOSE
new population
same population
checking of analysis
reanalysis of data
EX
TEND&GENERA
L
IZ
E
new goal
new
population
conceptual extension
exact replication,
same method
Replicability
Generalizability
Extensibility
Leiva et al.
(2017)
exact replication,
new method
same
population
exact replication,
same method
new data
7
8
Rust et al. (2014)
empirical
generalization
generalization and
extension
repurposing and
extension
empirical
repurposing
Soni et al. (2019)

8. RepliGES
Method
reproducibility
Research
question
Gesture
elicitation
experiment setup
Run
experiment
Data
annotation &
preprocessing
Data
analysis
K
nowledge
Initial observation,
prior work,
application domain
Gesture input
reflective of
user behavior
Results
reproducibility
Inferential
reproducibility
Results Interpretation
Design
considerations and
guidelines
What is the purpose of the
replication experiment?
(Goodman et al., 2016)
Referents,
apparatus, task,
hypotheses
Participants
are elicited for
gestures
Raw gesture
representations
Agreement
calculation
Gesture set,
empirical data
Prerequisite:
Research
question
specified
Experiment
design completely
specified
Participants
sampling
specified
Gesture
clustering
criteria specified
Agreement
measures
specified
Gesture set
documented
and available
Interpretation
method
specified
Replicability criteria for GES studies
paper, report, documentation
software, source code, scripts
data (raw/annotated/processed)
Full
replicability
No
replicability
1 2 3 4 5 6 7 8
I
II
III
A B C D E F
Steps
of GES

9. RepliGES
new data
same
method
new
method
same participants new participants
Vatavu & Wobbrock
(2015, 2016),
Tsandilas (2018)
5
Nacenta et al.
(2013), Schipor &
Vatavu (2019)
3
6
Nebeling et al.
(2014), Hoff et al.
(2016)
n/a
(and unlikely)
4
Repeatability
Reproducibility
Replicability
not yet done
new data
1
2
by the virtue of the
method itself
same data
Repurposability
10
9
Gheran et al. (2018)b,
Soni et al. (2020)
same data
REPLICATE REPURPOSE
new population
same population
checking of analysis
reanalysis of data
EXTEND & GENERALIZE
new goal
new
population
conceptual extension
exact replication,
same method
Replicability
Generalizability
Extensibility
Leiva et al.
(2017)
exact replication,
new method
same
population
exact replication,
same method
new data
7
8
Rust et al. (2014)
empirical
generalization
generalization and
extension
repurposing and
extension
empirical
repurposing
Soni et al. (2019)

10. GEStory: An interactive design space
To support the visualization of gesture-related information, dynamic query
filters are tightly coupled with a starfield display of the human body
https://sites.uclouvain.be/GEStory

11. GEStory:
Domain
model

12. GEStory - 1

13. GEStory - 2

14. GEStory – Replication, Generalization, Repurposing

15. A Scientific Literature Review of GES replications
Gher
an et al. (2018) Gher
an et al. (2018)b
9
Ts
andil
as(2018)
L
aheyet al. (2011), Bail
l
yet al. (2013),
Chan et al. (2016), Weigelet al. (2014) 2
Vatav
u &Wobbr
ock
(2016)
Vatav
u (2013), P
ium som boon et al. (2013),
Bail
l
yet al. (2013), Vatav
u and Zaiț
i(2014) 2
Nacentaet al. (2013) 3 3
Schipor&
Vatav
u (2018)
Vatav
u &Wobbr
ock
(2015)
Vatav
u (2012), P
ium s
om boon et al. (2013),
Bail
l
yet al. (2013), Vatav
u and Zaiț
i(2014) 2
REPL
ICA
TION
Wobbr
ocket al. (2009) Rus
tet al. (2014)
7
Soniet al. (2019) Soniet al. (2020)
9
GENERA
L
IZA
TION
Vatav
u (2017) L
eiv
aet al. (2017)
10
Mor
ris(2012) Nebel
ing et al. (2014)
6
Morr
iset al. (2014) Hof
fet al. (2016)
6
Rus
tet al. (2014), Wobbr
ocket al. (2009) Soniet al. (2019)
8
REPURPOSING

16. Conclusion
• We introduced the RepliGES conceptual space to address several
shortcomings (S1-S5) about the body of knowledge accumulated on user-
defined gestures in the scientific literature.
• Based on RepliGES, we developed GEStory, an interactive design space to
structure, visualize, and query user-defined gestures reported by 216 GESs,
which we demonstrated by identifying a set of replicated GESs.
• Our results have implications for the practice of end-user gesture elicitation,
in support of which we provide replicability criteria and recommendations
for researchers and practitioners

17. Conclusion
• With these tools, the body of knowledge on user-defined gestures is
concentrated in one single point of contact
• Gestures are systematically specified with a unified format in a consistent manner
• It is structured according to a domain model to identify RepliGES links between
different published GESs.

18. Thank you very much
for your attention