ICC2017 Washington - http://icc2017.org/ 5104.3 Researching the usability of a GeoVisual Analytics environment for the exploration and analysis of different data Irma Kveladze Aalborg University Menno-Jan Kraak University of Twente Corné P.J.M. Van Elzakker University of Twente

1. Irma Kveladze, Menno-Jan Kraak and Corne P.J.M. van Elzakker
Session: Evaluating Map Quality
Washington DC, USA 2 – 7 July 2017
The Usability of a GeoVisaul Analytics Environment for the
Exploration and Analysis of Different Datasets

2. July 2 – 7 2017 | Washington DC USA
Background
Framework
Experiment
Results
GeoVisual Environments
Kapler, et al. (2008) Andrienko and Andrienko (2004)
Ferreira, et all. (2013)
http://www.digitalattackmap.com/#anim=1&color=0&country=ALL&list=0&time=16692&view=map Nguyen and Schumann, 2011

3. July 2 – 7 2017 | Washington DC USA
Background
Framework
Experiment
Results
Usability perspective
Çöltekin, et al. (2009, 2010)
What do we know?
National Atlas of the USA Carto.net
Compare two interactive online map interfaces

4. GVA environments are often designed to solve only one particular
problem, however, the question is whether the same environment can
be suitable to tackle similar problems with different datasets?
The main objective of this research is to understand:
• How well GVA tools within this environment are utilized by the
test participants to execute similar tasks with different use
cases / datasets
• If so, what makes them useful
• if not, what are the issues to be considered
Background
Framework
Experiment
Results
Questions remain
July 2 – 7 2017 | Washington DC USA

5. Simple dataset
Napoleons Russian Campaign
Complex annotations
Travel log of a trip in Estonia
Background
Framework
Experiment
Results
Use case studies
July 2 – 7 2017 | Washington DC USA

6. GeoVisual analytics environment
July 2 – 7 2017 | Washington DC USA
Background
Framework
Experiment
Results

7. GeoVisual analytics environment
July 2 – 7 2017 | Washington DC USA
Background
Framework
Experiment
Results

8. Background
Framework
Experiment
Results
User task
July 2 – 7 2017 | Washington DC USA
TASK 1 Please describe how long different corps were traveling to Moscow and back?
EXPECTED TASK
EXECUTION
Participants should locate the 2D base map of the STC at the point of intersection between Moscow and the
trajectories to find the time of arrival. Then continue to move the 2D base map up to the time axis of the STC to
identify the time of departure from Moscow and the end of the campaign.
TASK 2 Please look at the STC and TG and compare the difference between the beginning and the end of the campaign.
Characterize the factors / reasons that influence the strength of the Grande Armée.
EXPECTED TASK
EXECUTION
The participants are expected to compare the number of soldiers at the beginning and at the end of the campaign
through the TW and STC. Then they should look at information on temperature on the TG to characterize change.
TASK 1 Please describe how long the traveler stayed at different places? What do you think was the purpose of the stay /
visit?
EXPECTED TASK
EXECUTION
Expected participant behaviour is to locate the base map at the places of the long stops (stations) to identify their
temporal duration. Then focus on TW, TG and the 2D map to characterize the purpose of the stay.
TASK 2 Please observe the types of annotations and find the area / place where the most diverse materials / information
were collected?
EXPECTED TASK
EXECUTION
Participants should compare different annotations in the STC to find the place with the most diverse information.
Napoleon’s march to Moscow
Travel log data of Estonia

9. Background
Framework
Experiment
Results
Experiment Design
July 2 – 7 2017 | Washington DC USA
• Test participants
• Materials
• Setup
• Procedure

10. User test environment
July 2 – 7 2017 | Washington DC USA
Qualitative
• Think aloud
• Video / audio recording
• Screen recording
• User interview
Quantitative
• Questionnaire
• Task execution
• Eye tracking
Background
Framework
Experiment
Results
Evaluation methods

11. Task performance
July 2 – 7 2017 | Washington DC USA
Background
Framework
Experiment
Results
0 1 2 3 4 5 6 7
locate
identify
characterize
compare
Number of participants completed tasks
Tasks
Task Effectiveness
Complex Annotations Simple datasets

12. Task performance
July 2 – 7 2017 | Washington DC USA
Background
Framework
Experiment
Results
Identify time
0
50
100
150
200
250
Simple datasets Complex annotations
Seconds
Locate
0
50
100
150
200
250
Simple datasets Complex annotations
Seconds
Identify
Locate the map

13. Different use of the GVA environment
July 2 – 7 2017 | Washington DC USA
Background
Framework
Experiment
Results
Locate the map

14. Task performance
July 2 – 7 2017 | Washington DC USA
Background
Framework
Experiment
Results
Characterize changeCompare difference
0
50
100
150
200
250
Simple datasets Complex annotations
Seconds
Compare
0
50
100
150
200
250
Simple datasets Complex annotations
Seconds
Characterize

15. Background
Framework
Experiment
Results
Use of the visual representations
July 2 – 7 2017 | Washington DC USA
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
2D Map
3D STC
TG
TW
2D Map
3D STC
TG
TW
SimpledatasetsComplexannotations
Task completed
Percentage of use of a particular visual representation in completing each of the four tasks
Locate Identify Compare Characterize

16. Background
Framework
Experiment
Results
Conclusions
July 2 – 7 2017 | Washington DC USA
• The coordination between views appeared to be very useful for
exploration.
• Representations implemented for one dataset are not completely useful
for other datasets.
• The participants could use tools and manipulate the GVA environment
better at every new task due to the learning effect during the experiments.
• Some usability aspects and research questions on efficiency metrics with
a bigger group of users would still need to be done.
• We suggest and encourage others to execute evaluation studies to better
understand the complex user-interface aspects of GVA environments.

17. Thank you!