Traning workshop on ‘Designing an conducting user studies” Wrap Up (Kristien Ooms) @ ICC&GIS June 15th, 2016 Albena, Bulgaria

1. Workshop on
Designing & Conducting User Studies
15 June 2016 - Albena, Bulgaria

2. Wrap up of the day
Module 1
Kristien Ooms
University of Ghent, Belgium
Chair, ICA Commission on UUUI
Methods and techniques of use, user and
usability research in geo-information
processing and dissemination
Module 2
Robert Roth
University of Wisconsin Madison, USA
Vice Chair, ICA Commission on UUUI
Quantitative versus Qualitative user
research: selecting the right approach
Module 3
Kristien Ooms
University of Ghent
ICA Commission on Use and User Issues
When and how to apply Eye Tracking?
Module 4
Corné van Elzakker
University of Twente, The Netherlands
Past Chair, ICA Commission on UUUI
An Introduction to Thinking Aloud
2

3. Tips & Tricks …
RESEARCH SET-UP:
Collecting and analyzing use, user & usability research data
• Prepare clear tasks to be executed during the test (based on the
formulation of geographical questions to which users want to find
answers)
• How to derive questions / tasks from research questions?
• Consider the test environment (realistic use context)
• Make test scenarios / protocols / checklists for you / the
researcher to create exactly similar conditions for all test
participants

4. Tips & Tricks …
RESEARCH SET-UP:
Collecting and analyzing use, user & usability research data
• Instruct all test participants in exactly the same way (usually
written instructions)
• Execute pilot tests to test the overall set-up and determine
the test duration
• Analyze the research data keeping your main research
questions in mind

5. Tips & Tricks …
TEST PERSONS:
Collecting use, user & usability data
• Number?
Depends on your research objective.
5-9 TP’s for explorative, qualitative research.
Many more TP’s required for statistical validation.
• Engaged?
Recruit and select TP’s who are problem owners, not TP’s who want
to do you a favour.
Establish TP characteristics (background, etc.)
• Ethical considerations ….

6. Tips & Tricks …
RESEARCH INVOLVING HUMAN SUBJECTS
Take care of your participants
• Provide incentives / do not force
• Ask permission / seek informed consent
• Do not waste subject’s time (test set-up)
• Maintain privacy / confidentiality (no names in reports,
permission for pictures / videos in publications)
• Do not expose subjects to risks /potential harms
• Sound reporting on test persons / methods
• Laws regulating use of human subjects
Partly after Olson, 2009

7. Tips & Tricks…
• Always realize WHY you are doing what you are doing with
geographical information in your studies or future work
• Always know for WHOM you are doing what you are doing
• Never take for granted that the geographic information or
geographic data product or service you have designed or
generated is usable for everyone
• Follow user-centered design approaches and investigate
the usability
7

8. 8
Tips & Tricks…
• Take the time to work out (and document)
every step of the research sequence:
– Formulate clear objectives
– Formulate clear research questions
– Only after that: decide on methods and
techniques of user research with which you will
probably find answers to your research questions
– Execute user research
– Analyze the data and answer your questions
– Validate your research method / technique
8

9. 9
Tips & Tricks…
• Nielsen’s Alertbox
– https://www.nngroup.com/articles/subscribe/
• Amazon Mechanical Turk
– https://www.mturk.com/mturk/welcome
9

10. 10
Tips & Tricks…
• Mixing Methods
• Provide Training Blocks
• Take care of
PARTICIPANTS – MATERIAL – PROCEDURE – ANALYSIS
10

11. Thank you for your attention!
Contact:
kristien.ooms@ugent.be
http://use.icaci.org