This document provides an overview of how to plan and execute an ecologically valid NeuroIS study. It discusses key elements to consider such as tasks and scenarios, measures, interfaces, protocols, instruments, recruiting subjects, and compensation. Examples of case studies and insights from NeuroIS research are also presented. The document emphasizes developing a complete test plan that addresses objectives, teams, interfaces, subjects, and other important details.

1. PIERRE-MAJORIQUE LÉGER, Ph.D.
NSERC-Prompt Industrial Research Chair in User Experience
Co-director Tech3Lab, Director ERPsim Lab
June 4 2019, Vienna, Austria.
How to plan and execute
an ecologically
valid NeuroIS study

2. © Copyright Léger (2019)
Pierre-Majorique Léger, Ph.D.
Lab Serious games to learn
enterprise systems and
business analytics
https://www.researchgate.net/profile/Pierre-Majorique_Leger
NSERC-Prompt
Industrial Research Chair
in User Experience
Director ERPsim Lab
Co-direct0r Tech3Lab
pml@hec.ca
GRADUATE STUDIES
POST-DOCTORATE
LABORATORIES FINANCIAL SUPPORT
GUEST PROFESSOR

3. Powered by:
© Copyright Léger et Fredette (2019)

4. Marc Fredette, Ph.D.
Data Sciences
Sylvain Sénécal, Ph.D.
Marketing
Pierre-Majorique Léger, Ph.D.
Information technologies

5. 2018-2019
50Scientific articles
and conference
proceedings
8Postdoc
researchers
5PhD Students
21Master
students
10summer
interns
420human subjects
20research
projects

6. Industrial research partners
from a wide range of industries
INSURANCE
ONLINE GROCERY
MEDIA
RAILWAYS
BANKING
AERONAUTICS
LOGISTICS
FINANCE
r partenaire de
PLUSIEURS BOURSES DE RECHERCHE
À TOUS LES NIVEAUX :
chaire_ux.hec.ca
INSCRIVEZ-VOUS COMME
PARTICIPANTS À NOS ÉTUDES :
panel.hec.ca
Organismes subventionnaires :
partenaire de

7. est fier partenaire de
SEVERAL RESEARCH GRANTS
FOR ALL ACADEMICS LEVELS:
chaire_ux.hec.ca
SUBSCRIBE NOW AS PARTICIPANTS
IN OUR RESEARCH PROJECTS:
panel.hec.ca
Proud Partner of
Powered by: Funding Organisations:

8. Tech3Lab:
A Real UX Playground
183 3000PROJECTS RESEARCH PARTICIPANTS
6YEARS

9. © Copyright Léger (2019)
A Bvs
How to plan
a NeuroIS
study
SCHEDULING
TASKS AND
SCENARIOS
MESURES
INTERFACES PROTOCOL INSTRUMENTS
RECRUITING SUBJECTS COMPENSATION

10. © Copyright Léger (2019)
How dynamic is the interface?
1 2 3 4- +IMAGE WEB STATIQUE MOBILE STATIQUE DYNAMIQUE

11. © Copyright Léger (2019)
Detection of events
AUTOMATIC MARKERS MANUAL MARKERS
WEB DYNAMIC STIMULI = 30 FPS
$ $
$
$

12. © Copyright Léger (2019)
Ecologically valid NeuroIS study
TASK
SUCCESS OR
FAILURE?
SCENARIO
SCREEN 1
EVENT EVENT EVENT EVENT
SCREEN 2 SCREEN 3 SCREEN 4

13. © Copyright Léger (2019)
Complete test plan
TASK 1
TASK 2
...
INTERVIEW
SURVEY
DEBRIEFING
INSTRUMENTS
PROTOCOL
COMPENSATION

14. © Copyright Léger (2019)
RANDO-
MISATION
INSTRUMENTS
Within subject research design
PROTOCOL
TASK A
TASK B

15. © Copyright Léger (2019)
Between subject
research design
INSTRUMENTS
PROTOCOL

16. © Copyright Léger (2019)
What is the research question?
What are the hypothesis?
What is the interface?
On which platform?
How dynamic is the interface?
What are the event markers?
What are the characteristics
of the research subjects?
How many subjects are needed
for the experiment?
What are the time constraints?
Who are the researchers involved
and what are the responsibilities?
OBJECTIVES
TEAM
INTERFACE
SUBJECTS
Questions to ask to define the project

17. © Copyright Léger (2019)
What are the tasks?
What is the scenario?
What are the main subtasks?
What information is required to perform
the task?
How will the subjects be recruited?
What will be the compensation?
When and where will the experiment
take place?
Which version of the interface
will be used ?
Which instruments will be used /
are available ?
Questions to ask to define the test plan
A Bvs
A Bvs
A Bvs
A Bvs
A Bvs
A Bvs
Bvs
SCENARIOS
AND TASKS
MESURES
SCHEDULE
INTERFACESPROTOCOL
INSTRUMENTSRECRUTEMENT
COMPENSATION

18. © Copyright Léger (2019)
Case
studies

19. © Copyright Léger (2019)
Which of these two ads creates
the most optimal emotional reaction?

20. © Copyright Léger (2019)
Presentation study
EMOTIONAL AROUSAL
(Biopac)
VISUAL ATTENTION
(Tobii)
EMOTIONAL VALENCE
(Facereader)
UX HEATMAP
+ ...
5 seconds 5 seconds 5 seconds
Powered by:
or or or
Happy Sad
Calm Excited

21. © Copyright Léger (2019)
Ad #1 Ad #2
Ad Emotional Response: Results and Insights
INSIGHT #1
For more than half the participants,
Ad #1 generated only positive emotion. As
the UX heatmaps show, the face and faded lights
in the background seem to have contributed
to the positive emotional response for this Ad.
INSIGHT #2
Even when participants preferred this Ad,
it generated mostly negative emotions.
The backward-facing person in this Ad
contributes to the negative emotion felt
by participants.
Positive valence
(Facereader)
Negative valence
(Facereader)
UX Heatmap powered by CubeHX

22. © Copyright Léger (2019)
Ecommerce experience (1/2)
EMOTIONAL AROUSAL
(Biopac)
VISUAL ATTENTION
(Tobii)
EMOTIONAL VALENCE
(Facereader)
EMOTIONAL
AND COGNITIVE
CIRCUMPLEX
Powered by:
RANDOMIZED
Checkout
AROUSAL(STANDARDIZEDAROUSAL(STANDARDIZED
PUPIL SIZE
VALENCE (FACEREADER)
Calm Excited Easy Difficult
Happy Sad

23. © Copyright Léger (2019)
Emotional Circumplex
METHODOLOGICAL NOTES
EMOTIONAL
VALENCE
EMOTIONAL INTENSITY
Sad Happy
Calm
Excited
UX No Man Land Excitement
SerenityTolereble
HOW TO READ THIS CIRCUMPLEX
On the vertical axis, we find the emotional intensity
experienced by consumers; the higher they are, the
more emotionally intense is the experience. Horizon-
tally, emotional valence is represented; on the right,
experience generates pleasure, while on the left, the
experience generates displeasure. Finally, the size of
the circle represents the mental workload; a large
circle represents a significant mental effort of con-
sumers.

24. © Copyright Léger (2019)
WALMART, THE LAGGER.
Low prices is not sufficient for a bet-
ter experience. Walmart is a generalist
and must find a way to compete against
websites that can afford to carve a
unique experience for users. A microsite
can be a way to compete on the emo-
tional front. Walmart has successfully
used this strategy in the past for niche
products.
INSIGHT #1
Experience on Walmart is as emotionally engaging on these two pages. EBGames and BestBuy
are able to carry over to the search page the excitement generated on the homepage.
EXAMPLE 2
Homepage Product Search
VALENCE (FACEREADER)VALENCE (FACEREADER)
AROUSAL(STANDARDIZED
VALENCE (FACEREADER)VALENCE (FACEREADER)
AROUSAL(STANDARDIZED
Product pages: Insights & Recommendations (1/2)
AROUSAL(STANDARDIZEDEDA)

25. © Copyright Léger (2019)
Product Specs Checkout
AROUSAL(STANDARDIZED
AROUSAL(STANDARDIZED
AROUSAL(STANDARDIZED
AROUSAL(STANDARDIZED
PUPIL SIZE PUPIL SIZE
THE ULTIMATE STEP IN
THE CONVERSION FUNNEL:
THE PRODUCT PAGE
The product page is the last page
before consumers make a purchase.
The experience on EBGames appears
to be the gold standard. It performs
significantly better than Walmart and
BestBuy. By having the best product
page the EBGames website, is making
the most of their efforts to convert
consumers to paying customers on
their website.
Product pages: Insights & Recommendations (2/2)
INSIGHT #2
EBGames product page generates emotions that are positive
and intense, while maintaining a low mental workload.
VALENCE (FACEREADER)
VALENCE (FACEREADER)VALENCE (FACEREADER)
EXAMPLE 2

26. © Copyright Léger (2019)
Journey maps
METHODOLOGICAL NOTES
1 2 3 ...
+
-
ACTIVATION:
(StandardizedEDA-Biopac)
Time
STEPS
Friction Point
EMOTIONAL
VALENCE
(Facereader)
+ -
HOW TO READ THIS JOURNEY MAP
Colors of tasks are presented on the right, the red
squares representing moments of physiological pain
points along the journey. On the vertical axis, we find
the emotional intensity experienced by consumers;
the higher they are, the more emotionally intense is
the experience. Horizontally, the time is presented in
seconds. The size of the circle represents the emotional
valence; a large circle represents negative emotions.
In other words, pain points are high and large dots
on the journey map!

27. © Copyright Léger (2019)
Ecommerce experience (2/2)
EMOTIONAL AROUSAL
(Biopac)
VISUAL ATTENTION
(Tobii)
EMOTIONAL VALENCE
(Facereader)
PAIN POINT IDENTIFICATIONS
Happy Sad
Powered by:
Calm Excited Easy Difficult
VALENCE (FACEREADER)
INSIGHT #1
Even after filtering for the price, consumers cannot find
the product he is looking for, which creates frustration.
INSIGHT #2
The total order amount does not adjust when an item
is removed from the cart, creating confusion.