The document describes a replication study that measured the user experience of different adaptive user interfaces using EEG brain activity data. The study aimed to analyze 20 different adaptive graphical menus to compare their influence on cognitive load, engagement, memorization, and attraction. 40 master's students participated in the within-subjects study, using each of the 20 menus in random order. EEG data was analyzed to measure the dependent variables, and questionnaires assessed perceived cognitive load and attraction. Statistical tests found significant differences between menus for all dependent variables, suggesting menu type influences user experience metrics.
Cognitive Science Perspective on User eXperience!Hamed Abdi
How can we use research and knowledge about the brain, the visual system, memory and Emotion to design more effective products, services and systems?
In this presentation i explain how we can synthesize two different disciplinary to reach to more effectiveness in developing services, products and systems.
Concept-Level Design Analytics for Blended CoursesLaia Albó
Paper presentation at ECTEL conference 2019 (http://www.ec-tel.eu/index.php?id=918).
Paper citation and link:
Albó, L., Barria-Pineda, J., Brusilovsky, P., Hernández-Leo, D. (2019). Concept-level design analytics for blended courses. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, & J. Schneider (Eds.), Transforming Learning with Meaningful Technologies. EC-TEL 2019. Lecture Notes in Computer Science (Vol. 11722, pp. 541–554). Delft, The Netherlands: Springer, Cham.
https://link.springer.com/chapter/10.1007/978-3-030-29736-7_40
Rubric-based Assessment of Programming Thinking Skills and Comparative Evalua...Hironori Washizaki
Hironori Washizaki, "Rubric-based Assessment of Programming Thinking Skills and Comparative Evaluation of Introductory Programming Environments," 4th International Annual Meeting on STEM Education (IAMSTEM 2021), Keynote, August 12-14, 2021, Keelung, Taiwan and Online
Cognitive Science Perspective on User eXperience!Hamed Abdi
How can we use research and knowledge about the brain, the visual system, memory and Emotion to design more effective products, services and systems?
In this presentation i explain how we can synthesize two different disciplinary to reach to more effectiveness in developing services, products and systems.
Concept-Level Design Analytics for Blended CoursesLaia Albó
Paper presentation at ECTEL conference 2019 (http://www.ec-tel.eu/index.php?id=918).
Paper citation and link:
Albó, L., Barria-Pineda, J., Brusilovsky, P., Hernández-Leo, D. (2019). Concept-level design analytics for blended courses. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, & J. Schneider (Eds.), Transforming Learning with Meaningful Technologies. EC-TEL 2019. Lecture Notes in Computer Science (Vol. 11722, pp. 541–554). Delft, The Netherlands: Springer, Cham.
https://link.springer.com/chapter/10.1007/978-3-030-29736-7_40
Rubric-based Assessment of Programming Thinking Skills and Comparative Evalua...Hironori Washizaki
Hironori Washizaki, "Rubric-based Assessment of Programming Thinking Skills and Comparative Evaluation of Introductory Programming Environments," 4th International Annual Meeting on STEM Education (IAMSTEM 2021), Keynote, August 12-14, 2021, Keelung, Taiwan and Online
Empirical user studies in Semantic Web contextsCatia Pesquita
My presentation at EKAW 2018 for our position paper that argues better user studies and their reporting are needed in the Semantic Web community, and proposes a framework to design and report empirical studies.
Read the paper at: http://steffen-lohmann.de/publications/2018_EKAW_user_studies_semweb.pdf
This proposal of work contains details and samples of the user centric design process I follow. I have been trying to find a good graph that represents the process, but at the end I have decided to make my own! ;)
CoMo Game Dev - usability and user experience methods Isa Jahnke
This month we'll be hearing from Dr. Isa Jahnke, Director of the User Experience and Learning Design Lab at the University of Missouri. She will be discussing usability, user experience, and how to get the most out of focus groups and testing nights for the CoMO game development community.
These slides cover various aspects of testing in the context of computer science and human-computer interaction (HCI). It includes topics like evaluation techniques, empirical methods in HCI, controlled experiments, and various testing approaches such as A/B testing, cognitive walkthrough, heuristic evaluation, and review-based evaluation. Additionally, it touches on issues related to internal and external validity, reliability in testing, and different experimental designs like between-subjects and within-subjects experiments. This material appears to be educational and is likely used in a course related to computer science or HCI.
All content within this presentation is the property of Royal Holloway, University of London. Unauthorized use, duplication, or distribution of the materials contained herein is strictly prohibited.
An Independent Study Comparing SPSS to Intellectus Statistics: Preliminary ...Statistics Solutions
At both the undergraduate and graduate levels, students struggle conducting and interpreting statistical analyses. The proliferation of professional doctorates, with less emphasis on research and quantitative analyses, have exacerbated students’ data analytic skill challenges.
Presents an introduction to some basic metrics for usability and some current trends in UX evaluation methods. Includes some indicative examples from UX evaluation studies conducted by the author
This presentation was provided by Gerald Benoit of Simmons College during the NISO webinar, Enabling Discovery and Retrieval of Non-Traditional and Granular Content, held on June 7, 2017
The purpose of this research proposal is to identify organizational principles for the development of online learning curriculum in higher education. This study will address the following research questions: Can educational psychology learning theories (such as cognitive load theory) be used to inform usability-testing methods? Can usability-testing methods be used to discover basic principles of online learning curricular organization? Are there basic principles of online learning curricular organization that can improve the efficiency, effectiveness, and user satisfaction of learning in online environments? While there are many theoretical directions one could take to examine the interface of instructional design and technology, this research proposal will use the lens of the cognitive load theory. This study will use the cognitive walkthrough method as established by usability testing standards. Cognitive walkthroughs use an explicitly detailed procedure to simulate a user’s problem solving process at each step through the dialogue, checking if the simulated user’s goals and memory content can be assumed to lead to the next correct action. Participants will be asked to complete a series of tasks in an online learning environment formulated to compare different methods of organization. This study has the potential to make significant contributions to the field of educational psychology and online education by providing substantive empirical data that sheds light on potential principles that improve the effectiveness, efficiency, and user satisfaction of Web-based education.
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)MdTanvirMahtab2
This presentation is about the working procedure of Shahjalal Fertilizer Company Limited (SFCL). A Govt. owned Company of Bangladesh Chemical Industries Corporation under Ministry of Industries.
Empirical user studies in Semantic Web contextsCatia Pesquita
My presentation at EKAW 2018 for our position paper that argues better user studies and their reporting are needed in the Semantic Web community, and proposes a framework to design and report empirical studies.
Read the paper at: http://steffen-lohmann.de/publications/2018_EKAW_user_studies_semweb.pdf
This proposal of work contains details and samples of the user centric design process I follow. I have been trying to find a good graph that represents the process, but at the end I have decided to make my own! ;)
CoMo Game Dev - usability and user experience methods Isa Jahnke
This month we'll be hearing from Dr. Isa Jahnke, Director of the User Experience and Learning Design Lab at the University of Missouri. She will be discussing usability, user experience, and how to get the most out of focus groups and testing nights for the CoMO game development community.
These slides cover various aspects of testing in the context of computer science and human-computer interaction (HCI). It includes topics like evaluation techniques, empirical methods in HCI, controlled experiments, and various testing approaches such as A/B testing, cognitive walkthrough, heuristic evaluation, and review-based evaluation. Additionally, it touches on issues related to internal and external validity, reliability in testing, and different experimental designs like between-subjects and within-subjects experiments. This material appears to be educational and is likely used in a course related to computer science or HCI.
All content within this presentation is the property of Royal Holloway, University of London. Unauthorized use, duplication, or distribution of the materials contained herein is strictly prohibited.
An Independent Study Comparing SPSS to Intellectus Statistics: Preliminary ...Statistics Solutions
At both the undergraduate and graduate levels, students struggle conducting and interpreting statistical analyses. The proliferation of professional doctorates, with less emphasis on research and quantitative analyses, have exacerbated students’ data analytic skill challenges.
Presents an introduction to some basic metrics for usability and some current trends in UX evaluation methods. Includes some indicative examples from UX evaluation studies conducted by the author
This presentation was provided by Gerald Benoit of Simmons College during the NISO webinar, Enabling Discovery and Retrieval of Non-Traditional and Granular Content, held on June 7, 2017
The purpose of this research proposal is to identify organizational principles for the development of online learning curriculum in higher education. This study will address the following research questions: Can educational psychology learning theories (such as cognitive load theory) be used to inform usability-testing methods? Can usability-testing methods be used to discover basic principles of online learning curricular organization? Are there basic principles of online learning curricular organization that can improve the efficiency, effectiveness, and user satisfaction of learning in online environments? While there are many theoretical directions one could take to examine the interface of instructional design and technology, this research proposal will use the lens of the cognitive load theory. This study will use the cognitive walkthrough method as established by usability testing standards. Cognitive walkthroughs use an explicitly detailed procedure to simulate a user’s problem solving process at each step through the dialogue, checking if the simulated user’s goals and memory content can be assumed to lead to the next correct action. Participants will be asked to complete a series of tasks in an online learning environment formulated to compare different methods of organization. This study has the potential to make significant contributions to the field of educational psychology and online education by providing substantive empirical data that sheds light on potential principles that improve the effectiveness, efficiency, and user satisfaction of Web-based education.
Similar to EASE Replication EEG experiment.pptx (20)
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)MdTanvirMahtab2
This presentation is about the working procedure of Shahjalal Fertilizer Company Limited (SFCL). A Govt. owned Company of Bangladesh Chemical Industries Corporation under Ministry of Industries.
Student information management system project report ii.pdfKamal Acharya
Our project explains about the student management. This project mainly explains the various actions related to student details. This project shows some ease in adding, editing and deleting the student details. It also provides a less time consuming process for viewing, adding, editing and deleting the marks of the students.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptxR&R Consult
CFD analysis is incredibly effective at solving mysteries and improving the performance of complex systems!
Here's a great example: At a large natural gas-fired power plant, where they use waste heat to generate steam and energy, they were puzzled that their boiler wasn't producing as much steam as expected.
R&R and Tetra Engineering Group Inc. were asked to solve the issue with reduced steam production.
An inspection had shown that a significant amount of hot flue gas was bypassing the boiler tubes, where the heat was supposed to be transferred.
R&R Consult conducted a CFD analysis, which revealed that 6.3% of the flue gas was bypassing the boiler tubes without transferring heat. The analysis also showed that the flue gas was instead being directed along the sides of the boiler and between the modules that were supposed to capture the heat. This was the cause of the reduced performance.
Based on our results, Tetra Engineering installed covering plates to reduce the bypass flow. This improved the boiler's performance and increased electricity production.
It is always satisfying when we can help solve complex challenges like this. Do your systems also need a check-up or optimization? Give us a call!
Work done in cooperation with James Malloy and David Moelling from Tetra Engineering.
More examples of our work https://www.r-r-consult.dk/en/cases-en/
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
Overview of the fundamental roles in Hydropower generation and the components involved in wider Electrical Engineering.
This paper presents the design and construction of hydroelectric dams from the hydrologist’s survey of the valley before construction, all aspects and involved disciplines, fluid dynamics, structural engineering, generation and mains frequency regulation to the very transmission of power through the network in the United Kingdom.
Author: Robbie Edward Sayers
Collaborators and co editors: Charlie Sims and Connor Healey.
(C) 2024 Robbie E. Sayers
Explore the innovative world of trenchless pipe repair with our comprehensive guide, "The Benefits and Techniques of Trenchless Pipe Repair." This document delves into the modern methods of repairing underground pipes without the need for extensive excavation, highlighting the numerous advantages and the latest techniques used in the industry.
Learn about the cost savings, reduced environmental impact, and minimal disruption associated with trenchless technology. Discover detailed explanations of popular techniques such as pipe bursting, cured-in-place pipe (CIPP) lining, and directional drilling. Understand how these methods can be applied to various types of infrastructure, from residential plumbing to large-scale municipal systems.
Ideal for homeowners, contractors, engineers, and anyone interested in modern plumbing solutions, this guide provides valuable insights into why trenchless pipe repair is becoming the preferred choice for pipe rehabilitation. Stay informed about the latest advancements and best practices in the field.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
Immunizing Image Classifiers Against Localized Adversary Attacks
EASE Replication EEG experiment.pptx
1. Measuring the User Experience
of Adaptive User Interfaces
using EEG: A Replication Study
Daniel Gaspar-Figueiredo <dagasfi@epsa.upv.es>
Jean Vanderdonckt <jean.vanderdonckt@uclouvain.be>
Silvia Abrahao <sabrahao@dsic.upv.es>
Emilio Insfran <einsfran@dsic.upv.es>
EASE2023
3. • Adaptive User Interfaces (AUI) – UI that can be changed considering the Context of use to
enhance the User eXperience (UX)
• Type of changes:
• Layout
• Colours
• Modalities
• Element interactions
• Element styling
• …
Motivation
3
…
User Interface
Context of
use
User
Platform
Environment
• Too wide spectrum of possibilities
Motivation Baseline Experiment Replication Study Data Analysis Study Results
4. Motivation
4
Different UI designs produce different UX
Measure UX through physiological
data such as brain activity (EEG)
Motivation Baseline Experiment Replication Study Data Analysis Study Results
6. Baseline Experiment: Summary
6
• Analyse a set of adaptive graphical menus for the purpose of comparing them with respect to
their UX produced in terms of cognitive load, engagement, memorisation, and attraction from
the point of view of both researchers and user interface designer in the context of end-users of
applications with graphical menus
• 20 graphical adaptive menus were compared (mail manager and web browser applications)
• 40 participants aged from 18 to 63 years old (mean 42)
• Cognitive load, engagement, memorisation, and attraction were computed using EEG analysis
• RQ: Do graphical adaptive menus have a different influence on the UX?
Motivation Baseline Experiment Replication Study Data Analysis Study Results
7. Baseline Experiment: Summary
7
Statistically significant differences were found between the menus regarding the Cognitive load,
engagement, memorisation, and attraction
May be due to diversity of users
• Multiple backgrounds
• Very wide range of ages
Motivation Baseline Experiment Replication Study Data Analysis Study Results
9. Replication Study: Goal
9
• Analyse a set of adaptive graphical menus
• For the purpose of comparing them
• with respect to their user experience produced in terms of cognitive load, engagement,
memorisation, and attraction
• From the point of view of both researchers and user interface designers
• In the context of and end-users' group with a background in computer science
RQ1: Do the 20 graphical adaptive menus have a different influence on the UX?
RQ2: Does the UX measured using EEG signals correlate with the subjective ratings obtained using traditional
questionnaires?
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
10. Replication Study: Experimental objects
10
Vanderdonckt, J., Bouzit, S., Calvary, G., & Chêne, D. (2019). Exploring a
design space of graphical adaptive menus: normal vs. small screens. ACM
Transactions on Interactive Intelligent Systems (TiiS), 10(1), 1-40.
Design space for graphical adaptive menus
(8 variables):
• Position
• Orientation
• Size
• Shape
• Value
• Color
• Texture
• Motion
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
11. • We selected 20 different graphical adaptive menus from
Replication Study: Experimental objects
11
Vanderdonckt, J., Bouzit, S., Calvary, G., & Chêne, D. (2019). Exploring a design
space of graphical adaptive menus: normal vs. small screens. ACM Transactions
on Interactive Intelligent Systems (TiiS), 10(1), 1-40.
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
ItemA
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
ItemA
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
ItemA ItemA
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
ItemA
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
ItemA
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
ItemA
MenuItem3
MenuItem6
MenuItem1
MenuItem2
MenuItem4
MenuItem5
MenuItem7
ItemA
Static menu Bolding
menu
Underlying
menu
Glowing
menu
Adaptive
activation-
area menu
Boxing
menu
Morphing
menu
Ephemeral
menu
Highlighting
menu
Split menu
without
replication
Greyscaling
menu
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
ItemA
12. Replication Study: Experimental objects
12
MenuItem2 MenuItem4
MenuItem3
MenuItem6
MenuItem1
MenuItem8
MenuItem7
MenuItem5 ItemA
Leaf menu
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
ItemA
Italicizing
menu
Blinking
menu
Pulsing
menu
Out-of-context
menu
Temporal menu Pink menu
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
ItemA
Twisting
menu
Rotating
menu
• We represented these menus in two different contexts
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
13. Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Replication Study: Experimental objects
New tab
New window
Zoom
Favorites
New InPrivate window
History
Downloads
Apps
Extensions
Collections
Print
Web capture
Share
Find on page
Read aloud
More tools
Settings
Help and feedback
Close browser
>
>
>
Browser configuration
13
• Context A: Thunderbird Mail manager
• Context B: Microsoft Edge configuration menu
Motivation Baseline Experiment Replication Study Data Analysis Study Results
Mensaje
New Message
Reply
Forward
Forward As
Reply to All
Edit As New Message
Open Message
Open in Conversation
Attachements
Tag
Mark
Archive
Move to
Copy to
Move Again
Create Filter From Message…
Ignore Thread
Ignore Subthread
Watch Thread
>
>
>
>
>
>
Message
14. Replication Study: Participants
14
• Convenience sampling
• 40 students (5 female, 35 male, and 0 non-binary) enrolled in a Master’s degree in Computer
Science at the UPV
• A common background in computer science
• Aged from 20 to 40 years old (mean = 24)
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
15. Replication Study: Variables
15
• Independent Variables:
• (20) Menu types
• Dependent Variables (Emotion-based):
• Attraction: positive/pleasant reaction to a negative/unpleasant reaction response to a situation
• Memorisation: intensity of cognitive processes related to the formation of future memories during an
experience.
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
• Cognitive load: The use of cognitive resources to carry
out a task or visualize a stimulus
• Engagement: degree of involvement or connection
between the participant and the stimulus or task
16. Replication Study: Variables
16
• Performance-based Variables (Dependent):
• Completion-Time: Measures in seconds the time elapsed between the initial state of the task and the final
state, when the task is correctly completed
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
• Perception-based Variables (Dependent):
• Perceived Attraction: Subjective perception of attraction in response to
stimuli or a situation
• Perceived Cognitive Load: Subjective perception of cognitive load or
mental effort required during the task
17. Replication Study: Hypotheses
17
• 𝐻𝑛11: There are no significant differences in the users’ attraction when using different graphical adaptive menus
• 𝐻𝑛12: There are no significant differences in the users’ memorisation when using different graphical adaptive menus
• 𝐻𝑛13: There are no significant differences in the users’ cognitive load when using different graphical adaptive menus
• 𝐻𝑛14: There are no significant differences in the users’ engagement when using different graphical adaptive menus
• 𝐻𝑛15: There are no significant differences in the users’ completion-time when using different graphical adaptive menus
• 𝐻𝑛21: There is no correlation between the Cognitive load (EEG) and the Perceived Cognitive Load (Questionnaire)
• 𝐻𝑛22: There is no correlation between the Attraction (EEG) and the Perceived Attraction (Questionnaire)
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
18. Replication Study: Study Design
18
• Within Subjects design: Each participant used the 20 different menu types
• Each menu type was randomly assigned to a specific domain (Thunderbird’s mail manager or Web Browser settings)
• Each participant used the menus in a different order (also randomly assigned)
Subject 1 Menu_1 Menu_2 Menu_19 Menu_20
…
Subject 2 Menu_8 Menu_5 Menu_2 Menu_1
…
Subject 3 Menu_11 Menu_13 Menu_8 Menu_6
…
Subject n Menu_18 Menu_2 Menu_13 Menu_19
…
… … … … … …
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
19. Replication Study: Instrumentation
19
• Interactive PowerPoint presentation representing the 20 different menu types
• EEG Headset to obtain brain activity
• UEQ-S and NASA-TLX questionnaires
Sensors are placed at
positions AF7, Fp1, Fp2, AF8,
F3, F4, P3, P4, PO7, O1, O2
and PO8 (10-10 standard)
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
20. Replication Study: Instrumentation
20
Data obtained: Raw EEG Data. Brain activity
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
21. Replication Study: Instrumentation
21
Attraction
Memorization
Cognitive load
Engagement
Raw EEG Data
Delta, 0-4 Hz Theta, 4-8 Hz
Alpha, 8-13 Hz Beta, 13-30 Hz
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
22. Replication Study: Instrumentation
22
Subject n Menu_A Menu_B Menu_C Menu_D
…
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
• Completion time: the time is measured from the beginning to the end of the interaction for each menu.
Motivation Baseline Experiment Replication Study Data Analysis Study Results
23. Replication Study: Instrumentation
23
Goal
Experimental
objects
Variables Hypotheses
Participants Study design Instrumentation
Motivation Baseline Experiment Replication Study Data Analysis Study Results
UEQ-Short
Nasa-TLX
Perceived Attraction: UEQ-Short Evaluation
Perceived Cognitive Load: NASA-TLX evaluation
25. Data Analysis
• Mean distance of the Dynamic Time Warping (DTW) for
each participant (40), menu (20), and EEG metric (4)
1. Group the time series by menu and metric
2. Compute the distance matrix
3. Calculate the mean distance for each
participant/menu
25
Motivation Baseline Experiment Replication Study Data Analysis Study Results
Cognitive load – Distance matrix for Blinking menu
Cognitive load per
participant - Blinking menu
1
2
3
40
D=3
D=10
Participant ID
Participant ID
Participant 3
Participant 2
Participant 1
Participant 2
26. Data Analysis
1. Descriptive Study of dependent variables
2. Kruskal-Wallis or ANOVA (hypotheses testing)
3. Pearson or Spearman (correlation)
4. Cliff′s δ (Statistical significances)
26
Motivation Baseline Experiment Replication Study Data Analysis Study Results
28. Study Results: Descriptive (emotion-based variables)
28
Motivation Baseline Experiment Replication Study Data Analysis Study Results
29. Study Results: Descriptive (emotion-based variables)
29
Motivation Baseline Experiment Replication Study Data Analysis Study Results
MenuItem1
MenuItem2
MenuItem3
MenuItem4
MenuItem5
MenuItem6
MenuItem7
ItemA
Out of context
Temporal
Coloured
33. Study Results: Hypotheses testing
33
Hypotheses Method p-value Menu Difference Cliff’s Delta Effect Size
𝑯𝒏𝟏𝟏 Kruskal-Wallis .00 YES 0,302 Medium
𝑯𝒏𝟏𝟐 Kruskal-Wallis .00 YES 0,667 Large
𝑯𝒏𝟏𝟑 Kruskal-Wallis .00 YES 0,585 Large
𝑯𝒏𝟏𝟒 Kruskal-Wallis .00 YES 0,277 Medium
𝑯𝒏𝟏𝟓 Kruskal-Wallis .00 YES 0,575 Large
Hypotheses Normal Distr. Method Correlation p-value
𝑯𝒏𝟐𝟏 YES Pearson 0.819 .045
𝑯𝒏𝟐𝟐 YES Pearson 0.817 .046
Motivation Baseline Experiment Replication Study Data Analysis Study Results
34. Conclusions
• Baseline experiment results are confirmed with a more homogeneous group of users
• Effect size shows that these results could be generalised to other contexts
• Objective assessment of UX (cognitive load and attraction) through EEG signals
• Trends:
• Different font styles or structures uncertainty on UX
• Different colour and temporal variance predictable user reaction
• Menus that are preferred by the users do not necessarily perform well
• Future work:
• Experiments with more UI elements
• Provide recommendations for UI developers on the dimensions more relevant for
UX
• How to adapt UI to better fit with user needs in different contexts
34
Motivation Baseline Experiment Replication Study Data Analysis Study Results
35. Measuring the User Experience of Adaptive User
Interfaces using EEG: A Replication Study
Thanks for your attention
Daniel Gaspar-Figueiredo <dagasfi@epsa.upv.es>
Jean Vanderdonckt <jean.vanderdonckt@uclouvain.be>
Silvia Abrahao <sabrahao@dsic.upv.es>
Emilio Insfran <einsfran@dsic.upv.es>
EASE2023
36. Study Results: Descriptive (performance-based variables)
38
Motivation Baseline Experiment Replication Study Data Analysis Study Results
Editor's Notes
The catalogue had almost 40 menus, we chose 20 of them…
The catalogue had almost 40 menus, we chose 20 of them…
