Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
UX Group: Eye-tracking with quantity
Mária Bieliková
20 eye-trackers in one room
(UXI Labs @ Slovak University of Technology)
20x Tobii X2-60 eye
trackers
Screen recording
Fac...
Mass data collection is excellent for
summative studies
• Same conditions for
every participant
• Saving staff time
3
Group studies need
infrastructure software
Our UX Group Research Software
• Data collection from all UX nodes
• Lifecycle management of user studies projects
• Proce...
UXI@FIIT = three laboratories
6
Tobii
TX300
PC,
EEG,
...
Tobii
X2-60
mobiles,
TV, PC
20x Tobii X2-60
gaze tracking
PC
Cont...
8
Setup
Data
Streams
9
Start
Session
Annotate recorded data
10
Export recorded data
11
Enjoy your data!
12
Enjoy your data!
13
Enjoy your data!
14
Enjoy your data!
15
UX Group Lab –
first experiences
Twin-experiment
(two experiments taken with the same group of 50 participants, ~2,5GB of
raw csv data)
Experiment #1:
Crow...
Experiment #1:
Crowdsourcing movie categorizations
Participant’s task:
1. View the description of a documentary movie
2. P...
19
Experiment #1:
Task user interface with example gaze plot.
The gaze reveals, what other options
the workers considered
20
“Saving rhino phila"
[["animals", 100], ["crime", 50]]
[["t...
Experiment #2:
Questionnaire deception detection
Participant’s task:
Fill out the big five personality trait questionnaire...
22
Experiment #2:
Big five personality trait question example
We were looking for fixation order, response
times, pupil dilation, first fixation…
23
Deception cases were characteristic...
Methodological
lessons learnt
Have some assistants
There were 3 of us: one experiment conductor and two
assistants. It was enough for 20 people, but bar...
Do a pilot study with multiple participants,
don’t just do it with individuals
If the study has to be parallel recording, ...
Group calibration is best done two-step
First, do a presentation (slide) demonstration.
• Do not allow participants to do ...
Never let participants to do anything
during the time you speak to them.
Stress to them the necessity of discipline.
Paper...
More group studies
in preparation
Visual search ability for target findability
evaluation
30
Target:
31
32
33
Join us at FB
http://www.facebook.com/groups/UXIsk/
34
Upcoming SlideShare
Loading in …5
×

WUD Slovakia 2015: Experiment v UX class / Prof. Ing. Mária Bieliková, PhD.

545 views

Published on

WUD Slovakia 2015: Experiment v UX class / Prof. Ing. Mária Bieliková, PhD.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

WUD Slovakia 2015: Experiment v UX class / Prof. Ing. Mária Bieliková, PhD.

  1. 1. UX Group: Eye-tracking with quantity Mária Bieliková
  2. 2. 20 eye-trackers in one room (UXI Labs @ Slovak University of Technology) 20x Tobii X2-60 eye trackers Screen recording Face recording + Infrastructure for transferring all the data to the central server 2 3D depth camera 60Hz eyetracker
  3. 3. Mass data collection is excellent for summative studies • Same conditions for every participant • Saving staff time 3
  4. 4. Group studies need infrastructure software
  5. 5. Our UX Group Research Software • Data collection from all UX nodes • Lifecycle management of user studies projects • Processing of gathered data • Annotations • Cleaning • Export • Archivation 5
  6. 6. UXI@FIIT = three laboratories 6 Tobii TX300 PC, EEG, ... Tobii X2-60 mobiles, TV, PC 20x Tobii X2-60 gaze tracking PC Control room
  7. 7. 8 Setup Data Streams
  8. 8. 9 Start Session
  9. 9. Annotate recorded data 10
  10. 10. Export recorded data 11
  11. 11. Enjoy your data! 12
  12. 12. Enjoy your data! 13
  13. 13. Enjoy your data! 14
  14. 14. Enjoy your data! 15
  15. 15. UX Group Lab – first experiences
  16. 16. Twin-experiment (two experiments taken with the same group of 50 participants, ~2,5GB of raw csv data) Experiment #1: Crowdsourcing study demonstrating the use of eye-tracking for human computation scenario. Participants had to categorize of documentary movies based on their descriptions. Experiment #2: Questionnaire deception detection using eye-tracking Participants had to fill out the big five personality trait questionnaire. 17
  17. 17. Experiment #1: Crowdsourcing movie categorizations Participant’s task: 1. View the description of a documentary movie 2. Pick a primary category for the movie from the list 3. [Optionally] Pick a secondary category Hypothesis: We can discover additional classification information, if we eye-track the workers during the task 18
  18. 18. 19 Experiment #1: Task user interface with example gaze plot.
  19. 19. The gaze reveals, what other options the workers considered 20 “Saving rhino phila" [["animals", 100], ["crime", 50]] [["traveling", 1150.0], ["geography", 1017.0], ["biography", 500.0], ["health", 400.0], ["animals", 367.0], Title: Picked categories: Viewed categories: Experiment #1: Observations
  20. 20. Experiment #2: Questionnaire deception detection Participant’s task: Fill out the big five personality trait questionnaire that matches 1. your personality (respond honestly) 2. an ideal worker for job position (faking good instruction) Hypothesis: We can detect deception by analysis of gaze data Van Hooft, E.A.J. & Born, M.P. (2012). Intentional response distortion on personality tests: Using eye-tracking to understand response processes when faking. Journal of Applied Psychology, 97(2), 301–316. doi:10.1037/a0025711 21
  21. 21. 22 Experiment #2: Big five personality trait question example
  22. 22. We were looking for fixation order, response times, pupil dilation, first fixation… 23 Deception cases were characteristic by: • First fixations at extreme response options p = 3.538e-14 < 0.001 • Longer reaction times p = 0.039 < 0.05 • Pupil dilation variance before and after answering a question p = 0.022 < 0.05 Experiment #2: Observations
  23. 23. Methodological lessons learnt
  24. 24. Have some assistants There were 3 of us: one experiment conductor and two assistants. It was enough for 20 people, but barely. 25
  25. 25. Do a pilot study with multiple participants, don’t just do it with individuals If the study has to be parallel recording, do the pilot likewise Pilot allowed us to properly update instructions • No browser window resizing, no zooming • Make the scenario as simple as possible. The more steps there are, the more mistakes the participants make. Instructions are crucial • Don’t just use paper instructions, use slides and presentation as well 26
  26. 26. Group calibration is best done two-step First, do a presentation (slide) demonstration. • Do not allow participants to do anything during the explanation. • Make them watch you, or they will screw up. Second, let everyone follow the remembered instruction from the first step. • Watch the calibration status on the dashboard • Attend everyone that raises any problems • 3 experiment conductors were enough to handle calibration problems 27
  27. 27. Never let participants to do anything during the time you speak to them. Stress to them the necessity of discipline. Paper instructions won’t always save you. Participants do not read them properly. Better communicate important things also verbally. 28
  28. 28. More group studies in preparation
  29. 29. Visual search ability for target findability evaluation 30 Target:
  30. 30. 31
  31. 31. 32
  32. 32. 33
  33. 33. Join us at FB http://www.facebook.com/groups/UXIsk/ 34

×