ICC2017 Washington - http://icc2017.org/ 6205.1 Exploring the possibilities of eye tracking and EEG integration for cartographic context Merve Keskin Istanbul Technical University Kristien Ooms Ghent University A. Ozgur Dogru Istanbul Technical University Philippe De Maeyer Universiteit Gent

1. Exploring the Possibilities of
Eye Tracking and Electroencephalogram Integration
for Cartographic Context
Merve Keskin
Kristien Ooms, A. Ozgur Dogru, Philippe De Maeyer

3. To provide better maps, get to know your user better

4. Which areas are activated more?
ET
covert attention
EEG
overt attention

5. Research questions
User behaviors
across tasks?
Cognitive load?
Search pattern?
Distracting
elements?
Activation
across tasks?
EEG & ET correspondence?
Added value of EEG?
Additional insight?

6. Task and Stimuli Visual search task Memory/remembering task
EEG (BIOPAC-Acqknowledge), ET (SMI), PC (as a drawing tool), post-test questionnaire
Participants Experts (13 females, 11 males), novices (8 females, 24 males)
Data collection
methods
User experiment design – mixed methods
Data analyses Open source MATLAB toolbox - EEGLAB with EYEEEG plug-in
Acquired data EEG statistics, ET statistics, sketch maps, users’ background information

7. ET & EEG integration is not straightforward
Synchronization Experiment design Preprocessing
>

8. Synchronization is established through TTL trigger signal
TTL: Transistor-transistor logic; widely used technology to make integrated circuits

9. Design as simple as possible
Stimuli not overly complex
contained a number of main structuring elements
did not contain well-known areas
Tasks Visual search task 3 maps, 3 labels for each map
Memory task 1 map
Drawing tool MS Paint
Evaluation Test before main test Simplicity of language
of design Clarity of tasks
Duration of stimuli
Communication with the participant

10. Fp: frontopolar
F: frontal,
P: perietal,
O: occipital,
C: central,
T: temporal
International 10-20 EEG system
Frontal
Perietal
Central
reference
ear electrode
ground
ear electrode
Some temporal and occipital
electrodes

11. Baseline: a period of resting physiology
Individual differences in baseline EEG
cognitive, emotional, motor processes & skills
Task-related changes in EEG measures
comparison
Accurate baseline measure is critical to
interpretation of EEG findings
Before tasks 60 seconds long
Between each task, 30 seconds long

12. Data management
Significant amount of time is required
for preprocessing the data
Synchronization
Merging and aligning
ET & EEG data
Artifact removal from
EEG data

13. We wrote a Python script to convert ET data
into a compatible format for EEGLAB

14. We wrote a Python script to convert EEG events
into a compatible format for EEGLAB

15. Total: 41 Channels
Channel location file should consist of
both EEG and ET channels
16 EEG channels
'fp1' 'fp2' 'f3' 'f4' 'c3' 'c4' 'p3' 'p4' 'o1' 'o2' 't5'
't6' 'fz' 'fpz' 'cz' 'pz'
8 digital channels
'dc1' 'dc2' 'dc3' 'dc4' 'dc5' 'dc6' 'dc7' 'dc8'
17 ET channels
‘Time’, ‘Trial’, ‘Stimulus’, ‘R_Raw_X_(px)’,
‘R_Raw_Y_(px)’, ‘R_Dia_X_(px)’,
‘R_Dia_Y_(px)’, ‘Pupil_Diameter_ (mm)’,
‘R_POR_X_(px)’, ‘R_POR_Y_(px)’, ‘R_GVEC_X’,
‘R_GVEC_Y’, ‘R_GVEC_Z’, ‘R_EPOS_X’,
R_EPOS_Y’, R_EPOS_Z’, ‘R_Event_Info’

16. Baseline
Correcting the measurements

17. EEG data includes noises orginated from different sources
EEG activity not elicited by stimuli - e.g. alpha waves
Trial-by-trial variations
Articfactual bioelectric activity - eye blinks, eye movement, muscle activity, skin potentials
Environmental electrical activity - e.g. from monitors
Highpass- to filter our low frequencies
Low-pass – to filter out high frequencies
FILTERING?

18. Removing correlations in the data (different channels to be uncorrelated)
Independent Component Analysis (ICA) is very powerful to
separate out artifacts embedded in the EEG data

19. Review
Synchronization Experiment design Preprocessing
>

20. Future work
We will focus on event-related potentials (ERP)*, not the whole recording
EEG recording of a single participant ~ 30 minutes long
Thus, exact time intervals when the response from a participant occur
* Event-related potential (ERP) is the measured brain response that is the direct result of a specific sensory,
cognitive, or motor event.
We will collaborate experts in psychological domain for interpretation of results.
We will compare the results with the previous work (Ooms, 2012) conducted
with only ET
event-related potentials (ERP)*