Your SlideShare is downloading. ×
Et sync uxpa2013
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Et sync uxpa2013


Published on

Published in: Technology, Business

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Today we are here to talk about how to synchronize Tobii eye trackers with external devices and what are the current possibilities to do it.
  • First I would give you a brief introduction of what devices are commonly synchronize with Eye Trackers and what are the applications.Then I will talk about the importance of timing and what things are important to bear in mind when synchronizing devices. Then I’m going to talk about the general concept of synchronization We will focus on synchronization possibilities and go through the most important setups, explaining the characteristics of each one and giving examples. Finally, I’ll talk a bit about the analysis data, the existing options to analyze the data collected and some examples of use. then questions.
  • What kind of devices are we referring to, When we talk about synchronizing an eye tracker with an external device?Well, if we are conducting a study with eye tracking, it’s because we are interested somehow in understanding better the cognitive processes that are behind some actions or decisions that are being testing. Sometimes it’s useful to combine the information that the eye tracking provides with other sources of information.For example, many times it’s useful to collect ET data together with EEG data, so that we don’t have only the reactions that takes place in the brain, but also we can know where the participant was focusing the attention during these reactions. The same as with EEG, we can combine ET with other electrophysiological measurements, such as Electrodermal Activity, Electrocardiography, Electromyography , etc. We may also be interested in synchronizing the ET data with some external camera source, for example a camera that is recording the scene or the participant’s face.Or we want to know how the participant was moving while paying attention to some stimulus, so we would want to synchronize the ET data with some motion capture device.And like these examples, there are other devices that for some reason we are interesting in synchronizing with Tobii Eye trackers.
  • And if we talk briefly about applications, what are the main applications where combining ET data with other data is useful??Here we have some of the main applications, For monitoring, We can use the ET data to make sure that the participant is looking at the stimulus at the moment of recording other source of data. One example would be when recording EEG on children, they get distracted really easily and the ET data could be use to select just the stimulus where the child was looking, making the results more robust. ET data can be also used to control the stimulus presentation of an experiment using gaze contingency, for example waiting for the participant to fixate in one Area on Interest for certain time before continuing with the next stimulus. Another application is using the ET data to detect blinks and saccades in order to remove artifacts from the EEG data. The eye blinks creates big artifacts in the EEG data, and with the ET data, it can be known which artifacts are due to eye blinks, making things easier to select valid data from the EEG. The combination of two synchronized sources of data can be used to analyze one of the data sources depending on the selected data in the other source. One example would be to analyze the ET data only during the period of time when the participant was below certain level of heart rate. Finally, depending on the application and outcome, we need to think if we want a separate analysis of the data, so even though we want to synchronize the data, we don’t need to analyze it together, we just need to complement it, or, the other possibility would be to merge the data and analyze it in the same software.
  • Let’s introduce now some general concepts about timing and latency and what things are important to bear in mind when trying to synchronize two devices.
  • First I’m going to introduce general concepts about computer timing. This is important because usually are computers the ones that are responsible for timestamp and communicate between devices. When we look at the environment of the computer and the different components of the computer there are several challenges when it comes to obtain accurate timing. One is that the processor speed will change depending on different aspects, like the temperature of the processor and since the clocks are related to the processor speed you will also have a change in the clock rate. This means that different hardware systems may have different drifts.The other challenge is that you have different applications trying to access this clock which may cause delays. The other issue is that nowadays computers have more than one processor and normally each of these processors have a hardware clock and programs may access each one of these clocks independently which means that if these two clocks have a certain drift from each other then the timing would be inaccurate.Being said this, the biggest challenge in terms of computer timing is that operations do not necessarily perform in a predictable time lapse and makes it hard to produce or register an event at any precise instant. This is true for different operating systems such as Windows, Mac OS or Linux.
  • Another variable that is relevant when we talk about synchronization of Eye Trackers and other devices, is the ET latency. The total latency of an eye tracker goes from the moment the image is acquired by the sensor until the gaze data is delivered to an application. Most of these time is spent processing the image data and transforming it into gaze data. This latency is dependent on the ET, being up to 10 ms for the TX300 Tobii ET and up to 67 ms in the 30 Hz X2 Tobii ET.It is important to highlight that the gaze data is not affected by the Eye tracker latency, because the timestamp always will reflect the mid-exposure of the image of the eye on the sensor.Latency is mostly important when you want to synchronize live ET data with other device. If you try to merge different sources of data during the recording, it’s likely that each data has a different latency and arrives to the computer at different times. So if you try just to merge the data as it comes from the devices, this will end up creating an inaccurate synchronization of data, as the latencies of the different devices are not totally known. That is why the most common scenario to synchronize data, as we will see in a moment, is to record both data sources independently and merge it after the recording has finished.Another application where latency is important is when we are using gaze contingency paradigms, that is when we want to access the gaze data in order to trigger events during the stimulus presentation.
  • For example, if we talk about the communication delays that can happen when a stimulus presentation computer wants to access to the screen to present a new stimulus. The stimulus presentation software needs to prepare the image for the presentation and send the image to the graphics card. The graphics card will render the image and transmit it to the screen of the eye tracker and then the actual time when the image will be displayed will depend on the refresh rate and the response time of the screen. All this process takes time and the real challenge for a stimulus presentation software is to know not only the timestamp when the image was sent to be displayed, but to know the timestamp when the image was actually displayed on the screen, which is the timestamp that we want to record to be accurate. This is just a real example to understand better the challenge that exist when communicating between devices, but the general conclusion about this brief introduction of timing is that the more you are interested in short events, the more you need to care about timing, and the more you need to know your systems and how they work and perform operations.*************Multitask environment - Different hardware may have different drifts – it’s hard to produce or register an event at any precise instantWhen using separate devices simultaneously, there will be internal and external delaysInternal: sending the triggersExternal: time lag for sending and receiving the triggerThe delays are not constant, but they vary between different ..HardwareStimuli presentation softwareMonitors
  • Taking into account the general concepts that we have introduced about timing, now we are going to talk about how to synchronize Eye tracking data with other devices.
  • let’s talk a bit more about timing, this time in relation to the latency that we might be introducing since we say to the Event sender that we want to create and event until the event is received and timestamped in the device data.There are three parts that can increase latency, the first is the event sender. If the event sender is software running in a computer, we have already learnt that computer may introduce a delay in the execution of a command. The event sender could also be a hardware device designed specifically for the purpose of sending events, which would reduced the event sender latency.Then we have the transmission latency, which normally can be ignored because we can consider transmission through cables almost instantaneous. And we also have to take into account the delay that can be introduced in the event receiver, since it receives the event until it actually includes it in the data with a certain timestamp. Again, the event receiver can be the recording software running on a computer or the actual device that is recording the data.This doesn’t mean that we are always introducing a big delay when sending events, the most common setups are already tested and are used because they have a really good latency numbers, but it’s important to keep in mind that the use of different strategies can lead to undesirable results that we won’t notice unless we study them.
  • The recommendation when we want to synchronize two devices is that each recording software run in a different computer for performance and efficiency reasonsThis means that each computer will have its own clock, so according with the timing concepts we talked about before, computer clock will have slightly different operation and systems might drift with time. Also, it has to be taken into account the latency that each device may introduce since the data is recorded until it is available to store and use it by the recording software. For that reason, to synchronize two sources of data, the most accurate way is create at least one common event on both data recordings that allows to align the Eye tracking data and the external device data. Normally, this alignment is done after the recording. This events should be sent at the same time to both devices and should be included in the data at the time the event was created and sent.
  • There are different strategies when synchronizing two sources of data. The simplest one would be to send one common event at the beginning of the recording and use this as the basis for synchronization. If we want to avoid system drifts and improve accuracy in synchronization we can send more than one event during the recording. One approach is to send equally spaced, repeated events during the recording to synchronize the data afterwards. Another possibility is to send events manually with keystrokes or response boxes. Finally, the most common approach is to send events at the time of stimulus onset. This means that each time a stimulus is presented on the screen, an event will be sent to both the eye tracking data and the external device data.
  • Let’s talk a bit about the types of events that can be used to synchronize eye tracking data and external devices.
  • The most common way to send events to different devices to synchronize them is using TTL technology, which stands for Transistor-Transistor Logic Pulses. It´s a digital signal that has two discrete states, 0V or 5V and the changes between these two those states are used to mark events in the devices. As it consists in an basic electrical signal, the delay in the signal transmission is almost zero and it is normally connected directly to the hardware part to the devices, so there is no computer involved causing delays. These reasons make this technology very convenient synchronize when timing is critical . It’s an old technology but still can be found in almost every computer in the parallel and serial ports.
  • Events can also be created via TCP/IP Protocol, the main difference between this option and the TTL synchronization is that the events are sent between computers through a LAN cable and this allows to send a lot of different types of events.
  • The NTP protocol is based on the idea of having one clock that Is used as the master and synchronize all the other computer or devices clocks based on this one.
  • We can create events manually by pressing keystrokes, using the mouse, or with response boxes for example.
  • Another option is to use light sensors attached to the stimulus presentation screen and trigger events based on the stimulus presented on the screen.
  • We can synchronize data with common physiological events for example, blinking, which can be easily detected in EEG and eye tracking data.
  • In some cases, we might want also to synchronize eye tracking videos with other sources of data.
  • We are going to focus now in the main possibilities that already exist to synchronize Tobii Eye Trackers with external devices
  • Speciffically, I will talk about three main different ways to synchronize devices. I will explain the setups needed and the main characteristics of each one and I will also give examples for each one of the setups for you to understand better what are the outcomes and results of each setup.First, I will introduce the video synchronization possibilities, with a simple synchronization based on live screen recording and with another a bit more advanced setup suitable for external devices that support external video recording. After, I will introduce some set ups based on TTL and some of them include also TCP/IP protocols for communicating between softwares. We well go through two main setups, one of them using TET packages to communicate with the ET and the other one using ClearViewAPI to communicate and record in studio. This options always communicate with external devices using TTL pulses, parallel or serial port.The last way of synchronizing Tobii eye trackers with external devices that I will present will be using a light sensor to generate events and send them at the same time to Tobii Studio as well as to the external device. This option is possible when we include in our setup a third device called StimTracker and only using Tobii TX300 Eye Tracker.
  • This is the most basic setup that we can have to record together eye tracking data and data coming from other device. In this case we just need one computer which runs Tobii Studio as well the software needed to record the external device data. Tobii Studio will record the Eye tracking data, and using the live viewer feature, a video with the information of where the person is looking can be shown in real time. This video running in the Tobii Studio application can be combined in the same screen with the live video of the data recorded from the other device. This setup is useful when we don’t care much about timing, because as we have explained during this presentation, showing a live video of the data that is being recorded will introduce certain latency, as well as recording both data in the same computer may create computer timing problems. Nevertheless, it’s a very easy and useful setup when we don’t need too much precision in synchronizing events in our experiment. On example can be synchronizing some kind of wireless device to record electrophysiological data.
  • There are software that support external video recording. Which means that a video can be synchronized and recorded together with the device data. In this case we have a two computer setup. Tobii Studio records the eye tracking data and has the live viewing feature enabled. In order to record the video coming from the TS computer, the external device computer must have a video capture device. This computer will record the data from the external device as well as synchronizing the live view video coming from the TS computer.It is also possible to run an external device from the TS computer but now you’d have to consider additional internal latencies and lags.
  • In this example, our external device is a Biopac MP150 amplifier, recording the galvanic skin response and electrocardiogram from the participant while the gaze is also being recording. The Biopac recording software Acqknowledge, allows synchronization of a live video source with electrophysiological data. To synchronize the video gaze live with the set up the systems as I just explained, having a computer running tobii studio recording the eye tracking data and a computer running acknowledge recording GSR and ECG data and synchronizing it live with the live viewer video coming from Tobii Studio.
  • Now, we are going to talk about the second method of synchronization, which is the TTL synchronizationTS cannot send TTL pulses so for this section, we will be referring to other software for stimulus presentation.
  • The first option TTL is stimpressf that can record ET data, can communicate with ET via TET packEprime, presentation, matlabComp runs stimpres, collecting data, and timestamp. 2 comps are synched so onset/offset are accurately stamped. TTL sent to ext device stim on/off.
  • Here is an example of output from this setup
  • I now will talk about the last option for synchronizing tobii Eye Trackers with external devices that I will present during this webinar. This option uses a third device called StimTracker for TX300 designed specifically to send synchronized events to Tobii Eye trackers and other external devices. The stim tracker will send events based on the status of a light sensor attached to the stimulus presentation screen.
  • First, I will give a brief introduction about the working concept and the advantages of synchronizing using optionThe StimTracker is a device manufactured by Cedrus that is capable of sending very accurate events both to the Tobii Eye tracker TX300 and to most of the leading EEG and biometric data collection systems, as well as a PC parallel port.The StimTracker will send a synchronized event based on the status of a light sensor that must be attached to one corner of the stimulus presentation Screen. As soon as the light sensor detects a color change in the area where it is attached, it will change its status and the stimtracker will send a simultanous event to both devices. The light sensor can have two values: a value of 0 or 0Vwhen it detects a black color on the screen and a value of 1 or 5V when it detects a with color on the screen. Changing the color from black to white and white to black each time a new stimulus is displayed on the screen, will allow to have a very accurate onset stimulus Event.The main advantage of using this system for synchronizing, Is that it is independent of delays in operating systems or graphic cards, as the light sensor will change its status only when detects a pixel change. This gives a really good timing accuracy recommended in studies that require high temporal resolution. Thestimtracker for TX300 is compatible with Tobii Eye Tracker TX300 and most of the EEG and biometric data collector systems, as well as with the PC parallel’s port of computers.
  • Tobii Studio will integrate the binary triggered events from the light sensor of the StimTracker TX300 in the tobii Studio recording and the eye tracking data. These events will be accessible in two locations in tobii Studio: The replay tab as events on the timeline as well as in the Events list and data available for export in the Data Export Tab.In the Replay tab, the events appearing in the timeline and in the Event list indicate when the signal from the external device changes. This will generate two Studio Events: EventMarkerOn and EventMarkerOff. The event markers can be used to automatically generate segments that can be used to create scenes for data analysis and visualizationsThe second data type that the EventMarkerValue, which provides the signal registered by the eye tracker. The registered signal will be continuous, each sample received from the eye tracker will be associated with a value of the current status of the sensor.Besides from Tobii Studio, when using the Tobii SDK 3.0 to control Tobii TX300 hardware features, it is possible to obtain the EventMarkerValue variable from the Sync Port of the TX300, where the StimTracker is connected, which means that any stimulus presentation software built on the SDK 3.0 is able to record the light sensor values coming from the StimTracker for TX300.
  • Once we have the data from the eye tracker and the external device recorded, it has to be decided if the we want to analyze the data separately or use the events to merge both data sources and analyze them together.
  • Let’s talk first about the separate analysis
  • Sometimes, we want to combine eye tracking with other data but we are not interested in analyzing both signals together, we want to obtain results and conclusions from each source of data separately. However, if we want to be accurate, it is useful to synchronize the Eye tracking data and the external device data by sending events at the onset of each stimulus. Then, both results will be consistent with the stimulus timingIn the case of the Eye tracking data, once it is recorded, it will be analyzed with Tobii Studio or other analysis software and we will obtain Eye tracking results. In the same way, the external device data will be analyzed with the analysis software selected and we will obtain the results just taking into account the information obtained with this source of data. Once we have both results, they can be studied together and complemented to reach more conclusions.
  • We need to have both data sets together in one file in order to start analyzing the data.The first step will be to export the raw Eye tracking data. If the recording software was tobii Studio, it can be included in the exported data variables as Areas of Interest, fixations or Event markers as mouse clicks or keystrokes. The dataset must include one variable that corresponds with the events created to synchronize.If the analysis software is not the same as the one used to record the external device data, the recorded data must be exported including one variable that also corresponds with the events create to synchronize with the eye tracking data.One concept that it hasn’t been introduced yet, it the sampling rate of the recordings. If we want to merge two data sources, we need to have the same number of samples in each dataset. Normally, the sampling rate of the Eye tracking data will not be the same as the sampling rate of the other external device. For example, if we recorded an eye tracking study with the Tobii Eye tracker T120 using a sampling rate of 120Hz, means that we collected 120 gaze data points per seconds. On the other hand, if we have recorded synchronized with the eye tracking data the ECG of the participant with a sampling rate of 250Hz, means that we collected 250 data points per second. We cant merge directly both data sets, as one them, in this case the ECG, will have more points and the points wont correspond in time. Here is where the concept of interpolation is introduced, the shortest data set, in this case, the eye tracking data, will be modified by matching the points per seconds of the other dataset. Once the data has been interpolated, both data set will have the same sampling rate. In this example, the eye tracking data will have 250 points per second. Then, both the eye tracking data and the external device data can be merge using as reference points the events created, and then analyze with the analysis software chosen.At the end, the results will reflect the combination of the eye tracking data and the external device data.
  • Here we can see an example of a merged file containing a recording of EEG synchronized with Eye tracking data.
  • In the case of biometric data, like EEG, ECG or GSR, we are interested only in analyzing the data if the participant was looking at the stimulus shown. To analyze only valid biometric data, the eye tracking data can be used to select the parts of the biometric recording where the stimulus caused a reaction because the participant was looking at it.
  • We may also be interested in the opposite, analyze Eye tracking data based on the levels of the biometric data recorded.For example, analyze eye tracking data only when the signal of the biometric data recorded is above certain level.
  • In EEG, blinks cause artifacts, easily detected in ET, ET can be used to remove artifacts
  • The last example of use is to analyze the external data based on certain areas of interest created in the Eye tracking data.The picture on the right has two areas of interest, the ball and the box. If we create a variable with two values, a value of 0 when the participant was not looking at the area of interest, and a value of 1 when the participant was looking at the area of interest, we can combine this information and analyze the external data For example analyze first the external data when the participant was looking at the ball, and separately when the participant was looking to the box.
  • Transcript

    • 1. Putting the Pieces Together:Synchronization, Analysis, and Presentation of EyeTracking and Complimentary Data StreamsWilkey WongTobii Technology
    • 2.  External devices and applications Basic issues in timing General synchronization concept Approaches to synchronization Data analysis/presentation Questions2Copyright Tobii Technology ABSynchronization of eye trackers with external devices
    • 3. MotionCapture3Copyright Tobii Technology ABEEGExternalCamerasEDA, ECG, EMG, ……Actiontriggers1. EXTERNAL DEVICESWhat external devices may be synchronized?
    • 4. Why would you want to do this?1. Triangulate2. Disambiguate3. Richer interpretationAttention, processing,engagement…
    • 5.  For monitoring online/on task state AOI triggering and gaze contingent experiments Artifact rejection Selection of the data of interest for analysis Synchronization with user actions/reactions5Copyright Tobii Technology AB1. EXTERNAL DEVICESAPPLICATIONSWhat are the applications?
    • 6. Copyright Tobii Technology AB 6TIMINGPutting the Pieces Together
    • 7. 7Copyright Tobii Technology ABHost Computer timing Processor, hardware speed drifts Multitasking environment Multiprocessor core timing2. TIMINGComputers
    • 8. 8Copyright Tobii Technology ABLatency – Inside the Eye TrackerWhen is latency important? When synchronizing eye tracking data with other data (latency) during a recording. Any applications where gaze data is needed to trigger responses (Gaze contingency)10 – 40 ms2. TIMINGEye Trackers
    • 9. Communication Chain – Up to Stimulus Presentation9Copyright Tobii Technology ABThe more you are interested in short events the more you need to care about timing2. TIMINGSystem
    • 10. Timing - Summary1. Host computer2. Eye tracker3. Entire system
    • 11. Copyright Tobii Technology AB 11APPROACHES TO SYNCHRONIZATIONPutting the Pieces Together
    • 12. 12Copyright Tobii Technology AB3. APPROACHES TO SYNCHRONIZETOTAL EVENT LATENCY CHAINDeviceEvent senderlatencySoftwareHardwareTransmission LatencySoftwareEvent receiverlatencyHardwareEventTotal Latency – END-TO-ENDEvent sender Event receiverDeviceEach system should be characterized at the component level to understand the system
    • 13. 13Copyright Tobii Technology AB3. APPROACHES TO SYNCHRONIZECOMMON SYNCHRONIZATION EVENTSTriggerEventExternaldevice
    • 14. 14Copyright Tobii Technology AB3. APPROACHES TO SYNCHRONIZESYNCHRONIZATION EVENT STRATEGYOne common eventStimulusEventEvenly spaced eventsStimulusEventEvents based on Stimulus onsetStimulusEventManually triggered eventsStimulus EventAccuracy and drift
    • 15. Copyright Tobii Technology AB 15TYPES OF EVENT TRIGGERSPutting the Pieces Together
    • 16. 16Copyright Tobii Technology AB4. HOW TO SYNCHRONIZETYPES OF EVENT TRIGGERS TTL pulses (Parallel/Serial Port) TCP/IP NTP Events (clicks, keystrokes) Light sensors Physiological events (blinks) Button presses Video0/5V low latency
    • 17. 17Copyright Tobii Technology AB4. HOW TO SYNCHRONIZETYPES OF EVENT TRIGGERS TTL pulses (Parallel/Serial Port) TCP/IP NTP Events (clicks, keystrokes) Light sensors Physiological events (blinks) Button presses VideoLANSignaling flexibility
    • 18. 18Copyright Tobii Technology AB4. HOW TO SYNCHRONIZETYPES OF EVENT TRIGGERS TTL pulses (Parallel/Serial Port) TCP/IP NTP Events (clicks, keystrokes) Light sensors Physiological events (blinks) Button presses VideoSingle master clock
    • 19. 19Copyright Tobii Technology AB4. HOW TO SYNCHRONIZETYPES OF EVENT TRIGGERS TTL pulses (Parallel/Serial Port) TCP/IP NTP User Events(clicks, keystrokes) Light sensors Physiological events (blinks) Button presses VideoDirect user input
    • 20. 20Copyright Tobii Technology AB4. HOW TO SYNCHRONIZETYPES OF EVENT TRIGGERS TTL pulses (Parallel/Serial Port) TCP/IP NTP Events (clicks, keystrokes) Light sensors Physiological events (blinks) Button presses VideoDirect measurement
    • 21. 21Copyright Tobii Technology AB4. HOW TO SYNCHRONIZETYPES OF EVENT TRIGGERS TTL pulses (Parallel/Serial Port) TCP/IP NTP Events (clicks, keystrokes) Light sensors Physiological events (blinks) Button presses VideoEEGEye TrackingRich possibilities
    • 22. 22Copyright Tobii Technology AB4. HOW TO SYNCHRONIZETYPES OF EVENT TRIGGERS TTL pulses (Parallel/Serial Port) TCP/IP NTP Events (clicks, keystrokes) Light sensors Physiological events (blinks) Button presses VideoExtremely simple
    • 23. 23Copyright Tobii Technology AB2. HOW TO SYNCHRONIZETYPES OF EVENT TRIGGERS TTL pulses (Parallel/Serial Port) TCP/IP NTP Events (clicks, keystrokes) Light sensors Physiological events (blinks) Button presses VideoEEGEffort-intensive
    • 24. Types of event triggers - Summary1. Choose base on flexibility, accuracy, action2. Options range from simple to complex andtime intensive3. Consider data integration needs
    • 25. Copyright Tobii Technology AB 25SYNCHRONIZATION EXAMPLESPutting the Pieces Together
    • 26. 5. SYNCHRONIZATION EXAMPLESOVERVIEWVideo TTL & TCP/IP Light Sensor Live screen recording Devices that supportexternal video recording Offline synchronization Stimulus presentationsoftware records EyeTracking data Stimulus presentationsoftware and Tobii Studiorecords Eye Tracking data StimTracker for TX300Copyright Tobii Technology AB 26
    • 27. Copyright Tobii Technology AB 27VIDEO SYNCHRONIZATIONPutting the Pieces Together
    • 28. 28Copyright Tobii Technology AB5. SYNCHRONIZATION EXAMPLESLIVE SCREEN RECORDINGNON-CRITICAL TIMING Tobii Studio : Records Eye Tracking data External device software: Records the external data.TX300 Eye Tracking dataLANVGA/DVITobii Studio&External device software Computer screen: Shows live both recordings.VGA/DVIExternal DeviceComputer screenEye Tracker
    • 29. EXTERNAL VIDEO RECORDING5. SYNCHRONIZATION EXAMPLES29Copyright Tobii Technology ABLive videoEye Gaze dataStimulus presentationStimulus presentation +gaze Tobii Studio: Presents the stimuli Records Eye Tracking data Has the Live Viewer feature enabled External device software: Records the physiological data synchronizedwith the video Captures a video in real time from Tobii StudioLive Viewer featureEye Tracker Tobii Studio computer External device computerVideoCapturedeviceExternaldevice
    • 30. 30Copyright Tobii Technology AB5. SYNCHRONIZATION EXAMPLESVIDEO – EXAMPLE BIOPACExample: ET video recording using Acqknowledge software and Tobii Studio Live viewerTX300 Eye Tracking dataCapturedeviceLANVGA/DVIVGA/DVI(Live Viewer)Eye Tracking VideoTobii Studio Computer Recording computer withAcqknowledge 4Biopac System:Amplifier MP150 +Isolation Module STP100C +GSR module GSR100C
    • 31. Copyright Tobii Technology AB 31TTL SYNCHRONIZATIONPutting the Pieces Together
    • 32. 32Copyright Tobii Technology AB5. SYNCHRONIZATION EXAMPLESTTL: E-PRIME / PRESENTATION / MATLAB / OTHER Stimulus Software presentation: Collects eye tracking data from ET Timestamps in the ET data the stimulus onset Sends through the parallel/serial port TTL signalsat the stimulus onset E-Prime / Presentation: Tobii Extensions needed Matlab / Other: Tobii SDKStimulus presentation software records Eye Tracking dataTET packagesExternaldeviceDVI / VGALANParallel/Serial PortExternal Device ComputerStimulus Presentation Computer
    • 33. 33Copyright Tobii Technology AB5. SYNCHRONIZATION EXAMPLESE-PRIME EXTENSIONS FOR TOBII E- Prime controls Eye Tracker with TET packages E- Prime send TTL pulses to the external device E-Prime collects raw data from the Eye Tracker• Fixation Filters (Fixation / Saccades)• Visualizations (Heat maps / Gaze plots)• AOI• Necessary a third software to analyze Eye TrackingdataExample: TX300 and Brain Products EEG recording using E-Prime and TET PackagesDVILANParallel CableBrainVision RecorderComputerUSB Adapter EEG BrainAmp
    • 34. 34Copyright Tobii Technology AB5. SYNCHRONIZATION EXAMPLESE-PRIME EXTENSIONS FOR TOBIIExample: TX300 and Brain Products EEG recording using E-Prime and TET PackagesDVILANParallel CableBrainVision RecorderComputerUSB Adapter EEG BrainAmpE-PrimeEvents
    • 35. Copyright Tobii Technology AB 35EXTERNAL TRIGGERING/SIGNAL DEVICESPutting the Pieces Together
    • 37. 37Copyright Tobii Technology AB5. SYNCHRONIZATION EXAMPLESEXTERNAL TRIGGERING/SIGNALINGTwo data types:- EventMarkerValue: 0=off, 1 = on- StudioEvents: EventMarkerOn, EventMarkerOffStudio
    • 38. 38Copyright Tobii Technology AB5. SYNCHRONIZATION EXAMPLESEXTERNAL TRIGGERING/SIGNALINGLight sensorTX300StimTrackerCoaxialParallel Cable DB-25DVILANTobii Studio ComputerExample: TX300 and ANT EEG Recording using StimTracker for TX300ASA-Lab ComputerANT EEG AmplifierTobii Studio Software ASA-Lab SoftwareStimTrackerEvents
    • 39. Copyright Tobii Technology AB 39DATA ANALYSISPutting the Pieces Together
    • 40. SEPARATE:JOINT:40Copyright Tobii Technology AB6. DATA ANALYSISOVERVIEWSeparate or joint analysis?External Device DataEye Tracking DataEventEvent
    • 41. Copyright Tobii Technology AB 41SEPARATE ANALYSIS BY DATA STREAMPutting the Pieces Together
    • 42. 42Copyright Tobii Technology AB6. DATA ANALYSISSEPARATE ANALYSIS BY DATASTREAMAnalyze each data stream independentlyExternal Device Data Eye Tracking Data. . .Eye TrackingResultsExternal DeviceResultsStimulusStimulus
    • 43. Copyright Tobii Technology AB 43COMBINED ANALYSISPutting the Pieces Together
    • 44. 44Copyright Tobii Technology AB6. DATA ANALYSISCOMBINED ANALYSISMerge data and analyze it together2. Match sampling rates!!External Device Data e.g.:Sampling rate: 250 Hz250 samples / secEye Tracking Data e.g.:Sampling rate: 120 Hz120 samples / secAfter interpolate data:250 samples / Sec1. Export raw data3. Merge dataEye Tracking &External DeviceResultsInterpolate. . .Analysis or display
    • 45. 45Copyright Tobii Technology AB6. DATA ANALYSISCOMBINED ANALYSISMerge data and analyze it together2. Match sampling rates!!1. Export raw data3. Merge dataEye Tracking &External DeviceResultsInterpolate. . .EEG dataEye Tracking data
    • 46. Copyright Tobii Technology AB 46JOINT ANALYSIS: EXAMPLES OF USEPutting the Pieces Together
    • 47. 47Copyright Tobii Technology AB6. DATA ANALYSISJOINT ANALYSIS: EXAMPLES OF USE Eye Tracking triggers - EEG, ECG, GSR,…EEG, ECG, GSR,…Eye TrackingAnalyze data based on the eye tracking data validity
    • 48. 48Copyright Tobii Technology AB6. DATA ANALYSISJOINT ANALYSIS: EXAMPLES OF USEAnalyze data based on electrophysiological signals Physiological measure triggers - EEG, ECG, GSR,…Eye TrackingEEG, ECG, GSR,…
    • 49. 49Copyright Tobii Technology AB6. DATA ANALYSISJOINT ANALYSIS: EXAMPLES OF USE Artifact removal – Blinks:Localize blinks in EEG dataEEGEye Tracking
    • 50. 50Copyright Tobii Technology AB6. DATA ANALYSISJOINT ANALYSIS: EXAMPLES OF USE Areas of InterestAnalyze the External data based on the AOIs of the Eye Tracking dataBALLBOXExternal DataAOI BallAOI Box
    • 51. Co-presentation – Animation and scrolling data
    • 52. Copyright Tobii Technology AB 52THANK YOU!QUESTIONS?