Think aloud A is widely used in usability research. During a task, the user is verbalising their actions and thoughts out loud whilst navigating the site.
The moderators role is then to probe feedback during the test
This technique gives immediate feedback during a task and means the participant is not relying on memory to provide feedback on things they experienced.
Retrospective Think Aloud
With eyetracking hardware and software participants are asked to complete a task silently and on their own. Once the task is complete, the moderator plays back the dynamic eye tracking video while asking the user about their underlying motivation. This allows the participant to get on with their task without being distracted during the task. The feedback is often more thought through when replaying the eye tracking video.
Tobii eye trackers use near infrared diodes to generate reflection patterns on the corneas of the eyes of the user.
These reflection patterns, together with other visual information about the person, are collected by image sensors.
Together with very sophisticated image analysis and mathematics a gaze point on the screen can be calculated, i.e. where the user is looking
Pupil Corneal reflection
...Fixations and Saccades Page A fixation describes points where the eye is relatively still and concentrating directly on a subject. A saccade describes the rapid movements between fixations. A series of fixations and saccades is known as a scan path . In Tobii Studio fixations appear as spots and saccades appear as the lines between fixations
Eye tracking is effective because it is non invasive however this massive advantage to studies is often negated by forcing people to be someone they aren’t!
The same thing can apply to test environments and test set-ups.
What Does That Gaze Plot Show? Traditional search strategy on Google – displaying the ‘golden triangle’ and then still clicking on the first result – even after a brief foray below the fold – but with no sign of interest or detailed interaction. More ‘golden triangle’ style behaviour but this time the participant fails to go below the fold and chooses a video link as their answer. Very detailed search behaviour with 5 times the number of fixations and each result inspected, the participant then moved onto page two of the results.
A “mouse marker” (see example opposite) can be a useful tool when doing web usability testing as it encourages the participant to park their mouse and click on a certain point.
The benefit of this is that you are controlling the initial ‘start’ point for any visual outputs, allowing you to clip data effectively.
You also reduce the risk of polluting data (especially statistical data) with false ‘hits’ as people search for the mouse. If you look below at the examples showing a pair of gaze plots with and without a marker prior to the webpage being displayed.
A fixation cross (a basic image as shown opposite) between static stimuli is another way of normalising your data, and is particularly useful when doing print ad testing, or when you may be randomising stimuli.
Naturally when someone reads text copy, see’s an advert or follows a user journey there will be a point where their gaze is carried over to the next stimuli as the image changes. By ‘resetting’ each participants vision to the centre of the screen.
When randomising images this also allows you to remove any random legacy gaze points which could pollute the data sets.
As everyone's gaze will be central you can then ‘clip’ the first 20-30ms to remove the forced hot spot of data, the can also ensure interaction elsewhere on the image is not biased due to an unrealistic peak of interaction... As shown below.
Page Data including legacy data from fixation cross Data shown with 30ms clipped from the initial exposure
Heat Maps – Different Metrics Page This heat map was created with the fixation count metric applied. This means that the hot spots indicate where the highest number of fixations were. In this case this does not necessarily gauge the level of engagements on the page, just the level of attraction. The gaze opacity map option has the same functionality but reverses the way the data is displayed. The transparent areas are where the largest numbers of fixations were. Remember to take into account peoples peripheral vision, as the output illustrates the fixations only.
Heat Maps are very commonly used outputs from eye tracking but it is important to know that there are three very different metrics that can be applied to the data, and each is suitable for different types of stimuli, methodology and testing.
The three available metrics are :
Fixation count : Here the data is collated around the total number of fixations totalled across all the selected recordings, regardless of the duration of each fixation. For example the red areas may indicate 25 fixations.
Absolute duration : With this filter the data represents the total amount of time spent fixating in an area, across all of the selected recordings. For example the hot spot areas may indicate 9.234 seconds. This metric is useful when each person has seen the stimuli for the same exposure period.
Relative duration : The filter is ideal for web based testing, or if each participant sees a stimuli for a variable amount of time. Each persons interaction is looked at individually and the data normalised across all the recordings. The results are shown as a percentage of time spent on that stimuli.
Retrospective Think Aloud vs Think Aloud Think aloud testing slows down the users processes due to the additional cognitive workload. Here the user was tasked with searching for a house, and while doing this the moderator asked about how he found the site, the search engine and if he had noticed the advertising. The participant felt obliged to go below the page fold, then they interacted with the advertising after the moderator had mentioned it. You can clearly see the long fixations, and the extensive journey to basically complete a task that was literally right in front of them from the outset. Think aloud also allows users time to find the next step in a process or journey, potentially further damaging findings from testing. We can fixate 5 times or more a second, potentially over 300 fixations in the one minute this user took to complete this page – far more than we can verbalise and rationalise. By allowing the user to complete their task and interview them retrospectively this user (of a similar PC literacy to the previous participant) got on with the task in hand. They completed the task in 32 seconds, just over half the duration of our first example. More tellingly their interactions show that they didn’t go below the page, the didn’t interact with the advertising and after a very brief scan around the page they determined that their first choice of action was the correct one and searched for their potential new home. When speaking to them after the event during an RTA interview we initially start with the gaze and mouse data hidden – asking them their first impressions and what they thought they did, and then reveal their actual actions and discuss this further with them. The result is far more relevant, realistic and valid data.
For users with a T or 1750/2150 series eye tracker mobile device testing is possible using the scene camera function in Tobii Studio, and placing a camera under a table as seen in the image opposite. The user then interacts physically with the phone under the table – and views their actions on the screen and is eye tracked as normal. For obvious reasons this is not the most natural way of testing mobile devices!
Utilising a X series eye tracker, inverted you can test mobile phone devices in a much more relaxed and realistic way. The eye tracker is inverted to adjust the viewing angle of the unit from 0 to 35 degrees into 0 to -35 degrees. This compensates for the fact you are looking underneath the tracker, not over it as you would with a screen based test. The calibration is done on a level plane with the tracker and there is an amount of movement allowed of the handset by the participant. A manual calibration is required, and a strict procedure to invert the calibration needs to be followed.
Testing Physical World Objects Scene Camera For Physical Stimuli
To further add to the flexibility of the X series tracker it can also be used with an external ‘scene’ camera to test physical objects such as mobile phones, printed material, toys, hand held devices and so on.
As there is no screen to base a calibration on a grid is printed or drawn onto card or similar and this is then mapped within the software to ensure the eye tracked data is accurate.
The camera is connected to the PC via a video capture board and live viewing is possible.