1. Expert Evaluation Method: GOMS-KLM
ABSTRACT
General Terms
1. INTRODUCTION
In this part of the final report, an expert review, evaluation and
inspection method will be selected.
For this to work, an application will be used in order to apply and
perform the evaluation method (an application with a substantial
interactive component). The report will consist of the following;
a description of the selected method, how the study was setup and
performed (including how the application was selected for testing)
and how the results was analysed.
2. EVALUATION METHOD VIA EXPERT
ANALYSIS
Before deciding on which evaluation method, there are many
types of techniques available for evaluating an interactive system.
Here are some evaluation methods:
 Immersion, Paired comparison, Perspective-Based
Inspection, Perspective-Based Inspection, Playability
heuristics, Product Semantic Analysis, Property checklists.
2.2 INSPECTION METHOD
GOMS is a modelling technique that analyses the expert user’s
complexity of interactive systems [2]. This model predicts the
time it will take for the expert user to carry out a goal such as the
time required to fill a specific web form.
Amir Mohamed Ali
Middlesex University
London NW4 4BT
07908855593
Aa3263@live.mdx.ac.uk
Dr Serengul Smith
School of Science and Technology
London NW4 4BT
8411 6747
s.smith@mdx.ac.uk
Filling forms is common when it comes down to web interaction
For this factor, designing web forms that enhance users’ efficiency
is vital task. This paper consists of a tool called KLM Form
Analyser (KLM-FA) which allows experts to make predictions of
execution times of web form filling tasks.
Categories and Subject Descriptors
H.5.2 [User Interfaces]: Interaction styles, Evaluation
Design, User interface, Form-Analyser, Human Factors, Human
performance.
Expert evaluation, KLM-GOMS, Keystrokes, KLM Form Analyser,
& Registration form.
Keywords
2.1 METHOD CHOSEN
The selected method is KLM-GOMS; GOMS is a type of modelling
user behaviour and it is modelled in terms of; Goals, Operators,
Method and Selection Rules.
Goals - What the user is trying to accomplish (execution time is
estimated). This is where the KLM used
Operators - The operations such as actions used to accomplish the
goal (double-clicking, press-ENTER, etc.) (Amount of time it took
for the user to execute)
Method - The expert user will need to follow an algorithm
(instructions) that the user has to follow to achieve the desired goal.
An example would be to accomplish the goal ENTER-DETAILS in
Dropbox and MOVE-MOUSE to the SUBMIT button.
Selection Rules – The user will be using a fixed method to achieve
the same goals. The rules will consist of a condition statement such
as the IF FUNCTION “if the word to be deleted is more than 4 lines
away from the cursor location, then use-arrows-delete-word, or else
use the use-mouse-delete-word method”
The way in which evaluation method is decided depends on what
type of aspects of the system is being tested. For example, if
quantitative data is being tested such as statistics (recording
keystrokes of the user) then the suitable method would be the
KLM-GOMS method as the results consists of numeric data which
can be easily analysed using statistics techniques.
Cognitive Modelling; This approach to HCI prompts to predict the
user performance based on a model of cognition. The model
predicts how humans would complete a set of tasks using a user
interface according to Dix [1].

2. This is a cognitive walkthrough which belong to a family of
techniques called Inspection Methods. Inspection Methods
involve Keystroke Level Modelling also known as an analytic
technique. They also referred to as Expert Reviews because the
testing is usually performed by an expert in usability or Human
Computer Interaction.
3. APPLICATION CHOSEN
3.2 INTERACTIVE WEB-FORM
The application that is going to be used is an interactive web
form. This is a critical aspect of the overall expert’s experience
when it comes down to the usability of the system. This report
will evaluate web form filling tasks using Dropbox registration as
an example of a web-filling form.
3.3 WHAT WILL BE TESTED
 How much time an expert will require filling a specific web
form in a straight forward manner.
 The completion time as well as the time required to move the
cursor to the form’s elements.
In addition to this, Fitts’ law is known for this type of evaluation
as it is computationally used to calculate the exact time required
to test these kind of elements such as how much time it took to
move a cursor to the forms elements without confusion or errors.
Fitts law is a descriptive model of human movement which is
often used in HCI (human-computer-interaction). This type of
model is a “scientific law that predicts the time required to freely
and quickly moving to a target which consists of a ratio between
the distance and width of the target” [3]. However the GOMS-
KLM will be used as the evaluation method as it suits the study
the best.
4. SOFTWARE SELECTED
4.1 KLM FORM-ANALYSER TOOL
This software enables experts to evaluate web forms in an
effective manner which is why this is the software selection. It
also enables predictions of how long it takes to fill in a web form.
This tool services models of human performance.
This tool employees with KLM and Fitts’ law modeling to
estimate the time to fill a web form using different interactions
methods to present data. Filling forms is a common in web
interaction [4]. Therefore the design of web forms which enhances
users is a vital task. This software enables predictions of
execution times of web form filling tasks. KLM supports couple
of evaluation scenarios, both in a formative and summative
context.
4.2 ORIGINAL KLM CLASS OPERATORS
K - Pressing a key
P - Pointing to a location on screen with the mouse
H - Moving hands to home position on the keyboard
M – Mentally preparing to perform an action
R – System response where the user waits for the system.
For each operator, there is an estimate of execution time- A
set of heuristic rules is accounted for the mental preparation
time.
5. HOW THE SETUP WAS PERFORMED
VIA KLM-FORM ANALYSER
Firstly, there are a set of mental operator rules. This is known as
the “mental preparation”.
There are set of rules that are set before the test. In a web-form
task, there are two activities and they are:
1. Reaching the element using the mouse
2. Using the keyboard once the element is reached
Appendix A, Figure 1 shows a set of rules that need to be
followed in order to process the elements.
The operations consisted of using the mouse and keyboard.
Using the mouse to reach the button then use keyboard to enter
text, using the mouse to reach the text box and using mouse to
reach the text area.

3. 6. HOW THE RESULTS WILL BE
ANALYSED
Before conducting the test, predefined time values were set for the
KLM operators. This is where predictions are made on how long
it would take to reach the element on the page and fill the form
out.
Every form element requires KLM operators (a sequence of
required actions).
KLM Operators Predictions
M - Mental Preparation: This is the taken for a
user to make a decision or think about the next
move.
1.2 seconds
P – Time to point an object with the mouse: The
time depends on the size and distance of the target.
1.1 seconds
B – Button press or release:
This is straightforward; there would be no need to
hold the click button for more than 0.1 seconds.
0.1 seconds
H – Hand from keyboard to mouse or vice
versa: This is straightforward; there would be no
need to hold the click button for more than 0.1
seconds
0.1 seconds
W – System response: This depends on the
system and the command. It is used when the
user has to wait for the system.
0
6.1 THE TWO ELEMENTS THAT ARE BEING
RECORDED VIA KEYSTROKES
1. Time taken to reach the element (ReachTime)
2. Time taken to enter text [keystrokes] (ManipulationTime)
6.2 ANALYSED RESULTS
6.2.1 TIME TAKEN TO REACH ELEMENT
Figure 3 shows the first element. Cursor is not in the element, user
hands are on keyboard and navigation is performed using mouse.
User moves hands to mouse (H), thinks (M) and points to the
element (P). Furthermore, user clicks to the element (2*B), since
it is editable, and the cursor is required to be in it.
KLM Sum up: H+M+P+2*B
Total time: 2.90
6.2.3 TIME TAKEN TO MANIPULATE ELEMENT
Element is editable therefore manipulation is performed using
keyboard. Since navigation is performed using mouse, user moves
hands on keyboard (H) and fills the element (+5*K).
KLM sum up: H+10*K
Total time: 1.20
6.2.4 SUBMIT BUTTON REACH TIME
When the form was filled out, the submit button required two
operators: M+P (Mentally prepare + Point to object). It took 2.30
seconds – This was the correct estimate as M was expected to take
1.2 seconds and P was estimated to be 1.1 seconds.
See Appendix B, Table 2.
Total time: 1.2 + 1.1 = 2.30 seconds
6.2.5 SUBMIT BUTTON MANIPULATION TIME
After the form was filled out and the user was mentally prepared
and pointed to the object, the user then had to press click the
submit button (B). Total: 20 seconds
Table 1: Prediction table for KLM method
Figure 3: Dropbox Registration
Form

4. REFERENCES
[1] Dix, A. (2004). Human-computer interaction. Harlow,
England: Pearson/Prentice-Hall, p.321.
[2] Cs.umd.edu, (2016). GOMS and Keystroke-Level Model.
[online] Available at:
http://www.cs.umd.edu/class/fall2002/cmsc838s/tichi/printer
/goms.html [Accessed 3 Mar. 2016].
[3] The Interaction Design Foundation, (2016). Fitts's Law.
[online] Available at: https://www.interaction-
design.org/literature/book/the-glossary-of-human-computer-
interaction/fitts-s-law [Accessed 3 Mar. 2016].
[4] Kieras, D. (2001). Using the Keystroke-Level Model to
Estimate Execution Times. [online]
KierasKLMTutorial2001.pdf. Available at: http://www-
personal.umich.edu/~itm/688/KierasKLMTutorial2001.pdf
[Accessed 10 Mar. 2016].

5. APPENDIX A
APPENDIX B
Table 2: Analysed Elements
Reach Time Operations: How the user
operated to reach the element.
Manipulation Time Operations: How the
user operated to enter data (10 keystrokes)
Reach Time Value: How long it took the
user to reach the element
Manipulation Time Value: How long it took
the user to finish.
Figure 2 – Predetermined valuesFigure 1 – KLM Rules