The document describes KLM-Form Analyzer (KLM-FA), a tool that uses human performance models to automatically evaluate the efficiency of web form filling tasks. KLM-FA predicts the time and actions required to complete forms based on modeled user profiles and interaction scenarios. It was validated against user testing data and shown to have error rates mostly within 20%. Two case studies further demonstrated KLM-FA's usefulness for benchmarking many forms and redesigning a form to significantly reduce completion times. The tool aims to provide an efficient, cost-effective method for early-stage form design and evaluation.

1. KLM Form Analyzer: Automated Evaluation
of Web Form Filling Tasks using Human
Performance Models
Interact 2013, Cape Town, 2-6 September 2013 1
Christos Katsanos | ckatsanos@eap.gr
Nikolaos Karousos | karousos@eap.gr
Nikolaos Tselios | nitse@ece.upatras.gr
Michalis Xenos | xenos@eap.gr
Nikolaos Avouris | avouris@ece.upatras.gr

2. Motivation & Purpose
 Usability of interactive forms is a critical aspect
of Web UX
 Form filling is a data entry task
 Task efficiency is important
 User efficiency affected by many factors
 e.g. fields’ size/position/type, interaction
device/strategy, age, typing expertise
 Lack of easy-to-use, efficient and flexible
simulation tools for practitioners
Interact 2013, Cape Town, 2-6 September 2013 2

3. KLM-FA
Algorithm
Our approach:
KLM-Form Analyzer (KLM-FA)
Interact 2013, Cape Town, 2-6 September 2013 3
Web Form
Modeled User Profile
(age, typing expertise)
Interaction Scenarios
(devices, initial state,
keystrokes, fields to fill)
Modeling parameters
(rules, values, adjustments)
or
Web Forms
Parsing Module
(forms & elements
identifier)
Analysis Module
(Keystroke Level
Model, Fitts’ Law)
Task-time
prediction
Trace model
steps
Simulated
actions
Save/Export
Project
Output

4. KLM-FA tool: Main Interface
Form URL
Preview Browser
Synchronized
Views Results Panel
Point time
calculations
Analysis preferences
(modeled user, interaction
scenarios)
Analysis
parameters
Mass-scale
evaluation
Trace step-by-
step modeling

5. Interact 2013, Cape Town, 2-6 September 2013 5
Flexible: Test different interaction
scenarios
Form Fields
to be filled
Modeled user profile
(e.g. old & poor typist)
Interaction strategy (e.g.
tab-based navigation)
Initial state (e.g. cursor position
in first field after form loads)

6. Interact 2013, Cape Town, 2-6 September 2013 6
Flexible: Employ rules & change
modeling parameters

7. Efficient: Mass scale evaluation
(saved as XML file)
Interact 2013, Cape Town, 2-6 September 2013 7

8. Interact 2013, Cape Town, 2-6 September 2013 8
Efficient: TextField names and
default keystrokes mappings
Research-based Defaults

9. Studies
Interact 2013, Cape Town, 2-6 September 2013 9
1) Validation study: KLM-FA VS User testing
2) Case Study1: Benchmarking with KLM-FA
3) Case Study2: Form redesign with KLM-FA

10. Validation study: Methodology
Interact 2013, Cape Town, 2-6 September 2013 10
 KLM-FA predictions VS User testing data
 6 form-filling tasks (3 forms x 2 interaction strategies)
 3 signup forms of social networking sites: Facebook | Myspace | Twitter
 2 interaction strategies: mouse-only* | keyboard only
* except for input in text fields
 15 participants (12 male, mean age 27, mean of 42wpm#)
#as measured by a typical typing speed test
 10 trials per task (to allow users to reach skilled performance)
 In-house developed software for experiments (interaction logging,
conditions counterbalancing, instructions)

11. Validation study: Results (1/2)
Interact 2013, Cape Town, 2-6 September 2013 11
30.7
27.3
33.2
29.6
22.5 23.1
35.3
25.6
39.1
28.3
26.4
24.2
0
5
10
15
20
25
30
35
40
45
FB_mouse FB_keyboard MS_mouse MS_keyboard TW_mouse TW_keyboard
Tasktime(sec)
KLM-FA vs User testing
User testing
KLM-FA
*Error bars inuser testing data represent 95% confidenceintervals

12. Validation study: Results (2/2)
Interact 2013, Cape Town, 2-6 September 2013 12
Signup
form
Interaction
strategy
KLM-FA
Error rate
Facebook Mouse-based -14.9%
Facebook Keyboard-based 6.1%
Myspace Mouse-based -17.6%
MySpace Keyboard-based 4.5%
Twitter Mouse-based -17.5%
Twitter Keyboard-based -4.7%
Error rate =
Users′ time − KLM − FA time
Users′time
 KLM-FA tended to
 overestimate (16.7% on average) task time in mouse-based interaction
 underestimate (5.1% on average) task time in keyboard-based interaction
Within the 20% margin of error
(Card et al. 1980; Ivory, 2001)

13. Two Pilot Case Studies
 Benchmarking study
 Time to register to a social networking
site?
 16 popular sites (e.g. Facebook,
Google+)
 30 mins to do it (<2 mins per signup
form)
Mouse-
based
Keyboard
-based
Mean 33.64 23.25
SD 13.54 11.46
Min 14.12
(LinkedIn)
5.92
(LinkedIn)
Max 62.25
(Flickr)
46.48
(Google+)
 Form redesign
 Webmaster unfamiliar with HCI models & our tool
 New design: decrease in signup time of 55.8% (mouse) and
60.6% (keyboard)
 Comments: “intuitive and easy to use”, “valuable asset to
effective form design”

14. Discussion - Advantages
Interact 2013, Cape Town, 2-6 September 2013 14
 Mass scale evaluation/benchmarking
 Cost-effective (time & resources)
 Can be used early in the
design/evaluation cycle
 Simple to learn and apply
 Can be used for educational
purposes
Possibility for
wider adoption

15. Future Research
 Conduct additional validation studies
 Teaching HCI with KLM-FA & students’ learning
outcome
 Add enriched models of KLM and Fitts (e.g.
stochastic model of errors)
Interact 2013, Cape Town, 2-6 September 2013 15

16. Summary
 Proposed an approach that predicts the actions and
time required for form filling tasks
 Case studies depicted the usefulness of the proposed
tool
 Free + Fast + Easy = Possibility for wider adoption
Complementary to user-based methods
Interact 2013, Cape Town, 2-6 September 2013 16

17. Tool free to download and use
Try it out! – Questions?
http://klmformanalyzer.weebly.com
Interact 2013, Cape Town, 2-6 September 2013 17
http://quality.eap.gr
http://hci.ece.upatras.gr