Paper presented at HCI Int'1995. According to human behavior studies, several disciplines (e.g., cognitive psychology, software ergonomics, visual design) have brought substantive results to improve the user friendliness of user interface (UI). One possible output of these disciplines come as recommendations that could be translated into ergonomic rules (or guidelines). Guideline knowledge is often contained in five sources : recommendation papers [1], design standards (e.g., ISO 9241 [2]), style guides which are specific to a particular environment (e.g., IBM Common User Access [3]), design guides (e.g., Scapin's guide [4], Vanderdonckt's guide [5]) and algorithms for ergonomic design (e.g., automatic selection of interaction objects [6]). Studies carried out with designers show that these guidelines are difficult to apply at design time:  average search time for a guideline in a design guide lasts 15 minutes [1];  about 58% of designers succeed to find guidelines relevant to their problem [1];  designers do not respect about 11% of guidelines [7];  designers experienced interpretation problems for 30% of guidelines [7].

1. LISLIS Université Toulouse I Université de Namur
User Interface Evaluation:
Is It Ever Usable ?
Christelle FARENC,
Philippe PALANQUE,
University of Toulouse I, (France)
Jean VANDERDONCKT,
University of Namur FUNDP, (Belgium)

2. LISLIS Université Toulouse I Université de Namur
Outline
 Design of a Dialog Box
 Design Evaluation
 Problems with the Evaluation Report
 Knowledge organisation
 Avantages of this organisation
 Implementation issues

3. LISLIS Université Toulouse I Université de Namur
Design of a Dialog Box (1)
 Requirements for a T-uple editor
– Object : T-Uple =
 a textual attribute
 a number between 1 and 3
 a boolean
– Actions :
 add / delete
 modify
 reset

4. LISLIS Université Toulouse I Université de Namur
Design of a Dialog Box (2)
designer/evaluator's viewUI seen by EVALUATOR
Factual UI description
User Interface (UI) UI seen by USER
user-centreddesignuser-centreddesign
UI seen by PROGRAMMER

5. LISLIS Université Toulouse I Université de Namur
Design of a Dialog Box (3):
programmer's view
UI seen by USER
user-centreddesign
Interaction Object
UI norm (standard, style guide)
UI seen by EVALUATOR
UI seen by PROGRAMMER
Factual UI description
User Interface (UI)
UI IO values
user-centreddesign
programming
iterativedesign
Interaction ObjectInteraction Object

6. LISLIS Université Toulouse I Université de Namur
Design of a Dialog Box (4):
programmer's result
 Dialog Box with widgets for managing
T-uples

7. LISLIS Université Toulouse I Université de Namur
Design Evaluation (1)
 List of unrespected ergonomic rules
– R1=Display label for each widget presenting data,
– R2=Text should be legible,
– R3=Provide clear visual distinction between areas having
different functions,
– R4=User should quit the application easily,
– R5=User should be able to adapt display to own requirements,
– R6=Allow shortcuts (for push buttons),
– R7=Indicate inactive actions with different format ,
– R8=Dialog box should not be resizable,
– R9=Dialog box should not have maximize/minimize icons.

8. LISLIS Université Toulouse I Université de Namur
Design Evaluation (2)
 Ergonomic Criteria
– Prompting : R1
– Legibility : R2
– Grouping : R3
– Explicit User Action : R4
– User Experience : R5, R6
– Distinction by Format : R7
– Minimal Actions : R8, R9

9. LISLIS Université Toulouse I Université de Namur
Design Evaluation (3)
 Factors
– Effectiveness
 Explicit User Action, Minimal Actions
– Efficiency
 Prompting, Legibility, Grouping, Distinction by
Format
– Satisfaction
 User Experience
– Suitability

10. LISLIS Université Toulouse I Université de Namur
Design Evaluation (4)
 Factors of Utility
 Effectiveness
 Suitability
 Factors of Usability :
 Efficiency
 Satisfaction

11. LISLIS Université Toulouse I Université de Namur
Design Evaluation : summary
Utility
SuitabilityEffectiveness
Usability
Efficiency Satisfaction
Explicit User
Action
Minimal
Actions
Prompting Legibility Grouping Distinction
by Format
User
Experience
R1 R2 R3 R4R5 R6R7 R8 R9
User Interface (UI)
No Mnemonic No quitPositionsNo label
List Box X,Y,Z Edit Boxes Add Push Button
T-Uple Editor Ms-Windows norm

12. LISLIS Université Toulouse I Université de Namur
Problems with
Evaluation Report
 Evaluator, Programmer and User all have
separate views of the same UI
– Evaluator : usability problems
– Programmer : programming problems
– User : task problems
 No intersection
– Evaluator : hard to evaluate in terms of widgets
– Programmer : hard to reuse evaluation in terms of factors
– User : hard to tell problems in terms of both widgets and factors

13. LISLIS Université Toulouse I Université de Namur
Knowledge organisation
Usability /Utility Dimensions
Factor
Ergonomic criteria
Ergonomic rule
Real Value
Interaction Object
UI norm (standard, style guide)
UI seen by EVALUATOR
UI seen by PROGRAMMER
Factual UI description
User Interface (UI)
UI IO values
UI seen by USER
user-centreddesign
expertise
report
programming
iterativedesign
Ergonomic criteriaErgonomic criteria
Ergonomic ruleErgonomic rule . . .
Value ValueReal Value
Interaction ObjectInteraction Object
Ergonomic rule
. . .
Factor Factor
user-centreddesign
Main Intersection

14. LISLIS Université Toulouse I Université de Namur
Advantages of the organization
 Reusability of evaluation report for
programmer
 Expressiveness of usability problems for
evaluator
 Independence of Ergonomic Rules for any
target environment
 Clear statement of utility/usability goals at
both design and evaluation time

15. LISLIS Université Toulouse I Université de Namur
Implementation Issues
 ERGOVAL
– surface evaluation
– rule base of ergonomic rules expressed in terms of widgets
abstraction (graphical objects)
– structural decomposition of UI
 TRIDENT
– automatic generation of UI
– rule base of ergonomic rules expressed in terms of widgets
abstraction (abstract interaction objects)
– structural decomposition of UI
 Possible coupling

16. LISLIS Université Toulouse I Université de Namur
That's all folks!