1. Evaluation of Text Editors
Roberts T, Moran T. 1982

2. Methodology
• Dimensions
– Time for experts to edit (PERFORMANCE)
– Errors (experts) (PERFORMANCE)
– Learning the basics (novices) (PERFORMANCE)
– Functionality
– Objectivity
• Not biased by editor’s conceptual model
• Observing people equally familiar
– Thoroughness
– Ease-of-use

3. • Over 200 tasks that can be performed, covers
modifying, layout, special (e.g. tables) etc
• Performance – 32 core tasks
• (insert, delete, etc) a (character, word,
sentence etc), locate something, + misc.
• Time – 50 tasks, randomly distributed. Small
tasks most normal
• 4 total participants 1 or 2 non tech, 1 or 2
tech.
• Stopwatch

4. Time
• +/- 30 minutes
• Score: Avg error free time minus the time
correcting errors.
• Keystroke-level-model: «predicts editing time
by counting physical and mental operations in
a task -> theoretical standard of expert
performance

5. Errors
• Not typographic errors, errors more than a
few sec.
• Also tasks incorrectly performed
• Score error time in % of error-fre time.

6. Learning
• Teaching four novices, even no computer xp.
• One on one orally
• one page summary sheet (to avoid «simple
memory difficulties»)
• Quiz – a marked document
• Not all tasks are neccesarily taught –> figure out
• Breaks between 5 cycles.
• Score: num of tasks learned divided by time to
learn the task (avg of 4)

7. Functionality
• Over 200 tasks checklist.
• Only checked if done efficiently
• Half credit if done «clumsily», none if not
possible.
• Score % of total number

8. Empirical Results
• TECO, WYLBUR, WANG, NLS, EMACS, BRAVO,
BRAVOX, GYPSY (last three experimental)
• Can be seen as a comparison of editors,
evaluation of the method itself or general
behaviour data

9. Comparison
None superior =>
tradeoffs

10. Time
• Gypsy fastest
• «Display-based»
(WYSIWYG?) fastest

11. Errors
• Keystroke-level-model
approx 70% of average
• X5 between best and
worst
• However this is small so
no conclusions

12. Learning/speed
• Emacs programmer’s
choice
• NS and BRAVOX
formatting+layout

13. Functionality

14. NLS wins
Taxonomy more weighted towards layout

15. Evaluation of methodology
Successfully provide an objective,
multidimensional picture. Approx 1 week.
Minus for occational users, more reliable
measures are needed to differentiate.

16. Behavioral results
• 30 seconds core editing for experts
• Two hours of training
• Error rates had greatest individual difference
(0-39)
• The slower users make more errors
• Only moderate variation among experts in
speed
• Same range between novice learners (1.5-2)