The main objective of this study was to identify potential usability problems, interaction differences, advantages, and disadvantages of two versions of a nursing documentation system: PDA and Tablet PC. A comparative study of tasks completion was made between these systems. The dependent variables of this study were tasks completion time, number of tasks completed, and user satisfaction. No significant differences were found in completion time of individual tasks between both systems. Significant differences in user satisfaction ratings for the use of the stylus, weight, and portability were found. No significant differences were found in the satisfaction of the nurses with interaction and system aspects between the two nursing documentation versions. The results of the study support the conclusion that PDAs are a better alternative for supporting nursing documentation tasks at bedside than Tablet PCs.

1. A Comparative Study of Nurses
Accessing Electronic Patient Record Systems
with PDAs and Tablet PCs
Gilberto Crespo Pérez
Electrical & Computer Engineering Department
University of Puerto Rico at Mayagüez
December 2005

2.  Hospitals in Puerto Rico & United States keep their
patients’ records in paper form.
– Problems with information management effectiveness.
– Handwriting may produce errors due to misunderstanding of
written information.
 Compromise patients’ health.
 PDAs & Tablet PCs are two technological devices with
potential for accessing and capturing clinical data at
bedside.
– Allow greater speed and effectiveness to compile patient’s
information.
– Reduce the risk of errors during treatment of patients.
– Improve physicians and nurses’ effectiveness and productivity at
point-of-care, as well as the quality of patient care.
Introduction

3.  A review of the literature reveals that there are
no formal studies comparing the use of PDAs
and Tablet PCs for accessing electronic patient
records.
 We believe that such study could help determine
which of the two technologies fits better for
collecting and accessing clinical information at
the point-of-care.
Introduction (Cont.)

4. Previous Work
 From 44,000 to 98,000 people die annually in U.S.
hospitals as a result of medical mistakes -- more than
half of them preventable [Rosenbloom03].
 Trying to correct some of these errors, the Veteran’s
Administration (VA) has taken effective measures
through electronic system controls and
implementation [Rosenbloom03]
– Over a five-year test period, the medication error rate
dropped 70%

5. Previous Work (Cont.)
 The Use of Portable Devices in Health Care:
 E-prescribing, ordering, checking labs tests, dictation
notes among others - [Fischer03, Ying03,
Rosenbloom03, Barret02,].
 Clinical Applications for Portable Devices:
 Drug references, pharmacopoeias, medical
calculators, and patient trackers among others -
[Adatia03, Rosenbloom03, Choi00, Kimura03, van
der Velde01, Berner04].

6. Previous Work (Cont.)
 Usability Studies of Portable Devices in Health Care:
 PDA vs Laptop & PDA vs Paper Based [Rodríguez02,
Rodríguez03, Rodríguez04, Staggers00, Stausberg03].
 Users performance, subjective satisfaction, and preference were
dependable variables.
 Usability of Tablet PC [Narayan04, Andon04].
 Physical Aspects.
 No more studies related to other usability issues of Tablet PCs
have been found, which indicates that this field is in its early
stages of research.

7. Objectives
 Conduct a usability study to compare the use of
PDA and Tablet PC-based applications to
support nurses’ tasks at point-of-care.
– These studies compare both systems in terms of
performance parameters:
 Time to complete tasks.
 Number of tasks completed.
 Subjective user satisfaction.
– Identify:
 Potential usability problems.
 Interaction differences.
 Advantages and Disadvantages.

8. Systems
 Tablet PC version
– Implemented on a Gateway tablet.
– Windows XP Tablet PC OS.
– Uses a Stylus as a pointing device.
– A soft keyboard provided by OS was used for text input.
 PDA version
– Implemented on a HP iPaq 5500.
– Windows Pocket PC 2003 OS.
– Uses a small stylus as a pointing device.
– A soft keyboard provided by OS was used for Text input.

9. Systems (Cont.)
 The Tablet PC version
– Developed in Java.
 The PDA version
– Developed in C Sharp.
 Both applications use:
– MSSQL to store and retrieve the patient’s clinical
information in a database.
– The standard IEEE 802.b to communicate with the
database server.

10. Graphical User Interface
Patient List Windows

11. Graphical User Interface (Cont.)
Medicine Windows

12. Graphical User Interface (Cont.)
Intake/Output Windows

13. Participants
 20 staff nurses
– Selected on a first-come first-served basis from those that
responded a call for participation.
– Experience as staff nurses ranged from 1 to 27 years
(Mean=12.3 years).
– Experience with computers ranged from 0 to 9 years
(Mean=4.6 years).
– On average, they used computers on their job for 3.8 hours
per day.
– None of them had prior experience with the systems used
for the study or with any similar application.
– None of them had experience using PDAs or Tablet PCs.

14. Experimental Design
 Participants were asked to fill out a questionnaire about
their work experience and their experience using
computers.
 They were asked to sign a letter of consent.
 An orientation script was read to each participant
explaining the objective of the test.
 Nurses were given a short tutorial session on the graphical
system.
– 18 minutes for the PDA version.
– 13 minutes for the Tablet PC version.
– They were allowed to use the system by themselves and were
guided to practice using each of the functions of the system.

15.  Half of the nurses performed the tasks first on
one system and the other half started on the other
system.
 Once the participants complete all the task in
both versions, they were asked to fill a user
satisfaction questionnaire.
Experimental Design (Cont.)

16. Tasks
1. Indicate patient’s age and weight. Say them aloud.
2. Indicate the most recent registered patient’s temperature.
3. Look for the most recent nurse note and read it aloud.
4. Acknowledge any pending medication order as
administered.
5. Enter the following set of vitals signs:
Temp: 37.0o
C; BP: 130/90; Pulse: 71; RR: 18; O2 Sat: 96%.
6. Indicate the total balance of intake/output of fluids in the
last 24 hours.
7. Look for a Dr. Colón note.
8. Enter the following patient assessment information
provided.
9. Enter the following I/O information:
Intake PO: 50 ml; Output Urine: 650 ml;

17. 10. Enter the following text as a note:
Patient presents fever and a large lung mass on left upper
lobe consistent on CT with multi-focal pneumonia
11. Enter the following patient’s pain information.
Classification: Four; Location: Right frontal shoulder;
Description: Hurt; Therapy: Massage; Administered
medicine.
12. Look for most recent physician consult order and
acknowledge it as that was notified by phone.
13. Show where is indicated in the record the reason why a
dose of Roboxine was omitted.
Tasks (Cont.)

18. User Satisfaction Questionnaire
 Interaction Aspects:
– Look up patient information
– Acknowledge medication orders
– Acknowledge consult order
– Enter vitals signs
– Document intake/output
– Document patient assessment
– Document pain assessment
– Read a note
– Write a note
Nurses were asked to rate on a 1 – 7 scale (1 being poor
& 7 being excellent)

19.  System Aspects:
– Record organization
– Trustworthiness of information
– Precision of Information
– Accessibility of information
– System security
 Physical Aspects:
– Use of the stylus
– Use of the screen keyboard
– The screen
– The weight
– The portability
 Level of Satisfaction with each system
 The system they would prefer for doing their
nursing documentation.
User Satisfaction Questionnaire (Cont.)

20. Statistical Analysis
 Dependent variables were:
– Task completion time.
– Number of tasks completed.
– Subjective user satisfaction.
 Time analysis – Paired t Test
 User Satisfaction – Wilcoxon Signed Ranks
 Number of Task Completed – Wilcoxon Signed
Ranks

21. Results: Overall Completion Times
656.2 667.3
191.7
215.6
0.0
100.0
200.0
300.0
400.0
500.0
600.0
700.0
Seconds
Mean Std. Deviation
PDA
Tablet PC

22.  Dependent Sample t test did not reveal
significant difference between average
time it took the participants to complete all
the tasks on both system.
Results: Overall Completion Time (Cont.)

23. 0
50
100
150
200
250
CompletionTime(Seconds)
1 2 3 4 5 6 7 8 9 10 11 12 13
Tasks
PDA
Tablet PC
Results: Completion Times for Each Task
on Both Versions of the System

24.  Dependent Sample t test did not reveal
significant difference in the time it took
the participants to complete each of the
tasks on both versions of the system.
Results: Completion Times for Each Task
on Both Versions of the System (Cont.)

25. Results: Number of Participants that Complete
Each Task on Each Version of the System
0
2
4
6
8
10
12
14
16
18
20
NumberofParticipants
1 2 3 4 5 6 7 8 9 10 11 12 13
Tasks
PDA
Tablet PC

26.  Wilcoxon Signed Ranks did not reveal
significant difference in the total number
of tasks completed by the participants.
 The participants completed an average of
12.0 tasks on the PDA and 11.7 on the
Tablet PC
Results: Number of Participants that Complete
Each Task on Each Version of the System (Cont.)

27. Results: Average Satisfaction Ratings
for Individual Interaction Aspects
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0
Rating
Look up patient information
Acknowledge medication orders
Acknowledge consult order
Enter vitals signs
Document intake/output
Document patient assessment
Document pain assessment
Read a note
Write a note
InteractionAspects
PDA Tablet PC

28.  Wilcoxon test did not reveal significant a
difference on each individual interaction
aspects considered.
 Also, Wilcoxon Test did not reveal a
significant difference in the overall
average satisfaction rating given to the 9
interaction aspects.
Results: Average Satisfaction Ratings
for Individual Interaction Aspects (Cont.)

29. Results: Average Satisfaction Ratings
for Individual System Aspects
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0
Rating
Record
organization
Trustworthiness
of information
Precision of
Information
Accessibility of
information
System
security
SystemAspects
Tablet PC
PDA

30.  Wilcoxon Test did not reveal a significant
difference in the overall average
satisfaction rating given to the 5 system
aspects.
 The overall average satisfaction rating for
the PDA and the Tablet PC was 6.8
Results: Average Satisfaction Ratings
for Individual System Aspects (Cont.)

31. Results: Average Satisfactions Ratings
for Individual Physical Aspects
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0
Rating
Use of the stylus
Use of the screen keyboard
The screen
The weight
The portability
PhysicalAspects
Tablet PC
PDA

32.  Wilcoxon Test revealed a significant difference
in overall satisfaction with the five physical
aspects considered.
 Wilcoxon Test also revealed significant user
satisfaction rating for:
– The use of the Stylus
– The weight of the device
– The portability of the device
Results: Average Satisfactions Ratings
for Individual Physical Aspects (Cont.)

33.  Participants were more satisfied with:
– The PDA computer (Average Rating = 6.6) than with the
Tablet PC computer (Average Rating = 5.2).
– The use of the stylus on the PDA (Average Rating = 6.7)
than on the Tablet PC (Average Rating = 6.2).
– The weight of the PDA (Average Rating = 6.5) than with
the weight of the Tablet PC (Average Rating = 4.1).
– The PDA portability (Average Rating = 6.6) than with the
Tablet PC portability (Average Rating = 3.5).
Results: Average Satisfactions Ratings
for Individual Physical Aspects (Cont.)

34. Overall Results
 No significant differences were found in theNo significant differences were found in the
overall satisfaction rating given by theoverall satisfaction rating given by the
participants to each version of the system.participants to each version of the system.
 However, the majority of the participantsHowever, the majority of the participants
preferred the PDA over the Tablet PC versionpreferred the PDA over the Tablet PC version
for performing their duties.for performing their duties.

35. Discussion
 No significant difference was found in the overall time it
took nurses to complete tasks on both portable devices.
– It contradicts our expectations that nurses would be faster using
a larger user interface than a smaller one.
 Our results are consistent with the results of the study by
Rodríguez et. al. [Rodríguez03]
– When the task of writing a note was not considered, nurses were
able to complete nursing tasks in similar overall times on the
PDA and on Laptop.
 The use of a stylus, the screen size, and resolution is not
a factor in nurses’ performance.

36. Discussion (Cont.)
 Nurses were as fast on a PDA as they would on a
Tablet PC.
 Younger nurses tend to take shorter times in
completing tasks.
 Due to the short training given to nurses to teach
them how to use the systems and the high percentage
of tasks completed:
– Results indicate that both nursing documentation systems
are very easy to learn .
– Similar results were found in the [Rodríguez03] study.

37. Discussion (Cont.)
 The word “document” in tasks T4, T6, T8, and T12
seems to create confusion in some of the nurses to the
point that they did not find the way to complete the task
successfully.
 Nurses expressed similar high levels of satisfaction with
the documentation activities performed in both, the PDA
and Tablet PC systems.
 They were significantly more satisfied with the physical
aspects of the PDA than those of the Tablet PC

38. Discussion (Cont.)
 Many of the participants (12 out of 20) expressed
discomfort in holding the Tablet PC in a standing
position during the test session.
– This result is consistent with the studies conducted by:
 Michael A. Narayhan [Narayhan04]

Christopher L. Andon [Andon04]
 Fourteen of the nurses indicated that they preferred
the PDA version for performing nursing
documentation tasks while only four preferred the
Tablet PC.

39. Conclusions
 Our results supports that it is possible to design PDA-
based applications that allow nurses to achieve similar
performance and satisfaction levels as with a Table PC-
based.
 Screen size and display resolution of the PDA are no
factors that limit nurses’ performance and satisfaction in
comparison to Tablet PCs.
 Nurses were significantly more satisfied with the physical
aspects of the PDA than with those of the Table PC.
– Aspects such as the use of the stylus, the weight, and portability
are factors that influence nurses’ satisfaction with the system.

40. Conclusions (Cont.)
 These applications exhibited a high degree of
learnability.
 Differences on user interfaces have not affected users
satisfaction.
 Usability engineering principles and guidelines
– Are critical part of the graphical user interface success.
– Proved to be powerful and important tools for measuring the
efficiency of the system.
 The age seems to be a factor that affects nurses’
performance..

41. Conclusions (Cont.)
 Considering that:Considering that:
– Satisfaction with the physical aspects of the systemsSatisfaction with the physical aspects of the systems
was the only dependent variable for which awas the only dependent variable for which a
significant difference was found in favor of the PDAsignificant difference was found in favor of the PDA
– That 14 out of 20 participants selected the PDA overThat 14 out of 20 participants selected the PDA over
the Tablet PCthe Tablet PC
– And the relatively low costs of the PDAAnd the relatively low costs of the PDA
We can conclude that PDAs are a better alternativeWe can conclude that PDAs are a better alternative
for supporting nursing documentation tasks atfor supporting nursing documentation tasks at
bedside than Tablet PCs.bedside than Tablet PCs.

42. Future Work
 Other usability studies should be conducted:
– After the users have had a year of experience with the system.
– With physicians interacting with each version of the system used
on this research.
 Application improvement:
– Incorporation of printing and supplemental language options
capabilities.
– Speech recognition functionality.
– Convert to a web based application.

43. Questions/CommentsQuestions/Comments

44. Usability Concepts Definitions
 Usability: the extent to which the intended user can meet his or her
goals using the system being tested.
 Learnability: easy to learn -> time a novice user takes to complete
tasks using the system.
 Efficiency: high level of productivity -> time that users with certain
expertise take to complete typical tasks.
 Memorability: easy to remember.
 Errors: Low error rate and easy to recover.
 Satisfaction: users are subjectively satisfied -> ask users opinions.
When replies from multiple users are average, the result is objective.
-questionnaire with Likert scale.

45. General Concepts Definitions
 Ethical Aspects with Human Subjects:
– Respect
– Comfortable environment
– Confidentiality
– Early success experience

46. Statistical Tools Definitions
 Dependent-samples t test: used to compare the
time to complete the tasks -> same users, 2
measurements.
 Wilcoxon Signed Rank test: compare differences
in user satisfaction -> dependent samples.
 Correlation analysis: determine associations
between time to complete the tasks, computer
literacy, typing skills, age, and eye glasses, among
others.
 Linear regression: evaluate the learning effect
between 2 systems.