Validity of Using the Assessment of Motor and ProcessSkills .docx

Validity of Using the Assessment of Motor and Process
Skills to Determine the Need for Assistance
Brenda K. Merritt
KEY WORDS
� activities of daily living
� disability evaluation
� motor skills
� sensitivity and specificity
� task performance and analysis
Brenda K. Merritt, PhD, OT Reg(NS), OTR, is
Assistant Professor, School of Occupational Therapy,
Dalhousie University, Room 215, 5869 University Avenue,
PO Box 15000, Halifax, Nova Scotia B3H 4R2 Canada;
[email protected]
OBJECTIVE. This study evaluated the validity of using
Assessment of Motor and Process Skills (AMPS)
measures as evidence of the need for assistance in the
community.
METHOD. In a retrospective analysis of existing data (N 5
64,466), receiver operating characteristic
(ROC) curves were generated, and a split-sample method was
used to validate the predictions.

RESULTS. When identifying people who need assistance versus
those who do not need assistance in the
community, activity of daily living (ADL) motor and ADL
process measures have fair and good discriminating
value, respectively (areas under the ROC curves were .78 and
.84). Evidence supports placing ADL motor and
ADL process independence cutoff measures at 1.50 logits
(sensitivity 5 .67, specificity 5 .72) and 1.00
logit (sensitivity 5 .81, specificity 5 .70), respectively.
Accuracy was highest when matched motor and
process decisions occurred (sensitivity 5 .85, specificity 5 .83).
CONCLUSION. Evidence supports using ADL ability measures
from the AMPS to provide evidence of
a client’s need for assistance in the community.
Merritt, B. K. (2011). Validity of using the Assessment of
Motor and Process Skills to determine the need for assistance.
American Journal of Occupational Therapy, 65, 643–650. doi:
10.5014/ajot.2011.000547
E
very day, health care professionals conduct evaluations and
make recom-
mendations regarding the most appropriate discharge plans and
living
environments for their clients. The overarching goal in this
process is to ensure

that clients can safely and efficiently manage, either
independently or with the
assistance of another, all the daily life tasks that are required
for independent
living (e.g., self-care, home maintenance, money management,
medication
adherence, leisure, social interaction, community transportation;
e.g., Wehman,
2001).
In their exploration of this complex evaluation process,
Meinow, Kåreholt,
and Lagergren (2005) found that the strongest predictors of the
amount of
assistance received in the home were dependency in personal
activities of daily
living (PADLs; R 2 5 .34, p < .001), dependency in instrumental
activities
of daily living (IADLs; R 2 5 .39, p < .001), and cognitive
function (R 2 5 .13,
p < .001). Because PADL ability and IADL ability (hereinafter
jointly referred
to as activity of daily living [ADL] ability) are two of the key
predictors of
the need for assistance in the community, occupational

therapists play a vital
role in this evaluation process. If occupational therapists are to
contribute to the
decision-making process from an evidence-informed
perspective, additional
research must be conducted to document the validity and
accuracy of using
ADL measures to predict the need for assistance in the
community.
The Assessment of Motor and Process Skills (AMPS; Fisher,
2006a, 2006b)
is an internationally recognized, occupational therapy–specific
assessment of the
quality of a client’s ADL performance. The linear ADL process
ability scale of
the AMPS has shown promise when used to identify clients who
need assistance
The American Journal of Occupational Therapy 643
to live in the community (Bernspång & Fisher, 1995;
Fisher, 2006a; Hartman, Fisher, & Duran, 1999; Kottorp,
2008; Merritt, 2010); however, additional research is

needed to determine the sensitivity and specificity of the
AMPS ADL cutoff measures. Sensitivity is an estimate of
the true positive rate and within this context represents
the proportion of the independent sample correctly
identified as independent. In contrast, specificity is an
estimate of the true negative rate, or the proportion of the
sample in need of assistance that is correctly identified as
needing assistance. Identifying the sensitivity and speci-
ficity estimates that arise when using AMPS ADL ability
measures to predict the need for assistance could provide
occupational therapists with additional evidence to sup-
port their clinical judgments. For example, this evidence
could potentially be used to acquire support from fund-
ing agencies to pay for a client’s care, establish the need
for additional health care services to maximize a client’s
independence and lessen caregiver burden, and document
the efficacy of occupational therapy services and the im-
pact they have had on the client’s need for assistance and
burden of care.

To address this gap in the literature, this study
addressed three issues: (1) whether significant association
exists between ADL ability (motor ability, process ability,
or both) and global functional level, (2) the sensitivity and
specificity estimates that arise when using the current
AMPS ADL motor and ADL process cutoff measures to
differentiate clients who are independent in the com-
munity from those who are in need of assistance, and
(3) the need to create new cutoff points that reflect more
acceptable estimates of sensitivity and specificity (i.e.,
lower rates of error).
Method
I conducted a retrospective, criterion-based validation
study of the AMPS ADL motor and ADL process ability
cutoff measures. Per standard protocol, before beginning
this research project, I obtained approval from the Col-
orado State University regulatory compliance office.
Participants and Site

I used existing data from the AMPS Project International
database in Fort Collins, Colorado. The data were from
people who were assessed by occupational therapy prac-
titioners within typical occupational therapy settings (e.g.,
hospitals, client homes, rehabilitation clinics). The sample
potentially included all available data records in the AMPS
database that were evaluated between January 1, 1999, and
January 1, 2005, and met the following criteria: (1) were
age 16 yr or older; (2) were not associated with rater
scoring error, as evidenced by artificially high ADL motor
or ADL process ability measures; (3) were not scored by
multiple raters as part of rater calibration; and (4) had
known sex, age, and global functional level ratings. To
clarify, only data from 1999 to 2005 were selected, because
new criteria for rating each global functional level were
created in 1999 and remained consistent until the data were
extracted in 2006 (Fisher, 1999). Before 1999, specific
criteria for rating functional level had not been developed.

For this study, 64,466 data records (58% of the
potential sample in the AMPS database) met the criteria
for inclusion. Data records eliminated from consideration
were those of people younger than age 16 (23% of the
potential sample), were associated with rater scoring
error (10% of the potential sample), or had incomplete
demographic data (9% of the potential sample). Data
records included those for men (42.9%) and women
(57.1%) from North America (14.8%), the United
Kingdom (22.3%), the Nordic countries (34.2%), other
European countries (11.6%), New Zealand and Australia
(8.7%), Asia (8.0%), and unknown regions (0.4%). Data
records were grouped into global diagnostic categories
(e.g., the psychiatric diagnostic category included people
with bipolar disorder, depression, schizophrenia, or per-
sonality disorder; see Table 1). Although specific di-
agnostic information could be lost by coding the data
in this manner, it ensured that sample sizes in each di-

agnostic group were sufficient for statistical analyses.
Instrumentation
AMPS. The AMPS was administered by occupational
therapists who attended a 5-day AMPS training workshop
and subsequently were calibrated as AMPS raters (e.g.,
demonstrated valid and reliable administration and
scoring of the AMPS). After observing the client perform
at least two AMPS tasks, each rater used the criteria in the
AMPS manual (Fisher, 2006b) to score the 16 ADL
motor skill and 20 ADL process skill items. Globally,
ADL motor skills are defined as the observable actions the
person enacts when moving the self or task objects
(Fisher, 2006a). ADL process skills are defined as “the
observable actions of performance the person enacts to
logically sequence the actions of the ADL task perfor-
mance over time, select and use appropriate tools and
materials, and adapt his or her performance when prob-
lems are encountered” (Fisher, 2006a, p. 4). According to
the criteria in the AMPS manual, each ADL skill item

was scored using a 4-point ordinal rating scale based
on the global criterion of competence (1 5 deficient
644 November/December 2011, Volume 65, Number 6
performance and/or task breakdown, 2 5 ineffective perfor-
mance, 3 5 questionable performance, 4 5 competent per-
formance). Each participant had at least two sets of ADL
motor scores and two sets of ADL process scores: one set
of scores for each task performed (i.e., two sets of scores
were used to generate one ADL motor ability measure and
one ADL process ability measure for each person).
Previous studies have supported using the AMPS cutoff
measures as indicators of the need for assistance (Bernspång
& Fisher, 1995; Hartman et al., 1999). Bernspång and
Fisher (1995) developed a “risk” zone, delineated by ADL
measures that fall within ±0.3 logit of the ADL motor
cutoff measure of 2.0 logits or ADL process cutoff measure
of 1.0 logit. Their results indicate that approximately two-
thirds of the clients who were frail or at risk of functional
decline had ADL ability measures within the risk zones.

With a sample of older adults (with and without de-
mentia), Hartman et al. (1999) documented a 94% overall
correct classification rate when using the AMPS ADL
process cutoff measure as an indicator of the need for as-
sistance. Previous studies have also supported the reliability
and validity of the AMPS ability measures across age
groups (Hayase et al., 2004), between men and women
(Merritt & Fisher, 2003), and with a variety of diagnoses
(e.g., Doble, Fisk, Fisher, Ritvo, & Murray, 1994;
Hartman et al., 1999; Rexroth, Fisher, Merritt, & Gliner,
2005). Last, the AMPS has also been shown to be a sensitive
outcome measure (e.g., Chard, Liu, & Mulholland, 2009;
Goverover, Johnston, Toglia, & Deluca, 2007; Kinnman,
Andersson, Wetterquist, Kinnman, & Andersson, 2000;
Oakley, Khin, Parks, Bauer, & Sunderland, 2002). Research
has demonstrated that the characteristics of the test
function independently of the person being tested (e.g.,
the item and task challenge calibrations do not vary

between samples; Fisher, 2006b); thus, the linear ADL
ability measures of men and women from different
world regions and with different diagnoses can be val-
idly generated and compared.
Global Functional Level. Considering the evidence that
clinical judgment appears to be one of the most accurate
means of determining global functioning within the
community (Pinholt et al., 1987; Rogers et al., 2003),
that direct observation of ADL performance tends to lead
to more accurate estimates of global functioning within
the community (Rogers et al., 2003), and that AMPS
ADL ability measures and AMPS global functional level
ratings are concurrently documented, I concluded that
a practical and valid external criterion available for eval-
uating the accuracy of the AMPS cutoff measures was the
AMPS global functional-level rating. Global functional
level was rated by trained and calibrated AMPS raters
according to the specific criteria outlined in the AMPS

manual (Fisher, 2006b; see Table 2). More specifically,
each occupational therapist used his or her clinical judg-
ment to determine whether the client was able to live
independently, in need of minimal assistance or super-
vision, or in need of moderate to maximal assistance to
live in the community.
The AMPS global functional level was not based solely
on the AMPS task observations; rather, the rating was
based on all information the therapist gathered about the
client. As a result, the therapist used multiple sources of
information to determine the most accurate rating (e.g.,
interview, caregiver report, reports from other health care
providers, observation of ADL performance, assessment
of body functions). In support, other researchers have
Table 1. Number of Participants by Diagnostic Group and
Global Functional Level
Diagnostic Category

TotalIndependent Minimal Assistance Moderate Assistance
Well 1,226 0 0 1,226
At risk 424 21 0 445
Frail older adult 0 61 6 67
Mild learning disability 42 52 32 126
Neurological developmental 57 315 636 1,008
Mental retardation 44 621 1,015 1,680
Other neurological 1,055 3,258 4,454 8,767
Hemispheric stroke 777 3,145 4,879 8,801
Musculoskeletal 2,598 3,003 1,780 7,381
Medical 705 1,009 851 2,565
Psychiatric 1,580 3,998 2,978 8,556
Dementia 99 681 1,708 2,488
Other memory 27 73 90 190
Other 1,824 7,442 11,900 21,166
Total 10,458 23,679 30,329 64,466

reported that using multiple sources of information, in-
cluding direct observation of ADL ability, is the most
accurate means of determining global functioning in the
community (Jette, Grover, & Keck, 2003; Rogers et al.,
2003). To minimize the risk of biased ratings, at the time
of rating the client’s AMPS global functional level the
raters were blind to the person’s final AMPS ability
measures and unaware that the functional level ratings
would be used to investigate the validity of the AMPS
ability measures.
Procedure and Data Analysis
A specialized, many-facet Rasch analysis program, Facets
(Linacre, 2006), was used to convert the raw ordinal ADL
skill item scores into linear ADL motor and ADL process
ability measures. Such analyses adjust the final ADL
ability estimates to account for task challenge, skill item
difficulty, and severity of the rater. Demographic in-
formation and ADL motor and ADL process ability

measures were imported into SPSS Version 17.0 for
Windows (SPSS, Inc., Chicago).
To evaluate the association between ADL ability
measures and global functional level, two one-way analyses
of variance (ANOVAs) were conducted (one for ADL
motor ability and one for ADL process ability), and the
size of the effect for each analysis was determined. Effect
size (h
2
) was interpreted on the basis of Cohen’s measure
of association (Cohen, 1988), in which .01, .06, and .14
indicate small, medium, or large effects, respectively.
To prepare the data for receiver operating charac-
teristic (ROC) curve analysis, AMPS global functional
level ratings were recoded into a dichotomous variable
denoting independent (n 5 10,458) or in need of assistance
to live in the community (n 5 54,008). Next, the data
were prepared to conduct a split-sample validation of the
predictive model (Kohavi, 1995). Therefore, approxi-
mately 60% of the sample (n 5 38,540) was randomly
selected using the SPSS random sampling function; this

subsample became the test sample and was used to gen-
erate ROC curves and subsequently develop the pre-
dictive model. The remaining data records in the sample
(n 5 25,926) became the validation sample, which was
used to validate the predictive model. The test and vali-
dation samples had the same proportion of data records
in each functional level group as did the entire sample;
16% of data records in each sample were independent,
36% were in need of minimal assistance, and 47% were
in need of moderate to maximal assistance.
Using the test sample (n 5 38,540), the dichotomous
state variable (need for assistance) was used to generate two
ROC curves, one with ADL motor ability as the test
variable and one with ADL process ability as the test
variable. Normal distribution and homogeneity of vari-
ance were not assumed, and thus a nonparametric model
was used to generate the ROC curves (Brown & Davis,
2006).
The area under the curve (AUC) was examined to

globally determine whether the ADL motor and ADL
process ability measures had merit in correctly categorizing
clients who were independent from those who needed
assistance to live in the community. Rough guidelines for
determining the discriminating value of a test by exam-
ining the AUC values are as follows: fail 5 .50–.60,
poor 5 .60–.70, fair 5 .70–.80, good 5 .80–.90, and
excellent 5 .90–1.00 (Perneczky et al., 2006).
After verification that the ADL motor and ADL process
scales had merit in predicting independence in the com-
munity (i.e., AUC ³ .70), the data were examined more
closely. First, the ROC curves were examined to determine
the sensitivity and specificity estimates of the current ADL
motor measure of 2.00 logits and ADL process ability
cutoff measure of 1.00 logit. Next, the ROC curves were
Table 2. Age and ADL Motor and ADL Process Ability Results
by Global Functional Level
Characteristic
Independent (n 5 10,458) Minimal Assistance (n 5 23,679)
Maximal Assistance (n 5 30,329)

Age, yr
M 55.1 57.6 61.7
SD 17.9 20.2 20.6
ADL motor ability, logits
Range 20.03–3.91 20.76–3.47 23.05–3.46
M 1.83 1.30 0.64
SD 0.69 0.76 0.98
ADL process ability, logits
Range 0.07–2.91 20.36–2.35 22.75–2.05
M 1.45 0.99 0.35
SD 0.51 0.48 0.96
Note. ADL 5 activity of daily living; M 5 mean; SD 5 standard
deviation.
examined to determine whether a need existed to create
new cutoff measures to reflect more accurate estimates of
independence or need for assistance to live in the community.

With regard to identifying community independence,
I found no specific recommendations for desired levels of
sensitivity and specificity in the literature. Potential cutoff
measures were thus determined by examining the ROC
curve at the clinically relevant area of the curve, defined as
the point on the curve where false positive rates (i.e., test
incorrectly indicates that the person is able to live in-
dependently) are minimized without resulting in large
false negative rates (i.e., test result incorrectly indicates
that the person needs assistance in the community). In
support of the decision to minimize false positive errors,
Dijkstra, Tiesinga, Plantinga, Veltman, and Dassen (2005)
also chose cutoff values that minimized errors that could
lead to clients not receiving the care, treatment, or support
that they actually needed.
Once the optimal cutoff measures for ADL motor and
ADL process ability were determined, the validation sample
(n 5 25,926) was used to create a predicted need for as-
sistance variable that was based on each person’s ADL
motor ability; people under the cutoff were coded as

needing assistance, whereas those above the cutoff were
coded as being independent in the community. In a simi-
lar manner, a second predicted need for assistance variable
was created on the basis of the ADL process cutoff mea-
sure. Congruence between predicted need for assistance and
the clinician’s judgment of the need for assistance (i.e.,
AMPS global functional level) was then analyzed to vali-
date the predictive model. To determine whether cate-
gorical accuracy could be improved on, the accuracy
achieved when decision points matched (i.e., when both
ADL motor and ADL process abilities were either above or
below the cutoff measures) was investigated.
Finally, from among all of the data records (n 5
64,466), separate ROC curves for samples within each
diagnostic category were generated using the occupa-
tional therapist’s original rating of need for assistance as
the state variable and ADL motor ability or ADL process
ability as the test variable. For this analysis, the AUC was
examined to globally determine the relative predictive
validity of the AMPS ability measures across the dif-
ferent diagnostic categories. Data were drawn from the

entire data set to maximize the sample size of each di-
agnostic grouping.
Results
Global functional level was found to be significantly as-
sociated with ADL motor ability, F(2, 64463) 5 8,728,
p < .01, and ADL process ability, F(2, 64463) 5 15,916,
p < .01. According to Cohen’s (1988) criteria, ADL
motor and ADL process ability measures have large
effects with regard to global functional level; h
2
values
were .213 and .331, respectively. Although age was
also found to be significantly associated with functional
level, F(2, 64463) 5 526, p < .01, the effect was small
(h
2 5 .016).
The ROC curve for examining the utility of using
ADL motor ability measures to determine community
independence revealed an AUC estimate of .78, indicating
fair discriminating value. Examination of the ROC curve
revealed that the sensitivity (i.e., rate of those correctly

identified as being independent within the community) of
the current ADL motor cutoff measure of 2.00 logits was
.40, and the specificity (i.e., rate of those correctly
identified as needing assistance to live in the community)
was .87. On further examination of the clinically relevant
area on the ROC curve, an ideal cutoff measure for ADL
motor ability was determined to be 1.50 logits, for which
sensitivity and specificity estimates were .67 and .72, re-
spectively. Using the validation sample, the accuracy
estimates were validated; when the predicted need for
assistance was compared with the clinician’s judgment of
the need for assistance, the new ADL motor cutoff of 1.5
logits resulted in a sensitivity of .68 and a specificity of
.72, thus validating the ROC curve estimates.
When ADL process ability measures were used to
determine community independence, the AUC was .84,
indicating good discriminating value. Examination of the
ROC curve revealed that the sensitivity and specificity

estimates of the current ADL process cutoff measure of
1.00 logits were .80 and .70, revealing an acceptable and
ideal cutoff measure. Within the validation sample, when
the predicted need for assistance was compared with the
clinician’s judgment of the need for assistance, the ADL
process cutoff of 1.0 logit resulted in a sensitivity rating of
.81 and a specificity rating of .70, thus validating the
ROC curve estimates.
To prepare the data to evaluate the accuracy of using
matched ADL motor and ADL process decisions, only
those participants in the validation sample (n 5 25,926)
with matched predicted ADL motor and ADL process
decision points were selected (i.e., ADL motor and ADL
process ability measures were either both above or both
below the respective cutoff measures). Approximately
65% (n 5 16,807) of the validation sample had matched
predicted decisions, and the sensitivity and specificity
estimates of this sample were .85 and .83, respectively.
Thus, accuracy estimates were highest when matched

ADL motor and ADL process decisions occurred.
Finally, separate ROC curves were generated for each
diagnostic group. Because of invariant ratings of the need
for assistance, the data coded with a diagnosis of either well
or frail older adult were not analyzed. AUC estimates
ranged from .68 to .85 (poor to good ratings) for ADL
motor ability and from .72 to .92 (fair to excellent rat-
ings) for ADL process ability (Table 3).
Discussion
The analyses indicate that both ADL motor ability and
ADL process ability have utility as indicators of the need
for assistance to live in the community. At best, when the
data records in the validation sample had matched ADL
motor and ADL process decision points (n 5 16,807),
85% of the independent sample and 83% of the sample
in need of assistance were correctly categorized. Until
now no study has investigated the use of matched ADL

motor and ADL process decisions, and thus the findings
have the potential to provide the clinical and research
communities with a new means of examining the con-
stellation of ability measures for the purpose of estimating
the need for assistance to live in the community.
When decision points did not match, the most ac-
curate predictions were obtained when using the ADL
process ability cutoff measure of 1.0 logit; 80% of the
independent sample and 70% of the sample in need of
assistance were correctly classified. The ADL process scale
of the AMPS continues to be more closely associated with
global functioning in the community than the ADL motor
scale, in line with previous studies (Fisher, 2006a; Hartman
et al., 1999; Kottorp, 2008; Merritt, 2010).
Although the current ADL motor cutoff measure of
2.00 logits has not been used as an indicator of in-
dependence in the community, the data revealed that the
current cutoff measure is too high for use in categorizing

independence versus the need for assistance (i.e., 60% of
the independent sample was incorrectly coded as needing
assistance). As a result, a new ADL motor cutoff measure
of 1.50 logits is proposed, resulting in more acceptable
estimates of sensitivity and specificity. Although additional
research is warranted, this study documents the first line of
evidence that ADL motor ability has some promise for use
as an indicator of community independence.
Global estimates of the accuracy of using AMPS ability
measures to predict the need for assistance were evaluated
across the various diagnostic groups (Table 3). The di-
agnosis-specific AUC estimates ranged from poor to good
for ADL motor ability and from fair to excellent for ADL
process ability. Although this global statistic does not
provide conclusive evidence of an assessment’s utility and
accuracy (Zweig & Campbell, 1993), the AUC statistics in
Table 3 suggest that the ADL motor and ADL process
ability scales are not equally accurate across all diagnostic

categories. More specifically, this is the first line of evi-
dence that suggests that the ADL motor ability scale may
be more accurate than the ADL process ability scale for
clients within specific diagnostic categories (e.g., muscu-
loskeletal, medical), whereas the ADL process ability scale
may be more accurate for clients with dementia, neuro-
logical developmental disabilities, and so forth.
Although the diagnosis-specific AUC statistics do not
provide conclusive evidence, they do provide a foundation
for comparing the accuracy estimates of other assessments.
For example, Mausbach and colleagues (2008) evaluated
the accuracy of using either the University of California,
San Diego, Performance-Based Skills Assessment (UPSA;
Patterson, Goldman, McKibbin, Hughs, & Jeste, 2001)
or the Dementia Rating Scale (DRS; Mattis, 1973) to
predict residential independence of people with schizo-
phrenia (n 5 434). They determined that the AUC es-
timates were .74 for the UPSA (an evaluation of ADL
ability) and .65 for the DRS (an evaluation of cognition).

In contrast, the current findings indicate that with
a sample of clients with psychiatric illness, the AUC
estimate for the AMPS ADL process scale was .77 (Table
3). Additional studies are warranted to fully evaluate and
compare the accuracy estimates of the AMPS cutoff mea-
sures across different diagnoses.
The results of this study are promising and indicate
that the AMPS ability measures have merit when used to
document the need for assistance to live in the community;
however, the predictions are not 100% accurate. One
Table 3. Area Under the Curve (AUC) Estimates by
Diagnostic Category
Diagnostic Categorya n
AUC
ADL Motor ADL Process
At risk 445 .75 .72
Mild learning disability 126 .67 .75
Neurological developmental 1,008 .73 .85

Mental retardation 1,680 .75 .83
Other neurological 8,767 .79 .83
Hemispheric stroke 8,801 .82 .82
Musculoskeletal 7,381 .82 .77
Medical 2,565 .85 .81
Psychiatric 8,556 .68 .77
Dementia 2,488 .78 .92
Other memory 190 .85 .91
Other 21,166 .79 .83
Note. ADL 5 activity of daily living.
aWell and frail older people were not analyzed because of
invariant ratings of
the need for assistance.
explanation for this is the fact that the AMPS was not
designed to measure global functioning within the com-
munity. The ability to live independently in the com-
munity requires the performance of tasks that extend
beyond those included in the AMPS (e.g., participation in

social tasks, community transportation, money manage-
ment, phone use, computer use); thus, the AMPS ADL
ability measures can be expected to explain some, but
not all, of the variation in the construct of community
independence.
Another area to explore is that of rater scoring error.
For example, occupational therapists working with clients
who have physical disabilities may overemphasize obser-
vation and scoring of ADL motor skill deficits relative to
ADL process skill deficits, resulting in inflated ADL
process ability measures and higher false positive errors.
Likewise, rater scoring error can occur when occupational
therapists working with clients who have cognitive or
psychiatric deficits focus their observations and scoring on
ADL process skills while failing to observe and accurately
score ADL motor skills, resulting in inflated ADL motor
ability measures and higher false positive errors. In such
instances, rater error is introduced; rater severity should

not change on the basis of the characteristics of the clients
being assessed.
Another potential source of error within the current
study is the accuracy of the occupational therapists’ ratings
of global functional level. The possibility exists that
clinicians are more accurate when determining the global
functional levels of those who are on the extreme ends of
the scale (e.g., clients who are clearly independent in the
community and those who require substantial assistance
to live in the community). When clients require some
assistance to live in the community or have inconsistent
needs for assistance, the possibility exists that clinicians’
judgments of global functional level are inaccurate. Ad-
ditional research is warranted to verify this assumption
and to further investigate the potential causes of error in
determining global functional level.
Although the lack of evidence supporting the validity
and reliability of the AMPS global functional ratings

constitutes a limitation of this study, use of this rating scale
created an economical opportunity to use a large existing
database to generate initial evidence of the merit of using
AMPS ADL ability measures to determine a person’s need
for assistance in the community. Clearly, follow-up
studies are warranted that prospectively gather ADL
measures or AMPS global functional ratings and measure
them against additional criteria (e.g., narrative reports
from caregivers, overall time that assistance was provided
over the course of a week, daily assistance logs). Such
studies could seek to validate use of the AMPS cutoff
measures as evidence of the need for assistance and could
document the validity and reliability of the AMPS global
functional level ratings.
Conclusion
Occupational therapists play a critical role in determining
clients’ discharge needs and need for assistance in the
community. This study provides evidence that occupa-

tional therapists can use the AMPS ability measures to
support decisions regarding a client’s potential need for
support or assistance in the community. The findings
support using AMPS ADL motor and ADL process
ability measures as evidence of a client’s need for assis-
tance to live in the community, but it is important to
stress that such decisions are rarely made on the basis
of the results from one assessment. Although many occu-
pational therapists use functional assessments to determine
discharge needs, they also consider constructs beyond
personal and domestic ADL performance, including the
client’s wants and needs, the client’s life context, the se-
verity and prognosis of the client’s impairment, and other
professionals’ opinions (Dijkstra et al., 2005). Thus, AMPS
ability measures should contribute to and support the
clinician’s judgments and recommendations and should
not serve as the sole piece of evidence when determining
the need for assistance. s

Acknowledgments
I thank Anne G. Fisher for her support and extensive
expertise. Previous versions of this article were presented at
the 2008 International Assessment of Motor and Process
Skills Symposium: Measuring, Planning, and Implementing
Occupation-Based Programs, Halifax, Nova Scotia, and the
2009 American Congress of Rehabilitation Medicine–
American Society of Neurorehabilitation Joint Conference,
Denver, Colorado.
References
Bernspång, B., & Fisher, A. G. (1995). Differences between
persons with right or left cerebral vascular accident on the
Assessment of Motor and Process Skills. Archives of Phys-
ical Medicine and Rehabilitation, 76, 1144–1151. doi:
10.1016/S0003-9993(95)80124-3
Brown, C. D., & Davis, H. T. (2006). Receiver operating
characteristics curves and related decision measures: A tu-
torial. Chemometrics and Intelligent Laboratory Systems, 80,
24–38. doi: 10.1016/j.chemolab.2005.05.004
Chard, G., Liu, L., & Mulholland, S. (2009). Verbal cueing
and environmental modifications: Strategies to improve
engagement in occupations in persons with Alzheimer

disease. Physical and Occupational Therapy in Geriatrics,
27, 197–211. doi: 10.1080/02703180802206280
Cohen, J. (1988). Statistical power and analysis for the behav-
ioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Dijkstra, A., Tiesinga, L. J., Plantinga, L., Veltman, G., &
Dassen, T. W. (2005). Diagnostic accuracy of the Care
Dependency Scale. Journal of Advanced Nursing, 50,
410–416. doi: 10.1111/j.1365-2648.2005.03406.x
Doble, S. E., Fisk, J. D., Fisher, A. G., Ritvo, P. G., & Murray,
T. J. (1994). Functional competence of community-
dwelling persons with multiple sclerosis using the Assess-
ment of Motor and Process Skills. Archives of Physical
Medicine and Rehabilitation, 75, 843–851. doi: 10.1016/
0003-9993(94)90107-4
Fisher, A. G. (1999). Assessment of Motor and Process Skills
(3rd
ed.). Fort Collins, CO: Three Star Press.
Fisher, A. G. (2006a). Assessment of Motor and Process Skills:
Vol. 1. Development, standardization, and administration
manual (6th ed.). Fort Collins, CO: Three Star Press.
Fisher, A. G. (2006b). Assessment of Motor and Process Skills:
Vol.
2. User manual (6th ed.). Fort Collins, CO: Three Star Press.
Goverover, Y., Johnston, M. V., Toglia, J., & Deluca, J.
(2007). Treatment to improve self-awareness in persons

with acquired brain injury. Brain Injury, 21, 913–923.
doi: 10.1080/02699050701553205
Hartman, M. L., Fisher, A. G., & Duran, L. (1999). Assess-
ment of functional ability of people with Alzheimer’s dis-
ease. Scandinavian Journal of Occupational Therapy, 6,
111–118. doi: 10.1080/110381299443690
Hayase, D., Mosenteen, D. A., Thimmaiah, D., Zemke, S.,
Atler, K., & Fisher, A. G. (2004). Age-related changes in
activities of daily living (ADL) ability. Australian Occupa-
tional Therapy Journal, 51, 192–198. doi: 10.1111/j.1440-
1630.2004.00425.x
Jette, D. U., Grover, L., & Keck, C. P. (2003). A qualitative
study of clinical decision making in recommending dis-
charge placement from the acute care setting. Physical
Therapy, 83, 224–236.
Kinnman, J., Andersson, U., Wetterquist, L., Kinnman, Y., &
Andersson, U. (2000). Cooling suit for multiple sclerosis:
Functional improvement in daily living. Scandinavian
Journal of Rehabilitation Medicine, 32, 20–24. doi:
10.1080/003655000750045695
Kohavi, R. (1995). A study of cross validation and bootstrap
for accuracy estimation and model selection. In Proceedings
of the 14th International Joint Conference on Artificial
Intelligence (pp. 338–345). San Francisco: Morgan
Kaufmann.
Kottorp, A. (2008). The use of the Assessment of Motor and
Process Skills (AMPS) in predicting need of assistance for
adults with mental retardation. OTJR: Occupation, Partic-
ipation and Health, 28, 72–80. doi: 10.3928/15394492-
20080301-04

Linacre, J. M. (2006). Facets Rasch measurement computer pro-
gram. Chicago: Winsteps.
Mattis, S. (1973). Dementia Rating Scale. Odessa, FL: Psycho-
logical Assessment Resources.
Mausbach, B. T., Bowie, C. R., Harvey, P. D., Twamley
E. W., Goldman, S. R., Jeste, D. V., et al. (2008). Use-
fulness of the UCSD Performance-Based Skills Assessment
(UPSA) for predicting residential independence in patients
with chronic schizophrenia. Journal of Psychiatric Research,
42, 320–327. doi: 10.1016/j.jpsychires.2006.12.008
Meinow, B., Kåreholt, I., & Lagergren, M. (2005). According
to need? Predicting the amount of municipal home help
allocated to elderly recipients in an urban area of Sweden.
Health and Social Care in the Community, 13, 366–377.
doi: 10.1111/j.1365-2524.2005.00570.x
Merritt, B. K. (2010). Utilizing AMPS ability measures to
predict level of community dependence. Scandinavian
Journal of Occupational Therapy, 17, 70–76.
Merritt, B. K., & Fisher, A. G. (2003). Gender differences in
the performance of activities of daily living. Archives of
Physical Medicine and Rehabilitation, 84, 1872–1877.
doi: 10.1016/S0003-9993(03)00483-0
Oakley, F., Khin, N. A., Parks, R., Bauer, L., & Sunderland,
T. (2002). Improvement in activities of daily living in
elderly following treatment for post-bereavement depres-
sion. Acta Psychiatrica Scandinavica, 105, 231–234. doi:
10.1034/j.1600-0447.2002.1sc021.x
Patterson, T. L., Goldman, S., McKibbin, C. L., Hughs, T., &

Jeste, D. V. (2001). UCSD Performance-Based Skills As-
sessment: Development of a new measure of everyday
functioning for severely mentally ill adults. Schizophrenia
Bulletin, 27, 235–245.
Perneczky, R., Pohl, C., Sorg, C., Hartmann, J., Komossa, K.,
Alexopoulos, P., et al. (2006). Complex activities of daily
living in mild cognitive impairment: Conceptual and di-
agnostic issues. Age and Ageing, 35, 240–245. doi: 10.
1093/ageing/afj054
Pinholt, E. M., Kroenke, K., Hanley, J. F., Kussman, M. J.,
Twyman, P. L., & Carpenter, J. L. (1987). Functional
assessment of the elderly: A comparison of standard instru-
ments with clinical judgment. Archives of Internal Medi-
cine, 147, 484–488. doi: 10.1001/archinte.147.3.484
Rexroth, P., Fisher, A. G., Merritt, B. K., & Gliner, J. (2005).
Ability differences in persons with unilateral hemispheric
stroke. Canadian Journal of Occupational Therapy, 72,
212–221.
Rogers, J. C., Holm, M. B., Beach, S., Schulz, R., Cipriani, J.,
Fox, A., et al. (2003). Concordance of four methods
of disability assessment using performance in the home
as the criterion method. Arthritis Care and Research, 49,
640–647. doi: 10.1002/art.11379
Wehman, P. (2001). Life beyond the classroom: Transition
strat-
egies for young people with disabilities. Baltimore: Paul H.
Brooks.
Zweig, M. H., & Campbell, G. (1993). Receiver-operating
characteristic (ROC) plots: A fundamental evaluation tool
in clinical medicine. Clinical Chemistry, 39, 561–577.

Copyright of American Journal of Occupational Therapy is the
property of American Occupational Therapy
Association and its content may not be copied or emailed to
multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users
may print, download, or email articles for
individual use.
Name: Chavita Green
Article Citation: Merritt, B. K. (2011). Validity of using the
Assessment of Motor and Process Skills to determine the need
for assistance. American Journal of Occupational Therapy, 65,
643–650. doi: 10.5014/ajot.2011.000547
DIAGNOSTIC ACCURACY STUDY: Are the results of the
study valid?
Was the diagnostic test evaluated in a Representative spectrum
of patients (like those in whom it would be used in practice)?
What is best?
Where do I find the information?
It is ideal if the diagnostic test is applied to the full spectrum of
patients - those with mild, severe, early and late cases of the
target disorder. It is also best if the patients are randomly
selected or consecutive admissions so that selection bias is

minimized.
The Methods section should tell you how patients were enrolled
and whether they were randomly selected or consecutive
admissions. It should also tell you where patients came from
and whether they are likely to be representative of the patients
in whom the test is to be used.
This paper: Yes ☐ No ☐ Unclear ☐
Comment:
Was the reference standard ascertained regardless of the index
test result?
What is best?
Ideally both the index test and the reference standard should be
carried out on all patients in the study. In some situations where
the reference standard is invasive or expensive there may be
reservations about subjecting patients with a negative index test
result (and thus a low probability of disease) to the reference
standard. An alternative reference standard is to follow-up
people for an appropriate period of time (dependent on disease
in question) to see if they are truly negative.
The Methods section should indicate whether or not the
reference standard was applied to all patients or if an alternative
reference standard (e.g., follow-up) was applied to those who
tested negative on the index test.
Comment:
Was there an independent, blind comparison between the index
test and an appropriate reference ('gold') standard of diagnosis?
What is best?
There are two issues here. First the reference standard should be
appropriate - as close to the 'truth' as possible. Sometimes there
may not be a single reference test that is suitable and a
combination of tests may be used to indicate the presence of
disease.
Second, the reference standard and the index test being assessed

should be applied to each patient independently and blindly.
Those who interpreted the results of one test should not be
aware of the results of the other test.
The Methods section should have a description of the reference
standard used and if you are unsure of whether or not this is an
appropriate reference standard you may need to do some
background searching in the area.
The Methods section should also describe who conducted the
two tests and whether each was conducted independently and
blinded to the results of the other.
Comment:
What were the results?
Are test characteristics presented?
What is the measure?
What does it mean?
Sensitivity (Sn) = the proportion of people with the condition
who have a positive test result.
The sensitivity tells us how well the test identifies people with
the condition. A highly sensitive test will not miss many people.
Specificity (Sp) = the proportion of people without the
condition who have a negative test result.
The specificity tells us how well the test identifies people
without the condition. A highly specific test will not falsely
identify many people as having the condition.
Application
Were the methods for performing the test described in sufficient
detail to permit replication?
What is best?

The article should have sufficient description of the test to
allow its replication and also interpretation of the results.
The Methods section should describe the test in detail.
Comment:
Threats to Validity
What were the potential and presented threats to validity?
What is best?
The authors should acknowledge areas of limitations. The
purpose of the appraisal is to identify threats that the authors
don’t identify.
The authors should list limitations in a “limitations” section.
Potential threats – are the ones that you identified throughout
your appraisal.
Comment:
1
Adapted from
Centre for Evidence-Based Medicine, University of Oxford.
(2010). Diagnostic critical appraisal sheet. Retrieved from
http://www.cebm.net/index.aspx?o=6913

Validity of Using the Assessment of Motor and ProcessSkills .docx

Recommended

Recommended

More Related Content

Similar to Validity of Using the Assessment of Motor and ProcessSkills .docx

Similar to Validity of Using the Assessment of Motor and ProcessSkills .docx (20)

More from dickonsondorris

More from dickonsondorris (20)

Recently uploaded

Recently uploaded (20)

Validity of Using the Assessment of Motor and ProcessSkills .docx