Gay1

This article was downloaded by: [202.138.134.156]
On: 07 February 2013, At: 00:41
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

Journal of Clinical Child Psychology
Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/hcap19

Factors influencing elementary school teachers' ratings
of ADHD and ODD behaviors
Jack Stevens & Alexandra L. Quittner
Version of record first published: 07 Jun 2010.

To cite this article: Jack Stevens & Alexandra L. Quittner (1998): Factors influencing elementary school teachers' ratings of
ADHD and ODD behaviors, Journal of Clinical Child Psychology, 27:4, 406-414

To link to this article: http://dx.doi.org/10.1207/s15374424jccp2704_4

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to
anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contents
will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should
be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims,
proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in
connection with or arising out of the use of this material.

Journal of Clinical Child Psychology Copyright O 1998 by
1998, Vol. 27, No. 4,406-414 Lawrence Erlbaum Associates, Inc.

Factors Influencing Elementary School Teachers' Ratings of
ADHD and ODD Behaviors
Jack Stevens and Alexandra L. Quittner
Department of Psychology, Indiana University-Bloomington
Howard Abikoff
New York University Medical Center

Examinedfactors that influence teachers' ratings of children with either attention def-
icit hyperactivity disorder (ADHD) or oppositional demnt disorder (ODD). 105
teachers watched 2 videotapes-1 depicting a normal child and the other a child with
either ADHD or ODD--and rated each child using 2 differentquestionnaires.Results
indicated that teachers accurately rated the child on the ADHD versus ODD tape as
having significantly more inattention and hyperactivity but significantly less
Downloaded by [202.138.134.156] at 00:41 07 February 2013

oppositionality. However, effectsizes indicated the presence of a unidirectional, neg-
ative halo efSect of oppositional behavwrs on ratings of hyperactivity and inattention.
Teachers appeared less biased in theirjudgments when using a welboperationalized
rating scale. Finally, knowledge, education, and experience with children withADHD
generally had no effect on the accuracy of teachers' ratings.

Elementary school teachers play a major role in childhood behavior. Often they lack the time or ability
the assessment of children's academic and behavioral to notice specific children's behaviors. Alternatively,
problems. Due to their extensive contact with chil- teachers may have frequent opportunities to observe
dren in a variety of structured and unstructured set- children's conduct but may be biased by certain char-
tings and their knowledge of age-appropriate skills acteristics of the students. For example, Stevens
and behaviors, teachers provide important informa- (1980) found that ethnicity and socioeconomic status
tion for both clinical and research purposes (Atkins, produced negative halo effects on teachers' ratings.
Pelham, & Licht, 1985; Shelton & Barkley, 1995). Specifically, the videotaped behaviors of African
Most often, educators are asked to evaluate children American and poor children were evaluated as more
with psychological problems by completing standard- deviant than Caucasian and middle-class children, de-
ized rating scales (Sandoval, 1981). These ratings are spite identical rates of disruptive behaviors.
then used to make decisions concerning diagnosis, treat- Negative halo effects may be associated not only
ment, and educational placement (see Brown, 1985). with certain demographic characteristics but also with
Teacher ratings are also frequently used in a research the presence of certain childhood behaviors. These bi-
context as inclusionary criteria, as a means of monitor- ases have been found to produce errors in teachers'
ing treatment progress, and as indicators of long-term judgments, particularly in the area of attention deficit
outcomes (DuPaul, Guevremont, & Barkley, 1991). hyperactivity disorder (ADHD). Mann et al. (1992)
Despite evidence indicating that teachers' ratings noted that the diagnosis is "based more on an assess-
can discriminate between children and adolescents ment of developmentally inappropriate intensity, fre-
with higher versus lower levels of psychological diffi- quency, andor duration of the behavior rather than its
culties (e.g., Dalley, Bolocofsky, & Karlin, 1994), mere presence. Such judgments increase the possibil-
teachers are not always accurate and objective raters of ity of observer bias" (p. 1539).
Researchers have found that although teachers' rat-
ings can reliably differentiate children with and with-
We thank Dr. Janice Bizzari for her support and assistance. We
out attention deficits (Atkins, Pelham, & Licht, 1985),
also thank the principals and teachers for their time and conscientious
effort. including even those who exhibit hyperactivity
Alexandra L. Quittner is now at the Department of Clinical and (Brown, 1985), teachers often do not distinguish chil-
Health Psychology, University of Florida. dren with ADHD from those with symptomatology of
Requests for reprints should be sent to Jack Stevens, Department the other disruptive behavior disorders--conduct dis-
of Psychology, Indiana University, Bloomington, IN 47405-1301,
order (CD) and ODD. For example, Schachar,
E-mail: jhsteven@indiana.edu,or to Alexandra L. Quittner, Depart-
ment of Clinical and Health Psychology, University of Florida, P.O. Sandberg, and Rutter (1986) investigated the relation
Box 100165, Gainesville, FL 32160, E-mail: aquittne@hp.ufl.edu between two respondents-teachers and blind observ-

TEACHERS' RATINGS OF ADHD AND ODD BEWAVIORS

ers-on measures of ADHD and ODD. They found a We also examined whether teachers' ratings would
negative halo effect of oppositional behaviors on hy- be less biased when more behaviorally anchored items
peractivity ratings but no halo effect of hyperactivity were administered (Abikoff et al., 1993). Qluestion-
on oppositional behaviors. Thus, regardless of their ac- naires often contain vague items that require a global
tivity level, children exhibiting defiance or aggression rating on some characteristic (e.g., "difficulty in learn-
were judged as having ADHD. ing"; Vaughn, Riccio, Hynd, & Hall, 1997). These
Abikoff, Courtney, Pelham, and Koplewicz (1993) items are often ambiguous with respect to the target be-
found a similar unidirectional bias in which havior and may be interpreted in different ways, inter-
oppositional behaviors exerted a negative halo effect fering with teachers' ability to make more fine-grained
on ratings of ADHD symptomatology. Specifically, distinctions in behavioral ratings. In contrast, descrip-
oppositional behaviors in a child on a classroom video- tive items that include greater operationaliiation of the
tape spuriously elevated teachers' reports of the sever- problem behaviors (e.g., "does not sit still", "blurts out
ity of his ADHD behaviors. However, hyperactive and answers") should lead to greater accuracy of' teacher
inattentive behaviors did not lead teachers to rate an- ratings. Thus, rating scales that employ concrete as op-
other child as being defiant or aggressive. This study posed to global items may be less subject to inegative
attempted to replicate these findings, testing the hy- halo effects (Mimtz & Collins, 1985).
pothesis that ODD behaviors lead teachers to rate chil- Abikoff et al. (1993) found evidence of the unidi-
dren as hyperactive or inattentive but that ADHD rectional bias even when teachers rated children using
behaviors do not cause teachers to rate these children the Diagnostic and Statistical Manual of Mental Dis-

as oppositional. orders (3rd editi~on, DSM-IZZ-R) ADHD diagnos-
rev.;
A central issue remains unclear from the Schachar tic category symptoms, which were specific and well
et al. (1986) and Abikoff et al. (1993) studies. Spe- operationaked. In that study, however, a categorical
cifically, the factors that might lead teachers to appear (all or non~e) approach to assessment was utilized for
less biased in their ratings have not been identified. the DSM-III-R symptoms, which did not permit a
Certain aspects of the assessment measures or particu- thorough examination of differencesin the magnitudes
lar characteristics of the teachers may affect their abil- of teacher ratings. A continuous distribution of ADHD
ity to differentiate between the two disorders. This ratings would permit greater detection of differencesin
study sough~t address this question by assessing three
to teachers' ratings. In this study we compared the impact
possible determinants of the bias in teachers' ratings of of using twlo differentcontinuousbehavioral measures,
disruptive blehavior disorders: measurement character- one with and one without global items. We bypothe-
istics of the rating scales, teachers' knowledge of and sized that teachers would be less biased in their ratings
education in ADBD, and teachers' professional expe- when using a rating scale that required behaviorally
rience with these children. anchored rather than global judgments.
First, we hypothesized that two properties of rating A second factor that may influence teachers' ability
scales may affect teachers' ability to distinguish to distinguish between ADHD and ODD is their
ADHD from ODD: level of confounding of symptoms knowledge of and educational background in ADHD.
across the two disorders and the extent to which items Little is known about how teachers' knowledge of the
are behaviorally anchored. Several commonly used disorder is associated with their ratings or their use of
teacher rating scale factors (e.g., Hyperactivity Factor particular intervention strategies (Greene, 1995). To
of the Conners Teachers Rating Scale; Conners, 1990) date, no study has systematically evaluated the relation
have individual items that reflect both ADllD and between knowledge of ADHD and teachers' ability to
ODD (e.g., "disturbs other children") behaviors. discriminate this childhood disorder from others. We
Therefore, it is not surprising that teachers' ratings of- hypothesized that greater knowledge of the disorder
ten do not differentiate between these disorders when would heighten teachers' awareness of ADHD
individual items include symptoms applying to both. symptomatology, resulting in better distinctil~nsbe-
Some researchers have found that pure items can help tween ADHD and ODD.
teachers distinguish between different types of prob- Finally, we examined whether professional experi-
lem behaviors (Ellen, 1989; Milich Idt Fitzgerald, ence with children with ADHD, their parents, a,ndvari-
1985). However, other studies (e.g., Abikoff et al., ous health care professionals would lead to more
1993; Schachar et al., 1986) have found no evidence accurate ratiugs. Interactions with students with
that omitting confounded items reduced the unidirec- ADHLI, as well as in-service presentations and conver-
tional bias. 'These results should be revisited to reex- sations with medical and mental health professionals
amine whether items that reflect either ADHD or ODD about children with ADHD may be related to more
prove useful in making this distinction. In the current positive perceptions of these children, as was found in
study, we expected to find less bias in teacher ratings a study examining teachers' attitudes about
when utilizing specific items that reflected only hyper- mainstreaming students with learning difficulties
activity and inattention as opposed to aggression. (Bender, Vail, & Scott, 1995). We hypothesized that

STEVENS, QUITTNER. & ABIKOPF

teachers with greater professional experience with tapes were designed so that the target child in the
children with ADHD and their caretakers would be ADHD tape engaged in behaviors characteristic of
less susceptible to negative halo effects when rating ADHD and did not behave in an oppositional fashion.
children displaying oppositional behaviors. In contrast, the target child in the ODD tape displayed
In sum, the central objective of this study was to ex- oppositional behaviors characteristic of the disorder
amine whether a unidirectional, negative halo effect but exhibited a rate of ADHD behaviors in the normal
was present in teachers' ratings of ADHD and ODD range.
behaviors. Furthermore, we investigated the role three Several steps were taken to ensure the validity of
factors-characteristics of the rating scales, knowl- the videotapes. Two independent sets of observers,
edge of ADHD, and experience with ADHD-might each using either the revised Stony Brook classroom
play in increasing the accuracy of teachers' ratings. observation code (COC; Abikoff, Gittelman-Klein, &
Klein, 1977) or the Classroom Observations of Con-
duct and Attention Deficit Disorders (COCADD;
Method Atkins et al., 1985; Atkins, Pelham, & Licht, 1989) and
blind to the condition of the target child, were asked to
Participants rate the frequency of ADHD and ODD behaviors.
Their ratings were in the expected direction. For exam-
One hundred and eight elementary school teachers ple, the rate of hyperactive behaviors was judged to be
were recruited from eight public schools through let- twice as high for the target child in the ADHD as op-

ters and phone calls to principals. In addition, three ele- posed to the normal or ODD tape. The frequency and
mentary school teachers returning for a graduate types of interactions amang classmates and teacher
degree in education also participated. Individual teach- were also controll~d each tape to remove any biases.
on
ers were not compensated for their participation; how- Finally, to minimize biases related to nonspecific char-
ever, schools in which teachers participated were acteristics (e.g., attractiveness of the child), the target
offered either a small monetary donation or an child in each psychopathology tape was played by the
in-service presentation on ADHD after completion of same actor; a differentchild played the "normal" child.
the study. Six of the 111 teachers failed to complete
75% of each of the standardized ratings scales de-
scribed in the following, and therefore their data were Procedure
excluded from the analyses.
The final sample for the current study consisted of Individuals with elementary school teaching expe-
105 elementary school teachers from a medium-sized rience were invited to participate in a study of teachers'
Midwestern city. The majority of teachers were judgments of childhood behaviors. The study was con-
women (90%) and Caucasian (5% were African Amer- ducted in a large room (e.g., library, conference room)
ican). Fifty-seven of the participants taught exclu- at each school. After signing informed consent state-
sively regular education students, 8 teachers taught ments, groups of teachers, ranging in size from 1 to 19,
only special education students, and the remaining 40 were assigned to one of four conditions.
taught both regular and special education students. On A between-subject design was employed in which
average, teachers had considerable teaching experi- teachers viewed, in counterbalanced order, two 10-rnin
ence ( M E 14.75 years, SD = 9.14). Seventy-seven had videotapes designed by Abikoff and colleagues
a master's degree, and two held a doctorate. (1993). All teachers viewed a videotape of a normal
fourth-grade boy in a classroom setting and either a
videotape of a boy with pure ADHD or a videotape of a
Stimuli boy with pure ODD. Forty-seven teachers viewed the
normal and ADHD tapes (i,e., 17 saw the normal tape
The three videotaped stimuli used in this study are first and 30 saw the ADHD tape first). The remaining
described briefly in the following paragraphs. A more 58 teachers viewed the normal and ODD tapes (i.e., 28
detailed description of the development and validation saw the normal tape first and 30 saw the ODD tape
of the videotapes can be found in Abikoff et al. (1993). first). Thus, the four conditions were: (a) nor-
Each 10-min videotape highlighted the activities of maVADHD, (b) ADHDInormal, (c) normaVODD, and
one specific 10-year-old boy, along with a teacher and (d) ODDInormal.
several peers as they engaged in a structured academic Prior to presentation of the two tapes, teachers were
activity (e.g., completing worksheets). The target child given three packets of questionnaires, and the investi-
in each tape exhibited different types and frequencies gators read a page of instructions telling teachers that
of disruptive childhood behaviors in order to depict a they will be presented with two videotapes showing
child who was either "normal," had "pure" ADHD, or children in a fourth-grade classroom containing chil-
had "pure" ODD. Specifically, the two "pathology" dren with mixed learning levels and that they can think

TEACHERS' RATINGS OF ADHD AND ODD BEHAVIORS

of each tape as a 10-min "slice of life" in the school standing of ADHD symptoms. Good internal consis-
day. They were instructed to watch a particular young- tency coefficients (i.e., .84 to .95) were obtained with
ster on the tape and were not informed of the diagnostic this sample.
status (i.e., "normal," "ADHD," and "ODD') of the
target children. Full instructions can be found in
Abikoff et al. (1993). Knowledge of ADHD. Teachers' knowledge of
After viewing the first tape, teachers were told to ADHD was assessed using the 17-item Knowledge
extrapolate from what they observed on the tape to scale from the Attention Deficit Hyperactivity Disor-
questions involving behaviors that were not present on der Knowledge and Opinion Survey-Revised; Bennett,
the tape, as they would in a real situation. After view- Power, Rostain, and Carr, 1996).This true-false test in-
ing each tape, teachers completed two different rating cluded questions concerning the etiology, assessment,
scales-the Conners Teacher Rating Scale (CTRS-28; treatment, and prognosis of the disorder. Correct re-
Goyette, Comers, & Ulrich, 1978) and the SNAP-N sponses were coded "1," and incorrect or bllank re-
Rating Scde (Swanson, personal communication, De- sponses were coded "0." Item-total correlations in the
cember 13, 1996). Finally, teachers completed two present sample of elementary education teachers were
questionnares in a third envelope that assessed their poor, ranging from -.22 to 30. Internal consistency
knowledge of and experience with ADHD. Teachers was also low (Cronbach a = 22).
were not told that the individuals appearing in the tapes
were actors until completion of the study. The entire

procedure lasted approximately 45 rnin. Experience with ADHD. A 23-item scale, the
ADHD Experience Scale, was developed for this study.
This questionnaire expanded on a similar measure de-
Measures veloped by Anastopoulos (personal commu~nication,
January 23,1997) andcontained three major sections:
Conners Teacher Ratings Scale. The CTRS-
28 is a shortened version of the 39-item Conners 1. Educational opportunities to learn about
Teacher Questionnaire (Conners, 1969) and is one of ADHD (e.g., in-service presentations).
the most frequently used teacher rating scales. 2. Professional experiences with ADHD (e.g.,
Teachers rate the target child's behavior on a 4-point number of previous students with ADHD, con-
scale ranging from not at all to very much. The ferences with parents, number of contacts with
CTRS-28 yields three factors (i.e., Conduct Problem, health care professionals about these students.
Hyperactivity, and Inattentive-Passive) and a 3. Attitudes about ADHD (e.g., how distressing
Hyperkinesis Index, which contains 10 of the most fre- these behaviors are).
quently endorsed items by teachers (Goyette et al.,
1978). Good internal consistency coefficients Two subscales were created from this measure: a
(.74-.94) were obtained for the four subscales of the six-item Educational Background scale (Cronbach a =
CIXS-28 for teachers rating either pathology tape in .68) and an eight-item Professional Contact scale
this study. (Cronbach a = .75).

SNAP-IV Rating Scale. A 39-item version of Results
the SNAP-IV Rating Scale was also completed by
teachers to rate the presence of ADHD and ODD be- Preliminary Analyses
haviors in the target child. Teachers rated the child's
behavior on a 4-point scale that was similar to the one There were no systematic differences in teachers'
used on the CTRS-28. In this version of the SNAP-IV, characteristics across the four conditions. Chi-square
there were the nine DSM (4th ed.; DSM-ZV) symptoms tests indicated that the number (ofteachers with special
of ADHD-Inattentive Type, the nine DSM-ZV symp- education experience or graduate degrees was similar
toms of ADHD-Hyperactive/Impulsive Type, and the across conditions,x2(3, = 105)= 1.72and3.54,p> .30.
N
eight DSM-Wsymptoms of ODD. The SNAP-IV was Three separate muItivariate analyses of variance
chosen because its items utilized more specific opera- (MANOVAs) revealed no significant differences
tional definitions and required more descriptive (e.g., across conditions in educational background, profes-
"fidgets with hands or feet or squirms in seat") as op- sional contact, and knowledge of ADHD, years of expe-
posed to global (e.g., "childish and immatureD') judg- rience, or ratings of distress in h~andlingADHDl behav-
ments than did the CTRS-28. In addition, the iors (Hotelling's Trangedfrom0.090to 1 . 4 7 5 ,> .05).
~
SNAP-IV specified behaviors according to DSM-N As expected, paired t tests indicated that teachers dif-
criteria, which reflects the field's most recent under- ferentiated between the normal and the two pathology

STEVENS, QUITTNER, & ABIKOFF
Table 1. Teachers' Mean CTRS-28 and SNAP-IV Ratings o Each Tape
f

Type of Child Depicted on Tape
- --
Normal ADHD ODD
Scale M SD M SD M SD Effect Size Indexs
CTRS-28
HA 0.19 0.08 1.91 0.57 1.66 0.55 0.27
IN-PA 0.36 0.42 1.56 0.61 1.36 0.58 0.17
HI 0.27 0.28 1.73 0.57 1.78 0.54 0.03
CP 0.03 0.08 0.74 0.53 2.05 0.57 1.23
SNAP-IV
HYAMP 0.17 0.17 1.72 0.66 1.10 0.66 0.50
INATT 0.38 0.46 1.90 0.67 1.38 0.72 0.38
ODD 0.03 0.21 0.77 0.67 2.13 0.62 1.08
- -

Note: N= 105for normal tape, N = 47 for ADHD tape, and N= 58 for ODD tape. CTRS-28 = Conners Teachers Rating Scale (28-itemversion);HA
=Hyperactivity factor; IN-PA = Inattentive-Passive factor; HI = Hyperkinesis Index; CP = Conduct Problem factor; HYIIMP =DSM-IV ADHD
Hyperactivity/Impdsivity scale; INATT = DSM-IV ADHD Inattention scale; ODD = DSM-IVOppositional Defiant Disorder scale.
%ffect size = differencesin ratings between the ADHD and ODD tape controlling for the order of the normal tape, special education experience,
and ratings of the normal tape. Effect sizes < S O are modest, .SO-.70 are moderate, and > .70 are large.

tapes. As can be seen in Table 1,teachers rated the target trary to our prediction, the ADHD tape was rated
child in the ADHD tape versus the normal tape as having significantly higher than the ODD tape on the
greater symptomatology on the CTRS-28 Hyperactiv- CTRS-28 Hyperactivity Factor, F(1, 96) = 7.17, p <
ity, Inattentive-Passive, andHyperkinesis scales andon .01. Note, however, that the mean CTRS Hyperactivity
the SNAP-IV Hyperactivity-Impulsivity, and Inatten- Factor scores were still above the clinical cutoff of 1.5
tion scales; t(46) ranged from 12.68 to 21.47, p < .001. on both tapes. Finally, the child in the ADHD versus
Similarly,teachersratedthe target child intheODD tape ODD tape was judged to have greater symptomatology
versus the normal tape as having more severe on the SNAP-IV Inattention and Hyperactiv-
oppositional behaviors on the Conners Conduct Prob- ity-Impulsivity scales, F(1, 96) = 13.76 and 23.92 re-
lem scale and on the SNAP-IV ODD scale, t(57) =27.83 spectively,p < .001.
and 24.76, p < .001.
Ratings of oppositional behaviors. As predicted,
the average CTRS-28 Conduct Problem and SNAP-IV
ODD scores were significantly lower for teachers
Ratings of Behavior on the watching the ADHD versus ODD tape, F(1, 96) =
ADHD Versus ODD Tapes 145.66 and 103.24 respectively, p < .001.
To test the presence of a negative halo effect (Hy- Effect sizes for judgments of ADHD versus ODD
pothesis I), a 2 (Type of Tape: ADHD vs. ODD) x 2 behaviors. The effect size of type of tape (ADHD
(Order of the Normal Tape: First vs. Second) x 2 (Type vs. ODD) was calculated1after controlling for order,
of Teacher: Special Education/Inclusion vs. Regular special education experience, and behavior ratings on
Education) analysis of covariance (ANCOVA) was the normal tape (see Table 1). As mentioned previ-
conducted on each of the four CTRS-28 scales and ously, teachers' ratings of the child on the ADHD ver-
three SNAP-IV scales. To account for differences in sus ODD tape were significantlydifferent on two of the
individual response styles across teachers, their ratings four CTRS-28 scales (Hyperactivity Factor, Conduct
on the normal tape were used as covariates. Across all Problem Factor) and on all three of the SNAP-IV
scales, we found no significant effects of order of tape scales. Thus, although several statistically reliable dif-
or teachers' special education experience as well as no ferences were found, the magnitude of the main effects
significant two-way or three-way interactions. F(1,96) for type of tape was considerably less for measures of
ranged from .OO to 3.35, p > .05. inattention and hyperactivity than for measures of
oppositional behavior. In sum, there was substantially
less discrepancy between the two pathology tapes in
Ratings of hyperactive and inattentive behaviors. ratings of ADHD versus ODD behaviors, suggesting
As expected, no significant differences were found be-
tween ratings of the ADHD versus ODD tape on the In- '~ffect indexes were calculated using the proportion of vari-
size
attentive-Passive scale, F(l, 96) = 2.84, p > .09, or on ance in ratings accounted for by the type of tape. See Cohen (1992)
the Hyperkinesis Index, F(l, 96) = .052, p > 3 0 . Con- for further details on effect sizes.

TEACHERS' RATINGS OF ADHD AND ODD BEHAVIORS

that oppositional behaviors did exert a negative halo ef- 1985). To assess the impact of these problemaltic items
fect on ratings of hyperactivity and inattention. on teachers' ratings, these four items were omitted
from the CTRS-28 Inattentivepassive scale, and the
remaining four items were subjected to an analysis of
Impact of Operationalization of the covariance. As expected, teachers rating the ADHD
Behavioral Measures on versus ODD tape endorsed significantly moire severe
Inattention and Hyperactivity Ratings inattention o this modified scale, F(1,96) = 8.89, p <
m
.01. In sum, negative halo effects appear to be at least
Teachers were expected to evidence less bias in partly due to measurement problems in these com-
their ratings of inattention and hyperactivity as monly used ratings subscales.
operationalization of the rating scale increased. This
hypothesis was supported. As mentioned previously,
differences in the predicted direction were obtained Factors Related to Teachers' ADHD
on both of the well-operationalized ADHD scales of Ratings of the Pathology Tapes
the SNAP-IV but on only one of the three ADHD
scales of the less behaviorally anchored CTRS-28. Teachers with greater knowledge about and more
Thus, item content and wording of the questions ap- professional experience with ADHD were expected to
peared to contribute significantly to the bias. rate the two pathology tapes more accurately. To ex-
Two pauticular problems with the CTRS-28 were amine the relation between these teacher chmacteris-

noted: confounding of content across the scales and the tics and their ratings of hyperactive and inattentive
global wording of some items, First, in terms of over- behaviors, hierarchical regression analyses were per-
lapping content, both the Hyperactivity factor and the formed separately for the PDHD and ODlD tapes.
Hyperkinesis Index of the CTRS-28 contained items Three-step hierarchical regressions were eomputed
that assessied both ADHD and ODD behaviors simulta- with the CTRS-28 InattentivePassive and Hyperac-
neously (e.g., "disturbs other children"). A measure of tivity scales and the SNAP-IV Inattention and Hyper-
pure inattention and hyperactivity, not confounded by activity-Impulsivity scales as dependent variables.
items reflecting aggression and noncompliance, might Ratings of the normal tape andl order of the tape presen-
reduce the occurrence of biased judgments of children tation were entered on the first and second steps, re-
with ODD. The five InattentionlOveractivity (10) spectively. Next, five variables reflecting teachers'
items from the IOWA Conners Teachers Rating Scale backgrounds and experience were entered on the third
(Loney & Milich, 1982) that were empirically related step. These teacher factors were: (a) ADHD Educa-
to hyperactivity but not aggression were subjected to tional Background, (b) ADHD Professional Contact,
an analysis of c~variance.~ partialling out ratings
After (c) ADHD Knowledge soore, (d) Special Elducation
of the normal tape, significantly higher ratings on these Experience, and (e) Distress Ratings (on ;I 5-point
five items for the target child on the ADHD versus scale) caused by ADHD behaviors.3
ODD tape were found, M = 10.8 and 8.3 respectively, The regression analyses indicated that neither
F(1,96) = 16.76, p < ,001. This indicated that the bias teacher's individual rating style (Step 1) nor the order
was weaker when using a scale with items reflecting of tape presentation (Step 2) was significantlly related
only hyperactivity and inattention. to any of the scale scores for teachers rating either pa-
Second, problems with the wording of the inatten- thology tape. The total R2 ranged from .02 to .07. The
tion items were noted in an item analysis of the only exception was that highes hyperactivity ratings of
CTRS-28 Inattentive-Passive scale. The Inatten- the normal tape were significantly related ito higher
tive-Passive scale contains two items reflecting social CTRS-28 hyperactivity ratings of the ODD tape O)<
skills as opposed to inattention (i.e., "appears to be eas- .05; total R2 = .19). On the third step, ADHD Profes-
ily led by other children" and "appears to lack leader- sional Contact, ADHD Knwwledge Test score, and
ship"). Two items also required global rather than Level of Distress caused by ADHD behaviors were not
descriptive judgments (i.e., "childish and immature" significantly associated with ratings of the M)HDtape
and "difficulty in learning"), which may be more sus- ( p > .05). However, two variables were significantly
ceptible to negative halo effects (Mintz & Collins, related to teachers' ratings of the ADHD tape, but in
the direction opposite to our prediction:

2 ~ our study, "restless in the 'squirmy' sense," "makes inappro-
n
priate noises when slhe shouldn't,""distractibilityor attention span a
problem," "excitable, impulsive," and "fails to finish things that slhe 3~ulticollinearity among the predictor variables did nc>tappear to
starts" were utilized. The items frrom the original (1982) scale were as be a problem. For ratings of the ADHD tape, wrrelati~ons among
follows: "fidgeting," "hums and makes other odd noises," "inatten- these variables = -0.26 to 0.57. For ratings of the ODD tape, wrrela-
tive, easily distracted," "excitable, impulsive," and "fails to finish tions among these variables=-0.27to 0.44. Multicollinearityis more
things he stants (short attention span)." likely when correlationsare above .70 (Hanushek & Jackson, 1977).

STEVENS,QUITTNER, & ABIKOFF
1. Teachers with more special education experi- mainstreaminghas occurred,resulting in more children
ence endorsed less symptomatology on the with various problems in regular education classrooms
CTRS-28 InattentivePassive scale. (Stainback & Stainback, 1995;Westwood, 1997). This
2. Teachers with more ADHD education had sig- shift has given a broad spectrum of teachers greater ex-
nificantly lower CTRS-28 Hyperactivity rat- posure to children with ADHD, which may have de-
ings ( p < .05; total R~ = .26 and .18, respec- creased the strength of the unidirectional bias in this
tively; A R~ coefficients = .21 and .l3). study.
We also found strong support for the second hypoth-
On the third step of the analyses of the ratings of the esis. As expected, teachers evidenced less bias when
ODD tape, ADHD Professional Contact, Special Edu- completing a well-operationalized rating scale, such as
cation Experience, and Level of Distress caused by the SNAP-IV, and exhibited greater bias on the less
ADHD behaviors were not significant predictors. well-operationalized CTRS-28. More fine-grained
However, as predicted, higher scores on the ADHD analyses indicated that individual items reflecting both
Knowledge test were associated with significantly ADHD and ODD behaviors from the CTRS-28
lower CTRS-28 InattentivePassive ratings of the Hyperkinesis scale (e.g., "disturbs other children") ap-
ODD tape. Moreover, ADHD Educational Back- peared to contribute to these inaccuracies. However,
ground was a significant predictor for all four scales (p when items reflecting only ADHD or ODD
< 05; total Rz ranged from .18 to .31; A R2 coefficients symptomatology were utilized, teachers accurately en-
ranged from "10 to .28). Contrary to Hypothesis 3, dorsed greater inattention and hyperactivity for the

teachers with more ADHD education endorsed more child in the ADHD versus ODD tape. Note that Abikoff
inattention and hyperactivity on the ODD tape. et al. (1993) still foundevidence of a unidirectionalbias
when utilizing ascalecontaining only hyperactivity and
inattention items, Because that study found generally
Discussion stronger evidence of a bias than the present one, purer
item content may not have played asr significant arolein
We obtained support for our first hypothesis that the attenuatingthis bias. Given the less extreme bias found
presence of oppositional behaviors would exert a uni- in the current study,purer items have appeamd to redu~e
directional, negative halo effect on teachers' ratings of this problem.
hyperactivity and inattention. In this study, teachers In addition, teachers were not able to differentiate
generally rated the child on the ADHD versus ODD the two symptomatic children on the CTRS-28 Inat-
tape as having significaqtly greater hyperactivity and tentive-Passive scale, suggesting that this measure of
inattention and significantly less oppositionality. inattention lacks discriminant validity. Other research-
However, an examination of the effect sizes for type of ershave also found that the Conners Teacher Rating
tape indicated inaccuracies in teachers' judgments. Scales lacked specificity and were not able to identify
Small differences in ratings of hyperactive and inatten- those children who had attention deficits per se from
tive behaviors were found between pathology tapes, in those who exhibit hyperactivity or aggression
contrast to the large differences obtained in (Ullmann, Sleator, & Sprague, 1985). However, when
oppositional ratings between tapes. Thus, teachers of- we excluded items that reflected social skills or were
ten erroneously rated the child exhibiting only global in nature, teachers correctly rated the target
oppositional behaviors as having substantial symp- child with ADHD versus ODD as more inattentive.
toms of ADHD. This is consistent with our premise that item$ reflect-
These findings differ somewhat from those reported ing specific behaviors, rather than global characteris-
by Abikoff etal. (1993). In this study, differences in the tics, improve the accuracy of teachers' ratings.
ratings of hyperactive and inattentive behaviors be- Finally, we found no support for our third hypoth-
tween tapes were small but statistically significant. esis. Having greater knowledge and experience with
However, in the Abikoff et al. (1993) study, ADHD was not generally associated with more accu-
nonsignificant differences in ratings of the two tapes rate ratings. The lack of a relation betwean knowl-
were generally found, suggesting a stronger negative edge of ADHD and teacher ratings should be inter-
halo effect. One possible explanation for these discrep- preted with caution, because the knowledge measure
ant results is that in recent years teachers have become we used had significant limitations. Fir& item-total
more aware that there are different types of childhood correlations were low, suggesting that individual
behavior disorders. The data from the Abikoff et al. items were not tapping the same central construct.
(1993) study were callected almost 8 years ago, and in Second, the majority of guestbns focused an factual
the meantime teachers may have had greater access to information about the prevalence, etidagy, treatment,
information about child behavior problems, which may and prognosis of the disordw, as opposed to knowl-
have attenuated the bias. Similsuly, in the past decade a edge about relevant symptoms, T@aohers7under-
strong shift in educational philosophy with regard to standing of the specific behaviors that do anid do not

TEACHEXIS' RATINGS OF ADHD AND ODD BEHAVIORS

constitute ADHD, rather than general facts, may be a would be advised when selecting rating scales to uti-
better indicator of their ability to recognize and rate lize well-operationalized questionnaires, which appear
ADHD symptoms accurately. However, the design of to facilitate differentiation of childhood behavior dis-
this stu4y precluded administration of this type of orders. Mote that Conners (1997) has recently revised
measure because the videotaped stimuli would have both the parent and teacher questionnaires to reflect the
influenced those responses. better-operationalized DSM-N criteria. R~esearchers
We also foundlittle evidencethat greater exposureto should further investigate whether improving the
educational materials about ADHD was related to more operationalization of questionnaires increases their
accurate ratings. In fact, our findings were in the oppo- discriminant validity, not only in the domain of child-
site direction, indicating that teachers exposed to more hood disruptive behaviors but also across all areas of
information about ADHD tended to rate the child with child and adult psychopathology.
ODD as bavingmoreinattentionandhyperactivity. This Second, factual knowledge and professioinal experi-
finding held true regardless of the operationalization of ence with ADMD do not appear to be sufficient for im-
theratings scales. Perhapsteacherswithmoreeducation proving teachers' recognition of ADHD and ODD
relevant to ADHD recognize that the behavior of chil- symptomatology. This is an important issue because
dren with ODD is problematic but overgeneralize all be- teachers are often the earliest and most frequent source
havior disorders to the one they "know bestY-ADHD. for identifying children with behavior problems. Two
Many h v e implied that the extensive coverage of important steps should be taken. First, a
ADHD by the media and other sources has led to an psychometrically sound measure of teachers' knowl-

overdiagnosis of the disorder (Hancock, 1996). Thus, edge of A O H D includes questions relevant to the
that
childrenwith diverseproblems (e.g., learningdisorders, discrimination of different childhood disord~rs should
aggression) may oftenbe incorrectly labeled ,as ADHD. be developed. Second, through college and continuing
Sirnildy, more contact with experts on ADHD and education courses teachers should be trained specifi-
children with this disorder was not related to the accu- cally to recognize the symptoms of ADHD, ODD, and
racy of teachers' ratings of the ODD tape. It is possible other psychol~ogicaldisorders (Abikoff et al., 1993).
that exposureto professionals and parents does not lead Other factors besides knowledge and experience
to bemr differentiation of specific childhood disorders. should also be considered, such as teachers' personali-
For example, contact with children with ADHD and ties and attitudes toward these students. For example,
their caretakers may sengitizeteachers to the key symp- teachers' preference for structured versus unstructured
toms of the disorder,but it may also result in greater ac- classrooms may influencetheir ratings of chiildren with
ceptance of children's behavior problems, and hence, hyperactivity (Eddowes, Aldridge, & (Julpepper,
less extreme ratings. In addition, it is widely acknowl- 1994). Given the national move toward the
edged that health care professionals often hold different mainstreaming of children with a variety crf disabili-
views with regard to diagnosis and appropriate treat- ties, it is vitally important that teachers have the
ment. Therefore, contact with these professionals may knowledge and experience relevant to facing these
not result in better-informed teachers. new challenges.
Two limitations of this study are worth noting. First,
it is not clear how generalizable findings based on rat-
ings of videotaped segments are to the more typical References
classroom setting.Teachers can rarely focus on one stu-
dent for extended period of time, as they did when Abikoff, H., Courtney, M., Pelham, W. E., & Kopl~ewicz,H. S.
viewing these videotapes. Moreover, teachers typically (1993). Teachers' rating of disruptive behaviors: The influence
evaluate their students in a variety of situations, from of halo effects. Journal of Abnormal Child Psychology, 21,
structured academic activities to less organized free 519-533.
Abikoff, H., Gittelman-Klein, R., L Klein, D. F.(1977). Validation
play. Examining teachers' ratings with children in ac- of a classroom observation code for hyperactive children. Jour-
tual settings, as their behavior varies across situations, nal of Consulting and Clinical Psychology, 4,772-783.
would increasethe generalizability of our findings.No- Atkins,M. S., Pelham,W. E., &Licht,M. H. (1985). Acomparisonof
tably, however, our results convergewith those investi- objective classroom measures and teacher ratings;of attention
gators who found a unidirectional bias in a classroom deficit disorder. Journal of Abnormal Child Psychology, 13,
155-167.
setting (Schachar et al., 1986). Second,in this study we Atkins, M. S., Pelham, W. E., & Licht, M. H. (1989). The differential
did not investigate teachers' knowledge of ODD, an- validity of teacher ratings of inattentiontovemtivity and ag-
other factor that may have influenced their ratings. gression. JoumalofAbnormal ChildPsychology, 17, 423--435.
Greater knowledge of ODD could have increased the Bender, W. N., Vail, C. O., &Scott, I<. (1995). Teachers'attitudesto-
accuracy of their ratings of children displaying only ward increased mainstreaming:Implementingeffective instruc-
tion for students with learning disabilities. Journar' of Learning
oppositioinal behaviors.
Disabilities, 28, 87-94.
Two important implications may be drawn from Bennett, D. S., Power, T. J., Rostain, A. L., &Cam,D. E. (1996). Par-
this investigation. First, clinicians and researchers ent aciceptability and feasibility of ADHD interventions: As-

STEVENS, QUITTNER, & ABIKOFF

sessment, correlates, and predictive validity. Journal of Pediat- Milich, R., & Fitzgerald, G. (1985).Validation of inattentionloverac-
ric Psychology, 21,643-657. tivity and aggression ratings with classroom observations. Jour-
Brown, R. T. (1985). The validity of teacherratings in differentiating nal of Clinical and Consulting Psychology, 53, 139-140.
between two subgroups of attention deficit disordered children Mintz, L. I, & Collins, B. (1985).Qualitative influences on the per-
.
with or without hyperactivity. Educational and Psychological ception of movement: An experimental study. Journal of Ab-
Measurement, 45,661-669. normal Child Psychology, 13, 143-153.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, Sandoval, J. (1981). Format effects in two teacher rating scales of hy-
155-159. peractivity. Journal ofAbnorma1Child Psychology, 9,203-21 8.
Conners, C. K. (1969). A teacher rating scale for use in drug studies Schachar, R., Sandberg, S., &Rutter, M. (1986).Agreement between
with children. American Journal of Psychiatry, 126, 884-888. teachers' ratings and observations of hyperactivity, inattentive-
Conners, C. K. (1990). Conners Rating Scalesmanual. Toronto, On- ness, and defiance. Journal ofAbnorma1 Child Psychology, 14,
tario, Canada: Multi-Health Systems, Inc. 331-345.
Comers, C. K. (1997). Conners' Ratings Scales-Revised North Shelton, T. L., & Barkley, R. A. (1995). The assessment and treat-
Tonawanda, NY:Multi-health Systems, h c . ment of attention-deficit/hyperactivitydisorder in children. In
Dalley, M. B., Bolocofsky, D. N., & Karlin, N. J. (1994). M. C. Roberts (Ed.), Handbook of pediatric psychology (pp.
Teacher-ratings and self-ratings of social competencyin adoles- 633-654). New York: Guilford.
cents with low- and high-depressive symptoms. Journal of Ab- Stainback, W., & Stainback, S. (1995). Contemplating inclusiveedn-
normal Child Psychology, 22,477-485. cation from a historical perspective. In R. Villa & J. Thousand
DuPual, G. J.,Guevremont, D. C., & Barkley, R. A. (1991). Attention (Eds.), Creating an inclusive school (chap. 2). Alexandria, VA:
deficit-hyperactivity disorder in adolescence: Critical assess- Association for Supervision and Curriculum Development.
ment parameters. Clinical Psychology Review, 11, 231-245. Stevens, G. (1980). Bias in attributivns of positive and negative be-
Eddowes, E. A., Aldridge, J., & Culpepper, S. (1994). Primary teach- havior in children by school psychologists, parents, and teach-
ers' classroom practices and their perceptions of children's at- ers. Perceptual and Motor Skills, 50, 1283-1290.

tention problems. Perceptual and Motor Skills, 79, 787-790. Ullmann, R. K., Sleator, E. K., & Sprague, R. L. (1985). A change in
Ellen, Arthur S. (1989). Discriminant validity of teacher ratings for mind: The Conners Abbreviated Rating Scales reconsidered.
n d , learningdisabled, and emotionally-handicapped boys. Journal of Abnormal Child Psychology, 13,553-565.
Journal qfScho01 Psychology, 27,15-25. Vaughn, M. L., Riccio, C. A., Hynd, G. W., &Hall, J. (1997).Diag-
Goyette, C. H., Conners, C. I. & Ulrich, R. F. (1978). Normative
S, nosing ADHD (predominately Inattentiveand Combined Type
data on the Revised Conners Parent and Teacher Rating Scales. subtypes): Discriminant validity of the Behavior Assessment
Journal ofAbnorma1 Child Psychology, 6,221-236. System for Children and the Achenbach Parent and Teacher
Greene, R. W. (1995). Students with ADHD in school classrooms: Rating Scales. Journal of Clinical Child Psychology, 26,
Teacher factors related to compatibility, assessment, and inter- 349-357.
vention. School Psychology Review, 24, 81-93. Westwood, P. (1997). Commonsense methodsfor children with spe-
Hancock, L. (1996, March 18). Mother's Little helper. Newsweek, cial needs. New York: Routledge.
I2?,51-56. Wilson, C. C. (1980). Behavioral assessment: Questionnaires.
Hanushek, E. A., &Jackson, J. E. (1977).Statistical methodsfor so- School Psychology Review, 9, 58-66.
cial scientists. New York: Academic.
Loney, J., & Milich, R. (1982). Hyperactivity, inattention, and ag-
gression in clinical practice. In M. Wolraich & D. K. Routh
(Eds.), Advances in developmental and behavioral pediatrics
(Vol. 12, pp. 221-251). New York: Plenum.
Mann, E. M., Ikeda,Y., Mueller, C.W., Takahashi, A., Tao, K. T.,
Hnmris, E., Li, B. L., &Chin, D. (1992). Cross-cultural differ-
ences in rating hyperactive-disruptive behaviors in children. Manuscript received November 11,1997
American Journal of Psychiatry, 149, 153S1542. Final revision received April 13, 1998

Gay1

More Related Content

What's hot

Viewers also liked

Similar to Gay1

Gay1