A Comparison Of The Mystery Motivator And The Get Em On Task Interventions For Off-Task Behaviors

Psychology in the Schools, Vol. 49(2), 2012 C
2012 Wiley Periodicals, Inc.
View this article online at wileyonlinelibrary.com/journal/pits DOI: 10.1002/pits.20627
A COMPARISON OF THE MYSTERY MOTIVATOR AND THE GET ‘EM ON TASK
INTERVENTIONS FOR OFF-TASK BEHAVIORS
ELISABETH E. KRAEMER, SUSAN C. DAVIES, KELLI JO ARNDT, AND SAWYER HUNLEY
University of Dayton
Attending to instruction is a critical behavior for academic success. Many elementary school teach-
ers, however, identify disruptive and inattentive classroom behaviors as key barriers to students’
successful educational performance. This study examined the impact of two class-wide positive
behavior support programs. The Mystery Motivator and Get ‘Em On Task interventions were im-
plemented in an alternating treatments design with fifth grade participants to decrease off-task
behaviors. Results indicated that both interventions effectively decreased off-task behavior at the
class-wide level. Implications and suggestions for future research on evidence-based behavioral
interventions are discussed. C
2012 Wiley Periodicals, Inc.
A COMPARISON OF THE MYSTERY MOTIVATOR AND THE Get ‘Em On Task
INTERVENTIONS FOR OFF-TASK BEHAVIORS
Disruptive behaviors are among the most prevalent behavior problems in childhood, accounting
for one half to one third of all referrals to child mental health settings (McMahon Estes, 1997).
Within the classroom, disruptive behaviors impact the learning process, reduce instruction time, and
make it more difficult for students to succeed academically (Luiselli, Putnam, Sunderland, 2002).
Positive Behavior Support (PBS) is a system to help parents and school staff members create and
maintain a safe, supportive learning environment. PBS practices and strategies are organized and
conceptualized to meet the needs of students with a vast range of behavioral challenges. To respond to
the challenges, PBS relies on a continuum of behavior supports based on implementing interventions
with differing specificities. The PBS model is a three-tiered system that focuses on school-wide
(Tier 1, or “primary”), classroom or small group (Tier 2, or “secondary”), and individual (Tier 3, or
“tertiary”) supports. The following study examines the efficacy of two Tier 2 behavior intervention
programs, The Mystery Motivator and Get ‘Em On Task.
PBS
Throughout the United States, schools and entire school districts are implementing PBS to im-
prove school-wide discipline (Hagan-Burke et al., 2005). Many schools implementing this approach
have reported a 20% to 60% reduction in office discipline referrals, as well as improved social
climate and academic gains (Cushing, 2000). PBS is based on the principles of applied behavioral
analysis (Safran Oswald, 2003).
The goal of PBS is to “apply behavioral principles in the community in order to reduce
problem behaviors and build appropriate behaviors that result in durable change and a rich lifestyle”
(Carr et al., 1999, p. 3). PBS initially evolved within the field of developmental disabilities and
emerged from three major sources including applied behavior analysis, the normalization/inclusion
movements, and person-centered values (Carr et al., 2002). We can also view PBS as rooted in
ecological theory. Bronfenbrenner (1979) viewed the relationships between individuals and their
environments as “mutually shaping” and saw the individual’s experience “as a set of nested structures,
each inside the next, like a set of Russian dolls” (Bronfenbrenner, 1979, p. 22). Bronfenbrenner’s
Correspondence to: Susan C. Davies, Department of Counselor Education and Human Services, 300 College
Park, University of Dayton, Dayton, OH 45469-0530. E-mail: sdavies1@notes.udayton.edu
163

164 Kraemer et al.
theory examines how several systems interact (e.g., family, workplace, and economy); therefore, the
study of PBS is a logical extension of his investigations. The theory is useful in helping educators
understand how systems within the child’s educational environment can interact to positively impact
his or her ability to access resources across the school system.
The Individuals with Disabilities Education Act (IDEA) of 1997, and its subsequent revision
in 2004, requires that local education agencies use PBS not only for students identified for special
education, but also for those whose problem behavior puts them at risk for special education
placements (IDEA, 2004). Thus, research is needed to identify strategies that are effective at different
levels of need. Typically, PBS is directed at three different levels of support: (a) primary (e.g., school-
wide), (b) secondary (e.g., classroom or small group), and (c) tertiary (e.g., individual) (Walker
Shin, 2002).
School-wide interventions focus on all students in all school settings, and typically all staff
members are involved in the implementation (Hagan-Burke et al., 2005). Turnbull and colleagues
(2002) conducted a study in an urban middle school in Kansas City to show the effects of imple-
menting a PBS. All staff members worked together to implement universal supports across all school
settings for all students by using and teaching clearly defined expectations. Data were systematically
collected to monitor progress toward the reduction of problem behavior as indicated by the reduc-
tion in the frequency of discipline. These collection techniques included interviewing school staff,
directly observing students across all school settings, tracking office discipline referrals, as well as
monitoring attendance, grades, and standardized test scores. Results indicated that after the first two
years the total number of office discipline referrals decreased by 19%, in-school conferences with
students decreased by 23%, timeouts (when students are required to sit in the office for a period
of time) decreased by 30%, in-school suspensions decreased by 12%, and short-term suspensions
decreased by 60% (Turnbull et al., 2002). Teachers and administrators also indicated that the rate
of progress was substantial given the challenges they face and their history of addressing those
challenges.
Successful implementation of a school-wide PBS should improve school culture and strengthen
pro-social behavior and learning outcomes for the majority (approximately 80%) of students (Horner
Sugai, 2000; Sugai et al., 2010). Recently, numerous publications have focused on school-
wide/systems level change (e.g., Deno et al., 2009; Lewis, Jones, Horner, Sugai, 2010). For those
students who need extra intervention, a secondary PBS can be implemented at the classroom or
small group level.
Tier 2 Interventions
Secondary-level interventions typically target students within a school who are considered to
be at risk for the development of chronic problem behavior patterns (Hagan-Burke et al., 2005).
Approximately 15% of students may require this level of intervention (Sugai et al., 2010). Several
recent articles have highlighted effective class-wide behavior interventions. These interventions
might involve consultation, such as “The Classroom Check-Up” (Reinke, Lewis-Palmer, Merrell,
2008) or have a self-management component, such as “It’s in the Cards” (Murphy Korinek, 2009).
Other strategies may incorporate group contingencies in which the class works together to achieve
goals, such as “Anchor the Boat” (Lohrmann Talerico, 2004). Group contingency interventions
may be implemented alone or used in conjunction with another strategy, such as Positive Peer
“Tootling” (Cihak, Kirk, Boon, 2009) or self-management (Davies Witte, 2000). The two Tier
2 interventions examined in the study were a group contingency intervention (“Mystery Motivator”)
and a teacher-monitored class-wide intervention (“Get ‘Em On Task”).
Psychology in the Schools DOI: 10.1002/pits

Behavior Interventions 165
Mystery Motivator
Mystery Motivators are recognition tools based on a lottery-like system that allows a person
to select from a variety of high- and low-value prizes for his or her engagement in targeted positive
behaviors (Wesley Mattaini, 1999). Mystery motivators are unknown rewards that have been shown
to be effective in improving disruptive behaviors (Kehle, Bray, Theodore, 2000). Anticipation
and interest are maintained as a result of the uncertainty of the reward. Each day targeted positive
behaviors are achieved, a person can select the corresponding day on a weekly chart. If the box
on the corresponding day contains a Mystery Motivator symbol, the person can select a reward
from the Mystery Motivator reward menu (Wesley Mattaini, 1999). PBS is best defined as a
system of support that includes proactive strategies for defining and supporting appropriate student
behaviors to create positive school environments; therefore, the Mystery Motivator is considered
a PBS because it is a proactive intervention in which appropriate behavior is defined, supported,
and rewarded. It can be implemented for targeted students or for an entire class. In this study, the
Mystery Motivator was a group contingency, in which the entire class worked together to achieve
rewards.
Get ‘Em On Task
Get ‘Em On Task is a computer-signaling program that helps teachers reward their students
based on an individualized auditory signal system for monitoring student behavior (Althouse, Jenson,
Likins, Morgan, 1999). This program can be used with an individual student or groups of students
to support any positive reinforcement or self-management program (Althouse et al., 1999). It allows
a teacher to use a classroom computer to generate random signals from 0 to 100 per hour with
additional imbedded bonus signals (Jenson, Olympia, Farley, Clark, 2004). The program can
run during the day and track when each signal occurs and what point value was assigned to
that signal. When a signal sounds, the teacher scans the classroom and assigns predetermined
points to students who are on task (Althouse et al., 1999). These students can then exchange the
earned points for rewards (Althouse et al., 1999). Thus, in the following intervention, Get ‘Em
On Task was used as a class-wide intervention, but students worked independently to accumulate
points.
Current research on PBS suggests that approximately 15% of students require targeted (Tier 2)
intervention and provides evidence that mystery motivators work as an intervention for many types
of behavior problems. The mystery motivator can be used with single students, teams, or whole
classrooms to increase or decrease types of behavior. Most studies conducted using the mystery
motivator as an intervention have been implemented at a class-wide level to focus on homework
completion. Little research has been conducted, however, on the implementation of the mystery
motivator as a prevention strategy for off-task behavior.
The purpose of this study was to determine the impact of two PBS programs, the Mystery
Motivator and the Get ‘Em On Task interventions, when implemented at the class-wide level. Al-
though intervention studies typically rate improvement with intervention in comparison to baseline,
this study potentially adds to prior research by comparing two different evidence-based classroom
interventions. It was hypothesized that students who were identified as demonstrating significant
levels of off-task behavior and participated in the Mystery Motivator intervention and the Get ‘Em
On Task intervention would decrease off-task behaviors when compared to those students with no
intervention.

166 Kraemer et al.
METHODS
Participants
One fifth grade English class (Class I) and one fifth grade Math class (Class II) were selected
from an elementary school in Ohio to participate in this study. The elementary school has a population
of 501 students. Each participating class has a total of 25 students (n = 25). The district serves a
population of 5,276 Pre-K to Grade 12 students. The community has a population of 12,380 (U.S.
Bureau of the Census, 2000). The school is composed of 3.2% Asian/Pacific Islander, 2.8% Multi-
Racial, 92.6% White, non-Hispanic, and 14.4% students with disabilities. The district is suburban,
and 8.9% of its students are classified as economically disadvantaged (on free or reduced lunch).
The study took place in the general education classroom. Classes were selected based on
teacher request, interviews with teachers regarding the severity of students’ off-task behavior, and
data collected through the use of the Behavioral Observation of Students in School (BOSS) (Shapiro,
1996).
Design
Single case designs are a valid methodology for establishing empirical interventions (Stoiber
Kratochwill, 2000). Zhan and Ottenbacher (2001) describe single case experimental designs as
practice based and practitioner oriented. In the school setting, such designs systematically docu-
ment the efficacy of interventions and do not significantly disrupt a classroom routine. A teacher,
intervention specialist, or school psychologist can easily record day-to-day behavior changes of
students, analyze the treatment, and modify the design as the intervention progresses. In fact, the
ability to modify treatment according to the child’s performance during the course of treatment is
one advantage of using this model (Zhan Ottenbacher, 2001), as it allows a school team to identify
optimum treatment for a specific child or group of children. Ultimately, single case experimental
designs can help bring research into the school setting and improve the likelihood that schools
implement research-based interventions to improve student learning. A single case design was cho-
sen for this study so that data could be collected and analyzed without significantly disrupting the
classroom routine. The intervention was implemented using an ABCACBA alternating treatment
design in which A was the baseline, B was the Mystery Motivator and C was the Get ‘Em On Task
intervention. Thus, the independent variables in this study were the Mystery Motivator and the Get
‘Em On Task intervention. The dependent variable was the off-task behavior.
Materials
The BOSS (Shapiro, 1996) was used by independent observers to conduct on-task/off-task
observations of participants throughout the study. It is an observation-coding system used for as-
sessing academic behavior in the classroom and assists observers in measuring levels of on-task
and off-task behaviors via momentary time sampling in 15-second intervals for a period of at least
15 minutes. The on-task behaviors are identified as active engagement and passive engagement, and
the off-task behaviors are identified as off-task motor, off-task verbal, or off-task passive (Hintze,
Volpe, Shapiro, 2002). For the purpose of this study, each observation period lasted 15 minutes
and was divided into 60 intervals, each of which was 15 seconds in length. Frequency of engagement
or off-task behavior was collected using a momentary time-sampling procedure at the beginning of
each 15-second interval. Off-task behavior was coded using a partial interval recording schedule,
where the occurrence of each behavior was recorded only once during each interval. The classrooms
were divided into four quadrants, with each row of students representing a quadrant. The purpose
of using quadrants was to allow the observer to collect a random sample of the room. The order of

quadrants was randomly assigned for each observation session at the beginning of the study. During
each time sample, the observer recorded the behavior of the students in the quadrant being observed.
During the next time sample, the next quadrant was observed, and so forth. For each observation,
a timer was set to 15 minutes. The students were in four rows of five to six students. The observer
started the timer and recorded the behavior of the first student in the first row by placing a “+” sign
in the box of the indicated behavior (e.g., off-task verbal if the student was talking or on-task passive
if the student was listening to the teacher). After 15 seconds, the observer recorded the behavior of
the first student in the second row. This behavior recording continued throughout the four rows.
When the observer came back to row one, the second student in the first row was observed and so
forth until the 15 minutes concluded.
Materials needed for the Mystery Motivator intervention included “invisible” markers, weekly
behavior charts, a reward menu for each student, and the identified rewards (Wright, 2004b). Crayola
Color Switchers are watercolor markers that were used for this study; they include various colors
and a special pen with transparent ink. A mark was placed on the Mystery Motivator chart using the
special marker. When the transparent ink was colored over with one of the watercolor pens, a symbol
“magically” appeared in the square. The weekly behavior chart contained a space for each day of the
week. A reward survey was created using the Online Reinforcer Survey Generator (Wright, 2004a).
This survey was conducted with the students to ensure that the reinforcers selected were significant
for the class. Students were asked to list three prizes they would most like to receive from the teacher,
as well as their three favorite classroom activities. A reward menu was then developed based on
the students’ responses to the survey (e.g., candy, 10 minutes of extra recess, and 5 minutes of an
in-class game).
Materials needed for the Get ‘Em On Task intervention included the Get ‘Em On Task computer
program, a computer, a Point Card for each student, a classroom bank where points were recorded
and saved by students, a reward menu with the cost (in points) for each reward, and the identified
rewards (Althouse et al., 1999). For example, a reward menu could be 100 points = homework pass
for spelling/grammar; 75 points = late homework pass for 1 day late; 50 points = computer time;
25 points = candy.
Procedure
After classrooms were chosen, a consent form was sent home that explained the purpose of
the study, the benefits to the student, and a description of how confidentiality was maintained. The
percentage of consent for Class I was 96% and for Class II was 100%. No data were collected
on the students whose parents did not consent. For those whose parents did consent, no records
were disclosed to others; data from all participants were pooled. After permission was obtained,
an informal interview was held with the participants’ teachers to indicate the specific behaviors of
concern that were observed in the classroom. Off-task behavior was a significant problem in both
classrooms; it was defined as calling out, getting out of their seats, and disturbing other students.
Baseline data were collected by the primary researcher and a secondary observer using the BOSS
to set appropriate goals for each class. Data were collected through observation twice a week for 2
weeks. The primary researcher also interviewed the class to create reward menus for the interventions
and trained the classroom teachers on how to implement both the Mystery Motivator and Get ‘Em
On Task interventions. Both teachers were given written directions for each intervention to read to
the class.
Next, the Mystery Motivator was implemented daily for 2 weeks during a 45-minute class
period. The Mystery Motivator Weekly Chart was created for the classes. The goal was the minimum
behavioral criteria the class needed to meet to earn a chance to fill in a blank on the Mystery Motivator

168 Kraemer et al.
Chart. The teacher introduced the Mystery Motivator by first explaining that the class would have
the chance to earn rewards for good behavior. The teacher reviewed the target behaviors that were
selected and used demonstration and modeling to ensure that students clearly knew either the negative
behaviors that should be avoided (e.g., calling out, bothering others) or the positive behavior that
should be increased (e.g., staying in seat, working quietly). The teacher then introduced the Mystery
Motivator Chart. The teacher explained that the students could earn a chance to fill in the blank on
the chart for the current day to uncover a possible reward if they demonstrated on-task behaviors.
The teacher reviewed the target behaviors and goals posted on the Mystery Motivator Weekly Chart.
Next, the teacher let the class know that the magical letter “M” had been secretly placed in some,
but not all, of the chart squares. If the class revealed the “M” in the chart, a reward could be selected
from the reward menu. If the letter “M” did not appear, the teacher would congratulate and praise
the class for their good behavior but no reward would be given. The teacher also let the class know
that they would have another chance to fill in the Mystery Motivator Chart the next day. At the end
of each week, if the class had met the criteria, the class was able to fill in the Bonus Points box in
which they could receive the reward that appeared in the box. For this intervention, the teacher used
Crayola Color Switchers to mark an “M” in 3 of the 5 days on the weekly chart and wrote a reward
from the reward menu in the Bonus Points box. The boxes varied each week and did not create a
pattern.
To implement the alternating treatment design, the Get ‘Em On Task computer program (Alt-
house et al., 1999) was then implemented daily for 2 weeks during a 45-minute class period. The
teacher first asked the students what they would like to earn in the classroom store. These items
included special activities, privileges, and treats. Next, the teachers had the students vote on the best
reward items and rank them. The highest ranked items cost the most points. Goals were created
based on points per day. Each student had the possibility of earning 10 points per day (100 points
per 2 weeks). After an interval time was set, the teacher implemented the intervention. A copy of
the Point Card was then given to each student. The teacher explained to the class that when the hour
begins and a signal sounds, she would scan the classroom. If students were off task, the teacher
would instruct those students (by name) to mark an X (no points) for the interval. The teacher would
then praise the rest of the class for being on task and instruct them to mark a point for themselves.
Bonus signals could also be used in which 2 points were allotted if students were on task.
At the end of the day, the students added the points in their bank total. This was also a good
time for teachers to review daily progress with students. A classroom store exchange was held
at the end of the 2 weeks in which students could exchange the earned points for items that had
been predetermined as a class. The cost of each item was then subtracted from that student’s bank
total. After each intervention was implemented for 2 weeks, the classrooms reverted to the “no
intervention” condition for 2 weeks. Next, Get ‘Em On Task was implemented again for 2 weeks,
followed by the Mystery Motivator, and a final return to baseline (no intervention) conditions.
Treatment Fidelity
Direct observation using the BOSS was conducted by the researcher two times per week for the
remainder of the study. These observation times occurred on Mondays and Wednesdays each week.
The ABCACBA alternating treatment design was implemented at 2-week intervals after baseline data
were collected. Data were collected at an interval level based on the BOSS direct observation tool.
The Pearson product-moment coefficient was used to measure (r), comparing the data to an alpha
level of 0.05. Interobserver agreement measures were collected by a secondary observer at the same
time as the primary researcher. Interobserver agreement estimates for on- and off-task behavior were
calculated by scoring an agreement when both observers recorded identical frequencies of on- or
off-task behavior during each 15-second interval. Interobserver agreement estimates were calculated

by finding the Kappa Coefficient. According to Fleiss (1981) and Cicchetti (1994), values of less
than .40 are poor, values of .40 to .60 suggest fair agreement, values of .60 to .75 represent good
agreement, and values greater than .75 indicate excellent agreement. For this study, interobserver
agreement was deemed acceptable if average agreement between two individual observers was
calculated at a Kappa Coefficient of .60 or higher.
Integrity checklists were given to the teachers to ensure that both interventions were imple-
mented as they were intended. The checklists stated the materials that were needed and listed each
step of the intervention in a script format. Treatment integrity checklists were completed by the
teacher for each day of baseline and intervention and were collected by the primary researcher on a
weekly basis.
Treatment Acceptability
Intervention acceptability was measured using the Behavior Intervention Rating Scale (Elliott
Treuting, 1991). The scale was developed as an instrument to measure teachers’ perceptions
of treatment acceptability and perceived efficacy of classroom interventions. The teacher, upon
completion of the study, completed this measure to assess satisfaction with the interventions. The
items were rated on a 6-point Likert scale ranging from 1, indicating a strong disagreement, to
6, indicating a strong agreement with the provided statements (e.g., “Most teachers would find
this intervention appropriate for challenging behaviors,” “The intervention would be an appropriate
intervention for a variety of children”). A survey was also used to measure intervention acceptability
for students. This survey contained six questions about each intervention on a 5-point Likert scale
ranging from 1 (strongly disagree) to 5 (strongly agree), such as “I liked the Mystery Motivator” and
“Get ‘Em On Task helped me behave better.” The survey was completed individually by students.
When students were finished, the survey was placed face-down on a table in the back of the room.
Students did not place names on the survey forms. Students who did not receive consent were asked
to quietly read a book until all students were finished filling out the survey.
RESULTS
AccordingtoHunleyandMcNamara(2010), whenconductingasinglecasedesignthefollowing
data should be collected when interpreting and analyzing results: visual analysis, effect size (ES),
and Goal Attainment Scaling (GAS). Visual inspection was conducted to determine general patterns
of the data across baseline and intervention conditions. ES measured the magnitude of change across
various phases of intervention. GAS determined how closely student performance aligned with
the expected level of outcome for the goal. Additional data, such as non-overlapping data points,
treatment integrity, and intervention acceptability, were also collected.
Figure 1 displays the intervention results for Class I. The graph displays the average percentage
of off-task behavior during baseline and at each stage of intervention. A represents the baseline, B
represents the Mystery Motivator intervention, and C represents the Get ‘Em On Task intervention.
Each week the class was monitored for progress on two occasions. The original baseline for Class
I was 34% (week 1) and 38% (week 2). Figure 1 demonstrates that during the first intervention
phase, the Get ‘Em On Task (C) intervention was slightly more successful at decreasing off-task
behavior than the Mystery Motivator intervention (B). By the end of the second phase of each
intervention, however, student performance was similar, and demonstrated significant improvement
when compared to the original baseline.
Figure 2 displays the intervention results for Class II. Each week the class progress was
monitored on two occasions in the same manner as for Class I. The original baseline for Class II
was 14.5% (week 1) and 31.5% (week 2). Figure 1 demonstrates that during the first intervention

170 Kraemer et al.
Class I Percentage of Off-Task Behavior
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
34
36
38
40
1 3 5 7 9 11 13 15 17 19 21 23 25 27
Days
%
of
off-task
behavior
% of off-task behavior
A
B
C
A
C
B
A
FIGURE 1. Baseline and progress-monitoring data for Class I.
Class II Percentage of Off-Task Behavior
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
34
36
38
40
1 3 5 7 9 11 13 15 17 19 21 23 25 27
Days
%
of
off-task
behavior
% of off-task behavior
A B C A C B A
FIGURE 2. Baseline and progress-monitoring data for Class II.
phase, the Get ‘Em On Task (C) intervention was slightly more successful at decreasing off-task
behavior than the Mystery Motivator intervention (B). By the end of the second phase, however,
student performance in Class II under both Mystery Motivator and Get ‘Em On Task interventions
demonstrated significant improvement when compared to the original baseline.
In this study, the specific type of ES used is referred to as the d-index. The d-index was
calculated as the intervention mean minus the baseline mean divided by the standard deviation of

Table 1
Percentage of Off-Task Behavior Corresponding to GAS Ratings
GAS Rating −2 −1 0 +1 +2
Class I – Percent Off-Task ≥43.2% 43.2% 36% 36% 36% 28.8% ≤28.8%
Class II – Percent Off-Task ≥27.6% 27.6% 23% 23% 23% 18.4% ≤18.4%
all the data for each intervention. The d-index was used to determine the magnitude of a change in
level when the data do not indicate a trend. When calculated, it takes into account each data point’s
actual score. Cohen (1992) recommends using ±.2, ±.5, and ±.8 as guidelines for approximating
a small, medium, and large effect. The d-index should be interpreted with caution and used along
with visual inspection of the data. The ES values for both interventions were calculated based on
the weekly average. The ES for Get ‘Em On Task was calculated as −1.72 for Class I and −1.46 for
Class II, which is considered to be a large effect. The ES for the Mystery Motivator was calculated
as −1.62 for Class I and −1.07 for Class II, which is also considered to be a large effect.
GAS was used to evaluate the overall rate of off-task behavior during both the Mystery Motivator
intervention and the Get ‘Em On Task intervention (see Table 1). The scale was created following
the guidelines for a percentage-of-change technique identified by Hunley and McNamera (2010) to
enhance scale validity. The mean baseline after 2 weeks for Class I was 36% off-task. A rating of 0
would indicate no change for Class I between baseline and intervention with a score of 36% off-task
during the intervention phase. Twenty percent of the baseline score (7.2) was added or subtracted
from the baseline score to create the ±2 ratings for the GAS; −2 = 43.2% and +2 = 28.8%. By the
end of the Mystery Motivator intervention, Class I decreased off-task behaviors to 5% for a GAS
score of +2. For the Get ‘Em On Task intervention, Class I decreased off-task behavior to 7% for a
GAS score of +2. This GAS indicates a much improved level of attainment for both interventions.
The mean baseline after 2 weeks for Class II was 23% off task. A rating of 0 would indicate
no change for Class I between baseline and intervention with a score of 23% off task during the
intervention phase. Twenty percent of the baseline score (4.6) was added or subtracted from the
baseline score to create the ±2 ratings for the GAS ; −2 = 27.6% and +2 = 18.4%.
Class II decreased off-task behavior to 6% by the end of the Mystery Motivator intervention to
be assigned a GAS score of +2. By the end of the Get ‘Em On Task intervention, Class II decreased
off-task behavior to 5% for a GAS score of +2. This GAS indicates a much improved level of
attainment for both the Mystery Motivator and Get ‘Em On Task interventions.
The percentage of non-overlapping data points (PND) was determined by examining the data
across both baseline and intervention phases and calculating the percentage of intervention points
that do not overlap the baseline points (the number of data points that do not overlap divided by the
number of total data points). For this study, the PND was calculated based on a weekly average. The
smaller the percentage of overlap, the greater the intervention effect. Scruggs, Mastropieri, Cook,
and Escobar (1986) suggest that the criteria for evaluating PND are 50% or lower PND = ineffective
or unreliable; 50-70% PND = questionably effective; 70-90% PND = moderately effective; and
90% or higher PND = highly effective. It is important to note that PND lacks sensitivity for highly
successful interventions (e.g., 100% PND), and it only uses the most extreme baseline data point to
compare to the intervention data. (“Extreme” is the highest baseline point if the intervention is to
increase a behavior or the lowest baseline point if the intervention is to decrease a behavior.) Thus,
in this study, because the goal is to decrease off-task behavior, one unusually low baseline data point
can affect the reliability of the PND. The PND for the Mystery Motivator was calculated at 100%
for Class I and 75% for Class II, which is considered to be highly effective in Class I and moderately

172 Kraemer et al.
Table 2
Summary of Data Analysis Results
A B C A C B A
Class I
Off-task behavior 36 23 13.45 24.5 5.5 8 20
ES ∗∗ −1.67 −1.51 ∗∗ −1.94 −1.80 ∗∗
GAS 0 +2 +2 +2 +2 +2 +2
PND ∗∗ 100% 100% ∗∗ 100% 100% ∗∗
Class II
Off-task behavior 23 16.25 5 23.5 3.5 7.5 16.5
ES ∗∗ −0.82 −1.51 ∗∗ −1.63 −1.49 ∗∗
GAS 0 +2 +2 −1 +2 +2 +2
PND ∗∗ 25% 100% ∗∗ 100% 100% ∗∗
∗∗Baseline data points.
effective in Class II. The PND for the Get ‘Em On Task intervention was calculated at 100% for
Class I and Class II, which is considered highly effective (see Table 2).
A treatment integrity checklist was completed for each day of baseline and intervention. The
results of the checklist indicated that the study was implemented as planned approximately 93% of
the time. On rare occasions, the teacher was unable to complete a step of the intervention, such as
instructing students to mark point cards at an appropriate interval.
Interobserver agreement was assessed for the occurrence or nonoccurrence of off-task behav-
iors using the Kappa Coefficient of Agreement. Fleiss (1981) and Cicchetti (1994) have provided
interpretative guidelines. Values of less than .40 are poor, values of .40 to .60 suggest fair agree-
ment, values of .60 to .75 represent good agreement, and values greater than .75 indicate excellent
agreement. Interobserver agreement was conducted once a week for the 14-week implementation
period and indicated a value of .71, which represents good agreement.
Intervention acceptability was assessed for both the Mystery Motivator and the Get ‘Em On Task
intervention using the Behavior Intervention Rating Scale (Elliott Treuting, 1991). The teacher
evaluated the intervention by circling the number (1–6) which best described the disagreement or
agreement with each statement. The teacher in Class I scored the Mystery Motivator as having an
average of 5.04 rating and the Get ‘Em On Task as having an average of 5.59 rating, indicating that
both interventions were deemed acceptable. The teacher in Class II scored the Mystery Motivator
as having an average of 5.46 rating and the Get ‘Em On Task as having an average of 5.76 rating,
indicating that this teacher also found both interventions acceptable. Both teachers noted strengths
and weaknesses for each intervention, felt the interventions were appropriate for challenging behav-
iors, and they both indicated they would suggest the interventions to other teachers. Both teachers,
however, noted that they felt the students’ behavior may not remain at an improved level after the
intervention is discontinued and that the intervention may not improve the students’ behavior in
other settings (e.g., other classrooms, home).
The students from each class were also asked to evaluate the acceptability of the Mystery
Motivator and Get ‘Em On Task interventions. This survey was a 5-point Likert scale where 1
indicated “strongly disagree” and 5 indicated “strongly agree.” Class I scored the Mystery Motivator
as having an average of a 3.59 rating; Class II’s rating was 3.5. Thus, both classes “somewhat liked”
the Mystery Motivator. Class I scored Get ‘Em On Task as having an average of 3.98 rating; Class
II’s rating was 3.97. Thus, Get ‘Em On Task was also “somewhat liked” by students.

DISCUSSION
The results of this study indicate that both the Mystery Motivator and the Get ‘Em On Task
interventions effectively decreased off-task behaviors when compared to no intervention. As seen
in the progress-monitoring charts, the Get ‘Em On Task intervention was somewhat more effective
than the Mystery Motivator, with a difference in the decrease in overall off-task behavior of 16.75%.
When we examine the end points of the interventions, however, both were equally successful. In
other words, Get ‘Em On Task decreased off-task behaviors more quickly, but both interventions
were equally effective after several weeks of implementation.
The Mystery Motivator involved a group contingency in which students had a mutual goal.
Students were aware that their individual behavior impacted other students in the class. Although
peer pressure can elicit negative side effects, no such results were noted. The intervention promoted
positive interdependence, and students reminded one another about the reward they were trying
to earn. In any group contingency, it is essential that teachers carefully monitor the classroom
environment, encourage positive interactions among students, and intervene if there are instances
of students becoming “scapegoats” or targets of belittling if classmates perceive they made the
group “lose” their reward. In the present study, no negative interactions were observed; however,
researchers conducting future studies and teachers implementing group contingencies should remain
aware of this possibility.
Get ‘Em On Task involved students earning points individually, which allowed for a greater
degree of individual student accountability. Although it required “banking” points and delaying
gratification (because rewards were given at the end of the 2-week period), students had the benefit
of receiving immediate feedback (“Was I on-task during the beep? Did I earn a point?”). Furthermore,
although individual student data sheets were not collected or evaluated as part of this study, Get ‘Em
On Task allowed the collection of ongoing progress monitoring through individual student point
sheets. Therefore, a teacher can implement the program for the entire class, but evaluate whether it
is or is not effective for specific students. These data would be excellent to use to determine whether
certain class members require more intensive (Tier 3, tertiary) behavior supports.
Teacher attitudes toward an intervention are important; both teachers participating in this study
had positive, open attitudes toward both interventions. They were interested in participating in
the study and implemented the programs with a high degree of integrity. The Get ‘Em On Task
intervention was rated as slightly more acceptable by both teachers and students. Student responses
indicated that Get ‘Em On Task was somewhat more successful. They reported that they could stay
on-task better when they monitored their own behavior, instead of having the teacher decide whether
the whole class was on-task as is done during the Mystery Motivator intervention. Students who
liked the Mystery Motivator better, however, noted this liking because there was a chance to receive
a reward every day and then earn a bonus reward if all squares were filled in for the week.
Limitations
One limitation to this study is the limited external validity of single-subject designs. Because
the participants were from two classrooms in one school district, it is not representative of the overall
population, thereby limiting generalizability of results. Another limitation is the scheduling con-
straints due to holiday breaks and the fifth grade class attending camp. Because of these constraints,
the time between alternating treatments was not consistent. A further limitation is the lack of counter-
balancing. Both classes followed the same ABCACBA alternating treatment design. Future research
could be conducted in which the intervention implementation pattern could be altered for each class.
Some limitations of the Get ‘Em On Task intervention include the cost of the program as well as
the extra cost of the licensures for any extra computers that the program will be placed on. Another

174 Kraemer et al.
limitation for the Get ‘Em On Task intervention is the need for access to a computer in the classroom.
Although many schools now have this access, some schools may have limited technology resources.
Implications and Future Research
Behavior problems in the classroom indicate a strong need for practical, effective interventions
that can be easily used within general education settings. The Mystery Motivator can be used
with single students, teams, or whole classrooms to increase or decrease specific behaviors (Wright,
2004b). Get ‘Em On Task can also be used with single students, teams, or whole classrooms to support
any positive reinforcement or self-management program (Althouse et al., 1999). When teachers or
school psychologists determine a need for Tier 2 interventions to decrease off-task classroom
behaviors, either intervention may be useful. The interventions allow teachers to concentrate more
on academics than behavior management. Both were “fun,” involved desirable rewards, and allowed
all students (not just the ones needing more intensive interventions) to participate.
Educators who want to incorporate a self-monitoring component and can pay the initial cost
associated with implementing Get ‘Em On Task may find it beneficial, particularly because results of
this study indicated that it was somewhat more effective in immediately decreasing off-task behavior.
The Mystery Motivator, however, was also effective. It may be more beneficial to educators who
want an inexpensive, easy classroom management program that involves a group contingency in
which the class works together to achieve a common goal.
Practitioners can use these interventions with any population and any level of behavior. To
strengthen the research for both the Mystery Motivator and Get ‘Em On Task interventions, however,
further investigation with both older and younger children, as well as with children from types of
school districts different than those targeted in this study, would be beneficial. Furthermore, this study
did not examine whether these interventions were associated with a reduction of off-task behaviors
in other settings. Because the ultimate goal of PBS is to improve behavior across school settings, a
future study examining generalization of behavior change as a result of these Tier 2 interventions
would be beneficial. Additional research may be conducted to investigate academic outcomes and
work production when implementing these interventions. The authors were encouraged by the results
from the study and encourage further examination of these Tier 2 interventions.
REFERENCES
Althouse, R. B., Jenson, W. R., Likins, M., Morgan, D. P. (1999). Get ‘em on task. Longmont, CO: Sopris West.
Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard
University Press.
Carr, E. G., Dunlap, G., Horner, R. H., Koegel, R. L., Turnbull, A. P., Sailor, W., et al. (2002). Positive behavior support:
Evolution of an applied science. Journal of Positive Behavior Interventions, 4(1), 4–16. Retrieved from EBSCO MegaFile
Database.
Carr, E. G., Horner, R. H., Turnbull, A. P., Marquis, J. G., McLaughlin, D. M., McAree, M. L., et al. (1999). Positive behavior
supports for people with developmental disabilities: A research synthesis. Washington, DC: American Association on
Mental Retardation.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments
in psychology. Psychology Assessment, 6, 284–290. Retrieved from EBSCO MegaFile database.
Cihak, D. F., Kirk, E. R., Boon, R. T. (2009). Effects of classwide positive peer “tootling” to reduce disruptive classroom
behaviors of elementary students with and without disabilities. Journal of Behavioral Education, 18(4), 267–278.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.
Cushing, L. S. (2000). Descriptive analysis in the school social culture of elementary and middle school students. Unpublished
doctoral dissertation, University of Oregon, Eugene.
Davies, S., Witte, R. (2000). Self-management and peer monitoring within a group contingency to decrease uncontrolled
verbalizations of children with attention deficit-hyperactivity disorder. Psychology in the Schools, 37, 137–147.

Deno, S. L., Reschly, A. L., Lembke, E. S., Magnusson, D., Callender, S. A., Windram, H., et al. (2009). Developing a school-
wide progress-monitoring system. Psychology in the Schools, 46(1), 44–55. Retrieved from EBSCO MegaFile Database.
Elliot, S. N., Treuting, M. V. (1991). The behavior intervention rating scale: Development and validation of a pretreatment
acceptability and effectiveness measure. Journal of School Psychology, 29(1), 43–51. Retrieved from EBSCO MegaFile
database.
Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley.
Hagan-Burke, S., Burke, M. D., Martin, E., Boon, R. T., Kirkendoll, D. (2005). The internal consistency of the school-wide
subscale of the effective behavioral support survey. Education and Treatment of Children, 28(4), 400–413. Retrieved
from EBSCO MegaFile database.
Hintz, J. M., Volpe, R. J., Shapiro, E. S. (2002). Best practices in systematic, direct observation of student behavior.
In A. Thomas J. Grimes (Eds.), Best practices in school psychology IV (pp. 993–1006). Bethesda, MD: National
Association of School Psychologists.
Horner, R. H., Sugai, G. (2000). School-wide behavior support: An emerging initiative. Journal of Positive Behavior
Interventions, 2, 231–232.
Hunley, S. A., McNamera, K. (2010). Tier 2 of the RTI Model. Washington, DC, National Association of School
Psychologists.
Individuals with Disabilities in Education Act (IDEA) (2004). Washington, DC: U.S. Government Printing Office.
Jenson, W. R., Olympia, D., Farley, M., Clark, E. (2004). Positive psychology and externalizing students in a sea of
negativity. Psychology in the Schools, 41(1), 67–79. Retrieved from EBSCO MegaFile database.
Kehle, T. J., Bray, M. A., Theodore, L. A. (2000). A multi-component intervention designed to reduce disruptive classroom
behavior. Psychology in the Schools, 37(5), 475–481. Retrieved from EBSCO MegaFile database.
Lewis, T. J., Jones, S. E., Horner, R. H., Sugai, G. (2010). School-wide positive behavior support and students with
emotional/behavioral disorders: Implications for prevention, identification and intervention. Exceptionality, 18(2), 82–
93. Retrieved from EBSCO MegaFile Database.
Lohrmann, S., Talerico, J. (2004). Anchor the boat: A classwide intervention to reduce problem behavior. Journal of
Positive Behavioral Interventions, 6, 113–120.
Luiselli, J. K., Putnam, R. F., Sunderland, M. (2002). Longitudinal evaluation of behavior support intervention in a public
middle school. Journal of Positive Behavior Interventions, 4, 182–188. Retrieved from EBSCO MegaFile database.
McMahon, R. J., Estes, A. M. (1997). Conduct disorders. In Lea A. Theodore (Ed.), Assessment of childhood disorders
(pp. 130–193). New York: Guilford Press.
Murphy, S. A., Korinek, L. (2009). It’s in the cards: A classroom management system to promote student success.
Intervention in School and Clinic, 44(5), 300–306.
Reinke, W. M., Lewis-Palmer, T., Merrell, K. (2008). The classroom check-up: A class wide teacher consultation model
for increasing praise and decreasing disruptive behavior. School Psychology Review, 37(3).
Safran, S. P., Oswald, K. (2003). Positive behavior supports: Can schools reshape disciplinary practices? Exceptional
Children, 69(3). Retrieved from EBSCO MegaFile database.
Scruggs, T. E., Mastropieri, M. A., Cook, S. B., Escobar, C. (1986). Early intervention for children with conduct disorders:
A quantitative synthesis of single-subject research. Behavioral Disorders, 11, 260–271.
Shapiro, E. S. (1996). Academic skills problems workbook. New York: Guilford Press.
Stoiber, K. C., Kratochwill, T. R (2000). Empirically supported interventions and school psychology: Rationale and
methodological issues. Part 1. School Psychology Quarterly, 15, 75–105.
Sugai, C., Horner, R. H., Algozzine, R., Barrett, S., Lewis, T., Anderson, C., et al. (2010). School-wide positive behavior
support: Implementers’ blueprint and self-assessment. Eugene, OR: University of Oregon.
Turnbull, A., Edmonson, H., Griggs, P., Wickham, D., Sailor, W., Freeman, R., et al. (2002). A blueprint for school wide
positive behavior support: Implementation of three components. Council of Exceptional Children, 63(3), 377–402.
Retrieved from EBSCO MegaFile database.
U.S. Bureau of the Census (2000). Profile of General Demographic Characteristics: Springboro City, OH. Retrieved December
18, 2009, from http://censtats.census.gov/data/OH/1603974076.pdf
Walker, H. M., Shinn, M. R. (2002). Interventions for achieving prevention goals and outcomes for at-risk children
and youth. In M. R. Shinn, H. M. Walker, G. Stoner (Eds.), Interventions for academic and behavior problems II:
Preventive and remedial approaches. Bethesda, MD: National Association of School Psychologists.
Wesley, J., Mattaini, M. (1999). Mystery motivator. Peace Power Tools. Retrieved February 1, 2009, from
www.peacepower.info
Wright, J. (2004a). Jackpot! On-Line Reinforcer Survey Generator. Retrieved October 30, 2008, from
http://www.jimwrightonline.com/php/jackpot/jackpot.php
Wright, J. (2004b). Mystery Motivator. Retrieved January 5, 2009, from www.interventioncentral.org
Zhan, S., Ottenbacher, K. J. (2001). Single subject research designs for disability research. Disability and Rehabilitation,
23, 1–8.

A Comparison Of The Mystery Motivator And The Get Em On Task Interventions For Off-Task Behaviors

Recommended

Recommended

More Related Content

Similar to A Comparison Of The Mystery Motivator And The Get Em On Task Interventions For Off-Task Behaviors

Similar to A Comparison Of The Mystery Motivator And The Get Em On Task Interventions For Off-Task Behaviors (20)

More from Addison Coleman

More from Addison Coleman (20)

Recently uploaded

Recently uploaded (20)

A Comparison Of The Mystery Motivator And The Get Em On Task Interventions For Off-Task Behaviors