7TH GRADE LIFE SCIENCE

McDougal Littell
Evidence-Based Small-Scale Study Final Report:
7TH
GRADE LIFE SCIENCE
Technical Report, 07-31-05
by
Catherine Callow-Heusser, Ph.D. (ABD), Principal Investigator and Director
cheusser@endvision.net
Douglas Allred, M.S., Project Coordinator
red@endvision.net
Daniel Robertson, Ph.D., Senior Research Analyst
danr@endvision.net
EndVision Research & Evaluation, LLC
41 E. University Blvd. #321
Logan, UT 84321
(435) 881-8811
Geoffrey D. Borman, Ph.D., Associate Professor
gborman@education.wisc.edu
Maritza Dowling, Research Assistant
mdowling@ccbc.education.wisc.edu
Educational Leadership and Policy Analysis
University of Wisconsin-Madison
1161D Educational Sciences Building
1025 West Johnson St.
Madison, WI 53706-1796
(608) 263-3688

i
EXECUTIVE SUMMARY
As a small-scale feasibility study, this study demonstrated that a randomized
experimental design could be successfully conducted in typical school settings. The study
initially included 31 7th
grade classrooms whose teachers were randomly assigned to either the
treatment group, which used the McDougal Littell Life Science, or to a comparison group using
science curriculum that teachers had previously used. Mixed methods were used to gather data
through student assessment, classroom observations, surveys, interviews, and focus groups.
After approximately 18-20 weeks of implementation (e.g., one half of the school year), 29
classrooms completed the study and were represented in the final analysis of student outcomes.
While this relatively small number limits statistical power, it was sufficient to show differences
that consistently favor the treatment group and that suggest McDougal Littell Life Science was
more effective than life science curriculum used in comparison classrooms.
The impact analyses focused on attitudinal and achievement changes for over 700
students nested within 29 classrooms. The overall unconditional standardized mean difference
effect size between treatment and comparison groups for average student gains on a science
content test was δ = 0.27. This difference shows a small but potentially important difference in
gains. Hierarchical linear model analyses, which take into account both student- and classroom-
level sources of variability in the outcome, were conducted to estimate the classroom-level effect
of random assignment. This method was used because students randomized together within any
one classroom or school are more likely to respond in a similar manner than students randomized
from different clusters. The analysis identified the unique impact of McDougal Littell Life
Science while accounting for both pre-existing student and classroom differences, and any effect
of classroom-clustering, including the impact from differences in characteristics of teachers. The
analysis revealed no statistically significant classroom-level effects of assignment to implement
McDougal Littell Life Science on the science achievement outcome. The magnitude of the
effect, δ = .15, was of some practical significance, but it was not large enough to achieve
statistical significance given the current design, which included a total of only 29 classrooms.
For the average McDougal Littell classroom in the sample, this effect is equivalent to moving the
entire class from the 50th
to the 56th
percentile while the average comparison classroom remained
at the 50th
percentile.
The hierarchical linear model analysis indicated that the impact of assignment on the
science attitudinal measure, though, did serve to narrow the gap between students who began the
treatment with poorer and better attitudes toward science. In addition, there was a statistically
significant difference between treatment teachers using McDougal Littell Life Science and
comparison teachers with respect to their implementation of high-quality research-based
classroom practices and curricula, with a magnitude of effect δ = 0.15. Relative to the
comparison teachers, treatment teachers implemented research-based classroom practices and
curricula with greater quality and consistency.
Utilizing an instrumental variables approach for estimating the treatment effect,
additional analyses specifically addressed program implementation and how it influenced
students’ achievement and attitudinal outcomes. Assignment to McDougal Littell Life Science
had a positive and statistically significant effect on teachers’ uses of research-based instructional
practices and curricular materials, but the instrumental variables analyses suggested that these
benefits for teachers did not immediately translate into large and statistically significant effects
© ΣndVision Research & Evaluation, Logan UT

ii
on the student outcomes. The most consistent predictors of student outcomes were student- and
classroom-level pretest scores on the achievement and attitude measures. That is, students and
classrooms beginning the study with higher achievement and more positive attitudes towards
science tended to end the study with better achievement and attitudinal outcomes. Exploratory
regression trees analyses revealed that McDougal Littell Life Science helped to counteract this
trend in the poorest performing classrooms. The 75 treatment students experienced a positive
and statistically significant impact relative to the 47 comparison students from the five lowest-
performing classrooms. This result provided some suggestive evidence that McDougal Littell
Life Science could be particularly beneficial for high-needs students from low-performing
schools and classrooms.
Overall, teachers viewed McDougal Littell Life Science favorably. While they made
many suggestions for improving the program, the positive comments and attitudes towards the
program were quite strong. Teachers and students stated that the most effective and valuable
components were (a) the notetaking and vocabulary strategies, (b) the organization of the
materials—including “Big Ideas,” “Key Concepts,” and chapter/section headings, (c) the
resources supporting differentiated instruction, and (d) the technology components that
supported instruction.
Because of the many positive findings of this study, we recommend that funding for a
larger-scale study be sought to investigate the effectiveness of the McDougal Littell Science
Series in grades 6-8. Changes in teaching practices and student outcomes should be measured,
and the magnitude of the relationship between teaching practices and student outcomes should be
investigated, including mediators and moderators of these relationships. Additionally, this study
identifies additional factors that would need to be considered in a larger study, and as such, lays
the groundwork for a larger-scale study to be conducted successfully.

iii
ACKNOWLEDGEMENTS
Many people were involved in helping us successfully conduct this study, and they deserve
recognition and thanks.
Carol Guziak, Manager of Educational Research at McDougal Littell, for working diligently to
grasp the nuances of educational research and for supporting an evidence-based study of
McDougal Littell Science.
Douglas Carnine, instructional designer for McDougal Littell Science, for providing advice to
design the study and support as we conducted it.
Kathy Zantal-Weiner, friend and colleague, for helping to lay the groundwork for the study.
Duncan Drummond and Dave Sutor at McDougal Littell, for making many more phone calls
than we did to request the participation of school districts and teachers. They have heard more
“No” responses than anyone should be subjected to in a lifetime!
Becky Canning, Hyrum Henderson, and Wendy Sanborn, for helping to develop and test
instruments, for uncomplainingly dealing with grueling travel schedules, and for striving to
collect reliable, high quality data.
Linda Allred, Bryan Elwell, Suzanne Yelton, and Danhui Zhang for reliably and efficiently
grading tests and entering data.
Katie Christiansen Griffiths, for helping to schedule visits to schools and communicate with
teachers.
District superintendents and science coordinators, for agreeing to participate in the study and
giving us access to classrooms, teachers, and students.
Teachers who willingly and enthusiastically participated in the study, for cheerfully allowing us
to observe what happens in middle school science classrooms across the country.
And students, for whom we’re working to improve science education…we hope you learn to
love science as much as we do!

iv
TABLE OF CONTENTS
Executive Summary ......................................................................................................... i
Acknoweldgements .........................................................................................................iii
Introduction ..................................................................................................................... 1
Background ................................................................................................................. 1
Meeting Criteria for Evidence-Based Research Designs............................................. 1
Research Design............................................................................................................. 4
Statistical Power .......................................................................................................... 5
Research Questions........................................................................................................ 6
Description of the Intervention....................................................................................... 10
McDougal Littell Science ........................................................................................... 10
Standards-Based Instruction and Assessment....................................................... 11
Research-Based Practices from Educational and Cognitive Research.................. 11
Scope of Use.......................................................................................................... 15
Cost to Schools and Districts ................................................................................. 15
Materials Provided to Treatment Teachers and Students ...................................... 15
Using McDougal Littell Life Science .......................................................................... 16
McDougal Littell Professional Development........................................................... 17
Teaching and “Full Implementation”....................................................................... 18
Factors Affecting “Full Implementation” and Student Outcomes ............................ 21
Using Other Curriculum in Comparison Classrooms ................................................. 24
Programs Used in Comparison Group Classrooms ............................................... 24
Teaching and “Full Implementation”....................................................................... 24
Description of Population and Sample .......................................................................... 26
Population.................................................................................................................. 26
Sample ...................................................................................................................... 26
Selecting the Sample ............................................................................................. 26
Randomly Assigning Teachers to Groups.............................................................. 28
Final Sample .......................................................................................................... 28
Target Classroom................................................................................................... 29

v
Demographics of the Sample................................................................................... 30
Description of Schools ........................................................................................... 30
Description of Teachers ......................................................................................... 30
Description of Classrooms and Students ............................................................... 30
Instrumentation and Data Collection ............................................................................. 31
Pretest/Posttest ......................................................................................................... 31
Student Attitude Survey............................................................................................. 34
Classroom Observation Instrument ........................................................................... 34
Using Science Materials Checklist Teacher Self-Report Checklist ............................ 36
Using Science Materials Observation Checklist ........................................................ 36
Informal Teacher Interviews ...................................................................................... 37
Informal Student Interviews ....................................................................................... 37
Journaling Questions................................................................................................. 37
Teacher Survey ......................................................................................................... 38
Student Questionnaire............................................................................................... 38
Treatment Teacher Focus Groups/Interviews............................................................ 38
Treatment Student Focus Groups ............................................................................. 38
Data Analysis ................................................................................................................ 39
Final Sample.............................................................................................................. 39
Hierarchical Linear Model Analyses of McDougal Littell Life Science Treatment
Effects...................................................................................................................... 42
Analysis of Research Questions................................................................................ 44
Differences in Achievement between Groups ........................................................ 44
Differences in Attitudes between Groups ............................................................... 45
Causal Effects of Implementation Quality .............................................................. 48
Differences in Treatment Effects on Achievement Attributable to Classroom
Context—Closing the Gaps ................................................................................ 51
Use and Impact of Notetaking................................................................................ 53
Use and Impact of Technology............................................................................... 54
Implementation of the Program.............................................................................. 56
Attitudes toward McDougal Littell Life Science....................................................... 58
Effectiveness of Professional Development ........................................................... 68

vi
Conclusions................................................................................................................ 70
Limitations of the Study................................................................................................. 71
Recommendations for Future Research........................................................................ 73
References.................................................................................................................... 75
Appendices ................................................................................................................... 78
Appendix A: Sample Agenda of Professional Development Provided for Treatment
Teachers....................................................................... Error! Bookmark not defined.
Appendix B: McDougal Littell Sample Script for Seeking Study Participation ......... Error!
Bookmark not defined.
Appendix C: Letters Faxed or Emailed to Interested Schools or Districts ............... Error!
Bookmark not defined.
Appendix D: Parent Letter of introduction and Passive Permission Error! Bookmark not
defined.
Appendix E: Pretest/Postest Forms A and B................... Error! Bookmark not defined.
Appendix F: Student Attitude Survey .............................. Error! Bookmark not defined.
Appendix G: Classroom Observation Instrument ............ Error! Bookmark not defined.
Appendix H: Using Science Materials Observation ......... Error! Bookmark not defined.
Appendix I: Using Science Materials Checklist ............... Error! Bookmark not defined.
Appendix J: Teacher Survey ........................................... Error! Bookmark not defined.
Appendix K: Student Questionnaire ................................ Error! Bookmark not defined.
Appendix L: Teacher Focus Groups................................ Error! Bookmark not defined.
Appendix M: Student Focus Groups ............................... Error! Bookmark not defined.
Appendix N: Journaling Email Questions ...................... Error! Bookmark not defined.
Appendix O: Descriptive Summaries and Statistics ....... Error! Bookmark not defined.
Appendix O.1: Descriptions of Samples and MeasuresError! Bookmark not defined.
Appendix O.2: Responses to Student Questionnaire .. Error! Bookmark not defined.
Appendix O.3: Responses to Teacher Questionnaire.. Error! Bookmark not defined.
Appendix O.4: Using McDougal Littell Science Materials Checklist...Error! Bookmark
not defined.
Appendix O.5: Attrition Analysis .................................. Error! Bookmark not defined.
Appendix O.6: Journaling Email Responses................ Error! Bookmark not defined.

1
INTRODUCTION
The small-scale study described in this report provides research-based evidence for the
effectiveness of the McDougal Littell Life Science curricula series targeted at the 7th
grade level.
The research design of this study meets the research criteria specified by the What Works
Clearinghouse (WWC) in the Study Design and Implementation Assessment Device (Study
DIAD), version 1.0. This report includes a description of McDougal Littell Life Science,
including a description of the program as implemented by teachers, in order to provide sufficient
evidence of the intervention and outcomes to meet WWC standards.
Background
McDougal Littell’s Market Research Team conducted field tests and evaluation studies during
the second semester of 2003 (McDougal Littell Market Research, 2004). Eleven teachers in nine
schools and districts from five states submitted results from implementing chapters from
McDougal Littell Science, including modules from Physical, Earth, and Life Science. The
classrooms represented a mix of urban, suburban, and rural locations as well as a wide range of
socio-economic backgrounds. Pilot testing included pre/post testing of students and results were
positive—students’ posttest scores were considerably higher than pretest scores and teachers’
attitudes towards the materials were favorable.
In order to be able to provide additional feasibility evidence for the purpose of obtaining larger-
scale funding, McDougal Littell’s Market Research Team contracted with EndVision Research
& Evaluation during the 2004-2005 school year to conduct a small-scale feasibility study of the
effectiveness of McDougal Littell Life Science. The randomized experimental design described
in the original proposal specified a minimum of 32 teachers, with 16 randomly assigned to
implement McDougal Littell Life Science curricula and 16 teachers assigned to a comparison
group using previously implemented curriculum. In order to meet McDougal Littell reporting
timelines, the study extended through the entire course of one school term, or approximately 18
weeks during the Fall of 2004. Mixed methods were used to gather data through student
assessment, classroom observations, surveys, interviews, and focus groups.
Meeting Criteria for Evidence-Based Research Designs
Classrooms, schools, and their surrounding environments are complex structures in which
experimental research conditions are difficult to achieve. As shown in the conceptual framework
in Figure 1, many factors affect students, teachers, classrooms, schools, districts, and the
communities in which they reside—most of which are difficult to control in a research study that
takes place in a “natural” or typical setting. For this reason, many contextual factors which could
impact student achievement outcomes were observed and measured through the course of the
study. Student attrition from the study was monitored and handled as necessary in the data
analysis to insure high quality, research-based evidence.

2
Student
Achieve-
ment
Figure 1. Conceptual Framework
The research design met the criteria specified by the What Works Clearinghouse (WWC) in the
Study Design and Implementation Assessment Device (Study DIAD), version 1.0. These criteria
specified that reporting of study findings, including descriptions of the intended curricula and the
curricula implemented by teachers, were required to provide sufficient evidence to meet WWC
standards. This final report for the study includes such descriptions.
Additionally, guidelines from the Institute for Education Sciences (IES, 2003) that recommended
providing evidence of intervention effectiveness using randomized experimental designs were
met in conducting the study and writing this report. The IES guidelines advocated the following
criteria: (a) the intended and implemented interventions were clearly described, including who
administered the intervention, who received it, and what it cost, (b) the ways in which the
intervention differed from what the comparison group received were measured and reported,
(c) the logic for how the intervention was supposed to affect outcomes was included,
(d) compromises to random assignment were articulated and considered in the data analysis,

3
(e) an analysis of baseline differences between intervention and comparison groups was
conducted and any pre-existing differences after random assignment were controlled in the data
analysis, (f) valid outcome measures were used, and (g) all research students were followed and
attrition was monitored, reported, and considered in the data analysis. In addition, (h) effect
sizes and results from statistical tests were reported, (i) the intervention’s effect on all subgroups
of students and outcomes were reported, regardless of direction of effect, (j) the intervention was
implemented in multiple sites with varying demographic locations and characteristics, and
(k) the intervention was carried out and delivered in typical school settings and under typical
conditions. Finally, (l) the study’s intervention and comparison groups were randomly assigned
prospectively (i.e., prior to the intervention), and (m) measures were chosen prospectively.
Overall, the research design described in this final study report dealt with the complexities of
school settings across time while addressing recommendations for providing high quality,
research-based evidence.
This experimental research design constituted a feasibility study to provide evidence for
effectiveness of McDougal Littell Life Science as implemented in the 7th
grade on a small scale.
As a small-scale feasibility study, it falls short of IES guidelines for “strong” evidence on two
points: (a) it did not include data collection on long-term outcomes of the intervention, and (b)
although the IES guidelines suggest outcome data should be reported for those in the intervention
group who did not complete the intervention, most students who dropped out of the study were
absent on test day and teachers were unwilling to sacrifice another instructional day to test them
separately, or they moved to other schools not participating in the study.
As a small-scale feasibility study, researchers followed students for one school term, so data
analysis and reporting could be completed in time for McDougal Littell to use the findings (a) for
the 2005-2006 school year’s sales, (b) to submit a proposal for a large-scale study to
competitions that we anticipated being announced in spring 2005, (c) to present timely evidence
to the What Works Clearinghouse, and (d) to improve other McDougal Littell curriculum under
development.

4
RESEARCH DESIGN
A randomized experimental design was used to answer the research questions through both
quantitative and qualitative methods and measures (e.g., mixed methods, see design matrix in
Table 1 on page 8). Teachers were randomly assigned to implement the “Cells and Heredity”
and “Ecology” units from McDougal Littell Life Science or to a comparison group using
previously implemented curriculum but covering the same units. One of the teachers’ class
periods was selected for the study, and classroom- and student-level data was collected
throughout the course of the study. McDougal Littell provided honorariums that were distributed
to both treatment and comparison teachers upon study completion.
An accredited Institutional Review Board (IRB) reviewed the study proposal and
instrumentation, approving the study under exemption numbers 45 CFR 46.101 (b1 & b2) as the
research was conducted in established or commonly accepted educational settings, involving
normal educational practices, such as the effectiveness of or the comparison among instructional
techniques and curricula. Additional district-level IRB clearance was obtained to the extent
required by some districts for research involving human subjects in public school settings.
Written permission was obtained from district or school administrators for researchers to collect
student, teacher, administrator, school, and district data through focus groups and interviews,
surveys, classroom observations, assessment/testing, and artifacts (e.g., lesson plans, student
products or portfolios, school policies).
We had planned to tell all teachers that they were participating in research to help learn more
about how teachers actually use curriculum materials/textbooks, and to not inform teachers that
researchers were investigating the effectiveness of McDougal Littell Life Science. In this way,
teachers would remain “blind” to the intended purposes of the study. However, district
coordinators who requested that teachers participate in the study told teachers that McDougal
Littell curriculum would be used by teachers assigned to the treatment group. Treatment
teachers did agree to not talk with comparison group teachers about the curricula, specifically
content, pedagogy, classroom activities, assessments, and the effects of the curricula on
themselves and students, and those few comparison group teachers whose classrooms were in the
same building as treatment teachers agreed to not question treatment teachers.
Researchers collected baseline data, including state or district assessment scores for students
assigned to treatment and comparison classrooms, and student, teacher, classroom, and school
demographic characteristics. Additionally, pretests and attitudinal measures were administered
to students. McDougal Littell provided “typical” professional development (as described in a
subsequent section) and materials to treatment teachers, with no costs for curricula
implementation incurred by schools, school districts, or researchers.
The intervention extended through the entire course of one school term, or approximately 18
weeks. All treatment teachers implemented the same curricula and agreed to “fully” implement
McDougal Littell Life Science for one term. Researchers collected data from both treatment and
comparison classrooms throughout the study by conducting classroom observations, focus
groups, interviews, and surveys; reviewing artifacts; and administering pre/post assessments.

5
Statistical Power
Power analysis for two-level Hierarchical Linear Models (HLM), in which students are nested
within classrooms or schools at level 1 and classrooms or schools are at level 2, is a relatively
recent area of methodological study (Hox, 2002; Raudenbush & Liu, 2000; Raudenbush, 1997).
In an HLM framework, statistical power may be affected by several factors. As with any other
statistical analysis, both the magnitude of the treatment effect and the selected probability of
making a type I error – the alpha level – affect power. In addition, in the two-level hierarchical
model, the number of classrooms or schools represented at level 2 of the analysis, the number of
students per classroom or school represented at level 1 of the analysis, and the variation between
classrooms or schools have an impact on statistical power.
As estimates of the magnitude of the level-2 classroom effect of McDougal Littell Life Science,
an effect size of δ = 0.25 was chosen. This effect size, which expresses the expected difference
between the experimental treatment classroom means and the comparison group classroom
means on the proposed outcome, corresponded to widely used standards that experts use to
understand the magnitude of educational effects. Specifically, Cohen (1988) classified effect
sizes of 0.20 as small, 0.50 as moderate in size, and 0.80 as large, and Slavin (1990) stated that
effect sizes at or above 0.25 should be considered to be educationally meaningful. The standard
chosen, therefore, provided a relatively conservative estimate of the expected effect, but was also
of practical educational importance.
In this power analyses, three estimates of the intraclass correlation were used: ρ = 0.05, ρ = 0.10,
and ρ = 0.15. These estimates represented the proportion of variance in achievement scores that
we might expect to find between classrooms. These estimates were derived from a recent study
of teacher effects on upper elementary students’ achievement outcomes by Rowan, Correnti, and
Miller (2002), who found that 3% to 13% of the variability in students’ test score gains was
associated with differences among classrooms across a national sample.
Finally, the initial power analysis summarized in Figure 2 employed two constants: an alpha
level of p < .10 and an estimated within-classroom sample size of 25 students. Given these
estimates, the power analysis for a two-level HLM shown in Figure 2 plots power, 1 - β, where β
represents the probability of failing to reject a false null hypothesis, by the total number
(including treatment and comparison) of sampled classrooms, J, for ρ = 0.05, 0.10, and 0.15.
With a total sample of 32 classrooms, each composed of 25 students, McDougal-Littell effects of
δ = 0.25 would be detected at an alpha level of p < .10 with power of 0.75 assuming an intraclass
correlation, ρ, of 0.05. Power of 0.75 for δ = 0.25 and ρ = .10 would be attained with a total
sample of 50 classrooms. Finally, for ρ = .15, acceptable power of 0.75 would be achieved with
a total sample of 75 classrooms. The results of the power analysis plotted in Figure 2, therefore,
suggested that statistical power would be adequate for detecting classroom-level treatment
effects using a two-level HLM model under a reasonable array of assumptions. Even with a
sample size as small as 32 total classrooms, statistical power should be sufficient to detect the
expected treatment effects for a small-scale feasibility study.

6
Figure 2. Statistical Power Estimates for Classroom-Level Random Assignment
Number of clusters
P
o
w
e
r
22 41 60 79 98
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
α = 0.100
n = 25
δ= 0.25,ρ= 0.05
δ= 0.25,ρ= 0.10
δ= 0.25,ρ= 0.15
RESEARCH QUESTIONS
The research questions that follow address three broad areas, prioritized to match WWC criteria.
An evaluation design matrix that lists indicators and data sources for these research questions
follows in Table 1.
I. The impact of the intervention on student achievement and attitudes:
A. To what extent do students experiencing McDougal Littell Life Science perform better by
(i) demonstrating a higher level of proficiency on curriculum-based assessments and (ii)
exhibiting more positive attitudes than their peers using other curriculum?
B. To what extent are achievement gaps reduced among traditionally underrepresented
groups (e.g., females, students living in poverty, underrepresented minorities)?
C. To what degree does notetaking promote higher assessment scores and increase student
achievement?
D. To what degree does technology use with the curricula promote higher assessment scores,
enhance student engagement, and increase student achievement?
II. The degree and quality of classroom implementation of the curricula:
A. To what extent do teachers follow the program and teachers’ guide?
B. To what degree do teachers encourage students to use Notetaking Guides?

7
C. To what extent are science labs implemented?
D. To what extent is technology used to enhance the curricula and instruction?
E. What are the relationships between teacher characteristics and degree of implementation?
F. What contextual factors promote or hinder successful implementation?
A post-hoc research question was added:
G. To what extent do teachers implementing the McDougal Littell Life Science demonstrate
increased use of research-based teaching practices and instructional strategies?
III. The effectiveness of teacher support materials and professional development:
A. To what extent does professional development follow research-based conceptual models
and cover the content and pedagogy contained in the curricula?
B. To what extent do support materials assist teachers in using the curricula?
C. To what extent do support materials assist teachers in implementing notetaking?
D. To what extent do the technology-based support materials assist teachers in implementing
the curricula?

8
Table 1. Design Matrix: Evaluation of McDougal Littell Life Science
Evaluation Questions Indicators or Data Elements Data Source
IMPACT OF CURRICULA ON STUDENT ACHIEVEMENT
1. To what extent do students
experiencing McDougal
Littell Science perform better
by demonstrating a higher
level of proficiency on
assessments and exhibiting
more positive attitudes than
their peers using other
curriculum?
2. To what extent are
achievement gaps reduced
among traditionally
underrepresented groups
(e.g., females, students living
in poverty, underrepresented
minorities)?
Student scores on standards-aligned
measures are higher than peers
using other curriculum
Students are:
- intellectually engaged with
important ideas relevant to the
concepts being taught;
- encouraged to use higher level
thinking and learning skills;
- engaged in learning activities that
are aligned with state and national
standards;
- confident in their content abilities;
and
- able to see value in the content.
Student performance on
formative, curricula-
based, or researcher-
administered measures
Classroom observations
Teachers interviews and
focus groups
Student focus groups
Other curriculum-related
artifacts (e.g., student
assignments, projects,
portfolios; teacher
lesson plans)
3. To what degree does
notetaking promote higher
assessment scores and
increase student
achievement?
4. To what degree does
technology use with the
curricula promote higher
assessment scores, enhance
student engagement, and
increase student
achievement?
Students are engaged in notetaking and
technology use with the curricula
Students believe notetaking and use of
technology supports their learning,
achievement
Notetaking strategies are:
- implemented as designed;
- integrated into daily
instructional lessons;
- used for intended purposes;
- perceived as useful.
Technology curricula supports are:
- implemented as designed;
- integrated into lessons;
- used for intended purposes;
- perceived as useful.
Student Notetaking
Guides, assignments,
projects, and related
artifacts
Student performance on
curriculum-based, or
researcher-
administered measures
Teacher surveys,
interviews and focus
groups
Student focus groups

9
Evaluation Questions Indicators or Data Elements Data Source
CURRICULA IMPLEMENTATION
5. To what extent do teachers
follow the curricula and
teachers’ guide?
6. To what degree do students
use notetaking?
7. To what extent are science
labs implemented?
8. How is technology used to
enhance instruction?
Frequency of units or lessons
skipped or expanded upon
Deviation from textbook strategies,
use of supplemental materials
Use of Notetaking Guides
Access/availability of computers
Frequency/type of resources used
Teacher surveys,
interviews, focus
groups, journaling
Lesson plans and other
related artifacts (e.g.,
student projects,
assignments)
9. What are the relationships
between teacher
characteristics and degree/
quality of implementation?
10. What contextual factors
affect implementation?
Classroom, school, district, and
community factors that may or
may not be explicitly
articulated by teachers,
students and that effect
curricula implementation
Teacher interviews,
journaling questions
Student focus groups,
class notes
11. (post hoc) To what extent do
teachers implementing the
McDougal Littell Life
Science demonstrate
increased use of research-
based teaching practices and
instructional strategies?
Use of review, vocabulary,
notetaking, questioning, and
assessment strategies
Use of effective teaching cycle:
review, new content, guided
practice, assessment
Teacher surveys, focus
groups, interviews,
journaling
Student focus groups,
class notes
CURRICULA SUPPORT
12. To what extent does
professional development
(PD) follow research-
based conceptual models
and cover the content and
pedagogy contained in the
curricula?
Recommended strategies for adult
teaching/learning are employed
Training materials include content/
pedagogy objectives/activities
Participants are actively engaged in
the curricula, and provided
sufficient learning opportunities
PD lesson plans, training
materials
PD observations
Teacher interviews,
journaling questions
13. To what extent do support
materials assist teachers in
using the curricula?
14. To what extent do support
implementing notetaking?
15. To what extent do
technology-based support
implementing curricula?
Support materials
− promote strategies and concepts
that align with standards;
− support increasing teachers’
pedagogy, content knowledge;
− guide sequencing to align with
standards and assessments;
− provide resources for additional
content or instructional support.
Review of materials
Teacher surveys,
interviews, focus
groups, journaling
questions
Lesson plans and other
related artifacts

10
DESCRIPTION OF THE INTERVENTION
The description of the intervention includes three parts: (a) a description of McDougal Littell
Science, including associated materials and resources, (b) a description of how teachers
implemented McDougal Littell Life Science to teach the “Cells and Heredity” and “Ecology”
units to students in their classrooms, and (c) a description of how the curriculum and teaching in
comparison classrooms differed from treatment classrooms. Figure 3 shows the proposed
mechanism for the impact of the intervention on student outcomes. In this model, the
intervention has an impact on both teaching practices and student outcomes. Teaching practices
also affect student outcomes.
Teaching
Practices
Life Science
Program
Student
Outcomes
Figure 3. Impact of the Intervention on Student Outcomes
McDougal Littell Science
McDougal Littell Science was published in 2004 and constitutes the publisher’s first foray into
publishing science curriculum at the middle school level. The series includes earth, life, and
physical science programs, as well as texts to meet integrated science standards, and individual
books for each unit in earth, life, and physical science. The 7th
grade curriculum used for this
study was “Life Science.” Because of the half-year length of this study, two units were selected:
“Cells and Heredity” and “Ecology.” McDougal Littell Educational Research staff, EndVision
researchers, and advisors to the project agreed that these units included concepts covered under
most states’ 7th
grade standards and would be appropriate for teaching during the first half of the
school year. The other units included in McDougal Littell Life Science are (a) Diversity of
Living Things, (b) Life over Time, and (c) Human Biology.
All treatment and comparison group teachers in the study agreed to cover—during the first half
of the school year—concepts that fall within these titles based on their state standards and the
requirements of the study. Because state standards from a number of the districts involved in the

11
study included integrated science in the 7th
grade, many of the teachers reordered the sequencing
of their instruction for this school year to accommodate our request to cover these topics early.
Additionally, two of the districts did not include ecology objectives in their science standards for
7th
grade, but teachers agreed—with district-level permission—to include an ecology unit. They
devoted less time to the concepts than they might have otherwise, so they could still cover
district-mandated objectives by the end of the school year.
Standards-Based Instruction and Assessment
Because the No Child Left Behind Act of 2001 requires all states to establish statewide
accountability systems based on challenging state standards in reading, mathematics, and science
for grades 3-8, the program content, activities, and assessments of McDougal Littell Science
were aligned to National Science Education Standards (NRC, 1996).
Research-Based Practices from Educational and Cognitive Research
A number of research-based practices were incorporated into McDougal Littell Science. First,
nine strategies from Classroom Instruction that Works: Research-Based Strategies for
Increasing Student Achievement (Marzano, Pickering, & Pollock, 2001) were incorporated into
the student and teacher texts, chapter resource books, and other ancillary materials. The
strategies follow.
• Identifying similarities and differences: This strategy includes comparing and classifying,
and suggests representing comparisons in graphic or symbolic form, e.g., contrasting units of
measure and types of classification systems, making concept maps and Venn diagrams.
• Summarizing and notetaking: Students learn a variety of note taking formats, e.g., outlines
and webbing, and learn when to delete, substitute, or keep information when writing a
summary. Examples include specific features contained in the texts (e.g., Know How to
Take Notes, Help with Notetaking) and the entire student Notetaking Guide, which is an
ancillary resource available at low cost (i.e., about $10 per student).
• Reinforcing effort and providing recognition: To help students make the connection
between effort and achievement, this strategy encourages teachers to provide recognition
to students for attaining specific goals.
• Homework and practice: Research shows that it is important that students understand the
purpose of their assignments. This strategy suggests providing homework assignments that
focus on specific elements of a complex skill to help make the purpose of the assignments
clear to students.
• Nonlinguistic representations: To help students understand content in a new way, this
strategy focuses on creating nonlinguistic representations. These include creating graphic
organizers, making physical models, generating mental pictures, drawing pictures and
pictographs, and engaging in kinesthetic activity.
• Cooperative learning: A description of the five defining elements of cooperative
learning—positive interdependence, face-to-face interaction, individual and group
accountability, interpersonal and group skills, and group processing—are described in
this strategy and suggestions are given for grouping techniques.

12
• Setting objectives and providing feedback: This strategy recommends that teachers use
instructional goals to narrow their students’ focus and provide criterion-based feedback. It
also encourages students to personalize their teacher’s goals and provide some of their own
feedback. Examples include Before/Now/Why lists in student lessons and scoring rubrics on
Building Test-Taking Skills.
• Generating and testing hypotheses: The variety of structured tasks included in this
strategy guides students through generating and testing hypotheses and using induction or
deduction. This strategy also advises asking students to clearly explain their hypotheses and
conclusions to deepen their understanding. Examples include Hands-On Activities and
Predict exercises throughout the book.
• Cues, questions, and advance organizers: This strategy includes asking questions or
giving explicit cues before a learning experience to provide students with a preview of what
they are about to experience. It suggests using verbal and graphic advance organizers, or
having students skim information before reading as an advanced organizer. Examples
include Before/Why/Now lists at the beginning of lessons, pre-reading lesson elements such
as Word Watch lists and Example heads throughout the book.
Another set of research-based instructional strategies that are incorporated into McDougal Littell
Science are those presented in Effective Teaching Strategies that Accommodate Diverse Learners
(Kameenui & Carnine, 1997). The features are summarized below.
• A focus on big ideas: Kameenui and Carnine describe big ideas as “those concepts,
principles and heuristics that facilitate the most efficient and broadest acquisition of
knowledge. They are the keys that unlock a content area for a broad range of diverse
learners.” Grossen and Burke (1998) wrote, “Big ideas in science do four things. First, they
represent central scientific ideas and organizing principles. Second, they have rich
explanatory and predictive power. Third, they motivate the formulation of significant
questions, and fourth, they are applicable to many situations and contexts common to
everyday experiences." The McDougal Littell Science provides a thorough development of
big ideas, or key concepts.
• Conspicuous strategies: Explicit teaching of problem solving strategies that expert
problem solvers find useful can be helpful to all students provided the strategies can be
applied to a wide range of situations. Examples include Problem Solving Strategies pages;
Building and Practicing Test-Taking Skills; and worked-out examples with step-by-step
annotated reasons.
• Mediated scaffolding: Providing extra support to students when they are first learning new
ideas can give them the skills and the confidence to succeed on their own. Examples include
Getting Ready to Learn; Getting Ready to Practice exercises with Vocabulary and Guided
Problem Solving questions; Help with Homework boxes; Practice and Problem Solving
exercises that move gradually from basic to challenging.
• Strategic integration: Kameenui and Carnine describe strategic integration as “the
combining of essential information in ways that result in new and more complex
knowledge.” Integration may be across or within disciplines. For example, Exploring Math
in Science; real world problem solving in lessons/exercises; What do you think? questions
relating to Math, Social Studies, or Art; topic integration and integration of representations.

13
• Primed background knowledge: In order to learn new information easily, students need to
be familiar with key background information (prerequisite concepts and skills). McDougal
Littell Science has a carefully sequenced curriculum in which prerequisite knowledge is
presented before students need it. In addition, the textbooks offer several ways to check
understanding of prerequisite skills and provide help if needed. For example, Pre-Course
Test and Practice; Brain Games, Getting Ready to Learn, Review What You Need to Know
questions, Basic Skills questions in Mixed Review Exercises; Help with Review Notes and
Skills Review Handbook.
• Judicious review: Carefully planned and paced review of important ideas can increase
students’ retention of concepts and facility in applying skills. Examples of judicious review
can be found in the Notebook Review, Mixed Review, and Chapter Review; as well as
teaching and learning support materials.
Because research shows that ongoing, embedded assessment provides feedback to help teachers
plan instruction, McDougal Littell Science provides materials for diagnosing how well students
understand the material, for differentiating instruction to reach all students, for assessing student
progress, and for providing remediation. The curriculum also emphasizes important test-taking
skills and problem-solving strategies for students based on the Creating Independence through
Student-owned Strategies, or CRISS (Santa, Havens, & Maycumber, 1996), model for student-
centered teaching. The following strategies and materials are emphasized.
• Ongoing diagnosis: Materials to diagnose student understanding are provided before,
during, and following each chapter and lesson. Tools include pre-course tests, chapter warm-
up games and Getting Ready to Learn exercises, Skill Check exercises, Your Turn Now and
Getting Ready to Practice to help monitor how well students are grasping the vocabulary,
skills and concepts. Additional assessment resources include Homework Check boxes, and
the Test and Practice Generator CD-ROM.
• Differentiating instruction and practice: There are a number of components identified in
the materials that help differentiate instruction and practice to reach all students. For
example, labs and worksheets are provided at three levels: Level A for struggling learners,
Level B for students who perform adequately, and Level C for advanced students.
• Building test-taking skills: It is more important than ever for students to build strong test-
taking skills in order to be successful on annual assessments required by the NCLB Act.
McDougal Littell Science provides instruction and practice with test-taking skills at the end
of every unit in the textbook. Included are (1) multiple-choice questions where students are
encouraged to use cognitive skills to decide whether answer choices are reasonable, (2) short-
response questions where guidance is provided about how to write complete answers and
show work, (3) context-based multiple choice questions that involve interpreting diagrams
and graphs, and (4) extended-response questions where students learn how to write complete
answers to multi-step problems.
• Assessment: The series provides diagnostic, formative, and summative assessment resources
for measuring student progress on an ongoing basis. These include test-practice questions at
the end of every exercise set, quizzes, reviews, traditional and standardized chapter tests, an
end-of-course test, quizzes for every lesson, alternative forms for quizzes and tests, a Test

14
and Practice Generator CD-ROM to create customized quizzes and tests, and Online Quizzes
and Standardized Test Practice.
• Reteaching and remediation: A variety of resources help students achieve success, such as
Help with Review notes, Notebook Reviews after every few lessons that summarize key
vocabulary and skills and provide practice, Chapter Reviews, Cumulative Practice, Skills
Review Handbook, Extra Examples, Common Error notes, and additional teacher resources.
• Problem solving strategies: Questions on state and national tests are often posed as word
problems. In order for students to demonstrate mastery of science skills, they must be able to
read and interpret word problems and apply appropriate strategies to solve them. McDougal
Littell Science incorporates problem solving throughout the textbook to help students learn to
apply skills in context.
Finally, students need strong skills in reading, writing, and notetaking in science in order to
understand course content, be successful on important state and national assessments, and
develop the ability to become independent learners. Recent brain research and classroom
research in reading and writing supports the value of well-known practices of successful
teachers, particularly practices supporting vocabulary development and reading comprehension.
Based on this research, McDougal Littell Science incorporates the following features.
• Vocabulary development: The textbook provides strong support to students in learning,
practicing, and reviewing vocabulary. The Getting Ready to Learn page at the start of each
chapter lists important review words practiced in the Using Vocabulary exercises. At the
beginning of each lesson, the key vocabulary for the lesson appears under the Word Watch
list, and new vocabulary in the lesson is emphasized by boldface type with yellow
highlighting. Other vocabulary building aids include Help Notes, Getting Ready to Practice,
Notebook Review pages, Chapter Reviews, and the complete Glossary that includes
examples and diagrams.
• Reading comprehension: Students are given tips for identifying the main idea,
understanding the vocabulary, knowing what’s important in a lesson, being an active reader,
and reading word problems. Other strategies include (1) establishing a context by connecting
new learning to prior knowledge and starting each lesson with a real-world example, a short
activity, or a visual presentation of a math idea to set the stage for new concepts in the lesson,
(2) facilitating understanding by presenting new concepts in short sentences that use simple
syntax and that are accompanied by appropriate tables, charts, and diagrams, (3) reflecting on
learning (i.e., metacognition) by encouraging students to consider whether an answer is
reasonable or to explain their reasoning, (4) using graphic organizers such as charts, Venn
diagrams, or concept maps to help classify mathematical objects.
• Writing opportunities: Students need frequent opportunities to practice writing skills.
These opportunities occur throughout the textbook in Exercises, Stop and Think questions,
activities, Notebook Review, and Exploring Math in Science sections.
• Effective notetaking: Taking effective notes is an important comprehension, learning, and
review strategy that increases engagement. Yet, teachers throughout the country report that
students enter middle school with few if any notetaking skills. Thus, the authors identified
the goal of helping students develop their notetaking skills as an important objective of the

15
program. They incorporated many notetaking aids into the program, such as Getting Ready
to Learn pages and Notebook Reviews in the textbooks, and the separate Notetaking Guides.
Scope of Use
McDougal Littell Science reached distribution channels in time for 2004-2005 school year sales.
Recently, Houghton Mifflin, McDougal Littell’s parent company, released a news
announcement, “North Carolina’s Top School Districts Choose State-of-the-Art McDougal
Littell Science for Middle School,” in which they claim, “In total, McDougal Littell Science will
reach about 100,000 North Carolina students next year” (April 25, 2005, available at
http://www.hmco.com/company/newsroom/news/news_release_042505.html). Widespread sales
are anticipated for the 2005-2006 school year.
Cost to Schools and Districts
Costs to schools and districts vary based on the base components and ancillaries purchased. For
this study, the components listed in the following section were provided for each student in the
treatment group, for a total cost per treatment student of under $100. Costs for the basic package
provided to each teacher in the treatment group were approximately $370, while the ancillary
package also given to each teacher included materials that would typically cost approximately
$1400 per teacher. For this study, teachers were also given student textbooks for each student in
their other Life Science classes, at a cost of approximately $57 per student.
Materials Provided to Treatment Teachers and Students
The following materials were provided to the teachers in the treatment group. Although most
materials were given to teachers at professional development sessions, not all materials were
mailed in time for the professional development session or prior to the start of the school year.
The delays in teachers receiving materials were due to districts agreeing just prior to the start of
the school year to participate in the study. Coordinating distribution of materials, professional
development scheduling, and the schedules of teachers for whom the contract year had already
started was difficult. For example, in one district that agreed to participate in the study just
before the start of the school year, teachers participated in the McDougal Littell professional
development after students had returned to school, and they received the Science Toolkit
approximately one month after the start of the school year.
The following Life Science materials were included in the basic package for treatment teachers:
• Teacher Edition (TE) Life Science Single Volume Edition
• Test Generator CD-ROM Kit (Life, Earth, and Physical Science)
• Lab Generator CD-ROM
The following Life Science ancillary materials were provided to treatment teachers:
• Cells and Heredity Unit Assessment Book
• Cells and Heredity Unit Resource Book
• Cells and Heredity Unit Transparency Book
• Diversity of Living Things Unit Assessment Book

16
• Diversity of Living Things Unit Resource Book
• Diversity of Living Things Unit Transparency Book
• Ecology Unit Assessment Book
• Ecology Unit Resource Book
• Ecology Unit Transparency Book
• Human Biology Unit Assessment Book
• Human Biology Unit Resource Book
• Human Biology Unit Transparency Book
• Life Over Time Unit Assessment Book
• Life Over Time Unit Resource Book
• Life Over Time Unit Transparency Book
• Problem Solving and Critical Thinking TE Grade 7
• Process and Lab Skills TE Grade 7
• Program Overview
• Content Review CD-ROM (Life, Earth, and Physical Science)
• EasyPlanner CD-ROM (Life, Earth, and Physical Science)
• e-Edition CD-ROM (Life, Earth, and Physical Science)
• English Learners Package
• McDougal Littell Science Issue Flyer-Big Ideas in Science
• McDougal Littell Science Issue Flyer-Differentiated Instruction
• McDougal Littell Science Issue Flyer-Reading Support
• Notetaking Guide for Life Science
• PowerPoint Presentations CD-ROM (Life, Earth, and Physical Science)
• Science Toolkit
• Scientific American Frontiers DVD Kit
• City Science (provided to teachers who requested it)
• Multi-language Glossary (provided to some teachers)
The following Life Science materials were provided for students in the treatment group.
Additionally, a Pupil Edition Life Science Textbook was provided for all students in each of the
treatment teachers’ 7th
grade science classes that were not included in the research.
• Pupil Edition Life Science Textbook
• Notetaking Guide for Life Science
• Standardized Test Practice Pupil Edition
• Problem Solving and Critical Thinking Pupil Edition
• Process and Lab Skills Pupil Edition
Using McDougal Littell Life Science
Teachers in the treatment group were asked to “fully implement” McDougal Littell Life Science.
Although full implementation was not operationally defined, we asked the teachers to use as
many of the instructional strategies, resources, and ancillary materials as possible. Because some
of the teachers said during the professional development sessions that they would not participate
in the study if they could not use some of their “tried and tested” activities, we did not require

17
that they eliminate all extra activities. For example, some had laboratory activities they would
not give up while others had supplementary activities they had always used, such as projects
jointly completed with teachers of other subjects. One of these was a project involving English,
history, and science classrooms, where students created exhibits about famous scientists over a
two week period and jointly participated in a history fair.
All treatment teachers taught other units prior to the units required for this study, including units
of measure, measurement, laboratory use, and others. All teachers started “Cells and Heredity”
and “Ecology” units during the first half of the school year, as requested for the study, although
some teachers were unable to complete one of the units until after the start of the second term
due to interruptions to school schedules from fires and hurricanes. For some teachers, teaching
these units meant restructuring their planned sequencing for the school year, or obtaining district
permission to not use district sequencing and pacing guides. In one district, the resequencing
resulted in teachers in both treatment and comparison groups not having to administer district-
level criterion-referenced tests (CRTs) to students in the classrooms participating in the study—
this was a huge motivator for teachers in that district to participate in the study!
McDougal Littell Professional Development
McDougal Littell provides complementary professional development to districts and schools
who purchase more than some minimal value of Life Science (C. A. Guziak, personal
communication, January 5, 2004). The training covers use of the materials in the series,
including the Notetaking Guide and other ancillary print materials, and use of computer and
online support resources by teachers and students. Professional development sessions typically
range from 3 hours to 2 days in length, depending on district requirements and availability of
time. Not all teachers in districts that purchase the program attend publisher-provided
professional development.
For this study, we agreed to plan one and a half days for professional development to train
treatment teachers how to use the program. However, not all districts were able to allot this
amount of time given teaching contracts and the amount of time available prior to students
returning to school, so some of the sessions were shorter than the planned time. Shortened
sessions typically did not include as much time for teachers to work with each other to develop a
lesson or to use the technology components. One trainer from McDougal Littell trained all
treatment teachers near their home location on the dates shown in Table 2. Teachers were
provided with a $100 honorarium for attending the in-service, although most did not know prior
to the in-service that they would be given an honorarium.
Because of the late date at which most districts agreed to participate and provided names of
teachers, the in-service dates were scheduled just before or just after students returned to school.
This meant that teachers did not have long to prepare lesson plans that used the materials prior to
teaching. EndVision researchers attended all in-service sessions except one in which training
was provided for a single teacher. A sample agenda for the in-service is included in Appendix
A, although the training varied somewhat in length, content, and sequencing between sites.

18
Table 2. Dates in 2004 for Professional Development and Student Return to School
District Dates of Professional
Development Sessions
Times Students Return
to School
School district in a southern state Thursday, August 5
Friday, August 6
8:00-4:30
8:30-noon
Monday, August 9
School district in a southern state Friday, August 13 8:00-5:00 Monday, August 16
School district in a western state Monday, August 16
Tuesday, August 17
8:00-4:30
8:30-noon
Wednesday, August 25
Two school districts in a northeast
state
Monday, August 30
Tuesday, August 31
8:00-4:30
8:30-noon
Tuesday, September 7 or
Wednesday, September 8
School district in a western state Thursday, September 30 9:00-3:00 Monday, August 30
The trainer included a variety of “hands-on” activities during the professional development
sessions. Some of these included the following:
• lab-type activities to demonstrate problem-solving processes or processes for teaching
inquiry-based investigations (some of these were not activities included in the McDougal
Littell Life Science materials),
• planning a sample lesson that included use of ancillary resources,
• exploring ClassZone.com and SciLinks websites,
• editing test questions within the Test Generator software, and
• using the EZ-Planner CD to edit lesson plans.
Additionally, the trainer discussed and demonstrated components of the material such as the “Big
Ideas and Key Concepts,” vocabulary-building strategies (e.g., decoding strategies, notetaking
strategies for learning vocabulary, multi-language glossary), notetaking in outline format using
text headings and subheadings, and multiple forms of assessment of student learning (e.g.,
questions embedded in text, section and chapter review questions, 3-minute warm-ups, chapter
and unit tests, standardized test practice).
Teaching and “Full Implementation”
To learn how to implement McDougal Littell Life Science, the 15 treatment teachers for this
study attended one or two day in-service sessions conducted by a McDougal Littell Science
trainer. The trainer introduced an extensive selection of 7th
grade life science instructional
resources. This broad array of materials included but was not limited to: (a) standard textbook
editions for teacher and students; (b) supplemental materials such as transparencies, Notetaking
Guides, reading study guides, and assessments; and (c) technology such as DVDs, videos,
PowerPoint presentations, and Internet websites.
During the on-site data collection activities, researchers noted that treatment teachers varied in
their level of implementing the McDougal Littell Life Science materials. Thus, for purposes of
this study, treatment teachers were categorized in one of three groups: (a) “full” implementers,

19
(b) moderate or partial implementers, or (c) low implementers. “Full implementers” were
teachers who regularly employed a wide range of McDougal Littell Life Science resources or
resources from the primary science program, including materials and instructional strategies. In
contrast, “low implementers” were treatment teachers who seldom used or used only limited
McDougal Littell Life Science materials and strategies or comparison teachers who did not use a
single published program consistently. Moderate or partial implementers may have used a single
published program, but they also regularly included teacher-developed or other materials and
activities. Partial implementers may have used McDougal Littell Life Science materials but did
not consistently use the embedded instructional strategies. General descriptions of the two
anchor points for this continuum of implementation, full and low implementers, follow.
Full Implementers. A number of factors were included in the rating of “full implementer.” The
following factors and descriptions help depict the qualities of those teachers.
Environmental Factors. Environmental factors refer to the physical arrangement of
classroom, equipment, and materials. Data collectors recorded the following classroom
characteristics for instructors who fully implemented Life Science resources:
(a) Student seating was arranged to encourage individual learning and cooperative
grouping. All students were able to see instruction.
(b) Teachers displayed science posters, student projects and various learning resources—
produced by the publisher of their primary Life Science program, by the teacher, or
by students—in a purposeful and orderly manner. In treatment classrooms, this
included display of “Big Ideas,” “Key Concepts,” and science vocabulary.
Materials and Instructional Strategies. Full implementers regularly followed the lesson
format identified in the teacher’s edition. Consequently, treatment teachers routinely applied
effective teaching practices embedded in the McDougal Littell Life Science, namely: review of
previous content, presentation of new content tied to prior knowledge, guided practice with
scaffolding to promote learning of increasingly complex concepts, and independent practice.
Within this framework of effective instruction, teachers who fully implemented frequently relied
on the following:
(a) Teacher and student edition textbooks
(b) Clearly stated objectives and set expectations for classroom routines and behavior
(c) Start of class warm-ups and other forms of review
(d) Transparencies for notetaking and Notetaking Guides
(e) Strategies for learning and practicing science vocabulary
(f) Graphic organizers to visually organize concepts and their properties
(g) Connections to prior knowledge and experiences
(h) Questioning strategies to check for student understanding
(i) Frequent feedback and reinforcement of appropriate student responses
(j) Organized sequencing of concepts using PowerPoint presentations
(k) Internet websites and other software and aids to visually depict concepts
(l) Science labs and other hands-on activities to develop concept understanding
(m) Frequent assessment of student learning, including homework assignments

20
Researchers further confirmed the frequent application of these resources and instructional
strategies through data collected in student focus groups.
Overall Tone of Classroom. Full implementers often established an overall classroom
tone conducive to student learning. Perhaps this tone can best be described as a teacher’s clear
expectation to students that they need to “be about the business of learning.” More specifically,
teachers who fully implemented demonstrated:
(a) Frequent opportunity for student response, practice, and feedback
(b) High rate of student engagement to maximize opportunities for learning
(c) Low frequency of off-task behavior with appropriate intervention to maintain
engagement
(d) High rate of positive reinforcement
(e) Differentiated instruction to meet the needs of all learners
(f) Materials and resources for English Language Learners (ELL)
Low Implementers. Low implementers used materials and instructional strategies poorly, or
included materials and strategies from other publishers or sources.
Environmental Factors. Environmental factors refer to the physical arrangement of
classroom equipment or materials. Data collectors recorded the following typical classroom
characteristics for instructors who were low implementers:
(a) Student seating divided in rows emphasizing individual work
(b) Classrooms appeared less welcoming, disorganized, or void of displays of student
work
(c) Little evidence of materials included with published resources was seen on walls,
desks, or in bookcases
Materials. Low implementers typically followed a less structured and more varied
routine or presentation than teachers who fully implemented a Life Science program.
Consequently, low implementers routinely presented new content without opportunities for
student responses or had students work independently while failing to regularly assess students’
skills through review of previous content or guided practice. Within this framework of less
effective instructional strategies, teachers who were low implementers more often relied on the
following limited resources:
(a) Teacher and student edition textbooks
(b) Reading study guides
(c) Chapter tests
(d) Teacher-generated activities or worksheets, or non-McDougal Littell materials
and assessments
Researchers further confirmed the infrequent use by treatment teachers of additional McDougal
Littell Life Science resources through data collected in student focus groups.

21
Overall Tone of Class. Low implementers often established an overall classroom tone
less conducive to student learning. More instructional time was spent managing off task
behaviors. Classrooms where teachers only partially or minimally implemented the Life Science
curriculum often demonstrated:
(a) Lower rates of positive reinforcement
(b) Infrequent opportunity for student response and feedback
(c) Lower rates of engagement for some or many students
(d) Increased frequency of off task behavior, including talking during instruction, out-of-
seat disruptions, physical interactions with other students, and other behavior that
disrupted other students and decreased learning opportunities
In summary, instructors who fully implemented McDougal Littell Life Science had similar class
sizes and available instructional time as teachers who were low implementers. However, full
implementers more often demonstrated (a) greater attention to environmental factors that
influenced instruction, including access to publisher’s materials, (b) a wider range of Life
Science resources and instructional strategies that were embedded in the program implemented
within the context of more consistent adherence to effective teaching strategies, (c) increased
student engagement and opportunities for student learning, and (d) an overall positive classroom
tone where student effort was recognized and that was more conducive to student learning.
Factors Affecting “Full Implementation” and Student Outcomes
As in any school setting, many contextual factors, including characteristics of schools, teachers,
and students, may affect implementation of any program, and the impact of the program on
student outcomes. In this study, some of these contextual factors that may have influenced
implementation or student outcomes included the following, non-exhaustive lists.
District or school characteristics. School environments included lock-down facilities and
security guards, access to and support of technology resources, Title I funds and resources,
socioeconomic status, proportions of minority students, number of students learning English,
number of students with disabilities and level of inclusion, school dress code or uniforms,
number of students per classroom, classroom aides, state or district standards, state or district
testing, and even the school’s response to weather and other external events.
Teacher characteristics. Teacher characteristics that could affect study outcomes included
voluntary or district-stipulated participation in the study, years of teaching experience and
academic preparation, preparation for managing instructional time and student behaviors, level
of interaction with students during instruction, expectations for students, attitudes towards
students, use of a positive or punitive reinforcement system, extracurricular demands, and
comfort with observers visiting the classroom.
Student characteristics. Contextual factors related to student characteristics included number
of days attending school (or absent from school), distractibility (potentially due to factors
external to school), classroom seating assignments, time of day, gender, socioeconomic or
minority status, disabilities, ability to read and communicate using English, interest and ability in
content area (e.g., science), and many others.

22
The following descriptions submitted by teachers, with identifying information removed,
describe the many uncontrollable factors that affect students’ opportunities to learn in school and
classroom settings. The first was submitted by a treatment teacher who participated in this
science study, and the second was sent to us by a treatment teacher participating in a math study
(conducted concurrently with the science study) whose comments reflected those we heard
frequently from teachers participating in the science study.
Study disruptions caused by factors mostly external to school settings.
Well, this year has been quite interesting! First of all, our Middle School was
struck by lightning within the first few days of school. A fire erupted which
engulfed our library and sent soot and smoke throughout our school. At that point,
our school was placed on double sessions with another Middle school.
What does that mean? It simply means that our students and faculty share the
same school building but at different times of the day. Instead of attending school
from 7:20 a.m.- 2:35 p.m.(Teachers attend for 7:00 a.m.-2:50 p.m.), our students
arrive for school at 12:20 p.m. - 4:30 p.m.(Teachers from 9:30 a.m.- 4:50 p.m.)
This shortened schedule has taken away important teaching time.
Also, the students have been home all morning and I find it very difficult to get
them to focus on their schoolwork. By the late afternoon, they are tired and not
exactly working to their potential. The school is still required to serve a bag lunch
during our first period class. The students receive P.E. three times a week. As you
can see, the biggest problem facing our students is the shortened amount of time
for classroom instruction.
The fire presented problems which our school had never faced before. (Example:
changes in bus schedules for bringing students to school at the later time,
preparing lunch for almost 3,000 students including both schools, after school
sporting events and dance lessons, etc).
Once we became accustomed to our new schedule changes, the threat of
Hurricane Ivan the Terrible threw us into a tailspin. We missed eight days of
school which must be made up during Thanksgiving, Christmas, New Year's, and
other scheduled holidays.
So, as you can see, this year has been very interesting! Someone once said that the
most important quality for a teacher to possess is "Flexibility"! I would have to
agree!
Study disruptions caused by factors mostly internal to school settings.
Since [the district agreed to participate in the study after students had returned to
school and] we received the textbooks [for the study] after the beginning of

23
school, … we had to learn a new series, correlate this to our standards and pacing
guide, change out books and get to work.
This coincided with the three days of assemblies related to fund raising, etc. and
one day of Fall Break. November brought a student holiday for Election Day, and
2 1/2 days vacation for Thanksgiving. December delivered assemblies (thus not
having classes) for Choral Concerts and Band Concerts. We were out from the
21st through January 4. Oh, and did I mention in-school ball games????
Second semester starts. Ahhh, January. January began with a 3-day week
followed by a 5-day week (with the assembly for the Spelling Bee), followed by a
4-day week (Martin Luther King Holiday), followed by a 5-day week (with a
Character Counts Assembly day and a Tag all day field trip.) The Flu Bug is
surfacing....
February brings 2 days out of school the first week for a Flu Epidemic. This is
followed by a 4-day week and then ANOTHER 4-day week, ANOTHER 4-day
week (Inservice day), and, yes, ANOTHER 4-day week (President's
Day/Inservice). Also, thrown in (on a day we were actually in class) was a poll on
drug use/knowledge that took 2 class periods.
March is off to a good beginning: a 5-day school week! However, this is
immediately followed by a 4-day week (staff development day). The 3rd
week is a
5-day week, followed by the week of Spring Break. The last week in March is a 4
day week, for the students have 6 school days off...teachers report on Monday the
28th
for Staff Development.
The 1st week in April (4-8) is a 5 day week, followed by a 4-day week. The 15th
is another in-school Administrative Day. We are now into the 3rd
week in April.
Monday, the 18th
, is another Student Holiday (Teacher Inservice Day). Beginning
on Tuesday the 19th
through Wednesday the 27th
, the calendar is blocked off for
State Testing. I do not know the exact schedule for this. Testing is not all day, and
some days are make-ups...
May. School is out the 20th
(Friday), with students returning on Monday to pick
up their report cards. Those 3 weeks we are here are filled with May Days,
assemblies, concerts, team assemblies, school-wide assemblies, AR Reading
reward day, etc., etc., etc...
As you can see, it is hard to get much traction!! We get started, just to be off a day
or two and have to review when we return. I have not included the 4 (FOUR)
pull-out programs that are constantly removing students from the classroom (but
NOT special areas!) to attend a variety of reform or enrichment programs. Add to
this the art and music field trips not previously mentioned, sickness, court dates,
death in families, etc. and it seems like our academic calendar shrinks each year.

24
Yet programs get more difficult, and the public expects higher test scores each
year. And EVERYONE expects ALL schools to be above average!!! (We really
need to focus on statistics a bit more!)
I hope this sufficiently addresses [the question about whether any disruptions may
have affected the study]...I know there are more things that occurred (the intercom
every few minutes comes to mind), but once you see the layout on the calendar, I
think you begin to get the Big Picture.
Using Other Curriculum in Comparison Classrooms
Comparison group teachers were asked to teach using whatever curriculum they had used
previously—none of the schools involved had adopted a new science program to be implemented
this school year, so all comparison group teachers, with the exception of one first-year teacher,
had taught using the same curriculum at least one year prior to this study.
Programs Used in Comparison Group Classrooms
Teachers in the 15 comparison classrooms reported that they used the following Life Science
programs. None of these teachers reported that they had attended professional development for
using these programs in the past three years.
Publisher Name of Text Publication Year Count
Addison-Wesley Science Insights-Exploring Living Things 1994 1
Holt, Rinehart, and
Winston
Holt Science & Technology
2001
2002
2003
1
3
3
Prentice Hall Science Explorer: Life, Earth, and Physical Science
Science Explorer: Life, Earth, and Physical Science
2001
2002
4
1
Not Reported 2
Teaching and “Full Implementation”
Teachers in the comparison group also ranged widely in the degree of implementation of the
materials and resources they used. Overall, they more frequently used materials from a variety
of sources and less often followed the sequencing in their primary texts. However, a few of the
teachers in the comparison group did more closely follow the publisher’s materials and lesson
plans, and incorporated instructional strategies similar to those embedded in McDougal Littell
Life Science.
Contextual factors that affected teachers in the treatment group also affected teachers in the
comparison group—with one exception. Teachers in the comparison group did not have to
change what they would have typically done in their classrooms, other than let the researchers
visit to observe. They were not required to develop new lesson plans, use new materials, or add

25
new materials to their classroom environment. The amount of stress experienced by the
comparison teachers as a result of participating in this study was substantially less than that
experienced by teachers randomly assigned to the treatment group. Treatment group teachers
were required to use a new Life Science program, which involved new lesson plans and
activities, use of technology resources, and for those who implemented fully, potential changes
in teaching practices to implement the instructional strategies embedded in McDougal Littell Life
Science. For comparison teachers, the year started with “business as usual,” while treatment
teachers whose districts agreed to participate in the study near the start of school scrambled to
learn how to implement a new Life Science program.

26
DESCRIPTION OF POPULATION AND SAMPLE
Because this study was a feasibility study conducted to provide evidence for product
effectiveness with the hope of obtaining Federal funding for a large-scale study, the sample size
was determined using a power analysis with slightly relaxed parameters. The power analysis is
further described in the analysis section of this report.
Population
The population of science teachers from which the sample could have been selected included
teachers in all schools or districts which (a) had not already purchased the McDougal Littell Life
Science, (b) were not actively discussing purchase of McDougal Littell Life Science with sales
representatives, and (c) had student populations that were reasonably close to national averages
for socio-economic status (SES) and ethnicity percentages.
Sample
The original proposal for this study specified participation by a minimum of 32 teachers, with
one of their 7th
grade science periods selected as the target classroom. Because we wanted to
reduce the potential for first year teachers, we told district science coordinators that we were
searching for 40-50 teachers who were interested in participating. We asked them to return a list
of 4-8 7th
grade science teachers from each middle school that had district and/or school
permission to participate in the study. Ideally, we planned to select only one teacher per school,
with no more than 3 teachers selected from any one large school if very large schools or fewer
districts agreed to participate in the study.
Selecting the Sample
Personnel from McDougal Littell’s Educational Research staff made the first contact with
districts, typically contacting the district science coordinator or curriculum director. A sample of
the script followed during these calls is included in Appendix B. When the response to a request
to participate in the study was favorable, letters that briefly explained the study and solicited
permission to participate were emailed or faxed to the district contact (see Appendix C), and the
contact name was submitted to EndVision’s Project Coordinator. The Project Coordinator
continued making contact and discussing the study with district representatives until the district
either (a) declined participation either directly or by not returning the promised signed letter of
participation, or (b) agreed to participate and returned the signed letter of permission that
included names and contact information for teachers agreeing to participate in the study.
EndVision started calling districts on May 11, 2004. Because teachers for three studies were
recruited simultaneously (e.g., McDougal Littell Life Science, Middle School Math and Algebra I
programs), districts were asked if they were willing to participate in one or more of the studies.
As of September 3, 2004, when the final district agreed to participate in the science study and
returned names of teachers, EndVision’s Project Coordinator had made over 1400 contacts with
93 districts to seek study participants. McDougal Littell had called many other districts that
expressed no initial interest.

27
Reasons cited by those declining districts that provided an explanation to EndVision staff are
included in the table below. Additionally, a few districts required extensive paperwork to be
filed, school board or district administrator approval, and/or district-level IRB approval.
Although a number of districts that required this more extensive approval route sought approval
early on, few school boards or district administrators approved participation. Given the time
constraints as the school year approached, we sought districts where decisions to participate
could be made more quickly.
Table 3. Reasons Given for Not Participating in the Study
Reason for not participating Frequency
Conflict with competing science programs, curriculum standards, or other ongoing research 15
District personnel initially expressed interest but time ran out to continue recruitment
activities
10
District personnel stated that there was insufficient time to prepare for the study 7
People were simply too busy 6
The decision maker was no longer available, (e.g., summer leave, change in personnel) 3
District agreed to participate but teachers declined 1
We had set 32 teachers as the absolute minimum for conducting the study in anticipation of
attrition. By the start of the school year, given varying start dates across districts, six school
districts had agreed to participate and had provided contact information for 37 teachers. One
School District provided 14 names, but we decided that 14 teachers from one district were too
many to include in the study. We chose 10 of the teachers, eliminating schools with student
populations from the highest ranked socioeconomic status. In all, 34 teachers in six school
districts representing 20 schools agreed to participate in the study. Overall, twelve schools with
teachers participating in the study had only one participating teacher, 4 schools contributed 2
teachers, 3 schools supplied 3 teachers, and one school included 4 participating teachers. The
participating districts included the following, with New York designated as the NORTHEAST
group, districts in Alabama and Tennessee labeled SOUTH, and Utah and California districts
included in the WEST group.
Table 4. Districts and Numbers of Teachers Agreeing to Participate
State # of Teachers # of Schools Region
New York 3 1 NORTHEAST
New York 5 5 NORTHEAST
Alabama 8 3 SOUTH
Tennessee 6 2 SOUTH
Utah 10 8 WEST
California 2 1 WEST
5 States 34 Teachers 20 Schools 3 Regions

28
Randomly Assigning Teachers to Groups
Because the start of school varied among districts and districts agreed to participate at different
times, teachers were randomly assigned by district to comparison or treatment groups. Random
assignment with replacement was completed using the following process. Names of teachers
from the same district were written on same-size slips of paper and folded in half. The papers
were then drawn from a hat. The first name drawn was assigned to the treatment group and
replaced in the hat. The second name drawn was assigned to the comparison group and replaced
in the hat, and so on alternating treatment and comparison groups until all but one name had been
assigned. The last name was automatically placed in the next group (e.g., comparison or
treatment). If a name was drawn that had been drawn previously, it was replaced in the hat and
drawing with replacement continued until a new name was drawn.
Final Sample
Although 34 teachers agreed to participate in the study, the final sample used in the analysis
included only 29 of the 34 teachers. The following table lists the sample selection, random
assignment, and participation.
Table 5. Sample Selection, Random Assignment, and Participation
Sample Number of
Treatment Teachers
Number of
Comparison Teachers
Comments
Original 16 17
Random assignment
repeated in one district
16
(+ 1 team teacher)
17 See Team Teaching
paragraph below
Two treatment teachers
withdrew
14
(+ 1 team teacher)
17 See Withdrew Prior to
Study paragraph
Final sample used in
outcome analysis
14 15 See Final Sample
paragraph below
Team Teaching. One teacher who had been randomly assigned to the comparison group team
taught with a teacher at the same school who had been assigned to the treatment group. Because
they refused to give up team teaching and we were concerned about treatment diffusion if they
remained in the groups to which they had been randomly assigned, we requested the name of a
replacement teacher. The names of the replacement teacher and the teacher who had been
randomly assigned to the treatment group were again put into a hat and the first of the two
names—to be assigned to the treatment group—was drawn. The original treatment teacher’s
name was the name drawn, so she was again assigned to the treatment group. Because of the
team teaching situation, both the treatment teacher and her partner were given McDougal Littell
materials and included in the data collection process. The additional teacher was not included in
the final analysis as she had not been randomly assigned to a group.

29
Withdrew Prior to Study. Two treatment teachers dropped out of the study prior to the start of
school. One teacher who had been assigned to the treatment group decided not to participate
prior to the professional development session. The professional development was conducted in
that district on Thursday and Friday before school started on Monday. The teacher would have
been unable to attend on Friday because she was directing soccer tryouts, and she was
uncomfortable changing to a new curriculum so soon before school started as she had already
laid out lesson plans for the first units of life science. Additionally, the contract days for her
district prior to school starting were few and she was unwilling to sacrifice a day in lieu of
preparing her classroom. Another teacher who left the study attended the in-service just prior to
their school start date (but during teacher contract time). After the morning session, she left in
tears explaining that her life was “too much of a mess”—she couldn’t face adding the stresses of
lesson development and teaching with new curriculum at that time.
Final Sample. The final sample included 14 treatment teachers and 1 team teacher using the
McDougal Littell materials from whom data was collected. Data from the additional teacher was
excluded from the data analysis, but may be used for later reporting and publications. Seventeen
were randomly assigned to the comparison group. Data including pretests and classroom
observations were collected from these teachers, but 1 comparison teacher would not participate
in a second classroom observation. That teacher and another did not return the final data
package, which included student content and attitude posttests, teacher questionnaire, and student
demographics, resulting in complete data from 15 comparison teachers.
Informed and Uninformed Volunteers. When we contacted teachers to tell them the group to
which they had been randomly assigned, we learned that some of the districts had asked for
volunteers who would agree to participate in the study prior to submitting names. Other districts
provided the names of teachers they expected to participate without asking for volunteers. After
discussing the study with those teachers who had not volunteered, even those who had been
reluctant decided to participate. The “selling points” included teacher honorariums, the materials
provided to treatment teachers at no cost, and the research design—a few of the reluctant
teachers who were assigned to the comparison group might have typically refused but decided to
participate because of the quality of the randomized research design.
Target Classroom
Target classrooms for the study were selected using the following criteria:
• Students were 7th
grade students, if the teacher taught multiple grades.
• First and final periods were excluded, as the researchers conducting this study have observed
teacher and student behavior that occurred during these periods that was considerably
different than other periods.
• Classrooms were selected that had no more than a “typical” number of “inclusion” students
or students receiving special services.
• If the school used some form of tracking (i.e., classrooms consisting almost exclusively of
struggling, “average”, or high achieving students), “average” classrooms were selected.

30
• Class periods were selected that maximized the number of classroom observations that could
be conducted per day.
Demographics of the Sample
Description of Schools
Four of the 20 schools in which the study was conducted included both treatment and
comparison teachers. For this reason, a school could be included in both the treatment and
comparison descriptions below.
Treatment teachers participating in this study taught at 10 different schools. Of the 10 schools,
two had minority enrollments that exceeded 50%, three had more than 50% of their students
eligible for free or reduced lunch, and four were categorized as Title-1 Eligible by the U.S.
Department of Education. The average pupil-teacher ratio was 19.4, and the average total school
enrollment was 957.8.
Comparison teachers participating in the study taught at 12 different schools. Of the 12 schools,
three had minority enrollments that exceeded 50%, four had more than 50% of their students
eligible for free or reduced lunch, and five were categorized as Title-1 Eligible by the U.S.
Department of Education. The average pupil-teacher ratio was 18.8, and the average total school
enrollment was 943.2.
Description of Teachers
Of the 14 treatment teachers who completed the study, all had bachelors degrees in either science
or science education, and three had advanced degrees. The average McDougal Littell teacher had
taught for 11.2 years, with a maximum of 35 years and a minimum of 1 year. On average,
treatment teachers had 9.7 years of experience teaching science, with a maximum of 20 years
and a minimum of 1 year.
Of the 15 comparison group teachers who completed the study, 12 had bachelors degrees in
either science or science education, and eight had advanced degrees. The average comparison
group teacher had taught for 12.5 years, with a maximum of 26 years and a minimum of 0 years.
On average, comparison group teachers had 10.6 years of experience teaching science, with a
maximum of 26 years and a minimum of 0 years.
Description of Classrooms and Students
The average total class size for treatment classrooms was 29.2 students, with a maximum size of
36 and a minimum size of 23 students. On average, parents of 2 students per classroom refused
to allow their child to participate in the study by returning the letter refusing permission (range,
0-5; see Appendix D). One classroom consisted entirely of male students. Minority composition
of classrooms ranged from 0.0% to 92.3%, with an average of 24.4%. On average, 8.7% of
students in each classroom had been categorized with a learning disability; this statistic ranged
across treatment classrooms from 0.0% to 26.3%. Finally, in the average treatment classroom
3.4% of students did not speak English as a primary language and 1.4% did not use English as

7TH GRADE LIFE SCIENCE

Recommended

Recommended

More Related Content

Similar to 7TH GRADE LIFE SCIENCE

Similar to 7TH GRADE LIFE SCIENCE (20)

More from Simar Neasy

More from Simar Neasy (20)

Recently uploaded

Recently uploaded (20)

7TH GRADE LIFE SCIENCE