NAEP Data Suggests Opportunity Gap in Learning US History

An Opportunity to Learn US History: What NAEP Data
Suggest Regarding the Opportunity Gap
Tina L. Heafner, Paul G. Fitchett
The High School Journal, Volume 98, Number 3, Spring 2015, pp. 226-249
(Article)
Published by The University of North Carolina Press
DOI: 10.1353/hsj.2015.0006
For additional information about this article
Access provided by Ebsco Publishing (26 Aug 2015 09:05 GMT)
http://muse.jhu.edu/journals/hsj/summary/v098/98.3.heafner.html

An Opportunity to Learn US History: What NAEP Data Suggest
Regarding the Opportunity Gap
Tina L. Heafner
University of North Carolina at Charlotte
Tina.Heafner@uncc.edu
Paul G. Fitchett
University of North Carolina at Charlotte
Paul.Fitchett@uncc.edu
The purpose of this study is to determine the degree to which Opportunity to Learn
(OTL), is associated with students’ achievement in US History. Opportunity to Learn
stems from the basic premise that there is an important relationship between the
quality and frequency of classroom instruction and students’ levels of academic suc-
cess. The authors examine National Assessment of Educational Progress in U.S. His-
tory (NAEP-USH) assessment data in order to better understand the relationship
between classroom- and student-level variables associated with historical knowledge
as measured in the 12th
grade. Findings document that instructional exposure (OTL)
is a factor associated with learning outcomes; however, the OTL in history is not the
same for all student populations. An opportunity gap is evident in these results and
suggests that Black students are underserved in U.S. History. However, when control-
ling for poverty and other socio-economic variables, students’ predictive achievement
on NAEP is positively associated with instructional exposure; yet, differences were
observed for some but not all strategies measured in the full model. In the aggre-
gate and disaggregate models, instructional exposure and motivational factors asso-
ciated with OTL account for a large and significant percentage of the proportional
variance (r2
) in NAEP-USH achievement. We contend that instructional exposure
is significantly predictor of historical knowledge. Findings from this study indicate:
1) student and school characteristics are significant factors associated with achieve-
ment, 2) pedagogical decisions are important and can greatly impact student learning
in US History, and 3) culturally congruent instructional decision-making is needed to
ensure positive learning experiences for Black students.
Keywords: Opportunity to Learn, Social Studies, NAEP US History, Student Variables,
Instructional Strategies, Instructional Exposure, Minority Opportunity Gap, Culturally
Responsive Pedagogy
Introduction
What remains untested is the fact that “we know how to educate poor and
minority children of all kinds—racial, ethnic, and language—to high levels.... But
the nation as a whole has not yet acted on that knowledge....” (Commission on
Chapter 1, 1992).
© 2015 The University of North Carolina Press
226

How students perform on high school standardized exams in U.S. History, such as the
National Assessment of Educational Progress (NAEP-USH) 12th
grade assessment,
provides a context for describing the national state of history education with broad
implications for social studies. A recent report from the Department of Education
suggests that U.S. schoolchildren have inadequate historical knowledge (National
Center for Education Statistics, 2011). In 2010, less than half of students (45%) scored
at or above the Basic level, and only 12% performed at or above the Proficient level
(National Center for Education Statistics, 2011). These findings are indicative of well-
documented trends and serve as a call for history and social studies reform (Evans,
2004; Ravich, 2010; Ravitch & Finn, 1987; Saye & The Social Studies Inquiry Research
Collaborative, 2013). Nonetheless, researchers contest the adequacy of these reports in
capturing the competency of young peoples’ history knowledge and the unique attributes
of discipline-specific learning (Barton, 2011; Rothstein, 2004; Wineburg, 2004).
Although there are limitations to NAEP data, and we do not contend that these data
provide the complete picture of instruction, NAEP data are recognized as the national
measure of what students know in the content areas and are widely accepted mea-
sures of achievement. Moreover, NAEP-USH is designed to evolve from a fact-laden,
culturally-biased assessment to a more inclusive test, featuring items representative of
various cultural perspectives and emphasizing critical thinking (House & Nancy, 1990;
National Assessment Governing Board, 2010; Zwick & Ercikan, 1989). NAEP-USH pro-
vides researchers with a national assessment of the content narrative and skills vetted
by teachers, historians, and other educational professionals (National Assessment
Governing Board, 2010). Results have been used over the years to direct educational
policy and to question, challenge, critique, and applaud various reform efforts such
as A Nation at Risk, No Child Left Behind, and Race to the Top.
Fundamentally, NAEP data provide an examination of the complex school-, class-
room-, and student-level variables associated with learning outcomes (cf. Smith &
Niemi, 2001). These data illuminate patterns in student performance. The uniqueness
of US NAEP highlights specific trends indicative of concerns in history education.
First, there is a persistent problem of low achievement among all students in history
(Stedman, 2009; National Center for Education Statistics, 2011). Second, performance
in the area of history remains the lowest among core content area assessments with
more than half the students falling below the basic level (Stedman, 2009). Third, the
gap between achievement scores for Blacks and Whites remains almost as large as it
was in the 1960s and a similar gap persists for Hispanic students (Konstantopoulos &
Borman, 2011; National Center for Education Statistics, 2011; Stedman, 2009). The
implications for U.S. history are noteworthy given recent emphases on culturally
responsive and relevant pedagogy, equity, and social justice (Epstein, 2009; Gay,
2000, 2002; Ladson-Billings, 1995, 2009; Salinas, 2006). Fourth, little has changed
in classroom instructional practices, which remain traditional and teacher-centered
(Crocco & Costigan, 2007; Henry, 1993; Saye & The Social Studies Inquiry Research
Collaborative, 2013; Stearns, Siexas, & Wineburg, 2000; Stedman, 2009; VanSledright,
2011). Given the growing national diversity in U.S. schools, these trends raise con-
cerns and merit serious attention.
Numerous studies identify the link between teacher instruction, opportunity to learn,
and schooling context with students’ historical knowledge (Epstein, 2009; Gradwell,
2006; Grant, 2003; Ladson-Billings, 2009; Pace, 2011; Saye & The Social Studies
Inquiry Research Collaborative, 2013). Drawing upon the association between the
teacher effect and achievement outcomes (Nye, Konstantopoulos, & Hedges, 2004),
the Opportunity to Learn (OTL) stems from the basic premise that there is an important
An Opportunity to Learn US History
227

relationship between the quality and intensity of classroom instruction and students’
levels of academic success. While quality is important, without some level of instruc-
tional exposure, learning will not occur and quality instruction will not exist. Thus,
frequency of instructional exposure can be viewed as a baseline measure of classroom
instruction and a gauge of learning opportunities. In this study, OTL is operationalized
as the degree to which a student experiences classroom instruction, including a variety
of approaches that address a range of cognitive processes, teaching practices, and group-
ing formats (Kurz, Elliott, Lemons, Zigmond, Kloo, & Kettler, 2014). While research
touts specific strategies to support and embrace the complexity and diversity of stu-
dents, there are limitations confounding the impact of these ideas. Small sampling
frames, a fluctuating definition of the goals and purposes of history and social studies
instruction, and limited replicability have inhibited a broad examination of history
teaching (Au, 2007; Thornton & Barton, 2010; VanSledright, 2011). However, it is well
established that longstanding traditional practices, especially lecture, continue to per-
vade history classrooms (Henry, 1993; Ravitch & Finn, 1987; Saye & The Social Studies
Inquiry Research Collaborative, 2013; Shaver, Davis Jr., & Helburn, 1979; Stearns,
Siexas, & Wineburg, 2000). Furthermore, teachers have yet to create equitable learning
environments for all students (Cummins, 2000; Litner & Schweder, 2008; Olson, 2006).
Only a handful of large-scale, quantitative research studies have explored the associa-
tion between measures of classroom context and teacher effectiveness on historical
knowledge. Of the national survey research conducted during the years 1987 to
2005, less than 5% examined history or social studies, providing the smallest amount
of generalizable quantitative evidence on instruction among the core subject areas
(Camburn & Won Han, 2011). While previous research has acknowledged race and
socioeconomic achievement gaps (Reardon, 2011), few studies have explored this
trend within history education. Recognizing that the perception of an achievement
gap shifts blame away from schools and to the student (Darling-Hammond, 2010), the
achievement gap is more accurately an indicator of an opportunity gap (Brown &
Brown, 2012; Boykin & Noguera, 2011; Rothstein, 2004). In addressing this opportunity
gap and responding to a call for more quantitative research on instruction in history
education (Camburn & Won Han, 2011; Saye & The Social Studies Inquiry Research
Collaborative, 2013), we examined high school NAEP-USH scores for the most recent
available data. Using a hierarchical regression model, we analyzed 2010 assessment
data in order to better understand the relationship between school-, classroom-, and
student-level variables associated with historical knowledge as measured in the
12th
grade. Our purpose was to examine variability within the long-standing gap in
history knowledge (The Bradley Commission on History in Schools, 1989). Deter-
mining the degree to which classroom practices and learning experiences are linked
with a narrowing of the opportunity gap has the potential to inform the development
of effective and equitable history teaching.
Theoretical Framework
Previous research has suggested myriad contextual factors associated with teaching
and learning in history (Au, 2007; Grant, 2003; Leming, 1991; Levstik, 2008; Saye &
The Social Studies Inquiry Research Collaborative, 2013; Shaver, Davis Jr., & Helburn,
1979). These contexts are situated within two interacting domains contributing to his-
tory knowledge outcomes: demographic/school-level characteristics and opportunity
to learn (OTL) variables. OTL variables include content access, instructional exposure,
and curriculum (Stevens, 1993b).
Student Demographics and School-Level Characteristics
Research in history education has found considerable variation in students’ outcomes
across school-level urbanity, standardized testing policy, and socioeconomic status
The High School Journal – Spring 2015
228

(Banks, 2008; Epstein, 2009; Konstantopoulos & Borman, 2011; Pace, 2011). In addi-
tion, NAEP-USH report card findings indicated an opportunity gap between students
of various demographic types (National Center for Education Statistics, 2011). There
are multiple factors that contribute to this performance gap, including background
characteristics such as parents’ education and poverty level (Abedi & Gándara,
2006; Konstantopouslos, & Borman, 2011; Nichols, Glass, & Berliner, 2012); gender
and race (Reich, 2011; Smith & Niemi, 2001; Zwick & Ercikan, 1989); the challenge
of second language learning (Abedi & Herman, 2010; Freeman, Freeman, & Mercuri,
2002; Schleppegrell, Achugar, & Oteiza, 2004); and cognitive and learning differences
for students with disabilities (Cawthon, Kay, Lockhart, & Beretvas, 2012; Grant, 2003;
Litner & Schweder, 2008). Reardon (2011) contended that the income opportunity
gap is nearly double that of the Black-White opportunity gap and that this gap has
grown steadily for Blacks due to a growth in income inequality since the 1980s. Reardon
suggested “income segregation has led to greater differentiation in school quality and
schooling opportunities” (p. 13).
Research has also shown that economically disadvantaged and culturally diverse sub-
groups have less access than other students to challenging curriculum (Cummins,
2000; Delpit, 2006; Nichols, Glass, & Berliner, 2012; Saye & The Social Studies Inquiry
Research Collaborative, 2013). Technical analyses of standardized history tests, includ-
ing previous additions of NAEP-USH and the NY Regents exams, pointed out substan-
tial discrepancies in achievement scores associated with race and gender (Reich, 2011;
Wineburg, 2004; Zwick & Ercikan, 1989). Likewise, school urbanity (urban, suburban,
or rural) is associated with performance outcomes (Konstantopouslos, & Borman, 2011;
Malloy & Wohlstetter, 2003; Morris & Monroe, 2009). Collectively, the forces behind
the gaps in achievement are complex and interconnected.
Student Opportunity to Learn Indicators
Previous studies have indicated a link between school-level characteristics and student
experience (Fitchett & Heafner, 2010; Fitchett, Heafner, & Lambert, 2014; Gerwin &
Visone, 2006; Konstantopouslos & Borman, 2011; Henke, Chen, & Goldman, 1999).
These experiences are often situated within Opportunity to Learn (OTL) frameworks
(Stevens, 1993b, 1996). OTL includes variables measuring content access, instructional
exposure, curriculum, and structures of classroom/school contexts. In this study, OTL
is defined as the degree to which a student is exposed to specific content, cognition, and
pedagogical practices (Kurz, Elliott, Lemons, Zigmond, Kloo, & Kettler, 2014). OTL has
evolved to encompass the instructional practices, curriculum, standards, and assess-
ments associated with student success (Carroll, 1963; Wang 1998). It includes equitable
access to educational resources, rigorous coursework and content, and high quality
instruction. OTL suggests that there is a strong relationship between the quality and
intensity of classroom instruction and students’ academic achievement (Monte-Sano &
De La Paz, 2012; Reisman, 2012; Saye & The Social Studies Inquiry Research Collabo-
rative, 2013). Furthermore, OTL is an indicator of instructional inequalities and can be
used to explain performance differences among students (Stevens, 1993a).
Research has also indicated instructional practices and teacher dispositions affect
learners’ development of historical knowledge (Grant, 2003; Levstik, 2008; Maggioni,
VanSledright, & Alexander, 2009; Ravitch & Finn, 1987; Smith & Niemi, 2001). The
instructional practices and purposes of history education privileged by the teacher
can influence how much and to what degree students learn historical content. Recent
studies examining how students make sense of historical thinking and test-taking have
substantial implications for OTL in history (Monte-Sano, 2012; Reich, 2009; Reisman,
2012; Saye & The Social Studies Inquiry Research Collaborative, 2013). In their
229

cross-state analysis, Saye and colleagues reported teachers who promoted authentic
intellectual work in their history classrooms (including group projects, discipline
inquiry, structured discussion, and student engagement) scored better on average
when compared to peers who used more traditional, rote instructional models. How-
ever, their analysis was unable to provide a standardized metric on the assessments
thus limiting any generalizations associated with ecological factors. Investigating test-taker
responses on the New York Regents Exam, Reich (2009) concluded that student’s success
on these tests “center around an ability to work with narrative and written text” (p. 348).
Similarly, Monte-Sano (2012) noted that reading and writing within the context of the
discipline are essential skills for content knowledge development. The connections
between reading and writing with the curriculum and historical knowledge development
are intertwined. Reisman’s (2012) quasi-experimental study of the Stanford-supported
Reading Like a Historian curriculum suggests that discipline-specific instruction
increases students’ access to reading and writing, in addition to improving overall his-
torical knowledge and comprehension. While these regional and localized studies of
history instruction and achievement are informative, recent studies have not examined
the relationship between students’ learning exposure and historical knowledge.
Complicating OTL is the inequity of access among students, which also hinders stu-
dent achievement (Boykin & Noguera, 2011). Like other subject areas, research in history
education has indicated that student socioeconomic status, race, and gender influence
history teachers’ instructional purposes and strategies (Bourdillion, 1999; Crocco, 2008;
Cruz & Thornton, 2009; Epstein, 2009; Smith & Niemi, 2001; Venkateswaran, 2004).
These dispositions impact students’ OTL. Culturally diverse and economically dis-
advantaged students are more likely to experience substandard classroom practices
and are given fewer opportunities to participate or engage with content (Abedi & Herman,
2010; Au, 2010; Grant, 2001; Ladson-Billings, 2009; Pace, 2011; Salinas, 2006; Stevens,
1993a). In addition, race is associated with NAEP-USH OTL indicators such as content
coverage and instructional strategies (Abedi & Gándara, 2006; Cruz & Thornton, 2009;
Nichols, Glass, & Berliner, 2012; Smith & Niemi, 2001).
Moreover, teachers’ decision-making and professional attitudes are connected to the
demographics and background of the students they teach (Abedi & Herman, 2010;
Cawthon, Beretvas, Kaye, & Lockhart, 2012; Epstein, 2009; Fitchett, Starker, & Salyers,
2012). For example, Abedi and Herman (2010) suggested that the opportunity gap
between English learners (ELs) and native speakers, while typically associated or
explained by language acquisition, is more commonly evidence of differences in
instructional decisions, which frequently constrict the OTL. Although the academic
language of social studies can be challenging for ELs, teachers can instruct students
in reading interventions to reveal how discipline-specific knowledge is constructed
within texts (Schleppegrell, Achugar, & Oteiza, 2004). Yet, teachers make assumptions
about abilities of students based on cultural and socioeconomic backgrounds and
rarely provide the texts needed to advance cognitively adept learners (Cruz & Thornton,
2009; Litner & Schweder, 2008; Salinas, 2006). As a result, opportunities to engage in
content-rich discussion or to read complex, authentic texts are significantly limited,
especially for students of color (Cummins, 2000; Hess, 2009; Ladson-Billings, 2009;
Venkateswaran, 2004; Williams, 2001).
Policy over the last four decades has raised the awareness of a racial and cultural
academic divide in U.S. schools (Roach, 2004). The affirmation that Black and Latino
students have more limited educational opportunities than White and Asian stu-
dents is a departure from the avoidance of decades earlier. Research affirms that
schools and teachers are significant factors associated with the persistent opportunity
230

gap (Cole-Henderson, 2000; Cummins, 2000; Nye, Konstantopoulos, & Hedges, 2004;
Roach, 2004; Rockoff, 2004). Studies suggest acknowledgement of the wide chasm of
racial OTL is not a reflection of inferiority of any race, but rather a reminder to gal-
vanize and mobilize change in schools and instructional practices. Similarly, history
teachers and teacher educators have made culturally responsive teaching a priority
(Epstein, 2009; Gay, 2000, 2002; Ladson-Billings, 1995, 2009; Salinas, 2006). However,
we contend that these efforts have not been enough to improve the OTL, especially for
Black students, whose achievement levels remain stagnant. While macro-level studies
have consistently pointed out a racial OTL-gap, the majority of work examining the
context of learning among students of color in history classrooms remains primarily
small-scale and case-based (Epstein, 2009; Epstein, Mayorga, & Nelson, 2011; Salinas,
2006). Few large-scale studies have examined the unique contribution that instruc-
tional experiences can have on the opportunity to learn history for students of color.
Method
Building upon existing research on students’ knowledge of history, we used data from
the NAEP 2010 U.S. History Assessment to examine the complex relationship among
school characteristics, demographics, and OTL on history achievement. Our study
focused on four research questions:
1. To what extent do student background and school-level variables relate to 12th
grade
students’ U.S. History achievement?
2. To what extent do opportunity to learn (OTL) variables (both curricular and instruc-
tional) relate to 12th
grade students’ U.S. history achievement?
3. Among Black students, to what extent do student background and school-level vari-
ables relate to students’ U.S. History achievement (as measured by NAEP-USH)?
4. Among Black students, to what extent do opportunity to learn (OTL) variables (both
curricular and instructional) relate to 12th
grade students’ U.S. history achievement?
Data
We used data from the NAEP 2010 US History Assessment, 12th
grade. The test
includes a set of student background items and a battery of multiple choice, short
response, and constructed response items. NAEP sampling approximates U.S. popu-
lation estimations. For NAEP-USH, estimates of student populations are determined
through inverse, probability-weighted cluster sampling procedures (National Center
for Educational Statistics, 2011). Using matrix sampling, individual respondents take
a portion of the overall NAEP-USH assessment at a given grade level. NAEP does not
provide individual student outcomes or scores. Item-response theory procedures
generate approximate estimates of students’ overall performance, known as plausible
values (PVs). The composite of five PVs serves as the dependent variable and with a
range of 0 to 500, mean is normalized at 250. These composite scores allow researchers
to reliably compare achievement across student populations. For our study, we
selected public school respondents to avoid confounding demographics and cur-
ricular issues nested within private schools (n full 5 8,160)1
. For the second part of
our analysis (research questions 3 and 4), we disaggregated students identified in
school records as Black (n Black 5 1350).2
The extent to which NAEP-USH measures historical understanding is highly conten-
tious. The National Assessment Governing Board (2010) claims that NAEP measures
not only content knowledge, but critical thinking. Conversely, history educators and
researchers have challenged the ability of a multiple-choice focused assessment like
1
All samples are rounded to the nearest 10 in conjunction with NCES data reporting procedures.
2
The student designation of “black” is a US Census indicator of race and include various Afro-ethnicities.
231

NAEP-USH to appropriately measure historical inquiry (Gaudelli, 2002; Rothstein, 2004;
Wineburg, 2004). Given these theoretical considerations, we chose to operationalize
NAEP achievement as content knowledge and the resulting implications of our analy-
sis should be interpreted as such.
Variables
In addition to providing PVs for each respondent, students and schools provided infor-
mation on school-level demographics, student background information, and opportunity
to learn (OTL) items, such as instructional experiences and exposure to the curriculum.
Our statistical models were designed to control for various factors (SES, cultural back-
ground, gender, and etc.) that collectively comprise student and school differences. For
student demographic variables, we analyzed school-reported data on student race,
gender, free/reduced lunch status, Limited English Proficiency (LEP) status and Indi-
vidual Education Plans (IEP) (block 1). The terms we use to identify groups of students
are derived from the language used in NAEP. We also include a student-reported item
on mother’s level of education. We then incorporated school-level variables: school
percentage quartiles of free/reduced lunch status and percentage of Black students
(block 2). Previous research has offered a precedent for including these contextual vari-
ables at the student-level model, while maintaining traditional indicators for school-level
social capital (Coleman, 1966; Hox, 2010; Raudenbush & Bryk, 1986; Raudenbush, Bryk,
Cheong, Congdon, & du Toit, 2004). We also included dummy codes for school urbanity
(as determined by national census data). These school-level variables are unique eco-
logical contributions in educational research. Earlier research found that school commu-
nity (i.e. urbanity) influences how subject area curricula are learned (Delli Carpini &
Keeter, 1989; Kozol, 2005). Delli Carpini and Keeter (1989) noted rural communities
shared distinct traits in their development of political knowledge and civic education
that separates them from urban areas. Moreover, urbanity represents numerous cul-
tural capital factors that contribute to the learning climate (Kozol, 2005; Levinson,
2012). While we acknowledge that these indicators do not reflect the cultural distinc-
tiveness and diversity of those classified or labeled in these categories (c.f. Castania,
2003), these terms allowed us to examine the outcomes of racial and socioeconomic
divisions as well as make comparisons to the normed group.
To investigate student OTL, we examined items from the NAEP student survey. This
battery of Likert-type items included inventories of students’ instructional experi-
ences, interest in history, and effort on the test. However, student responses to instruc-
tional exposure are often biased (Henke, Chen, & Goldman, 1999). Smith and Niemi
(2001) recommend controlling for this bias by including student responses to sub-
ject area interest and effort. We incorporated respondents’ effort on the test, interest
in history, and perceived difficulty of the subject in the model. To examine students’
effort on the test, interest in US history, and perception of subject area difficulty, we
normalized (using Z-scores) the single Likert items related to a student’s effort on
a test, interest in history, and subject easiness respectively.
Research on the pedagogy of social studies/history teaching suggests that history instruc-
tion and learning is experienced within separate orientations (Fitchett & VanFossen,
2013; Maggioni et al., 2009). These orientations are often comprised of several different
individual instructional types (Fallace, 2010; Ross, 2006). In a nationwide study of
social studies teachers, Fitchett and VanFossen (2013) noted that history teachers’
instructional decision-making coalesced around three pedagogical types labeled:
teacher-centered instruction, student-centered instruction, and discipline-specific
instruction. Their analyses demonstrated that teachers who reportedly relied on
textbooks were also more likely to use worksheets. Conversely, teachers who
232

reportedly exposed students to primary source materials were more likely to
emphasize mapping and writing within the content area. Thus, they concluded that
examining a single instructional exposure item is insufficient.
In order to discern underlying instructional exposure orientations associated with
students’ responses to the instructional exposure inventory of NAEP, we conducted
a principal axis factor analysis (PAF) with orthogonal rotation. Prior to analysis, we
removed a single item, frequency of tests and quizzes, because it did not fit con-
ceptually with the other instructional items. This single item was normalized and
included separately in the model. Conducting the PAF, Kaiser-Meyer-Olkin measures
of adequacy were sufficient (>.820). Analysis of eigenvalues greater than one and
screen plots indicated two factors, which accounted for 43% of the variance among
items (see Table 1). To label the factors, we looked at commonalities in the learning
interactions among students and pedagogical types associated with instruction. The
first category represents a continuum of social interaction such as field trips, use of
film, online, group projects, and guest speakers (Guthrie, Klauda, & Ho, 2012). This
factor includes instructional experiences in which information is learned in multiple,
experiential modes frequently championed in history education (Brophy, 2006;
Heafner & Friedman, 2008; Hicks, Doolittle & Lee, 2004; Marcus, Levine, & Grenier,
2012; Saye & The Social Studies Inquiry Research Collaborative, 2013; Stoddard &
Marcus, 2010; Thornton, 2005). We labeled this category multimodal instruction.
Table 1: Orthogonally rotated principal axis factor loadings for NAEP student
instructional exposure inventory (n 5 8160)
Item
Factor
(% contributed to overall
inventory variance)
Extracted
Communalities
When you study history or social
studies in school, how often do
you do each of the following?
(5 point Likert items)
Multimodal
Instruction
(29.09)
Text-Dependent
Instruction
(13.91)
Work on group project .627 .418
Give presentation to the class .619 .403
Write a report .519 .324 .374
Use books or computers in
library for schoolwork
.493 .260
Listen to information
presented online
.476 .260
Go on field trips or have
outside speakers
.380 .144
Watch movies or videos .337 .115
Discuss material studied .658 .434
Read extra material not
in textbook
.608 .354
Read material from textbook .592 .404
Use letters, diaries, essays by
historical people
.545 .362
Write short answers to questions .514 .338
Note. Loadings <.30 suppressed for simple solutions.
233

The second category includes pedagogy closely associated with text-oriented learn-
ing. These NAEP items are reading-related tasks where knowledge is drawn directly
from texts and learning occurs through inquiry, recursive writing, and reflective dis-
course in response to texts. Text content comprises information within and beyond
textbooks, including primary source history texts such as letters, diaries, and essays
by historical people. We named this category text-dependent instruction. This cate-
gory fits conceptually with the extant theory and research on text-based history
instruction and learning (Monte-Sano, 2012; Monte-Sano & De La Paz, 2012; Reich,
2009; Reisman, 2012; Wineburg, 2001). Cronbach’s alpha indicated adequate reliability
for multimodal instruction (a5 0.71) and text-dependent instruction (a5 0.74). Factor
scores were then calculated for each respondent (mean 5 0, standard deviation 5 1).
These factor scores were used as composite variables in order to account for students’
instructional exposure.
Analytical Procedures
For this study, we employed Hierarchical Regression Plausible Values Model to
examine the unique contributions of each variable block on student achievement (as
measured by the PV composite for each student). This statistic, an acceptable analysis
with NAEP data (Li, Oranje, & Yanlin, 2009), allowed us to account for the unique
variance contributed by various predictor types. Using American Institute of Research
AM software, jackknife procedures were conducted and replicate weights were cal-
culated in order to obtain accurate estimates and reliable standard errors. For the pur-
pose of this study, we grouped variables around the four research questions. For
research question 1, student and school contextual variables were analyzed (blocks 1
and 2). For research question 2, OTL items were grouped (block 3). We then ana-
lyzed a subset of Black students, replicating the same procedure (research questions 3
and 4). Results of our analyses follow. Table 3 indicates how variables were included
within each model.
Among the independent variables, we wanted to determine whether key demo-
graphics variables (race and gender) were statistically associated with OTL instruc-
tional exposure predictors (text-dependent instruction and multimodal instruction).
Such an association could confound estimates and make it difficult to isolate effects
associated with demographics and instructional exposure. Table 2 illustrates the
demographic background of the sample across instructional exposure scales
(standardized z-scores, with a mean equal to zero). The findings suggest slight vari-
ation in instruction exposure across demographics. Inferential analyses (ANOVA)
found the shared variance between race and gender across instructional exposure
scales to be minimal (h2
5 0.001 to 0.03) with little threat of multicollinearity. There-
fore, we determined that the variables of interest could be examined independently
of one another.
Results
Finding 1: A U.S. History Demographic Gap
Results from model one (Table 4) indicate that approximately 27% of the vari-
ance in NAEP-USH achievement can be predicted by students’ individual demo-
graphics. Specifically, Black and Hispanic students score significantly lower than
their White peers when controlling for socioeconomic status (free/reduced lunch
status) and student accommodation (EL status and IEP status). The Black-White
student opportunity gap was the most profound, whereby a Black student scores
approximately 20 points below a White student (on average). This gap is over
two times as large as the gap between Whites and Hispanics and for students with
free/reduced lunch status. Males and students with mothers who have college
234

degrees scored significantly higher than females and learners whose parents did
not attain a college degree. These findings reflect the importance of background
knowledge and social capital while also suggesting the male-orientation of U.S. His-
tory (Crocco, 2008; Frederickson, 2004; Reich, 2011; Wineburg, 2004; Venkateswaran,
Table 2: Variables specifications for NAEP US History 2010
Block Variable Description
Block 1—Student
Level Indicators
Male Comparison category Female
Race (White) Dummy coded Black, Hispanic,
Asian/Pacific, American Indian,
and Unclassified (comparison
category White)
Mother graduated
from college
Comparison categories mother did
not graduate from HS, mother
graduated from HS, mother
attended some college, and
don’t know
ELL (yes) Language Learner classification
(comparison category no)
IEP (yes) Individualized Education Plan
(comparison category no)
Free/Reduced
Lunch (yes)
Comparison category no
Block 2—School
Level Indicators
Urbanity Dummy coded, school is located in
urban, rural, or small-town setting
(comparison group is urban)
School Free/
Reduced Lunch
Dummy-coded into quartiles
(comparison group <25%)
School Percent Black Percentage of school enrollment
identified as Black
Block 3—Student
OTL Indicators
Effort on History Test Standardized score (z-score) of
Likert response to effort on test
Interest in History Standardized score (z-score) of
Likert response to interest of
the subject
History is Easy Standardized score (z-score) of
Likert response to History is Easy
Online Took course online (comparison
category no)
Frequency of
Assessment
Standardized score (z-score) of
Likert response to frequency
of test/quizzes
Multimodal Instruction Standardized factor scores
associated with variables loading
onto Multimodal instruction factor
Text-Dependent
Instruction
Standardized factor scores
associated with variables
loading onto Text-Dependent
instruction factor
235

2004; Zwick & Ercikan, 1989). Specifically, our findings suggest that race, poverty
level, and gender are significant predictors of students’ historical knowledge. As
Table 4 illustrates, this effect remains powerful even after controlling for school-
level and OTL variables.
Finding 2: Socioeconomic ecology of Schools Matter, but to What Degree?
In model 2 (Table 4), we added two school-level contextual variables. Findings indi-
cated that the contributions to variance explained in NAEP-USH were not significant
(1%). Moreover, urbanity of school location at the student level was not signifi-
cantly associated with history achievement. When accounting for the full sample
of 12th
grade public school students, the percentage of Black students enrolled was
not significant. However, findings suggested that increased numbers of students from
low socioeconomic schools are associated with a decrease in NAEP-USH achievement.
A student whose school is populated by 75–100% students receiving free/reduced
lunch was associated with an approximately 11-point decrease compared to a student
from a school with >25% free/reduced lunch population. Thus, while school locale
and minority enrollment were not significantly associated with achievement, students
in schools with high concentrations of poverty were associated with lower achieve-
ment. Given that high poverty schools are associated with lower reading and math
levels, we wonder if literacy may be a factor limiting students’ opportunity to learn
U.S. History. Furthermore, this may be an indicator of an emphasis on remediation
for these subjects common in high poverty school contexts (Gordon, 2012; Reardon,
2011; Wills & Sandholtz, 2009). More research is needed to determine if this latter
connection exists.
Finding 3: Instructional Experiences Improve U.S. History Achievement
In contrast to model 2, taking into account various OTL predictors in model 3 increases
the amount of variance explained NAEP-USH by a significant 16% as reported in
Table 4. Findings suggested that motivational indicators such as perceived easiness
of history and subject-area interest were significantly associated with students’ perfor-
mance. Interestingly, taking U.S. History online was associated with almost 4 fewer
points on average compared to students who took the class face-to-face, implying that
online instruction should not be uniformly championed as an appropriate pedagogy
for history instruction. Analysis of the exposure to instruction factors indicated that
for each standard deviation increase in text-dependent instruction, NAEP-USH scores
Table 3: Race and Gender Representation Across Instructional Exposure Standardized
Estimates (n 5 8,170)
Variable
Text-Dependent
Instruction
Multimodal
Instruction
Race
White n 5 4540 0.03 −0.12
Black n 5 1360 0.01 0.27
Hispanic n 5 1600 −0.05 0.11
Asian/Pacific Isl. n 5 510 −0.11 0.01
American Ind. n 5 100 0.13 −0.07
Other n 5 60 0.04 −0.08
Gender
Male n 5 4060 −0.05 0.02
Female n 5 4110 0.04 −0.02
236

increased by approximately 8 points. Conversely, each standard deviation increase in
exposure to multimodal instruction was associated with a decrease of approximately
7 points. These findings suggest that text-dependent instruction types—including a
combination of discussion, textbook reading, and document analysis—were positively
associated with U.S. history knowledge acquisition. These findings mirror previous
research proposing that multifaceted text-oriented instruction improves historical con-
tent knowledge (Barton & Levstik, 2004; Grant, 2003; Monte-Sano, 2012; Reisman,
2012; VanSledright, 2011). The composite of higher amounts of reading from various
source materials, conducting whole class discussion, and writing in the subject area
Table 4: Summary of Hierarchical Regression Analysis for Variables Predicting 12th
grade US History Achievement (n 5 8,160)
Variable
Model 1 Model 2 Model 3
B SE B SE B SE
Constant 291.80*** 0.80 296.65*** 1.37 295.2*** 1.29
Male 6.28*** 0.70 6.3*** 0.70 5.85*** 0.68
Race 5 Black −20.31*** 1.52 −17.62*** 1.37 −13.65*** 1.41
Race 5 Hispanic −7.60*** 1.33 −6.35*** 1.33 −5.37*** 1.24
Race 5 Asian, Pacific 4.82 2.53 3.84 2.38 6.60** 2.01
Race 5 American Ind. −10.97* 3.16 −7.99* 2.96 −10.25* 3.05
Race 5 Unclassified −0.30 4.87 −0.33 4.85 −0.35 4.86
Mother graduated
from College
9.73*** 0.85 8.69*** 0.86 7.15*** 0.84
LEP (yes) −35.31*** 2.38 −34.40*** 2.38 −27.07*** 2.13
IEP (yes) −26.58*** 1.68 −26.61*** 1.71 −22.97*** 1.56
Free/Reduced
Lunch (yes)
−9.25** 0.80 −6.77** 0.74 −5.12*** 0.77
Urbanity 5 Suburb −0.30 1.43 0.19 1.22
Urbanity 5 Town −3.66 1.89 −2.07* 1.82
Urbanity 5 Rural −3.36 1.37 −2.47 1.37
School Percent Black −0.04 0.025 −0.03 0.02
School Free/Reduced
Lunch 5 26–50%
−3.81** 1.22 −4.03** 1.16
School Free/Reduced
Lunch 5 51–75%
−7.23*** 1.59 −7.31*** 1.64
School Free/Reduced
Lunch 5 76%–100%
−10.54*** 1.85 −11.16*** 1.60
Effort on History Test −0.42 0.38
Interest in History 4.52*** 0.50
History is Easy 4.34*** 0.33
Took US History Online −3.76*** 0.99
Frequency of Assessment −0.72 0.42
Multimodal Instruction −7.48*** 0.49
Text-Dependent
Instruction
8.61*** 0.38
Model R2
0.27 0.28 0.42
F for D R2
– 1.69 280.45***
Note. *p<.05, **p<.01, ***p<.001.
237

were associated with greater U.S. history content knowledge (Monte-Sano & De La Paz,
2012; Hess, 2009). Moreover, it confirms the recommendations of researchers (Monte-
Sano, 2012; Monte-Sano & De La Paz, 2012; Reich, 2009) that text-dependent learning is
associated with students’ historical content knowledge acquisition and the narrow aims
of content-specific tests like NAEP.
Interestingly, increased exposure to multimodal pedagogies was negatively asso-
ciated with outcomes. This finding does not suggest that individually the use of
fieldtrips and group projects are not beneficial for learning history. Rather, when
these instructional activities are overemphasized (one standard deviation), they
may hinder historical content knowledge development. Brophy (2006) argued that
although multimodal, constructivist learning supports higher order thinking (HOT),
a common instructional practice in history classrooms (Saye & The Social Studies
Inquiry Research Collaborative, 2013), it is only effective when background knowledge
exists. He, like other researchers (Barton & Levstik, 2004; Grant, 2003; Monte-Sano,
2012; Reisman & Wineburg, 2008; VanSledright, 2011), associated the development
of historical knowledge with text-dependent learning. While multimodal forms of
instruction increase student interest and provide students who struggle with reading
an opportunity to engage with content (Heafner, 2004; Heafner & Friedman, 2008;
Hicks, Doolittle & Lee, 2004; Kawashima-Ginsberg, 2013; Marcus, Levine, & Grenier,
2012; Stoddard & Marcus, 2010), these forms of instruction have limitations in sup-
porting historical knowledge achievement as measured by NAEP-USH.
It is also plausible that the negative impact of increased intensity of multimodal instruc-
tion may be due to the way in which these strategies are implemented by teachers.
Teachers may be more skilled and confident with text-dependent instruction than
multimodal (Beeson, Journell, & Ayers, 2014; Stoddard, 2013; VanFossen & Waterson,
2008). This difference could be associated with lower quality multimodal instruction.
However, NAEP-USH variables are not measures of quality and this hypothesis is
untestable in our study. We recommend OTL research examining instructional quality
differences between text-dependent and multimodal instruction as associated with
student learning to explore this phenomenon.
Finding 4: Among Black Students, Demographic Effects Shift
In the second part of our analysis, we disaggregated students identified by their school
as Black and conducted a hierarchical regression model (Table 5). Findings indicated
that models 1 and 2 accounted for approximately 16% and 22% in the proportion of
variance in NAEP-USH achievement respectively. Whereas in the overall sample
males outperformed females, Black males did not score significantly differently from
Black females. The absence of a gender gap among Black students cannot be fully
explained by the variables included in our model.
Other factors such as culturally-based gender roles among females or attrition rates
of minority students prior to 12th
grade were not examined. Perhaps the lack of
gender difference is an indicator of attributes of Black females rather than Black
males. Researchers (Eccles, Barber, Jozefowicz, Malenchuk, & Vida, 1999; Gilligan,
1990; Steele, 1997) have documented cultural gender differences in Black girls’ self-
confidence, self-esteem, self-worth, and academic perceptions in comparison to
other races. This research, however, is limited to efficacy and measures a dis-
tinctly different outcome variable than achievement. While we cannot draw direct
correlations to our study, results do lead us to wonder if an untested associa-
tion among gender, culture and student content knowledge may be impacting
our results. We recommend that research be conducted to explore the interaction
among these variables.
238

Similar to the previous models, free/reduced lunch status of the student and school
context were associated with a significant decrease in U.S. History achievement
scores. Unlike the previous full-scale analyses, variance explained by the addition
of school context variables was statistically significant (6%). Black students enrolled
in schools with a higher percentage of Black students were associated with a decrease
in scores. For each 10% increase in Black student enrollment, Black student scores
decreased by an average of 1.4 points. For a school that enrolls >50% Black students,
a Black student scores between 7 to 14 points less (on average). This finding suggests
that schools with high minority enrollment have unique contextual and cultural con-
siderations that can affect Black student achievement in U.S. History. This finding also
reaffirms previous research, which has indicated that majority-minority schools,
symptomatic of school de-facto segregation, can have negative consequences for
learners (Boger & Orfield, 2005; Mickelson, 2001; Muller, Riegle-Crumb, Schiller,
Wilkinson & Frank, 2010).
Table 5: Summary of Hierarchical Regression Analysis for Variables Predicting
12th
grade US History Achievement of Black Students (n 5 1,350)
Variable
Model 1 Model 2 Model 3
B SE B SE B SE
Constant 275.85*** 1.87 287.52*** 3.12 289.55*** 3.26
Male −0.02 1.82 0.45 1.85 0.37 1.77
Mother graduated
from College
8.43*** 1.83 6.93*** 1.58 6.28*** 1.57
ELL (yes) −30.86*** 9.06 −32.40*** 8.13 −33.45*** 7.46
IEP (yes) −26.83*** 3.10 −27.01*** 3.14 −25.52*** 3.04
Free/Reduced
Lunch (yes)
−10.85*** 2.21 −6.71** 1.97 −5.26** 1.91
Urbanity 5 Suburban 1.39 2.21 −0.68 2.10
Urbanity 5 Town −3.10 4.93 −0.42 5.03
Urbanity 5 Rural −0.93 2.68 −0.38 2.57
School Percent Black −0.14** 0.04 −0.12** 0.04
School Free/Reduced
Lunch 5 26–50%
−8.46* 3.71 −8.42** 3.66
School Free/Reduced
Lunch 5 51–75%
−8.27* 3.41 −9.12** 3.34
School Free/Reduced
Lunch 5 76%–100%
−11.41** 3.32 −11.91** 3.45
Effort on History Test −0.43 0.65
Interest in History 4.29*** 0.99
History is Easy 2.76** 0.83
Took US History Online −0.97 2.31
Frequency of Assessment −1.22 0.79
Multimodal Instruction −6.98*** 0.90
Text-Dependent
Instruction
6.95*** 1.07
Model R2
0.16 0.22 0.34
F for D R2
– 89.51*** 211.38***
Note. *p<.05, **p<.01, ***p<.001.
239

Finding 5: Among Black Students, Instructional Exposure Matters
While demographic and school-level contexts are associated with increased U.S. His-
tory achievement, our findings also indicate that variables directly related to instruc-
tional decision-making were associated with Black students’ history outcomes.
Model 3 (Table 5), including OTL factors, was associated with an 11% increase
in the proportion of NAEP-USH explained. Like previous models, indicators of
motivation and effort were associated with increased history scores among Black
learners. Similar to the aggregate model, Black 12th
graders exposed to greater
text-dependent instruction were associated with an increase in U.S. history
achievement. Moreover, the exposure to multimodal instruction was associated
with a decrease in NAEP-USH achievement. In agreement with researchers (Banks,
2008; Gay, 2000, 2002; Ladson-Billings, 1995, 2009), our findings suggest that cul-
turally relevant ways of teaching draw upon effective pedagogy that is beneficial
for all students. Furthermore, the opportunity to learn should not be hindered by
decisions to limit text-dependent instruction for certain populations (Au, 2010; Grant,
2001; Gordon, 2012; Landsman, & Lewis, 2011).
Implications
The fact that Black students perform significantly lower than all other races pro-
vides evidence that this population of students is clearly underserved in U.S. His-
tory. First, the OTL as defined by the frequency of instruction in social studies is
not the same for all students. Results suggested that groups commonly excluded
from the curriculum (e.g. Blacks, Hispanics, Native Americans, females, and the
poor) are unlikely to perform at the same level as White or Asian males who come
from educated, high socioeconomic family backgrounds. A wide chasm, bounded
by race, exists in the educational achievement for Black and Hispanic students
(Konstantopoulos & Borman, 2011; Kozol, 2005). Similar to other studies (i.e. Goldhaber,
Brewer, & Anderson, 1999), an overwhelming proportion of achievement variance
can be attributed to student and school backgrounds. These findings affirm that, like
other subjects areas, a racial opportunity gap exists in history. We suggest that there
may also be other factors influencing outcomes, such as limitations of the test or
cultural and socio-economic differences in how students are taught history. Addi-
tional research is needed to determine the extent to which these variables impact
student learning.
Second, low achievement on standardized history tests by minority and low-income
children (Smith & Niemi, 2001; Zwick & Ercikan, 1989) is often subsumed as
inherent traits of student and family demographics (Olson, 2006). We contend that dif-
ferences in achievement associated with student demographics cannot (and should
not) be the sole predictors of historical learning. Rather, race, cultural capital, and
school contexts reflect institutional-level limiting factors that inversely affect student
achievement. Content knowledge imbalances created by demographic variables
can be offset by focusing on OTL attributes associated with learning outcomes (see
Figure 1). Our findings from block three of the full student population affirm that
OTL factors under the control of teachers, such as the intensity of instructional
exposure, can have a positive impact on students’ history knowledge. When con-
trolling for race and socioeconomic differences, learning experiences can improve
history achievement for all students. Furthermore, gender differences present in
the full model did not exist among Black students and require further examination
to begin to understand gender-related cultural differences associated with content
knowledge. Thereby, results challenge the myth that female students or students
of diverse backgrounds are resigned to lower performance in history classes.
240

Results from the analysis of Black 12th
graders affirm that student background and
school context were associated with U.S. History achievement scores. Moreover,
with greater levels of minority enrollment, achievement among Black students incre-
mentally declined. However, model three of the disaggregate analysis, confirms
the positive impact of OTL variables on Black student achievement. Specifically, stu-
dent interest and perceived ease of the subject along with increased text-dependent
instruction was associated with positive outcomes on the NAEP-USH. These OTL
variables accounted for approximately 12% of the variance in achievement. Pre-
vious studies have noted that children learn differently and that culturally respon-
sive instructional decision-making is necessary to ensure positive learning experiences
in history classrooms (Chikkatur, 2013; Martell, 2013; Ukpokodu, 2006). Thus, we
infer that motivating students through various culturally responsive instructional
practices may contribute to greater U.S. history content knowledge. Furthermore,
OTL attributes of content access and curriculum exposure are foundational to content
knowledge. This content knowledge, as measured by NAEP, represents the traditional
canon of U.S. History, which may or may not resonate with the historical positionality
of students. However, providing students access to the pervasive narrative offers a
starting point for inquiry and critical analysis, skills valued among history educators
and culturally responsive educators alike (Barton, 2008; Gay, 2002; Ladson-Billings,
2009; VanSledright, 2011).
Third, analyses indicate that OTL variables are important and instructional exposure
matters. While seemingly small in comparison to student-level variables (12% of
variance explained in disaggregate sample comparison to 14% for the full sample),
instructional exposure is associated with significant increases in students’ historical
knowledge as measured on the 12th
grade NAEP-USH. Students exposed to greater
text-dependent instruction are associated with higher content knowledge gains on
the NAEP-USH. Evidence of research-based practice in social studies can provide
guidance for how to improve instruction opportunities for all students. These findings
support previous research indicating emphasis on reading and writing within the
Figure 1: Creating Balance in the Opportunity to Learn US History with Pedagogy.
241

discipline is associated with greater history content knowledge and better test-taking
(Monte-Sano, 2012; Monte-Sano & De La Paz, 2012; Reich, 2009; Reisman, 2012). Text-
dependent instruction is clearly linked to achievement. Thus, our research implies that
students who frequently encountered vocabulary and syntax related to U.S. history,
whether through reading, analysis, or discourse, are more likely to achieve higher levels
of U.S. history content knowledge.
Surprisingly, increased exposure to multimodal instruction was inversely related
to student achievement. We posit this finding suggests that frequent non-text-based
instructional activities (at the expense of more text-dependent instruction) may
hinder students’ historical content knowledge and test-taking acumen. History edu-
cators have noted the importance of content knowledge in the progression of his-
torical knowledge (Lee & Shermlit, 2003; Reisman & Wineburg, 2008). Moreover,
Reich (2009) and Reisman (2012) concluded that experiential learning practices,
while engaging, do not expose students to the vocabulary and historical contexts
associated with achievement on U.S. history tests. This belief was also expressed
by Brophy (2006), who suggested that providing students with basic historical con-
tent allowed for more effective constructivist practice. Conversely, engaging students
with experiential learning prior to providing them appropriate context might con-
tribute to improper analyses or false assumptions. It is critical to note that our find-
ings do not suggest various instruction associated with multimodal exposure (such as
field trips, online learning, and group work) are unimportant for historical knowl-
edge. Rather, frequent exposure is not optimal for content knowledge acquisition
as measured by NAEP-USH. The amount and type of instructional exposure is asso-
ciated with achievement.
However, we must be careful not to assume causation. The inverse might be true as
well. Classes with higher-achieving students might receive more or less of a type
of instruction due to teachers’ biases. Results may also be confounded by ineffective
applications of these instructional practices. Researchers have found teacher peda-
gogy as well as teacher preparation lacking in regards to the development of multi-
modal instructional skills (Beeson, Journell, & Ayers, 2014; Stoddard, 2013; VanFossen
& Waterson, 2008). Additional research is recommended to examine the quality of
learning experiences as well as the extent to which multimodal instruction impacts
student content understanding. Because the data interpretation was limited to histori-
cal knowledge outcomes, future research should also explore the role that these
instructional practices have on higher-order historical thinking concepts such as inter-
pretation, empathy, and inquiry (Lee & Shermlit, 2003).
Fourth, when examining the full model, males scored significantly higher than
females. This may be evidence of a gender bias within the U.S. History curricula
(Bourdillion, 1999; Crocco, 2008; Frederickson, 2004; Reich, 2011; Wineburg, 2004;
Venkateswaran, 2004; Zwick & Ercikan, 1989) that cannot be explained by differences
in achievement in reading. In light of statistically significant findings from Saye &
The Social Studies Inquiry Research Collaborative (2013) in which teachers demon-
strated a propensity toward authentic pedagogy in classrooms with a greater presence
of female students, these results raise questions. Furthermore, gender differences,
while present in the full model, did not exist within the subset of Black students.
Unlike the analysis of the full population, Black males did not score significantly
different from Black females. This finding could possibly be understood in light of
previous research indicating an opportunity gap among black males as a result of
institutionalized feelings of neglect and disconnect with schools (Gordon, 2012;
Mickelson, 2001; Noguera, 2003; Ogbu, 2003). However, the absence of a gender gap
242

among Black students cannot be fully explained by the variables included in our
model. Other factors such as culturally-based gender roles or attrition rates of
minority students (i.e. Black males) prior to 12th
grade were not examined. Thus,
we suggest a need for more research to explore these gender-specific findings.
One intriguing gender difference that we observed between model 2 and model 3 was
the lack of significance between Black male and female students which differed from
the full model wherein gender was a significant predictor of achievement. While
NAEP data cannot be used to explain this difference, we sought to contextualize if
other gendered racial differences could be documented in prior research. From a
review of earlier research we found several studies related to female efficacy. Eccles
et al. (1999) sought to examine differences in perceptions of Black and White ado-
lescent girls using a path analysis employing simultaneous regression analyses to
estimate efficacy parameters. Results of their study mirrored earlier findings (Gilligan,
1990; Steele, 1997) that Black girls do not exhibit the same decline in confidence,
self-esteem, self-worth, and academic perceptions that White girls do. Black girls
consistently held a more positive view of themselves than White girls did in all
dimensions. Each of these researchers observed similar ethnic group differences
and found these racial differences to be statistical mediators of efficacy. While we
observed gender differences, we did not measure confidence among Black and White
females; therefore, results cannot be used to determine if a gender cultural difference
exists or why. It is plausible that Black males perform no better than Black females
due to a lack of curricular identity or access (Epstein, 2009; Epstein, Mayorga, &
Nelson, 2011). Perhaps these issues of cultural congruency are more pervasive and
influential than gender issues among Black students. Our results cannot address either
of these points but do raise unanswered questions that can only be examined through
additional research.
Supplementary research is also needed to explore the cultural differences in instruc-
tional exposure and to examine why specific strategies, such as online learning, per-
ceived to be effective methods were either negatively associated or not significantly
associated with NAEP scores. Of interest, teacher dispositions toward students are
not measured in this study. Despite efforts to recognize structural factors that curtail
Black students’ opportunities to learn, teacher decision-making can “unintentionally
open the possibility to homogenize African Americans in a social discourse of spe-
cialized need and intervention . . . whereby African-American students are posi-
tioned as different and deviant from what is considered normal” (Brown & Brown,
2012, p. 13). Thus, how teachers respond to and interact with students of color stu-
dents may attribute to the differences in educational opportunities and experiences
(Au, 2010; Avery, 2002; Banks, 2008; Gay, 2002; Gordon, 2012; Ladson-Billings, 2009;
Landsman, & Lewis, 2011; Stevens, 1996). Lastly, we echo the sentiment of Saye and
The Social Studies Inquiry Research Collaborative (2013) who called for greater state-
wide analysis of teaching and learning in relationship to student outcomes. Unlike
other assessments, NAEP is a norm-referenced test that is not high-stakes. Given the
recent push toward value-added models, we suggest that history and social education
researchers examine how various pedagogical approaches contribute to student
achievement on state-specific assessments. Such research could have specific policy
implications at local and state levels.
Limitations
In acknowledging the limitations of this study, we recognize that researchers, such
as Gaudelli (2002), Rothstein (2004), and Wineburg (2004) question how well find-
ings from the NAEP-USH reflect historical learning. Given its item design and need
243

for psychometric reliability, NAEP relies heavily on multiple-choice and constructed-
response items. This, however, is not a shortcoming of test construction, but rather
an indication of what can and cannot be measured. Well-developed multiple choice,
short answer, and extended response items, like those on NAEP, can capture some
elements of historical thinking (i.e. evaluating a source, identifying perspective, and
cause and effect). What these items cannot evaluate is how students perform when
actually engaged in such activity or when questions are motivating, relevant, or cul-
turally responsive. Items cannot describe how well instruction is enacted (e.g., the
viewing of a film), how that instruction is used to develop deeper understanding
(e.g., the use of film to develop historical context), or if instruction draws upon quality
resources (e.g., historical authenticity of film content). Hence, NAEP can provide evi-
dence of instructional exposure with strategies associated with historical inquiry, but it
cannot evaluate the quality of those learning experiences. We contend that the test,
while not a flawless assessment of historical understanding, can provide some insights
into identifying instructional behaviors that are associated with students’ content
knowledge. However, additional research is needed to determine to what extent these
instructional variables impact student learning.
Fundamentally, NAEP is an accepted measure of content knowledge. Yet, a limi-
tation exists in terms of how content is interpreted. Students’ perception of their
instructional experiences may confound content to include both history and other
social studies courses. As a culmination of content learning, 12th
grade, self-
reported student data may not delineate cumulative instructional experiences in
history from those in other social studies courses. Twelfth graders may have
taken social studies electives or other required social science courses and not just
history courses. Furthermore, students are not asked to limit their responses to
history only.
Moreover, student-reported learning experiences as reported in the NAEP-USH only
offer frequencies and cannot be used to make claims of instructional quality. Yet,
as VanSledright (2011) and Reich (2009) noted, there is no perfect assessment and
the nature of social science theory and research only complicates the matter. The
NAEP does, however, provide reliable student population estimates and it allows
researchers to examine how various student populations across the nation compare
on various levels of achievement. For the purpose of our study, we maintain that
the NAEP-USH findings are an appropriate data source for an analysis of this scope.
Furthermore, our research determined that school-level characteristics (percentage of
students qualifying for free/reduced lunch and percentage of Black students) were
significantly associated with various models. Our study does not account for the
unique variance attributed to school-level effects. We posit that in addition to the
substantial findings at the student level, school-level characteristics offer a unique
insight into student achievement, particularly for Black students. Future analysis
might attempt to account for these unique contributions.
Conclusion
We examined National Assessment of Educational Progress (NAEP-USH) assessment
data in order to better understand the relationship between classroom- and student-
level variables associated with historical knowledge as measured in the 12th
grade.
Findings from this study indicate that: 1) student and school characteristics are sig-
nificant factors associated with achievement, 2) pedagogical decisions are important
and can greatly impact student learning in U.S. History, and 3) cultural and gender
differences need to be taken into account when examining diverse students’ OTL
and should inform instructional and curricular decision-making.
244

As an outcome of the racial opportunity gap evident in these results, we conclude
that Black students are underserved in U.S. History. Addressing the racial and cul-
tural opportunity gap in social studies as with other subjects continues to be a central
concern. We suggest greater emphasis in professional development for teachers and
administrators that draws upon research exploring how access, curriculum, instruc-
tion, and dispositions can engage students’ cultural assets, diverse backgrounds, and
experiences. Lessons can be gleaned from programs and schools that have been
successful in improving the achievement of Black and Latino students (Borman,
Stringfield, & Rachuba, 2000; Cole-Henderson, 2000; Landsman, & Lewis, 2011;
Nye, Konstantopoulos, & Hedges, 2004; Rockoff, 2004).
When controlling for poverty and other socio-economic variables, students’ achieve-
ment on NAEP is positively associated with instructional exposure. In the aggregate
and disaggregate models, instructional exposure, and motivational factors associated
with the opportunity to learn accounted for a significant and large percentage of the
total variance in NAEP-USH achievement. Pedagogy can have a substantial effect on
students’ historical knowledge.
Specifically, text-dependent instruction is associated with higher levels of achieve-
ment. These results are, however, constrained to frequency of text-based instruction
and do not explain the quality of instruction students received. At a most basic level,
to promote academic success, students need to read, write, and talk about U.S. History.
Recommendations for instructional practice are that what students read, the frequency
of their exposure to informational and complex texts including primary sources, and
their recursive discourse through discussion and writing should be core components
of all students’ history learning. Using effective methods put forth in history education
research as well as supporting student motivation and efficacy are likely to produce
the learning outcomes desired for all students.
Data analyses reported in a previous study confirmed that over-exposure to instruc-
tional strategies can have an inverse, non-linear effect on history achievement
(Fitchett & Heafner, 2013). Our findings guide us in cautioning against the per-
ception that greater frequency of instructional exposure to pedagogical strategies
will be associated with higher achievement. Rather, diversification of instruction
is optimal based on NAEP outcomes. From our models, we conclude that varied
use of specific strategies can produce higher achievement outcomes for all stu-
dents. Moreover, not all methods had similar results when comparing the aggregate
and disaggregate models. We recommend further research to examine culturally
responsive instructional decision-making as a possible solution for equalizing learning
opportunities for Black students.
Results shed light on contextual factors associated with history achievement. Findings
document that OTL is linked to positive learning outcomes. Achievement for all stu-
dents is impacted by utilizing instructional strategies that: a) support diverse student
interests and motivation, and b) encourage content-area literacy, discourse, and his-
torical thinking. Nevertheless, achievement in U.S. history remains unequal for
diverse student populations. School ecology, cultural capital, and student demo-
graphics influence and moderate curricular access; yet, in both models OTL variables
contributed a substantial amount to the overall variance in NAEP-USH achievement.
Therefore, we assert that the opportunity to learn (OTL), the inherent right of every
child in America, is a social studies matter, one in which documented political, social,
economic and cultural gaps are themes associated with achievement and fundamental
to curriculum as social reform.
245

References
Abedi, J., & Herman, J. (2010). Assessing English language learners’ opportunity to learn mathematics: Issues
and limitations. Teachers College Record, 112(3), 723–746.
Abedi, J., & Gándara, P. (2006). Performance of English language learners as a subgroup in large-scale assess-
ment: Interaction of research and policy. Educational Measurement: Issues and Practices, 26(5), 36–46.
Au, K. H. (2010). Isn’t culturally responsive instruction just good teaching? In W. C. Parker (Ed.), Social
studies today: Research & Practice (pp. 77–86). New York, NY: Routledge.
Au, W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher,
36(5), 258–267.
Avery, P. G. (2002). Teaching tolerance: what research tells us. Social Education, 66(5), 270–275.
Banks, J. A. (2008). Diversity, group identity, and citizenship in a global age. Educational Researcher, 37(3),
129–139.
Barton, K. C. (2008). Research on studens’ ideas about history. In L. Levstik, & C. Tyson (Eds.), Handbook
of research on social studies teaching and learning (pp. 239–258). Erlbaum Publishing.
Barton, K. C. (2011). Wars and rumors of war: Making sense of history education in the United States. In
T. Taylor & R. Guyver (Eds.), History wars in the classroom: Global perspectives (pp. 189–204). Charlotte,
NC: Information Age Publishing.
Barton, K. C., & Levstik, L. S. (2004). Teaching history for the common good. Mahway, NY: Lawerence
Erlbaum Associates, Inc.
Beeson, M. W., Journell, W., & Ayers, C. A. (2014). When using technology isn’t enough: A comparison of
high school civics teachers’ TPCK in one-to-one laptop environments. Journal of Social Studies Research,
38, 117–128.
Bourdillion, H. (1999). On the record: The importance of gender in teaching history. In Bourdillion, H.,
(Ed.), Teaching history (pp. 62–75). New Fetter Lane, London: Routledge.
Boger, J. C., & Orfield, G. (Eds.). (2005). School resegregation: Must the South turn back? Chapel Hill, NC:
The University of North Carolina Press.
Borman, G. D., Stringfield, S., & Rachuba, L. (2000). High achievement: National trends and promising
programs and practices. John Hopkins University Center for Social Organization of Schools, A Report
Prepared for the National Task Force on Minority High Achievement: The College Board.
Boykin, A. W., & Noguera, P. (2011). Creating the opportunity to learn. ASCD: Alexandria, VA.
Brophy, J. (2006). Graham Nuthall and social constructivist teaching: Research-based cautions and quali-
fications. Teaching and Teacher Education, 22(5), 529–537.
Brown, K. D., & Brown, A. L. (2012). Useful and dangerous discourse: Deconstructing racialized knowledge
about African-American students. Educational Foundations, 26(1–2), 11–26.
Camburn, E. M., & Won Han, S. (2011). Two decades of generalizable evidence on US instruction from
national surveys. Teachers College Record, 113(3), 561–610.
Castania, K. (2003). Diversity: The evolving language of diversity. Resource Center, Cornell University.
Ithaca, NY.
Cawthon, S. W., Beretvas, S. N., Kaye, A. D., & Lockhart, L. L. (2012). Factor structure of opportunity to learn
for students with and without disabilities. Educational Policy Analysis Archives, 20(41), 1–30.
Cawthon, S. W., Kaye, A. D., Lockhart, L. L., & Beretvas, S. N. (2012). Effects of linguistic complexity
and accommodations on estimates of ability for students with learning disabilities. Journal of School
Psychology, 50(3), 293–316.
Carroll, J. B. (1963). A model for school learning. Teachers College Record, 64, 723–733.
Chikkatur, A. (2013). Teaching and learning African American history in a multicultural classroom. Theory
and Research in Social Education, 41(4), 514–534.
Cole-Henderson, B. (2000). Organizational characteristics of schools that successfully serve low-income
urban African-American students. Journal of Education for Students Placed at Risk, 5(1–2), 77–91.
Coleman, J. S. (1966). The concept of equality of educational opportunity. Washington, DC: US Office
of Education.
Commission on Chapter 1. (1992). Making schools work for children in poverty. Washington, DC: American
Association for Higher Education. (ERIC Document Reproduction Service No. ED362618).
Crocco, M. S. (2008). Gender and sexuality in the social studies. In L. Levstik, & C. Tyson (Eds.), Handbook
of research on social studies teaching and learning (pp. 172–196). Erlbaum Publishing.
Crocco, M. S., & Costigan, A. T. (2007). The narrowing of curriculum and pedagogy in the age of account-
ability urban educators speak out. Urban Education, 42(6), 512–535.
Cruz, B. C., & Thornton, S. J. (2009). Teaching social studies to English language learners: Teaching English
language learners across the curriculum. New York, NY: Routledge.
Cummins, J. (2000). Language, power, and pedagogy: Bilingual children in the crossfire. London, UK:
Cambrian Printers Ltd.
Darling-Hammond, L. (2010). The flat world and education: How America’s commitment to equity will
determine our future. New York, NY: Teacher’s College Press.
Delli Caprini, M. X., & Keeter, S. (1989). What Americans know about politics and why it matters. New
Haven, CT.: Yale University Press.
Delpit, L. (2006). Other people’s children: Cultural conflict in the classroom. New York: The New Press.
Eccles, J., Barber, B., Jaozefowicz, D., Malanchuk, O., & Vida, M. (1999). Self-evaluations of competence,
task values, and self-esteem. In N. G. Johnson, M. C. Robers, and J. Worell (Eds.), Beyond Appearance:
A New Look at Adolescent Girls (pp. 53–83). Washington, DC: American Psychological Association.
246

Epstein, T. (2009). Interpreting national history. New York, NY: Routledge.
Epstein, T., Mayorga, E., & Nelson, J. (2011). Teaching about race in an urban history class: The effects
of culturally responsive teaching. The Journal of Social Studes Research, 35(1), 2–21.
Evans, R. (2004). The social studies war: What should we teach the children? New York: Teachers College Press.
Fallace, T. (2010). Exploring three orientations to the social studies. In E. E. Heilman (Ed.), Social studies
and diversity education: What we do and why we do it (pp. 23–25). New York, NY: Routledge.
Fitchett, P. G., & Heafner, T. L. (2010). A National Perspective on the Effects of High-Stakes Testing and
Standardization on Elementary Social Studies Marginalization. Theory & Research in Social Education,
38(1), 114–130.
Fitchett, P. G., & Heafner, T. L. (2013). Making critical connections between social studies teaching and
student achievement using NAEP Data Explorer. The Teacher Educator, 48(4), 296–310.
Fitchett, P. G., Heafner, T. L., & Lambert, R. (2014). Examining social studies marginalization: A multilevel
analysis. Educational Policy, 28(1), 40–68.
Fitchett, P. G., Starker, T. V., & Salyers, B. (2012). Examining culturally responsive teaching self-efficacy in
a preservice social studies education course. Urban Education, 47(3), 585–611.
Fitchett, P. G. & VanFossen (2013). Survey on the Status of Social Studies: Development and analysis. Social
Studies Research and Practice, 8(1), 1–23. Retrieved from: http://www.socstrpr.org/wp-content/uploads/
2013/04/MS_06462_no1.pdf
Frederickson, M. (2004). Surveying gender: Another look at the way we teach United States history. The
History Teacher, 38(4), 476–484.
Freeman, Y., Freeman, D., & Mercuri, S. (2002). Closing the achievement gap: How to reach limited formal-
schooling and long-term English langauge learners. Portsomouth: Heinemann.
Gaudelli, W. (2002). US kids don’t know US history: The NAEP study, perspectives, and presuppositions.
The Social Studies, 93(5), 197–201.
Gay, G. (2000). Culturally responsive teaching: Theory, research, and practice. New York, NY: Teachers
College Press.
Gay, G. (2002). Preparing for culturally responsive teaching. Journal of Teacher Education, 53(2), 106–116.
Gerwin, D., & Visone, F. (2006). The freedom to teach: Contrasting teaching in elective and state-tested
courses. Theory & Research in Social Education, 34(2), 259–282.
Gilligan, C. (1990). Making connections: The relational world of adolescent girst at the Emma Willard
School. Cambridge, MA: Harvard University Press.
Goldhaber, D. D., Brewer, D. J., & Anderson, D. J. (1999). A three-way error components analysis of edu-
cational pro- ductivity. Education Economics, 7(3), 199–208.
Gordon, B. M. (2012). “Give a brotha a break!”: The experiences and dilemmas of middle- class African
American male students in White suburban schools. Teachers College Record, 114, 1–26.
Gradwell, J. M. (2006). Teaching in spite of rather than because of, the test. In S. G. Grant (Ed.), Measuring
history: Cases of state-level testing across states (pp. 157–176). Greenwich, CT: Information Age Publishing.
Grant, S. G. (2001). Its just the facts, or is it? The relationship between teachers’ practices and students’
understandings of history. Theory & Research in Social Education, 29(1), 65–108.
Grant, S. G. (2003). History lessons: Teaching, learning, and testing in US high school classrooms. Mahway,
NJ: Lawrence Erlbaum Associates
Guthrie, J. T., Klauda, S. L., & Ho, A. N. (2012). Modeling the relationships among reading instruction,
motivation, engagement, and achievement for adolescents. Reading Research Quarterly, 48(1), 9–26.
Heafner, T. L. (2004). Using technology to motivate students to learn social studies. Contemporary Issues
in Technology and Teacher Education, 4(1), 42–53.
Heafner, T. L., & Friedman, A. M. (2008). Wikis and constructivism in social studies: Fostering a deeper
understanding. Computers in the Schools, 25(3–4), 288–302.
Henke, R. R., Chen, X., & Goldman, G. (1999). What happens in classroms? Instructional practices in ele-
mentary and secondary schools, 1994–95. Washington, DC: US Department of Education, National Center
for Educational Statistics.
Henry, M. (1993). A fifty-year perspective on history teaching’s “crisis.” OAH Magizine of History, 7(3),
5–8.
Hess, D. (2009). Controversy in the classroom: The democratic power of discussion. New York: Routledge.
Hicks, D., Doolittle, P., & Lee, J. K. (2004). Social studies teachers’ use of classroom-based and web-based
historical primary sources. Theory and Research in Social Education, 32(2), 213–247.
House, E. R., & Nancy, L. (1990). Report on the Content Definition Process in Social Studies Testing (CSE
Technical Report 310). Los Angeles, CA: UCLA Center for Research on Evaluation, Standards, and
Student Testing.
Hox, J. (2010). Multilevel analysis: Techniques and applications. Philadelphia, PA: Taylor and Francis.
Kawashima-Ginsberg, K. (2013). Do discussion, debate, and simulations boost NAEP civics performance?
Medford, MA: The Center for Information and Research on Civic Learning and Engagement.
Konstantopouslos, S., & Borman, G. (2011). Family background and school effects on student achieve-
ment: A multilevel analysis of the Coleman data. Teachers College Record, 113(1), 97–132.
Kozol, J. (2005). Shame of a nation: The restoration of apartheid schooling in America. New York, NY:
Crown Books.
Kurz, A., Elliott, S. N., Lemons, C. J., Zigmond, N., Kloo, A., & Kettler, R. J. (2014). Assessing opportunity-
to-learn for students with disabilities in general and special education classes. Assessment for Effective
Intervention, 40, 24–39.
Ladson-Billings, G. (2009). The dreamkeepers: Successful teachers of African American children (2nd
ed.).
San Francisco: Jossey Bass.
247

Ladson-Billings, G. (1995). Toward a theory of culturally relevant pedagogy. American Educational
Research Journal, 32(3), 465–491.
Landsman, J. G., & Lewis, C. W., Eds. (2011) White teachers/Diverse classrooms: Creating inclusive schools,
building on students’ diversity, and providing true educational equity. Sterling, VA: Stylus Publishing, LLC.
Lee, P., & Shemilt, D. (2003). A scaffold, not a cage: Progression and progression models in history. Teaching
History, 113, 13–24.
Leming, J. S. (1991). Teacher characeristics and social studies education. In J. P. Shaver (Ed.), Handbook
of research on social studies teaching and learning (pp. 222–236). New York, NY: MacMillan.
Levinson, M. (2012). No citizen left behind. Cambridge, MA: Harvard University Press.
Levstik, L. S. (2008). What happens in social studies classrooms? Research on K-12 social studies practice.
In L. S. Levstik & C. A. Tyson (Eds.), Handbook of research in social studies education (pp. 50–62).
New York, NY: Routledge.
Li, D., Oranje, A., & Yanlin, J. (2009). On the estimation of hierarchial latent regression models for large-
scale assessments. Journal of Educational and Behavioral Statistics, 34(4), 433–463.
Litner, T., & Schweder, W. (2008). Social studies in special education classrooms: A glimpse behind the
door. Journal of Social Studies Research, 32(1), 3–9.
Maggioni, L., VanSledright, B., & Alexander, P. A. (2009). Walking on borders: A measure of epistemic cog-
nition in history. The Journal of Experimental Education, 77(3), 187–214.
Malloy, C. L., & Wohlstetter, P. (2003). Working conditions in charter schools: What’s the appeal for
teachers? Education and Urban Society, 35(2), 219–241.
Marcus, A. S., Levine, T. H., & Grenier, R. S. (2012). How secondary history teachers use and think about
museums: Current practices and untapped promise for promoting historical understanding. Theory &
Research in Social Education, 40(1), 66–97.
Martell, C. C. (2013). Race and histories: Examining culturally relevant teaching in the U.S. history class-
room. Theory and Research in Social Education, 41(1), 65–88.
Mickelson, R. A. (2001). Subverting Swann: First- and second-generation segregation in the Charlotte-
Mecklenburg Schools. American Educational Research Journal, 38(2), 215–252.
Monte-Sano, C. (2012). Building skills by doing history. Phi Delta Kappan, 94(3), 62–65.
Monte-Sano, C., & De La Paz, S. (2012). Using writing tasks to elicit adolescents’ historical reasoning.
Journal of Literacy Research, 44(3), 273–299.
Morris, J. E., & Monroe, C. R. (2009). Why study the US south? The nexus of race and place in investigating
Black student achievement. Educational Researcher, 38(21), 21–36.
Muller, C., Riegle-Crumb, C., Schiller, K. S., Wilkinson, L., & Frank, K. A. (2010). Race and academic
achievement in racially diverse high schools: Opportunity and stratification. Teachers College Record,
112(4), 1038–1063.
National Assessment Governing Board. (2010). US History Framework for 2006 National Assessment of
Educational Progress. Washington, DC: US Department of Education.
National Center for Education Statistics. (2011). The Nation’s Report Card: US History 2010. Washington,
DC: Institute of Educational Sciences, Department of Education.
Nichols, S. L., Glass, G. V., & Berliner, D. C. (2012). High-stakes testing and student achievement:
Updated analyses with NAEP data. Educational Policy Analysis Archives, 20(20), 1–30. Retrieved Feb-
ruary 21, 2013, from http://epaa.asu.edu/ojs/article/view/1048.
Noguera, P. A. (2003). The trouble with black boys: The role and influence of environmental and cultural
factors on the academic performance of African American males. Urban Education, 38(4), 431–459.
Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation
and Policy Analysis, 26(3), 237–257.
Ogbu, J. (2003). Black American students in an affluent suburb: A study of academic disengagement. Mahwah,
NJ: Erlbaum.
Olson, L. (2006, June 22). The down staircase. Diplomas count: Education Week, 25(41S), 5–6, 10–11.
Pace, J. (2011). The complex and unequal impact of high stakes accountability on untested social studies.
Theory & Research in Social Education, 39(1), 32–60.
Raudenbush, S., Bryk, A., Cheong, Y. F., Congdon, R., & du Toit, M. (2004). HLM 6: Hierarchical linear and
nonlinear modeling. Lincolnwood, IL: Scientific Software International.
Raudenbush, S., & Bryk, A. (1986). A hierarchical model for studying school effects. Sociology of Education,
59(1), 1–17.
Ravitch, D. (2010). The death and life of the great American school system: How testing and choice are
undermining education. New York, NY: Basic Books.
Ravitch, D., & Finn, C. (1987). What do our 17-year olds know? A report on the National Assessment of
History and Literature. New York: Harper Collins.
Reardon, S. F. (2011). The widening of the academic achievement between rich and poor: New evidence
and possible explanations. In G. J. Duncan & R. Murnane (Eds.), Whiter Opportunity? Rising Inequality
in schools, and children’s life chances (pp. 91–116). New York, NY: Russell Sage Foundation.
Reich, G. A. (2009). Testing historical knowledge: Standards, multiple-choice questions, and student reasoning.
Theory & Research in Social Education, 37(3), 325–360.
Reich, G. A. (2011). Testing collective memory: Representing the soviet union on multiple-choice questions.
Journal of Curriculum Studies, 43(4), 507–532.
Reisman, A. (2012). Reading like a historian: A document-based history curriculum intervention in urban
high schools. Cognition and Instruction, 30(1), 86–112.
Reisman, A., & Wineburg, S. (2008). Teaching the skill of contextualization in history. The Social Studies,
99(5), 202–207.
248

Roach, R. (2004). The great divide: Racial achievement gap gains recognition as national concern, but Solution
continues to elude educators, scholars and policy-makers. Black Issues in Higher Education, 21(1), 1–4.
Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data.
American Economic Review, 94(2), 247–252.
Ross, E. W. (2006). The struggle for the social studies curriculum. In E. W. Ross (Ed.), The social studies
curriculum: Purposes, problems, and possibilities (Third ed., pp. 17–36). Albany, NY: SUNY.
Rothstein, R. (2004). We are not ready to assess history performance. The Journal of American History, 90(4),
1381–1391.
Salinas, C. (2006). Educating late arrival high school immigrant students: A call for a more democratic cur-
riculum. Multicultural Perspectives, 8(1), 20–27.
Saye, J., & The Social Studies Inquiry Research Collaborative. (2013). Authentic pedagogy: Its presence
in social studies classrooms and relationship to student performance on state-mandated tests. Theory &
Research in Social Education, 41(1), 89–132.
Schleppegrell, M. J., Achugar, M., & Oteiza, T. (2004). The grammar of history: Enhancing content-based
instruction through functional focus on language. TESOL Quarterly, 28(1), 67–93.
Shaver, J. P., Davis Jr., O. L., & Helburn, S. W. (1979). The status of social studies education: Impressions
from three NSF studies. Social Education, 43(2), 150–153.
Smith, J., & Niemi, R. G. (2001). Learning history in school: The impact of course work and instructional
practices on achievement. Theory & Research in Social Education, 29(1), 18–42.
Stearns, P. N., Siexas, P., & Wineburg, S. (Eds.). (2000). Knowing, teaching and learning history: National
and international perspectives. New York, NY: New York University Press.
Stedman, L. C. (2009). The NAEP long-term trend assessment: A review of its transformation, use, and
findings. National Assessment Governing Board 1988–2008 Report.
Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and performance.
American Psychologist, 52(6), 613–629.
Stevens, F. I. (1993a). Applying an opportunity-to-learn conceptual framework to the investigation of the
effects of teaching practices via secondary analysis of multiple- case-study summary data. The Journal
of Negro Education, 62(3), 232–248.
Stevens, F. I. (1993b). Opportunity to learn: Issues of equity for poor and minority students. Washington,
DC: National Center for Education Statistics.
Stevens, F. I. (1996). The need to expand the opportunity to learn conceptual framework: Should students,
parents, and school resources be included? New York, NY: American Education Research Association.
Stoddard, J. D. (2013). Hillary: The movie, the History Channel, and the challenge of the documentary for
democratic education. Teachers College Record, 115(3), 1–32.
Stoddard, J. D. & Marcus, A. S. (2010). More than “showing what happened”: Exploring the potential of
teaching history with film. The High School Journal, 93(2), 83–90.
The Bradley Commission on History in Schools. (1989). Building a history curriculum: Guidelines for
teaching history in schools. The History Teacher, 23(1), 7–35.
Thornton, S. J. (2005). Teaching social studies that matters. New York: Teachers College Press.
Thornton, S. J., & Barton, K. C. (2010). Can history stand alone? Drawbacks and blind spots of a “disciplinary”
curriculum. Teachers College Record, 112(9), 2471–2495.
Ukpokodu, O. (2006). Essential characteristics of a culturally conscientious classroom. Social Studies
and the Young Learner, 19(2), 4–7.
VanFossen, P. J., & Waterson, R. A. (2008). “It is just easier to do what you did before…”: An update
on Internet use in secondary social studies classrooms in Indiana. Theory and Research in Social
Education, 36(2), 124–152.
VanSledright, B. (2011). The challenge of rethinking history education: On practices, theories, and policy.
New York, NY: Routledge.
Venkateswaran, U. (2004). Race and gender issues on the AP United States History examination. The History
Teacher, 27(4), 501–512.
Wang, J. (1998). Opportunity to learn: The impacts and policy implications. Educational Evaluation and
Policy Analysis. 20(3), 137–156.
Williams, J. A. (2001). Classroom conversations: Opportunities to learn for ESL students in mainstream
classrooms. Reading Teacher, 54(8), 750–757.
Wills, J. S., & Sandholtz, J. (2009). Constrained professionalism: Dilemmas of teaching in the face of test-
based accountability. Teachers College Record, 111(4), 1065–1114.
Wineburg, S. (2001). Historical thinking and other unnatural acts. Philadelphia, PA: Temple University Press.
Wineburg, S. (2004). Crazy for history. The Journal of American History, 90(4), 1401–1414.
Zwick, R., & Ercikan, K. (1989). Analysis of differential item functioning in the NAEP history assessment.
Journal of Educational Measurement, 26(1), 55–66.
249

Copyright of High School Journal is the property of University of North Carolina Press and
its content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email
articles for individual use.

NAEP Data Suggests Opportunity Gap in Learning US History

Recommended

Recommended

More Related Content

Similar to NAEP Data Suggests Opportunity Gap in Learning US History

Similar to NAEP Data Suggests Opportunity Gap in Learning US History (20)

More from Brandi Gonzales

More from Brandi Gonzales (20)

Recently uploaded

Recently uploaded (20)

NAEP Data Suggests Opportunity Gap in Learning US History