2. ple with disabilities?” will depend on how “disability” is first
formulated. Making explicit the social, political, historical, and/
or cultural accounts of “disability” can reveal the hermeneutic
circle in which a research synthesis is involved and suggest pos-
sibilities for including alternative accounts of constructs.
Although many narrative (Petticrew & Roberts, 2006) and sys-
tematic review methods—such as metanarrative (Greenhalgh,
Macfarlane, Bate, Kyriakidou, & Peacock, 2005), metastudy
(Paterson, Thorne, Canam, & Jillings, 2001), and critical
interpre-
tive synthesis (CIS; Dixon-Woods et al., 2006)—can be drawn
upon to critique constructs, none fully answers our call, through
a
systematic process, to understand how constructs and
methodologi-
cal elimination decisions frame the results of research
syntheses. In
Table 1, we clarify research synthesis as our object of critique.
We
characterize and differentiate research syntheses, such as meta-
analysis and best-evidence synthesis, from other systematic
reviews
that do not include methodological elimination as an integral
part
of their study screening process. We also show how neither
tradi-
tional nor qualitative systematic review methods include a
system-
atic process to critically analyze constructs in existing research
syntheses. Interrogating constructs in research synthesis is
needed
(we argue) in the field of education, where governments and
educa-
tionalists privilege these syntheses to answer questions about
3. what
703946 EDRXXX10.3102/0013189X17703946Educational
ResearcherEducational Researcher
research-article2017
1University of South Florida, Tampa, FL
2University of Kansas, Lawrence, KS
Unpacking Assumptions in Research Synthesis:
A Critical Construct Synthesis Approach
Jennifer R. Wolgemuth1, Tyler Hicks2, and Vonzell Agosto1
Research syntheses in education, particularly meta-analyses and
best-evidence syntheses, identify evidence-based
practices by combining findings across studies whose constructs
are similar enough to warrant comparison. Yet constructs
come preloaded with social, historical, political, and cultural
assumptions that anticipate how research problems
are framed and solutions formulated. The information research
syntheses provide is therefore incomplete when the
assumptions underlying constructs are not critically understood.
We describe and demonstrate a new systematic review
method, critical construct synthesis (CCS), to unpack
assumptions in research synthesis and to show how other
framings
of educational problems are made possible when the constructs
excluded through methodological elimination decisions
are taken into consideration.
Keywords: critical theory; educational policy; disability studies;
meta-analysis; qualitative research
FEATURE ARTIClES
http://edr.aera.net
4. http://doi.org/10.3102/0013189X17703946
http://crossmark.crossref.org/dialog/?doi=10.3102%2F0013189
X17703946&domain=pdf&date_stamp=2017-04-04
132 EDUCATIONAl RESEARCHER
works in policy and practice (Donmoyer, 2012; National
Research
Council, 2002). To support dialogue and decision making that is
more fully informed, contextualized, and critical, inquiry must
also
examine the social, historical, political, and/or cultural
assumptions
underlying constructs.
In this article we describe and demonstrate a new systematic
review method, critical construct synthesis (hereafter CCS). A
CCS
explores and critiques constructs included and excluded from
research synthesis, particularly through the process of screening
out
studies that fail to meet methodological standards for providing
best
or quality evidence. It shows how reexamining research
syntheses in
light of constructs identified in methodologically excluded
literature
may open up possibilities for reframing educational problems.
Exclusion, Constructs, and Critique in
Research Syntheses
In broad terms, the aim of any research synthesis is to
summarize
5. and evaluate research and knowledge on a topic. With the intro-
duction of meta-analysis, research syntheses became tools for
Table 1
Types and Characteristics of Literature Reviews
Type of Literature
Review Description
Systematic Process
for Reviewing
Primary Literature
Includes All
Primary
Literature
Includes Critical
Analysis of
Constructs
Critiques
Existing Research
Synthesis
Traditional review A review that provides an overview of
literature on a topic. Does not use
systematic review methods.
No Yes No No
Critical review A review of literature that critically
examines primary literature.
6. Does not use systematic review
methods.
No Yes Yes No
Systematic review
Research synthesis:
Best-evidence
synthesis
A systematic review that synthesizes
only studies that meet
predetermined methodological
standards
Yes No No No
Research synthesis:
Meta-analysis
A systematic review that summarizes
studies through statistical
comparison of findings
Yes Noa No No
Research synthesis:
Narrative review
A systematic review that summarizes
studies narratively, rather than by
meta-analysis
Yes Noa No No
7. Metanarrative A systematic review that seeks to
show various ways researchers
have understood a heterogeneous
topic area, often over time.
Yes Yes No No
Metastudy A systematic review that aims
to generate new insights into
phenomena through an analysis
of theory, methods, and finding of
qualitative research.
Yes Nob Yes No
Critical interpretive
synthesis
A systematic review that aims to
develop an interpretive model
of a phenomenon from existing
literature.
Yes Noc Yes No
Critical construct
synthesis
A systematic review that aims
to critique the constructs in
literature included and excluded
from an existing research
synthesis.
Yes Yes Yes Yes
8. Note. Literature reviews relevant to the object of critique and
methods in this article. For different and more exhaustive
typologies of literature views, see, for example,
Petticrew and Roberts (2009); Gough, Thomas, and Oliver
(2012); and Kastner, Antonya, Soohiaha, Strausa, and Triccoa
(2016).
aSome meta-analyses and narrative reviews do not exclude
quantitative studies based on the quality of their designs
(Cooper & Hedges, 2009). They do, however, exclude
qualitative and conceptual literature.
bMetastudy does not exclude qualitative studies based on the
quality of their designs (Paterson, Thorne, Canam, & Jillings,
2001). They do, however, exclude quantitative
and conceptual literature.
cCritical interpretive synthesis includes quantitative and
qualitative studies but excludes those deemed “fatally flawed”
(Dixon-Woods et al., 2006).
ApRIl 2017 133
quantifying an intervention’s effect by pooling estimates across
stud-
ies (Glass, 1976). As a quality control measure, reviews often
restrict
research syntheses to studies passing exacting standards of
method-
ological rigor. The U.S. Department of Education’s What Works
Clearinghouse (WWC; 2014), for example, reviews only
well-designed quasi-experimental studies, single-case designs,
and
randomized experiments. Yet, excluding studies not up to
method-
ological par conflicts with the idea of reviewing the full
knowledge
9. base. Glass (2000), for one, has remained “staunchly committed
to
the idea that meta-analyses must deal with all studies, good bad
and
indifferent” (para. 33). Restricting research syntheses to studies
that
meet methodological standards may be an effective tool for
estab-
lishing evidence-based practices, but choosing only well-
designed
studies does not inoculate research syntheses from the influence
of
constructs and the assumptions they carry.
Under previous positivist notions of social science inquiry, con-
structs, thought of as latent variables, were operationally
defined in
order to sanitize them of tacit assumptions (Phillips & Burbules,
2000). Such treatment rested on a hard distinction between
theoreti-
cal constructs, which were loaded with assumptions, and
empirical
constructs, which were taken to be assumption free. But the
distinc-
tion between these two types of constructs failed to hold after
Kuhn
(1962) showed that scientific communities work within
prevailing
frameworks (paradigms), which are central to empirical claims.
Despite advantages that may accrue from operationally defining
constructs in the process of a research synthesis, doing so does
not
isolate constructs from their social, political, historical, and
cultural
contexts. The identification of underlying assumptions in the
research synthesis process is not a warrant for “anything-goes”
10. rela-
tivism or a call to increase objectivity. Rather it supports the
position
that methodological decisions about what research designs to
include
in a research synthesis, and constructs that inform and emanate
from
those decisions, ought to be unpacked and subject to critique.
The recognition and use of critique in reviews is not new.
Cooper
(1985) noted early on that the purpose of conducting a review
could
be critical to show how previous conclusions were unwarranted
based on “the literature’s incommensurability with the
reviewers’
theoretical stance and/or criteria for methodological validity”
(p.
10). Similarly, Petticrew and Roberts (2006) described critical
reviews as aiming to critique methods and results of primary
litera-
ture but without “using the formalized approach of a systematic
review” (p. 41).
A host of qualitative systematic review methods now employ
more formalized approaches to critique primary literature in the
review process. For example, the metasynthesis phase of
metastudy
explores how various theoretically informed analytic options
influ-
ence research findings (Paterson et al., 2001). Dixon-Woods and
colleagues (2006) also characterize CIS as not just summarizing
the
literature’s data but also tracing the sociopolitical origins of
entrenched constructs. However, CCS stands out from these sys-
tematic review methods because it takes research synthesis as
11. its
object of critique and shows how research synthesis results are
sensi-
tive to constructs in the literature it includes.
Introducing CCS
Before demonstrating our use of CCS, we describe it in terms
of its philosophy of inquiry, methodology, and methods.
Philosophies of inquiry are attempts to provide consistent
answers
to a constellation of questions, such as What is the notion of
inquiry? What should count as evidence in educational inquiry?
(Biesta, 2015). Methodology delineates what the inquiry entails
and provides justifications for how it will be conducted within
the philosophies of inquiry it is embedded. Methods describes
the procedures scholars follow while inquiring within the
overall
methodology.
Philosophy of Inquiry
We conceptualize CCS as philosophically dexterous,
transferable
among different paradigms of inquiry within the critical tradi-
tion. By critique, we mean a sociopolitical analysis that
highlights
how power and ideology operate to structure and stratify
society, to
marginalize, oppress, and limit possibility. Below we illustrate
this
dexterity with two stances: critical realism (Bhaskar, 1998)
(informed
by Marxism) and poststructuralism (e.g., Foucault, 1980).
Although
12. these stances diverge in many respects, they can be situated
within
the critical tradition and thus supply the intellectual resources
needed for CCS.
Critical realism. Realism, including critical realism, holds that
the deep structure of reality is external to human cognition
(Gorski, 2013; House, 1991). Although reality may be multi-
layered (Bhaskar, 1998), realism values constructs that map
onto
external reality (Sider, 2014). The construct “human being,” for
example, may directly map onto reality’s structure (Armstrong,
1978), and if so, any full cognition of reality would require the
inclusion of that construct to be adequate. It would not be a
proper object of criticism. If “human being” does not map onto
reality, however, then it can be the object of criticism and aban-
doned, revised, or refined to meet social needs. Constructs such
as “disability” may fit this latter category (Searle, 1997).
Empiri-
cal inquiry can determine if such constructs actually function to
legitimate social oppression and so neutralize them. A dosage of
criticality is thus injected into realism when advocates of
realism
recognize the need to unmask hegemonic constructs wrongly
assumed to map onto reality, hence, critical realism (Bhaskar,
1988, 1993).
Following Ian Hacking (2002, 2004), critical realism pos-
its a dynamic relationship between reality and dominant ide-
ology, which creates a looping effect. For instance, when
researchers classify a subset of schoolchildren as “high achiev-
ers,” this dynamic relationship is reflected when schoolchil-
dren accept, resist, or redefine this identity option. Their
response can reinforce or discourage researchers from using
the construct or adapting it. The “high achievers” construct
thus evolves due to a feedback loop between labelers and
13. labeled. Looping makes vocabulary in education different
from vocabulary in natural science, wherein no such interac-
tion between labeling and labeler exists (e.g., rocks do not
contest or admire geologists’ labels of them). A critical real-
ist–informed CCS might evaluate the impact of looping
effects on research syntheses to provide emancipatory expla-
nations of findings. For example, one might wonder if the
suspect construct of “disability” harmfully delimited the pos-
sible findings of a research synthesis.
134 EDUCATIONAl RESEARCHER
Poststructuralism. In contrast to critical realism, poststructural-
ism holds that reality is entirely ideological. Poststructuralism
engages boundaries between prevailing ideologies and sees real-
ity as blurred (Peters & Burbules, 2000), one can never step
outside of ideology to “objectively” define reality according
to neutral facts. No external or objective structure (like the
looping effect) exists to distinguish between what is real and
what is defined by a construct. A poststructuralist analysis, for
example, would critique the “looping effect” as producing a
distinction between “labeler” and “labeled,” rather than reflect-
ing a “real” process.
Drawing on Foucault’s (1980) description of power and
knowledge (power/knowledge), a poststructuralist analysis for-
mulates ideology as laden with power. The role of power in
lever-
aging some ideologies above others (e.g., neoliberalism) takes
many forms and is often masked in liberal democracies (Giroux,
2004). But the incessant flux and productive work of power
means no ideology ultimately prevails over the others or even
remains stable over the long haul (Foucault, 1980). Ideological
disagreements are not resolvable by evidence but through
14. power.
A poststructuralist-informed CCS might seek to disrupt prevail-
ing ideologies in research syntheses by showing that the con-
structs studied are not inevitable, natural, or immutable but
instead contestable as they manifest through fallible social and
historical discourse.
Methodology
CCS draws on two types of well-established methodologies in
qualitative research. The first methodology is the qualitative
sys-
tematic review (Hannes & Macaitis, 2012), most closely resem-
bling the processes of a CIS (Dixon-Woods et al., 2006). CIS
involves an iterative and reflexive approach to synthesizing a
body of literature on a topic (Dixon-Woods et al., 2006). Our
CCS, although similar in approach, differs from a CIS in its
aim.
The aim of CCS is not to generate an interpretive synthesizing
argument across the literature as is done in a CIS but to under-
stand how the results of research syntheses are sensitive to the
constructs included and excluded from review.
The second methodology is any “construct analysis” approach
to analyze latent meaning in text. For example, constructs might
be analyzed using techniques from ethnographic content analy-
sis (Altheide & Schneider, 2013), semiotic analysis (e.g.,
Barthes,
1964), and/or (critical) discourse analysis (e.g., Gee, 2010).
These analytic techniques assist researchers in interrogating
con-
structs and their assumptions in research syntheses and stem
from the broad swath of the critical tradition.
Method
15. CCS involves multiple steps: (1) Identify a research synthesis of
interest. (2) Focus on a bundle of constructs encompassed in the
topic of interest (e.g., kinds of people, outcomes, interventions,
or contexts). (3) Replicate the synthesis’ literature search strat-
egy. (4) Screen gathered articles to eliminate works not on
topic.
The first four steps largely replicate the original synthesis,
whereas the next steps are unique to CCS: (5) Divide the
remainder into distinct sets, literature included and excluded
from the original synthesis. Importantly, the excluded set
should
include conceptual and empirical research originally deemed not
up to par. (6) Develop coding sheets that track key characteris-
tics of construct formulation (e.g., questions posed, key terms
defined, answers given). (7) Code both sets of literature,
included
and excluded. (8) Resolve any coding discrepancies. (9) Proceed
to analyze the constructs of interest in both sets of articles.
Comparison of the two sets can identify what assumptions
underlying the constructs informed the findings of the original
research synthesis and how findings were sensitive to them.
These steps define the method, but it need not be a linear pro-
cess. Researchers may cycle back and forth between the steps,
as
needed.
Demonstrating CCS
Below we demonstrate how we proceeded through eight steps of
CCS to explore the constructs “work” and “autism” in literature
on employment for youth with autism (Wolgemuth et al., 2016)
from a research synthesis on postschool transition for youth
with
disabilities (Cobb et al., 2013).
16. Step 1: Identify a Research Synthesis of Interest
The primary aim of CCS is to critically explore constructs in an
existing research synthesis. Therefore, in the first step, the
research team selects a research synthesis of interest. Reasons
for
selecting a particular synthesis will vary, but it is important that
the search strategy used in the original research synthesis be
pub-
lished or otherwise known so that it can be replicated.
We chose to consider a research synthesis based on the first
author’s participation in a U.S. Department of Education–
contracted systematic review of literature on postsecondary out-
comes for youth with disabilities (Cobb et al., 2013) that used
WWC guidelines (with some modifications) to screen primary
literature. The aim of the research synthesis was to identify
effec-
tive programs and strategies that support students with
disabilities
in the United States to transition from high school. Reflecting
on
the synthesis and its processes, the first author worried that of
the
738 studies passing the abstract screening process, only 16 met
WWC standards (with reservations). She wondered about
assump-
tions the constructs “disability,” “employment,” “independent
liv-
ing,” and “postsecondary education” advanced in the final
report
and what understandings might have been enabled in a more
inclusive review. To explore these questions, she brought
together
a team of researchers to study a subset of literature in the
research
17. synthesis: literature on autism and employment.
Step 2: Focus on Constructs That Encompass a
Topic of Interest
In Step 2, the review team determines the constructs it wants to
explore, including its rationale for doing so. The first author
noted that three of the 16 studies that met standards for inclu-
sion in the research synthesis were about work for people with
disabilities, but none was specifically about “work” for people
ApRIl 2017 135
with “autism,” although two included participants with autism
diagnoses. She wondered what possible understandings of work
and autism the overall report and its literature base privileged
and excluded.
For our CCS, we focused on the constructs of “work” (con-
ceptualized as both an intervention and an outcome construct in
the research synthesis) and “autism.” We chose to make these
constructs focal given a global increase in the prevalence of
autism diagnoses (Elsabbagh et al., 2012) and reported lower
employment and compensation of people with autism in the
United States as compared to other disability categories
(Wehman et al., 2014). We argued that although literature
examined the construction of autism in fiction literature (e.g.,
Hacking, 2009), its construction in the academic literature had
not been investigated.
Step 3: Replicate the Literature Search Strategy
In Step 3, the review team identifies the original search
strategy;
18. determines, based on the constructs it wishes to explore,
whether
terms need to be added and/or eliminated; and replicates the
search using the original databases.
The research synthesis reported the key terms and Boolean
operators used to search the literature: disability (e.g., autis*
OR
Asperger*), AND population (e.g., adolescent OR youth) AND
outcome (e.g., work OR job OR employment) AND program
(e.g.,
work experience OR supported employment) AND research
design
(e.g., RCT OR quasi-experimental). Because the original
synthe-
sis sought to synthesize literature for three postschool outcomes
(employment, higher education, and independent living) for all
youth with disabilities, we replicated the search for our CCS
using only terms that would yield literature on work for youth
with autism. We eliminated key terms in disability and outcome
and program that were not associated with autism and employ-
ment to generate an initial set of primary literature germane to
our review. That is, we removed disability key terms, such as
intellectual disability; program terms, such as independent
living;
and outcome terms, such as higher education.
Although the original synthesis had a broader scope with
regard to disability, intervention, and outcome, the methodolo-
gies it sought were only those with potential to meet WWC evi-
dence standards (e.g., single-case designs, quasi-experimental
designs, and randomized controlled trials). We therefore added
key terms in research design, whose original set included only
terms associated with quantitative studies, to capture conceptual
and qualitative work (e.g., qualitative, commentary). We ran our
literature search using the same databases (e.g., ERIC,
19. PsycINFO,
Medline) as the original review, which yielded 13,076 sources.
Also duplicating the original review, we limited our search to
only research published in peer-reviewed journals. We
recognize
this as a source of publication bias that is a limitation of the
original review and a delimitation to the scope of our critique.
Step 4: Screen Gathered Articles to Eliminate Works Off
Topic
In Step 4, the review team screens the articles identified in the
search strategy following the original review parameters and, if
applicable, narrowing those parameters based on the constructs
the team chose to explore.
We began screening the 13,076 articles by first identifying
only those that contained the words autism or Autism Spectrum
Disorder or ASD or Asperger’s Syndrome and work or
employment
or vocation or job or career either in their titles or abstracts. We
chose to conduct this initial screen in order to more efficiently
eliminate articles that were not of interest to our review and to
yield a more manageable set of article abstracts to screen for a
team of four researchers. This process left us with 2,738
articles.
Consistent with the original review, we then screened the
abstracts to identify articles that discussed work training or
employment that did or could occur during high school. Also
consistent, we defined (a) work as employment, vocation, or
par-
ticipation in paid or unpaid labor; (b) transition programs as
including career training, career therapy, or counseling; and (c)
youth as people between the ages of 13 and 22. We included lit-
erature about adults when it offered retrospective examinations
20. of their work experiences as youth. As is typical in this stage of
the review process, we erred on the side of inclusion. The
abstract
screen process resulted in 252 sources for which we conducted
a full-text screen. The full-text screen yielded 62 articles for
extraction.
Step 5: Develop an Extraction Pro Forma
In Step 5, the review team develops a pro forma to extract key
information from the articles and capture all statements in the
articles about the constructs of interest. Dixon-Woods and col-
leagues (2006) reported that the pro forma they developed for
their CIS was ultimately impractical to use, especially for large
documents, and wondered at the utility of formal data extraction
for interpretive syntheses. Given the focused nature of CCS on
specific constructs and the inclusion of only published articles
in
the original review, we felt a pro forma would help us identify
features of articles and specific sections for later analysis.
Our pro forma, summarized in Table 2, included summary
information about the article, including its purpose, methods,
population studied, findings, and conclusions. It also included
a section about methodology to enable us to analyze potential
connections between methodology and constructions of autism
and work. The remainder of the pro forma focused on the con-
structs of interest (autism, work, and the worker with autism).
We used the pro forma to record positive and negative state-
ments/definitions of autism, work, and the worker with autism.
By positive and negative, we did not mean “good” and “bad”
but
meant statements about what autism/work is (positive) and
what autism/work is not (negative). We also included spaces for
notes on our initial impressions of the constructs and other
observations.
21. Step 6: Use the Pro Forma to Extract Information
In Step 6, the research team uses the pro forma to extract rele-
vant information about the articles and statements about the
constructs of interest. This process will likely be iterative with
Step 5; that is, several team members can use the first draft of
the
pro forma to extract one study, come together and discuss their
136 EDUCATIONAl RESEARCHER
findings, identify commonalities, and potentially revise the pro
forma to better suit their research aims. Step 6 is also likely to
result in further elimination of articles based on the inclusion/
exclusion criteria in Step 4.
We used our pro forma to extract information from the 62
articles that passed the full-text screen. We extracted the first
eight articles as a team and met twice to share our extractions,
reconcile differences, and discuss the pro forma. These initial
extractions resulted in some revisions to the pro forma (e.g., we
clarified what we meant by positive and negative). Once we felt
confident in our process, we extracted the remaining 54 articles
in pairs. The pairs conducted their extractions individually and
then met to reconcile differences and create a final pro forma.
During this time, we met biweekly as a team to discuss our
prog-
ress and any questions or concerns. This process also resulted in
the elimination of an additional 45 articles that did not meet the
aims of the original review (e.g., employment studies conducted
in postsecondary, instead of secondary, settings). A final set of
17
primary sources was included in our CCS.
22. Step 7: Analyze the Pro Forma and Full Texts
Up to this point, we have described CCS as more or less system-
atic, adopting the steps and processes common to many system-
atic reviews. In Step 7, the research team conducts the analysis
of
the constructs and their underlying assumptions, using both the
pro forma and full texts. This includes identifying and compar-
ing articles that were and were not included in the original syn-
thesis. As noted above, this step is as variable as there exist
approaches for analyzing text and its latent meanings, and the
analysis technique selected will reflect the overarching philoso-
phy (e.g., critical realist, poststructural) and the aim of the
CCS.
Theoretical literature in the field (in our case, disability
studies)
is engaged to think through the analysis and interpret findings.
Reflexivity takes on a heightened importance during this step as
the team reflects on its own assumptions about the constructs
and the CCS process.
Our analysis involved a close reading and textual analysis of
the articles and extractions, seeking to understand how they
depicted autism, work, and the worker with autism. The philoso-
phy undergirding our CCS was broadly critical, and therefore
we
used several of Gee’s (2010) discourse analysis tools, including
the
significance-building tool and the identity-building tool, to cri-
tique what the articles featured and lessened as significant and
the identities they made possible and prevented. Working with
these tools, we categorized autism and work in a matrix with
two spectra: from simple to complex and from asset to deficit.
We developed this strategy based on Gee’s (2000) view that dis-
cursive “identities can be placed on a continuum in terms of
23. how active or passive one is in ‘recruiting’ them” (p. 104). In
our analysis, a simple construction ignored intersections of
identities (e.g., autism, class, race) in a unidimensional and
easy-to-follow narrative of autism and work. A complex con-
struction included multiple dimensions and intersections of
autism and work—depicting them in a more tentative, detailed,
and/or multiperspectival narrative. Deficit constructions
depicted autism or work in negative terms (e.g., students [situ-
ated] on the autism spectrum are lacking in social skills),
whereas asset constructions depicted autism or work more posi-
tively (e.g., students [situated] on the autism spectrum are good
visual learners).
Next, to understand what the constructions of autism and
work said about the worker with autism, we grouped articles
that
seemed to put forward similar constructions of autism and work
(e.g., simple deficit accounts of autism, simple asset accounts
of
work) and asked if there was a common story being built and, if
so, how. This process resulted in two major stories and variants.
The first, intervention story, identified autism as a problem for
which people on the autism spectrum needed treatment to ren-
der them useful as workers to society. Work in the intervention
story was usually presented as a set of discrete skills or tasks.
Complex stories, in contrast, invited positive accounts of autism
and broader notions of work that problematized the interven-
tion story. Connecting research methodologies to constructs, we
Table 2
Summary of Worker-With-Autism CCS Pro Forma
Extraction Area Information Extracted and Observations
Summary information Study code, citation, empirical or
nonempirical, peer-reviewed or practitioner journal, author’s
24. field/discipline/
occupation, population, major construct/theory investigated,
genre/design of study, purpose of the work/
study, nature of sample/group under discussion (total number,
age range, sex, ethnicity, education, other
characteristics), description of the research
approach/intervention (setting and work, design and
procedures),
major findings, conclusion, discussion, implications,
suggestions for future research
Methodology Methodologists, theory of method (e.g.,
postpositivist, interpretivist, pragmatist), role/voice of
researcher, role/voice
of participant
Construction of autism Positive statements/definitions of
autism, negative statements/definitions of autism, comments
about the way
autism is constructed
Construction of work Positive statements/definitions of work,
negative statements/definitions of work, comments about the
way work is
constructed
Construction of the worker with autism Positive
statements/definitions of the role of work for people with
autism at work, negative statements/definitions
of the role of work for people with at work, comments about the
role of work for the person with autism.
Other thoughts or concerns Concerns about the method or
writing, other thoughts
Note. CCS = critical construct synthesis. Full pro forma
25. available at
http://www.coedu.usf.edu/main/departments/me/documents/Wol
gemuth_CCS_ProForma_Webpage.pdf
ApRIl 2017 137
noted that only two of the 17 primary sources in our CCS met
standards for inclusion in the original research synthesis based
on their designs (quasi-experiment and single subject). Both
were coded as intervention stories, which discussed the impact
of
interventions on observable and/or measurable outcomes and
drew on behaviorist principles to conduct the interventions.
We read the articles and conducted the extractions individu-
ally and in pairs, and the process of coming to and interpreting
the two stories occurred during biweekly team meetings over
the
course of several months. During this time, we read widely
about
the social construction of autism, critiques of neoliberalism, and
discourses of social science. On the basis of our readings, and
consistent with our critical orientation, we interpreted the con-
structions of autism and work and stories in terms of broader
sociopolitical (e.g., neoliberal, disability) and academic (e.g.,
postpositivist) discourses. In particular, we relied on critical
dis-
ability literature that emphasized the social construction of dis-
ability alongside neoliberal accounts of work, literature that
critiques describing people with disabilities in terms of their
pro-
ductivity (e.g., McKenzie, 2013).
Reflexivity. Throughout the abstract screen, extraction, and
26. analysis processes and into the writing phase, we discussed and
reflected on our individual and collective assumptions about
work and autism. This reflection informed the aim of our
inquiry, how we approached our analysis, and the terms we
chose to use in our write-up. We became aware of the ways our
writing and talk constructed particular versions of autism and
work. Through reflective conversations, we became clear about
the constructions we wanted to privilege—critical, discursive
ones that would challenge essential, humanist, and neoliberal
accounts of disability and work. For example, we began our
conversations using “people-first” language (i.e., students with
autism) to place the person ahead of the disability. However,
we became concerned this humanist language was not aligned
with our readings of the disability studies literature that
theorize
autism as a social construction produced in systems of power.
We
wanted our conversations and write-up to suggest that autism is
a constructed condition that shapes presumptions and identities
about who one is/can become. Therefore, we adopted the phrase
(situated) on the autism spectrum to emphasize the constructive
power of language, relationships, and labels, such as autism.
Step 8: Write Up the CCS Report
The final step in CCS is the report write-up. Here researchers
decide on the structure, style, and voice of their report. Should
the manuscript be written more traditionally or experimentally?
Should it include or seek to downplay the research team’s per-
spectives and experiences conducting the review? Should it be
written in first person or third? As in most research dissemina-
tion deliberations, these decisions will be made in light of the
anticipated venue (e.g., brief report, academic journal) and
audi-
ence (e.g., policymakers, other researchers).
27. Our CCS write-up was first a conference paper presented at
the American Educational Research Association. Like many
syn-
thesis write-ups, it was a lengthy, including three tables
describ-
ing the features of all 17 primary sources and an appended pro
forma. Because we hoped to reach those who study autism and
might be open, if not sympathetic, to a critical approach, we
targeted a prominent disability studies journal for publication.
This meant our 60-page manuscript had to be cut in half. We
deleted most of the tables and the pro forma and made them
available as supplemental material; we also cut descriptions of
our methodology and worked elsewhere to streamline our intro-
duction, findings, and conclusions. The result was that the final
manuscript we submitted looked less like a traditional
systematic
review write-up, which often involves in-depth descriptions of
methods, and included primary literature (Petticrew & Roberts,
2006). Still, we chose to write up our CCS fairly
conventionally,
including introduction, methods, results, and discussion sec-
tions, to facilitate navigation. Whether this is the best way to
approach the CCS write-up is something we continue to wonder
about, especially in light of the connections CCS can illuminate
between the norms of academic writing and the kinds of under-
standings write-ups produce. In the conclusion section of our
CCS write-up, we recommended academics experiment with
writing in ways that engender less restrictive and more positive
accounts of work for youth situated on the autism spectrum. We
suggested CCS reviewers might also play with form and content
in their write-ups to attend to the constructions of constructs
they enable.
Discussion
28. Critically examining constructs in scholarly literature is
important
for understanding underlying assumptions about what counts as
good education and for whom. Empirical evidence derived from
experimental research can provide information about “what
works”
but always underdetermines the answer in the final analysis.
Practitioners, policymakers, and scholars need more than this
infor-
mation to make sense of what works in education (Donmoyer,
2012). We argue a broader understanding is needed, an
understand-
ing that is aware of the productive power of research, the
hermeneu-
tic circles in which research is produced, and the possibilities of
reframing educational problems and their solutions. We
introduce
CCS as a methodology for unpacking constructs in research
synthe-
ses. We do so to promote a “better” research synthesis, one that
does
not take constructs at face value and takes seriously the ways in
which review methodologies (inclusion and exclusion decisions)
construct and privilege some accounts over others.
Inclusionary and exclusionary methodological decisions are
grounded in disciplinary and sociopolitical assumptions about
what should count as “valid” research. Findings produced by
any
research synthesis are constituted within these assumptions, and
exploring excluded evidence reveals the implications of those
decisions, showing what might be found and known under a
different set of assumptions about what evidence counts. CCS
reveals the implications of these methodological elimination
decisions by comparing assumptions about constructs in
included and excluded literature. It asks, What methodologies
29. entail particular understandings at the exclusion of others?
What
might be thought, concluded, recommended differently? How
might problems and solutions be reframed?
In the example CCS, we noted that only two of our 17 pri-
mary sources were eligible for inclusion in the original research
138 EDUCATIONAl RESEARCHER
synthesis based on their designs (quasi-experiment and single
case). These were both coded in our CCS as intervention stories
that explored the impact of interventions on observable or mea-
surable outcomes and drew on behaviorism to design and con-
duct the interventions (Wolgemuth et al., 2016). The
intervention stories depicted autism in deficit terms and work
narrowly as a set of tasks for hourly pay. Absent from the
research
synthesis, and found in our CCS, were primary sources that dis-
cussed autism as a form of neurodiversity or in strengths-based
terms. Also absent from the research synthesis and found in our
CCS were primary sources that discussed work as an individual
right, a career, activism, or unpaid labor. We worry about the
limited and rather bleak understandings of people with disabili-
ties and their life possibilities enabled by a research synthesis
of
studies that met criteria for inclusion. We worry, alongside oth-
ers (cf. Van Cleave, 2012) critical of “scientifically based
research,” that “narrow definitions of research or science [in
research syntheses] might trivialize rather than enrich our
under-
standing of education policy and practice” (Feuer, Towne, &
Shavelson, 2002, p. 4).
30. The aim of CCS is not to demonstrate that research syntheses
should not be conducted or that all systematic reviews should be
as inclusive as possible. Instead, the aim is to empirically trace
and reveal the limitations of exclusionary decisions in order to
inform a more tentative, critical, and contextualized understand-
ing of the terms and conclusions produced in research
syntheses.
Constructs are indispensable to scholarly inquiry, but using
them without understanding both their history and the work
they do may produce unnecessarily limited understandings on
which to base policy and practice decisions. Through CCS, we
can better understand connections between research designs,
constructs we use, and their potential effects, with the aim
to reveal how research synthesis might not yield best ethics—
optimistic accounts of people that reframe their “problems” and
open up possibilities for their lives.
NOTE
1Following Cooper and Hedges (2009), we use the term
research
synthesis to refer to systematic reviews that “attempt to
integrate empiri-
cal [quantitative] research for the purpose of creating
generalizations”
(p. 6). We describe critical construct synthesis as a systematic
review method
particularly well suited to interrogate constructs in research
syntheses that
exclude primary literature based on methodological criteria.
REFERENCES
Altheide, D., & Schneider, C. J. (2013). Qualitative media
analysis (2nd
ed.). Newbury Park, CA: Sage.
31. Armstrong, D. M. (1978). Universals and scientific realism
(Vol. 1).
Nominalism and realism. Cambridge, MA: Cambridge
University
Press.
Barthes, R. (1964). Elements of semiology. New York, NY: Hill
& Wang.
Bhaskar, R. (1998). The possibility of naturalism: A
philosophical cri-
tique of the contemporary human sciences (3rd ed.). New York,
NY:
Routledge.
Bhaskar, R. (1993). Dialectic: The pulse of freedom. New York,
NY:
Verso.
Biesta, G. (2015). On the two cultures of educational research,
and how
we might move ahead: Reconsidering the ontology, axiology
and
praxeology of education. European Educational Research
Journal,
14(1), 11–22.
Cobb, R. B., Lipscomb, S., Wolgemuth, J. R., Schulte, T.,
Veliquette,
A., Alwell, M., . . . Weinberg, A. (2013). Improving
postsecond-
ary outcomes for transition-age students with disabilities: An
evidence
review (NCEE 2013-4011). Washington, DC: National Center
32. for Education Evaluation and Regional Assistance, Institute of
Education Sciences, U.S. Department of Education.
Cooper, H. M. (1985). A taxonomy of literature reviews. Paper
presented
at the Annual Meeting of the American Educational Research
Association, Chicago, IL. (ERIC Document Reproduction
Service
No. ED254541)
Cooper, H., & Hedges, L.V. (2009). Research synthesis as a
scientific
process. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.),
The
handbook of research synthesis and meta-analysis (pp. 3–16).
New
York, NY: Russell Sage Foundation.
Dixon-Woods, M., Cavers, D., Agarwal, S., Annandale, E.,
Arthur, A.,
Harvey, J., . . . Sutton, A. J. (2006). Conducting a critical
interpre-
tive synthesis of the literature on access to healthcare by
vulnerable
groups. BMC Medical Research Methodology, 6, 35.
Donmoyer, R. (2012). Can qualitative researchers answer
policymak-
ers’ what-works question? Qualitative Inquiry, 18, 662–673.
Eisenhart, M., & DeHaan, R. L. (2005). Doctoral preparation of
scientifi-
cally based education researchers. Educational Researcher,
34(4), 3–13.
Elsabbagh, M., Divan, G., Koh, Y., Kim, Y., Kauchali, S.,
33. Marcin,
C., . . . Fombonne, E. (2012). Global prevalence of autism and
other pervasive developmental disorders. Autism Research,
5(3),
160–179.
Feuer, M. J., Towne, L., & Shavelson, R. J. (2002). Scientific
culture
and educational research. Educational Researcher, 31(8), 4–14.
Foucault, M. (1980). Power/knowledge: Selected interviews and
other
writings (1972–77) (C. Gordon, Ed.). Essex, UK: Harvester.
Gee, J. P. (2000). Identity as an analytic lens for research in
Education.
Review of Research in Education, 25(1), 99–125.
Gee, J. P. (2010). How to do discourse analysis: A toolkit. New
York, NY:
Routledge.
Giroux, H. A. (2004). The terror of neoliberalism:
Authoritarianism and
the eclipse of democracy. Boulder, CO: Paradigm.
Glass, G. V. (1976). Primary, secondary, and meta-analysis of
research.
Educational Researcher, 5(10), 3–8.
Glass, G. V. (2000). Meta-analysis at 25. Retrieved from
http://www
.gvglass.info/papers/meta25.html.
Gorski, P.S. (2013). What is critical realism? And why should
you care?
34. Contemporary Sociology, 42(5), 658–670.
Gough, D., Thomas, J., & Oliver, S. (2012). Clarifying
differences
between review designs and methods. Systematic Reviews, 1,
28.
Retrieved from http://www.systematicreviewsjournal.com/con-
tent/1/1/28
Greenhalgh, T., Macfarlane, R. G., Bate, P., Kyriakidou, O., &
Peacock, R. (2004). Storylines of research in diffusion of
innova-
tion: A meta-narrative approach to systematic review. Social
Science
Medicine, 61, 417–430.
Hacking, I. (2002). Historical ontology. Cambridge, MA:
Harvard
University Press.
Hacking, I. (2004). Between Michel Foucault and Erving
Goffman:
Between discourse in the abstract and face-to-face interaction.
Economy & Society, 33(3), 277–302.
Hacking, I. (2009). How we have been learning to talk about
autism: A
role for stories. Metaphilosophy, 40, 499–516.
Hannes, K., & Macaitis, K. (2012). A move to more systematic
and transparent approaches in qualitative evidence synthesis:
Update on a review of published papers. Qualitative Research,
12, 402–442.
http://www.gvglass.info/papers/meta25.html
35. ApRIl 2017 139
House, E. R. (1991). Realism in education. Education
Researcher, 20,
2–9.
Kastner, M., Antonya, J., Soohiaha, C., Strausa, S. E., &
Triccoa,
A.C. (2016). Conceptual recommendations for selecting the
most
appropriate knowledge synthesis method to answer research
ques-
tions related to complex evidence. Journal of Clinical
Epidemiology,
73, 43–49.
Kuhn, T. S. (1962). The structure of scientific revolutions.
Chicago, IL:
University of Chicago.
Latour, B., & Woolgar, S. (1986). Laboratory life: The
construction of
scientific facts. Princeton, NJ: Princeton University. (Original
work
published 1979)
McKenzie, J. A. (2013). Models of intellectual disability:
Towards
a perspective of (poss)ability. Journal of Intellectual Disability
Research, 57(4), 370–379.
National Research Council. (2002). Scientific research in
education
(R. J. Shavelson & L. Towne, Eds.). Washington, DC: National
Academy Press.
36. Paterson, B. L., Thorne, S. E., Canam, C., & Jillings, C. (2001).
Meta-
study of qualitative health research. Thousand Oaks, CA: Sage.
Peters, M. A., & Burbules, N. C. (2000). Poststructuralism and
educa-
tional research. Lanham, MA: Rowman & Littlefield.
Petticrew, M., & Roberts, H. (2006). Systematic reviews in the
social
sciences: A practical guide. Malden, MA: Blackwell.
Phillips, D. C., & Burbules, N. C. (2000). Postpositivism and
educa-
tional research. Lanham, MA: Rowman & Littlefield.
Searle, J. R. (1997). The construction of social reality. New
York, NY:
Free Press.
Sider, T. (2014). Writing the book of the world. New York, NY:
Oxford
University Press.
Skrtic, T. M. (1991). Behind special education: A critical
analysis of pro-
fessional culture and school organization. Denver, CO: Love.
Slavin, R. E. (1986). Best-evidence synthesis: An alternative to
meta-
analytic and traditional reviews. Educational Researcher, 15(9),
5–11.
Van Cleave, J. (2012). Scientifically based research in
education as
37. a regime of truth: An analysis using Foucault’s genealogy and
gov-
ernmentality (Unpublished doctoral dissertation). University of
Georgia, Athens. Available from ProQuest Digital Dissertation
and Theses Database. (Accession No. gua4088456)
Watt, J. H., & Van Den Berg, S. (2002). Elements of scientific
theories:
Concepts and definitions. Research methods for communication
sci-
ence. Boston, MA: Allyn and Bacon.
Wehman, P., Carol, S., Carr, S., Targett, P., West, M., & Cifu,
G.
(2014). Transition from school to adulthood for youth with
autism spectrum disorder: What we know and what we need to
know. Journal of Disability Policy Studies, 25(1), 30–40.
What Works Clearinghouse. (2014). What Works Clearing
House:
Procedures and standards handbook version 3.0. Washington,
DC:
Institute of Education Sciences, U.S. Department of Education.
Wolgemuth, J. R., Agosto, V., Lam, Y. H., Riley, M., Hicks, T.
A., &
Jones, R. (2016). Storying transition-to-work for/and youth on
the
autism spectrum in the United States: A critical construct
synthesis
of academic literature. Disability & Society, 31(6), 777–797.
AuThORS
JENNIFER R. WOLGEMUTH, PhD, is an assistant professor at
38. the
University of South Florida, 4202 E. Fowler Ave., Tampa, FL
33620;
[email protected] Her research focuses on the ethics and validity
of
social science research, with attention to its (unintended)
impacts on
participants, researchers, and research audiences.
TYLER HICKS, PhD, is a postdoctorate researcher at the
University of
Kansas, 1122 West Campus Rd., Lawrence, KS 66045;
[email protected]
His research focuses on issues in critical realism, Bayesian
methodology,
and inclusive school reform.
VONZELL AGOSTO, PhD, is an associate professor of
curriculum
studies at the University of South Florida, 4202 E. Fowler Ave.,
Tampa,
FL 33620; [email protected] Her research agenda focuses on
curriculum
leadership as anti-oppressive education, with an emphasis on
race, gen-
der, and dis/ability.
Manuscript submitted June 28, 2016
Revisions received January 16, 2017, and March 3, 2017
Accepted March 4, 2017
Please edit based on comments and expand the paper by 1200
39. words more
Crimes and Violence in Societies from an Interdisciplinary
Approach
As far-reaching and consequential social and political issues,
crime and violence are perennial problems. Modern society
faces violence as one of the world's most serious public health
challenges. Rates of violent crime continued to grow during the
early 1960s. Then they dropped significantly between 1975 and
1978, only to start climbing again a year later. Numerous
factors are known to affect crime rates, such as the rate of
change in society, unemployment, racism, declining resources
and output, inflation, and ineffective bureaucracy. There have
been billions of dollars spent, but no major reductions in crime
as a result of these efforts. Part of the problem with gaining a
complete understanding of violence is that researchers have a
hard time conceptualizing differentiating between licit and
unlawful forms of violence (Lee, 2019). A multidimensional
approach, looking at the continuum between legal and criminal
kinds of violence, is advocated by researchers that studied this
topic. Notably, the study of violence has traditionally been
fragmented over many academic fields with little cross-
disciplinary interaction, which has hampered our ability to
comprehend the root causes of violence and slowed down
efforts to investigate and prevent it. With this research, we hope
to learn more about crime and violence from many different
perspectives. Comment by Fatema Al Sadi: Where?
Worldwide? Pay attention to the importance of context in
everything that you write. Comment by Fatema Al Sadi: US
Comment by Fatema Al Sadi: What exactly is this more?
their relationship? Their connection? Their influence on one
another? Be more precise in the final paper Comment by
Fatema Al Sadi: Your choice of topic is interesting. But it is
important to both narrow down the topic and to contextualize it
–
Crime and violence, from the standpoint of social psychology,
are universal. For instance, the field of social psychology
40. investigates the mental effects that people have on one another.
In this approach, crime and violence are explained in terms of
contextual, cognitive, and structural factors. For instance, the
approach argues that people are more likely to agree with the
group's dominant evaluations and less likely to resist authority
when the group's judgment is incorrect. In addition, elites who
make policy are subject to the pressure to conform. This is
because the Family, community, ethnic, social, national, and
international levels of violence and crime are all exacerbated by
these structures. There is a lot of feedback between the several
degrees of violence since one level might inspire more violence
on higher levels. For instance, biases and other mental
processes can cause individuals to act in discriminating ways,
which can then feed into a national trend toward violence and
exclusion (Cikara, Martinez, & Lewis, 2022). An additional way
in which intergroup conflict can manifest itself is through an
increase in the tendency to avoid association with "the other,"
which can then lead to polarization at both the community and
interpersonal levels. Although it is recognized that different
levels of social complexity have emergent properties, social
psychological analyses of the interaction of cognitive and
situational variables have provided a useful lens for
understanding not only individual and interpersonal behavior
but also group and intergroup behavior. Comment by Fatema
Al Sadi: In what sense? Explain more Comment by Fatema
Al Sadi: Under all conditions? What is this based on?
Comment by Fatema Al Sadi: Spurred by what? I think you
just need to elaborate a bit more here to make the idea clearer.
You could explore that further in the final paper
Politics as a tactic may contribute to an increase in crime
and violence. Notably, even during the era of tribal rule, there
was unfortunately a connection between politics and crime. A
political crime is defined as any act that is performed with the
intent to damage a country, its government, or its political
system. Violent acts committed before an election are used on
purpose by criminal organizations to influence the results of the
41. vote and the behavior of politicians. As things stand, various
political bodies have been taken over by criminal gangs.
Captured politicians cause a skewing of public resources toward
the territories controlled by organized crime (Phillips, 2018).
Every initiative to improve the criminal justice system, -the US
for example- is seen as either useful or threatening, depending
on the perspective of some political-interest group. Here,
idealistic political reformers engage in questionable electoral
behavior, and legislators are alarmed at the prospect of a
complex enforcement agency slipping into the hands of their
enemies. Violence is not required for a political crime to be
considered criminal. Despite the absence of a uniquely
American category of criminal offense dedicated to political
crimes, Americans have been and continue to be charged with
and convicted of such offenses. Therefore, the accused may
justify their actions by saying they were afraid for their country
and would do anything to protect it or that they genuinely
believed that manipulating others was required to achieve their
goals. The criminal's motives are irrelevant; their actions are
nonetheless illegal and carry varying degrees of punishment.
Rather than lumping all of these offenses into one category,
they are dealt with individually because of the wide range of
severity. Comment by Fatema Al Sadi: Try to be more precise
and specific. Avoid generalizations and assumptions and better
support your ideas.
Health can be dilapidated in the global context due to titian
caused by crime and violence. Notably crime and violence a
social determinant of health; it is not intended to be
comprehensive and may not address all relevant facets of the
issue. It is possible for everyone to feel the effects of crime and
violence on occasion, whether directly or through secondhand
exposure. Because of this, while crime and violence affect
everyone, some communities are disproportionately affected.
For instance, homicide rates among Black children and young
adults are consistently higher than those among White youth in
the US. Victims of violent crimes face not only the possibility
42. of death, disability, and other damage, but also the pain and
suffering of recovering from their injuries and the loss of
quality of life that can accompany them (Hlongwane, 2021).
Regularly numerous studies have shown that exposure to any
form of violence, at any time in a person's life, can have
negative effects on their physical and mental health. Long-term
health repercussions, including chronic illness and mental
issues, have been linked to adverse childhood experiences
(ACEs) such as violence or abuse for decades. Being exposed to
violence as an adult can have detrimental effects on health.
Women who have experienced abuse at the hands of an intimate
partner, for example, are more likely to suffer from
psychological and physiological issues like eating disorders,
depression, and even suicidal ideation. Thus, it may be useful to
address exposure to crime and violence as a public health
concern to avoid and lessen the consequences to individual and
community health and well-being. The primary goals of public
health interventions to prevent crime and violence are to
increase resilience and decrease vulnerability, promote positive
gender norms, foster positive relationships, and provide safe
environments. Comment by Fatema Al Sadi: Pay attention to
your transitions Comment by Fatema Al Sadi: Great. More
of this. More critical engagement with the issue at hand is
needed. Work on the connection between all three in a smoother
manner by focusing on the links and the transitions. For the
final paper, focus your analysis on a more specific and
contextualized issue, instead of giving a general picture.
In conclusion, there is a need to ensure that we are coexisting
in a world and environment which is free from crime and
violence. In the global context crime and violence are
insinuated in different interdisciplinary and there is an
indication that they cause a lot of harm in the community. It’s
rational that, the social psychology aspect can be rationalism
which can induce the group or the society to regroup and
instigates crime and violence. The thinking capacity of many
people at the same time can trigger animosity which can agitate
43. crime and violence in the community. Globally, the agility of
political animosity and differences have been the building
ground that has mounted to crime and violence through the use
of the mafia and political leaders in the community. There is an
association agitated between better healthcare and violence and
crime. It’s instigated that some adult disorders can occur in case
the individual had the experience of crime and violence at a
younger age which could have caused depression and trauma.
Comment by Fatema Al Sadi: Pay attention to your use of
commas. There is no need for one here. Comment by Fatema
Al Sadi: What does this mean? Comment by Fatema Al Sadi: Be
more critical and provide more theoretically substantial material
to tackle this issue in the final paper
References
Cikara, M., Martinez, J. E., & Lewis, N. A. (2022). Moving
beyond social categories by incorporating context in social
psychological theory. Nature Reviews Psychology, 1(9), 537-
549.
Hlongwane, N. P. (2021). A critical analysis of the offense of
rape in the Criminal Law (Sexual Offences and Related Matters)
Amendment Act 32 of 2007: identifying challenges and
providing solutions thereto (Doctoral dissertation).
Lee, B. X. (2019).
Violence: An interdisciplinary approach to causes,
consequences, and cures. John Wiley & Sons.
Phillips, B. J. (2018). Terrorist tactics by criminal
organizations: The Mexican case in context. Perspectives on
Terrorism, 12(1), 46-63.