Lesson 9: Evidence-Based Practices Fall 2014 Readings Brown, R.C., Fielding, J.E. & Maylahn, C.M. (2009). Evidence-based public health: A fundamental concept for public health practice. Annual Review of Public Health, 30, 175-201. Jones, A., Bond, G.R., Peterson, A.E., Drake, R.E., McHugo, G.J. & Williams, J.R. (2014). Role of state mental health leaders in supporting evidence-based practices over time. Journal of Behavioral Health Services & Research, 41(3), 347-355. SAMHSA National Registry of Effective Programs and Practices http://nrepp.samhsa.gov/. Office of Juvenile Justice and Delinquency Prevention (OJJDP) Model Programs Guide. http://www.ojjdp.gov/mpg/ Other Evidence-based Registries and Toolkits Blueprints for Violence Prevention. http://www.colorado.edu/cspv/blueprints California Child Welfare Clearinghouse. http://www.cebc4cw.org. Substance Abuse and Mental Health Services Administration (SAMHSA) (2008). Evidence-based Practice Kits. D.H.S.S. Publication No. SAM-08- 4344, Rockville: MD. http://store.samhsa.gov/list/series?name=Evidence-Based-Practices-KITs Suicide Prevention Resource Centers Best Practices Registry for Suicide Prevention. http://www.sprc.org/bpr Summary “Evidence-based practice” has been defined as “an approach to practice that requires the examination of research findings from systematic clinical research (e.g., randomized-controlled clinical research) in making decisions about the care of a specific population with a specific problem” (Levine, 2004). In the area of mental health, this term began to be used in the late 1990’s, as untested mental health and substance abuse programs were implemented and promoted that in many cases lacked any empirical evidence of effectiveness. For example, the D.A.R.E. program (Drug Abuse Resistance Education) was implemented across the country in thousands of schools, yet in a review of six evaluations of the D.A.R.E. program, the U.S. General Accounting Office found "no significant differences in illicit drug use between students who received DARE in the fifth and sixth grade and ...students who did not." (http://www.gao.gov/new.items/d03172r.pdf). A subsequent meta-analysis of D.A.R.E found the program to have less than small overall effective on drug use and psychosocial behaviors (Pan, W. & Bai, H., 2009). Due to concerns regarding the implementation of programs without positive outcome data, “evidence-based program” lists began to emerge in the areas of mental health, substance use, education, violence and other behavioral concerns. Lists included the SAMHSA National Registry of Effective Programs and Practices (nrepp.samhsa.gov), SAMHSA’S mental health evidence-based toolkits, the University of Colorado’s Blueprints for Violence Prevention, the Office of Juvenile Justice and Delinquency Prevention’s (OJJDP) Model Programs Guide, the Department of Education’s Exemplaryand Promising Safe, Disciplined and Drug-Free Schools Programs and the Su ...

1. Lesson 9: Evidence-Based Practices
Fall 2014
Readings
Brown, R.C., Fielding, J.E. & Maylahn, C.M. (2009). Evidence-
based public health: A fundamental concept for public health
practice. Annual Review of Public Health, 30, 175-201.
Jones, A., Bond, G.R., Peterson, A.E., Drake, R.E., McHugo,
G.J. & Williams, J.R. (2014). Role of state mental health
leaders in supporting evidence-based practices over time.
Journal of Behavioral Health Services & Research, 41(3), 347-
355.
SAMHSA National Registry of Effective Programs and
Practices
http://nrepp.samhsa.gov/.
Office of Juvenile Justice and Delinquency Prevention (OJJDP)
Model Programs Guide. http://www.ojjdp.gov/mpg/
Other Evidence-based Registries and Toolkits
Blueprints for Violence Prevention.
http://www.colorado.edu/cspv/blueprints
California Child Welfare Clearinghouse.
http://www.cebc4cw.org.
Substance Abuse and Mental Health Services Administration
(SAMHSA) (2008).
Evidence-based Practice Kits. D.H.S.S. Publication No. SAM-
08- 4344,
Rockville: MD.
http://store.samhsa.gov/list/series?name=Evidence-Based-

2. Practices-KITs
Suicide Prevention Resource Centers Best Practices Registry for
Suicide Prevention.
http://www.sprc.org/bpr
Summary
“Evidence-based practice” has been defined as “an approach to
practice that requires the examination of research findings from
systematic clinical research (e.g., randomized-controlled
clinical research) in making decisions about the care of a
specific population with a specific problem” (Levine, 2004). In
the area of mental health, this term began to be used in the late
1990’s, as untested mental health and substance abuse programs
were implemented and promoted that in many cases lacked any
empirical evidence of effectiveness. For example, the D.A.R.E.
program (Drug Abuse Resistance Education) was implemented
across the country in thousands of schools, yet in a review of
six evaluations of the D.A.R.E. program, the U.S. General
Accounting Office found "no significant differences in illicit
drug use between students who received DARE in the fifth and
sixth grade and ...students who did not."
(http://www.gao.gov/new.items/d03172r.pdf). A subsequent
meta-analysis of D.A.R.E found the program to have less than
small overall effective on drug use and psychosocial behaviors
(Pan, W. & Bai, H., 2009).
Due to concerns regarding the implementation of programs
without positive outcome data, “evidence-based program” lists
began to emerge in the areas of mental health, substance use,
education, violence and other behavioral concerns. Lists
included the SAMHSA National Registry of Effective Programs
and Practices (nrepp.samhsa.gov), SAMHSA’S mental health
evidence-based toolkits, the University of Colorado’s Blueprints
for Violence Prevention, the Office of Juvenile Justice and
Delinquency Prevention’s (OJJDP) Model Programs Guide, the
Department of Education’s Exemplaryand Promising Safe,
Disciplined and Drug-Free Schools Programs and the Suicide
Prevention Resource Center’s Best Practices Registry.

3. Programs were evaluated and labeled based upon their evidence
of effectiveness and their readiness for dissemination.
Depending on the list they appeared on and the scores they
received, programs were often sorted according to the strength
of evidence, often labeled as “model”, “exemplary”, “effective”,
“promising” or “emerging.”
There were several problems with this initial effort to create
evidence-based practice lists. Because there were multiple lists
generated from a broad array of governmental and state
agencies, it was difficult for program planners to sift through
these lists to determine which programs were most effective.
Three major federal departments (Education, Health and Human
Services, and Justice) had lists with many of the same topic
areas on each but different scoring methodology.
The duplication of lists led to a move in the mid-2000s to merge
all federal lists onto the SAMHSA National Registry of
Effective Programs and Practices (NREPP) to make the
selection of programs simpler for policy makers and program
implementers. This combination led to an expansion of NREPP
programs beyond substance use to include mental health,
education, employment, violence prevention, suicide
prevention, and trauma. Furthermore, NREPP no longer rated
its programs as model, effective or promising. Rather, all
programs that are submitted for review are scored for the
quality of the research and their readiness for dissemination.
Programs do not necessarily have to score highly to be on the
NREPP list. Program planners need to review the strength of
evidence of each program and decide for themselves whether a
program, despite some of its weaknesses, might be most
appropriate for the population they are serving. This new
scoring system, however, has led to some confusion because the
former criteria only allowed strong programs to be listed.
According to criteria for the submission of programs in FY2014,
minimum review criteria for NREPP required that a program:

4. · be evaluated using an experimental or quasi-experimental
study design
· demonstrate one or more positive change outcomes in mental
health and/or substance use among individuals, communities or
populations
· have results that are published in a peer-reviewed publication
or other professional publication or documented in a
comprehensive evaluation report
· have implementation materials and quality assurance
procedures in place that are ready for widespread distribution
(http://www.nrepp.samhsa.gov/ReviewSubmission.aspx)
Presently, more than 350 programs have been evaluated and
appear on the NREPP list.
Even if a program is not on a federal list, many will consider a
program to be evidence-based if peer-reviewed articles
demonstrate its effectiveness through a set of criteria including
the quality of the study design. In the area of mental health,
Lehman, Goldman, Dixon and Churchill (2004) compiled a
comprehensive list of evidence-based practices and programs
that had been evaluated in multiple studies. According to their
criteria, programs were required to have been studied in a
clinical trial, been evaluated using reliable and valid measures,
have replicated results in a variety of settings and used expert
consensus.
In addition to studies identifying evidenced-based programs,
SAMHSA has developed a series of Evidence-Based Program
Toolkits that specifically train practitioners in how to
implement best practices in the area of mental health. To date,
toolkits have been developed for eleven programs, including

5. Supported Education, Treatment of Depression in Older Adults,
Interventions for Disruptive Behavior Disorders, Consumer-
Operated Services, MedTEAM (Medication Treatment,
Evaluation, and Management), Permanent Supportive Housing,
Family Psychoeducation, Illness Management and Recovery,
Supported Employment, Integrated Treatment for Co-Occurring
Disorders, and Assertive Community Treatment (ACT) (see
http://store.samhsa.gov/home for individual toolkits).
The toolkit review process does not differ dramatically from
other review processes, and includes a systematic assessment
of peer-reviewed articles, including a review of the evidence of
program effectiveness derived from randomized control trials,
analysis of program replicability, assessment of fidelity to the
program as planned, and review of program curricula for its
adaptability to cultural and other subpopulations.
The use of evidence-based practice lists is not without its
critics. Arguably, the use of lists can be seen as a cookie-cutter
solution to complex social problems that require adaptability
rather than rigid adherence to program components. This has
certainly been the case in tribal communities who members have
felt that existing programs do not reflect the cultural values of
their communities and are inappropriate for their populations.
More recently, the term “evidence-based practice” has being
expanded to include those practices that are “research-based”
but not necessarily present on an existing national registry.
Despite this more expansive definition of effective practices,
there is no question that compared to 20 years ago, significantly
more pressure is being placed on program developers to show
strong evidence of effectiveness supported by rigorous
evaluation.
Assignment and Group Discussion
Your response to the question below and your participation in
the group discussion will be worth 8 points.
Using any of the evidence-based practices lists or the SAMHSA
toolkit series, pick a program or practice and describe the

6. program, including the intended audience, program components
and intended outcomes. Then discuss how the evidence was
built to support its inclusion on a list. For example, was it
evaluated by an independent evaluator? What specific design
was used (experimental design, quasi-experimental, pre/post,
etc.)? How many evaluations were conducted? Was the fidelity
of the program measured? Were the instruments that were used
reliable and valid? How strong was the evidence? Most of this
information will be on the NREPP list so you can get your
information from there. Then discuss whether you think the
evidence is strong enough to support its inclusion on the list,
and if it is not, why not?
Again, please try to have your initial answers to the question on
the Discussion Board by Friday so that you can respond to
others in the class by Sunday night.
References
Lehman, A. F., Goldman, H. H., Dixon, L. B., & Churchill, R.
(2004). Evidence-based mental health treatments and services:
Examples to inform public policy. New York, N.Y.: Milbank
Memorial Fund.
Levine RS, Husaini BA, Briggs N, et al. Translating prevention
research into practice. Nashville, TN: Meharry Medical
College/Tennessee State University; 2004. Final progress report
to AHRQ. Grant No. 5U 18HS011131.
Pan, W. & Bai, H. (2009). A multivariate approach to a meta-
analytic review of the
effectiveness of the D.A.R.E. Program. International
Journal of Environmental
Research and Public Health 6(1), 267-277.
U.S. Governmental Accountability Office (2003, January).
Youth illicit drug

7. use
prevention: DARE Long-term evaluations and federal efforts to
identify
effective programs. (Publication No. GAO-03-172R).
Retrieved from:
https://www/gao.gov/new.items/d03172r.pdf.
6