something helpfulIMG_3902(20200513-114013).JPGsomething hel.docx

something helpful/IMG_3902(20200513-114013).JPG
1
Assignment Cover Sheet
Program: BSSAS - BSocSc Applied Sociology
Course Code: SS3422
Course Title: Programme Evaluation
(Student Number) (51588988)
Lecturer: CHAN Wing Tai JP

Submission Date: 30 April, 2010
A Critical Review on “An Evaluation Report on Third
Path Man Services Batterer Intervention Program”
2
Logic Model
An Evaluation Report on Third Path Man Services Batterers
Intervention Programme
*URICA-DV: University of Rhode Island Change Assessment –
Domestic
violence, which is developed to assess the readiness of batterers
to end their
violence behaviors.
Inputs Activities Outputs Outcomes
(Immediate)
No one showed
regression at the
stage of

measurement.
19 of the 21
remaining
participants showed
progression.
14 among the 19
participants, who
showed progression,
reached the highest
stage of URICA-
DV*.
Average number of
stage the 21
participants
progressed was
2.72.

49 male batterers
joined the programme.
28 of them stayed for
less than 3 months in
the programme.
21 stayed in the
programme for more
than 3 months.
5 participants of the
remaining 21 stayed
for 6 months.
2 stayed of the
remaining 21for more
than 10 months.
Average duration of

intervention for 21
participants was 4.8
months.
Individual meeting - Social
worker met with male batterers to
discuss their problems.
Counseling service - according
to each batterer’s particular need.
Group meeting - participants
were invited to join voluntarily.
Contents of the individual and
group meeting include:
1. Knowledge of domestic
violence
2. Beliefs and myths
contributing to domestic
violence
3. Awareness of gender beliefs

and stereotypes
4. Anger management skills
5. Effective communication
skills
6. Problem-solving skills
Staff and social
workers – from
Harmony House
Funding – from
Hong Kong Jockey
Club
Time – 14 months
Partner – The
Hong Kong
Polytechnic
University for the
evaluation section.

3
Chan, Y. C. (2007). An evaluation report on third path man
services batterers intervention
program. Hong Kong: Harmony House Limited
A critique on “An Evaluation Report on Third Path Man
Services Batterers
Intervention Program”
Abstract
The Evaluation Report on Third Path Man Services Batterers
Intervention Programme was
conducted from 2004 to 2005 and published in 2007. Several
problems of the evaluation
report was recognized: ambiguous objectives and usage of the
evaluation; lacking indicator or
standards on the successful level of the participants; and high
dropout rate. Need assessment,
process evaluation, factorial evaluation design, survey, in-depth
interview were suggested to
make improvement on the programme. The reliability of the
evaluation report can be

strengthened by reporting the alpha value or test-retest result;
and the internal validity of the
evaluation report should be taken care of by keeping logbook or
journal of the participants.
4
Introduction
Programme details
The Batterers Intervention Programme aimed at stopping male
batterers’ violence behaviors
on their partners. At first, the programme was put forth in 2000
with a three-year pilot project
from 2000 to 2003. An evaluation on the pilot project was
conducted in 2004. The result of
the evaluation showed the success and usefulness of the pilot
project. Thus, the project was
extended for another two year, from 2004 to 2005. Data in this
report were mainly from 2004
to 2005.
Male batterers who came to the service would meet with the
social worker would meet with

the social worker individually to discuss their problems. Then,
counseling service would be
rendered according to each batterer’s needs. Lastly, participants
were invited to join group
meeting voluntarily. The contents of the individual and group
meeting included (1)
Knowledge of domestic violence, (2) Beliefs and myths
contributing to domestic violence, (3)
Awareness of gender beliefs and stereotypes, (4) Anger
management skills, (5) Effective
communication skills and (6) Problem-solving skills.
Evaluation Background
Dr. Chan Yuk Chung, the assistant professor of Department of
Applied Social Sciences, The
Hong Kong Polytechnic University, conducted the evaluation on
“The Third Path - Man
Services – Batterers Intervention Programme”. The evaluation
aimed at assessing the changes
made by the participants before, during and at the end of the
programme. All data collected
and evaluated were from November 2004 to December 2005.
Evaluation target
Male batterers who joined the Batterer Intervention Program

from November 2004 to
December 2005 could join the evaluation voluntarily. No
decision makers, social workers or
staff involved in the intervention were interviewed nor
evaluated.
Measuring instrument
The Transtheoretical Model of Change was used. It provided a
framework for the evaluation.
This model also showed directional change of the services users
and thus, provided a clear
5
effect of the programme. Based on this model, a 20-item
University of Rhode Island Change
Assessment – Domestic Violent (URICA - DV) developed by
Levesque, Gelles & Velicer
was used. 7 stages of progression were recognized as (1)
reluctant, (2) immotive, (3) non-
reflective action, (4) unprepared action, (5) pre-participation,
(6) decision-making and (7)
participation. Stages (1) to (3) are grouped as earlier stage and
stages (5) to (7) are grouped as

advanced stage.
Baseline was set by two raters, rating the score of every
participant before the treatment
started. The inter-rater reliability was 80%.
Data collection
Before joining the treatment, participants had to complete the
20-item Chinese version of
URICA-DV. At the end of the 3
rd
and 6
th
month, participants had to complete the same
questionnaire for the second and last assessment. 49 participants
completed the URIC-DV
voluntarily. 28 of them had completed the scale once before
leaving the programme. 21 of
them who stayed for more than 3 months had completed the
scale at least once.
Output
49 male batterers joined the programme, 28 of them stayed in
the programme for less than 3
months and the remaining 21 stayed more than 3 months. Out of
the 21 participants, 5 stayed

for 6 months and 2 stayed for more than 10 months. The average
duration of intervention for
the 21 participants was 4.8 months.
Outcomes
No one showed regression at the stage of measurement. 19 out
of 21 participants showed
progression. 14 of them even reached the highest stage of
URICA-DV, in which stating that
participants are actively working on ending their violence
action. The average number of
stage the 21 participants progressed was 2.72.
(Logic Model to be inserted here)
6
Critique
Objective of the evaluation report
This was an outcome evaluation. It focused on the extent the
outcomes have achieved and

gave every detail on it. It also emphasized on the effectiveness
of the programme on the
service users. Although the evaluator did not mention any
reasons for conducting the
evaluation, there seems to have two latent reasons: to fulfill
accreditation requirement and
account for funds.
This Batterers Intervention Programme had been carried out in
2000 to 2004; coming to its
second phase (from 2004 to 2005), showing its effectiveness
may serve as an indicator for
decision makers and sponsors to continue offering support and
funds to the programme. The
original objective of the evaluation report is to show the
improvement made by the
intervention programme to its service users. Yet, when the
latent reasons outweigh the main
objective, the evaluation report becomes the means to determine
the survival of the
programme. The effectiveness and usefulness of the evaluation
become very doubtful and
may be biased.
Stufflebeam’s advocated that the purpose of evaluation is not to

prove the effectiveness of the
programme, but to improve it (陳永泰, 1991, p.44). This
evaluation focused too much on
proving the effectiveness of the programme to its services users
and ignoring other problems
which improvement should be made. Although the evaluation
demonstrated the changes of
the services users after joining the programme, it did not collect
comments or opinions from
the decision makers, staff, social workers and even the services
users on the contents of the
programme. This showed that the evaluation only proved the
effectiveness of the programme
but failed to recognize any content or procedure problem of the
programme.
Also, the usage of the evaluation is not clearly defined. It seems
like the evaluation was
conducted based on the needs the interested groups like the
sponsors, not the main services
users. Since the evaluation showed the effectiveness of the
programme so that it could gain
support or fund again and the programme can be extended in the
future. This could make the
evaluation inadequate by obtaining favorable information and

hiding its weaknesses. The
7
objectives of the evaluation and the main users of the evaluation
report should be clearly
defined.
Measuring instrument
The reliability of the Chinese version of the URICA-DV was not
mentioned. Without showing
the reliability of the scale, the internal consistency of the scale
is not known and it is hard to
determine whether the items in the scale are reliable. The alpha
value or the test-retest value
should be included.
Baseline was set by two raters, which was important and useful
when comparing the result of
the participants in later stage. However, no standards or
indicators were made for the success
level. For a better evaluation, there should have at least an
indicator stating what should be
achieved by when. The application SMART goal helps
determine the progress of the

participants, whether they are above or below standards.
SMART GOAL stated that any goal
or objective should be specific, measurable, achievable,
relevant and within a given time
bound. Without it, the success level will be as ambiguous as its
objective – participants would
show progression and make no regression. Since the time bound
and the extent of standard
progression are not mentioned. It only benefits the evaluator by
allowing him to give flexible
explanation on the evaluation outcomes but failed to give a
deeper understanding on the
progress of the participants.
Data collection
This is a quasi-experimental, Single Group Time Series Design,
in which data were collected
in different condition, the progress of the participants could be
observed and compared easily.
However, referring to the MAXMINCON principle, this
evaluation design failed to minimize
the error variance and control extraneous variance. Without
control group, the power to detect
the real impact of the programme on its services users is weak,

and the extraneous variance,
such as religious background and personality of the participants,
cannot be ruled out.
Individual variance or third variables may change the
participants’ attitude or behaviors,
instead of the programme. It is difficult to determine whether
the improvement of the
participants is really due to the programme. Its small sample
size and high mortality rate
(57%) also make the conclusion less convincing and sound.
8
Besides, there are other threats to its internal validity such as
repeated testing, history,
maturation and regression to mean:
Some participants completed the scale for three times or even
four times in this evaluation, it
is possible that participants may give out expected answers in
the later stage of assessment
after gaining familiarity in the testing area or being aware of
the evaluation purpose. Biased
answers may be obtained.

History refers to the possibility that the changes of the
participants are made by other current
events rather than the intervention itself. Since the participants
were not totally isolated in a
testing environment, other events such as the encouragement of
his family members or
recently joining any religious group can cause the improvement
of the participants, instead of
the programme. An effective use of logbook or journal, such as
asking the participants to
write down their relationship with their partners once a week
and record down any special
event happened between them, can eliminate this threat.
In addition, participants’ changes of behaviors or attitudes may
be caused by their normal
development such that they learned their lesson and were
determined to make changes. This is
the maturation of the participants which makes the results in the
evaluation report unrelated to
the programme effectiveness.
Last but not least, regression to mean affects the significance of
the evaluation. Since all the
49 participants had battering record before joining the

programme. It is not surprised to see
them making changes. Thus, the intervention may just play one
part in improving
participants’ attitude and behaviors but giving all the credits to
the programme may make the
evaluation doubtful. Random assignment of the participants into
experimental group and
control group can help rule out the maturation and regression to
mean effect. However, the
small sampling size may make the process more difficult to
carry out.
Outputs and Outcomes
Among 21 participants stayed for more than 3 months, 5 stayed
for more than 6 months and 2
stayed for more than 10 months. The evaluation makes no
explanation on this phenomenon.
Since no standard was made, the appropriate treatment period
was not defined. Thus, it cannot
be concluded that the longer the participants stayed in the
programme, the better or vice versa.
9

The high dropout rate (57%) remains the most critical problem
in this evaluation. It may
indicate the weakness of the programme and imply for the needs
of amendment. Most of the
participants leave the programme after reaching the highest
stage - “participation”. It is logical
that participants will leave the programme when they perceive
themselves as “recovered” or
“cured”. These participants progressed faster than others and
could complete the programme
within 3 months. On the other hand, there were 5 participants
who stayed in the programme
for more than 6 months, the great variation among them makes
it hard to look for patterns or
characteristics of the participants.
The usefulness of the contents of the individual and group
meeting was also questioned. The
topics included: (1) Knowledge of domestic violence, (2)
Beliefs and myths contributing to
domestic violence, (3) Awareness of gender beliefs and
stereotypes, (4) Anger management
skills, (5) Effective communication skills and (6) Problem-
solving skills. However, the
evaluation report did not specify the sequence of the topic

covered, then, it is possible that
different participants would cover different topics during
different time phase during the
treatment. Otherwise, if the topics covered are in a systematic
and logical sequence, then a
process evaluation may be needed to identified the advantage
and disadvantages of the current
treatment process as well as to make amendment. As some
topics covered were quite effective
to certain group of people so that they completed the treatment
earlier and some topics were
not very effective so that some of them have o stayed for longer
treatment. The 49 cases
would then become very useful in determining the effectiveness
of certain topics.
Moreover, it revealed another weakness of the evaluation, which
is the lacking of needs
assessment. Without a proper needs assessment on the needs of
the services users, their deep-
rooted problem or obstacles are not known and the steps for
planning a programme may be
too messy to be effectively organized. Needs assessment may
even help in reducing the high
dropout rate and setting an achievable standard for the

completion time of the participants.
If resources are available and the sample size is large enough,
factorial design can determine
what kind of programme is suitable to what kind of participants;
as well as the sequence of the
treatment:
Group 1a O1 T1 + T2 O2 O3
10
Group 1 Group 1b O1 T2 + T3 O2 O3
Group 1c O1 T1 + T3 O2 O3
Group 2
Group 2a O1 T1 + T2 O2 O3
Group 2b O1 T2 + T3 O2 O3
Group 2c O1 T1 + T3 O2 O3
Participants can be participants are assigned to tow different
groups according to their stage

before joining the evaluation. The six stages are originally
subdivided into two groups: stage
(1) to stage (3) is the earlier stage and stages (5) to (7) are
grouped as advanced stage. The
participants in group 1 can further divide into 2 sub-groups
randomly or according to their
actual stage. Different combination of the treatment can be
applied to different groups. Pretest,
posttest and maintenance test can be made to assess the progress
of the participants. Thus, the
contents and sequence of the topics covered can be assessed;
and the change pattern of the
participants in different group can be observed. A more flexible,
useful and suitable treatment
combination can be planned according to the needs and stage of
the participants in the future.
Discussion
The report suggested that the high dropout rate was due to
inappropriate intake procedure in
which some cases were selected inappropriately. However, it
did not mention the meaning of
“inappropriate cases”, and the criteria of selecting a case were
not mentioned in the report.
Moreover, there is no standard for the time that participants

should stay in the programme. It
is possible that completion within 3 months is the standard and
the 21 participants who spent
more than 3 months were below standard. Thus, without this
crucial indicator, the role of the
high dropout rate remains undefined, whether it is a big
weakness of the programme or a
satisfactory result is unknown yet.
Second, ineffective engagement skills of the staff were also
reported as a failure to keep
batterers in treatment. However, the cases showed that 66.7% of
the participants who leave
the treatment have already reached the highest stage -
“participation”. Thus, it is possible that
the high dropout rate is due to the self-perceived completion of
treatment by the participants,
instead of lacking effective engagement skills of the staff.
The report also suggested have “court-ordered treatment” so
that the participants will have to
join the programme compulsively and thus, reduces the number
of dropout cases. Since the
11

evaluator has put much emphasis on improving the high dropout
rate, I would suggest have a
services users opinion survey so as to collect participants’
opinion on the programme. The
data collected reflect the real needs of the participants and their
reasons of early completion;
this data will be useful for making needs assessment. Decision
makers can plan for a more
effective and efficient programme and evaluation to suit the
needs of the participants.
If available, an in-depth interview with the participants can find
out the reasons for staying in
the treatment and early dropout of the treatment. It can be used
to double-check the
participants to see whether their progression achieved is real or
just an effect of the repeated
testing.
Conclusion
As an outcome evaluation, the report has given a very detailed
assessment on the outcomes
and effectiveness of the programme, the level of progression
each participant has achieved

and the time taken for the achievement. The evaluation was also
well constructed with a
sound theoretical framework. Thus, the evaluator could stay
focus on the theoretical model
and avoid distraction. The framework also offers an observable
changes of the participants so
that their progress over time.
However, the objectives and the main user of the evaluation
report are not clear, causing
doubts on the favorable results obtained or even failing to
reflect the effectiveness of the
whole programme to its services users.
Furthermore, this evaluation emphasized on the effectiveness of
the intervention programme,
and so relied heavily on the assessment results before and after
the programme. It overlooked
the contents of the intervention programme. The great variance
among the 21 cases makes it
hard to find out the characteristics and pattern of the
participants, such like which groups of
people are more likely to complete the programme within three
months and which groups will
stay for six months or even more. Lacking of indicator and
standards of the evaluation causes

a failure in the report – there is no way to decide an acceptable
length that participants should
be staying in the programme. With this piece of information,
instead of giving out ambiguous
conclusion – 66.7% of the participants reached the highest stage
– a more meaningful and
useful result can be drawn.
12
The evaluator perceived the high dropout rate as a problem to
the evaluation. Yet, the
meaning of the high dropout rate was undefined because there is
no indicator for the length
that participants should stay in the programme. It is possible
that the dropout rate is within an
acceptable range. The answer in this area remains open. On one
hand, need assessment may
be conducted beforehand to evaluate the needs of such kind of
programme and the needs of
the participants so that a more flexible and useful programme
can be planned for the services
users. On the other hand, process evaluation can be another

option for conducting an
evaluation. It reveals the strengths and weakness of the
programme process. Decision makers
can make use of the results in the process evaluation for
amendment and improvement of the
programme. Survey or in-depth interview with the participants,
decision makers, staff and the
social workers could also help collect opinions about the
process of the programme and thus,
made appropriate amendment.
Moreover, if more resources are available and the sample size is
large enough, factorial
evaluation design can be conducted so that the most appropriate
sequence of the treatment can
be tested and found. The patterns of participants can also be
found such like which group of
participants is suitable to which type of treatment or which type
of treatment works most
effective on which type of participants. This can increase the
usefulness and flexibility of the
programme.
Other recommendation such as reporting the reliability of the
scale and asking the participants

to write journal once a week can help increase the reliability
and internal validity of the
evaluation, internal threats such as history and maturation can
be ruled out.
Yet, it is also noticed that to implement the above suggestions
would require much resources
and may not be possible to carry out at once. Step by step, the
objectives and usage of the
report, standard or indicator, reliability and internal validity
must be stated and well
considered in the future evaluation report; need assessment can
be conducted before planning
any batterers intervention programme; process evaluation can be
conducted instead of
outcome evaluation. Survey or in-depth interview can be treated
as a subset of information to
the decision makers for improvement of the programme; lastly,
factorial design can be
considered if the sample size is large enough and the resources
are available. Then, the
evaluation can be more convincing and sound.
13

References:
Chan, Y. C. (2007). An evaluation report on third path man
services batterers intervention
program. Hong Kong: Harmony House Limited
陳永泰 (1991)。社會服務評估法。香港：香港基督教服務處。
RESEARCH ON SOCIAL WORK PRACTICE Mayfield Arnold
et al. / EFFECTS OF SEX EDUCATION
The Effects of an Abstinence-Based
Sex Education Program on
Middle School Students’ Knowledge and Beliefs
Elizabeth Mayfield Arnold
Thomas E. Smith
Dianne F. Harrison
Florida State University
David W. Springer
University of Texas at Austin
A statewide evaluation of the Education Now and Babies Later
(ENABL) program was con-
ducted to assess its ability to increase adolescents’ knowledge
and beliefs about pregnancy pre-
vention. ENABL is aimed at preventing teenage pregnancy
through abstinence. Using a

quasiexperimental research methodology, middle school
students (N= 1,450) comprised a treat-
ment group (n = 974) and a comparison group (n = 476).
Subjects completed a pretest and post-
test reflecting knowledge and beliefs about teenage pregnancy.
ANCOVA revealed significant
differences (F = 8.98,p < .001) on posttest scores between the
two groups. The treatment group
showed marked improvement from pretest to posttest scores,
whereas the comparison group did
not. These findings support the claim that the Postponing
Sexual Involvement (PSI) curriculum is
effective in affecting students’ knowledge and beliefs about
teenage pregnancy and illuminate
the need for social work intervention at a variety of levels to
address this multifaceted problem.
Although the birth rate for adolescents declined 8% between
1991 and 1995
(U.S. Department of Health and Human Services [HHS], 1997),
adolescent
pregnancy still looms as a social problem with a current annual
national rate
of 112 pregnancies per 1,000 female adolescents (Alan
Guttmacher Institute,
10
Authors’Note: Correspondence may be addressed to Elizabeth
L. Mayfield Arnold, Institute for
Health and Human Services Research, 2035 E. Paul Dirac Drive,
Suite 236, HMB, Florida State
University, Tallahassee, FL 32306-2810, or via the Internet to
[email protected]
This research was supported by the Florida Department of
Health to evaluate Education Now and

Babies Later (ENABL). Special thanks is extended to Cheryl L.
Robbins and Virginia A. Miller
of the Florida Department of Health for their assistance with
this project, and to Sandi Miller,
ENABL Coordinator, Osceola County, Florida, for granting
permission to use her ENABL
evaluation tool in this study.
Research on Social Work Practice, Vol. 9 No. 1, January 1999
10-24
© 1999 Sage Publications, Inc.
1997). In Florida, the estimated number of pregnancies in 1994
for females
ages 19 and younger was 52,639, with approximately 25%
ending in abortion
(Lopez, Westoff, Perrin, & Remmel, 1995).
Many adolescents who give birth find themselves raising their
children as
single parents, a difficult task that places them at a higher risk
for poverty. In
1992, the federal government spent $34 billion on support to
teenage moth-
ers, which represents a substantial 36% increase from 1990
(Poole, 1996). Of
female-headed households in Florida, more than half (53.9%) of
these fami-
lies live below the poverty line (1996 Annual Report, 1996). As
a result of the
alarming personal and societal costs of adolescents becoming
pregnant,
President Bill Clinton has vowed to step up efforts to reduce
adolescent preg-

nancy (Lameiras & Gaulin, 1997). A national, multiphased
public education
campaign called Girl Power! was launched by theHHS in 1997
to promote
self-esteem, provide accurate information on health issues, and
support
abstinence-based education initiatives. Five promising
pregnancy prevention
programs have been targeted for federal funds. One of these
programs is Post-
poning Sexual Involvement (PSI) (HHS, 1997), a nationally
recognized
abstinence-based pregnancy prevention program developed by
Marion How-
ard of Emory University. This article presents the results of an
evaluation of
the Education Now and Babies Later (ENABL) program that
used the PSI
curriculum as its core component.
RECENT HISTORY OF PREGNANCY PREVENTION
EFFORTS
The new federal welfare law passed in August 1996 allocated
$50 million
per year in new funding for abstinence-based sex education
programs begin-
ning fiscal year 1998 through the Maternal and Child Health
block grant. In
addition, the five states that reduce their out-of-wedlock births
by the largest
margin are eligible to receive $20 million if they also keep
abortion rates
lower in comparison to the 1995 rates (HHS, 1997). The
availability of such
funding provides incentives for the development and expansion

of pregnancy
prevention programs that previously had been hindered by
inadequate fund-
ing. In 1989, five states devoted $64,000 and one state
(Wisconsin) allocated
$200,000 through direct grants to sex and AIDS education
programs at the lo-
cal school district level; no other states allocated any funds
(Kenney, Guar-
dado, & Brown, 1989). The scant amount of funding allocated
by states illus-
trated either the lack of interest in the issue of sex and AIDS
education or the
desire of states to maintain direct control over the funding and
operation of
such programs.
Mayfield Arnold et al. / EFFECTS OF SEX EDUCATION 11
The incidence of adolescent pregnancy was not viewed as a
significant
problem prior to 1960, but by the end of the next decade,
recognition of the
seriousness of the issue began to develop (Schinke, 1998).
Although the
1970s and 1980s saw an increase in sex education, the numbers
of adoles-
cents reportedly reached by these programs varied, which may
be because
sex education has no universally accepted definition. However,
during these
decades, courses focused on anatomy and biology, and
proponents of sex
education were labeled as being “antifamily” (Scales, 1981).

Problems of
definition are complicated from a historical perspective because
few prior
studies have explored the content of programs or the
sociopolitical context in
which curricular decisions are made. Furthermore, educators
may be dis-
suaded from using effective but unpopular teaching strategies,
thus hindering
the potential positive impact of a program (Firestone, 1994).
The importance of public opinion cannot be overestimated as a
factor in
the success or failure of program implementation. The fate of
sex education
appears to depend on a consensus by decision makers that, in
turn, is influ-
enced by medical and moral issues (Thomson, 1994). Public
opinion polls
have found support for sex education, “but it appears to be a
passive rather
than active and visible” (Rosoff, 1989, p. 64). In some states,
statutes allow
parents to exempt their children from participation in sex
education pro-
grams. Health educators have encountered opposition from
people who want
to keep all children out of such programs, even when parents
and community
members have an opportunity to provide input into the program
and curricu-
lum development (State of Florida, 1990). Some parents believe
that sex edu-
cation should only be provided at home. However, a low level
of parental
knowledge regarding human sexuality is a significant barrier in

parents at-
tempting to teach their children basic information about
pregnancy preven-
tion (Gilbert & Bailis, 1980).
Although a national consensus may never exist regarding the
content of
pregnancy prevention programs, HHS adheres to and promotes
five princi-
ples to reduce adolescent pregnancy: parental and adult
involvement, absti-
nence, clear strategies for the future, community involvement,
and a sus-
tained commitment to youth over time (HHS, 1997). Many
legislators,
parents, and educators support the promotion of abstinence as
the best form
of pregnancy prevention. Kenney et al. (1989) surveyed large
school districts
throughout the nation and found that more than two fifths (43%)
of districts
require abstinence be taught in their curricula, and less than
half (48%) en-
courage that abstinence be taught. Although abstinence is
emphasized in the
majority of programs, educators must contend with the fact that
many youth
involved in such prevention efforts may already be sexually
active, thus docu-
menting the need for programs to target kids earlier (Rosoff,
1989).
12 RESEARCH ON SOCIAL WORK PRACTICE

The increasing popularity of abstinence-based programs has
brought with
it the need for program development, evaluation, and
monitoring. The Na-
tional Association of County and City Health Officials
(NACCHO) and HHS
contend that the goal of pregnancy prevention efforts should be
to conduct re-
search to “understand more fully the determinants and
antecedents of unin-
tended pregnancy” (HHS & NACCHO, 1996, p. 9). As Scales
(1981) noted,
the focus has historically been training, curriculum
development, and pro-
gram implementation, with some states actually preventing
evaluation from
taking place.
Demands cannot be made to answer the question “does it work”
if fundamental
restrictions are placed on the collection of essential data with
which to answer
that question. And unless more research is conducted that
contributes to the an-
swer, those who support sex education will continue to face
obstacles of skepti-
cism from those who, not unreasonably, would like to see
convincing data on
the impact of sex education. (Scales, 1981, p. 564)
Without information about factors that influence teenage
pregnancy, ef-
forts at primary prevention are limited and are less likely to
make a substan-
tial impact.

EXISTING PROGRAMS AND EVALUATIONS
Many sex education programs aimed at pregnancy prevention
exist na-
tionwide with varying degrees of formal structure, content, and
evaluation.
Frost and Forrest (1995) reviewed evaluations of five popular
pregnancy pre-
vention programs and found that of the four programs that
examined rates of
sexual initiation following program completion, all had
significant reduc-
tions in rates of initiation of sexual activity; the highest success
rates were
found when younger adolescents were targeted. In addition, two
of the five
programs reduced pregnancy rates among participants.
The Program Archive on Sexuality Health and Adolescence
(PASHA) has
identified 11 primary pregnancy prevention programs (which
includes all 5
reviewed by Frost and Forrest, 1995), in which three common
behavioral im-
pacts were found: increased abstinence, improved contraceptive
use, and
lower pregnancy rates (Card, Niego, Mallari, & Farrell, 1996).
Despite some promising results from evaluations of pregnancy
prevention
programs, their long-term benefits can be difficult to assess.
Koo, Dunteman,
George, Green, and Vincent (1994) studied the School and
Community-
Based Intervention Program in Denmark, South Carolina. They
found that al-

though the pregnancy rates among program participants
declined initially,
the pregnancy rate began increasing steadily after the
discontinuation of key
components of the program, the cessation of related services,
and a loss of
momentum. This highlights the difficulty in maintaining lasting
changes and
reinforces the need for continued program follow-up and
evaluation.
Despite the abundance of pregnancy prevention programs, a
small number
of evaluations have been conducted to assess long-term gains
from such in-
terventions. A contentious issue is whether gains in knowledge
about sexual-
ity issues influence sexual behavior. Kirby (1984) found no
relationship be-
tween knowledge and behavior change in a study of sex
education programs.
Eisen and Zellman (1987) also found little relationship between
formal sex
education and self-reported sexual activity when students were
followed at 3-
and 6-month intervals following a 15-hour (six-session)
program based on
the Health Belief Model. By contrast, a later study by Eisen,
Zellman, and
McAlister (1985) found support for the assertion that sex
education can lead

to changes in behavior and decision making regarding sexual
issues.
One issue of concern to health educators is whether school-
based pro-
grams are more effective than those in other community
settings. Shamai and
Coambs (1992) examined sex education programs using a
theoretical per-
spective that supports the relative autonomy of schools; this
theory suggests
that schools are autonomous from the larger society. Using this
framework,
they assert that school-based programs have been ineffective
because they
exclude the societal influences outside of the school as a
contributing factor
to teenage pregnancy. They go on to argue that government
agencies, the
community, and the media must be used to influence society and
ultimately
change the behavior of youth these programs are targeting.
Comprehensive
programs that go beyond a narrow biological approach to
include societal and
contextual factors are believed to be crucial to program
effectiveness
(de Gaston,Jensen, & Weed, 1995). In addition, programs must
reflect the
current research on adolescent sexual activity and have a clearly
articulated
theoretical base (Schinke, 1998).
PSI: HISTORY AND EVALUATION
PSI is an experiential educational program for adolescents

based on the
social influence or social inoculation model that theorizes that
young people
engage in potentially harmful behaviors due to inadequate
knowledge. The
aim of PSI is to provide adolescents with information about the
general na-
ture of social relationships, their rights in relationships, and
ways to deal as-
sertively with peer pressure so that they can postpone sexual
involvement.
The programmatic philosophy is that those youth ages 16 and
younger are
often pressured into activities for which they do not fully
understand the im-
plications (Howard & McCabe, 1990). The PSI curriculum
consists of five
sessions, each lasting 45 to 60 minutes, in which the following
subjects are
covered: risks of sexual involvement, social pressures, peer
pressure, asser-
tiveness techniques, and reinforcing skills (Grady Health
System, 1995).
PSI was developed at Grady Memorial Hospital in Atlanta,
Georgia, in
1983 by hospital staff of the Emory University School of
Medicine/Grady
Memorial Hospital. The staff of the Teen Services Program,
who provide
family planning services to adolescents ages 16 and younger,

realized that
traditional approaches to influencing sexual behavior among
teens were inef-
fective. They subsequently made the decision to revise their
curriculum, thus
resulting in the development of PSI. The program was field
tested in Atlanta
and Cleveland and subsequently implemented in the Atlanta
public school
system through a grant from the Ford Foundation (Howard &
McCabe,
1990). An evaluation of the Grady Hospital program found that
those who re-
ceived the PSI program were just as likely to have a boyfriend
or girlfriend
but less likely to begin having sex when compared to those who
did not par-
ticipate in PSI. However, pregnancy rates among the
experimental and com-
parison groups were similar. The program appeared to be
effective for those
who had never had sex prior to the program but not for those
who had already
become sexually active (Howard & McCabe, 1990).
PSI is the core curricular component of the ENABL program, an
award-
winning educational and informational program adopted in
several states,
that is aimed at preventing teenage pregnancy through
abstinence. In addition
to PSI, ENABL has three other components: a statewide media
and public re-
lations campaign, the provision of training for individuals who
teach PSI, and
an evaluation of the program’s effectiveness.

The California ENABL program was launched in 1992 by
Governor Pete
Wilson to try to reduce the state’s teenage pregnancy epidemic
(Vanzi, 1995).
Although still in use in several other states, the California
program was aban-
doned following a study by the University of California at
Berkeley that pro-
duced equivocal results about its effectiveness. Kirby, Korpi,
Barth, and Ca-
gampang (1995) evaluated the ENABL program for youth ages
12 to 14 years
from April 1992 to June 1994. They found a small positive
impact on some of
the variables at 3 months, but at 17 months, the results
indicated that PSI had
no long-lasting impact on any of the mediating variables,
including the deci-
sion to have sex. Although the results were disappointing to
proponents of
ENABL, the withdrawal of support for the program was viewed
by some as
premature. Judith McPherson Pratt, a health education
consultant who devel-
oped ENABL, called the disabling of the California program an
“early aban-
donment of a promising program” that illustrates society’s
investment in the
“quick fix” approach rather than long-term planning aimed at
producing real

change (Pratt, 1996).
Hence, the development of ENABL came from practice-based
ideas about
the most effective method of trying to instill in young teens the
desire to delay
sexual involvement. With some measure of success found by the
Grady Hos-
pital Study, and equivocal results in California, the need for
further evalua-
tions of ENABL is apparent in light of new funding and
legislative mandates
to improve sex education at the state and local levels. To
succeed in meeting
the needs of today’s youth, evaluation of sex education
programs must take
place. As Dr. Marion Howard, founder of PSI, stated about her
experience
with pregnancy prevention efforts, “Essentially, it was learned
that continu-
ing programs merely on the basis of good feelings about what is
being done
can do a tremendous disservice to young people” (Howard,
1991, p. 604).
Likewise, the social work literature has reflected the
professional concern
that interventions should be empirically demonstrated to be
effective
(Fischer, 1973; Harrison & Thyer, 1988; Rubin, 1985).
METHODOLOGY
An evaluation of the effectiveness of the ENABL program was
conducted
using a quasiexperimental research methodology. A formative
evaluation

study of the efficacy of the ENABL program was necessary to
determine if
the PSI curriculum was effectively implemented. The need for a
widely ac-
ceptable survey questionnaire lead the research team to select
one that had
been previously piloted in Florida (personal communication, S.
Miller, June
1996). The need to develop new items for pregnancy prevention
programs
evaluations has been demonstrated in previous evaluations of
ENABL
(Kirby, Korpi, Barth, & Cagampang, 1997). Although the
survey question-
naire did not include items about sexual activity, it examined
subjects’knowl-
edge, beliefs, and attitudes about sexuality. For programs
targeting younger
students, indications of a program’s success is commonly
judged by its abil-
ity to affect skills or attitudes as a preliminary measure of
efficacy (Card
et al., 1996).
The study examined whether students who completed the PSI
curriculum
achieved significantly higher scores on the evaluation
instrument than those
who did not receive the training. This component of the analysis
allowed the
researchers to measure the formative success of the ENABL
program. Imple-
menting a systematic evaluation of ENABL required at least a
quasiexperi-
mental research design. A quasiexperimental design allowed
researchers to

not mandate random assignment to treatment and comparison
conditions.
Such a design allows for some conclusions, albeit limited, about
the efficacy
of the interventions. A large sample size (N = 1,450) supported
the use of
parametric statistical procedures.
To assess the effectiveness of the PSI curriculum in increasing
students’
knowledge about abstinence-based issues related to sexual
behavior and
pregnancy, subjects completing the pretest were assigned a
score based on
the number of items answered correctly out of 14 dichotomous
(true/false)
items. These items reflected information taught in the
curriculum or informa-
tion that could be easily extrapolated based on the curriculum,
reflecting re-
spondents’ knowledge, beliefs, and attitudes about sexual
behavior and teen-
age pregnancy (e.g., you can get pregnant thefirst time you have
sex, you
alwayshave the right to say “NO” to having sex). For example, a
perfect score
reflecting a high degree of knowledge about sexual behavior
and pregnancy
would be 14, whereas a student who answered half of the
questions correctly
would receive a score of 7. The brevity of the instrument

ensured that the ado-
lescent participants would not become fatigued in completing
the measure
and thus introduce measurement error. At this time, the
psychometric proper-
ties of the measure are unknown. However, it has been used
previously in
Florida and was tested for its acceptability and feasibility
(personal commu-
nication, V. Miller, July 1996).
Sampling
Initially, the sampling strategy was to recruit experimental
group subjects
from counties receiving ENABL state funding or from counties
participating
in ENABL without receiving state money (N = 35). In addition,
the research
team planned to recruit comparison group subjects from
counties not partici-
pating in ENABL (N= 32). Finding a comparison group proved
difficult after
receiving refusals from 55 school districts out of 67 counties.
Therefore, the
comparison group was formed from counties scheduled to
receive ENABL
but who had not done so during the period of data collection.
Letters requesting participation were sent to every school
district in Flor-
ida (N = 67); follow-up telephone contact with representatives
from each
school district yielded the eventual study sample described
below. After re-
ceiving permission from each school district to conduct the

evaluation, pa-
rental consents were obtained. Research assistants then traveled
to each
school, obtained student consent, and administered the
questionnaire.
The cohort of students (N = 1,450) was drawn from 20 middle
schools rep-
resenting 10 school districts in rural counties or counties
bordering large ur-
ban areas. Roughly two thirds (67%,n = 974) completed the PSI
curriculum
and one third (33%,n = 476) were in the comparison group. This
cohort was
composed of the following ethnic groups: one fifth (20.3%)
African Ameri-
can, more than half (57%) Caucasian, 17.7% Hispanic, 1.3%
Native Ameri-
can, and .6% Asian American, with the remainder of the
respondents mark-
ing Other (.3%). Approximately half of the respondents were
male (44.6%)
and half were female (52.2%). The cohort predominantly
(95.7%) came from
the sixth grade. The majority of the students (89%) were doing
well in school
(making As, Bs, or Cs). Nearly three fourths (73.5%) stated that
they had a
boyfriend or girlfriend, with the remainder (25.2%) stating that
they had
never had a boyfriend or girlfriend. The respondents averaged

2.1 days per
week in which there was no parental supervision for at least 1
hour. More than
four fifths (81.9%) of the cohort spoke mostly or only English
at home. More
than half (55.1%) of the respondents lived with both their
mother and father;
more than one quarter (28.3%) lived with their mother compared
to only a
few (3.9%) who lived with their father; 7.9% lived half-time
with their
mother and half-time with their father; and 4.4% had other
arrangements.
Nearly one third (30.9%) of mothers had completed high school
and 36.7%
had completed college; the remainder (32.3%) of the
respondents’ mothers
had a middle or elementary school education or the students did
not know
their mothers’ educational level.
RESULTS
Students receiving the PSI curriculum (experimental group) and
those in
the comparison group were compared using analysis of
covariance (AN-
COVA), with the pretest as the covariate (see Table 1). Those
cases where a
respondent completed a pretest but not a posttest (15.7%) were
omitted from
the ANCOVA analysis. TheF score (F = 8.98) was significant at
thep < .001
level, indicating that a significant difference existed on posttest
scores (hold-
ing the pretest scores constant) between those respondents who

were exposed
to the PSI curriculum and those who were not.
Further analysis of the paired pretest and posttest scores found
that the
treatment group (composed of 4 counties) receiving the PSI
curriculum
showed improvement from mean pretest (10.06,SD= 2.17) to
mean posttest
(11.36,SD = 1.79) scores (see Table 1). By contrast, the
comparison group
(composed of 7 counties) not exposed to the PSI curriculum did
not show
marked improvement from mean pretest (10.02,SD= 1.93) to
mean posttest
(10.31,SD= 1.90) scores. It is worth noting that 2 of the
comparison counties
had a decrease in their mean scores from pretest to posttest.
Both changes
were statistically significant at the .05 level.
An effect size (ES) of .55 was calculated. Cohen (1977) has
simplified the
quantification of ES by proposing that ES be expressed as a
function of the
overall population standard deviation whenever possible
(Brewer, 1978).
This approach has been used here. Furthermore, Cohen (1977)
provides a
guide for interpreting ES values with an ES of .20 as small, .50
as medium,
and .80 as large ES.

Using this criteria, the effect size in this study (.55) would be
considered
medium. In the present study, the ES is interpreted as the
proportion of vari-
ance explained (PVE) in the posttest by the pretest scores. With
an ES of .55,
the independent variable (pretest scores) explains about 8% of
the variation
in the dependent variable (posttest scores). It is, therefore,
reasonable to con-
clude that the intervention is credible in terms of its
effectiveness when com-
pared to the average social work intervention reported in
published outcome
research.
DISCUSSION AND APPLICATIONS TO SOCIAL WORK
PRACTICE
This study provided useful guidance for states considering the
adoption of
teenage pregnancy prevention programs in public school
systems. Although
there are several methodological limitations, the study provides
support for
the short-term efficacy of the PSI curriculum. However, it is
important to note
that this program is not designed as a one-time quick fix for the
complex issue
of adolescent pregnancy. Adolescents must continue to have
reinforcement
of the ideas and concepts learned in the program through
continued exposure
to the program in school, home, and/or other community
settings.

Despite positive outcomes, the varying reactions to
presentations of sex
education curricula outside of the home illustrate the need for
increased com-
munity education regarding the benefits of teaching adolescents
about absti-
nence. Thus, one conclusion from this study is that broad-based
community
support and parental involvement are critical to the
implementation and
TABLE 1: Analysis of Covariance of Posttest Scores With
Pretest Scores as the
Covariate
Mean Pretest (SD) Mean Posttest (SD)
Treatment Control Treatment Control
Measure (n = 1,143) (n = 566) (n = 974) (n = 476) df F Score
Knowledge 10.06 10.02 11.36 10.31 1 8.98*
and beliefs (2.17) (1.93) (1.79) (1.90)
*p < .001.
success of adolescent pregnancy prevention efforts. Because
parents have the
right for their children not to participate in pregnancy
prevention programs in
the state where this evaluation was conducted, all adolescents
cannot be

reached through formalized programs. A lack of community
support can also
prevent programs from even being adopted by local school
districts if parents
oppose such programming.
An intervention’s effect size, in and of itself, does not reflect
that interven-
tion’s clinical significance, or practical value. In considering
the clinical or
substantive significance of this study’s results, other factors,
along with ef-
fect size, must be taken into consideration. Once the curriculum
is accepted
by schools, it is relatively easy to implement and inexpensive.
In addition to
these benefits, the program is producing a medium effect size
around an issue
(adolescent pregnancy) that is of considerable concern to any
family or com-
munity with adolescents. This is particularly true for
disadvantaged popula-
tions in lower socioeconomic neighborhoods where adolescent
pregnancy
rates tend to be higher. Collectively, these conclusions lend
support to the
clinical significance of these findings.
The current limitations on pregnancy prevention efforts
illustrate the need
for social work intervention in this area. The diversity of
practice settings in
social work allows for social workers to be involved at various
points of inter-
vention, including health care settings, community agencies,
and schools.

Social workers must promote prevention efforts but can also be
instrumental
in enhancing the messages presented in formalized programs
such as PSI.
Viewing adolescent pregnancy from an ecological perspective
clearly illumi-
nates the need to account for the social context of teenage
sexuality and col-
laborate across systems with other health care professionals
(Smith, 1997).
Those invested in preventing adolescent pregnancy must
translate empirical
knowledge into practice efforts while realizing that the
multifaceted nature of
the problem precludes simplistic solutions (Schinke, 1998).
Our enthusiasm about finding the positive outcomes was
tempered by the
number of school boards and administrators who refused to even
consider
implementing the evaluation, much less the curriculum. This
suggests that
implementing a school-based strategy to curtail teenage
pregnancy faces an
uphill struggle with little assurance of eventual success.
Although the com-
mitment of local advocates of ENABL …

something helpfulIMG_3902(20200513-114013).JPGsomething hel.docx

Recommended

Recommended

More Related Content

Similar to something helpfulIMG_3902(20200513-114013).JPGsomething hel.docx

Similar to something helpfulIMG_3902(20200513-114013).JPGsomething hel.docx (20)

More from rronald3

More from rronald3 (20)

Recently uploaded

Recently uploaded (20)

something helpfulIMG_3902(20200513-114013).JPGsomething hel.docx