VIP Call Girl Sector 32 Noida Just Book Me 9711199171
School based rh program evaluation report the case of hawassa
1. Process Evaluation of School-based Adolescent
Reproductive Health Education Programs:
The Case of Awassa City
Master’s Thesis
The First of its Kind on Program Evaluation in Ethiopia
By:
Fikru Tessema (B.Sc, M.Sc)
April, 2008
2. Process Evaluation of School-Based RH Education Programs
Fikru Tessema i April, 2008
Abstract
Background: The existing young people reproductive health (RH) related education programs in
Ethiopia have received attention fairly by the Government and NGOs in the past. Some pilot programs are
running with regard to RH related education in-schools and out-of-schools in a community in different
parts of the country with the support from NGOs. It is recognized that young people need a range of
information on RH to support them in making responsible decisions regarding their sexuality. Teenage
pregnancy, abortion, sexually transmitted infections (STIs)/HIV and linking adolescent reproductive
health (ARH) related education with ARH service are common problems in adolescent RH.
Evaluation questions & objective: This evaluation seeks answers for questions how in-
school adolescents were reached with RH knowledge and skills and how ARH related education was
linked with ARH services. The overall objective of this evaluation is to assess the implementation process
of adolescent RH related education in high schools in Awassa City, South Ethiopia.
Methods: A case study design with both quantitative survey and qualitative in-depth interview was
conducted in Awassa City (23 September to 08 October 2007). A pre-tested self administered
questionnaires and in-depth interview guide were used to collect data. The data was checked, cleaned,
coded and analyzed using EpiInfo and SPSS 12.0.1.
Results: A total of 436 adolescents participated in student survey, of which about 61.7% were male
and 38.3% female. Of the total respondents, 368 (84.4%) reported being taught RH related education in
schools. RH related education was being taught in high schools during Biology class with related topics;
some times during campaigns and public events. Of the total respondents, 362 (83.0%) also reported
having RH related education sessions some times and 6 (1.4%) mentioned during public events. Of the
topics included in RH related education in high schools, HIV/AIDS (76.4%), pregnancy (64.9%) and STIs
(48.4%) were the most frequently mentioned topics. Of the total respondents, 285 (65.4%) were reported
that they participated in extracurricular activities related to RH education in high schools, of which
29.0% respondents participated in drama club, 23.2% in mini media, 32.3% in health club, 14.0%
in RH IEC/BCC and others 1.1% in harmful traditional club, youth counseling club. With regard
to training of peers, about 146 (33.5%) of the adolescents were trained in ARH by youth center. Of
the total respondents, 204 (46.8%) were also knew counseling & referral services providers for
youths with RH problems in high schools. Most RH problems frequently mentioned were HIV/AIDS
(71.6%) and followed by unwanted/unintended pregnancy (68.0%) and STIs (43.0%). Most in-depth
interviews (IDIs) participants explained that the IEC/BCC materials supplied to high schools were too
3. Process Evaluation of School-Based RH Education Programs
Fikru Tessema ii April, 2008
small amount. The student survey also revealed that about 38.8% of the total respondents read IEC/BCC
materials on RH. In the IDIs, when mentioning the role of parents, peer educators and teachers, they have
to develop the habit of free and open discussion on sexuality issues with children and other siblings and
facilitate youth friendly environment. Youth dialogue, a new initiative, designed by youth center in which
adolescents discuss RH issues and find out healthy ways of dealing with RH problems by themselves.
Conclusions: The overall provision of RH related education, participation of in-school youths in
extracurricular activities, supply and distribution of IEC/BCC materials on RH in high schools for
adolescents were achieved fair results (59%) in compliance with the intentions to reach adolescents with
RH related education and information in high schools.
Recommendations: Outreach services of the youth center and RH IEC/BCC materials have to
target RH related education in high schools for fostering consistently adolescents with knowledge and
skills on regular basis with sufficient quantity of the materials by increasing distribution outlets and
locally printed materials at regional health bureau and youth center level. New initiative like youth
dialogue should be included in youth friendly RH service standards, hence, the Federal MoH should revise
its youth friendly RH service standards.
4. Process Evaluation of School-Based RH Education Programs
Fikru Tessema iii April, 2008
Acknowledgment
My sincere appreciation goes to FMOH, TUE, the Brazil FC and JU for their efforts they made to realize
this M&E training course at M.Sc level in our country for the first time.
It is also my pleasure to extend my deep gratitude to Professor Elizabeth M. and Professor Carl K. for
their valuable guidance and comment from the very beginning to this end.
My sincere thanks go to Dr Binyam Ayele for his valuable advice and comments on evaluation proposal
and Dr Frehiowt Brehane, Dr Mirkuzie Woldie and Mr Yohannes Dibaba for their valuable advice and
comments on this evaluation result report with my sincere gratitude to Dr Frehiowt Brehane for her advice
and comment throughout the process during proposal preparation and final report writing.
I am also grateful with SNNP Regional Health Bureau, Awassa City Health Department, FGAE Branch
Office and Youth Center and school principals and unit leaders for their cooperation in facilitating data
collection.
I would like to thank all my colleagues and those who contributed their ideas and provided me data to
enrich this proposal and Dr Evan for her assistance in data analysis and Ato Hiwot Tesfaye and Tsegaye
Asefa for their IT support.
Finally, it is my pleasure to extend my sincere gratitude to my wife Sr Genet Wondimu and family for
their day-to-day encouragement and moral support that makes me more attentive in my study.
5. Process Evaluation of School-Based RH Education Programs
Fikru Tessema iv April, 2008
List of Acronyms
AIDS : Acquired Immuno Deficiency Syndrome
ARH : Adolescent Reproductive Health
BSS : Behavioural Surveillance Survey
CDC : Center for Disease Control & Prevention
CORHA : Consortium of Reproductive Health Associations
EPHA : Ethiopian Public Health Association
FGAE : Family Guidance Association of Ethiopia
FGM : Female Genital Mutilation
HAPCO : HIV/AIDS Prevention & Control Office
HIV : Human Immuno Deficiency Virus
HR : Human Resources
HSDP : Health Sector Development Program
IDIs : In-depth Interviews
ISY : In-School Youth
MCH : Maternal and Child Health
NGOs : Non-Governmental Organizations
NRHS : National Reproductive Health Strategy
NRHTF : National Reproductive Health Task Force
OSY : Out-of-School Adolescent
REB : Regional Education Bureau
RH : Reproductive Health
RHB : Regional Health Bureau
SB : School-based
SNNPR : South Nations & Nationalities People Region
STIs : Sexually Transmitted Infections
UNAIDS : United Nations Acquired Immuno Deficiency Syndrome
UNICEF : United Nations Fund for Children
USA : United State of America
USAID : United State Agency for International Development
WHO : World Health Organization
WoHO : Woreda Health Office
6. Process Evaluation of School-Based RH Education Programs
Fikru Tessema v April, 2008
Table of Contents
Page
Abstract i
Acknowledgment iii
List of Acronyms iv
Table of Contents v
List of Tables viii
List of Figures ix
CHAPTER ONE: BACKGROUND 1
1.1 INTRODUCTION 1
1.2 READINESS OF THE PROGRAMME FOR EVALUATION 2
1.3 OVERVIEW OF THE PROGRAM 3
1.3.1 Global adolescent RH related education program context 3
1.3.2 National adolescent RH related education program context 4
1.3.3 SNNPR adolescent RH related education program context 4
1.4 PROBLEM STATEMENT 5
1.5 PROGRAM DESCRIPTION 6
1.5.1 Program development stage 6
1.5.2 Program implementation level 6
1.5.3 Program Resources 6
1.5.4 Program objectives 7
1.5.5 Program components 7
1.5.6 Program logic model 7
1.5.7 A Need for Evaluation of the program 9
CHAPTER TWO: STAKEHOLDERS ENGAGEMENT 10
2.1 STAKEHOLDERS IDENTIFICATION 10
2.1.1 Key stakeholders‟ roles and use of findings of evaluation 11
2.2 PROCESS OF STAKEHOLDERS PARTICIPATION 13
2.2.1 Key stakeholders involved in the process 13
2.3 STAKEHOLDERS COMMUNICATION 13
2.3.1 Communication process 13
CHAPTER THREE: EVALUATION QUESTIONS AND OBJECTIVES 14
3.1 QUESTIONS OF THE EVALUATION 14
3.1.1 Main- and sub-evaluation questions 14
7. Process Evaluation of School-Based RH Education Programs
Fikru Tessema vi April, 2008
3.2 OBJECTIVES OF THE EVALUATION 15
3.2.1 General Objective 15
3.2.2 Specific Objectives 15
3.3 CONCEPTUAL FRAMEWORK FOR THE EVALUATION 15
3.3.1 Theoretical Framework of Evaluation 15
CHAPTER FOUR: EVALUATION METHODS 17
4.1 Focus, approach and purpose of the evaluation 17
4.2 Study area and period 17
4.3 Study design and technique 18
4.4 Source and study population 18
4.5 Inclusion and exclusion criteria 19
4.6 Sampling technique and sample size 19
4.7 Data collection instruments development 20
4.8 Data collection procedure 21
4.9 Data analysis procedure 21
4.10 Data quality management 21
4.11 Ethical Consideration 23
4.12 Operational Definitions 24
4.13 Matrix of Analysis for Judgment 25
4.14 Evaluation Report Dissemination Plan 25
CHAPTER FIVE: RESULTS OF THE EVALUATION 26
5.1 QUANTITATIVE RESULTS 26
5.1.1 Socio-Demographic Characteristics of the Study Population 26
5.1.2 Provision of RH related education for adolescents in high schools 28
5.1.3 Adolescent reproductive health education session in high schools 30
5.1.4 Extracurricular activities of ARH related education 31
5.1.5 Linkage between ARH related education and ARH services 33
5.1.6 RH problems of adolescents 35
5.2 QUALITATIVE RESULTS 36
5.2.1 Provision of RH related education for adolescents in high schools 36
5.2.2 ARH related education session in high schools 36
5.2.3 Importance of ARH related education in schools 37
5.2.4 Extracurricular activities of ARH related education 37
5.2.5 Linkage between ARH related education and ARH services 38
5.2.6 RH problems of adolescents 39
8. Process Evaluation of School-Based RH Education Programs
Fikru Tessema vii April, 2008
5.2.7 Judgmental Matrix of Analysis for Program Activities and Program Outputs 40
CHAPTER SIX: DISCUSSION 43
6.1 DISCUSSION 43
6.2 LIMITATIONS OF THE EVALUATION 47
6.3 STRENGTH OF THE EVALUATION 47
CHAPTER SEVEN: CONCLUSION AND RECOMMENDATIONS 48
7.1 CONCLUSIONS 48
7.1 RECOMMENDATIONS 49
REFERENCES 51
ANNEX 1: Closed-ended self-administered questionnaire for BIOLOGY TEACHERS
teaching in grade 9th
& 10th
of high schools………………………………………...………...55
ANNEX 2: Closed-ended self-administered questionnaire for STUDENTS in
grade 9th
and 10th
of senior secondary schools…………………………………...….….........63
ANNEX 3: Open-ended questions and guides for informants‟ in-depth interview with
EXPERTS and SERVICE PROVIDERS …………………………………..………..….…...71
ANNEX 4: Consent form for evaluation of school-based ARH related education
programs in Awassa City, SNNP Region, 2007 ……………………………………….....….73
ANNEX 5: Training guideline for data collectors ……………………………………..…........................75
ANNEX 6: Definition of evaluation indicators ……..……………………………..…...………......….....82
ANNEX 7: Judgmental Matrix of Analysis Template ……………………………............…..….....….....84
9. Process Evaluation of School-Based RH Education Programs
Fikru Tessema viii April, 2008
List of Tables
Page
Table 1: Socio-Demographic Characteristics of the Respondents (Biology teachers) by selected
characteristics, Awassa town, October, 2007 ..............................................................................26
Table 2: Socio-Demographic Characteristics of the Respondents (in-school adolescents) by selected
characteristics, Awassa town, October, 2007 ..............................................................................27
Table 3: Distribution of respondents, who were taught RH related education and cited types of RH
related education topics, Awassa town, October, 2007 ...............................................................28
Table 4: Percentage of respondents to student survey, who cited different methods of avoiding
pregnancy and preventing STIs/HIV transmission, Awassa town, October, 2007......................29
Table 5: Percentage of respondents to student survey who cited ideal age of onset of sex and getting
pregnant for a girl with first time of sexual intercourse, by selected characteristics, Awassa
town, October, 2007 ....................................................................................................................30
Table 6: Percentage of respondents who cited RH related education sessions and its importance and
teachers‟ feeling in teaching RH in class, Awassa town, October, 2007.....................................31
Table 7: Percentage of respondents who participated extracurricular activities of RH related education in
high schools for adolescents, Awassa town, November, 2007....................................................33
Table 8: Sources of information about RH services and its providers, counseling and referral services in
school, Awassa town, October, 2007...........................................................................................34
Table 9: Distribution of RH related problems of in-school adolescents and their exposure to information
on ARH services by selected characteristics, Awassa town, October, 2007 ...............................35
10. Process Evaluation of School-Based RH Education Programs
Fikru Tessema ix April, 2008
List of Figures
Page
Figure 1: Logic Model for School-based RH related education......................................................8
Figure 2: Conceptual framework of the evaluation.......................................................................16
Figure 3: Map of Sidama Zone......................................................................................................18
Figure 4: Sampling Scheme...........................................................................................................20
Figure 5: RH service providers reported by adolescents...............................................................32
Figure 6: Sources of information on RH services reported by adolescents...................................34
11. Evaluation of School-Based RH Education Program
Fikru Tessema 1 April, 2008
CHAPTER ONE: BACKGROUND
1.1 INTRODUCTION
Program evaluation is “the systematic collection of information about the activities,
characteristics, and outcomes of programs to make judgments about the program, improve
program effectiveness, and/or inform decision makers about future program development.”1, 2, 3
Numerous evaluations around the world show that comprehensive adolescent reproductive health
education (RH) programs help adolescents to delay the onset of sexual intercourse. It also helps
sexually active young people to protect themselves from unintended pregnancy, STIs and
HIV/AIDS and develop RH services seeking behaviours. 4, 5
Educating young people about reproductive health improves their ability to make informed and
responsible choices. Teaching them skills and transferring knowledge in negotiation, critical
thinking, decision-making, and communication also improves their self-confidence. Family life
education is an especially effective way to teach young people critical life skills. It can help them
to postpone sex until they are mature enough to protect themselves from reproductive health
related problems. Messages of abstinence also appear to work best when aimed at younger
adolescent who are not yet sexually active. Especially in girls, it has achieved a delay in sexual
initiation of about a year. 6, 7
Evaluation has identified many effective, comprehensive, school-based ARH related education
programs for young people in less developed and advanced nations. School-based ARH related
education programs generally have considerable role in sexual health by sharing information on
sexuality, HIV/AIDS transmission and prevention, contraception, condoms, sexually transmitted
diseases, and decision making and refusal skills. Schools should require teaching RH related
education because they are the most appropriate place of fostering knowledge and skills for
adolescents-educating for life. Adolescents have been reached with various RH programs by
bringing together a group of adolescents to establish youth centers in urban areas and maximize
health information dissemination; and providing youth-friendly clinics that improves services
utilization.8
12. Evaluation of School-Based RH Education Program
Fikru Tessema 2 April, 2008
1.2 READINESS OF THE PROGRAMME FOR EVALUATION
An evaluability assessment (EA) was conducted to ensure that reproductive health education
program in school is evaluable.9
During the EA, meetings and telephone call with and visits to
program planners, and supporters have been done prior to the development of evaluation plan.
During the visits interviewing and document review was carried out to clarify program
objectives, logical frame work for its implementation and stages of its development and level of
its implementation and made ready documents and staffs ready for evaluation. 10
A visit to Family Health Department of MoH was carried out and discussion was made with team
leaders and experts of the department. Some NGOs was contacted and discussed on program data
for evaluation and their role in the evaluation. Available programme resources were also
assessed, in which, beside the overall government lead and support, most budget and support
comes from NGOs for RH in general.
A logic model is prepared for the program and clearly structured. Within the model, the
objectives are measurable so that the degree to which they have been achieved can be assessed.
Program managers have a thought objectively; that is, what data can be collected that will
provide clear evidence that the goals and objectives have been met. Overall, program managers,
the higher officials, ensure that this program is serving those school children they set out to serve.
The program is also staffed with people with the appropriate qualifications and knowledge.
The stakeholders, program mangers, donors and NGOs, are engaged to ensure that their
perspectives are understood. This is helpful to execute the evaluation because without their
involvement, the evaluation might not address important elements of a program‟s objectives,
operations, and outcomes. As a consequence, evaluation findings might be ignored, criticized, or
resisted because the evaluation did not address the stakeholders‟ concerns or values. Schools
have been included to assess whether ARH education programs can operate together as a system
of interventions to effect change within a community. Program objectives are clearly and
strategies are in place for its implementation. In conclusion, since these all elements are in place,
the program is proved to be evaluable.11
13. Evaluation of School-Based RH Education Program
Fikru Tessema 3 April, 2008
1.3 OVERVIEW OF THE PROGRAM
1.3.1 Global adolescent RH related education program context
Many of adolescent reproductive health education programs demonstrated reductions in
reproductive health related problems among participants relative to control adolescents.12
In the
US, sixty three evaluated programs have been compiled to see the effectiveness of RH related
education in schools. Most of them effectively demonstrated a delay in the timing of first sex, an
ability to assist sexually active adolescent to increase their use of condoms, the ability to reduce
the incidence of unprotected sex and success at increasing contraceptive use other than condoms
among adolescents reached by the programs. Some of them resulted in a reduction of the number
of sex partners, assisted sexually active adolescent to reduce the frequency of sexual intercourse,
showed declines in teen pregnancy, HIV and STIs among program participants, compared to
controls.13
In Latin America, in Jamaica, a program promoted abstinence for adolescent ages 10 to 12,
emphasized increased self-knowledge and abstinence for adolescent ages 13 to 15, and promoted
protection from unintended pregnancy, HIV, and other STIs for older adolescent.14
Evaluation of
comprehensive ARH related education programs in Africa in Nigerian junior and secondary
schools showed decreased incidence of STIs, increased abstinence, increased condom use, and
reduced numbers of sex partners.15
Evaluation has also identified effective community-based
ARH programs for young people in Zimbabwe, Kenya, Cameroon, Guinea, and Uganda-
programs that successfully delay initiation of sex and effectively foster risk reduction among
sexually active adolescent.16
During the late 1980s and early- to mid-1990s, a wide-scale anti-
AIDS campaign in Uganda that included messages of abstinence resulted in a decline in HIV
rates. From 1989 to 1995 in Uganda, the proportion of 15- to 19-year-olds reporting that they
“never had sex” rose from 31% to 56% among males and from 26% to 46% among females after
they have been given ARH related education in school.16, 17
In India, a study was conducted
(1989-90) on teachers‟ comfort in teaching adolescent reproductive health, explicitly sexuality
education, 61.6% feel comfortable in discussing in non-sexual ways.18, 19
14. Evaluation of School-Based RH Education Program
Fikru Tessema 4 April, 2008
1.3.2 National adolescent RH related education program context
The existing young people RH related education programs in Ethiopia are attempting to address
RH issues of the in- and out-of school youths. GOs and NGOs, both international and local,
carried out a variety of RH activities such as training on RH for health care providers,
community-based RH, general clinic, youth–friendly clinics, youth centers/clubs, peer education,
school-based HIV/RH related education, and health information and VCT.20
Most assessments
and studies carried out to know RH related education promotion in-and out-of schools revealed
that most adolescents have low level of awareness and lack of youth friendly RH services.21, 22
Adolescent less than 20 years of age who have begun childbearing were about 37%. Unintended
births among women less than 15 years of age were about 50%. Unintended births among 15-24
year olds were about 33%. Adolescent receiving antenatal cares were about 27%. Adolescents
currently using modern contraception were about 5%. In-school adolescents who knew at least
three STIs were about 44.0% and two methods of prevention 44.2%; two modes of acquiring of
HIV/AIDS 54.3% and three methods of prevention 55.8%.23
According to a study on young
people‟s RH, STI and HIV/AIDS needs and utilization of services in eight regions (2005), both
in-school youth (ISY) and out-of-school youth (OSY) reports indicate that they are at risk of
unprotected sexual activity that expose them to RH problems and STIs/HIV infections.24
1.3.3 SNNPR adolescent RH related education program context
RH related education programs in the region are carried out by public sector and NGOs, of
which, regional health bureau, Awassa city health departments and Awassa health center are
public sector organizations and FGAE youth centers and clinics and some other faith-based
organizations are NGOs that are participating in RH related education for in-school adolescents.
The FGAE Youth Center is the one who actively involved in the program.
There are RH related education initiatives, such as training on RH, community-based RH
services, youth–friendly clinics, youth centers, peer education, school-based HIV/RH related
education, and health information and VCT. About one third (33.3%) of high schools in Awassa
City promote RH related education.20, 21
15. Evaluation of School-Based RH Education Program
Fikru Tessema 5 April, 2008
1.4 PROBLEM STATEMENT
Mostly information on sexual and reproductive health is presented in a so-called value neutral
way. Sexuality in human RH must be related to moral values. Young people need information
about sex, but it must be placed in a moral context. The greatest problem among young people
today is not only a lack of education about sexual reproductive health, but a lack of moral
instruction about sexual and reproductive health.25
Due to various reasons referred to culture and
religion in Ethiopia there is no openly discussing issue related to RH, specifically about sexuality,
family planning, STIs, and HIV/AIDS. This cultural unwillingness and embarrassment is barrier
to ARH related education programs promotion to reduce ARH problems like unintended
pregnancies and STIs/HIV in Ethiopia.20
Other barriers to ARH related education are lack of teachers and peers adequate training and
having up-to-date RH related health information. Thus, teachers and peers must be adequately
trained and given up-to-date resources to assure that RH related health information is being
properly communicated. 26, 27
Most adolescent lack basic knowledge of reproductive anatomy and
physiology, how pregnancy or STIs and HIV occurs, how to prevent them, and where to obtain
information and services.28
Adolescents do not have enough specific facts about sexual issues themselves to adequately do
safe sex and also consider that accessing contraception would disclose their sexual activity;
contraceptives to be unaffordable and family planning distribution centers are for married women
only. Prioritizing ARH related education in school timetable/curriculum is also least practiced if
not nil. Generally, the existing ARH related education and service are inadequate and large
number of adolescents is not reached by the program.29, 30
Thus, the overall aim of this evaluation is to assess how ARH related education programs are
going on and working in delivering knowledge and skills to in-school adolescents and its linkage
to ARH services in Awassa City so as to draw recommendations based on the findings of the
evaluation.
16. Evaluation of School-Based RH Education Program
Fikru Tessema 6 April, 2008
1.5 PROGRAM DESCRIPTION
1.5.1 Program development stage
ARH related education program in school is underway under the 1993 National population policy
framework. Several strategies outlined in the policy, which the National Office for Population
(NOP) is in charge of helping its implementation, specifically pertain to adolescents. These
include: 1) Reducing the high attrition rate of females in the educational system; 2) Providing
career counseling in secondary schools and universities; 3) Establishing adolescent reproductive
health counseling centers; and 4) Raising the minimum age of marriage for girls from 15 to 18
years of age.31
The MoH has also National Reproductive Health Strategy (2006 – 2015),32
a Five-Year Action
Plan for Adolescent Reproductive Health Programs (2002-2007) and services standards &
guidelines.17
1.5.2 Program implementation level
With regard to the implementation of the program, it has received attention fairly by the
Government and NGOs in the past. Some pilot programs are running with regard to RH related
education in-schools and out-of-schools in a community in different parts of the country with the
support from NGOs.22, 34
1.5.3 Program Resources
Government support for YRH is fairly strong and reflects the concern on the part of Government
authorities to prevent unintended pregnancies, STIs, and HIV among adolescent.33
Examples of
governmental support for YRH include the adoption of the 1993 population policy (still in effect
today), Passage in the Parliament of the Family Law (raising the minimum age of marriage,
among other supportive articles), and revision of the penal code, decriminalizing the
advertisement and sale of contraceptives. NGOs both international and local and faith-based
organizations are also at the forefront of ARH programs in Ethiopia.34
17. Evaluation of School-Based RH Education Program
Fikru Tessema 7 April, 2008
1.5.4 Program objectives
Objectives of RH related education programs are:17, 29
1. To reach in-school adolescents with knowledge and skills needed to foster and sustain
health-affirming behavior.
2. To increase access and utilization of adolescent reproductive health services.
1.5.5 Program components
Adolescent RH program components include:17, 29
1. Provide RH related education in school for adolescents
2. Train peer educators on RH to provide peer education in schools.
3. Train teachers in RH to facilitate school-based RH related education
4. Aware youths on minimum package of youth friendly RH services
5. Provide counseling on RH and sexual issues
6. Create referral linkage between RH service outlets
1.5.6 Program logic model
The logic model describes the sequence of events for bringing about changes to the ARH
problems by synthesizing the main program elements into a picture of how the program is
supposed to work to solve the problems35
(Figure 1).
18. Process Evaluation of School-Based RH Education Program
Fikru Tessema 8 April, 2008
Figure 1: Logic Model for School-based RH related education
Provided Work & immediate results Expected/Intended Results
Logic Model for School-Based RH Education Programs
INPUTS ACTIVITIES OUTPUTS OUTCOMES
Short Medium
IMPACTS
Long Term
Decreased
prevalence of
early onset of
sexual
intercourse, teen
pregnancy and
STIs, HIV/AIDS
Provide RH education in
school for adolescents
No of adolescents reached
by RH education
Participate adolescents in
supplementary activities
Train school teachers in RH
Train peer educators in RH
Supply and distribute RH
related IEC/BCC materials
Aware youths on minimum
package of RH services
Provide counseling on RH
and sexual issues
Create referral linkage
between RH service outlets
No of adolescents
participate
No of teachers trained
No of peers trained
No of adolescents reached
by IEC/BCC materials
No of adolescents aware of
RH services
No of adolescents reached
by counseling services
No of adolescents reached
by referal services
Increased No of
adolescents
accessing health
information on
RH
Increased No of
adolescents who
understood health
information
Increased No of
adolescents
accessing
services
Increased No of
adolescents
practicing the
relevant behaviours
Increased No of
adolescents having
favorable attitude
to practice good
behaviors
Increased No of
adolescents having
good life skills
HCF
Policy,
guidelines
and
standards
Trained
human
resources
IEC/BCC
materials
School
clubs
Youth
centers
Program
fund
Mini
media
19. Process Evaluation of School-Based RH Education Program
Fikru Tessema 9 April, 2008
1.5.7 A Need for Evaluation of the program
Adolescent RH related education programs have been implemented for decades in the study area
to prevent and control teen pregnancy, abortion and STIs/HIV infections by shaping their RH
behaviours. This is the point where to decide evaluating ARH program is important in general
and particularly in identifying insufficiencies in delivering RH related education or accurately
portraying to stakeholders how the program truly operates. Focusing on in-school RH related
education program is very important for two reasons. One is that the young people are spending
more of their adolescence in school. The other one is that school-based RH education is
educating adolescents at the right time to shape their RH behaviours at the right time.36
As school enrollment rates rise, ARH related education programs have the potential to reach a
large number of young people. It also lays a basis for RH knowledge and skills building of
adolescents in lower and middle adolescence age group.11
This is the reason why most programs
are focusing on in-school adolescents than out-of-school adolescents. Out-of-school RH
programs are more of maintaining a positive behavioral change they had during schooling time. It
also encompasses both youth and community. So that it is complex and cost wise, expensive.37
Hence, it is not included in this evaluation.
So that focusing on evaluation of reproductive health education programs in schools is more
targeted to facilitate managers‟, supporters‟ and services providers‟ thinking how their school
based RH education program is going on to meet its objectives. It is helpful to make a
comparison among program activities and decide which activities should be retained, improved
or included as new initiatives.38, 39
20. Process Evaluation of School-Based RH Education Program
Fikru Tessema 10 April, 2008
CHAPTER TWO: STAKEHOLDERS ENGAGEMENT
2.1 STAKEHOLDERS IDENTIFICATION
Stakeholders are individuals, groups, or organizations that tended to have significant interests on
how well a program functions. Stakeholders‟ identification depends on the role they have in the
program and evaluation. A number of stakeholders (NGOs and GOs) have a role and interest in
ARH related education interventions in schools and community.2, 3
Major stakeholders are categorized as: (1) Stakeholders having lead role: MoH, Regional Health
and Education Bureaus and District Health and Education Offices. (2) Stakeholders having
supporting role: NGOs (e.g., Family Guidance Association of Ethiopia). Most budget and support
for ARH programs comes from NGOs. (3) Stakeholders having implementation role: health
facilities, schools and youth centers. Most youth centers are established by NGOs and
implementing ARH services in- and out-of-schools in collaboration with schools and community.
21. Process Evaluation of School-Based RH Education Program
Fikru Tessema 11 April, 2008
2.1.1 Key stakeholders’ roles and use of findings of evaluation
Key
stakeholder
Role in the
program
Expectation from
the program
Area of
interest/involvement in
the evaluation
Expectation from the
evaluation
Use of evaluation
findings
Ministry of
Health
Policy, strategy and
national action plan
formulation and
advocacy
Reduced Prevalence of
unintended teen
pregnancy, STIs and
HIV among
adolescents.
Logistic supply and funding
of the evaluation
Letter of support for visit and
interviewing of program
operators and supporters
Information for program
improvement
Status of services provision by type
Information on population served
Conclusions and recommendations
Strengthening and revising
strategies, standards
Share experiences from
lessons learnt
Jimma
University
Support in policy,
strategy and national
action plan
formulation and
advocacy
Reduced Prevalence of
unintended teen
pregnancy, STIs and
HIV among
adolescents.
Logistic supply and funding
of the evaluation
Letter of support for visit and
interviewing of program
operators and supporters
Academic purpose: to expose
students to program evaluation
research
Share experiences from
study results
SNNPR Health
& Education
Bureaus
Coordinating and
mobilizing resources
and collaborating
with partners
Reduced Prevalence of
unintended teen
pregnancy, STIs and
HIV among
adolescents.
Letter of support for visit and
interviewing of program
operators and supporters
Information for program
improvement
Status of services provision by type
Information on population served
Conclusions and recommendations
Taking corrective
measures
Strengthening and revising
strategies
Re-plan for over all
program improvement
Share experiences from
lessons learnt
SNNPR District
Health &
Support ARH related
education provision
Reduced Prevalence of
unintended teen
Information for program
improvement
Taking corrective
measures
22. Process Evaluation of School-Based RH Education Programs
Fikru Tessema 12 April, 2008
Key
stakeholder
Role in the
program
Expectation from
the program
Area of
interest/involvement in
the evaluation
Expectation from the
evaluation
Use of evaluation
findings
Education
Offices
for in- school
youths
pregnancy, STIs and
HIV among
adolescents.
Cooperation in data
collection
Status of services provision by type
Information on population served
Conclusions and recommendations
Strengthening and revising
action plans
Re-plan for over all
program improvement
Share experiences from
lessons learnt
NGOs (Family
Guidance
Association of
Ethiopia)
Promoting RH related
education and
supporting in-and
out-of-school
programs
Reduced Prevalence of
unintended teen
pregnancy, STIs and
HIV among
adolescents.
Cooperation in data
collection
Information for program
improvement
Status of services provision by type
Information on population served
Conclusions and recommendations
Re-plan for over all
program improvement
Strengthening
management of
program resource
Capacity building in
human resources
Youth Centers,
Health Centers,
Schools
Promoting RH related
education and
providing services
Reduced prevalence of
teen pregnancy, STIs
and HIV among
adolescents.
Cooperation in data
collection
Information for program
improvement
Status of services provision by type
Information on population served
Conclusions and recommendations
Raising awareness of
adolescents
Capacity building in
human resources
Share experiences from
lessons learnt
23. Process Evaluation of School-Based RH Education Program
Fikru Tessema 13 April, 2008
2.2 PROCESS OF STAKEHOLDERS PARTICIPATION
The success of stakeholders‟ participation in the evaluation is greatly enhanced during
evaluability assessment (EA).
2.2.1 Key stakeholders involved in the process
The SNNP Regional Health Bureau , Awassa city health department and Awassa health center
were key actors in RH related education programs promotion in schools and community. There
are also many NGOs working on school health in general and some NGOs working on RH
related education in- and out-of-school in particular. FGAE is the one who intensively involved
in school-based ARH related education programs in the country.20
FGAE has begun providing ARH information and services directly to adolescent through the
establishment of youth centers in 1990. Twenty-four youth centers presently exist in different
parts of the country. Most of them are located in Addis Ababa, Jimma, Dessie and Awassa Cities.
More than 200 peers‟ educators work in youth centers and through these centers they provide a
variety of services to ISY and OSY. 21
2.3 STAKEHOLDERS COMMUNICATION
2.3.1 Communication process
Primarily, stakeholders have been consulted on the process of evaluation, basis for judgment and
main questions of evaluation. A preliminary agreement was made to continue with planning for
evaluation; hence RH related education is evaluable according to the concise evaluability
assessment results. Secondly, a consensus was also reached on sharing of information and
advisory and participatory role in resolving problems during conducting the actual evaluation.
Finally, it was aimed at conducting evaluation result dissemination for stakeholders participating
in the evaluation.
24. Process Evaluation of School-Based RH Education Program
Fikru Tessema 14 April, 2008
CHAPTER THREE: EVALUATION QUESTIONS AND OBJECTIVES
3.1 QUESTIONS OF THE EVALUATION
Evaluation questions comprises main questions for addressing the major thematic area of the
program with sub questions that will help detail information generation.40, 41
3.1.1 Main- and sub-evaluation questions
1. How in-school adolescents were reached with RH related education in schools?
To what extent were adolescents knew the importance of RH education?
What are the main topics included in RH related education in school?
To what extent each topic on RH was given in schools?
To what extent were adolescents knew the details of each topic of RH education?
How RH related education was taught in schools?
Who were the RH educators in schools?
2. To what extent were adolescents involved in extracurricular activities in schools?
What are the extracurricular activities carried out in schools?
In what extracurricular activities were adolescents participated mostly?
Were teachers and peers trained in RH in schools?
To what extent were adolescents found IEC/BCC materials relevance to convey
health information/messages on RH?
3. How adolescent RH services linked to RH related education in schools?
Why adolescents were in need for RH services?
What are RH problems of adolescents?
How adolescents have got RH services?
Who were the most RH service providers for adolescents?
How health information on RH services provided for adolescents?
What are the most health information providers for adolescents?
How often schools provided with supportive supervision with regard to adolescent
RH related education?
25. Process Evaluation of School-Based RH Education Program
Fikru Tessema 15 April, 2008
3.2 OBJECTIVES OF THE EVALUATION
3.2.1 General Objective
The overall objective of this evaluation is to assess how adolescent RH related education is
carried out in high schools in Awassa City, South Ethiopia.
3.2.2 Specific Objectives
1. To assess the enrollment status of adolescents in RH related education in high schools.
2. To assess the implementation status of extracurricular activities carried out in relation to
adolescent RH related education in high schools.
3. To assess the status of linkage between adolescent RH related education and adolescent
RH services in high schools.
3.3 CONCEPTUAL FRAMEWORK FOR THE EVALUATION
3.3.1 Theoretical Framework of Evaluation
Basically adolescent RH related issues are the bases to initiate adolescent RH related education
programs and define its context. The context of the program encompasses stakeholders and legal
framework and needs to be met, which leads to identify: (1) the availability of program input that
determine available resources, and possible alternative strategies. (2) the continuity of program
activities that examine how well plan was implemented and resources utilized. (3) the compliance
of program products with intended deliverables and retain interventions or take corrective
measures required in the future within the larger framework of the implementation process of
adolescent RH related education programs in schools. 42, 43, 44
26. Process Evaluation of School-Based RH Education Program
Fikru Tessema 16 April, 2008
Figure 2: Conceptual framework of the evaluation
Program products:
In-school
adolescents reached
with RH education,
training of
educators and
health information
on RH services
Formatting the program
Compliance
Program activities:
- RH education in
schools
- RH related
extracurricular
activities
- HI on RH services
and providers
ARH education
program & its
context:
- Program
stakeholders,
- Legal
framework
Continuity
Program
inputs:
Program
funds, HR,
strategy,
standards,
IEC/BCC
materials,
HFs
Availability
Adolescent RH related issues
Retain
interventions
or
take corrective
measures
Sustaining the program
27. Process Evaluation of School-Based RH Education Program
Fikru Tessema 17 April, 2008
CHAPTER FOUR: EVALUATION METHODS
4.1 Focus, approach and purpose of the evaluation
Focus of the evaluation: The focus of this evaluation was process of RH related education
program implementation in school that will help to understand how it really works, and “verify
what the program is and delivered to targeted recipients”.40, 45
Approach of the evaluation: The approach of this evaluation was formative that helps to know
whether existing program procedures and activities are working. It also helps to guide and refine
program components and activities that may not be seen in the program initial stage and ensure
the participation of the stakeholders in the evaluation process and utilization of the findings for
program improvement. 46
Purpose of the evaluation: This program evaluation was done to help stakeholders credibly
claim progress and success of the program or how to improve on their efforts.43
Generally, the
purpose of this evaluation is to extract relevant information on program implementation that can
subsequently be used as the basis for corrective measures and planning to make ARH related
education programs in school more effective and efficient.
4.2 Study area and period
The study was conducted in Awassa town, South Ethiopia from 23 September to 08 October 2007.
Awassa is the capital of Southern Nations, Nationalities and Peoples Regional State and located
in Sidama Zone. Awassa is about 275 km away from Addis Ababa. It has 14 administrative
„Kebeles‟ and the total population size is estimated to be 119,623, of which 60,378 are males and
59,245 are females.47
The regional health bureau is responsible for the implementation of RH
strategy in general and ARH related education for ISY and OSY. Awassa City Health
Department has an overall facilitating role in ARH related education and provision of services
through its health facility. There are three hospitals (one private and one referral Hospital owned
by Hawassa University as a teaching Hospital) and the other one is Army Hospital) and two
health centers (one governmental and one non-governmental) and several private clinics.
28. Process Evaluation of School-Based RH Education Program
Fikru Tessema 18 April, 2008
Awassa branch office of FGAE also supports and provides RH related education and services to
in- and out-of-school adolescents through its youth center and clinic. In-school ARH related
education programs are primarily supported by FGAE and some other NGOs in the city. There
are six high schools (three governmental and three non-governmental) namely Addis Ketema,
Awassa, Komboni Catholic Missionary, Adventist, SOS and Alamura senior secondary schools
in Awassa City. Of the six high schools, two high schools: Addis Ketema and Awassa senior
secondary schools included through purposive selection. The total number of students in the two
high schools who are in grades 9 and 10 were about 2925.
Figure 3: Map of Sidama Zone
4.3 Study design and technique
This evaluation employed a case study method with both qualitative in-depth interview and
quantitative survey.48
4.4 Source and study population
The source population for this evaluation includes all health professionals in the SNNPR
Regional Health Bureau, Awassa City Administration Health Department, Awassa Health Center,
and FGAE branch office with all students and teachers in high schools in Awassa city.
29. Process Evaluation of School-Based RH Education Program
Fikru Tessema 19 April, 2008
The study population includes Family Health Department of the Regional Health Bureau, Disease
Prevention and Control of Awassa Town Administration Health Department, MCH Unit of
Awassa Health Center, FGAE Awassa Branch Office Model Clinic and Model Youth Center and
Biology Department and grade 9th
and 10th
of high schools.
4.5 Inclusion and exclusion criteria
Inclusion criteria: Teachers and students from schools provided with ARH related education
and who have RH related education related experiences and can fill the questionnaire with no
assisstance, RH related service providers and program managers have been included in the study.
Exclusion criteria: Schools not promote ARH related education, health facilities with no
exclusive ARH service, individual teachers who has no experiences related to RH related
education and individual students and teachers who cannot complete the self-administered
questionnaire without assistance such as blind were not included in the study.
4.6 Sampling technique and sample size
Probability and non-probability sampling methods have been used in cases selection for the
study.49, 50
Regional health bureau and Awassa town Administration Health Department have
been selected purposively because they have a managerial role in ARH program implementation.
Awassa Health Center, FGAE Branch Office Clinic and youth center in Awassa town have been
also selected purposively because they provide RH related education and services at the facility
and the center and support RH related education in high schools.
Addis Ketema and Awassa High Schools have been selected because they are schools promoting
adolescent RH related education programs. Teachers teaching Biology in grade 9th
and 10th
of
these two high schools participated in the survey because they know about RH through teaching
RH related education, i.e., human anatomy and reproductive system in schools. Grades 9th
and
10th
from these two high schools were also selected purposively for surveying of students because
they are grades with students in the adolescence age group and target of adolescent RH related
education.
For in-depth interview, six experts: Family Health Expert of the Regional Health Bureau,
Coordinator of Disease Prevention and Control of Awassa town Administration Health
30. Process Evaluation of School-Based RH Education Program
Fikru Tessema 20 April, 2008
Department, MCH Coordinator of Awassa Health Center, FGAE Awassa Branch Office Model
Clinic and Model Youth Center Heads and Youth Counselor of the center have been interviewed.
For surveying of teachers: There was no sampling for teachers because all teachers teaching
Biology in grade 9th
and 10th
of the two high schools have been invited for the survey.
For surveying of students: The sample size was calculated by using EpiInfo version 6 StatCalc
Utility, considering the following parameters: according to an assessment done in four big
regions, unintended births among young women less than 15 years of age were about 50%, 95%
confidence level and worst acceptable value + 5%. Accordingly from the total population of 2925
students in grades 9 & 10, the required sample size was 416. Considering the non responses and
absentees of 10% gave the final sample size 458.
The total sample size distributed to the selected schools proportionate to their student population
size. By using sampling frame, the respondents distributed to each school were selected from all
sections of the two grades randomly generated by MS Excel. All assigned students have been
then invited to participate in the survey.
Figure 4: Sampling Scheme
4.7 Data collection instruments development
An in-depth interview guide (Annex 1) for program supporters and service providers and closed-
ended questionnaires (Annex 2 & 3) for teachers and students (program recipients) have been
used to get evidence/data from multiple sources that increased opportunities for the investigator
to explore about the program across multiple sources of evidences. The closed-ended
Aawasa & Addis
Ketema High Schools
Awasa FGAE
Branch Office
Clinic
(HC)
Youth
Center
(H&YC)
Awassa City
Health
Department
(DPCC)
Awassa
Health Center
(MCHC)
SNNP
Regional
Health Bureau
(FHE)
Biology teachers
Students in
grade 9&10
Public sector NGO sector
31. Process Evaluation of School-Based RH Education Program
Fikru Tessema 21 April, 2008
questionnaires were standard questionnaires adopted from WHO, FHI and EPHA standard
questionnaires. They were translated to Amharic and back translated to English for checking of
consistent translation and finally was administered in Amharic.
4.8 Data collection procedure
The principal investigator did the in-depth interviews. Two supervisors and four data collectors
were hired to facilitate surveying of students and teachers. The supervisors were senior and
bachelor holders with health background. The data collectors were also bachelor holders with
health/social science background. They were divided into two teams during data collection to
complete data collection in maximum of three days to avoid contamination. Visits were made to
the selected schools to obtain student lists to develop sample frame and identify the respondents.
The in-depth interview and survey time were arranged with the agreement of the respondents and
Unit Leaders in the high schools.
4.9 Data analysis procedure
The collected quantitative data first checked for completeness and internal consistency. Then the
data entered in to EpiInfo version 3.3.2 with coding by using double data entry technique. The
entered data cleaned by using Compare Utility of EpiInfo version 3.3.2 to work on clean data for
analysis. It was analyzed by using EpiInfo version 3.3.2 and SPSS version 12.0.1 by running
simple frequency distribution and the results presented using tables with cross tabulations and
graphs.
The qualitative field notes first entered in to the computer with placing in to arrays. A description
of the evidence was made from the qualitative data placed in a matrix of categories with thematic
and content analysis
4.10 Data quality management
Pre-test: A pre-test study was conducted to maximize validity and reliability of the study
instruments. A questionnaire for surveying of students were tested by taking one section from
grade 9th
of one high school and 10th
of another high school, which were not included in the
survey. A questionnaire for surveying of teachers also tested by taking Biology teachers teaching
32. Process Evaluation of School-Based RH Education Program
Fikru Tessema 22 April, 2008
in grade 10th
of one high school, which was not be included in the survey. Accordingly
corrections were made for appropriate wording, clarity and consistency of questions. At the end
of the pretest, discussion were held with the respondents on skipping pattern, sensitiveness of the
questions, and their honest and frank response, the relevance of the study.
Training: Data collectors were familiarized with the instruments through training. They have
been given three days training on surveying techniques before and after pre-testing by using
training guide (Annex 5).
Supervision: The principal investigator had overall project coordination. A daily supervision and
follow up were done by principal investigator with the assistance of supervisors. Data collectors
submitted the completed questionnaires every day to supervisors and were checked for
completeness with principal investigator.
Finally, the draft report was presented to study participants who participated in in-depth
interviews to check for proper presentation of results and comments. The IDIs field notes and
analyzed questionnaires were also kept well for further cross checking.
Meta Evaluation: Moreover, before the execution of this evaluation project, it was reviewed
against the pertinent standards for evaluating evaluation projects by using utility, feasibility,
accuracy and propriety standards.51
Accuracy: to ensure the accuracy of data generated by the evaluation, all the data collection,
analysis, and presentation techniques were planned based on scientific methods. Information to
be gathered was maximized by using multiple data sources and triangulation of data and carrying
out in-depth interview by the investigator and data collection by qualified data collectors by using
standard questionnaires. Furthermore, the proposal presented to proposal committee for defense
and revision.
Utility: to ensure the utilization of the findings, this evaluation plan was communicated with
stakeholders and participated in facilitating data collection. Thus, there was a high likelihood of
addressing the information needs of stakeholders; hence, they use findings for program
improvement.
33. Process Evaluation of School-Based RH Education Program
Fikru Tessema 23 April, 2008
Feasibility: to ensure the feasibility of the evaluation, since ARH related education programs in
school are programs going on for the last decades, this increases the availability of data required
for the evaluation. Moreover, all the techniques proposed for data collection were so easy to
implement. In addition, there was a possibility to hire human resources required for the
evaluation and support from stakeholders during the process of evaluation.
Propriety: The absence of procedures that may cause untoward effects on participants,
stakeholders, program managers and others involved along with the issues stated in the “Ethical
Consideration” was not a problem. Beside the ethical clearance from Jimma University, the
Regional Health Bureau Research and Laboratory Department also cleared the ethical issues.
4.11 Ethical Consideration
Ethical clearance: Ethical clearance for the protocol was obtained from Jimma University,
Public Health Faculty Ethical Clearance Committee and from SNNP Regional Health Bureau
Research and Laboratory Department prior to its implementation.
Confidentiality: Respondents‟ view and opinion treated as confidential and anonymous. With
regard to protecting participants‟ confidentiality, participants‟ identities were protected and
respected during final presentation of the data in public dissemination events, as well as in
printed publications.
Informed consent: Informants informed about the evaluation research in a way they can
understand, finally reached on consensus and have got verbal consent with signed consent form
by data collectors (Annex 4). The information to informants included: the purpose of the
evaluation research, how confidentiality protected and expected benefits.
Letter of support: JU and MoH signed a letter of support for cooperation of regional officials.
The principal investigator contacted officially with Regional Health Bureau, City Administration
Health Department, FGAE Branch Office, Principals of the selected schools, and Unit Leaders of
each school. There was good acceptance and cooperation in the whole process of data collection.
34. Process Evaluation of School-Based RH Education Program
Fikru Tessema 24 April, 2008
4.12 Operational Definitions
Adolescent: is defined as those who are in the age-group of adolescents between 10 and 24 years
old (UNICEF, WHO definition).
Availability: used as evaluation dimension that refers to ARH related education program
resources across providers and recipients. It is measured in terms of strategic program
components running in schools, youth centers and health facility
Case Study: An in-depth examination of ARH related education programs in schools, with
multiple sources of information from program experts and recipients to provide as
complete picture as possible.
Compliance: used as evaluation dimension that refers to the implementation of each ARH
related education program component in line with the intended strategies and standards
across providers and recipients. It is measured in terms of fulfillment of strategic program
component implementation.
Continuity: used as evaluation dimension that is the measure of the extent to which the program
for specified users is provided over time across providers and recipients. It is measured in
terms of strategic program components operating over time.
Evaluation standards: Standards are predetermined cut-off points (>90%, 75-89%, 50-74% and
<50%) agreed with stakeholders for judging performances.
Expert: A person who knows about adolescent reproductive health education programs in
schools.
Indicators: They are input, process and output indicators for which data were collected to
evaluate adolescent RH related education program implementation in schools.
Knowledge: refers to the ability of teachers and adolescents in schools to explain at least two to
three ARH problems and their prevention methods, type of health information on RH and
RH services for adolescents.
Parameters for judgment: Judgment parameters (successful, adequate, fair and in-adequate) are
used to describe the level of implementation of program components by comparing
standards with findings of the evaluation.
35. Process Evaluation of School-Based RH Education Program
Fikru Tessema 25 April, 2008
4.13 Matrix of Analysis for Judgment
Matrix of analysis for judgment developed with consultation of stakeholders and comprised
program objectives and activities, core evaluation dimensions (availability, compliance and
continuity), core evaluation indicators, data sources for indicators, tools to collect data, weight
and values for program activities and achievement, standards for evaluation and parameters for
judgment.
4.14 Evaluation Report Dissemination Plan
The findings were presented to the IDIs‟ participants to validate the findings and incorporated
their comments. The dissemination plan also will comprise presenting the evaluation results to
different stakeholders by distributing hard copies of the evaluation result reports. The final report
will be prepared after this final examination and electronic (PDF format) and hard copies will be
submitted to Jimma University. It is also planned to publish on peer reviewed journals.
36. Process Evaluation of School-Based RH Education Program
Fikru Tessema 26 April, 2008
CHAPTER FIVE: RESULTS OF THE EVALUATION
5.1 QUANTITATIVE RESULTS
5.1.1 Socio-Demographic Characteristics of the Study Population
A total of 16 Biology teachers participated in the survey carried out in the two high schools
located in Awassa town with 94.0% (16 out of all 17 teachers) response rate. Hence, the final
analysis was made based on 16 completed questionnaires. About 11 of the respondents were male
and 5 female respondents. The age range varied from 32-56 years and the mean age was 43.6
years (SD.7.72). The predominant ethnic group of the respondents is Amhara (8) followed by
Guragie (3). Of the total respondents, 11 were Orthodox followers and 4 Protestant (see Table 1).
Table 1: Socio-Demographic Characteristics of the Respondents (Biology teachers) by selected
characteristics, Awassa town, October, 2007
Characteristics
Respondent’s
Number %
Age 16 100%
30-40 5 31.3%
41-50 8 50.0%
51+
3 18.8%
Sex 16 100.0%
Male 11 68.8%
Female 5 31.2%
Education 16 100.0%
First degree 15 93.8%
Second degree 1 6.2%
Ethnicity 16 100.0%
Amhara 8 50.0%
Guragie 3 18.9%
Wolaita 2 12.5%
Sidama 1 6.2%
Oromo 1 6.2%
Missing 1 6.2%
Religion 16 100.0%
Orthodox 11 68.8%
Protestant 4 25.0%
Other 1 6.2%
A total of 436 in-school adolescents participated in the survey carried out in two high schools
located in Awassa town with response rate of 95.0%. Hence, the final analysis was made based
on 436 completed questionnaires (see Table 2).
37. Process Evaluation of School-Based RH Education Program
Fikru Tessema 27 April, 2008
Table 2: Socio-Demographic Characteristics of the Respondents (in-school adolescents) by selected
characteristics, Awassa town, October, 2007
Characteristics
Respondent’s
Number %
Age 436 100%
12-14 54 12
15-19 360 83
20-24 22 5
Sex 436 100.0%
Male 269 61.7%
Female 167 38.3%
Grade 436 100.0%
Grade 9 236 54.1%
Grade 10 200 45.9%
Ethnicity 436 100.0%
Sidama 218 50.0%
Wolaita 68 15.6%
Amhara 62 14.2%
Oromo 37 8.5%
Guragie 26 6.0%
Others 24 5.5%
Missing 1 0.2%
Religion 436 100.0%
Protestant 260 59.5%
Orthodox 111 25.5%
Muslim 27 6.2%
Catholic 27 6.2%
Other 9 2.1%
Missing 2 0.5%
With whom they live currently 436 100.0%
Parents 315 72.2%
Relatives 78 17.9%
Other 43 9.9%
Educational status of parents 436 100.0%
Unable to read and write 33 7.6%
Can read and Write 60 13.8%
1-4 grade 37 8.5%
5-8 grade 65 14.9%
9-12 grade 113 25.9%
Above grade 12 127 29.1%
Missing 1 0.2%
Occupational status of parents 436 100.0%
Self employed 248 56.8%
Civil servant 133 30.5%
Teacher 29 6.7%
Health worker 26 6.0%
As it is indicated in Table 2, out of the total respondents to the student survey, 54 (12%) were in
the age group 12-14 years, 360 (83%) in 15-19 years and 22 (5%) in 20-24 years. The age range
varied from 12-24 years and the mean age was 16.2 years (SD.1.83). About 61.7% of the
respondents were male and 38.3% female. The predominant ethnic group of the respondents is
Sidama 50.0% followed by Wolaita 15.6% and Amhara 14.2%. About 59.6% of the respondents
were protestant followers, 25.5% Orthodox, 6.2% Muslim and 6.2% Catholic.
38. Process Evaluation of School-Based RH Education Program
Fikru Tessema 28 April, 2008
5.1.2 Provision of RH related education for adolescents in high schools
The student survey revealed that 368 (84.4%) of the total respondents, reported being taught RH
related education in schools. Regarding topics included in RH related education in schools, they
mostly reported HIV/AIDS (76.4%), pregnancy (64.9%) and STIs (48.4%) (see Table 3).
Table 3: Distribution of respondents, who were taught RH related education and cited types of RH
related education topics, Awassa town, October, 2007
Variables
Respondents
Yes %
Being taught RH related educations in school 368 84.4%
Type of topics included in ARH related
education
HIV/AIDS 281 76.4%
Pregnancy 239 64.9%
STIs 178 48.4%
Abortion 113 30.7%
Communication skill 87 23.6%
Concept of RH 61 16.6%
Physical and social development 52 14.1%
Negotiation skills 32 8.7%
Harmful traditional practice 5 1.4%
Note: Because of multiple responses the percentages did not add up to 100%.
Of the total respondents to student survey, 408 (93.5%) knew methods of avoiding pregnancy.
Contraceptive (67.9%), use of condom (62.7%) and abstinence (57.6%) were the most commonly
reported methods of avoiding pregnancy. About 413 (94.7%) respondents also knew methods of
preventing STIs. Abstinence (64.9%), treatment (49.9%) and use of condom (47.3%) were also
the most frequently reported methods of preventing STIs. The majority (99.3%) of respondents
also knew methods of prevention of HIV/AIDS transmission. mostly reported abstinence
(70.2%), get married/one-to-one (67.7%) and use of condom (65.0%) were the most commonly
mentioned methods of preventing HIV/AIDS transmission (see Table 4).
39. Process Evaluation of School-Based RH Education Program
Fikru Tessema 29 April, 2008
Table 4: Percentage of respondents to student survey, who cited different methods of avoiding
pregnancy and preventing STIs/HIV transmission, Awassa town, October, 2007
Variables Cited methods of
prevention Total
Freq
Yes %
Methods to avoid pregnancy 408 (93.5%)
Contraceptive 277 67.9%
Use condom 274 67.2%
Abstinence 235 57.6%
Female sterilization 59 14.5%
Male sterilization 58 14.2%
Natural method (menstrual cycle, ejaculation
outside of female organ)
12 2.9%
Methods to prevent STIs 413 (94.7%)
Abstinence 268 64.9%
Treatment 206 49.9%
Use condom 195 47.3%
Methods to prevent HIV/AIDS 433 (99.3%)
Abstinence 304 70.2%
Use condom 281 65.0%
Get married/one-to-one 293 67.7%
Avoid syringes and needles sharing 166 38.4%
Break transmission from mother to child 137 31.6%
Note: Because of multiple responses the percentages did not add up to 100%.
With regard to onset of sex, about 70.4% of the respondents were also able to report 20 years and
above as an ideal age to commence sexual intercourse or marry. About 52.1% respondents knew
that a girl can get pregnant with first time of sexual intercourse (see Table 5).
40. Process Evaluation of School-Based RH Education Program
Fikru Tessema 30 April, 2008
Table 5: Percentage of respondents to student survey who cited ideal age of onset of sex and getting
pregnant for a girl with first time of sexual intercourse, by selected characteristics,
Awassa town, October, 2007
Variables Characteristics Total
Male % Female % Freq. %
Ideal age to commence sexual
intercourse or marry
269 61.8% 167 38.2% 436 100.00%
17 years 7 1.6% 4 0.9% 11 2.5%
18 years 74 17.0% 44 10.1% 118 27.1%
20 years and above 188 43.2% 119 27.2% 307 70. 4%
Getting pregnant for a girl with
first time of sexual intercourse
269 61.8% 167 38.2% 436 100.0%
Yes 137 31.4% 90 20.6% 227 52.1%
Note: Because of multiple responses some percentages did not add up to 100%.
5.1.3 Adolescent reproductive health education session in high schools
An attempt that was made to know adolescent RH related education sessions in high schools
showed that RH related education was being taught in high schools during Biology class with
related topics; some times during campaigns and public events (see Table 6).
Of the total respondents to student survey, 362 (83.0%) also reported having RH related
education sessions some times and 6 (1.4%) mentioned during public events. During the student
survey, 375 (86.0%) of the total respondents feel that having reproductive health education in
school is important in relation to its problems (see Table 6).
41. Process Evaluation of School-Based RH Education Program
Fikru Tessema 31 April, 2008
Table 6: Percentage of respondents who cited RH related education sessions and its
importance and teachers’ feeling in teaching RH in class, Awassa town, October,
2007
Variables
Teachers Students
M F Total M % F % Total
RH related education
sessions
Some times
6 1
7
215 98.2% 147 98.7%
362
During public event
3 3
6
4 1.8% 2 1.3%
6
During Biology
class
1 0
1
- - - -
Campaigns
1 1
2
- - - -
Teachers comfortable
with teaching ARH 9 3
12
- - - -
Importance of RH
related education
11 5
16
238 63.5% 137 36.5%
375
5.1.4 Extracurricular activities of ARH related education
Furthermore, the student survey revealed that 285 (65.4%) of the total respondents have
participated in extracurricular activities related to RH related education that have been carried out
in high schools, of which 29.0% respondents participated in drama club, 23.2% in mini media,
32.3% in health club, 14.0% in RH IEC/BCC and others 1.1% in harmful traditional club, youth
counseling club (see Figure 5).
42. Process Evaluation of School-Based RH Education Program
Fikru Tessema 32 April, 2008
Figure 5: RH service providers reported by adolescents
5.1.4.1 Supply and distribution of IEC/BCC materials on RH
The student survey also revealed that about 38.8% of the total respondents read IEC/BCC
materials on RH. Most respondents were able to list some IEC/BCC materials on RH. HIV/AIDS
posters, FGM posters, leaflets on pregnancy, FGM, abortion, STIs and early marriage were the
most commonly mentioned materials (see Table 7).
5.1.4.2 Training of teachers and peer educators
The teachers survey results showed that 6 (37.5%) of the respondents were trained by FGAE
youth center and 10 respondents were not trained. The student survey also revealed that 146
(33.5%) of the total respondents were trained in ARH (see Table 7).
43. Process Evaluation of School-Based RH Education Program
Fikru Tessema 33 April, 2008
Table 7: Percentage of respondents who participated extracurricular activities of RH related
education in high schools for adolescents, Awassa town, November, 2007
Variables
Respondents
Yes %
Adolescents who thought that IEC/BCC
materials are important to convey RH
information
158 93.5%
Adolescents who read IEC/BCC materials on
RH
169 38.8%
Teachers training in ARH related education
6 37.5%
Peers training in ARH related education
146 33.5%
Teachers read IEC/BCC materials on ARH
5 31.2%
Note: Because of multiple responses the percentages did not add up to 100%.
5.1.5 Linkage between ARH related education and ARH services
In this regard, it was revealed that there were health information on youth friendly RH services
and RH service providers. Of the total respondents to a student survey, 367 (84.2% had
information about RH services and RH service providers. Mass media (45.5%) and health club
(43.6%) were the commonly mentioned sources of health information about RH services (see
Figure 6). Health facility (78.2%) and youth center (40.0%) were also the commonly mentioned
RH service providers. Of the total respondents, 204 (46.8%) were also knew counseling &
referral services providers in high schools. Health club members (49.5%) were the most
frequently mentioned counseling & referral services providers in high schools. In teacher survey,
it was also found that 11 of the respondents reported that they participated in counseling &
referral services for in-school adolescents with RH problems (see Table 8).
44. Process Evaluation of School-Based RH Education Program
Fikru Tessema 34 April, 2008
Figure 6: Sources of information on RH services reported by adolescents
Note: Because of multiple responses the percentages did not add up to 100%.
Table 8: Sources of information about RH services and its providers, counseling and
referral services in school, Awassa town, October, 2007
Variables Reported
Total
Freq.
Yes
Freq
%
ARH service providers
367 (84.2%)
Health facility 287 78.2%
Youth center 150 40.9%
Drug store/pharmacy 78 21.3%
Traditional healers 36 9.8%
Other (NGOs) 1 0.3%
Counseling and referral service providers in
school
204 (46.8%)
Health club 101 49.5%
Trained peers 74 36.3%
Teachers 73 35.8%
Others (Youth counseling club) 5 2.5%
Teachers participated in counseling & referral
services
11 68.8%
Note: Because of multiple responses the percentages did not add up to 100%.
45. Process Evaluation of School-Based RH Education Program
Fikru Tessema 35 April, 2008
5.1.6 RH problems of adolescents
The student survey results also showed that about 94.5% of the total respondents thought that
adolescents faced RH problems. Regarding major RH problems of adolescents, the most
frequently mentioned RH problems of adolescents were HIV/AIDS (71.6%) and followed by
unwanted/unintended pregnancy (68.0%) and STIs (43.0%) (see Table 9).
Table 9: Distribution of RH related problems of in-school adolescents and their exposure to
information on ARH services by selected characteristics, Awassa town, October, 2007
Variables Characteristics Total
Male
Freq
% Female
Freq
%
Freq %
Adolescents need for RH services
cited by teachers
11 68.7% 5 31.3% 16 100.0%
Yes 10 62.5% 5 31.3% 15 93.8%
Do not know 1 6.2% 0 0.0% 1 6.2%
Adolescents faced RH problems
269 61.7% 167 38.3% 436 100.0%
Yes 255 58.5% 157 36.0% 412 94.5%
No 5 1.1% 2 0.5% 7 1.6%
Do not know 9 2.1% 8 1.8% 17 3.9%
RH problems of adolescents
HIV/AIDS 188 45.6% 107 26.0% 295 71.6%
Unintended pregnancy 182 44.2% 98 23.8% 280 68.0%
STIs 112 27.2% 65 15.8% 177 43.0%
Lack of information on RH
services 103 25.0% 48 11.7% 151 36.7%
Abortion 81 19.7% 67 16.3% 148 35.9%
Lack of information how to
prevent pregnancy 72 17.5% 39 9.5% 111 26.9%
Lack of information how to
prevent STI/HIV 69 16.7% 32 7.8% 101 24.5%
Lack of emergency
contraceptive 15 3.6% 11 2.7% 26 6.3%
Others (Rape, FGM, etc) 10 2.4% 5 1.2% 15 3.6%
Note: Because of multiple responses some percentages did not add up to 100%.
46. Process Evaluation of School-Based RH Education Program
Fikru Tessema 36 April, 2008
5.2 QUALITATIVE RESULTS
A total of 6 experts‟ in-depth interviews were conducted to extract relevant information how RH
related education for adolescents were provided in-school adolescents in Awassa City. Experts from
Regional Health Bureau, Awassa town Health Department, Awassa Health Center, and FGAE
Model Clinic and Youth Center participated in the interviews. Of the respondents, five were male
and one female.
5.2.1 Provision of RH related education for adolescents in high schools
The findings of the IDIs indicated that all program activities were targeting the implementation of
National RH strategy (2006-2015). All respondents explained that there is no exclusive regional
or internal strategy for ARH related education in schools. They use the National RH strategy to
facilitate the provision of ARH related education in schools. In the IDIs, the respondents
explained that adolescent RH related education was integrated in family health program, one of
the components of HSDP (2006 – 2010) at regional and city administration health department
level. It was also included in the five-year (2005-2009) strategic plan at youth center level.
With regard to funds for the provision of RH education, the IDIs respondents explained that there
is no exclusive fund, but, a lump sum fund was allocated for RH program that included
adolescent RH IEC/BCC. Consistent with this, RH education for adolescents in high schools is an
ongoing program. It is part of health education programs. The budget for RH programs did not
have breakdown for adolescent RH education. There was no attempt made to solicit exclusive
funds for adolescent RH education. The budget allocation is based on priority health programs. It
was also reported that there was a shortage of funds at city administration health department,
health center and youth center level for adolescent RH activities in general. Hence, there were
insufficiencies of funds for RH education in general.
5.2.2 ARH related education session in high schools
Concerning ARH related education session in high schools; the findings of the IDIs indicated that
there were no regular sessions carried out by public sector and NGOs. Hence, it was not included
47. Process Evaluation of School-Based RH Education Program
Fikru Tessema 37 April, 2008
in school-time table as a subject. It was carried out by collaborating with schools during outreach
services and special performances like Regional or National public events.
5.2.3 Importance of ARH related education in schools
Regarding the importance of ARH related education in high schools, all respondents explained
that schools are the best place to work on ARH related education by including in school time
table as a subject.
5.2.4 Extracurricular activities of ARH related education
The IDIs with respondents revealed that there are some initiatives of extracurricular activities of
ARH related education in high schools. Among some initiatives: the provision of schools with
trained peer RH services providers; out-reach RH related education to high schools with the help
of school principals and club coordinators in high schools; drama and music show including
poems on RH were the major mentioned initiatives.
5.2.4.1 School clubs, mini media and peer educators
All IDIs respondents were mentioned that school clubs, mini media and peers were supported by
public sector and youth center to build their capacity and enhance their involvement in health
issues of in-school adolescents. The support was mainly focused on IEC/BCC materials provision
to use them as service outlets for IEC/BCC materials distribution in high schools. Hence, the
members of the clubs lack adequate refresher courses to up-date their knowledge of RH.
5.2.4.2 Training of teachers and peer educators
According to the IDIs respondents‟ report, currently the trainings were not carried out regularly
every year because of budget constraints and its inconsistent availability. Peer educators‟ family
life education, adolescent sexual RH and counseling trainings in 2006 and 2007 were some of the
trainings carried out at youth center.
5.2.4.3 Supply and distribution of IEC/BCC materials on RH
According to the IDIs respondents‟ report, there were two approaches in supply and distribution
of IEC/BCC materials on RH. There is hierarchical approach from federal to region and local
48. Process Evaluation of School-Based RH Education Program
Fikru Tessema 38 April, 2008
authorities or from head quarter to branch offices. The other approach is printing some IEC/BCC
materials at local level, in which the regional health bureau and FGAE Model Clinic and Youth
Center were the one who printed IEC/BCC materials on RH for adolescents.
One of the respondents said, “…the IEC/BCC materials supplied to high schools were too small
amount, it cloud not reach all in-school adolescents because they are many in number.”
Concerning the type of IEC/BCC materials, most IDIs participants explained that most materials
are hard copies and there are also some electronic materials (CDs). Similarly, one of the
respondents also said, “…IEC/BCC materials on RH printed and produced and distributed to
services delivery points in FGAE Model Clinic, client youths/adolescents coming from schools,
colleges, and other youth groups, and client youths come from Anti-AIDS clubs in- and out-of
schools.”
The high schools reached at district level for their supply of IEC/BCC materials on RH. High
schools were highly benefited from IEC/BCC materials distributed by the youth center. VCT
posters, VCT brochures, leaflets on gender and contraceptive and take home or pocket booklets
were the most usually supplied and distributed materials. During public events by sending the
materials to high schools without charge, peer educators to their colleagues in high schools were
the means of distribution of IEC/BCC materials. In this regard, one of the respondents said, “We
distributed IEC/BCC materials to every high school in the region… this is our best side…”
5.2.5 Linkage between ARH related education and ARH services
In this regard, an attempt made to know how linkage was there between ARH related education
and ARH services. The IDIs respondents explained that the linking points are part and parcel of
extracurricular activities of ARH related education in high schools. These are health information
on ARH services providers, counseling for RH problems and referral for ARH services.
The IDIs with respondents revealed that it is an integral part of health services systems where all
health services providers were expected to provide health information on available services. It
was scheduled in a routine health services for adolescents. Use of audio-videos and one-to-one
communication, printed materials like brochures at service delivery points, morning IEC/BCC
49. Process Evaluation of School-Based RH Education Program
Fikru Tessema 39 April, 2008
sessions, local radio media and outreach services were the major commonly mentioned means of
disseminating health information on RH services.
With regard to supportive supervision to high schools: the IDI participants explained that
there is no regular supportive supervision to high schools. Sometimes high schools contacted
during public events, in conjunction with campaigns and outreach services.
5.2.6 RH problems of adolescents
In the IDIs with the attempt made to know major RH problems of adolescents, they need free
services, confidentially caring, welcoming and warming reception approaches. Cultural barriers,
secretive to discuss openly, male dominance and lack of awareness were also the commonly
mentioned RH problems of adolescents.
An 18 years old male 10th
grade student, he made a comment in relation to reproductive health
problems, for which they need RH services, he said that ….. we, Ethiopians are in a competition
with population growth not in economy; abandon this image; we, ourselves do struggle.
The direct Amharic version of his comment says, “… እኛ ኢትዮጵያዉያኖች በኢኮኖሚ ሳይሆን በህዝብ ብዛት
ነዉ እየተወዳዯርን ያሇነዉ ይህን ገጽታ ቀይሩልን እኛም እንታገላሇንÝÝ”
When mentioning roles of parents and teachers: One of the IDIs respondents said, “… they need to develop
the habit of free and open discussion on sexuality issues with children and other siblings.” Similarly, one of the
respondents also said, “… parents, teachers and peers need to explain states of body/physical development and
RH problems as they exist in reality and facilitate youth friendly environment in which adolescents discuss and
find out healthy ways of dealing with the problems by themselves.”
When mentioning initiatives for disseminating health information on RH: One of the IDIs participants
explained that “youth dialogue” is a new initiative by which health information on RH education was
disseminated for adolescents. It is designed by youth center to make a discussion by the youths among
themselves to share ideas and find out healthy ways of dealing with RH issues by themselves.
50. Process Evaluation of School-Based RH Education Programs
Fikru Tessema 40 April, 2008
5.2.7 Judgmental Matrix of Analysis for Program Activities and Program Outputs
Program
Objectives
Program
activities
Evaluation
Dimensions
Core Evaluation Indicators Data sources Tools Weight Value Findings Evaluation
standards
Parameters for
judgment
To reach in-
school
adolescents
with knowledge
and skills
needed to foster
and sustain
health-
affirming
behavior.
Provide RH
related education
in school for
adolescents
Compliance with
teaching RH and type
of RH topics in school
for adolescents
(1) No of in-school adolescents
who reported being taught
RH related education in
schools
Survey Survey
questionnaire
15 12.7 84.4%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(2) No of in-school adolescents
who reported contraceptive
as a method of avoiding
pregnancy
Survey Survey
questionnaire
10 6.8 67.9%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(3)No of in-school adolescents
who reported use of condom
as a method of avoiding
pregnancy
Survey Survey
questionnaire
10 6.3 62.7%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(4) No of in-school adolescents
who reported abstinence as a
method of prevention of
STIs
Survey Survey
questionnaire
10 6.5 64.9%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(5) No of in-school adolescents
who reported treatment as a
method of prevention of
STIs
Survey Survey
questionnaire
10 5.0 49.9%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(6) No of in-school
adolescents who reported
abstinence as a method of
prevention of HIV/AIDS
Survey Survey
questionnaire
10 7.0 70.2%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(7) No of in-school adolescents
who reported use of
condom as a method of
prevention of HIV/AIDS
Survey Survey
questionnaire
10 6.5 65.0%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Compliance with
informing ideal age
for onset of sex and
getting pregnancy
with first sexual
intercourse for a girl
(8) No of in-school adolescents
citing 18 years and above
Survey Survey
questionnaire
5 4.9 97.5%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(9) No of in-school adolescents
who reported getting
pregnant with first sexual
intercourse for a girl
Survey IDI guide
Survey
questionnaire 5 2.6 52.1%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Compliance with
arranging RH
education sessions and
its importance in
(10) No of in-school
adolescents reported having
RH related education
sometimes
Survey Survey
questionnaire
10 8.3 83.0%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
51. Process Evaluation of School-Based RH Education Programs
Fikru Tessema 41 April, 2008
Program
Objectives
Program
activities
Evaluation
Dimensions
Core Evaluation Indicators Data sources Tools Weight Value Findings Evaluation
standards
Parameters for
judgment
schools for adolescents (11) No of in-school
adolescents reported having
RH related education is
important
Survey Survey
questionnaire
5 4.3 86.0%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Overall for providing RH related education in school for adolescents 100 70.6 70.6% 50-74% Fair
Participate
adolescents in
activities related
to RH related
education in
schools
Compliance with
participating
adolescents in
extracurricular
activities
(1) No of in-school adolescents
participated in
extracurricular activities of
RH related education
Survey Survey
questionnaire
100 65.4 65.4%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Train school
teachers in RH
Availability of
training for teachers
(1) No of trained teachers in
RH
Survey Survey
questionnaire
80 30.0 37.5%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Continuity of training
for teachers
(2) No of training for teachers
in RH conducted every year
Survey Survey
questionnaire
20 0.0 0.0%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Overall training of school teachers in RH 100 30.0 30.0% <50% Inadequate
Train peer
educators in
RH
Availability of
training for
adolescents
(1) No of in-school adolescents
trained in peer education
Survey Survey
questionnaire
80 26.8 33.5%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Continuity of training
for adolescents
(2) No of training for in-school
adolescents in peer
education conducted every
year
Survey Survey
questionnaire
20 0.0 0.0%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Overall training of peer educators in RH 100 26.8 26.8% <50% Inadequate
Supply and
distribute RH
related IEC/BCC
materials
Compliance with
accessing relevant
IEC/BCC materials on
RH for adolescents
(1) No of in-school adolescents
read IEC/BCC materials
Survey Survey
questionnaire
40 15.5 38.8%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(2) No of adolescents thought
that IEC/BCC materials
were important to convey
RH information
Survey Survey
questionnaire
60 56.1 93.5%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Overall supply and distribute RH related IEC/BCC materials 100 71.6 71.6% 50-74% Fair
To increase
access and
utilization of
adolescent
Aware youths
on minimum
package of RH
services
Availability of HI
dissemination on RH
services and providers
in schools
(1) No of in-school adolescents
who have HI on RH services
and providers
Survey Survey
questionnaire
70 58.9 84.2%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(2) No of in-school adolescents Survey Survey 20 9.1 43.6% >90% Successful
52. Process Evaluation of School-Based RH Education Programs
Fikru Tessema 42 April, 2008
Program
Objectives
Program
activities
Evaluation
Dimensions
Core Evaluation Indicators Data sources Tools Weight Value Findings Evaluation
standards
Parameters for
judgment
reproductive
health services
who reported health club as
a source of HI on RH
services
questionnaire 75-89%
50-74%
<50%
Adequate
Fair
Inadequate
(3) No of in-school adolescents
who reported health facilities
as RH services providers
Survey Survey
questionnaire
10 7.5 75.2%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Overall awareness of youths on minimum package of RH services 100 75.1 75.1% 75-89% Adequate
Provide
counseling on
RH and sexual
issues
Compliance with
addressing RH
problems of
adolescents
(1) No of in-school adolescents
who recognize adolescents
encountered RH problems
Survey Survey
questionnaire
100 94.5 94.5%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Create referral
linkage between
RH service
outlets
Availability of
counseling & referral
services
(1) No of in-school adolescents
who knew counseling &
referral services providers in
high schools
Survey Survey
questionnaire
30 14.0 46.8%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(2) No of in-school adolescents
reported health club
members as counseling &
referral services providers
Survey Survey
questionnaire
20 9.9 49.5%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(3) No of in-school adolescents
reported trained peers as
counseling & referral
services providers in high
schools
Survey Survey
questionnaire
20 7.3 36.3%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
(4) No of in-school adolescents
reported teachers as
counseling & referral
services providers in high
schools
Survey Survey
questionnaire
20 7.2 35.8%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Availability of
supportive
supervision to schools
(5) No of supportive
supervision carried out to
schools with regard to ARH
related education
Survey Survey
questionnaire
10 0.0 0.0%
>90%
75-89%
50-74%
<50%
Successful
Adequate
Fair
Inadequate
Overall linkage between adolescent RH service outlets 100 38.4 38.4% <50% Inadequate
Over all program achievement 800 472.4 59.1% 50-74% Fair
Note: Texts in bold and italics indicate the status of program achievement as compared to evaluation standards with judgment parameter. Weights, Standards
and Parameters are expert judgment and literature values and agreed with stakeholders for this program evaluation for judgment. Findings are what
program achieved as immediate results.
53. Process Evaluation of School-Based RH Education Programs
Fikru Tessema 43 April, 2008
CHAPTER SIX: DISCUSSION
6.1 DISCUSSION
This evaluation of the program has attempted to assess how ARH related education program was
going on and working in high schools by focusing only on the process of the program
implementation. Over all there was high response rate in teacher and student survey and in-depth
interviews of experts with the participation of both male and female.
In this evaluation it is evident that RH related education provision in high schools were found
to be adequate (84.4%) in compliance with the intended RH related education provision for in-
school adolescents as compared to the agreed evaluation standards. In addressing topics to be
included in RH related education for adolescents in schools were also found to be fair in
compliance with the intended major topics to be included in RH related education. This was
showing to be true by the fact that most respondents were able to report some methods of
avoiding pregnancy (mostly contraceptive and use of condoms) and preventing STIs and
HIV/AIDS transmission (mostly abstinence and use of condoms). In a study done on in-school
youth RH in eight regions in Ethiopia (21), indicated that use of condom and abstinence were
most commonly cited methods of avoiding pregnancy and prevention of STIs and HIV/AIDS.
This suggests no change with regard to mentioning methods of avoiding pregnancy and
prevention of STIs and HIV/AIDS by in-school adolescents.
In educating in-school adolescents about ideal age for commencing sexual intercourse was
successful in compliance with achieving the level of awareness of in-school adolescents, wherein
97.5% of respondents reported 18 years and above as an ideal age for onset of sex. Teaching in-
school adolescents about getting pregnant with first sexual intercourse for a girl was fair in
compliance with achieving the level of awareness of in-school adolescents. A study done on
youth RH need and services in four selected regions of Ethiopia (20) showed 18 years as a
minimum age and 20 and above as an ideal age for onset of sex.
The in-depth interviews (IDIs) also showed that all program activities were in line with the
National RH strategy to facilitate the provision of ARH related education in schools. The ARH
related education in high schools exists as an integral part of the health care delivery system in
54. Process Evaluation of School-Based RH Education Programs
Fikru Tessema 44 April, 2008
the health sector. Thus, it was addressed in family health at regional health bureau and city
administration health department level and in routine health services at health facilities and youth
center level.
The qualitative study also identified that the FGAE Model Youth Center has extended approach.
The focal person assisted by trained peer service providers and educators that enhances reaching
in-school adolescents with ARH related education. A study conducted in selected regions also
indicated the FGAE Youth Center engaged actively in the provision of wide range of
integrated youth friendly services (14, 21).
The results of this evaluation showed that RH related education session in schools mainly
available sometimes, which were found to be adequate in compliance with the intended
arrangement of the sessions and reaching adolescents with RH education, wherein the majority
(83.0%) of the respondents reported RH related education was available sometimes in high
schools.
The IDIs participants acknowledged the importance of imparting more of non-sexual RH related
education in high schools for adolescents because schools provide a chance of getting high
number of adolescents at earlier age (5). A study done in selected regions of Ethiopia (21),
indicates about 85.8% of the young people stated that it was very important obtaining early
information and knowing about RH matters. A study done in USA on the potential role of
schools in addressing comprehensive RH related education also indicated that it is a place to
reach a large number of young people and lay basis for reproductive health knowledge and skills
building of adolescents in lower and middle adolescence age group (26).
The qualitative results indicated that ARH related education is not included in school time
table. Hence, extracurricular activities carried out in schools by different actors were found to be
the only strategy to address ARH related education in high schools. This practice helps to reach
more adolescents in high schools once at a time. But, it cannot be considered as educating for
life, compared to many experiences elsewhere in the world (5, 13, 15), in which it requires RH
related education in school time table because it is the most effective way of educating for life.
55. Process Evaluation of School-Based RH Education Programs
Fikru Tessema 45 April, 2008
In this evaluation, the overall participation of adolescents in extracurricular activities was
found to be fair (65.4%) in compliance with the intention to participate adolescents in
extracurricular activities as compared to the agreed evaluation standards.
With regard to the availability of training in RH for teachers and peers, it was found to be
inadequate in compliance with the intention to provide the training for teachers and peers.
Furthermore, its continuity was also inadequate and not at hand, if not nil. This suggests that
training of teachers and peers has no continuity to reach a good number of teachers and peers
with RH knowledge and skills needed to impart RH for adolescents in schools.
The qualitative results indicate only the youth center that actively promote extracurricular
activities of ARH related education and services in high schools (14, 20). The results of this
evaluation showed that school clubs, mini media and peer educators have a remarkable role in
consulting and discussing with adolescents in high schools, which was comparable with a an
assessment done some two years back in major regions of Ethiopia (20), in which young people
acknowledged consulting and discussing with peer educators, friends, and school clubs for their
RH issues.
In this evaluation it was found that the distribution of IEC/BCC materials on RH was
inadequate in compliance with the intention to distribute the materials, in which about 38.8% of
respondents to student survey have got chance of reading the materials. Most adolescents were
able to list some IEC/BCC materials, in which HIV/AIDS posters, FGM posters, leaflets on
pregnancy, FGM, abortion, STIs and early marriage were the most commonly listed materials,
which was similar to other study done in Ethiopia (19, 33).
The availability of health information dissemination on ARH services and its providers in
schools was found to be adequate as compared to the agreed evaluation standards in which most
(84.2%) of the respondents to student survey have information about RH services and RH service
providers.
The IDIs demonstrated that health information was available for adolescents through various
outlets, in which school clubs, mini media and peer educators were serving as health information,