how to evaluate health services,indicators used and various study designs used,efficacy and efficiency of those methods.

1. By: Dr. Kavita Yadav
MPH 1st yr
Moderator:Dr.Kavitha HS
JSSMC,Mysore

2.  Introduction to epidemiology
 Uses of epidemiology
 Definition and Need of Evaluation
 Steps in evaluation
 Application of epidemiological studies in
evaluation of health services
 Various study designs
 Merits and demerits
 Conclusion
 References

3.  The study of distribution and determinants of
health related states or events in a specified
populations, and the application of this study
to the control of health problems.

4. • Observational studies
Descriptive studies:Ecological
Cross-sectional
Analytical studies: Case-control
Cohort
• Experimental studies : RCT,field trials,
Community trials

5.  To study historically the rise and fall of
disease in a population
 Community diagnosis
 Planning and evaluation
 Evaluation of individual’s
risk and chances
 Syndrome identification
 Completing the natural
history of the disease
 Search for cause and
risk factor

6.  A systematic process to assess the
achievement of the stated
objectives of a programme,its
adequacy,efficiency, and its
acceptance by all parties involved.

7.  A planned, systematic process of observation
that closely follows a course of activities, and
compares what is happening with what is
expected to happen

8. 1)It determines programme 1)It determines programme
efficiency effectiveness
2)It establishes standard of 2)It identifies inconsistencies
performance at the activity level between programme
3)It alerts the management , objective and activities.
of discrepancy 3)It suggests changes in
4)It identifies strong & programme procedure
weak points of operation & objectives
programme operation 4)It identifies the possible
side effects of programme

9.  To review the implementation of services
provided by health programmes so as to
identify problems and recommend necessary
revisions of the programme.
 To assess progress towards desired health
status at national or state levels and identify
reasons for gap, if any.

10.  To contribute towards better health planning
 To document results achieved by a project
funded by donor agencies. To know whether
desired health outcomes are being achieved
and identify remedial measures.
 To improve health programmes and the
health infrastructure.
 Allocation of resources in current and future
programme.
 To render health activities more relevant,
more efficient and more effective.

11.  The Policy makers(Those responsible for
programme development and implementation)
 Adhoc research group
 By the Community
(students,NGOs)

14.  Determine what is to be evaluated
 Establish standards and criteria
 Plan the methodology
 Gather information
 Analyse the results
 Take action
 Re-evaluate

17.  Structural criteria: physical criteria,facilities
equipments(cost benefit,cost effectiveness)
 Process criteria: every prenatal mother must
receive 6 check ups.
 Outcome criteria: alteration in patient
health status or behaviour resulting from
health care.

18.  An indicator is a standardized, objective
measure that allows—
 A comparison among health facilities
 A comparison among countries
 A comparison between different time periods
 A measure of the progress toward achieving
program goals.

19.  Valid: should actually measure what they
are supposed to measure.
 Reliable: answer should be same if measured
by different people in same conditions
 Sensitive: sensitive to change in situation
 Specific: reflect changes only in situation
 Feasible : ability to obtain
needed data
 Relevant :contribute
to understanding of
phenomenon of interest

20.  Relevance
 Adequacy
 Accessibility
 Acceptability
 Effectiveness
 Efficiency
 Impact

21.  Efficacy:it is a measure in a situation in
which all conditions are controlled to
maximize the effect of the agent
 Effectiveness:If we administer the agent in a
“real-life” situation, is it effective?(cost
effectiveness)
 Efficiency: Is it possible to achieve our goals
in a cheaper and better way? Cost includes
not only money, but also discomfort, pain,
absenteeism, disability, and social stigma.
(cost benefit ratio)

22.  Purpose of evaluation
 Standards and criteria must be included

23.  Data required may include political, cultural,
economic, environmental and administrative
factors influencing the health situation as
well as mortality and morbidity statistics.

24.  Focus on the health service as the
independent variable, with a reduction in
adverse health effects as the anticipated
outcome (dependent variable) if the
modality of care is effective

25.  Randomized design
 Non randomized design
Before after design
Simultaneous Nonrandomized Design
Combination design
Case control studies

26.  Eliminates problem of selection bias.
 For ethical and practical reasons,randomizing
patients to receive no care is not considered.
 Assign different types of care and then
evaluate.

27.  Chemotherapy of tuberculosis in India, which
demonstrated that domiciliary treatment of
pulmonary TB was as effective as the more
costlier hospital or sanatorium. Results
gained international acceptance and ushered
in new era.
 Evaluation of Multiphasic screening in South
East London, led to withholding of vast
outlay of resources required to mount a
national programme

28.  RCT trials are logistically complex and
extremely expensive.
 Ethical problems
 Long time for completion, so relevance is
questionable.
 Alternative approach- outcome research.

29.  Denotes studies comparing the effects of two
or more health care interventions or
modalities- such as treatment, forms of
health care organization, or type and extent
of insurance coverage and provider
reimbursement on health or economic
outcomes.
 Uses data from large data sets that were
derived from large population.

30.  Refers to real world population and issue of
representativeness or generalizability is minimized
 As the data already exists, analysis can be
completed and results generated rapidly
 Sample size is not a problem except when smaller
sub-groups are examined
 Cost effective.

31.  Data gathered for fiscal and administrative purpose
may not suit research questions addressed in study
 New questions (as knowledge is more complete
now) wouldn’t have been framed
 Data on independent and dependent variable may
be limited
 Data relating to possible confounders may be
inadequate or absent
 Certain variables that are relevant today,were not
included in original data set

32.  Investigator may create surrogate variable or
may change original question which he wanted
to address
 Investigator becomes progressively more
removed from the individual being studied

33.  Before –after design
 Simultaneous nonrandomized design
a)Comparison of utilizers and non utilizers
b)Comparison of eligible and non eligible
 Combination designs
 Case control studies

34.  Data obtained in each of two periods are not
comparable in terms of quality and
completeness.
 Difference is due to programme or due to
other factors which changed over time like
housing,nutrition,lifestyle
 Problem of selection exists

35.  A cohort study in which the type of health
care being studied represents “exposure”
 Problem arises as in how to select exposed
and non-exposed group for study

36.  To compare a group of people who use a
health service with a group who do not.
 Problem of self selection exists
 Address this problem by characterizing the
prognostic profile of people in both groups.
 We cant say someone
not to utilize the
programme.

37.  Assumption being made that eligibility and
non-eligibility is not related to either
prognosis or outcome,
 So no selection bias is being introduced
 For eg:employer or census
tract of residence
 May relate to
socioeconomic status

38.  In all above mentioned designs that compare
the morbidity level in people who receive
care and who do not, assumption is made
that the original level of morbidity in 2
groups at time T1 were comparable before
the care was provided.
 Combination of both the designs viz. before
and after ,programme and no programme

39.  The case-control design has been applied primarily to
etiologic studies, when appropriate data are obtainable,
this design can serve as a useful, but limited, surrogate
for randomized trials.
 But this design requires definition and specification of
cases, it is most applicable to studies of prevention of
specific diseases. The “exposure” is then the specific
preventive or other health measure that is being
assessed.
 As in most health services research, stratification by
disease severity and by other possible prognostic
factors is essential for appropriate interpretation of the
findings.

40.  Should take place within shortest time
feasible
 Discussion of results should be done.

41.  For evaluation to be truly effective emphasis
should be on actions- to support,strengthen
or modify the services.
 Calls for shifting priorities,revising objectives
or developing new programmes to meet
previously unidentified needs.

42.  Aims at rendering health services more
relevant, more efficient and more effective.

43.  The 5 year RCH phase II was launched in
2005 with a vision to bring about outcomes
as envisioned in the Millennium Development
Goals, the National Population Policy 2000
(NPP 2000), the Tenth Plan, the National
Health Policy 2002 and Vision 2020 India,
minimizing the regional variations in the
areas of RCH and population stabilization
through an integrated, focused, participatory
programme meeting the unmet needs of the
target population, and provision of assured,
equitable, responsive quality services.

44.  Goal: “Health For All”
 Objective: Population stabilization by 2045
 Programme: Comprehensive R.C.H services
 Monitoring & Evaluation: RCH indicators/feedback
data

45.  No. of eligible couples registered/ANM
 No. of Antenatal Care sessions held as planned
 % of sub Centres with no ANM
 % of sub Centres with working equipment of ANC
 % ANM/TBA without requisite skill
 % of sub centres with infant weighing machine
 % sub centres with vaccine supplies
 % sub centres with ORS packets
 % sub centres with FP supplies

46.  % Pregnancy Registered before 12 weeks
 % ANC with 5 visits
 % ANC receiving all RCH services
 % High risk cases referred
 % High risk cases followed up
 % deliveries by ANM/TBA
 %PNC with 3 PNC visits
 % PNC receiving all counselling
 % PNC complications referred
 % Eligible couple offered FP choices
 % women screened for RTI/STDs
 % Eligible couple counselled for prevention of RTI/STDs
 % ARI treated
 % children fully immunized

47.  % Deaths from maternal causes
 Maternal mortality ratio
 Prevalence of maternal morbidity
 % Low birth weight
 Neonatal mortality ratio
 Prevalence of post natal maternal morbidity
 % Baby breast feed within 6 hrs of delivery
 Couple protection rate
 Prevalence of terminal method of sterilization
 Prevalence of spacing method
 %Abortion related morbidity
 Prevalence of RTI/STDs

48.  Gordis leon. Epidemiology. 4th edition. Philadelphia:
Elsevier Saunders:2009
 Park K. Park’s Textbook of Preventive and Social
Medicine. 22nd ed.
 WHO: UNFPA. Programme Manager’s Planning
Monitoring & Evaluation Toolkit. Division for
oversight services, August 2004,
 UNICEF. “A UNICEF Guide for Monitoring and
Evaluation: Making a Difference?”, Evaluation Office,
New York, 1991. • ). “Framework for Program
Evaluation in Public Health”, 1999. Available in
English at http://www.cdc.gov/eval/over.htms