MUHIMBILI UNIVERSITY OFHEALTH AND ALLIED SCIENCIES COMMUNITY HEALTH PRESENTATION FESTO KOMBA, BSCN3 RE.NO 2009-04-01817
QUESTION• Describe the usefulness of the pretest-posttest (both one group and two group design) to evaluation.
OBJECTIVES At the end of this precise presentation one will acquire the following knowledge.• Meaning of evaluation.• Meaning of experimental design as the method of data collection.• Kind of experimental designs which are feasible and appropriate to the health settings: pretest-posttest (both one -group and two- group design).• Usefulness of pretest-posttest to evaluation.
DEFINITIONS• Evaluation is the “final” step of the nursing process, but it is linked to assessment, which is the first step. (Anderson et al 2011)• Nursing practice is cyclic as well as dynamic, and, for community- based interventions to be timely and relevant, the community database, nursing diagnoses, and health program plans must be evaluated routinely.• The effectiveness of community nursing interventions depends on continuous reassessment of the community’s health and on appropriate revisions of planned interventions.
Cont.. Experimental Design is the method of data collection which can provide an answer to the crucial questions:• Did the program make a difference?• Are health behaviors, knowledge, and attitudes changed as a result of the program activities?• Is the community healthier because of the programs offered?• However, the problem with experimental studies in program evaluation is that they require selective implementation, meaning that people who participate are selected through a process such as random assignment to a control group and an experimental group.
Cont..• For many ethical, political, and community health reasons, selective implementation is difficult to complete and is sometimes impossible.• Despite these problems, the experimental study remains the best method to evaluate summative effects (outcomes) of a program and the only way to produce quantified information on whether the program made a difference. The following designs are the most feasible and appropriate to health care settings. – Pretest–Posttest One-Group Design. – Pretest–Posttest Two-Group Design
1.PRETEST–POSTTEST ONE-GROUP DESIGN.• Two observations are made, - the first at Time 1 and - the second at Time 2.• The observation can be the prevalence of a health state (e.g., the percentage of adults who exercise regularly, the teenage pregnancy rate, cases of child abuse, and so on), knowledge scores, or other important community health facts.• Between Time 1 and Time 2, an experiment is introduced.
Cont..• The experiment may be a planned program aimed at a target group, such as teen sexuality classes, or with a community-wide focus, like a crime-prevention program.• The evaluation of the program is measured by considering the difference between the health state at Time 1 and the health state after the program at Time 2.
Cont..Pretest–Posttest One-Group Design table. Time 1 Time 2 Experimental Observation 1 Experiment Observation 2 Group
2.PRETEST–POSTTEST TWO-GROUP DESIGN• The design has both an experimental group and a control group.• At Time 1, an observation is made of both the experimental and control groups.• Between Time 1 and Time 2, an experiment is introduced with the experimental group.• At Time 2, second observations are made on both the experimental and control groups.• Program evaluation is the difference between Observations 1 and 2 for the experimental group when compared to the comparison group (which has been selected to be as similar as possible to the experimental group).
Cont..Pretest–Posttest One-Group Design table. Time 1 Time 2 Experimental Observation 1 Experiment Observation 2 Group Control Group Observation 1 Observation 2
Cont..• Will the pretest–posttest with a control group design eliminate the effect of outside factors that occurred simultaneously with the experiment and that might account for the change between Observation 1 and Observation 2, the very problem that plagued the pretest–posttest one-group design?• The answer is yes, if the experimental and control groups are similar.
USEFULNESS TO EVALUATION• An experimental design can yield data on whether a program has produced the desired outcomes when compared to the absence of such a program . or alternatively• whether one program strategy has produced better results with regard to the desired outcomes than some other strategy. NB: The experimental design is not useful for the evaluation of program progress or program cost efficiency.
REFERENCE• Anderson, E.T., McFarlane, J: (2011 ) Community As Partner, Theory and Practice in Nursing, 6th Edition.