As the global prevalence of obesity and chronic diseases continues to rise, the need for effective health promotion programs is imperative. Whilst research into effectiveness of health promotion programs is needed to improve population health outcomes, translation of these research findings into policy and practice is crucial. Translation requires not only efficacy data around what to implement, but also information on how to implement it.
http://medcraveonline.com/MOJPH/MOJPH-02-00007.pdf
Introduction
Planning
Definitions
Components
Types of health planning
Steps in planning process
Introduction
Planning
Definitions
Components
Types of health planning
Steps in planning process
Evaluation
Definitions..
Types
Steps in evaluation
Frame work for evaluation of public health program.
Conclusion.
References.
National Institute of Health: Theory at a Glance, A Guide for Health Promotio...Zach Lukasiewicz
Introduction viii
Audience and Purpose 1
Contents 1
Part 1: Foundations of Theory in Health Promotion and Health Behavior 3
Why Is Theory Important to Health Promotion and Health Behavior Practice? 4
What Is Theory? 4
How Can Theory Help Plan Effective Programs? 4
Explanatory Theory and Change Theory 5
Fitting Theory to the Field of Practice 5
Using Theory to Address Health Issues in Diverse Populations 7
Part 2: Theories and Applications 9
The Ecological Perspective: A Multilevel, Interactive Approach 10
Theoretical Explanations of Three Levels of Influence 12
Individual or Intrapersonal Level 12
Health Belief Model 13
Stages of Change Model 15
Theory of Planned Behavior 16
Precaution Adoption Process Model 18
Interpersonal Level 19
Social Cognitive Theory 19
Community Level 22
Community Organization and Other Participatory Models 23
Diffusion of Innovations 27
Communication Theory 29
Media Effects 30
Agenda Setting 30
New Communication Technologies 31
Introduction
Planning
Definitions
Components
Types of health planning
Steps in planning process
Introduction
Planning
Definitions
Components
Types of health planning
Steps in planning process
Evaluation
Definitions..
Types
Steps in evaluation
Frame work for evaluation of public health program.
Conclusion.
References.
National Institute of Health: Theory at a Glance, A Guide for Health Promotio...Zach Lukasiewicz
Introduction viii
Audience and Purpose 1
Contents 1
Part 1: Foundations of Theory in Health Promotion and Health Behavior 3
Why Is Theory Important to Health Promotion and Health Behavior Practice? 4
What Is Theory? 4
How Can Theory Help Plan Effective Programs? 4
Explanatory Theory and Change Theory 5
Fitting Theory to the Field of Practice 5
Using Theory to Address Health Issues in Diverse Populations 7
Part 2: Theories and Applications 9
The Ecological Perspective: A Multilevel, Interactive Approach 10
Theoretical Explanations of Three Levels of Influence 12
Individual or Intrapersonal Level 12
Health Belief Model 13
Stages of Change Model 15
Theory of Planned Behavior 16
Precaution Adoption Process Model 18
Interpersonal Level 19
Social Cognitive Theory 19
Community Level 22
Community Organization and Other Participatory Models 23
Diffusion of Innovations 27
Communication Theory 29
Media Effects 30
Agenda Setting 30
New Communication Technologies 31
Quality
Degree of adherence to pre-established criteria or standards.
Not an easy subject to get quality healthcare services.
Quality management
Doing the right thing, at the right time, for the right person, and having the best quality result.
4 main components:
Quality planning
Quality control
Quality assurance
Quality improvement
Focused on product/service quality & means to achieve it
What is implementation science and why should you careLisa Muldrew
This seminar will discuss the emerging field of implementation science with a focus on its application within clinical settings. Topics will include an overview of implementation science, how implementation science is positioned within the translation continuum, common conceptual models and analytic frameworks used in implementation science and a study example.
Policy Implications of Healthcare Associated InfectionsAlbert Domingo
On February 19, 2014 at the Ateneo School of Medicine and Public Health in Pasig City, Dr. Albert Domingo presented an introduction to the economic impact of healthcare associated infections (HAIs) as well as related concepts in health policy and management. The speaker discussed common approaches taken to ascertain the economic impact of HAIs, followed by factors/considerations in Philippine health policy and management that must be understood and adjusted in order to minimize HAIs.
A document prepared by Dr. Mustafa Salih, the former director of the Directorate General of Health Policy, planning and research at the Federal ministry of Health in Sudan.
An overview on priorities in health research was a part of a course for research methodology delivered in King Saud University College of Medicine August 2015
Quality
Degree of adherence to pre-established criteria or standards.
Not an easy subject to get quality healthcare services.
Quality management
Doing the right thing, at the right time, for the right person, and having the best quality result.
4 main components:
Quality planning
Quality control
Quality assurance
Quality improvement
Focused on product/service quality & means to achieve it
What is implementation science and why should you careLisa Muldrew
This seminar will discuss the emerging field of implementation science with a focus on its application within clinical settings. Topics will include an overview of implementation science, how implementation science is positioned within the translation continuum, common conceptual models and analytic frameworks used in implementation science and a study example.
Policy Implications of Healthcare Associated InfectionsAlbert Domingo
On February 19, 2014 at the Ateneo School of Medicine and Public Health in Pasig City, Dr. Albert Domingo presented an introduction to the economic impact of healthcare associated infections (HAIs) as well as related concepts in health policy and management. The speaker discussed common approaches taken to ascertain the economic impact of HAIs, followed by factors/considerations in Philippine health policy and management that must be understood and adjusted in order to minimize HAIs.
A document prepared by Dr. Mustafa Salih, the former director of the Directorate General of Health Policy, planning and research at the Federal ministry of Health in Sudan.
An overview on priorities in health research was a part of a course for research methodology delivered in King Saud University College of Medicine August 2015
Health planning steps and types of evaluation in community health nursing.pptxSapna Thakur
Planning is defined as: The process of. Analyzing the system for defining the problem. Assessing the extent to which the problems exists as a need. Formulating goals and objectives to alleviate the needs. Assessment of resources.
Introduction to Program Evaluation for Public Health.docxmariuse18nolet
Introduction to
Program Evaluation
for Public Health Programs:
A Self-Study Guide
Suggested Citation: U.S. Department of Health and Human Services
Centers for Disease Control and Prevention.
Office of the Director, Office of Strategy and Innovation.
Introduction to program evaluation for public health
programs: A self-study guide. Atlanta, GA: Centers
for Disease Control and Prevention, 2011.
OCTOBER 2011
Acknowledgments
This manual integrates, in part, the excellent work of the many CDC programs that have used
CDC’s Framework for Program Evaluation in Public Health to develop guidance documents and
other materials for their grantees and partners. We thank in particular the Office on Smoking
and Health, and the Division of Nutrition and Physical Activity, whose prior work influenced the
content of this manual.
We thank the following people from the Evaluation Manual Planning Group for their assistance in
coordinating, reviewing, and producing this document. In particular:
NCHSTP, Division of TB Elimination: Maureen Wilce
NCID, Division of Bacterial and Mycotic Diseases: Jennifer Weissman
NCCDPHP, Division of Diabetes Translation: Clay Cooksey
NCEH, Division of Airborne and Respiratory Diseases: Kathy Sunnarborg
We extend special thanks to Daphna Gregg and Antoinette Buchanan for their careful editing
and composition work on drafts of the manual, and to the staff of the Office of the Associate
Director of Science for their careful review of the manual and assistance with the clearance
process.
Contents
Page
Executive Summary
Introduction..................................................................................................................................... 3
Step 1: Engage Stakeholders .................................................................................................. 13
Step 2: Describe the Program ................................................................................................ 21
Step 3: Focus the Evaluation Design ..................................................................................... 42
Step 4: Gather Credible Evidence ......................................................................................... 56
Step 5: Justify Conclusions ...................................................................................................... 74
Step 6: Ensure Use of Evaluation Findings and Share Lessons Learned ......................... 82
Glossary ......................................................................................................................................... 91
Program Evaluation Resources ..................................................................................................... 99
Introduction to Program Evaluation for Public Health Programs Executive Summary - 1
Executive Summary
This documen.
Introduction to Program Evaluation for Public Health.docxbagotjesusa
Introduction to
Program Evaluation
for Public Health Programs:
A Self-Study Guide
Suggested Citation: U.S. Department of Health and Human Services
Centers for Disease Control and Prevention.
Office of the Director, Office of Strategy and Innovation.
Introduction to program evaluation for public health
programs: A self-study guide. Atlanta, GA: Centers
for Disease Control and Prevention, 2011.
OCTOBER 2011
Acknowledgments
This manual integrates, in part, the excellent work of the many CDC programs that have used
CDC’s Framework for Program Evaluation in Public Health to develop guidance documents and
other materials for their grantees and partners. We thank in particular the Office on Smoking
and Health, and the Division of Nutrition and Physical Activity, whose prior work influenced the
content of this manual.
We thank the following people from the Evaluation Manual Planning Group for their assistance in
coordinating, reviewing, and producing this document. In particular:
NCHSTP, Division of TB Elimination: Maureen Wilce
NCID, Division of Bacterial and Mycotic Diseases: Jennifer Weissman
NCCDPHP, Division of Diabetes Translation: Clay Cooksey
NCEH, Division of Airborne and Respiratory Diseases: Kathy Sunnarborg
We extend special thanks to Daphna Gregg and Antoinette Buchanan for their careful editing
and composition work on drafts of the manual, and to the staff of the Office of the Associate
Director of Science for their careful review of the manual and assistance with the clearance
process.
Contents
Page
Executive Summary
Introduction..................................................................................................................................... 3
Step 1: Engage Stakeholders .................................................................................................. 13
Step 2: Describe the Program ................................................................................................ 21
Step 3: Focus the Evaluation Design ..................................................................................... 42
Step 4: Gather Credible Evidence ......................................................................................... 56
Step 5: Justify Conclusions ...................................................................................................... 74
Step 6: Ensure Use of Evaluation Findings and Share Lessons Learned ......................... 82
Glossary ......................................................................................................................................... 91
Program Evaluation Resources ..................................................................................................... 99
Introduction to Program Evaluation for Public Health Programs Executive Summary - 1
Executive Summary
This documen.
Contribution of Multiple Assessment Methodologies in a Theory-driven Evaluati...JSI
Poster from the Innovations for MNCH Initiative team at the 2015 Evaluation Conference, summarizing research methods for evaluating the impact of an mHealth intervention with frontline health workers in five districts in Ghana.
Learn more about the project at http://www.innovationsformnch.org
Chapter 5 Program Evaluation and Research TechniquesCharlene R. .docxchristinemaritza
Chapter 5 Program Evaluation and Research Techniques
Charlene R. Weir
Evaluation of health information technology (health IT) programs and projects can range from simple user satisfaction for a new menu or full-scale analysis of usage, cost, compliance, patient outcomes, and observation of usage to data about patient's rate of improvement.
Objectives
At the completion of this chapter the reader will be prepared to:
1.Identify the main components of program evaluation
2.Discuss the differences between formative and summative evaluation
3.Apply the three levels of theory relevant to program evaluation
4.Discriminate program evaluation from program planning and research
5.Synthesize the core components of program evaluation with the unique characteristics of informatics interventions
Key Terms
Evaluation, 72
Formative evaluation, 73
Logic model, 79
Program evaluation, 73
Summative evaluation, 73
Abstract
Evaluation is an essential component in the life cycle of all health IT applications and the key to successful translation of these applications into clinical settings. In planning an evaluation the central questions regarding purpose, scope, and focus of the system must be asked. This chapter focuses on the larger principles of program evaluation with the goal of informing health IT evaluations in clinical settings. The reader is expected to gain sufficient background in health IT evaluation to lead or participate in program evaluation for applications or systems.
Formative evaluation and summative evaluation are discussed. Three levels of theory are presented, including scientific theory, implementation models, and program theory (logic models). Specific scientific theories include social cognitive theories, diffusion of innovation, cognitive engineering theories, and information theory. Four implementation models are reviewed: PRECEDE-PROCEED, PARiHS, RE-AIM, and quality improvement. Program theory models are discussed, with an emphasis on logic models.
A review of methods and tools is presented. Relevant research designs are presented for health IT evaluations, including time series, multiple baseline, and regression discontinuity. Methods of data collection specific to health IT evaluations, including ethnographic observation, interviews, and surveys, are then reviewed.
Introduction
The outcome of evaluation is information that is both useful at the program level and generalizable enough to contribute to the building of science. In the applied sciences, such as informatics, evaluation is critical to the growth of both the specialty and the science. In this chapter program evaluation is defined as the “systematic collection of information about the activities, characteristics, and results of programs to make judgments about the program, improve or further develop program effectiveness, inform decisions about future programming, and/or increase understanding.”1 Health IT interventions are nearly always embedded in ...
HCM 3305, Community Health 1 Course Learning Outcom.docxaryan532920
HCM 3305, Community Health 1
Course Learning Outcomes for Unit VIII
Upon completion of this unit, students should be able to:
3. Recognize effective organization and promotion of health programming for community health on a
global scale.
3.1. Assess the steps for organizing a community health program.
3.2. Identify steps needed to effectively evaluate the community health program.
Reading Assignment
Chapter 15:
Systems Thinking and Leadership in Community and Public Health
Unit Lesson
In this unit, we will discuss systems thinking and community health programming.
Community organizing is a process that involves the engagement of individuals, groups, and organizations.
Program planning is not required in community organizing; however, it is often times used. Program planning
is a process where a health intervention is planned to meet the needs within a population. Antiviolence
campaigns and stress management courses are examples of program planning (McKenzie, Pinger, & Kotecki,
2012).
When deciding which community health interventions to create, the Centers for Disease Control and
Prevention (CDC) uses Guide to Community Preventive Services (Community Guide). The Community Guide
is considered credible because it is based off the scientific systematic review process. The guide answers
many questions that are critical to community health on subjects such as
interventions that have worked/did not work,
populations in which the intervention worked/did not work,
cost of the intervention,
benefits/risks of the intervention, and
future research recommendations (Centers for Disease Control and Prevention, 2015).
Community health programs are intricate and are a key factor in disease prevention, improving health, and
increasing quality of life. Health status and behaviors are determined by personal, environmental, policy, and
organizational influences. Community health programming is targeted at reaching the goals of Healthy People
2010. Community health programs are generally held within healthcare settings; however, other settings are
becoming more popular. Programs are being held at schools, worksites, religious organizations, and within
communities (Healthy People 2020, 2015). There are instances where healthcare organizations are
collaborating with schools to offer health programs. For instance, nutrition and exercise programs are being
offered at an increased rate. Employers see the value of employee health. Therefore, many employers offer
incentives to employees who take part in employee wellness programs. It is not far fetched to hear about
employers checking cholesterol, blood pressure, quality of life, weight, BMI, and sometimes glucose. The
rationale is that healthy employees are less likely to call in sick with health-related conditions.
Community health professionals must identify their health issue, and then create specific and measurable
goals and objectives. ...
Running head: QUALITY IMPROVEMENT
Quality improvement 1
Introduction
Health care system consists of various areas that have different functions, and these areas need improvement from time to time to improve the quality of services offered. One of these areas is health care literacy of patients especially the least served; it is defined as the ability of people to access, process and understand basic health information (Lie et al., 2012). An elaborate quality improvement is needed to ensure the provision of quality services. Therefore in a quality improvement plan, each and everyone has a role to play. From the board of directors, middle to department staff in data collection and reporting, reporting implementation progress, orientation and education of staff about the plan and finally evaluation of the plan. Comment by Earl: ok
Roles
Board of directors need to review the quality improvement plan, once approved oversee its implementation by CEO, directors, managers and the staff. Executive leadership oversees the implementation of the plan by the staff. The quality improvement committee analyzes the performance data, evaluates the data and determines the effectiveness of the plan, and makes recommendations on the progress. Medical staffs implement the quality improvement plan. Middle management manages staff and ensures implementation of the plan and is answerable to the executive leadership. The departmental staff handles ensuring that they play their specific role required of them in the implementation of the plan that involves their department (Barrera Jr et al., 2013). Comment by Earl: Discuss roles specific to your project in depth – this is too generic
Data collection and data reporting
Quality improvement committee handles data collection and reporting. The committee should collect data, evaluate and analyzes it and make the necessary recommendations. If the plan is adopted, they determine the functionality of the plan and what changes need to be made to ensure its effectiveness. Comment by Earl: Be specific; explore in more depth
The board of management responsible for reviewing the recommendations and decides whether to adopt them or not. Once they approve they give a go-ahead for its implementation. The management team will take the responsibility of overseeing its implementation.
Changes implemented
There are various changes that need to be implemented to improve health literacy among patients, especially in the underserved population. Firstly is to promote universal access to health information. There needs to be readily accessible health either through their Internet or read materials such as brochures to every patient and should be presented in the simplest manner for the patients to understand..
A case study approach to comprehend sustainability's framework valéry ridde
Presentation by Mathieu Seppey (Université de Montréal).
Global Health Workshop: Methods For Implementation Science in Global Health.
http://www.equitesante.org/implementation-science-methods-in-global-health/
Process of Planning and EvaluationThe process of planning and ev.docxstilliegeorgiana
Process of Planning and Evaluation
The process of planning and evaluation is cyclical and their activities are interdependent. The activities happen in stages; the end of one activity or program leads to the next. The ideas, insights, and learning derived from a particular stage are likely to affect the decisions and activities of the next stage.
Despite this, the process of planning and evaluation is usually presented in a linear manner with sequential steps. The cycle is often affected by external influences. Planners and evaluators need to be flexible in responding to these influences
Health Program Models
Good health programs involve a good deal of effort and a well-developed model. These models provide direction and structure to the program to be built on. The models may not be used completely during the planning process, or various parts of the model may be combined to suit the program. There are some commonly used models in health program planning:
· Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation (PRECEDE)-Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development (PROCEED): Commonly known as the PRECEDE-PROCEED model, it is the most well-known model in this field because it is theoretically grounded and comprehensive. Planning, implementation, and evaluation are all combined in this model.
· The Model for Health Education Planning (MHEP) analyzes:
· Planning through six phases
· Content through three dimensions―subject matter, method, and process
· The Comprehensive Health Education Model (CHEM) consists of six major steps and several suggested procedures within each step.
· The Model for Health Education Planning and Resource Development (MHEPRD) is a less known model. The five major components of this model are―health education plans, demonstration programs, educational programs, research programs, and information and statistics. Each component represents the end result of the planning process. In this model, evaluation plays an integral part in each phase of the model.
Written Assignment 1: Quality of Simulation
Chapter 3 discusses methods to assess the quality of simulations. You learned about three different views of simulation quality.
Suppose you lead a task force that is developing a simulation to provide strategic planning recommendations for property use zoning for a county of 750,000 residents. The zoning board and county commissioners want a simulation that allows them to assess the impact of various zoning decisions based on a variety of dynamic factors, including age, race, education, and income status.
Submit a 2-page (double-spaced) paper addressing the following:
1. First, identify which of the three views discussed in the chapter that would provide the best quality assessment for the situation described above, and explain your decision.
2. Explain how would you ensure the highest level of accuracy with your simulation, and how w ...
Healthy People 2020Healthy People was a call to action and an.docxpooleavelina
Healthy People 2020
Healthy People was a call to action and an attempt to set health goals for the United States for the next 10 years.
Healthy People 2000 established 3 general goals:
Increase the span of healthy life.
Reduce health disparities.
Create access to preventive services for all.
Healthy People 2010 introduced 2 general goals:
Increase quality and years of healthy life.
Eliminate health disparities.
Practical Policy for Preventive Services
The U.S. health care system faces significant challenges that clearly indicate the urgent need for reform.
There is broad evidence that Americans often do not get the care they need even though the United States spends more money per person on health care than any other nation in the world.
Preventive care is underutilized, resulting in higher spending on complex, advanced diseases.
Practical Policy for Preventive Services
Patients with chronic diseases too often do not receive proven and effective treatments such as drug therapies or self management services to help them more effectively manage their conditions.
These problems are exacerbated by a lack of coordination of care for patients with chronic diseases.
Reforming our health care delivery system to improve the quality and value of care is essential to address escalating costs, poor quality, and increasing numbers of Americans without health insurance coverage.
Why policies need to be developed?
Basic needs are not being met (e.g., People are not receiving the health care they need)
People are not being treated fairly (e.g., People with disabilities do not have access to public places)
Resources are distributed unfairly (e.g., Educational services are more limited in neighborhoods of concentrated poverty)
Why policies need to be developed?
Current policies or laws are not enforced or effective (e.g., The current laws on clean water are neither enforced nor effective)
Proposed changes in policies or laws would be harmful (e.g., A plan to eliminate flextime in a large business would reduce parents' ability to be with their children)
Existing or emerging conditions pose a threat to public health, safety, education, or well-being (e.g., New threats from terrorist activity)
Marjory Gordon’s Functional Health Patterns
Marjory Gordon was a nursing theorist and professor who created a nursing assessment theory known as Gordon's functional health patterns.
It is a method to be used by nurses in the nursing process to provide a more comprehensive nursing evaluation of the patient.
Gordon's functional health pattern includes 11 categories which is a systematic and standardized approach to data collection.
List of Functional Health Patterns
1. Health Perception – Health Management Pattern
describes client’s perceived pattern of health and well being and how health is managed.
2. Nutritional – Metabolic Pattern
describes pattern of food and fluid consumption relative to metabolic need and pattern indicators of loca ...
Summary Various industries, including health care, have adop.docxpicklesvalery
Summary
Various industries, including health care, have adopted quality
improvement (QI) to enhance practices and outcomes. As
demands on the U.S. public health system continue to increase,
QI strategies may play a vital role in supporting the system and
improving outcomes. Therefore, public health practitioners, like
leaders in other industries, are developing QI approaches for
application in public health settings.
Quality improvement in public health involves systematically
evaluating public health programs, practices, and policies and
addressing areas that need to be improved to increase healthy
outcomes. Although QI methods and techniques have only
recently been applied to public health, public health systems offer
a wide range of opportunities for implementing, managing, and
evaluating QI efforts.
The growing field of Public Health Systems and Services Research
(PHSSR) offers the potential to contribute to and support QI efforts
in public health. PHSSR examines the delivery of public health
services within communities as well as the outcomes that result from
dynamic interactions within the public health system. By examining
the public health system, stakeholder interactions, delivery of services,
and outcomes, PHSSR can inform and support the implementation
of QI initiatives.
Most recently, national, state, and local levels have made notable
progress in quality improvement in public health.1, 2 One initia-
tive credited with achieving progress is the Multi-State Learning
Collaborative (MLC). The MLC aims to inform the national accredi-
tation program, incorporate quality improvement practice into pub-
lic health systems, promote collaborative learning across states and
partners, and expand the knowledge base in public health.
Bringing together state and local practitioners and other stakeholders
in a community of practice to achieve MLC goals has yielded several
best practices and lessons for public health stakeholders. However,
more work is needed if QI is to become standard practice in public
health—particularly in understanding health departments’ readiness
for change, building the evidence base for effective public health QI
practices in the context of the public health system, and examining the
sustainability of successful projects, and identifying the determinants
of transformational change.
ÆResearchInsights
Quality Improvement in Public Health: Lessons Learned
from the Multi-State Learning Collaborative
Background: AcademyHealth’s 2009 Annual Research Meeting
At the 2009 Annual Research Meeting (ARM), June 28–30, in Chicago, AcademyHealth convened a panel of three experts, members of the
Multi-State Learning Collaborative (MLC), to discuss their experiences in implementing quality improvement collaboratives in public health.
Leslie Beitsch, M.D., J.D., associate dean for health affairs and professor of family medicine and rural health at the College of Medicine, Florida
State ...
Similar to Developing comprehensive health promotion - MedCrave Online Publishing (20)
Changes in psychological adjustment of female - MedCrave Online PublishingMedCrave
Problems of studying failure in the form of breaks and prolonged studying are increasingly taking place in contemporary research. The reason for this lies in the observations that 30% to 50% of enrolled students do not finish studies and at the same time three quarters of them break studying during the first two years of study.
http://medcraveonline.com/JPCPY/JPCPY-02-00051.pdf
Ebola Associated Genes in the Human Genome Implications for Novel TargetsMedCrave
Ramaswamy Narayanan, Ph.D., professor in the Charles E. Schmidt College of Science at Florida Atlantic University, is working to blend the power of computers with biology to use the human genome to remove much of the guesswork involved in discovering cures for diseases.
Medcrave Group - Open Locked Nailing Using an Expandable NailMedCrave
A retrospective study was performed using the hospital records. The mechanism of injury, the time between injury and surgery, blood transfusion requirements, blood loss, surgical times, time taken to weight bear (for the femoral/
tibial fractures), time for commencement of upper limb use (for humeral fractures), complication rates and the average follow up times were documented. Fifty-seven long bone fractures in 57 patients were included in this study. Complete results including preoperative X-Rays were available for 27 patients. In 30 cases, the actual X-Rays were not located but documentation by the treating surgeons was available.
Medcrave - Vertical distribution of different forms of potassiumMedCrave
Potassium is a major constituent of the earth crust contained more in igneous rocks than the sedimentary rocks. Potassium comprise on an average of 2.6 % of the earth crust, making it the seventh most abundant element and fourth most abundant mineral nutrient in the lithosphere .
Medcrave Group - Microfluidic technologiesMedCrave
Exosomes are cell-released small membrane vesicles derived from the endolysosomal pathway with a size range of 30-150 nm. Since the first discovery in 1981, exosomes have been found to be released from various cell types and present in many biological fluids, including blood, urine, erebrospinal fluid and ascites. Significant attention has been focused on exosome molecular components (e.g. roteins, mRNA and miRNA) which have been implicated in a variety of physiological functions and pathological disease states.
Medcrave - MERS coronavirus - current statusMedCrave
CDC: Centers for Disease Control; MERS-CoV: Middle
East Respiratory Syndrome Coronavirus; RT-PCR: Reverse
Transcriptase Polymerase Chain Reaction; VLP: Virus Like
Particles.
Recently, a new virus started to infect certain individuals in the Middle-East. It was soon identified as a previously unknown coronavirus that caused severe respiratory disease with a high rate of mortality. This virus, MERS-CoV, is still closely watched by health authorities as it has the potential to evolve and cause a major epidemic.
Medcrave - Long term follow up of regnauld’s procedureMedCrave
We performed a retrospective study to assess the long-term outcome of regnauld’s procedure, as originally described by Regnauld [1], for the treatment of hallux valgus. This procedure includes the treatment of hallux limitus, hallux rigidus and hallux valgus with associated degenerative joint disease.
Since the first heart transplant, refinement of donor and recipient selection methods, better donor heart management, and advances in immunosuppression have
significantly improved survival. In this first of two articles, a perspective of the current realities of cardiac transplantation is shown, as well as the challenges to sustain services worldwide, and some of the new developments, both resently available and just beyond the horizon. Topics that will be covered in this first part include the donor and recipient demographics, as well as recent advances in transplantation immunology, allograft vasculopathy, and immune tolerance.
Medcrave Group - Association analysis of the polymorphismMedCrave
Behcet’s disease (BD) is a chronic inflammatory autoimmune disease and strongly associated with human leukocyte antigen (HLA) gene such as HLA-B and -A. To
determine the association of the HLA-A, HLA-B and HLA-E alleles with BD in Japanese cohort as well as other ohorts, we performed sequencing-based typing method in 382 BD cases and healthy controls. Then, we analyzed the association of these alleles with BD.
Medcrave Group - Association analysis of the polymorphism
Developing comprehensive health promotion - MedCrave Online Publishing
1. MOJ Public Health 2015, 2(1): 00007Submit Manuscript | http://medcraveonline.com
MOJ Public Health
Abbreviations
CDC: Centre for Disease Control and Prevention; MRC:
Medical Research Council Framework; RE-AIM: Reach, Efficacy,
Adoption, Implementation and Maintenance; CIPP: Context,
Input, Process and Product; NHMRC: National Health and Medical
Research Council
Introduction
As the global prevalence of obesity and chronic diseases
continues to rise, the need for effective health promotion
programs is imperative. Whilst research into effectiveness of
health promotion programs is needed to improve population
health outcomes, translation of these research findings into
policy and practice is crucial. Translation requires not only
efficacy data around what to implement, but also information
on how to implement it. Evaluation seeks to optimise translation
by answering questions related to how to implement evidence-
Developing Comprehensive Health Promotion
Evaluations: A Methodological Review
Research Article
Volume 2 Issue 1 - 2015
Teede HJ1,2
*, Kozica SL1
, Lombard CB1
,
Hider K1
and Harrison CL1
1
Monash Centre for Health Research and Implementation
(MHRCI), Monash University, Australia
2
Diabetes and Vascular Medicine Unit, Monash University,
Australia
*Corresponding author: Helena J Teede, Monash
Centre for Health Research and Implementation (MHRCI)
School of Public Health and Preventive Medicine, Monash
University, Locked Bag 29, Monash Medical Centre,
Clayton, Victoria, 3168, Australia, Fax: +61 39594 7554;
Tel: +61 3 9594 7500; Email:
Received: November 18, 2014| Published: December
30, 2014
based interventions under real world conditions [1]. Evaluation
studies the implementation strategies and adds value to efficacy
outcomes[2],potentiallyinformingonprogramreach,fidelityand
sustainability. Hence evaluation is now recognised as an integral
component of all health promotion programs [2]. Ultimately,
comprehensive evaluation provides essential knowledge about
program implementation practices and processes, which is vital
to the translation of programs into varied setting and population
contexts [3,4].
Presently, evaluation has been applied inconsistently to
health promotion programs [5], limiting the translation of
knowledge. Health promotion programs have further been
criticised for insufficient evaluation planning [6] and scrutinised
as “fail (ing) to contribute to their own quality enhancement”
[5]. Barriers to evaluation include funding constraints, lack of
knowledge, skills and familiarity with evaluation methodologies
and poor availability of measurement tools. Information related
Abstract
Background: Evaluation has an integral role in effective health promotion, yet few
large scale health promotion interventions prospectively incorporate rigorous
evaluation into their planning processes. To promote greater use of evaluation
we reviewed the health promotion literature incorporating evaluations and have
produced a succinct summary.
Methods:MEDLINEandSCOPUSdatabaseswereaccessedtoidentifyhealthpromotion
publications, which utilised formal evaluation. Publication were then summarised and
reviewed informing on evaluation methods relevant to health promotion.
Results: Numerous health promotion evaluation levels and approaches exist some
of which include objective-based, needs-based, collaborative and utilization focused.
Multiple evaluation frameworks are available including the RE-AIM, Centre for Disease
Control and Prevention (CDC) and Medical Research Council framework (MRC). Data
collection methods often include both qualitative and quantitative methods and
validated tools are limited but important. Evaluation approaches and frameworks
have been applied effectively to diverse health programs with useful learnings.
Evaluation enablers include:
I. Prospective planning
II. Tailoring the evaluation to the study context
III. Engaging with evaluation target audience early
IV. Ample time to undertake the evaluation
Conclusion: Evaluation of health promotion interventions is imperative to deliver
benefits of research into improved health outcomes. Understanding of evaluation
concepts including levels, approaches and framework and methods is needed to
facilitate consistent use of evaluation in research. Ultimately, health promotion
programs require context specific adaptable evaluations. Greater opportunities exist
for shared learnings to build evaluation capacity and to deliver greater health impacts.
Keywords
Evaluation; Health promotion; Evaluating health programs; Evaluation frameworks;
Translational research
2. Developing Comprehensive Health Promotion Evaluations: A Methodological Review
Citation: Teede HJ, Kozica SL, Lombard CB, Hider K, Harrison CL (2015) Developing Comprehensive Health Promotion Evaluations: A Methodological
Review. MOJ Public Health 2(1): 00007. DOI: 10.15406/mojph.2015.02.00007
Copyright:
2015 Teede et al.
2/9
to evaluation is often difficult to access through conventional
academic literature and is more often located in non-academic
grey literature. Additionally, terminology in this field is
inconsistent and there is considerable overlap between areas
such as implementation research and evaluation.
In this context, we aimed to review and summarise
the literature and discuss planning and development of
comprehensive health promotion evaluations. This is relevant to
health professionals, researchers and end users, who seek insight
into evaluation designs and are engaged in driving evidence into
policy and practice. An enhanced appreciation of evaluation
methodologies and terminology aims to provide a foundation for
those new to evaluation.
Methods
MEDLINE and SCOPUS databases were used to identify
publications related to the evaluations of health promotion
interventions with no publications date restrictions. Techniques
such as “snowballing” otherwise referred to as reference tracking
were also utilised. This method involves scanning reference lists
of all full text papers and using judgment to decide whether to
pursue texts further [7]. A systematic review was not conducted,
as it was essential to locate literature outside databases
containing published journals (text books, government and non
for profit websites), as this is a regular repository for evaluation
literature. Publications were included and reviewed if it clearly
described an evaluation planning process or methodology and
had applied these methods to a health promotion intervention.
Publicationswereexcludedifitdidnotincludeaformalevaluation
methodology inclusive of evaluation levels, approaches and
frameworks within the context of health promotion programs.
Results
Evaluation Planning and Purpose
Literature suggests that in depth planning is critical to
comprehensive evaluation. The first step in evaluation planning
is to identify the purpose for undertaking the evaluation and
formulating clear evaluation objectives and questions. Broadly,
health promotion evaluation purposes and key questions will
commonly focus on program effectiveness and/or program
implementation. Evaluation is not an absolute science [8] and a
program evaluation wills unlikely represent all perspectives and
components of an intervention, resulting in a “balancing act” or
“trade-off” between priorities [9]. Gathering information on a
large number of issues may result in the evaluation lacking depth
to draw a confident conclusion [10]. Deciding which aspects to
evaluate will need to be informed by relevant health literature,
resources allocated, stakeholder needs and determined based
on the primary targets audience of the evaluation purpose of
the health program [11], mandating the importance of early
stakeholder engagement. Stakeholders may include funders, end
users, service providers, government employees or the general
public. Importantly, providing ample time to plan and conduct an
evaluation is imperative. For an overview of the common phases
of evaluation planning and implementation please refer to figure
1 based on the work of Øvretveit [9] and Van Marris [12].
Evaluation Concepts
Areas for consideration when developing an evaluation
plan can include evaluation levels (also referred to as types),
approaches, framework (also referred to as models) and data
collection tools. We recommend having an understanding of
these commonly used evaluations ‘concepts’ and terminology.
Evaluation levels or types: Evaluation ‘levels’ include
process, formative, summative, impact and outcome evaluations.
The levels are determined by the purpose of the evaluation and
are influenced by the state of the program (under development,
settled) and the timing of data collection (before program roll
out, during implementation or post implementation) [13]. In
Table 1,2 we describe evaluation levels (or types) and note
that elements can overlap. Common elements assessed in a
process evaluation include program reach, fidelity in relation
to program protocol, program context, quality and dose
delivered and received by participants [14,15]. A community
based obesity prevention study in adolescents living in Tonga
completed a process evaluation by research staff recording
all intervention-related activities, the frequency of the
activity, the reach of that activity (how many people were
involved in the program) and the resources required [16].
The purpose of a formative evaluation is to improve the quality
of performance or the delivery of the program prior to roll-
out and is conducted either before or during the program [17].
On the contrary, summative, impact and outcome evaluations
typically occur at the completion of an intervention, yet these
three evaluation methods differ significantly. In the large
scale “Be Active Eat Well” program which promoted healthy
lifestyles in disadvantaged communities, impact and outcome
evaluation results were assessed as environmental changes,
policy implementation and anthropometric data two years post
program commencement [18]. When used alone an impact,
outcome and summative evaluation may miss the depth of the
data available during intervention roll-out. However, process
evaluation requires preplanning with stakeholders to enable
data collection throughout program delivery. We suggest that
combining a range of evaluation levels [E.g. process, summative
and outcome evaluation] can strengthen capacity to translate
successful health promotion interventions to an array of settings
and target groups.
Evaluation approaches: An evaluation approach informs
on ‘how’ we will conduct the evaluation i.e. will stakeholders be
engaged to participate in the evaluation or will the evaluation be
conducted by the researcher only? Each evaluation approach has
a number of associated steps, guiding the processes and activities
of the evaluation (Table 3). There are a number of evaluation
approaches available some of which include objective-based,
needs-based, theory based, collaborative, utilization focused and
realistic evaluation. Typically there is a large degree of overlap
between varied evaluation approaches; however the emphasis
and the tasks related to each step of the evaluation varies in
accordance with the nature and purpose of the evaluation [19].
Please note this is not an extensive list of approaches, rather we
have included approaches that are frequently applied to health
promotion evaluations.
3. Developing Comprehensive Health Promotion Evaluations: A Methodological Review
Citation: Teede HJ, Kozica SL, Lombard CB, Hider K, Harrison CL (2015) Developing Comprehensive Health Promotion Evaluations: A Methodological
Review. MOJ Public Health 2(1): 00007. DOI: 10.15406/mojph.2015.02.00007
Copyright:
2015 Teede et al.
3/9
Describe program being
evaluated, outline
objectives, target audience
and program activities
Engage stakeholders, clarify
the purpose, aims &
resources available for
evaluation
Explore the literature,
what is already known and
have other similiar
programs been evaluated?
Focus and design the
evaluation strategy selecting
appropriate methodologies
(level, approach, framework)
Determine data collection
methods and tools and
develop as required to
ensure credible and useful
results.
Undertake data collection.
Utilise mixed-methods if
possible.
Anaylse and interpret
results, judge the worth and
dissmeniate to stakeholders.
Document and share
evaluation learnings.
Figure 1: Evaluation planning and implementation.
Table 1: Glossary of evaluation and implementation research terms.
Clinical Research
Clinical research as defined by the National Institute of Health incorporates; 1) patient orientated research (direct human
interaction), 2) epidemiological and behavioural studies, 3) outcomes and health service driven research [53].
Evaluation
Evaluation is a comparative assessment and comparison of an intervention of interest, against a standard of acceptability
[54], utilising systematically collected data [9]. In order to determine the merit, worth or significance of the activity [55].
Evaluation Research
Is the systematic application of social research procedures for assessing the conceptualisation, design, implementation and
utility of social intervention programs [56].
Implementation
Research
“Implementation research is the scientific inquiry into questions concerning implementation—the act of carrying an
intention into effect, which in health research can be policies, programs, or individual practices (collectively called
interventions)” . The intent is to understand what, why, and how interventions work in “real world” settings and to test
approaches to improve them [57].
Efficacy Research
Explored the initial impact of the intervention (whether it has done more good than harm) amongst a target population under
specific conditions [1].
Translational
Research
Describes the notion of moving health knowledge generated into products, practices, policies and can include knowledge
exchange, transfer and mobilization [58]. Translatsional reserach is dynamic and iterative process aiming to improve health
ouctomes, provide more effective health serevices and products to strengthen the health care system [59]. More simply,
translational research is the transfer of basic science discoveries to represent a movement towards the goal of improved
health [58].
Scaling-up
‘Scaling up’ is the notion to describe the ambition or process of expanding the exposure of health interventions to maximize
population benefits, but can also refer to increasing the financial, human and capital resources needed to increase coverage
[60].
Process
Evaluation
Measures the activities of the program including reach, implementation, satisfaction, quality and capacity of the program. It
determines whether a program is delivered as intended to the target audience [8].
Impact
Evaluation
Measures the immediate effect of the health intervention [61]. It measures changes in awareness, knowledge, skills, attitudes and
behaviours [62].
Outcome
Evaluation
Measures the longer term program impacts, inclusive of subsequent effects on health [63].
Formative
Evaluation
A combination of measurements are obtained and judgements are made before or during the implementation phase of materials,
methods, activities, in order to improve the quality of performance or the delivery of the program [17].
Summative
Evaluation
Conducted after completion of a program and draws conclusions regarding the quality, impact, outcomes and benefits of a program
[17].
Table 2: Evaluation levels (types).
4. Developing Comprehensive Health Promotion Evaluations: A Methodological Review
Citation: Teede HJ, Kozica SL, Lombard CB, Hider K, Harrison CL (2015) Developing Comprehensive Health Promotion Evaluations: A Methodological
Review. MOJ Public Health 2(1): 00007. DOI: 10.15406/mojph.2015.02.00007
Copyright:
2015 Teede et al.
4/9
Table 3: Evaluation approaches and previous applications within the health promotion setting.
Evaluation
Approach
Description Previous Application in the Health Promotion Setting
Objective-
based
The effectiveness and worth of a program is based
purely on whether pre-defined objectives have been
successfully achieved [13].
The objectives of the Diet, Exercise and Weight Loss Trial (DEW-IT) were
to determine the effectiveness of a healthy lifestyle program to reduce
cardiovascular disease risk factors. Whilst, this study was not strictly
described as an objective based evaluation, large elements of this study
support this type of evaluation, as unintended outcomes were not explored.
In this program the predefined end-points were determine as weight, blood
pressure, serum lipids and fitness [64].
Needs-based
Evaluation
An evaluation in which the determination of worth is
identified by the communities wants or needs, which
will then be addressed by the program planned [13].
A needs assessment was conducted in patients with advanced cancer
to determine the most suitable delivery methods of health information.
Participants were asked to complete a survey and indicated that the
preferred mode of delivery of information was one-to-one education
sessions [65].
Theory
based/logic
Development
The evaluation is based on program theory and the
logical relationship between program inputs and
outcomes. This approach involves having a good
appreciation of the nature of the program, context and
environment [66].
The Being Active Eat Well program was developed to address childhood
obesity by promoting healthy lifestyles. In this programme the intervention
activities (inputs) focused on capacity building, policy development and
community empowerment. As a result of the program inputs the children in
the intervention group had significantly lower increases in weight and BMI
scores [67].
Collaborative/
Participatory
Evaluation
This approach compels evaluators to include program
staff, consumers and/or community members in
the evaluation design and conduct of the evaluation.
Stakeholders are involved in the evaluative
endeavour, including interpreting and conclusions
[43].
To determine the success of a community based rehabilitation program
in individuals with disabilities, a participatory evaluation was conducted.
Program participants, staff and managers of the program were engaged
in the evaluation process by part-taking in interviews and focus groups to
explore satisfaction of the service. Stakeholders reported that the program
had supported the needs of the community [68].
Utilization
Based
The approach is formulated on the premise of who
the primary intended users and stakeholders are and
how the results will be employed. Utilization-focused
evaluation is highly individualised, flexible and
situational [69].
A utilization focused evaluation was conducted to assess the role and
effectiveness of nurse practitioners in an acute hospital. The evaluation
focused on stakeholder’s satisfaction with the nurse practitioners and
provided valuable information for other organizations interested in
introducing nurse practitioners into their health care system [70].
Realistic
Evaluation
This theory-driven evaluation focus on the context
in which a program is implemented and describes
the mechanism responsible for outcomes achieved.
The findings highlight ‘what works for whom in a
set of given circumstances’, suggesting that it is not
possible to universally generalize cause-and-effect
relationships.
A realistic evaluation approach was employed to determine the effectiveness
of a theory driven school based program, designed to improve children’s
social, emotional and physical health. The mixed-methods evaluation
explored the processes used during program implementation to provide
contextual information and establish relationships between program inputs
and outcomes [71].
Evaluation frameworks or models: Similarly, evaluation
frameworks provide detailed guidance for evaluators, as they
ensure the evaluation design considers the origins and contexts
of the program being examined. An evaluation framework can
encourage the prioritisation of evaluation purpose and the
selection of data collection tools [20]. Moreover, evaluations are
program specific as there is no “one size fits all” evaluation design
in evaluation [19]. Evaluators will often utilise more than one
evaluation level, approach and framework in order to meet the
needs and purpose of their evaluation.
Evaluation Frameworks
From the literature, the commonly applied evaluation
frameworks used in health promotion research include the RE-
AIM, Context, Input, Process and Product (CIPP), Predisposing,
Reinforcing, Enabling, Constructs in Educational/Ecological
Diagnosis and Evaluation (PROCEED, PRECEDE), Centre for
Disease and Control and Prevention (CDC), Medical Research
Council (MRC) and programme logic model. Here we describe
briefly these evaluation frameworks, prior application and
practical insights from our experiences of utilising these
frameworks.
RE-AIM framework: The RE-AIM (Reach, efficacy, adoption,
implementation and maintenance) framework is an adaptable
and simple framework for evaluating large scale projects,
which considers both the individual and population impacts of
the program [21]. RE-AMI focuses on informing and optimising
translation and has been applied numerous times to health
promotion programs for example to a diabetes self-management
programs [22] and school based physical activity programs
[23]. In our experience this framework is useful because it is
adaptable, easy to use and supports the development of a focused
evaluation. The associated resources provide clear examples of
evaluation questions, potential measurement indicators and
data collection tools which are helpful for novice evaluators
[24]. However, it is worth noting that within this framework
‘maintenance’ is measured at two years post program delivery;
therefore consideration of evaluation timelines is essential.
5. Developing Comprehensive Health Promotion Evaluations: A Methodological Review
Citation: Teede HJ, Kozica SL, Lombard CB, Hider K, Harrison CL (2015) Developing Comprehensive Health Promotion Evaluations: A Methodological
Review. MOJ Public Health 2(1): 00007. DOI: 10.15406/mojph.2015.02.00007
Copyright:
2015 Teede et al.
5/9
In comparison to other frameworks the RE-AIM is less flexible
and the relationship between the five domains of the RE-AIM
framework remains unclear.
CIPP framework: The Context, Input, Process and Product
(CIPP) framework for evaluation is comprehensive and useful
for guiding formative and summative evaluations. The context of
the program refers to the evaluation environment in which the
program will be implemented. The input refers to the resources
and activities required to meet the objectives of the program. The
final stage of the CIPP model is the product, which refers to the
judgment of the effectiveness and success of the program [6,25].
This framework has been applied to health promotion programs
previously and examples include a diabetes management
program [26] and a behavioural intervention targeting reduction
in metabolic syndrome [27]. The CIPP framework is highly
flexible and adaptable and specifically useful for evaluations
seeking insight into a range of factors that influence program
effectiveness such as environmental and contextual factors.
However, this framework provides less guidance for evaluation
planning, including developing evaluation objectives and key
questions, compared to the RE-AIM framework.
PROCEED/PRECEDE framework: The PROCEED/PRECEDE
is a theoretically grounded and comprehensive framework,
combining program planning, implementation and evaluation.
It is based on the premise that a thorough social, educational
and ecological assessment (Precede) should take place prior to
program development and that evaluation is required to assess
effectiveness (Proceed). Phase include:
i. Social assessment
ii. Epidemiological, environmental and behavioural
intervention
iii. Educational and ecological assessment
iv. A. Intervention alignment
B. Administrative and policy assessment
v. Implementation
vi. Process evaluation
vii. Impact and outcome evaluation [28]
The framework has been applied to multiple health
promotion programs including mental health programs [29].
The PROCEED/PRECEDE models is a structured and valuable
framework for multi-strategy program evaluations and considers
multiple determinants of health. However, this framework is
complex and has been criticised for being a very medically based
and requires significant time to prospectively plan and undertake
the evaluation [19].
Centre for disease and control and prevention
(organisational framework): The Centre for Disease and
Control and Prevention (CDC) framework suggests that effective
program evaluation is a systematic way to improve public health
actions by conducting evaluations that ensure useful, feasible,
ethical and accurate results. The framework outlines 6 inter-
connected steps to planning and delivering an evaluation
I. Engage stakeholder
II. Describe the program
III. Focus the evaluation design
IV. Gather credible evidence
V. Justify conclusions
VI. Share lessons learnt [19]
The framework is highly reputable and has been applied
to numerous health promotion program including physical
activity programs [30] and tobacco control initiatives [31]. In our
experience the CDC framework is very practical and easy to follow
and is useful for the evaluation of simple through to complex
programs and partnerships. It is particularly valuable for capacity
building evaluations as it incorporates multiple stakeholder
perspectives into the evaluation. Another advantage of the CDC
model is that it is underpinned by four central standards for
ensuring ethically sound evaluations (utility, feasibility, propriety
and accuracy). However, the framework is broad and may be less
useful to novice evaluators, seeking a supportive framework to
assist in the development of evaluation objectives and questions.
Also this framework may require significant funding and strong
evaluation facilitation [19].
Medical research council (organisational framework):
The Medical Research Council (MRC) framework is theory based
and describes the development, evaluation and implementation.
The stages involved include:
V. Development of an intervention,
VI. Pilot and feasibility assessment,
VII. Evaluation of the intervention,
VIII. Implementation, dissemination, monitoring and long
term follow up [32].
Examples of previous application include a physical activity
program targeting a population at high risk of diabetes [33] and
evaluation of a cardiovascular disease management program
[34]. The advantages of the MRC framework are that it is highly
flexible and promotes a non-linear approach to evaluation with
many supportive documents to assists evaluation planning and
implementation. However, the framework is broad and does
not provide depth and detail around developing evaluation
objectives and key questions. Similarly to the CDC framework,
the MRC framework may be less useful to novice evaluators, who
may prefer a more structured and supportive framework.
Evaluation Methodological Approaches
Logic model/programme theory methodological
approach: A logic model is a methodological approach to
program planning and evaluation, describing the assumptions
and resources needed to support the activities of the program of
interest [35]. A logic model typically identifies the background
6. Developing Comprehensive Health Promotion Evaluations: A Methodological Review
Citation: Teede HJ, Kozica SL, Lombard CB, Hider K, Harrison CL (2015) Developing Comprehensive Health Promotion Evaluations: A Methodological
Review. MOJ Public Health 2(1): 00007. DOI: 10.15406/mojph.2015.02.00007
Copyright:
2015 Teede et al.
6/9
of the program, the resources (inputs), the activities (outputs),
immediate impacts and short and long term outcomes [36]. Logic
models have been applied to diverse health programs including
the evaluation of community nutrition education programs [37]
and a school based programs aimed at reduce the frequencies
of teenage pregnancies [38]. The value of logic models is well
established in health promotion and resources available to
develop logic models [39]. Logic models provide a practical
means to assess the relationship between program inputs and
anticipated outcomes. However, a caution is that logic models
may not capture all important aspects of programs and associated
policies. Logic models are also time consuming requiring
prospective planning to consider the underlying program theory
and relationships between processes and outcomes [19].
Overall
Ultimately, evaluation methods selected will vary according
to the purpose, timing, resources available, stakeholders
involved in the evaluation and their prior experience conducting
evaluations. Combining a range of evaluation approaches and
frameworks is commonly accepted in the literature, enabling
evaluators to account for potential methodological limitations.
Structured evaluation frameworks may not be appropriate
for all evaluations contexts, emphasising the importance of
rigorous evaluation planning and the need for context specific
adaptation of evaluation frameworks. However, the importance
of stakeholder engagement as an integral component of
evaluation conduct is recognised across the diverse evaluation
frameworks, emphasising the importance of communication and
engagement of stakeholders to yield useful evaluation results
[40]. In planning a health promotion program, resources ideally
need to be allocated for the evaluation. However, funding bodies
commonly fund research but less commonly fund evaluation and
advocacy is needed to address this. Providing ample funding for
major evaluation strategies would likely shift evaluations from
being perceived as a “minor after thought” to a major integral
component of a health promotion program [5].
Data Collection Methods and Tools
A summary of the advantages and disadvantages of
quantitative and qualitative data collection methods are outlined
in Table 4. The emphasis of qualitative research is depth, detail
and individual perceptions [10]. Quantitative research methods
use standardised data collection tools to fit diverse opinions
and experiences into often predetermined categories [8]. Most
evaluators recognise the synergistic benefits of mixed methods,
which incorporate quantitative and qualitative research [41].
Data “triangulation” provides an opportunity to integrate data in
mixed-methods studies. Data triangulation protocols commonly
involve initially analysing qualitative and quantitative data
separately and then comparing the findings from each method,
to determine whether they support or contradict each other [42].
The purpose of data triangulation is to enhance the reliability
and confidence of research findings. A practical example of
data triangulation in the health promotion setting could involve
comparing anthropometric measurements with results from a
physical activity questionnaire and focus groups (qualitative),
following a specific health promotion intervention [14,43].
Existing validated data tools are preferable for large pre- and
post-interventionsurveys,astheyprovidevalidityandconfidence
that any change identified is likely to reflect a meaningful result.
Yet, it is important to only employ validated tools if they are
useful for the evaluation [44]. Currently there are limited reliable
and validated evaluation tools available [14], with a clear need
to develop program specific tools. However, limited large scale
studies are beginning to share evaluation tools on websites for
public use and descriptions of process evaluation objectives
and associated data collections tools employed are increasingly
being published in peer reviewed journals. This has occurred
in interventions in mental health [45], weight management
[16] and physical activity [46]. There are additionally some
evaluation guides available online (grey literature) that
incorporate evaluation templates as well as sample questions
for semi-structured interviews and focus groups [47,48]. There
are also guides available to describe the optimal methods of
reporting evaluation outcomes, specific to the health promotion
context [49]. Ultimately, data collection tools frequently require
adaptation to accommodate and capture within health program
processes, which are unique to an intervention, increasing
sensitivity and usefulness of results [8]. Combining a range of data
Methods Advantages Disadvantages
Qualitative
• Focus groups
• In-depth interviews
• Structured interviews
• Open-ended survey
• Journals
• Participant
observations
• Case-studies
• Narrative evaluation
• Photograph [72]
• Permits depth and detail [10]
• Opportunity to study the meaning and
experiences
• Data collection process is not restrained by
predetermined categories of analysis
• Useful in determining unintentional effects of a
health intervention [8].
• Time consuming and laborious
• Costly
• Smaller number of cases
• High quality analysis depends on the skills
and integrity of the researcher [73]
Quantitative
• Questionnaires
• Measurements
• Tracking templates
• Reports
• Can be less time consuming
• Typically increased sample size
• Facilities comparison among subjects and
statistical rigorous of data
• Less detailed information
• Less sensitive
Table 4: Qualitative and quantitative evaluation methods advantages and disadvantages.
7. Developing Comprehensive Health Promotion Evaluations: A Methodological Review
Citation: Teede HJ, Kozica SL, Lombard CB, Hider K, Harrison CL (2015) Developing Comprehensive Health Promotion Evaluations: A Methodological
Review. MOJ Public Health 2(1): 00007. DOI: 10.15406/mojph.2015.02.00007
Copyright:
2015 Teede et al.
7/9
collection methods can partly address the inherent weaknesses
of each method. Determining evaluation objectives upfront is
integral to ensure that data gathering methods yield useful and
appropriate information [9]. In future, increased availability of
validated evaluation tools will improve evaluation standards and
enable better comparability between studies [14].
Discussion
Developing a robust evaluation is challenging due to the
lack of guidance in conventional published literature and the
wide range of evaluation concepts across evaluation level,
approach and framework. There are five available evaluation
levels which include formative, process, impact, outcome and
summative evaluation. Multiple evaluation approaches are
available some of which include objective-based, needs based,
collaborative, realistic and utilization focused evaluations. In
addition to evaluation levels and approaches, numerous health
promotion evaluation frameworks are available such as RE-
AIM, CDC, MRC, logic model, CIPP and PREECE/PROCEDE. These
evaluation components have been applied effectively to diverse
health promotion programs and target populations. Commonly,
evaluators will combine and use various aspects of evaluation
levels, approaches and frameworks to meet their evaluation
purposes. We highlight, that each evaluation will require
planning and context specific adaptation to produce useful and
credible results [50].
Moving forward, funders as key stakeholders are increasingly
seeking knowledge about how to translate evidence into practice.
Increasingly, researchers are likely to be held accountable for
optimising opportunities for translating research into practice
or at least for ensuring that the evidence generated during
their research also informs and enables translation [51].
Subsequently, the importance of up skilling our workforce to
build greater evaluation capacity has been established [52].
Evaluation is a vital step to inform translation; consequently
researchers face the challenge of deciding which evaluation
level, approach and framework to apply to drive translation. A
structured evaluation approach has some practical limitations
such as inherently being both resource and labour intensive
[14]. Presently, most evaluations involve a combination of
program specific questionnaires (developed and/or validated
tools) and qualitative research methods, in order to ensure a
robust evaluation design [14,43]. An evaluation consultant can
provide guidance for making major decisions such as selecting
the evaluation level, appropriate framework and data collection
methods.
Broad scale evaluations of health promotion interventions
are lacking and use of evaluation terminology is ambiguous
and inconsistent in the literature. Many health promotion
interventions may show efficacy but commonly do not have
comprehensive integrated prospective evaluations, limiting the
ability to translate outcomes and deliver return on research
investments [5]. This is further confounded by difficulties in
accessing published evaluation literature through accessible
conventional academic avenues. However, utilising existing
levels, approaches and framework creates a greater opportunity
to critique and compare evaluations across different health
interventions. Ultimately, standardisation of evaluation
reporting would enhance the field across a diverse array of
health settings. Opportunities for shared learnings, avoidance of
repeated mistakes and efficient allocation of resources also exist
in this field. Future directions require clear definition of terms
in, evaluation, as well as further development of well tested
evaluation tools. Improved sharing of evaluation processes,
planning and tools will assist novice evaluators to more easily
incorporate comprehensive evaluation methods into their
research plans and will improve translation of evidence into
policy and practice.
Conclusion
Comprehensive within program evaluations is integral to
enable translation and scale-up of effective health promotion
programs. As health promotion interventions vary significantly,
the authors do not suggest a “one size fits all” in evaluation,
as each evaluation needs to be tailored to the context and
circumstance to ensure results provide credible and useful
information. Evaluation concepts including level, approach,
framework along with data collections methods and tools all
require consideration. Enablers include prospective planning,
providing adequate time to develop the evaluation, defining key
objectives and target audiences for the evaluation and ensuring
the evaluation is practical and within resources allocated. Sound
evaluation focuses on the importance of real world impacts and
attempts to bridge the gap between research, clinical practice
and policy. Arguable evaluation should not be optional, but
rather an obligatory in future health promotion research and the
summary provided here will assist health promotion researchers
to integrate evaluation into future health promotion programs.
Acknowledgements
This work was supported by the National Health and Medical
Research Council (NHMRC). Samantha Kozica is an NHMRC
postgraduate scholar, Cheryce Harrison is a National Heart
Foundation Postdoctoral Research Fellow (100168) and Helena
Teede is a NHMRC Practitioner Fellow.
References
1. Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL
(2008) A glossary for dissemination and implementation research in
health. J Public Health Manag Pract 14(2): 117-123.
2. Kumanyika S, Jeffery R, Morabia A, Ritenbaugh C, Antipatis V, et al.
(2002) Obesity prevention: the case for action. Int J Obes Relat Metab
Disord 26(3): 425-436.
3. Oldenburg B, Absetz P (2011) Lost in translation: overcoming the
barriers to global implementation and exchange of behavioral
medicine evidence. Transl Behav Med 1(2): 252-255.
4. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F (2005)
Implementation research: A synthesis of the literature: National
Implementation Research Network.
5. Swinburn B, Bell C, King L, Magarey A, O Brien K, et al. (2007) Obesity
prevention programs demand high-quality evaluations. Aust N Z J
Public Health 31(4): 305-307.
8. Developing Comprehensive Health Promotion Evaluations: A Methodological Review
Citation: Teede HJ, Kozica SL, Lombard CB, Hider K, Harrison CL (2015) Developing Comprehensive Health Promotion Evaluations: A Methodological
Review. MOJ Public Health 2(1): 00007. DOI: 10.15406/mojph.2015.02.00007
Copyright:
2015 Teede et al.
8/9
6. Fleming ML, Parker E (2007) Health promotion: Principles and
practice in the Australian context. Allen & Urwin, New South Wales,
Australia.
7. Greenhalgh T, Peacock R (2005) Effectiveness and efficiency of search
methods in systematic reviews of complex evidence: audit of primary
sources. BMJ 331(7524): 1064-1065.
8. Hawe P, Degeling D, Hall J (1990) Evaluating health promotion:
MacLennan & Petty, Sydney.
9. Ovretveit J (1997) Evaluating health interventions: An introduction to
evaluation of health treatments, services, policies, and organizational
interventions. Open University Press.
10. Patton MQ (1987) How to use qualitative methods in evaluation. Sage
Publications, USA.
11. Rossi, Lipsey, Freeman (2004) Evaluation: A systematic approach. (7th
edn), Sage Publications, USA.
12. Marris BV, King B (2007) Evaluating Health Promotion Programs
Version 3.6: The Health Communication Unit, Centre for Health
Promotion, University of Toronto.
13. Owen JM (2006) Program evaluation: Forms and approaches. Allen &
Unwin, Crows Nest, N.S.W Australia.
14. Nutbeam D (1998) Evaluating health promotion-progress, problems
and solutions. Health Promot Int 13(1): 27-44.
15. SaundersRP,EvansMH,JoshiP(2005)Developingaprocess-evaluation
plan for assessing health promotion program implementation: a how-
to guide. Health Promot Pract 6(2): 134-47.
16. Fotu KF, Moodie MM, Mavoa HM, Pomana S, Schultz JT, et al. (2011)
Process evaluation of a community-based adolescent obesity
prevention project in Tonga. BMC public health 11: 284.
17. Green LW, Lewis FM (1986) Measurement and evaluation in health
education and health promotion. Mayfield Publishing Company, Palo
Alto, California.
18. de Silva-Sanigorski AM, Bolton K, Haby M, Kremer P, Gibbs L, et
al. (2010) Scaling up community-based obesity prevention in
Australia: background and evaluation design of the Health Promoting
Communities: Being Active Eating Well initiative. BMC public health
10: 65.
19. Kahan B (2008) Review of Evaluation Frameworks. Saskatchewn
Minstry of Education.
20. Fink A (2004) Evaluation fundamentals: Insights into the outcomes,
effectiveness, and quality of health programs. (2nd
edn), Sage
Publications, USA.
21. Glasgow RE, McKay HG, Piette JD, Reynolds KD (2001) The RE-AIM
framework for evaluating interventions: what can it tell us about
approaches to chronic illness management? Patient Educ Couns
44(2): 119-27.
22. Glasgow RE, Nelson CC, Strycker LA, King DK (2006) Using RE-AIM
Metrics to Evaluate Diabetes Self-Management Support Interventions.
Am J Prev Med 30(1): 67-73.
23. De Meij JS, Chinapaw MJ, Kremers SP, Wal MFVD, Jurg ME, et al. (2010)
Promoting physical activity in children: The stepwise development of
the primary school-based JUMP-in intervention applying the RE-AIM
evaluation framework. Br J Sports Med 44(12): 879-87.
24. McKenzie R, Naccarella L, Stewart A, Thompson C (2007) Targeting
what matters in health promotion evaluation. Evaluation Journal of
Australasia 7(1): 19-26.
25. Stufflebeam LD (2005) Cipp Model (Context, Input, Process, Product).
Encyclopedia of Evaluation. Sage Publications, Thousand Oaks, USA.
26. Clinton J, McNeill R, Irvine R, Silva M, Brown P (2009) An evaluation
of the Gardening for Health and Sustainability initiative: an initiative
supported by Let’s Beat Diabetes. Counties Manukau District Health
Board 1-44.
27. Intarakamhang U, Patcharee D (2012) Effects of health behavioral
modification program on metabolic diseases in risk Thai clients. Asian
Biomedicine 6: 2.
28. Green LW, Kreuter KW (1998) Health promotion planning: An
educational and Ecological Approach. (2nd
edn), Mayfield Publishing
Company, Mountain View, USA.
29. Wright A, McGorry PD, Harris MG, Jorm AF, Pennell K (2006)
Development and evaluation of a youth mental health community
awareness campaign–The Compass Strategy. BMC public health 6:
215.
30. Gamez R, Parra D, Pratt M, Schmid TL (2006) Muevete Bogota:
promoting physical activity with a network of partner companies.
Promot Educ 13(2): 138-143.
31. MacDonald G, Starr G, Schooley M, Yee SL, Klimowski K, et al. (2001)
Introduction to program evaluation for comprehensive tobacco
control programs. Centers for Disease Control and Prevention,
Atlanta, GA, USA.
32. Medical Research Council (2000) A framework for development and
evaluation of RCTs for complex interventions to improve health.
Medical Research Council, UK.
33. Hardeman W, Sutton S, Griffin S, Johnston M, White A, et al. (2005)
A causal modelling approach to the development of theory-based
behaviour change programmes for trial evaluation. Health Educ Res
20(6): 676-687.
34. Robinson L, Francis J, James P, Tindle N, Greenwell K, et al. (2005)
Caring for carers of people with stroke: developing a complex
intervention following the Medical Research Council framework. Clin
Rehabil 19(5): 560-71.
35. Cooksy LJ, Gill P, Kelly PA (2001) The program logic model as an
integrative framework for a multimethod evaluation. Evaluation and
Program Planning 24(2): 119-28.
36. Helitzer D, Hollis C, de Hernandez BU, Sanders M, Roybal S, et al.
(2010) Evaluation for community-based programs: the integration of
logic models and factor analysis. Eval Program Plann 33(3): 223-233.
37. Medeiros LC, Butkus SN, Chipman H, Cox RH, Jones L, et al. (2005)
A logic model framework for community nutrition education. J Nutr
Educ Behav 37(4): 197-202.
38. Hulton LJ (2007) An evaluation of a school-based teenage pregnancy
prevention program using a logic model framework. J Sch Nurs 23(2):
104-110.
39. Renger R, Titcomb A (2002) A three-step approach to teaching logic
models. American Journal of Evaluation 23(4): 493-503.
40. Johnson K, Greenseid LO, Toal SA, King JA, Lawrenz F, et al. (2009)
Research on Evaluation Use A Review of the Empirical Literature
From 1986 to 2005. American Journal of Evaluation 30(3): 377-410.
41. Creswell JW, Clark VLP (2007) Designing and conducting mixed
methods research: Australian and New Zealand Journal of Public
Health 31(4): 388.
42. O Cathain A, Murphy E, Nicholl J (2010) Three techniques for
integrating data in mixed methods studies. BMJ 341.
9. Developing Comprehensive Health Promotion Evaluations: A Methodological Review
Citation: Teede HJ, Kozica SL, Lombard CB, Hider K, Harrison CL (2015) Developing Comprehensive Health Promotion Evaluations: A Methodological
Review. MOJ Public Health 2(1): 00007. DOI: 10.15406/mojph.2015.02.00007
Copyright:
2015 Teede et al.
9/9
43. Davidson EJ (2005) Evaluation methodology basics: The nuts and
bolts of sound evaluation. Sage Publications, USA.
44. Hider K Frequently Asked Questions about Evaluation. In: SatCfC
(eds.), Southern Health C-CCH, Effectiveness. Melbourne.
45. Nakkash RT, Alaouie H, Haddad P, El Hajj T, Salem H, et al. (2012)
Process evaluation of a community-based mental health promotion
intervention for refugee children. Health Educ Res 27(4): 595-607.
46. Jenkinson KA, Naughton G, Benson AC (2012) The GLAMA (Girls! Lead!
Achieve! Mentor! Activate!) physical activity and peer leadership
intervention pilot project: a process evaluation using the RE-AIM
framework. BMC public health 12: 55.
47. Zarinposh F (2006) Project Evaluation Guide For Nonprofit
Organizations: Fundamental Methods and Steps For Conducting
Project Evaluation. Canada Volunteer Initiative, Canada.
48. de Groot F, Robertson N, Swinburn B, de Silva-Sanigorski A (2010)
Increasing community capacity to prevent childhood obesity:
challenges, lessons learned and results from the Romp & Chomp
intervention. BMC public health 10(1): 522.
49. Moskowitz JM (1989) Preliminary guidelines for reporting outcome
evaluation studies of health promotion and disease prevention
programs. New directions for program evaluation 1989(43): 101-112.
50. Greene JC, Lipsey MW, Schwandt TA, Smith NL, Tharp RG (2007)
Method choice: Five discussant commentaries. New Directions for
Evaluation 2007(113): 111-127.
51. Adams J, Dickinson P (2010) Evaluation training to build capability
in the community and public health workforce. American journal of
evaluation 31(3): 421-433.
52. Suarez-Balcazar Y, Taylor-Ritzler T (2013) Moving From Science
to Practice in Evaluation Capacity Building. American Journal of
Evaluation.
53. Rubio DM, Schoenbaum EE, Lee LS, Schteingart DE, Marantz PR, et al.
(2010) Defining translational research: implications for training. Acad
Med 85(3): 470-475.
54. Green LW, Kreuter MW (1999) Health Promotion Planning; an
Educational and Ecological Approach. (3rd
edn), Mayfield Publishing
Company, Mountain View, USA.
55. Scriven M (2007) Key evaluation checklist. Evaluation Checklists
Project.
56. Rossi PH, Freeman HE, Lipsey MW (1993) Evaluation: A systematic
approach. (6th
edn), SAGE Publications, USA.
57. Peters DH, Adam T, Alonge O, Agyepong IA, Tran N (2013)
Implementation research: what it is and how to do it. BMJ 347: f6753.
58. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, et al. (2006) Lost
in knowledge translation: time for a map? J Contin Educ Health Prof
26(1): 13-24.
59. Straus SE, Tetroe J, Graham I (2009) Defining knowledge translation.
CMAJ 181(3-4): 165-168.
60. Mangham LJ, Hanson K (2010) Scaling up in international health: what
are the key issues? Health Policy Plan 25(2): 85-96.
61. Slatter T (2003) Measuring healt promotion impacts: A guide to impact
evaluation in integrated health promotion. Victorian Government
Department of Human Services, Melbourne, Victoria.
62. McKenzie JF, Neiger B, Smeltzer J (2005) Planning, implementing, and
evaluating health promotion programs: a primer. Pearson/Benjamin
Cummings, San Francisco.
63. Naidoo J, Wills J (1994) Health promotion: foundations for practice.
Bailliere Tindall, UK.
64. Miller ER, Erlinger TP, Young DR, Jehn M, Charleston J, et al. (2002)
Results of the diet, exercise, and weight loss intervention trial (DEW-
IT). Hypertension 40(5): 612-618.
65. Wong RK, Franssen E, Szumacher E, Connolly R, Evans M, et al. (2002)
What do patients living with advanced cancer and their carers want
to know?-a needs assessment. Support Care Cancer 10(5): 408-415.
66. Donaldson SI, Gooler LE (2003) Theory-driven evaluation in action:
Lessons from a $20 million statewide work and health initiative.
Evaluation and Program Planning 26(4): 355-366.
67. Sanigorski A, Bell A, Kremer P, Cuttler R, Swinburn B (2008) Reducing
unhealthy weight gain in children through community capacity-
building: results of a quasi-experimental intervention program, Be
Active Eat Well. Int J Obes 32(7): 1060-1097.
68. Sharma M, Deepak S (2001) A participatory evaluation of community-
based rehabilitation programme in North Central Vietnam. Disabil
Rehabil 23(8): 352-358.
69. Patton MQ (1996) Utilization-Focused Evaluation: The New Century
Text. (3rd
edn), SAGE publications, London, United Kingdom.
70. Stetler CB, Effken J, Frigon L, Tiernan C, Zwingman-Bagley C (1998)
Utilization-focused evaluation of acute care nurse practitioner role.
Outcomes Manag Nurs Pract 2(4): 152-160.
71. Pommier J, Guevel MR, Jourdan D (2010) Evaluation of health
promotion in schools: a realistic evaluation approach using mixed
methods. BMC public health 10(1): 43.
72. Russ-Eft D, Preskill H (2001) Evaluation in organizations: A systematic
approach to enhancing learning, performance, and change: Basic
Books, Arizona.
73. Pope C, Ziebland S, Mays N (2000) Analysing qualitative data. BMJ
320(7227): 114-116.