This document discusses performance standards and program evaluation for public health organizations. It provides the following information: 1. It outlines an upcoming assignment and exam for a class on performance standards in public health. The exam will cover readings from weeks 1-5 and be available online for a week. 2. It discusses how performance standards are used and some key questions for deciding on standards, including how to evaluate programs and use quality improvement. 3. It provides an overview of different types of program evaluation including process, impact, outcome, and cost-benefit evaluations and discusses how these can help organizations assess their mission, support, finances, and impact.

1. UIC School of Public Health
Lara Jaskiewicz, PhD
*

2. *
*Read Fallon & Zgordzinski Ch. 16 & 17
*Work on your new group assignment –
performance standards (due June 23)
*Participate in class discussion

3. *
*Available on Blackboard from June 24 through June
30
*You will have 3 hours to complete it
*You will be able to choose the time you wish to
start during a 1 week period
*The exam will cover the readings (only) for weeks
1-5
*There will be 30 questions
*The questions will be multiple choice or true/false
in format

4. *

5. *
We often hear performance standards used:
*Student to teacher ratio in the classroom
*Minutes per patient
How do we decide on a standard?
*Evaluation
*Use of quality improvement in public health
*Public health performance standards

6. *
*Is your organization functioning at a high level?
*Are you staffed appropriately to carry out your
work?
*Is the division of labor amongst staff
appropriate?
*Do you have the right external partners to help
you reach your goals?
*What impact is the organization having?

7. *
*Process Evaluation=what did you do?
* Typically counts of activities, participants, etc.
* Answers the question of ―Did I do what I planned to do?‖
*Program and Impact Evaluation=does the program work?
* May be qualitative or quantitative
* Typically measuring change in the participants, e.g. 88% of
students correctly identified the food groups and serving
sizes described in the food pyramid
* Be careful to only measure participants
* Often is short-term change
* May be best if can do pre- and post-tests, e.g. knowledge
before and after participation

8. *
*Outcome Evaluation
*May be qualitative or quantitative
*What are the longer-term or broader impacts of your
programs? (norms or behavior change, reduction in
incidence or prevalence)
*Measure change in broader population, and towards
goals
*Is measuring behavior change as a result of new
knowledge, not knowledge change due to a program
*It is tricky to identify the effects of the program as
opposed to other factors over which you may have no
control

9. *
*Cost-Benefit and Cost-Effectiveness Analysis
* How much does it cost to carry out the program versus the
health benefit received?
* Can you show the benefit in monetary value? For
instance, the WIC program saves $4 in future health care
costs for every $1 spent
* Benefits may be financial or other, such as absenteeism or
productivity savings in worksite wellness programs
* Does the program create the same results if services are
reduced, or the scale changed (sized up or down)?
* What aspects of the program make the biggest difference
in its effect? What parts are essential and which less so?

10. *
* Evaluation, especially process evaluation and cost-benefit
analysis can help organizations identify answers to the following
points:
* How well the mission fits the work
* Organization image and reputation
* Ability to obtain both monetary and nonmonetary support
* Match of Board, staff and volunteers to the need
* How well finances are managed and if expenses are
reasonable
* Ability to raise funds needed and breadth of funding sources
* Match between need and activities and quality of activities
provided
* Quality of planning and evaluation efforts—do they help the
organization reach its goals?

11. *
*Deliberate improvement efforts
*Focus on improving population health
*Is an ongoing effort
*Improvement is measurable

12. *
*National standards
*Based on the 10 Essential Public Health
Services
*Focused on the overall public health system
*Not specific programs or activities
*Are benchmarks for performance measurement
*How your organization compares to standard
*Goal is quality improvement

13. *
*TQM: a process & way of constant improvement
*Employees are responsible for the quality of their
work
*Compliance Management: a focus on results
*Problem solving approach—why not meeting goals
*Management by Objectives
*Define common goals, objectives and levels of
responsibility
*All levels of organization have input
*Is very productive, but has several
limitations, including being paper and reporting
intensive

14. *
*Management by Walking Around
*The manager is in the work environment with
employees, to see what is really going on and why
*Six Sigma
*Measure output throughout organization
*Focus on ―error‖ reduction
*Define, measure, analyze, improve, and control
problems
*Plan-Do-Check-Act
*Easy to understand process

15. *
*Answers question of whether should perform
program or step, and whether it helps to reach
goals
*Can streamline process to simplify and make more
efficient and effective
*However, a lot of process reflects how people work
or interact, simply changing the steps may not
work. Need to explain to people doing the work
what you’re trying to do and give opportunities for
meaningful input
*Don’t reengineer obsolete, illogical, or inefficient
processes. Fix them first, then reengineer them

16. *
*Do you have someone who can help problem solve or
facilitate improvements?
*Internal: know organization, may buy-in to status
quo, inexpensive
*External: objective view, but not know
organization, expensive
*Follow-up is the key to success
*This is an ongoing process
*Build in incentives to follow process
*Change is difficult and staff may resist