Slides.ppt

1,236 views
1,186 views

Published on

Published in: Health & Medicine, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,236
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
45
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Today’s presentation was developed for the Turning Point Program’s Performance Management National Excellence Collaborative by UIC School of Public Health’s Center for Public Health Practice. It is intended as an introduction and skill building course focusing on the use of performance management in pubic health practice.
  • Established in 2000, the Performance Management Collaborative is a 4-year project funded by the Robert Wood Johnson Foundation as part of the national program Turning Point: Collaborating for a New Century in Public Health . The National Turning Point Program seeks to reform and improve state-local public health systems in 21 states and supports 5 national excellence collaboratives examining performance management, public heath law, leadership development, information technology, and social marketing. The Performance Management Collaborative works to refocus performance management activities on public heath organizations and systems rather than on categorical programs. The title of its recent report “From Silos to Systems” appropriately summarizes its aspirations.
  • The intent of this training program is to promote an understanding of the strategies, principles and tools of performance management in a way that will assist public health professionals in applying these concepts to and within public health organizations and systems in their work setting.
  • There are 7 parts to today’s program as indicated on this slide. These include a pre-test and post-test. We have already completed Part 1 which was the briefest of overviews as to the origin of this training program and what we hope to accomplish here today. After the pre-test is completed we will move on to Part III in which basic terms and principles of performance management are examined. Then, in Part IV, we move into more complex applications involving performance management in public health organizations and systems. Along the way, we will look at a variety of examples and case studies. Part V involves case studies that will be examined and discussed in depth. Part VI is a review and summary. [One or more case studies can be examined in varying degrees of depth, depending on available time and on the size and composition of the audience. Initially, 5 case study scenarios will be available, although it is possible that several more could be developed for even greater flexibility in tailoring the program to different audiences. For large audiences, it may be desirable to develop smaller groups of 10-15 persons to examine the case studies. A basic set of questions can be applied to a case study in 60-75 minutes. A more extensive set of questions, derived from an accompanying self-assessment tool, can be applied in a session lasting 120 minutes or longer. The complete program is designed to take about 6-7 hours of learning and can organized into three sessions (I-II-III-IV = 2-3 hrs, V = 1-4 hrs, VI-VII = 1 hr). The program can be shortened to 4 hours be eliminating one or more examples in Part IV and/or one of the Part V case studies and slightly compressing the remaining components.]
  • Part II of today’s program involves a brief pre-test that we ask all participants to complete in order to benchmark your present familiarity with the basic concepts and use of performance management in public health practice. There will also be a post-test at the conclusion of today’s program. [Registrants can be given 10-15 minutes to complete the pre-test. Alternatively, registrants could be asked to complete the pre-test before the session begins.]
  • This part of the program presents basic terms, concepts and information on what performance management is, including several common examples that foster an understanding of a basic framework for identifying performance management’s key components. This framework will be extended in subsequent parts of today’s program. Performance Management 101 addresses the learning objective: Define and describe the key concepts and components of performance management (Learning Objective #1)
  • Before we get into the substance of this topic, however, it is useful to consider the literal meaning of the basic terms that we will be using. Performance, for example, is used to connote something that is completed or accomplished in a positive and effective manner. Management seems like a more familiar term, but it has many different nuances. We will be mainly focusing on the control and manipulation connotations in which management decisions lead to specific actions that change or modify something, hopefully, resulting in improvements.
  • Combining these terms suggests that performance management involves actions that move someone or something towards accomplishment of something such as its mission, goals and objectives. Performance management then is not a free standing science, topic or subject, rather it involves the application of various principles and practices in the real world. Key considerations are who or what is performing and what it is that is to be accomplished. This is somewhat similar to the basic relationships that we all learned in our grade school grammar classes involving subjects, predicates, and objects. If we consider performance management as something similar to an action verb or predicate, then it is critical that we know the who or what, the subject, and the object, or what will be done. Rather than frequently repeating these awkward phrases, we will refer to these as “the context” for performance management as we proceed through today’s program.
  • There are countless real world examples of performance management applications from some very simple ones to some that are extremely complex. In examining how performance management can be useful in various public health practice applications, it often helps if we use a variety of examples – including some not necessarily drawn from public health. Here is a list of examples that we will use in our program today. Several will be the focus of more extensive examination and discussion as case study exercises. At first glance, it may not appear that these items have anything in common – but hopefully we can tie them all together using themes related to performance and performance management.
  • Let’s begin with a very simple example that we can all relate to at one level or another (running or watching others run). It is also an example that allows us to readily identify some of the important concepts we associate with the term performance management . In this example, it is easy to determine the who, and equally apparent should be what this person is seeking to accomplish. Even for those of us who are not runners, it should be clear that training at various distances, and keeping track of time for these distances are potentially useful activities that can assist a serious runner in determining whether some change in their race strategy might make sense in order to maximize performance and improve on last year’s time in this event.
  • Next let’s look at an automobile as our second example. Cars are critical to moving and transporting people to all sorts of destinations, and their performance characteristics therefore are important to all of us. The modern automobile is a complex machine composed of many different materials that are fashioned into various systems that assist the car to perform its basic transportation tasks. Several of these systems provide information about the car’s performance that allow for adjustments to be made in order to improve performance. For example, information on a car’s tire pressure can lead us to make changes that improve both comfort and safety, increase the likelihood of a problem-free trip, and even reduce the overall cost of the car’s use. Computer chips in today’s cars often receive and analyze some of this information and initiate changes without the benefit of human involvement. The who/what here is obviously the automobile. The do-what is its performance in carrying out its personal transportation mission.
  • A little less information is provided for our third example, but I am confident that you can see basic performance management concepts at work here as well. Again we can appreciate that the employee is the who (our subject) and improved job performance is the object or objective. Although most of us are reasonably familiar with this particular process, we know that, in many situations, employees are not evaluated based on previously established performance expectations, and that even when they are, the results of that analysis may not influence the next cycle of expectations or the salary and promotion status of that employee.
  • A very common venue for many of us in public health is the program focus since most of us work within a defined program area and maybe even have administered or managed a discrete program or two in our careers. Much has been written to help program managers to plan, implement and evaluate their programs. And the basic components of performance management are definitely evident in professional program management circles. In this example we know who or what the subject of performance here is, and we have a sense that the program needs to perform in accord with its goals and objectives. Regular tracking of performance might lead to rethinking the program’s workplan and such information could result in program changes being made at some point in the future.
  • These first four examples reflect situations similar to those we all have encountered on many different occasions, and we should be reasonably comfortable with them. They each incorporate some common features related to the basic concepts and components of performance management that we will be examining today. All have something to do with accomplishing something important, targets, relevant information, and changes that are made to improve performance. These elements are both apparent and important in these simple examples. We will further examine these concepts and components as we move onto some more complex applications. But first we should spend a little time defining and describing them, a task that has already been completed for us by the work to date of the Performance Management Collaborative. Let’s look at what they found when they sought to identify the key components of performance management.
  • As we can see on this slide, the Turning Point Performance Management Collaborative characterizes these same simple concepts in somewhat more formal terms. The Collaborative views Performance Management as the integration of four fundamental components that respond to four basic questions: (1) What will be measured? (2) How will it be measured? (3) What will be done with the information collected? And (4) What changes will be made as result of the information collected? The PM components corresponding to these questions are: 1. Applying Appropriate Standards: the identification and establishment of organizational or system performance standards, targets and goals and relevant indicators; 2. Actually Measuring Key Aspects of Performance: the application and use of performance indicators through appropriate data collection and information management practices; 3. Reporting and Interpreting Measurements: documentation and reporting of progress toward meeting standards and targets and sharing of such information through feedback; and 4. Making Changes Based on Information derived from Measuring Performance . We might also call this Quality Improvement: establishment of a program or process to manage change and achieve quality improvement based on performance standards, measurements and reports. The Performance Management Collaborative views Performance Management as the continuous use of these four fundamental practices such that they are integrated into an organization's (or system’s) core operations. The Collaborative has developed several graphic models to convey these relationships.
  • This graphic presents the four critical components of Performance Management and several key aspects of each component: applying performance standards – the basic direction-setting activities of an organization or a system, involves at least four elements which we will discuss in a few minutes measuring performance – related to performance standards, but a very different set of activities – as we will soon see - HEDIS is an example of a process where a huge amount of time has been spent on refining measures and defining numerators and denominators reporting progress – this is about accountability – and we will discuss these elements in just a few more slides – this is an often forgotten part of a coherent process that is really about converting data into useable information Improving quality – this is about managing change – the elements of which we will encounter in a few minutes - creating a learning environment that uses information dynamically and continuously to make decisions and create change
  • This is another view of the Performance Management components as embodied in a cycle. Note that the mission and strategy drive all components of performance management. Each component in the model interacts together as an integrated process to improve performance.
  • Now that we have an idea of what performance management entails, it is time to move our focus to how performance management can be useful in public health practice and other complex situations. We actually were introduced to several different applications of performance management in public health practice when we examined how supervisors can work with their employees to improve performance and when we looked at program management, an activity that is very prevalent in public health. Before we look at performance management related to public health organizations and public health systems, we should revisit some of the basic terminology used in measuring and managing performance to be sure we know what these terms mean and how they are used. This includes terms such as performance measurement, performance measures, performance standards, performance targets, and performance indicators.
  • The terms we will be using can best be described as moving targets and sometimes mean something different to different persons, but they all advance the cause of measuring performance. Performance measurement can be simply understood to be the regular collection and reporting of data to track work produced and results achieved. This simple definition is from a primer on public health performance management developed by the Turning Point Performance Management Collaborative. It is critical to appreciate that performance measurement is not an end itself, but rather a means to an end. Measuring performance is of little value unless it leads to further decisions and actions that seek to improve performance. Nonetheless, it makes sense to eliminate that perception by using a term like performance management to clearly connote both measurement and improvement activities.
  • The terms used in performance measurement and performance management circles can be confusing although they generally convey the concepts described on this slide. In any event, don’t get hung up on the specific terms that may be used in various performance management applications if they differ from those you are most comfortable with! Look beyond the terminology for the underlying concepts. As they say, “the map is not the territory.” The terms used are far less important than the underlying concepts. The terminology outlined here is consistent with how these terms are used by several national organizations that assess performance of healthcare providers and organizations, including the Joint Commission on the Accreditation of Healthcare Organizations (JCAHO). This scheme views performance measure as a generic term that basically includes different forms of measures. Some of those measures are often benchmarks, professional guidelines, or commonly used objective standards. When specific required or expected levels of performance relative to those standards are identified, these are generally considered performance targets. Benchmarks or standards or targets, however, convey concepts that often can be assessed from different perspectives or using different evidence. The specific data or information used to asses progress toward a performance standard is a performance indicator. Let me illustrate these terms with a simple example. One possible performance measure for a public health organization or system might be the completion of a community health needs assessment. A performance standard could require that this contained several key components that are consistent with the APEXPH, IPLAN or IOM CHIP Model and is completed at least once every 5 years. The performance target might be that it is 100% completed by a certain date. The performance indicator could be evidence from the state health agency that this was done, or maybe a letter from the local board of health certifying this was done, or an onsite inspection of the completed document. Different information or evidence from different sources could be used as the performance indicator in this example although the performance standard – completion of a community health needs assessment – remains the same.
  • Let’s look at another situation in which different indicators, providing slightly different views or perspectives, can be used to assess a single standard: Performance Measure – competent workforce Performance Standard – meet national Bioterrorism and Emergency Readiness Competencies Performance Target – 90% of staff meet BT/EP competencies Performance Indicators – self assessment information from staff; supervisor assessment; drill/exercise/event performance; external credentialing process
  • It is not surprising that performance measures are useful to managers in many ways. This slide illustrates a variety of ways performance measures can be used within a program, organization, or system. These include forecasting, improvement, incentives, accountability, and communications benefits.
  • To better understand what performance management is and how it works, let’s return to this graphic presenting the 4 critical components of Performance Management and walk through the key aspects of each component. We will spend the next few minutes examining each of these 4 components in a bit more detail in terms of what each means and how it works. This should help us frame questions that will allow us to ascertain whether or not these components are present and what form they take.
  • For example, applying performance standards has several key aspects that merit elucidation. Fortunately, you are now familiar with the subtle distinctions among the terms used here: standards, indicators, targets, and measures. With those distinctions in mind, the key aspects of applying appropriate performance standards include identifying existing relevant standards, identifying appropriate indicators related to those standards, establishing targets for the expected level of accomplishment, and communicating those expectations to the various stakeholders who contribute to or benefit from their attainment.
  • At a basic level, we can look for a performance standards component by assessing whether appropriate benchmarks or standards are used, whether specific achievement targets for those standards have been developed, whether there are appropriate sources of information that can be used to measure progress related to those standards, and whether these performance expectations are communicated to those who contribute to or stand to benefit from their accomplishment. [The information below can be excluded or used to amplify on the items above by quickly communicating or synthesizing the following questions.] A more extensive set of criteria that can be used to assess the presence and adequacy of a performance standards component is provided in the Performance Management Self-Assessment Tool. This tool is included in the course packet and CD and will be used more extensively in the case study components of this course. Do you use relevant performance standards ? Do you set specific performance targets to be achieved in a certain time period? Are personnel and financial resources assigned to make sure efforts are guided by relevant performance standards and targets? Are managers and employees held accountable for meeting standards and targets? Have you defined processes and methods for choosing performance standards, indicators , and targets? 6. Are your performance standards, indicators, and targets communicated throughout the organization and its stakeholders ? 7. Do you test your standards and targets so, at minimum, you are sure people understand them? 8. Do you coordinate so multiple programs, divisions, or organizations use the same performance standards and targets?
  • The measuring performance component embodies several key features, including refining appropriate indicators and developing that the data and information systems that collect and maintain data on these indicators.
  • Assessing whether or not there is a performance measurement component centers on determining whether indicators have been refined and whether the necessary data collection and information systems are in place. Refining indicators relies on assessing key attributes such as: Validity – a valid measure is one that captures the essence of what it professes to measure Reliability – a reliable measure has a high likelihood of yielding the same results in repeated trials, so that there are low levels of random error in measurement Responsiveness – a response or sensitive measure should be able to detect change Functionality – a functional measure is directly relate to mission and goals Credibility – a credible measure is supported by stakeholders Understandability – an understandable measure is easily understood by all, with minimal explanation Availability – an available measure is readily obtained through means at hand Abuse-Proof – an abuse-proof measure is unlikely to be used against that which is, or those who are, measured [The information below can be excluded or used to amplify on the items above by quickly communicating or synthesizing the following questions.] A more extensive set of criteria that can be used to assess the presence and adequacy of a performance standards component is provided in the Performance Management Self-Assessment Tool. This tool is included in the course packet and CD and will be used more extensively in the case study components of this course. Do you have specific measures for all or most of your established performance standards and targets? Are personnel and financial resources assigned to collect performance measurement data? Are measures selected in coordination with other programs, divisions, or organizations to avoid duplication of data collection? Have you defined methods and criteria for selecting performance measures? What tools do you use? Do you have a data collection system for each measure?
  • The progress reporting component calls for analyzing data that has been collected, converting data in understandable and useable formats, and providing feedback to key constituencies on a regular basis.
  • Converting data into understandable and useable formats merits special attention as a key to successful performance management.
  • Assessing whether or not there is a progress reporting component centers on determining whether data is appropriately analyzed, converted into understandable formats and provided back to key constituencies on a regular basis. [The information below can be excluded or used to amplify on the items above by quickly communicating or synthesizing the following questions.] A more extensive set of criteria that can be used to assess the presence and adequacy of a performance standards component is provided in the Performance Management Self-Assessment Tool. This tool is included in the course packet and CD and will be used more extensively in the case study components of this course. Do you document your progress related to performance standards and targets? Do you make this information regularly available to key stakeholdeers? Are personnel and financial resources assigned to analyze performance data and report progress? Are managers held accountable for reporting performance? Have you decided the frequency of analysis and reporting on performance progress for the following types of measures? Do you have a reporting system that integrates performance data from programs, agencies, divisions, or management areas (e.g., financial systems, health outcomes, customer focus and satisfaction)? Is training available to help staff effectively analyze and report performance data? Do you test your reports so, at minimum, you are sure people understand them and can use them for decision-making?
  • The quality improvement component results in performance measures being used to improve programs and their outcomes, and fostering change management activities in a learning organization, one that continuously expands its capacity to create its own future.
  • [This slide sets the stage for examining performance in more complex systems, which is the focus of Part IV.] A system is a set of interdependent elements with a common purpose. Donald Berwick’s First Law of Improvement: Results are properties of systems. This law looks very simple but it is not! Accepting it leads into a succession of implications that eventually abrade many of the more comfortable assumptions and activities that tend to dominate in well-intended, but less well-grounded improvement theories. Results are properties of systems: every system is perfectly designed to achieve exactly the results it gets Results do not occur by new goals or targets, but through systemic change Improvement comes only with change; but change doesn’t always improve results Smart improvement relies on understanding how systems work Systems rely on interdependencies which are as important as the system’s elements Change is more difficult than setting goals, measuring or holding people accountable. “I would rather (measure, complain, blame, accept good enough, fight) than change!”
  • Assessing whether or not a quality improvement component is present centers on identifying processes to make change, manage change, and maintain a learning organization. [The information below can be excluded or used to amplify on the items above by quickly communicating or synthesizing the following questions.] A more extensive set of criteria that can be used to assess the presence and adequacy of a performance standards component is provided in the Performance Management Self-Assessment Tool. This tool is included in the course packet and CD and will be used more extensively in the case study components of this course. Do you have a process(es) to improve quality or performance? Are personnel and financial resources allocated to your QI process? Are managers and employees evaluated for their performance improvement efforts? Are performance reports used daily for decision-making? Do you have the capacity to take action to improve performance when needed? Does the organization regularly develop performance improvement or QI plans that specify timelines, actions, and responsible parties? Do your coordinate QI efforts among programs, divisions, or organizations that share the same performance targets?
  • The 4 components must be present and they must be integrated into the organization's or system’s core operations. These overall capacity and accountability features are essential for successful performance management. [The information below can be excluded or used to amplify on the items above by quickly communicating or synthesizing the following questions.] A more extensive set of criteria that can be used to assess the presence and adequacy of a performance standards component is provided in the Performance Management Self-Assessment Tool. This tool is included in the course packet and CD and will be used more extensively in the case study components of this course. Is there leadership commitment to a performance management system? Do leaders nurture an organizational culture focused on performance improvement? Are personnel and financial resources assigned to performance management functions? Is performance being managed for at least some priority areas that are critical to your mission and function? Is performance actively managed? Are managers trained to manage performance? Are managers held accountable for developing, maintaining, and improving the performance management system? Are there incentives for performance improvement? Is there coordination to align the various components of the performance management system (i.e., performance standards, measures, reports, and improvement processes focus on the same things)? Is there coordination to align your performance management system with your strategic plan? Is there coordination to align your performance priorities with your budget?
  • Let’s recap the important concepts and components that we might try to look for in the various examples we will be using today. Here is a brief list. Anything important that has been omitted?
  • Now let’s use this basic framework to summarize the key performance management features from our first 4 examples in terms of the basic concepts and components that we have described for performance management. Although we advertised these as simple examples that did not apply to public health practice, a strong case can be made that the second and third examples are commonly encountered in public health practice organizations and settings. Our first example seems very straightforward: Context: who/what = our long distance runner; do what = complete the race Standard = time needed to complete the race with a special interest in bettering last year’s time Measuring = information on how fast he/she ran various training segments Reporting = a log or whatever form the time measurements took Improvement = this performance information was used to devise a new race strategy in order to do better than last year
  • Now let’s apply this framework to our second example: Context: who/what = the car; do what = serve our personal transportation needs in terms of safety, reliability, efficiency, etc. Standard = miles per gallon, time and cost of various trips Measuring = variety of key indicators routinely tracked by car’s instruments Reporting = many via car dashboard or other instruments Improvement = change tire pressure, add oil or water, adjust speed, etc.
  • Our third example is also relatively easy to analyze: Context: who/what = we are concerned about the job performance of an employee; do what = the tasks and duties for the job Standard = probably a job description but with specific deliverables and accomplishments to be achieved by the next reporting period Measuring = work products, supervisor observations, etc. related to the employee’s expectations Reporting = perhaps monthly or quarterly reports to the boss Improvement = based on analysis of this year’s performance, new expectations are established for next year; perhaps a salary adjustment is also tied to this year’s level of performance (meets or exceeds expectations)
  • This fourth example occurs in a more complicated environment, but its features may also be quite familiar: Context: who/what = the program as an entity; do what = achieve program mission and goals Standard = specific measurable accomplishments or program objectives approved by funding agency Measuring = data and information collected by the program that allows for determination if objectives were accomplished Reporting = often monthly or quarterly and based on progress in achieving objectives Improvement = revised workplan for next year, perhaps there are even financial incentives or penalties linked to this year’s performance
  • So far, so good – but it is necessary to extend our discussion of these elements to more complex situations encountered in public health practice. So on we go.
  • Next we will focus on applications of performance management in public health practice, the central topic of today’s program. This part of the program addresses the following key learning objectives: Identify performance management concepts and components in a variety of public health organization and system applications (Learning Objective #2) Assess and enhance performance management practices and opportunities in public health work settings (Learning Objective #4) Identify and access resources to support performance management applications (Learning Objective #6)
  • Let’s pick up where we left off and look at some slightly more complex situations involving public health programs. Let’s assume the program management example we used in Example 4 (we should now call it Example 4A) was a program to reduce tobacco use among health department employees. Let’s then extend this same program to one that targets all tobacco users in the community (we can call this Example 4B) and then to a network of such programs throughout the state (we will call this Example 4C). Let’s now see how this might affect the basic concepts and components involved in performance management for these programs of quite different scales. What is readily apparent is that the complexity of the “program” has increased from Example 4A as we examine it at the community level and than at the statewide level. Although each level still has an identifiable program with clearly established program goals, virtually all components have taken on increased levels of complexity. Certainly the program’s scope is more complex as is the range of necessary participants. Although standards and targets may be similar, more information from more sources makes measuring and reporting to more parties more difficult. Finally, efforts to make change and improve outcomes face many more intervening variables and factors in order for success to occur and be measured. Which performance management components make performance management more difficult in Example 4C than in Example 4A? [Elicit ideas from the audience if possible!]
  • Here is another example, this one at the national level, perhaps with somewhat less complexity. We might consider this a new national health priority. In view of heightened awareness of childhood obesity, let’s build an example of what this national program might look at – based on what we have seen with another recent national health priority, bioterrorism preparedness.
  • Let’s use our simple framework as a quick and dirty screening tool to see which components of performance management are evident in this example. We may need to dig a little deeper to identify some of these components since the information provided for this example was pretty skimpy. So let’s see what we can identify and what we need to dig a little deeper for.
  • For this Example, we will attempt to answer the questions posed on the previous slide. We realize that very little information has been provided for this example, so we will all have to make some assumptions and fill in some gaps in order to do this. But this is just for practice and is a prelude to what we will be doing in later parts of this program with more detailed case studies. [Consider having the entire audience write down brief responses to each of these queries or, alternatively, have 5 different sections of the audience do one of the 5. Then ask for volunteers to indicate their responses in order to engage and interact with the audience as time permits.]
  • Summarizing the audience contributions: Context: who/what = not easy to determine who or what this massive program or system is and who is involved; do What = accomplishment here is conceptually easy to understand Standards = certainly there are standard approaches to assessing individual and population-wide childhood obesity Measuring = there are even sources for information to track this Reporting = less clear who needs to be reached with measurement information Improvement = and even less clear exactly what changes would be useful
  • Keeping our focus on the national level, and arguably more complex than our previous examples, we might examine performance of the national network of state-based public health systems. Subjects and objects are similar to state systems, but approaches to setting targets, measuring and reporting performance, and performance improvement appear less well developed.
  • This figure illustrates how standards (similar to the national public health performance standards) might be used at several points in time to benchmark performance of public health core functions against national information. Standards, measuring, and reporting all illustrated here. What’s missing here is the change or improvement piece.
  • Let’s use our simple framework as a quick and dirty screening tool to see which components of performance management are evident in this example. We may need to dig a little deeper to identify some of these components since the information provided for this example was pretty skimpy. So let’s see what we can identify and what we need to dig a little deeper for.
  • For this Example, the national Public Health System, we will attempt to answer the questions posed on the previous slide. We realize that very little information has been provided on this case study, so we will all have to make some assumptions and fill in some gaps in order to do this. But this is just for practice and is a prelude to what we will be doing in later parts of this program with more detailed case studies. [Consider having the entire audience write down brief responses to each of these queries or, alternatively, have 5 different sections of the audience do one of the 5. Then ask for volunteers to indicate their responses in order to engage and interact with the audience as time permits.]
  • For this Example, the national Public Health System, this analysis leads us to conclude that performance management components are not in place and therefore not driving improvement efforts. We might make the same conclusions from our cursory look at the national public health priority example focusing on childhood obesity. Perhaps the vast scale here, the national public health system, is too large to permit effective performance management strategies.
  • Perhaps these national level examples involve applications so complex that performance management strategies are not readily employed. Or not as readily employed as some of the less complex program focused examples we examined earlier today. So let’s focus our attention on applications somewhere in between programs and national efforts such as local and state public health organizations and systems. One key characteristic of performance management is the notion of “active and strategic use”. Data mean nothing if they are not applied and used regularly and effectively. Performance management needs to be viewed as a balanced management model for public health, not just relying on measurement. Remember the discussion of “change” from our earlier session!
  • The Performance Management Collaborative views public health performance measures as “q uantitative measures of capacities, processes, or outcomes relevant to the assessment of public health performance.” This slide provides some simple examples of capacity, process and outcome measures for a common public health duty, prevention and control of communicable diseases. To better understand these various types of measures it is necessary to think about public health organizations and systems in terms of their structures, processes and outcomes. The next few slides present these concepts graphically.
  • This graphic presents a conceptual framework of the public health system. To the Performance Management Collaborative, this chart represents the big picture of how performance measurement fits into the larger system. This framework was published in the APHA Journal in 2001 and has been widely accepted by public health system researchers as a useful conceptual model. [This slide can be eliminated for some audiences – it may be too abstract. The slide that follows may be more readily understood.] [This complex slide can be broken down and presented in a step by step manner to facilitate understanding of its concepts and logic.] The macro context is open-ended. Issues arise or emerge, and become defined or redefined as public health issues. An example is violence, which 20 years ago was not a public health issue. Today, we would have trouble not thinking of it as a public health concern. In the macro context, changes in law or funding are ways in which issues become defined as public health issues and become incorporated into the public health system, which is also open-ended to accommodate these emerging public health issues. Within the public health system, the public health mission and purpose is our core. It is stable and solid, regardless of issues that are being addressed, and it drives program or policy. The public health system mission and purpose is operationalized by structure, process and outcome. Structural capacity, or infrastructure, includes organizational resources like partnerships. List the other elements of structural capacity. The elements of structural capacity allow us to perform the 10 essential public health services, now well recognized as the processes of public health practice. Performance of these 10 essential services leads to outputs in program and policy, not shown on the chart, and hopefully to outcomes. Structure, process and outcome are shaded in the chart because they are the targets of effective system-wide performance management.
  • This animated graphic adds some additional detail and insights into performance measures related to the performance of public health organizations and systems. This model suggests that it is necessary to assess different types of performance measures rather than focusing on outcome measures alone. The chain of events – or logic model – here indicates that capacity measures influence processes (and process measures) that, in turn, influence outputs and outcomes. Outcomes themselves feedback to influence capacity measures. Although many complex concepts and issues are captured in this animated graphic, it helps us to understand how public health systems work and how we can measure and assess their performance. This and the previous slide also illustrate that overall public health organization or system performance requires the collection and analysis of measures at all levels of the system (capacity, process, outputs, outcomes) and relating these measures to each other.
  • Using these models and frameworks, the Performance Management Collaborative identified several important public health applications that might benefit from the application of performance management principles and strategies. The Collaborative then surveyed the 50 states to learn more about how states were using performance management for these various applications. The results of that survey provide information on the current state of the art!
  • This slide summarizes several important lessons identified in the Turning Point Collaborative survey of performance management activities in the various states. There are a variety of performance management activities in place, although these are often not statewide and not often linked with quality improvement. Where performance management was used, however, there was the perception that they were effective in achieving goals and outcome objectives. There is no consensus among states on useful frameworks although the Healthy People 2010, Core Public Health Functions, Essential Public Health Services, and National Public Health Performance Standards are the most widely used approaches.
  • To what areas of public health practice has performance management been applied to date? This survey found the components of performance management were most likely to be in place for health status (not surprising, since there are a number of tools available to assist users in health status assessment). More interesting, perhaps, is that performance management activities were least likely to be in place for human resource development or management of public health capacity.
  • The Performance Management Collaborative’s survey of states found that nearly all states have some performance management efforts, but only about half of these are statewide efforts. For purposes of this survey, statewide was defined very broadly, to encompass state-level efforts, state plus local health department efforts and local health department efforts only. The other (approximately) half are targeted at categorical programs only or had no PM effort at all.
  • The Turning Point Performance Management Collaborative found that state health agencies were very different in the array of performance management components they have in place. Overall, they were least likely to have a process for quality improvement or change. They were most likely to have performance measures or standards.
  • About two-thirds of the states reported that they don’t use incentives or disincentives to manage performance. And only a very small percentage of states use incentives and disincentives within their organizations, or in managing their outside grants and contracts, and even less use incentives or disincentives with staff.
  • The Performance Management Collaborative survey found that the first 3 sources of performance measures listed on this table (Healthy People 2010, Core Functions, and EPHS) were the most widely used by states, followed by: (4) state specific performance frameworks (5) community assessment and planning frameworks like APEX-PH, MAPP, and PATCH (6) Healthy People 2010 Leading Indicators (7) National Public Health Performance Standards (8) HEDIS or other clinical performance management systems (9) federal performance frameworks such as GPRA (10) Healthy Cities/Healthy Communities (11) Baldridge Award Criteria (12) Others (13) Balanced Scorecard This survey antedated the availability of the National Public Health Performance Standards. APHEX-PH and MAPP are tools used at the local level and therefore were not frequently cited in the survey of states, even though APEX-PH and MAPP appear to be more robust sources of public health performance measures. This slide suggests that a good combination of tools for use by states would include the Healthy People 2010 objectives and the State NPHPS Instrument. Similarly, for local health jurisdictions, Healthy People 2010 and either MAPP or APEX cover the gamut of capacity, process, and outcome measures of performance for public health agencies and systems. As some in the audience may not be familiar with one or more of these sources, descriptions are available in the online Performance Management Toolkit and in other references and resources that are identified later in this program.
  • Our examples to this point have included some relatively simple ones that most of us encounter frequently in our personal and professional lives and a series of somewhat more complex public health applications related to specific programs and to the national public health enterprise. Modern public health practice also involves some pretty complex organizations and systems at the state and local level whose performance is very important to us. One example might be a large state health agency that operates all the local public health units in a state. There are a number of examples of this arrangement in the U.S. Here it is still possible to identify the who and the what is to be accomplished.
  • Let’s use our simple framework as a quick and dirty screening tool to see which components of performance management are evident in this example. We may need to dig a little deeper to identify some of these components since the information provided for this example was pretty skimpy. So let’s see what we can identify and what we need to dig a little deeper for.
  • For this Example, the complex State Health Agency, we will attempt to answer the questions posed on the previous slide. We realize that very little information has been provided on this case study, so we will all have to make some assumptions and fill in some gaps in order to do this. But this is just for practice and is a prelude to what we will be doing in later parts of this program with more detailed case studies. [Consider having the entire audience write down brief responses to each of these queries or, alternatively, have 5 different sections of the audience do one of the 5. Then ask for volunteers to indicate their responses in order to engage and interact with the audience as time permits.]
  • For this Example, the complex State Health Agency, we will attempt to answer the questions posed on this and the previous slide. We realize that very little information has been provided on this case study, so we will all have to make some assumptions and fill in some gaps in order to do this. But this is just for practice and is a prelude to what we will be doing in later parts of this program with more detailed case studies. This first set of questions basically asks us to identify the key performance management components in this case. Context: who/what = state health agency with its local units under its direct control; not clear whether other entities are included – do what hat = promote, protect, improve health Standards = HP 2010 for health status and health service use; Baldridge criteria for quality Measuring = outcomes, activities, costs, satisfaction, etc. Reporting = monthly to agency management Improvement = incentive for progress; information for resource allocation [Discuss missing, strong, and weak components .]
  • A somewhat similar, and arguably more complex, example might involve s state-local public health network of state and local public health agencies. Here the who and the what is to be accomplished are largely the same as in the previous example, but how this would be accomplished might well be different if we had more in depth information on this system.
  • Let’s use our simple framework as a quick and dirty screening tool to see which components of performance management are evident in this example. We may need to dig a little deeper to identify some of these components since the information provided for this example was pretty skimpy. So let’s see what we can identify and what we need to dig a little deeper for.
  • For this Example, the complex State-Local Public Health Network, we will attempt to answer the questions posed on this and the previous slide. We realize that very little information has been provided on this case study, so we will all have to make some assumptions and fill in some gaps in order to do this. But this is just for practice and is a prelude to what we will be doing in later parts of this program with more detailed case studies. [Consider having the entire audience write down brief responses to each of these queries or, alternatively, have 5 different sections of the audience do one of the 5. Then ask for volunteers to indicate their responses in order to engage and interact with the audience as time permits.]
  • Summarizing the suggestions and input from the audience for this example: Who/What = state health agency, local public health agencies independent but part of network Do What = promote, protect, improve health Standards = HP 2010 for outcomes, core function based LHD standards, activities reported? Measuring = outcomes, LHD standards, activities? Reporting = unclear what is reported back to state health agency Improvement = not clear if decisions or resource deployment affected [Discuss missing, strong, weak components.]
  • Let’s revisit our catalog of examples. We have touched on many of these already but, in order to take our understanding of these issues to the next level, we will now focus on two of these applications in greater depth. Depending on the size of the group here today, we may need to break up into smaller groups to dissect and analyze these case studies in terms of their performance management components. As part of this exercise, we will ask each of you to describe an activity that you are interested in or currently involved in and have you identify how you might approach that activity from a performance management perspective. The ultimate aim is not just for you to be familiar with these concepts but we want for you to be able to use them in looking at current efforts and in designing future ones. The general application related to the two case studies that we will examine today are highlighted on the slide.
  • [The case study component of this program can be tailored to the time available and to the size and particular needs of the audience. Program coordinators and presenters can select one or more of 5 case study scenarios that involve performance management at the local or state level. Most involve both state and local public health agencies. A basic set of questions, similar to those used in Part IV, provides a framework for the entire audience or break-out groups to analyze a case study in a 60-75 minute session. A more extensive set of questions, derived from the Self-Assessment tool developed by the PM Collaborative, provides the framework for more in depth analysis in a session lasting up to 120 minutes. Depending on time, audience and logistics, various combinations may be appropriate (2 one-hour sessions, 1 one-hour and 1 two-hour session, 2 two-hour sessions).]
  • Our first case study examines a community health improvement effort taking place in Moose County where the decision has been made to use the NACCHO’s new Mobilizing for Action through Planning and Partnerships (MAPP) process.
  • Community health improvement activities emerged as the cornerstone of local public health practice in the 1990s, responding to the 1988 IOM report on The Future of Public Health and to several more focused IOM reports released in the mid and late 1990s. Beginning in the early 1990s, local public health agencies became more actively engaged in community health planning with many local health jurisdictions adopting or adapting the APEX-PH model developed by NACCHO. MAPP is the next generation version of APEX-PH embodying virtually all the principles and strategies for community heath improvement processes (CHIPs) promoted by the IOM and other national public health organizations. CHIPs seek to assess and address priority community health problems based on a view of the community as system and community health problems as phenomena affecting many different sectors of the community and thereby requiring multi-sectoral responses and broad community collaborations and partnerships for health purposes. A community health needs assessment drives the process, leading to program development and implementation. Evaluation of intervention strategies feeds back into an ongoing assessment of community needs.
  • The MAPP model for community health improvement builds on the successes and lessons from a decade’s worth of experiences with APEX-PH and similar tools. MAPP adopts the view that the community is a system and seeks to assess community needs through several different lens using 4 separate assessment tools: Community health status Forces of change Community themes Local public health system assessment (based on the local National Public Health Standards instrument)
  • Another view of the MAPP Model is provide here, emphasizing how these 4 assessments fit into a strategic planning process for community health improvement.
  • Key elements of a community health plan are illustrated on this slide. A priority health problem is identified (in this case study, cerebrovascular disease was identified as a priority, and analyzed for important contributing factors that exist within the community and that therefore can be addressed by community actions and resources. Measures of health problems and their causative factors are used both to plan and to evaluate intervention strategies.
  • The community health plan for this priority identifies broad strategies and specifies actions to be taken, responsible parties for implementation, and evaluation activities.
  • Let’s use our simple framework as a quick and dirty screening tool to see which components of performance management are evident in this example. We may need to dig a little deeper to identify some of these components since the information provided for this example was pretty skimpy. So let’s see what we can identify and what we need to dig a little deeper for.
  • For the Moose County Community Health Improvement Case Study, we will attempt to answer the questions posed on the previous slide. We realize that very little information has been provided on this case study, so we will all have to make some assumptions and fill in some gaps in order to do this. But this is just for practice and is a prelude to what we will be doing in later parts of this program with more detailed case studies. [The entire audience, or an entire break-out group, will write down brief responses to each of these queries or, alternatively, have 5 different sections of the audience do one of the 5. Then ask for volunteers to indicate their responses in order to engage and interact with the audience as time permits.] Who/What = Moose County Health Partnership Do What = improve community health status by addressing priority health problems Standards = HP 2010, MAPP assessments (including local version of NPHPS) Measuring = outcomes and community contributing factors Reporting = Partnership, policy makers, public Improvement = not clear, depends on commitment of partners to resource deployment or redeployment Discuss missing, strong, and weak components.
  • Summarizing the information from the audience or break-out group: Context: who/what = Moose County Health Partnership; do what = improve community health status by addressing priority health problems Standards = HP 2010, MAPP assessments (including local version of NPHPS) Measuring = outcomes and community contributing factors Reporting = Partnership, policy makers, public Improvement = not clear, depends on commitment of partners to resource deployment or redeployment [Discuss missing, strong, and weak components.]
  • [These questions build on our ability to identify performance management concepts and components in this case study and ask course participants to relate this case study to their own experience. It may not be possible to address these issues in case study discussions if time is limited. This entire case study could focus on how the state in which Moose County resides carries out a statewide health improvement process.] [Course participants may have been involved in one or more previous cycles of a CHIP process. If so, what worked? What didn’t work? What needs to work better the next time around? How do they see a similar type activity taking place at the state level? What role would local assessments play in a statewide CHSIP?]
  • Let’s now switch gears to examine a second case study, this one focusing on public health workforce development in a state-local public health system. Our case is that of the Coyote County Health Department, which is one of 98 local health jurisdictions in Grey State.
  • One approach to competency-based workforce preparedness involves four major components all organized around a consistent set of practice relevant competencies for public health workers, and linked together by a learning management system that is web-accessible and user friendly.
  • Here is slide we saw earlier today that illustrates how different indicators, providing slightly different views or perspectives, can be used to assess a single standard such as a set of workforce competency expectations. Performance Measure – competent workforce Performance Standard – meet national Bioterrorism and Emergency Readiness Competencies Performance Target – 90% of staff meet BT/EP competencies Performance Indicators – self assessment information from staff; supervisor assessment; drill/exercise/event performance; external credentialing process
  • Grey State and Coyote County are working to increase the preparedness of the public health workforce for specific bioterrorism and emergency readiness competencies for public health workers identified by CDC and other national public health organizations. This competency formulation identifies 9 core competencies for all public health workers and a panel of more function-specific competencies for 9 different categories of public health workers, including leaders, epidemiology and communicable disease staff, clinical staff, environmental health staff, public health laboratory staff, medical examiner/coroner staff, public information staff, other professional staff, and technical and support staff.
  • Virtually all Coyote County Health Department staff were assessed within a two week period.
  • This slide catalogs the training plan developed by Coyote County based on the initial assessment of competency-based training needs.
  • This chart summarizes the experience of Coyote County Health Department’s use of the Grey State’s LMS to document pre and post training competency levels among their staff (~210 employees). A training plan was developed based on the initial assessment of agency staff and carried out over approximately a 6 week period. Agency staff were then re-assessed and the impressive results are shown here. Notably, a parallel objective assessment was carried out by an external evaluator who found the post-training levels to actually be even higher than the figures derived from self-reporting! This training assessment and training plan resulted in Coyote County meeting its target of 90% of its workers meeting competency expectations for each of the 9 core public health emergency readiness competencies.
  • Let’s use our simple framework as a quick and dirty screening tool to see which components of performance management are evident in this example. We may need to dig a little deeper to identify some of these components since the information provided for this example was pretty skimpy. So let’s see what we can identify and what we need to dig a little deeper for.
  • For the Coyote County Health Department’s Workforce Development Case Study, we will attempt to answer the questions posed on the previous slide. We realize that very little information has been provided on this case study, so we will all have to make some assumptions and fill in some gaps in order to do this. [The entire audience, or an entire break-out group, will write down brief responses to each of these queries or, alternatively, have 5 different sections of the audience do one of the 5. Then ask for volunteers to indicate their responses in order to engage and interact with the audience as time permits.]
  • Summarizing the participant input for the Coyote County Health Department’s Workforce Development Case Study: Context: who/what = public health workforce (state and local or one local public health agency); do what = competent public health practice Standards = competency expectations Measuring = multiple views beginning with self-assessment Reporting = back to individual, agency management, state Improvement = via tailored training plans, incentives [Discuss missing, strong and weak components.}
  • [These questions build on our ability to identify performance management concepts and components in this case study and ask course participants to relate this case study to their own experience. It may not be possible to address these issues in case study discussions if time is limited. This entire case study could focus on how Grey State uses information from its learning information management system to further statewide public health workforce preparedness objectives.]
  • Welcome back – this is the final session of today’s program. It is an opportunity to review and summarize the themes, information, and insights that we have encountered in the earlier parts of the program. As a touchstone, let’s focus once again on the critical components involved in performance management. This part of the program addresses the following learning objectives: Describe potential benefits of performance management across a range of public health organization and system applications (Learning Objective #3) Assess and enhance performance management practices and opportunities in public health work settings (Learning Objective #4) Advocate for the use of performance management in public health practice settings (Learning Objective #5) Identify and access resources to support performance management applications (Learning Objective #6)
  • This graphic once again presents the four critical components of Performance Management and several key aspects of each component: applying performance standards – the basic direction-setting activities of an organization or a system, involves at least four elements which we will discuss in a few minutes measuring performance – related to performance standards, but a very different set of activities – as we will soon see - HEDIS is an example of a process where a huge amount of time has been spent on refining indicators and defining numerators and denominators reporting progress – this is about accountability – and we will discuss these elements in just a few more slides – this is an often forgotten part of a coherent process that is really about converting data into useable information Improving quality – this is about managing change – the elements of which we will encounter in a few minutes - creating a learning organization that can use information dynamically and continuously to make decisions and create change
  • Let’s review our various examples in terms of how each applied standards and established target, remembering that this component of performance management has several important components. The key aspects of applying appropriate performance standards include identifying existing relevant standards, identifying appropriate indicators related to those standards, establishing targets for the expected level of accomplishment, and communicating those expectations to the various stakeholders who contribute to or benefit from their attainment. These component can be assessed by asking: Have performance goals and targets been established, and appropriate indicators identified? Do these benchmark against similar organizations or use national, state, or scientific guidelines? Are expectations communicated? Expectations are key here. Clear and understandable expectations are critical. The value of using standard approaches to characterizing key performance concepts such as outcome, processes and capacity cannot be overstated. Here we see that standard approaches for many of the public health applications involved Healthy People 2010 objectives and measures developed as part of the National Public Health Performance Standards Program.
  • Next, let’s review our various examples in terms of how each actually measured performance, remembering that this component of performance management has several important components, including refining appropriate indicators and developing that the data and information systems that collect and maintain data on these indicators. These component can be assessed by asking: Have the indicators been refined? Have data systems been developed? Are data being collected? How various phenomena will be measured is key here, as are the specific measures to be collected for analysis. Selection of measures depends on several considerations and the measures used must be important in the chain of causation or logic model.
  • Next, let’s review our various examples in terms of how each reports measurement information, remembering that this component of performance management has several important components which answer questions such as: Are data analyzed? Are data fed back to managers, staff, constituents, etc. in useable and understandable format? Is there a regular reporting cycle in place? The form and format of reporting varies considerably across these examples. For some, it appears this component is less well developed than applying standards and measuring performance.
  • Next, let’s review our various examples in terms of how each structures change designed to improve results, remembering that this component of performance management has several important components which answer questions such as: Where is the change process? Is there a process to manage changes being made? Is there a learning organization? Even more so than for the reporting component, the form of change and quality improvement varies from example to example and is lacking or unclear in more than a few.
  • This is another view of the Performance Management components as more of a cycle. Note that the mission and strategy drive all components of performance management. Each component in the model interacts together as an integrated process to improve performance. The interdependencies within this cycle should be apparent from the examples we have examined. Applying standards is quite frequently done Measuring performance, a little less so Reporting somewhat less frequent than measuring, and Quality improvement component is least likely among the four.
  • This last point is especially critical for effective, successful, and sustainable performance management efforts!
  • Finally, let’s review our various examples in terms of how each structures capacity and accountability to improve results, remembering that all four components of performance management should be integrated into the core operations of an organization or system so that it becomes a true learning organization like that depicted on the following slide. Several features are common to all our examples: commitment, leadership, informed decision making, system incentives, and accountability. To paraphrase the wisdom of Aristotle, excellence is not an isolated event; rather it is a habit. Capacity and accountability are essential elements of performance management if it is to become ingrained in the culture of an organization or system.
  • Public Health Performance Management [date] Assuring capacity and accountability to promote learning organization is a complex task in its own right. A learning organization is an organization that is continually expanding its capacity to create its future. In a learning organization, the system becomes the guiding mechanism for change. This allows the organization to keep pace with the rapid rate of change in its environment, to function in a more interdependent manner, and to respond to the changing needs of society.
  • Increasingly, there are success stories and effective tools to foster performance management strategies in tools for various public heath practice applications. These can be found in several sources provided in the resources and references cited at the end of this program. The experience to date from successful performance management applications suggests there are several factors necessary for success. Integrate PM into routine public health processes Sustainable PM activities meet state and local needs and political realities NPHPS, management models and tools provide a head start Early stakeholder involvement increases support and chances of success
  • Continuing, the experience to date from successful performance management applications suggests there are several factors necessary for success. Align PM measures, activities, and spending with public health priorities Trained staff, dedicated resources, and PM culture are essential Baseline information and trends important New or adapted information and management systems are necessary for cross-program management Incentives motivate performance and quality improvement
  • These various success factors support the contention that performance measures are useful to managers in many ways. This slide illustrates a variety of ways performance measures can be used within a program, organization, or system. These include forecasting, improvement, incentives, accountability, and communications benefits.
  • The basic concepts and principles involved with performance management are not new. They have been around for many years and in a variety of different forms. They are now foundations of the way we develop and evaluate programs. Despite this, Turning Point’s Performance Management Collaborative concluded that performance management was not being effectively applied to more complex public health applications, including state health agencies, local and state public health systems, and to national public health efforts. After more than four years of examination, the Performance Management Collaborative moved to promote performance management applications for public health organizational and systems performance issues. The key is to integrate the four components into a continuous system.
  • Public health, like other sectors, needs management results directed toward quality improvement, policy development, resource allocation, and programmatic direction. These are the useful outputs of managerial action and these are facilitated by the concepts and principles of performance management that we are discussing today.
  • There are several very useful resources related to performance management in public health practice. The Performance Management Collaborative has produced four documents on the topic. The Guidebook (#2) and the Silos to Systems report (#5) are especially relevant. The Public Health Foundation worked closely with the Performance Collaborative in the development of three of these documents and will serve as a legacy coordinating role for promoting performance management after funding for the Collaborative expires. The Performance Management Self-Assessment Tool is an especially useful resource for examining and analyzing performance management components in various public health practice applications. The Public Health Foundation manages an online resource catalog titled the Performance Management Toolkit, which brings together a variety of tools, reports, case studies, and implementation experiences.
  • Today’s program will end in a manner similar to which it began, with a post-test to measure what learners have attained. There will also be a course evaluation that focuses on other aspects of this training program so that it can be refined and improved consistent with the performance management principles and concepts we have discussed today.
  • Slides.ppt

    1. 1. Public Health Performance Management Turning Point Performance Management National Excellence Collaborative [meeting name] [location] [date] Public Health Performance Management Curriculum Prepared by Center for Public Health Practice, UIC School of Public Health
    2. 2. Genesis <ul><li>Turning Point Performance Management National Excellence Collaborative States (AK, IL, MO, MT, NH, NY, WV) </li></ul><ul><ul><li>“..to move the field of public health from simply measuring performance of individual programs to actively measuring and managing the performance of an entire agency or system.” </li></ul></ul><ul><ul><li>“..from managing silos to managing a system” </li></ul></ul>Source: Silos to Systems: Using Performance Management to Improve the Public’s Health. Turning Point Performance Management National Excellence Collaborative: Seattle WA; Turning Point National Program, 2003.
    3. 3. Learning Objectives <ul><li>Learners will be able to: </li></ul><ul><li>Define and describe the key concepts and components of performance management </li></ul><ul><li>Identify performance management concepts and components in a variety of public health organization and system applications </li></ul><ul><li>Describe potential benefits of performance management across a range of public health organization and system applications </li></ul><ul><li>Assess and enhance performance management practices and opportunities in public health work settings </li></ul><ul><li>Advocate for the use of performance management in public health practice settings </li></ul><ul><li>Identify and access resources to support performance management applications </li></ul>
    4. 4. Today’s Program <ul><li>Introduction </li></ul><ul><li>Pre-Test </li></ul><ul><li>Performance Management 101 </li></ul><ul><li>Applications of Performance Management in Public Health Practice </li></ul><ul><li>Case Studies </li></ul><ul><li>Review and Summary </li></ul><ul><li>Post-Test/Course Evaluation </li></ul>
    5. 5. Part II Pre-Test
    6. 6. Part III Performance Management 101
    7. 7. The Words <ul><li>“Performance” </li></ul><ul><ul><li>Execution, Accomplishment, Completion, Effectuation </li></ul></ul><ul><li>“Management” </li></ul><ul><ul><li>Supervision, Direction, Control, Manipulation </li></ul></ul>
    8. 8. Basic Concepts <ul><li>“Performance Management” </li></ul><ul><ul><li>Control/Manipulation towards Accomplishment </li></ul></ul><ul><ul><li>Key considerations = the context </li></ul></ul><ul><ul><ul><li>Who or what performs? </li></ul></ul></ul><ul><ul><ul><li>What is performed? </li></ul></ul></ul>
    9. 9. Focus on Performance <ul><li>Marathon Runner </li></ul><ul><li>Not Your Father’s Oldsmobile </li></ul><ul><li>Unit Supervisor </li></ul><ul><li>Program (and Mega-Program) Manager </li></ul><ul><li>National Health Priority Initiative </li></ul><ul><li>National Public Health System </li></ul><ul><li>State Health Agency </li></ul><ul><li>State-Local Public Agency Network </li></ul><ul><li>Community Health Improvement Process </li></ul><ul><li>Public Health Workforce Development </li></ul>
    10. 10. Managing Performance (Ex 1) <ul><li>Marathon Runner </li></ul><ul><li>trains 5x per week at various distances </li></ul><ul><li>records times </li></ul><ul><li>seeks to better last year’s time </li></ul>
    11. 11. Managing Performance (Ex 2) <ul><li>Your Father’s Oldsmobile? </li></ul><ul><li>Performance dimensions include fuel economy, comfort, safety, etc. </li></ul><ul><li>Raw materials include steel, plastic, rubber </li></ul><ul><li>Tire air pressure measurements </li></ul><ul><li>Dashboard instruments assist operators in making some adjustments; onboard computer chips also monitor performance </li></ul>
    12. 12. Managing Performance (Ex 3) <ul><li>Unit Supervisor </li></ul><ul><li>meets with employees annually to review job performance and set expectations for next year </li></ul><ul><li>next year’s salary increment is tied to this year’s job performance? </li></ul>
    13. 13. Managing Performance (Ex 4) <ul><li>Program Manager </li></ul><ul><li>establishes objectives </li></ul><ul><li>reports quarterly </li></ul><ul><li>seeks continuation funding year after year </li></ul>
    14. 14. Common Elements from These Simple Examples? <ul><li>Context (who or what is to perform and what is to be accomplished) </li></ul><ul><li>Goals or targets (form & level of accomplishment) </li></ul><ul><li>Information about performance </li></ul><ul><li>Collection of information </li></ul><ul><li>Modifications/Change possible </li></ul>
    15. 15. Key Components of Performance Management <ul><li>Applying Appropriate Standards </li></ul><ul><li>Measuring Key Aspects of Performance </li></ul><ul><li>Reporting and Interpreting Measurements </li></ul><ul><li>Making Changes Based on Measures of Performance </li></ul>
    16. 16. Critical components of performance management Source: Silos to Systems: Using Performance Management to Improve the Public’s Health. Turning Point Performance Management National Excellence Collaborative: Seattle WA; Turning Point National Program, 2003.
    17. 17. <ul><li>In the performance management cycle... </li></ul><ul><li>All components should be driven by the public health mission and organizational strategy </li></ul><ul><li>Activities should be integrated into routine public health practices </li></ul><ul><li>The goal is continuous performance and quality improvement </li></ul>Source: Silos to Systems: Using Performance Management to Improve the Public’s Health. Turning Point Performance Management National Excellence Collaborative: Seattle WA; Turning Point National Program, 2003.
    18. 18. Is Performance Management the Same Thing as Performance Measurement? [Maybe Yes / Maybe No]
    19. 19. <ul><li>Performance Measurement is the regular collection and reporting of data to track work produced and results achieved. </li></ul><ul><li>Performance Management is what you do with information you’ve developed from measuring performance. </li></ul><ul><ul><li>Set agreed-upon performance goals </li></ul></ul><ul><ul><li>Allocate or prioritize resources </li></ul></ul><ul><ul><li>Inform management decisions </li></ul></ul><ul><ul><li>Report on progress </li></ul></ul><ul><li>Caution! Some view Performance Measurement as only the first 3 components of Performance Management </li></ul>Source : lichiello P. Guidebook for Performance Measurement . Seattle WA: Turning Point National Program Office, 1999:48. Based on Hatry HP et al, Monitoring the Outcomes of Economic Development Programs. Washington DC: The Urban Institute Press, 1990.
    20. 20. Jargon Alert! [Terms may be used differently] <ul><li>Performance Measure </li></ul><ul><ul><ul><li>A specific quantitative representation of something deemed relevant to the assessment of progress toward a goal or objective </li></ul></ul></ul><ul><li>Performance Standard </li></ul><ul><ul><ul><li>Standards are one form of performance measure; they are objective measures or guidelines that are commonly used to assess performance (“standard” here means a standard way of measuring rather than something specific to achieve) </li></ul></ul></ul><ul><li>Performance Target </li></ul><ul><ul><ul><li>The planned or expected level of performance (generally expressed in standard terms) </li></ul></ul></ul><ul><li>Performance Indicator </li></ul><ul><ul><ul><li>Indicators are another form of performance measure; they are the data or information that is used to assess progress toward a performance standard or target </li></ul></ul></ul>
    21. 21. Different Indicators Used to Measure Performance (Workforce Competency) Credentialing Application External Certifying Body Assessment Personnel Mgt/ Competency Demo Application Workplace Assessment Competency Acquisition Application Course Instructor Assessment Self Improvement Application ---------> ---------> ---------> ---------> Self Assessment
    22. 22. Performance Measures Are Useful to … <ul><li>Identify aspects of the work that have and have not resulted in satisfactory results </li></ul><ul><li>Identify trends </li></ul><ul><li>Further investigate the nature of particular problems </li></ul><ul><li>Set targets for future periods </li></ul><ul><li>Motivate managers and staff to improve performance </li></ul><ul><li>Hold managers and staff accountable </li></ul><ul><li>Develop and improve programs and policies </li></ul><ul><li>Help design policies and budgets and explain these to stakeholders </li></ul>Source : lichiello P. Guidebook for Performance Measurement . Seattle WA: Turning Point National Program Office, 1999:48. Based on Hatry HP et al, Monitoring the Outcomes of Economic Development Programs. Washington DC: The Urban Institute Press, 1990.
    23. 23. Critical components of performance management Source: Silos to Systems: Using Performance Management to Improve the Public’s Health. Turning Point Performance Management National Excellence Collaborative: Seattle WA; Turning Point National Program, 2003.
    24. 24. Applying Appropriate Standards <ul><li>Identify and apply relevant standards (what will be measured) </li></ul><ul><ul><ul><li>Standards are one form of performance measure; they are generally objective standards or guidelines that are used to assess performance (“standard” here means a standard way of measuring rather than something specific to achieve). </li></ul></ul></ul><ul><li>Identify appropriate indicators (how it will be measured) </li></ul><ul><ul><ul><li>Indicators are another form of performance measure; they are the data or information that is used to assess progress toward a performance standard. </li></ul></ul></ul><ul><li>Set goals and targets </li></ul><ul><ul><ul><li>The planned or expected level of performance </li></ul></ul></ul><ul><li>Communicate expectations </li></ul>
    25. 25. Is there a Performance Standards component? <ul><li>Have performance goals and targets been established, and appropriate indicators identified? </li></ul><ul><li>Do these benchmark against similar organizations or use national, state, or scientific guidelines? </li></ul><ul><li>Are expectations communicated? </li></ul>
    26. 26. Measuring Performance <ul><li>Relies on indicators that document where performance is in relation to the target established for a standard </li></ul><ul><li>Develop data systems </li></ul><ul><li>Collect data </li></ul>
    27. 27. Is there a Performance Measurement component? <ul><li>Have the indicators been refined? </li></ul><ul><li>Have data systems been developed? </li></ul><ul><li>Are data being collected? </li></ul>
    28. 28. Reporting Progress <ul><li>Analyze data </li></ul><ul><li>Convert data into useable information </li></ul><ul><li>Feedback to managers, staff, policy makers, and constituents </li></ul><ul><li>Develop a regular reporting cycle </li></ul>
    29. 29. Reporting Progress <ul><li>Provide context for the report </li></ul><ul><ul><li>How do the performance measures relate to mission and goals </li></ul></ul><ul><li>Create clear, easy to read, report designs </li></ul><ul><ul><li>Use simple charts and tables </li></ul></ul><ul><li>Determine Reporting Frequency </li></ul><ul><ul><li>When and how often </li></ul></ul>
    30. 30. Is there a Reporting of Progress component? <ul><li>Are data analyzed? </li></ul><ul><li>Are data fed back to managers, staff, constituents, etc. in useable and understandable format? </li></ul><ul><li>Is there a regular reporting cycle in place? </li></ul>
    31. 31. Improving Quality <ul><ul><ul><li>Establish a program or process to manage change and achieve quality improvement in public health policies, practice and infrastructure based on what is learned through performance measures </li></ul></ul></ul><ul><li>Use data for decisions to improve policies, programs, and outcomes </li></ul><ul><li>Manage change </li></ul><ul><li>Create a learning organization </li></ul>
    32. 32. A Few Words about Change <ul><li>Results are properties of systems: every system is perfectly designed to achieve exactly the results it gets </li></ul><ul><ul><li>Results do not occur by new goals or targets, but through systemic change </li></ul></ul><ul><ul><li>Improvement comes only with change; but change doesn’t always improve results </li></ul></ul><ul><ul><li>Smart improvement relies on understanding how systems work </li></ul></ul><ul><ul><li>Systems rely on interdependencies which are as important as the system’s elements </li></ul></ul><ul><ul><li>Change is more difficult than setting goals, measuring or holding people accountable. “I would rather (measure, complain, blame, accept good enough, fight) than change!” </li></ul></ul>
    33. 33. Is there a Quality Improvement component? <ul><li>Where is the change process? </li></ul><ul><li>Is there a process to manage changes being made? </li></ul><ul><li>Is there a learning organization? </li></ul>
    34. 34. For Successful Performance Management <ul><ul><ul><li>All four Performance Management components should be present! </li></ul></ul></ul><ul><ul><ul><li>And they should be integrated into the organization’s or system’s core operations! </li></ul></ul></ul>
    35. 35. Concepts and Components to Assess in Examples <ul><li>Context for “improvement” (is/is not) clear? </li></ul><ul><li>Goals or targets for improvement (are/are not) established? </li></ul><ul><li>Appropriate measures of performance (are/are not) collected? </li></ul><ul><li>Reports of measurements (do/do not) reach the proper parties? </li></ul><ul><li>Information from measurements (are/are not) used to make improvements? </li></ul>
    36. 36. Summary - Example 1 Strategy Improvement Log Reporting “ Splits” Measuring Previous time Standards Runner; complete race in shortest possible time Context Marathon Runner (Ex 1)
    37. 37. Summary - Example 2 Adjustments as indicated Improvement Dashboard and other instruments Reporting Tire air pressure; oil, water, battery, speed Measuring Miles per gallon, cost Standards An automobile: safe, reliable, efficient personal transportation Context Not Your Father’s Oldsmobile (Ex 2)
    38. 38. Summary - Example 3 New objectives, more $ ? Improvement Performance appraisal Reporting Work prods Measuring Set annually Standards Worker; job duties Context Supervisor (Ex 3)
    39. 39. Summary – Example 4 Revised workplan Improvement Quarterly reports Reporting Info on activities Measuring Approved objectives Standards Program; program goals Context Program Manager (Ex 4)
    40. 40. OK, Sounds Simple Enough! But How Does This Work for More Complex Examples?
    41. 41. Part IV Applications of Performance Management in Public Health Practice
    42. 42. Examples 4A, 4B, 4C Revised workplan Quarterly reports Info on activities Approved objectives Program; program goals HD Program Ex 4A Revised workplan Revised workplan Improvement Quarterly reports Quarterly reports Reporting Info on activities Info on activities Measuring Approved objectives Approved objectives Standards Program; program goals Program; program goals Context Statewide Ex 4C Community Ex 4B
    43. 43. Managing Performance (Ex 5) <ul><li>Congress appropriates $1 billion annually x5 years for state-based efforts to respond to childhood obesity epidemic </li></ul><ul><li>States must: </li></ul><ul><ul><li>Appoint State Coordinators </li></ul></ul><ul><ul><li>Establish broadly based advisory committee </li></ul></ul><ul><ul><li>Prepare timelines for state and local plans, information and surveillance systems, lab services, professional training, media strategies, etc. </li></ul></ul><ul><ul><li>Renewal funding available to continue and extend these activities </li></ul></ul>
    44. 44. Components Present In These Examples? <ul><li>Context for “improvement” (is/is not) clear? </li></ul><ul><li>Goals or targets for improvement (are/are not) established? </li></ul><ul><li>Appropriate measures of performance (are/are not) collected? </li></ul><ul><li>Reports of measurements (do/do not) reach the proper parties? </li></ul><ul><li>Information from measurements (are/are not) used to make improvements? </li></ul>
    45. 45. Summary – Example 5 ? Improvement ? Reporting ? Measuring ? Standards ? Context National Childhood Obesity (Ex 5)
    46. 46. Summary – Example 5 Mechanisms for change poorly defined Improvement Since who or what is unclear, reporting target? Reporting Data and data collection systems in place Measuring Standard approaches certainly exist Standards Who or what unclear; performance task clear Context National Childhood Obesity (Ex 5)
    47. 47. Managing Performance (Ex 6) <ul><li>National Public Health System </li></ul><ul><li>Healthy People 2000 Objective 8.14 calls for 90% of population to be served by LHD effectively carrying out IOM core functions </li></ul><ul><li>Core functions further described in Essential Public Health Services framework </li></ul><ul><li>Replaced by Infrastructure Chapter in Healthy People 2010 </li></ul>
    48. 48. Comparison of Performance Measures Aggregated by Core Function
    49. 49. Components Present In This Example? <ul><li>Context for “improvement” (is/is not) clear? </li></ul><ul><li>Goals or targets for improvement (are/are not) established? </li></ul><ul><li>Appropriate measures of performance (are/are not) collected? </li></ul><ul><li>Reports of measurements (do/do not) reach the proper parties? </li></ul><ul><li>Information from measurements (are/are not) used to make improvements? </li></ul>
    50. 50. Summary – Example 6 ? Improvement ? Reporting ? Measuring ? Standards ? Context National Public Health System (Ex 6)
    51. 51. Summary – Example 6 Not driven by performance measurement Improvement Not clear since system components not specified Reporting No consensus as to how to collect information Measuring Core function related measures not widely used Standards System components/participants unclear; “ effectively carry out core functions” = ? Context National Public Health System (Ex 6)
    52. 52. Performance Management in Public Health Practice <ul><li>Active and strategic use of performance measures to improve the public’s health </li></ul><ul><ul><li>Ex: Achieving Healthy People 2010 national health goals and objectives </li></ul></ul><ul><ul><li>Ex: Improving public health organization and system performance (core functions, essential public health services, capacity, preparedness, etc.) </li></ul></ul>
    53. 53. Public Health Performance Measures <ul><li>Quantitative measures of capacities, processes, or outcomes relevant to the assessment of public health performance </li></ul><ul><ul><li>Ex: The number of trained epidemiologists available to investigate outbreaks (capacity measure) </li></ul></ul><ul><ul><li>Ex: The percentage of notifiable diseases reports submitted within the required time lines (process measure) </li></ul></ul><ul><ul><li>Ex: The annual incidence of selected infectious diseases in the community (outcome measure) </li></ul></ul><ul><ul><li>Ex: Percentage of clients who rate health department communicable disease services as “good” or “excellent” (outcome measure) </li></ul></ul>
    54. 54. Conceptual Framework of the Public Health System as a Basis for Measuring Public Health System Performance Source : Handler A, Issel M, and Turnock B. 2001. A conceptual framework to measure performance of the public health system. Am J Public Health , 91:1235-1239. PHS Mission and Purpose Philosophy Goals &quot;Core Functions&quot; Structural Capacity ----------------- Information Resources Organizational Resources Physical Resources Human Resources Fiscal Resources Outcomes --------------- Effectiveness Efficiency Equity Processes ------------------- The 10 Essential Public Health Services M A C R O C O N T E X T P U B L I C H E A L T H S Y S T E M
    55. 55. Public Health Performance Measures: Capacity, Process, and Outcomes Capacity Process (Essential Public Health Services) Outputs Outcomes Improved organizational performance Improved program performance Key Processes Improved Outcomes & Customer Satisfaction Increased Value & Public Support Develop Policy Source : Turnock BJ. Public Health: What It Is and How It Works, 3 rd Edition . Boston MA; Jones & Bartlett, 2004 . System Inputs Workforce Information Organization & Relationships Facilities Funding Assess Assure Programs and Services consistent with mandates and community priorities
    56. 56. Public Health PM Applications <ul><li>Public Health Infrastructure Capacity </li></ul><ul><ul><li>Human resource development </li></ul></ul><ul><ul><li>Data and information systems </li></ul></ul><ul><li>Public Health Processes </li></ul><ul><ul><li>Essential Public Health Services </li></ul></ul><ul><ul><li>Management practices </li></ul></ul><ul><li>Public Health Outcomes </li></ul><ul><ul><li>Health status </li></ul></ul><ul><ul><li>Customer focus and satisfaction </li></ul></ul>
    57. 57. Lessons Learned <ul><li>State performance management practices are widespread, although often not system-wide or with processes leading to quality improvement or changes. </li></ul><ul><li>States generally report their efforts result in improved performance, with positive outcomes broadly defined. </li></ul><ul><li>No single or composite framework is used in most states, and there are insufficient data to inform choices in performance management approach. </li></ul>
    58. 58. SHAs Most Likely to Have Components of Performance Management for Health Status; Least Likely for Human Resource Development Figure 8. Areas most and least likely to have performance targets, measures or standards, reports, and processes for quality improvement (QI)/change, of SHAs that apply performance management efforts SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=25) <ul><ul><li>Source : Turning Point Survey on Performance Management Practices in States: Results of a Baseline Survey of State Health Agencies . Seattle WA; Turning Point National Program Office, 2002. </li></ul></ul>
    59. 59. Nearly All SHAs Have Some Performance Management Efforts However, only about half apply performance management efforts statewide beyond categorical programs Figure 1. Agencies or programs to which SHAs apply performance management efforts (N=47) <ul><ul><li>Source : Turning Point Survey on Performance Management Practices in States: Results of a Baseline Survey of State Health Agencies . Seattle WA; Turning Point National Program Office, 2002. </li></ul></ul>
    60. 60. Most SHAs Have Performance Measures, Targets, and Reports, While Fewer States Have Process for Quality Improvement or Change* Figure 15. Percentage of SHAs that have specified components of performance management for public health capacity (N=25) *Correlation analysis revealed that there is a comparatively weak relationship between having performance targets, performance measures or performance reports and having a process for quality improvement (QI)/change. <ul><ul><li>Source : Turning Point Survey on Performance Management Practices in States: Results of a Baseline Survey of State Health Agencies . Seattle WA; Turning Point National Program Office, 2002. </li></ul></ul>
    61. 61. Most States Use Neither Incentives nor Disincentives to Improve Performance Figure 18. Percentage of SHA performance efforts that include incentives or disincentives to improve performance (N=40) Note: Respondents could choose more than one response, so total does not equal 100. <ul><ul><li>Source : Turning Point Survey on Performance Management Practices in States: Results of a Baseline Survey of State Health Agencies . Seattle WA; Turning Point National Program Office, 2002. </li></ul></ul>
    62. 62. Performance Measure Sources <ul><ul><li>Source : Turning Point Survey on Performance Management Practices in States: Results of a Baseline Survey of State Health Agencies . Seattle WA; Turning Point National Program Office, 2002. </li></ul></ul>✔ ✔ ✔ MAPP ✔ ✔ ✔ APEX-PH ✔ ✔ NPHPS (based on EPHS and CF) ✔ EPHS (#3) ✔ Core Functions (#2) ✔ ✔ Healthy People 2010 (#1) Outcome Process Capacity
    63. 63. Managing Performance (Ex 7) <ul><li>State Health Agency </li></ul><ul><li>Mission: to protect and promote the health of the state’s population </li></ul><ul><li>Local public health agencies are units of state health agency and work under its direction </li></ul><ul><li>Health status & other info (including activity counts, costs, staffing, etc.) collected </li></ul><ul><li>Resources deployed to local units based on progress toward pre-established targets </li></ul>
    64. 64. Components Present In This Example? <ul><li>Context for “improvement” (is/is not) clear? </li></ul><ul><li>Goals or targets for improvement (are/are not) established? </li></ul><ul><li>Appropriate measures of performance (are/are not) collected? </li></ul><ul><li>Reports of measurements (do/do not) reach the proper parties? </li></ul><ul><li>Information from measurements (are/are not) used to make improvements? </li></ul>
    65. 65. Summary – Example 7 ? Improvement ? Reporting ? Measuring ? Standards ? Context State Health Agency (Ex 7)
    66. 66. Summary – Example 7 Incentives for progress; info for resource allocation Improvement Agency management Reporting Outcomes, costs, satisfaction, etc. Measuring HP 2010, Baldridge Quality Stds Standards State agency/local unites; promote/protect public health Context State Health Agency (Ex 7)
    67. 67. Managing Performance (Ex 8) <ul><li>State-Local Public Health Agency Network </li></ul><ul><li>Mission to protect and promote </li></ul><ul><li>State certifies local public health agencies </li></ul><ul><li>Core function based standards applied </li></ul><ul><li>Annual grants made to local agencies meeting standards via formula based on population and need </li></ul>
    68. 68. Components Present In This Example? <ul><li>Context for “improvement” (is/is not) clear? </li></ul><ul><li>Goals or targets for improvement (are/are not) established? </li></ul><ul><li>Appropriate measures of performance (are/are not) collected? </li></ul><ul><li>Reports of measurements (do/do not) reach the proper parties? </li></ul><ul><li>Information from measurements (are/are not) used to make improvements? </li></ul>
    69. 69. Summary – Example 8 ? Improvement ? Reporting ? Measuring ? Standards ? Context State-Local PH Network (Ex 8)
    70. 70. Summary – Example 8 Decisions/resource allocations based on info? Improvement State agency? Reporting Outcomes, LHD stds, activities? Measuring HP 2010, core function based LHD stds Standards State and LHDs (+ partners?); promote/protect public health Context State-Local PH Network (Ex 8)
    71. 71. Examples and Case Studies* <ul><li>Marathon Runner </li></ul><ul><li>Not Your Father’s Oldsmobile </li></ul><ul><li>Unit Supervisor </li></ul><ul><li>Program (and Mega-Program) Manager </li></ul><ul><li>National Health Priority Initiative </li></ul><ul><li>National Public Health System </li></ul><ul><li>State Health Agency* </li></ul><ul><li>State-Local Public Agency Network** </li></ul><ul><li>Community Health Improvement Process* </li></ul><ul><li>Public Health Workforce Development** </li></ul>* Indicates number of case studies available for various PM applications
    72. 72. Part V Case Studies
    73. 73. Part V Example 9 Case Study Moose County Community Health Improvement Process Using MAPP
    74. 74. The CHIP Process Community Health Needs Assessment Community Health Plan Program Development Implementation Evaluation
    75. 75. The MAPP Model
    76. 76. Another View of M AP P Model Community Health Status Assessment Local Public Health System Assessment Organize For Success Review Mandates, Mission, Stakeholder Expectations, and Goals Vision For Success Identify Strategic Issues Formulate Strategies Implement Strategies Evaluate Community Generated Themes Contextual Environment Assessment
    77. 77. Tobacco Use By 2002, reduce cigarette smoking to a prevalence of no more than 24 percent among people aged 18 and older. (Baseline: 1996 BRFS percentage 27.3). Addiction Sedentary Lifestyle Hypertension Advertising (Indirect) Coalition will implement CDC = s community-based tobacco control program, focusing on delaying initiation, cessation programs, and advertising control. Coalition will provide support to LHD with in-kind donations, staff, and clinical counseling space. Cerebrovascular Disease (Stroke) By 2004, reduce stroke deaths to no more than 70 per 100,000 (Baseline: 1997 crude rate 76.5 per 100,000). Limited number smokers seeking counseling. Media messages promote smoking. Peer pressure. Health Problem Outcome Objective Risk Factor Impact Objective Contributing Factors Community Health Plan Proven Intervention Strategies Resources Available Barriers
    78. 78. Description of the Health Problem, Risk Factors and Contributing Factors Corrective Actions Proposed Community Organizations Evaluation Plan The problem is the high death rate associated with Cerebrovascular disease (county CR = 76.5, state CR =61.7 p/100,000 or 23% above state rate). The major risk factor is cigarette smoking and county has higher percentage of smokers (27.6%) , especially males (31%) than state (24%, 28%, respectively). Addiction appears to be the chief contributing factor. In order to effectively address this health problem a multi-sited and multi- targeted campaign is required to delay initiation of smoking behavior, assist persons attempting to quit, and provide counter advertising to tobacco promotions. A coalition of community groups, including the LHD, Cancer Society, Lung Association, General Hospital, Teens Against Butts will collaborate with in- kind donations, staff, and clinical counseling space to assist in meeting objectives. Coalition will meet quarterly to assess progress in achieving goals. Coalition will use the evaluation tool developed by the CDC to determine effectiveness of programming efforts. Surveys will be conducted amongst coalition members to ascertain stakeholder satisfaction with program. Community Health Plan
    79. 79. Components Present In This Example? <ul><li>Context for “improvement” (is/is not) clear? </li></ul><ul><li>Goals or targets for improvement (are/are not) established? </li></ul><ul><li>Appropriate measures of performance (are/are not) collected? </li></ul><ul><li>Reports of measurements (do/do not) reach the proper parties? </li></ul><ul><li>Information from measurements (are/are not) used to make improvements? </li></ul>
    80. 80. Summary – Example 9 ? Improvement ? Reporting ? Measuring ? Standards ? Context Moose County CHIP Using MAPP
    81. 81. Summary – Example 9 Not clear, commitment of partners? Improvement Partnership, policy makers, public Reporting Outcomes and community contributing factors Measuring HP 2010, MAPP assessments (incl. NPHPS) Standards Moose County Health Partnership; Improve community health via health priorities Context Moose County CHIP Using MAPP
    82. 82. Moose County CHIP Case Study to the Next Level <ul><li>Have you been (are you now) involved in a similar effort? If so which PM components are in place? Which PM components are missing or could be enhanced? How? </li></ul><ul><li>For a companion effort at the state level, describe how the various PM components would be used. </li></ul>
    83. 83. Part V Example 10 Case Study PH Workforce Development in Grey State/Coyote County
    84. 84. 1 – Assess Competency Using Consistent Methods and Tools 2 – Enhance Specific Competencies Based on Assessment 3 – Document Competent Performance in Workplace via Human Resource Management 4 – Recognize Competent Performance via System Incentives such as Credentialing Public Health Workforce Preparedness Core Public Health Practice & BT/ER Competencies 2 3 4 1
    85. 85. Different Indicators Used to Measure Performance (Workforce Competency) Credentialing Application External Certifying Body Assessment Personnel Mgt/ Competency Demo Application Workplace Assessment Competency Acquisition Application Course Instructor Assessment Self Improvement Application ---------> ---------> ---------> ---------> Self Assessment
    86. 86. <ul><li>9 core competencies for all public health workers </li></ul><ul><li>PLUS Function-specific competencies for 8 emergency response functional roles: </li></ul><ul><ul><li>leaders </li></ul></ul><ul><ul><li>communicable disease </li></ul></ul><ul><ul><li>clinical </li></ul></ul><ul><ul><li>environmental health </li></ul></ul><ul><ul><li>public health laboratory </li></ul></ul><ul><ul><li>medical examiner </li></ul></ul><ul><ul><li>public information </li></ul></ul><ul><ul><li>other professionals </li></ul></ul><ul><ul><li>technical and support </li></ul></ul>
    87. 88. Coyote County Health Dept Training Plan <ul><li>ALL STAFF HRS </li></ul><ul><ul><li>Anthrax/BT Primer (150 Staff) – 1.5 hrs 225 </li></ul></ul><ul><ul><li>Smallpox Primer (125 Staff ) - 1-5 hrs 118 </li></ul></ul><ul><ul><li>SNS Drill (90 staff) – 8 hrs 720 </li></ul></ul><ul><ul><li>CERT Training (10 staff) - 21 hrs 210 </li></ul></ul><ul><ul><li>Phase I Training (210 staff) - 2 hrs 420 </li></ul></ul><ul><ul><li>Phase II Training (211 staff) - 7 hrs 1477 </li></ul></ul><ul><li>BT-IC TEAM (Health Department Leaders) </li></ul><ul><ul><li>Weapons of Mass Destruction (8 staff) – 12 hrs 96 </li></ul></ul><ul><ul><li>Forensic Epidemiology (4 staff) – 12 hrs 48 </li></ul></ul><ul><ul><li>Risk Communications (10 staff) – 11 hrs 110 </li></ul></ul><ul><ul><li>Community-wide ER Response Drill (6 staff) – 6 hrs 36 </li></ul></ul><ul><ul><li>Incident Command Drill (10 staff) – 3 hrs 30 </li></ul></ul><ul><li>SPECIALIZED STAFF </li></ul><ul><ul><li>Smallpox Vaccine Administration (25 RN staff)- 1 hr 25 </li></ul></ul><ul><ul><li>RN Team Captains (20 RN staff ) – 4 hrs 80 </li></ul></ul><ul><ul><li>PH Training Network Satellite - (50+ staff/15+ presentations) 1-6 hrs 150 </li></ul></ul><ul><li>TRAINING INVESTEMENT (including Phase I and II) </li></ul><ul><ul><ul><li># Staff hours 3800 + </li></ul></ul></ul><ul><ul><ul><li># Trainings – 14 </li></ul></ul></ul><ul><ul><ul><li># Estimated cost (including materials, space, prep.) >$98,000 </li></ul></ul></ul>
    88. 89. CCHD Workers Before & After
    89. 90. Components Present In This Example? <ul><li>Context for “improvement” (is/is not) clear? </li></ul><ul><li>Goals or targets for improvement (are/are not) established? </li></ul><ul><li>Appropriate measures of performance (are/are not) collected? </li></ul><ul><li>Reports of measurements (do/do not) reach the proper parties? </li></ul><ul><li>Information from measurements (are/are not) used to make improvements? </li></ul>
    90. 91. Summary – Example 10 ? Improvement ? Reporting ? Measuring ? Standards ? Context Coyote County Health Dept. PH Workforce Development
    91. 92. Summary – Example 10 Individual, group, agency training plans Improvement Individuals, agency mgt, state Reporting Multiple views beginning with self-assessment Measuring Competency expectations Standards CCHD; skilled workforce Context Coyote County Health Dept. PH Workforce Development
    92. 93. CCHD Workforce Development Case Study to the Next Level <ul><li>Have you been (are you now) involved in a similar effort? If so which PM components are in place? Which PM components are missing or could be enhanced? How? </li></ul><ul><li>For a companion effort at the state level, describe how the various PM components would be used. </li></ul>
    93. 94. Part VI Review and Summary
    94. 95. Critical components of performance management Source: From Silos to Systems: Using Performance Management to Improve the Public’s Health. Turning Point Performance Management National Excellence Collaborative: Seattle WA; Turning Point National Program, 2003.
    95. 96. Apply Standards, Set Targets Outcomes – Healthy People 2010 Objectives Process - National Public Health Performance Standards National Public Health System Target established for childhood obesity prevalence National Health Priority Previous personal best time Marathon Runner Core public health practice competencies PH Workforce Development Outcomes – Healthy People 2010 Objectives Process - National Public Health Performance Standards Community Health Improvement Process Outcomes – Healthy People 2010 Objectives Process - National Public Health Performance Standards State-Local Public Agency Network Outcomes – Healthy People 2010 Objectives, Baldridge Quality Criteria Process - National Public Health Performance Standards State Health Agency Objectives approved by granting agency Program Manager Expectations established for specific job duties Unit Supervisor Safe, reliable, efficient transportation Your Father’s Oldsmobile
    96. 97. Measure Performance Surveillance of core function or EPHS performance National Public Health System Population studies of health status National Health Priority Training times for specific distances Marathon Runner Various views of individual competency PH Workforce Development Outcomes and contributing factors in community Community Health Improvement Process Outcomes, compliance with standards, activities? State-Local Public Agency Network Outcomes, activities, costs, satisfaction, etc State Health Agency Program goals and objectives Program Manager Specific performance expectations for employee Unit Supervisor Air, oil, water, battery, speed, direction, etc. Father’s Oldsmobile
    97. 98. Report Information Annual surveillance National Public Health System Annual review National Health Priority Personal training log Marathon Runner Individuals, agency management PH Workforce Development Community Health Partnership, policy makers, public Community Health Improvement Process Unclear State-Local Public Agency Network Agency management State Health Agency Quarterly progress reports Program Manager Monthly or quarterly progress reports Unit Supervisor Dashboard and other instruments Father’s Oldsmobile
    98. 99. Quality Improvement Incentives; redeploy resources National Public Health System Secure new commitments; redeploy resources; new policies National Health Priority Revise race strategy Marathon Runner Tailored training plans for individuals and agencies PH Workforce Development Secure new commitments; redeploy resources Community Health Improvement Process Incentives; resource allocation? State-Local Public Agency Network Management decisions; incentives; resource allocation State Health Agency Revise program objectives and workplan Program Manager Set new performance expectations Unit Supervisor Adjust tire air pressure, oil, water, speed, direction Father’s Oldsmobile
    99. 100. <ul><li>In the performance management cycle... </li></ul><ul><li>All components should be driven by the public health mission and organizational strategy </li></ul><ul><li>Activities should be integrated into routine public health practices </li></ul><ul><li>The goal is continuous performance and quality improvement </li></ul>Source: From Silos to Systems: Using Performance Management to Improve the Public’s Health. Turning Point Performance Management National Excellence Collaborative: Seattle WA; Turning Point National Program, 2003.
    100. 101. For Successful Performance Management <ul><ul><ul><li>All four Performance Management components should be continuously integrated into the core operations of the agency or system </li></ul></ul></ul>
    101. 102. Capacity & Accountability National Public Health System National Health Priority Commitment Leadership Informed Decision Making Incentives Accountability Marathon Runner PH Workforce Development Community Health Improvement Process State-Local Public Agency Network State Health Agency Program Manager Unit Supervisor Father’s Oldsmobile
    102. 103. Public Health Agency as a Learning Organization: Core Organizational Competencies Support Strategic Planning / Change Processes Forces / Trends Stakeholders Health System Planning to Plan Internal Environment Capacity Competencies Barriers External Environment Strategic Issues Strategies Organizational Systems Design & Development Actions Results MANDATES Vision Mission Values < Strategy Formation > < Implementation > Adapted from John M. Bryson (c) 1985. In Bryson, J.M. & Roering, W.D. (1988). Initiation of strategic planning by governments. Public Administration Review, Nov.- Dec. , 995 -1004. Opportunities / Threats Strengths / Weaknesses Core Functions & Essential Public Health Services
    103. 104. Success Factors <ul><li>Integrate PM into routine public health processes </li></ul><ul><li>Sustainable PM activities meet state and local needs and political realities </li></ul><ul><li>NPHPS, management models and tools provide a head start </li></ul><ul><li>Early stakeholder involvement increases support and chances of success </li></ul>
    104. 105. Success Factors (cont’d) <ul><li>Align PM measures, activities, and spending with public health priorities </li></ul><ul><li>Trained staff, dedicated resources, and PM culture are essential </li></ul><ul><li>Baseline information and trends important </li></ul><ul><li>New or adapted information and management systems are necessary for cross-program management </li></ul><ul><li>Incentives motivate performance and quality improvement </li></ul>
    105. 106. Performance Measures Are Useful to … <ul><li>Identify aspects of the work that have and have not resulted in satisfactory results </li></ul><ul><li>Identify trends </li></ul><ul><li>Further investigate the nature of particular problems </li></ul><ul><li>Set targets for future periods </li></ul><ul><li>Motivate managers and staff to improve performance </li></ul><ul><li>Hold managers and staff accountable </li></ul><ul><li>Develop and improve programs and policies </li></ul><ul><li>Help design policies and budgets and explain these to stakeholders </li></ul>Source : lichiello P. Guidebook for Performance Measurement . Seattle WA: Turning Point National Program Office, 1999:48. Based on Hatry HP et al, Monitoring the Outcomes of Economic Development Programs. Washington DC: The Urban Institute Press, 1990.
    106. 107. Why Use Performance Management? <ul><li>To improve public health practice and maximize its effectiveness. This requires </li></ul><ul><ul><li>More than setting goals/targets alone; more than measurement alone. These are necessary, but not sufficient, components </li></ul></ul><ul><ul><li>All four PM components should be continuously integrated into the core operations of the agency/system </li></ul></ul>
    107. 108. <ul><li>Quality improvement efforts </li></ul><ul><li>Policy change </li></ul><ul><li>Resource allocation change </li></ul><ul><li>Program change </li></ul>Managerial Action Why Use Performance Management?
    108. 109. Performance Mgt Resources <ul><li>Turning Point PM National Excellence Collaborative (online via www.turningpointprogram.org/Pages/perfmgt.html ) </li></ul><ul><li>Guidebook for Performance Measurement . Seattle WA: Turning Point National Program Office, 1999. </li></ul><ul><li>Performance Management in Public Health: A Literature Review. Seattle WA; Turning Point National Program Office, 2002. </li></ul><ul><li>Turning Point Survey on Performance Management Practices in States: Results of a Baseline Survey of State Health Agencies . Seattle WA; Turning Point National Program Office, 2002. </li></ul><ul><li>From Silos to Systems: Using Performance Management to Improve the Public’s Health. Seattle WA; Turning Point National Program Office, 2003. </li></ul><ul><li>Performance Management Self-Assessment Tool . Washington DC: Public Health Foundation, 2004. </li></ul><ul><li>Performance Management Toolkit . Public Health Foundation (online via www.phf.org ) </li></ul><ul><li>Performance Measurement and Improvement. Chapter 18 in Public Health Administration: Principles for Population-Based Management . Sudbury MA; Jones & Bartlett, 2000. </li></ul>
    109. 110. Part VII Post-Test & Course Evaluation

    ×