3.
Introduction Healthcare reform has strengthened the link between performance and reimbursement, exemplified by value-based purchasing and accountable care organizations. This has led to heightened interest in improving physician performance. However, providing actionable assessments of physician performance is not as straightforward as it may seem. Many organizations, for example, launch physician performance improvement initiatives without addressing relevant aspects of performance, such as the following: §§ What proportion of total performance variability is attributable to physicians? §§ Are there statistically significant differences in physician performance? §§ How is physician performance distributed across the outcome categories of “better than expected,” “as expected,” and “worse than expected”? Consequently, it is not uncommon for a physician improvement initiative to be launched without establishing relevant, quantifiable objectives. Furthermore, and more importantly, the strategy to realize physician improvement may not be clearly defined. Subsequently, after many meetings and hours of expended work, the initiative may be abandoned due to lack of direction and progress. Fortunately, however, there is a more thoughtful approach that enhances the success of physician improvement initiatives. Physician Performance Improvement: An Analytical Approach — Part one 1
4.
Harnessing the data at your disposal and conducting analytics to answer the questions posed in the aforementioned bullet points will provide the knowledge required to successfully engage physicians and improve organizational performance. In this white paper, we address how to approach answering these questions. In a subsequent white paper titled, “Physician Performance Improvement: Case Studies,” we will provide case studies applying these principles and using the information to formulate a performance improvement strategy that engages physicians. Physician Variability All healthcare outcomes (length of stay, cost, mortality, etc.) have two components that contribute to the overall variability in the achieved performance: §§ Organizational factors, such as policies, procedures, staffing, etc. §§ Physician practice patterns Quantifying what percentage of the total variability that physician practice patterns comprise is strategically valuable information. With this knowledge, a performance improvement strategy can be formulated. Figure 1 depicts the percentage of total variability in risk-adjusted excess length of stay (defined as observed length of stay minus expected length of stay) that is attributable to physician practice patterns for clinical conditions that have an opportunity for improvement. Figure 1: Physician Variability Contribution Back and Neck Procedures 93.7 Circulatory Disorders 77.8 Appendectomy 64.7 Infectious Disease 62 Sepsis 41.2 Red Blood Cell Disorders 38.6 Respiratory Failure 25.3 Metabolic Disorders 20.0 Seizures 17.0 Gynecology Procedures 15.7 Cellulitis 8.6 COPD 6.2 Pneumonia 5.4 Newborn 3.8 Rehabilitation 3.1 Asthma 0.0 Cardiac Arrhythmia 0.0 Physician Variability Percentage2 Physician Performance Improvement: An Analytical Approach — Part ONe
5.
Engaging physicians to explore and standardize practice patterns to reducevariability in risk-adjusted excess length of stay for “Back and Neck Procedures” isthe strategy of choice for this clinical condition, since physician practice patternsaccount for 93.7 percent of the total variability. On the other hand, if “Pneumonia”is the clinical condition selected for improvement, focusing the improvement effortson organizational factors is the strategy that will yield the greatest benefit. That isbecause organizational factors represent 94.5 percent of the variability and physicianpractice patterns represent 5.4 percent.One can readily see the value this information provides for successful performanceimprovement and physician engagement. Without this information, one may pursueexploring physician practice patterns for pneumonia with the hopes of performanceimprovement only to be disappointed with the results and potentially incurringresentment and disengagement among physicians in the process.Note: The physician variability percentage is the intraclass correlation coefficientderived using hierarchical regression techniques. By assessing the degree to whichmeasured outcomes are correlated within physicians, the intraclass correlationprovides an estimate of the degree to which outcome variation can potentially beexplained by variation in physician practice patterns. Several currently availablestatistical software packages provide this capability. See references in the footnotesfor further reading on the subject.1, 2Physician PerformanceDetermining if statistically significant differences in physician performance existprovides another piece of information that assists in deriving the improvementstrategy. If the goal is to improve risk-adjusted excess length of stay, the questionto answer is: Is risk-adjusted excess length of stay significantly different amongphysicians? If the answer is yes, then reducing variability for risk-adjusted excesslength of stay among physicians will likely yield meaningful improvements inhospital performance. On the other hand, if the answer is no, then variability inphysician performance does not exist and any attempt to reduce this variability willlikely not yield meaningful results.To illustrate, let us take two examples of physician performance variability anduse the common p-value of ≤ 0.05 to determine whether statistically significantdifferences exist in physician performance.Figure 2 below depicts the median risk-adjusted excess length of stay for attendingphysicians treating patients undergoing vascular procedures. The graph depictsperformance that ranges from a median of -0.7 to 1.9 days. The one-way test ofsignificance for these data yields a p-value of 0.0001. Since this value is less than0.05, the answer to the above question — Is there a statistically significant differencein risk-adjusted excess length of stay among physicians — is yes. Therefore,reducing the variability in physician performance will likely produce meaningfulimprovement. Physician Performance Improvement: An Analytical Approach — Part one 3
6.
Figure 2: Vascular Procedures: Attending Physician Performance by Risk-Adjusted Excess Days 2 -0.7 39 5 -0.4 36 3 -0.4 54 12 -0.3 24 13 -0.1 31 14 -0.1 58 Physician ID 6 0.1 33 10 0.4 27 8 0.6 33 4 0.8 27 9 0.8 36 7 0.9 36 11 1.1 33 1 1.9 33 Median Excess Days Case Count P<0.05 The scenario depicted in Figure 3 for renal failure depicts variation in median performance that ranges from a -0.4 to 1.4 days, and the one-way test of significance for these data yields a p-value of 0.4873. Given that this probability is considerably greater than 0.05, the answer to the above question: — Is risk-adjusted excess length of stay significantly different among physicians — is no. Simply put, there is insufficient evidence in the data to conclude that there is true variability in physician performance. Given this, one could not expect that attempts to reduce physician performance variability would yield meaningful improvement results. Figure 3: Renal Failure: Attending Physician Performance by Risk-Adjusted Excess Days 4 -0.4 45 3 -0.4 17 -0.0 19 Physician ID 5 1 0.6 50 7 0.5 21 6 1.3 19 2 1.4 22 Median Excess Days Case Count P<0.05 Physician Performance Classification The last piece of information in evaluating physician performance is the distribution of performance across the performance categories of “better than expected,” “as expected,” and “worse than expected.” The “expected” component of this measurement is the predicted risk-adjusted outcome based on the severity of illness among the physician’s patient population. In our aforementioned examples, we derived risk-adjusted excess length of stay by subtracting a patient’s “expected” length of stay from their observed length of stay. Negative differences represent a shorter length of stay than expected, zero differences represent a length of stay that is equal to expected, and positive differences signify a length of stay that is longer than expected. By statistically summarizing these differences across a group of patients attributed to a specific physician, one can determine whether or not systematic departures from risk-adjusted expected length of stay are present.4 Physician Performance Improvement: An Analytical Approach — Part ONe
7.
Classifying physician performance involves deriving risk-adjusted confidenceintervals and comparing each physician’s confidence interval to an appropriatereference. Before we delve into examining performance using confidence intervals,let’s first understand what a confidence interval is. A confidence interval is a range ofvalues that are likely to occur given the variability present in the data. Performancemeasurements are not static; they vary from one time period to the next. Therefore,to accurately measure physician performance, this variability must be accounted forin the measurement system. When deriving confidence intervals, one can select thelevel of precision desired. Typical levels are 99 percent, 95 percent, and 90 percent,with the most common being 95 percent.3Let’s use an example to demonstrate how to interpret confidence intervals and derivean improvement strategy. Figure 4 depicts 95-percent confidence intervals of themedian risk-adjusted excess length of stay. The left-most bar of the interval is thelower confidence limit (LCL), the right-most bar is the upper confidence limit (UCL),and the median is depicted by the dot. The appropriate reference here is the linelocated at the zero value, which represents performance that is as expected.The physicians at the bottom of the graph, highlighted in orange, are performingbetter than expected, since their confidence intervals do not intersect the referenceline, and the UCL lies to the left of the reference line. The physicians depictedin green are performing as expected, since their confidence intervals intersectthe reference line. And the physicians depicted in red are performing worse thanexpected, since their confidence intervals do not intersect the reference line and theLCL lies to the right of the reference line.In this scenario, a viable improvement strategy is to examine the practice patterns ofthe physicians with better than expected performance and disseminate the findingsto the other physicians. A standardized clinical protocol could also be developedbased on the findings; and garnering the cooperation of the other physicians inutilizing the protocol will likely yield meaningful improvement. Figure 4: Diabetes: Attending Physician Performance by Risk-Adjusted Length of Stay (LOS) ComparisonAttending Physician Better Than Expected As Expected Worse Than Expected -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 Risk-Adjusted Median Excess LOS Confidence Interval Physician Performance Improvement: An Analytical Approach — Part one 5
8.
Figure 5 depicts a scenario where all physicians are performing as expected. Hence, there are no physicians who can be used as role models for performance improvement purposes. The improvement strategy in this case consists of researching and implementing best practices and practice guidelines. Figure 5: Renal Failure: Attending Physician Performance Risk-Adjusted Length of Stay Comparison Attending Physician Better Than Expected As Expected Worse Than Expected -3 -2 -1 0 1 2 3 4 5 6 7 8 9 Risk-Adjusted Median Excess LOS Confidence Interval Summary Deploying an effective strategy that will engage physicians in performance improvement requires a comprehensive understanding of physician performance. There are three questions that facilitate a comprehensive understanding of physician performance: §§ What proportion of total performance variability is attributable to physicians? §§ Are there statistically significant differences in physician performance? §§ How is physician performance distributed across the outcome categories of “better than expected”, “as expected” and “worse than expected”? Once these questions are answered an appropriate strategy to engage physicians in performance improvement can be derived that will likely yield meaningful performance improvement. Deploying a physician performance improvement strategy by relying on only one measurement of performance is less likely to result in meaningful performance improvement. With this approach, one is hoping they have selected an appropriate performance improvement strategy. In part two of this white paper, we will apply these concepts to three commonly encountered scenarios. In practice, other scenarios will be encountered, however, it is our hope that these scenarios will provide some general guidance on deploying an effective physician performance improvement strategy. 1. Harman, JS, et al; “Profiling Hospitals for Length of Stay for Treatment of Psychiatric Disorders.” The Journal of Behavioral Health Services & Research. 2004;31(1):66-74. 2. Snijders TAB, Bosker RJ. “Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling.” Sage Publications Inc: 2012. 3. Martin JG, Altman DG: “Statistics With Confidence: Confidence Intervals and Statistical Guidelines.” British Medical Journal: 1989.6 Physician Performance Improvement: An Analytical Approach — Part ONe
9.
Physician Performance Improvement: An Analytical Approach — Part one 7
Be the first to comment