INFORMS Rocky Mtn Presentation 03-17-11

2,175 views
2,096 views

Published on

Quantitative Approaches to
Improve Healthcare Access and Quality, Rocky Mountain INFORMS Chapter Meeting, A panel presentation, featuring the work of:
Linda LaGanga, Ph.D.,Steve Lawrence, Ph.D.,
C.J. McKinney, Ph.D. Candidate1,
Antonio Olmos, Ph.D., Michele Samorani, Ph.D. Candidate
(Mental Health Center of Denver,
University of Colorado-Boulder,
University of Colorado-Denver,
University of Northern Colorado)Thursday, March 17, 2011

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,175
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

INFORMS Rocky Mtn Presentation 03-17-11

  1. 1. Quantitative Approaches to Improve Healthcare Access and Quality Rocky Mountain INFORMS Chapter Meeting A panel presentation, featuring the work of: Linda LaGanga, Ph.D.1,3 Steve Lawrence, Ph.D.2 C.J. McKinney, Ph.D. Candidate1,4 Antonio Olmos, Ph.D.1 Michele Samorani, Ph.D. Candidate2 1. Mental Health Center of Denver 2. University of Colorado-Boulder 3. University of Colorado-Denver 4. University of Northern Colorado 1Rocky Mountain INFORMS, March 17, 2011
  2. 2. Healthcare Issues we address  To overbook or not?  If we schedule them, will they come?  What would Deming do to improve healthcare?  To achieve efficiency and effectiveness of healthcare 2Rocky Mountain INFORMS, March 17, 2011
  3. 3. Where is our work developed and documented?  Experience and data from the Mental Health Center of Denver  Community mental health center serving over 14,000 people per year  Surveys and interviews of other healthcare providers/systems  Presented at INFORMS annual conferences  Other conferences:  Production & Operations Management Society  Decision Sciences Institute  Mayo Clinic Conference on OR/Systems Engineering in Healthcare  American Evaluation Association 3Rocky Mountain INFORMS, March 17, 2011
  4. 4. Read more about it…  Decision Science Journal (May, 2007)  Journal of Operations Management (2010, in press)  Conference presentations and proceedings at http://www.outcomesmhcd.com/Pubs.htm  Research posters on the wall opposite this room 4Rocky Mountain INFORMS, March 17, 2011
  5. 5. Appointment Scheduling and Overbooking  Clinic Overbooking to Improve Patient Access and Increase Provider Productivity  LaGanga, L. R., & Lawrence, S. R. (2007). Clinic overbooking to improve patient access and provider productivity. Decision Sciences, 38(2), 251 – 276. 5Rocky Mountain INFORMS, March 17, 2011
  6. 6. Simple Overbooking Example 6Rocky Mountain INFORMS, March 17, 2011
  7. 7. Model Assumptions  Number of patients booked, K:  E(K) = SK = N  S = Show rate, N = target n of patients  K = N/S  Patients scheduled at even intervals throughout the day  T = N/K = S  Inter-appointment times compressed by the show rate  Patients arrive on time with probability S  Patient service times deterministic  Added variability in final version 7Rocky Mountain INFORMS, March 17, 2011
  8. 8. Overbooking: Best Case 10 appointment slots / session; 50% show rate Regular Time Overtime Time Slot 1 2 3 4 5 6 7 8 9 10 11 12 13 Start Time 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 Best Case Expected number of patients (5) arrive, evenly spaced Arrivals A1 X2 A3 X4 A5 X6 A7 X8 A9 X10 Service D1 D3 D5 D7 D9 No overtime Waiting No patients wait 5 patients seen; no provider idle time; no patients wait; no clinic overtime 8Rocky Mountain INFORMS, March 17, 2011
  9. 9. Overbooking: Bunched Early 10 appointment slots / session; 50% show rate Regular Time Overtime Time Slot 1 2 3 4 5 6 7 8 9 10 11 12 13 Start Time 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 Case 1 Expected number of patients (5) arrive, bunched early Arrivals A1 A2 A3 X4 A5 X6 A7 X8 X9 X10 Service D1 D2 D3 D5 D7 No overtime Waiting W2 W3 W5 W7 5 patients seen; no provider idle time; 4 patients wait; no clinic overtime 9Rocky Mountain INFORMS, March 17, 2011
  10. 10. Overbooking: Late Arrival 10 appointment slots / session; 50% show rate Regular Time Overtime Time Slot 1 2 3 4 5 6 7 8 9 10 11 12 13 Start Time 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 Case 2 Expected number of patients (5) arrive, one late arrival Arrivals A1 X2 A3 X4 A5 X6 A7 X8 X9 A10 Service D1 D3 D5 D7 I D10 OT Waiting No patients wait5 patients seen; 10% provider idle time; no patients wait; 10% clinic overtime 10Rocky Mountain INFORMS, March 17, 2011
  11. 11. Overbooking: Bunched Late 10 appointment slots / session; 50% show rate Regular Time Overtime Time Slot 1 2 3 4 5 6 7 8 9 10 11 12 13 Start Time 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 Case 3 Expected number of patients (5) arrive, bunched late Arrivals A1 X2 A3 X4 X5 X6 A7 A8 A9 X10 Service D1 D3 I I D7 D8 D9 OT Waiting W8 W95 patients seen; 20% provider idle time; 2 patients waiting; 20% clinic overtime 11Rocky Mountain INFORMS, March 17, 2011
  12. 12. Overbooking: Extra Arrival 10 appointment slots / session; 50% show rate Regular Time Overtime Time Slot 1 2 3 4 5 6 7 8 9 10 11 12 13 Start Time 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 Case 4 More patients arrive (6) than expected (5) Arrivals A1 A2 A3 X4 A5 X6 A7 X8 A9 X10 Service D1 D2 D3 D5 D7 D9 OT Waiting W2 W3 W5 W7 W9 6 patients seen; no provider idle time; 5 patients waiting; 20% clinic overtime 12Rocky Mountain INFORMS, March 17, 2011
  13. 13. Overbooking Utility Model 13Rocky Mountain INFORMS, March 17, 2011
  14. 14. Overbooking Utility Model  Maximize clinic “utility”  Trade-off  Patient access (number of patients seen)  Average patient waiting times  Expected clinic overtime  Note that provider productivity is implicit in this model 14Rocky Mountain INFORMS, March 17, 2011
  15. 15. Relative Benefits and Penalties   = Benefit of seeing additional patient   = Penalty for patient waiting   = Penalty for clinic overtime  The values of , , and  don’t matter  Just their ratios or relative importance 15Rocky Mountain INFORMS, March 17, 2011
  16. 16. Utility Function Expected utility without overbooking U   SN Expected utility with overbooking U O   A  W   O Expected net utility with overbooking U N  U O  U   ( A  SN )  W   O 16Rocky Mountain INFORMS, March 17, 2011
  17. 17. Utility Function Described U N   ( A  SN )  W   O Utility Benefit of Less Utility Benefit Patient Less Less Clinic Patients that w/o Overbooking Penalty Waiting Overtime Penalty “Show” Net Utility Benefit from Overbooking (could be negative) 17Rocky Mountain INFORMS, March 17, 2011
  18. 18. Simulation Experiments  Five clinic size levels N  N = {10, 20, 30, 40, 50}  Ten show rates S  S = {100%, 90%, … , 10%}  Full factorial experiment  SN = 5 × 100 = 500 factor levels  10,000 replications per factor  500,000 observations 18Rocky Mountain INFORMS, March 17, 2011
  19. 19. Regression Analysis  Results from simulation analyzed using regression analysis  Regression equations obtained  Expected patient wait times  Expected clinic overtime  Expected provider productivity  All coefficients significant  R2 = 98%+ 19Rocky Mountain INFORMS, March 17, 2011
  20. 20. Sensitivity to Service Uncertainty 40 N50R90 30 N30R90 Average Net Utility N50R50 20 N30R50 N10R90 10 N10R50 N10R10 0 N30R10 0.0 0.2 0.4 0.6 0.8 1.0 N50R10 -10 Service Time Variability Average of net utility UN with overbooking as a function of service time variability cs , with and (=1,  =0.5, τ =1.2) 20Rocky Mountain INFORMS, March 17, 2011
  21. 21. Conclusions  Overbooking is one solution for appointment no-shows  Can significantly improve performance  Patient access (more patients seen)  Clinic utility  But with a cost  Increased patient waiting & clinic overtime  Good for some clinics, not for others 21Rocky Mountain INFORMS, March 17, 2011
  22. 22. Directions for Future Work  Scheduling policies  Double booking  Wave scheduling  Optimal overbooking policies  Current overbooking policy is not “optimal”  Dynamic programming  Nonlinear waiting & overtime functions  Long waits much worse than short waits 22Rocky Mountain INFORMS, March 17, 2011
  23. 23. Lean Options for Walk-In, Open Access, and Traditional Appointment Scheduling in Outpatient Health Care Clinics Mayo Clinic Conference on Systems Engineering & Operations Research in Health Care Rochester, Minnesota – August 17, 2009 Linda R. LaGanga, Ph.D. Stephen R. Lawrence, Ph.D. Director of Quality Systems Leeds School of Business Mental Health Center of Denver University of Colorado Denver, CO USA Boulder, CO USAAdditional information available at: http://Leeds.colorado.edu/ApptSched 23 Rocky Mountain INFORMS, March 17, 2011 © 2008 – Linda LaGanga and Stephen Lawrence
  24. 24. Data Mining in Appointment Scheduling Michele Samorani PhD Candidate Leeds School of Business, University of Colorado at Boulder 24Rocky Mountain INFORMS, March 17, 2011
  25. 25. Finding Patterns with Data Mining 25Rocky Mountain INFORMS, March 17, 2011
  26. 26. DECISION TREEYoung clients are more likely to keep appointments with no reminder call 26Rocky Mountain INFORMS, March 17, 2011
  27. 27. CLUSTERINGIf clients are under the age of 26.3 years old and have low average CRM (<.5), then they are more likely to keep their appointments 27 Rocky Mountain INFORMS, March 17, 2011
  28. 28. Using Data Mining to Schedule Appointments 28Rocky Mountain INFORMS, March 17, 2011
  29. 29. Overbooking – Shortcomings Suppose service time = 30 minutes 1 1 0 1 0 1 0 Little waiting time 1 1 and no overtime 11:20 11:40 12:00 11:00 10:00 10:20 10:40 9:409:00 9:20 0 0 1 1 0 1 1 1 Some waiting time 1 11:20 and a high overtime 11:40 12:00 11:00 10:00 10:20 10:40 9:409:00 9:20 If we could predict which patients show up and which don’t, we could obtain  a more controllable schedule 29 Rocky Mountain INFORMS, March 17, 2011
  30. 30. The method Every time a visit request arrives: 1)A classifier is used to predict if it shows or not (for each day) 2)The visit request is scheduled by solving a stochastic program through column generation Non‐controllable parameters Controllable parameters •Service time •Number of slots K •Revenue from seeing a patient •Scheduling horizon h •Clinic overtime cost •Classification  •Waiting time cost performance: – Sensitivity (sn) – Specificity (sp)How good we are at retrieving showing patients How good we are at retrieving non‐showing patients 30 Rocky Mountain INFORMS, March 17, 2011
  31. 31. Productivity vs Punctuality  Productivity: number of patients seen. It is increased by:  Punctuality: 1/(overtime + waiting time). It is increased by: 31Rocky Mountain INFORMS, March 17, 2011
  32. 32. Real world case: MHCDShow rate Same day 1 day 2 days 3 days 4 days RLow .74 .64 .65 .62 .61 .65MHCD .87 .74 .75 .72 .71 .76 • Goal: Find the best policy for MHCD in terms of: – Overbooking – Open Access – Data Mining  After playing for a few hours with the MHCD data set, I can achieve any of the following classification performances:  sn = 0.9, sp = 0.5  sn = 0.7, sp = 0.7  sn = 0.6, sp = 0.8 32Rocky Mountain INFORMS, March 17, 2011
  33. 33. Data Mining Open Access . Policy DM OB OA ∗ ∗ (min) (min) Overbooking 1 No No No 6.39 0.00 0.00 5.99 8 4 2 No No Yes 6.39 0.00 0.00 5.99 8 1 3 No Yes No 7.10 36.22 20.61 8.37 12 4 4 No Yes Yes 7.22 35.33 21.37 8.40 12 1 .6, .8 No No 6.82 0.00 0.00 6.44 8 5 5 .7, .7 No No 6.99 0.00 0.00 6.62 8 4 .9, .5 No No 7.36 0.00 0.00 7.00 8 5 .6, .8 No Yes 6.84 0.00 0.00 6.44 8 1 6 .7, .7 No Yes 6.83 0.00 0.00 6.43 8 1 .9, .5 No Yes 6.66 0.00 0.00 6.27 8 1 .6, .8 Yes No 7.24 21.11 14.96 7.78 12 3 7 .7, .7 Yes No 7.42 29.33 17.88 8.33 12 5 .9, .5 Yes No 7.58 40.78 23.56 9.03 12 2 .6, .8 Yes Yes 7.35 25.00 15.92 8.03 12 1 8 .7, .7 Yes Yes 7.44 28.44 18.51 8.28 12 1 .9, .5 Yes Yes 7.32 35.22 19.83 8.47 12 1 33Rocky Mountain INFORMS, March 17, 2011
  34. 34. . Policy DM OB OA ∗ ∗ (min) (min) 1 No No No 7.28 0.00 0.00 6.88 8 4 2 No No Yes 7.27 0.00 0.00 6.87 8 1 3 No Yes No 7.47 29.07 15.32 8.39 10 5 4 No Yes Yes 7.52 28.00 15.62 8.39 10 1 .6, .8 No No 7.49 0.00 0.00 7.11 8 5 5 .7, .7 No No 7.56 0.00 0.00 7.18 8 2 .9, .5 No No 7.85 0.00 0.00 7.47 8 2 .6, .8 No Yes 7.56 0.00 0.00 7.17 8 1 6 .7, .7 No Yes 7.59 0.00 0.00 7.19 8 1 .9, .5 No Yes 7.52 0.00 0.00 7.12 8 1 .6, .8 Yes No 7.60 20.73 13.26 8.14 10 2 7 .7, .7 Yes No 7.65 12.11 8.69 7.83 9 5 .9, .5 Yes No 7.86 15.22 9.81 8.18 9 2 .6, .8 Yes Yes 7.62 21.87 13.83 8.20 10 1 8 .7, .7 Yes Yes 7.64 24.87 14.53 8.36 10 1 .9, .5 Yes Yes 7.57 28.13 15.82 8.44 10 1 34Rocky Mountain INFORMS, March 17, 2011
  35. 35. Conclusions  Data mining can improve appointment scheduling in the presence of no-shows  If adopted in conjunction with overbooking, data mining can either increase punctuality or productivity, depending on sensitivity and specificity  In case of low show rate, the advantage obtained by using overbooking is similar to the one obtained with data mining.  On the other hand, in case of high show rate, data mining is a superior technique  Interestingly, if we can achieve a decent classification performance, using open access is the worst choice  Thank you for your attention. Questions? 35Rocky Mountain INFORMS, March 17, 2011
  36. 36. What about the scheduling horizon h?  h does not have any significant impact by itself:  But its interaction with sn and sp is significant: 36Rocky Mountain INFORMS, March 17, 2011
  37. 37. High sensitivity classifier Classifier 37Rocky Mountain INFORMS, March 17, 2011
  38. 38. Driving Clinical Quality Improvement through Mental Health Recovery Control Charts INFORMS Annual Meeting 2009 San Diego, CA October, 11th, 2009 CJ McKinney, MA* Antonio Olmos, PhD Linda Laganga, PhD Mental Health Center of Denver Denver, CO, USA * - Corresponding Author 38Rocky Mountain INFORMS, March 17, 2011
  39. 39. Literature  Olmos-Gallo, P.A. DeRoche, K.K. (2010, August). Monitoring Outcomes in Mental Health Recovery: The Effect on Programs and Policies. Advances in Mental Health (9)1, 8-16. http://amh.e- contentmanagement.com/archives/vol/9/issue/1/ contact P. Antonio Olmos for a copy of the publication  McKinney, C.J., Olmos-Gallo, P.A. McLean, C., LaGanga, L.R. (August 2010). Driving Clinical Quality Improvement through Mental Health Recovery Control Charts. Presented at the 3rd Annual Mayo Clinic Conference on Systems Engineering & Operations Research in Health Care, Rochester, MN. Awarded First Place for Best Poster Presentation.  Clark, C.R., Olmos-Gallo, P.A. (2007). Performance Measurement: A signature approach to outcomes measurement improves recovery. National Council Magazine, 3, 26-28.  Glover, H. (2005). Recovery based service delivery: Are we ready to transform the words into a paradigm shift? Australian e-Journal for the Advancement of Mental Health, 4(3), www.auseinet.com/journal/vol4iss3/glovereditorial.pdf (accessed 15 May 2009)  Montgomery, D. C. (2005) Introduction to Statistical Quality Control, Fifth Edition. Hoboken, NJ: John Wiley and Sons, Inc.  Olmos-Gallo, P. A., DeRoche, K. K., McKinney, C. J., Starks, R., & Huff, S. (2009). The Recovery Markers Inventory: Validation of an instrument to measure factors associated with recovery from mental illness. Working paper 39Rocky Mountain INFORMS, March 17, 2011
  40. 40. The Heart of Recovery Measurement 40Rocky Mountain INFORMS, March 17, 2011
  41. 41. Act Plan Continuous Improvement Check Do 41Rocky Mountain INFORMS, March 17, 2011
  42. 42. Quality Components in Mental Health Services Quality Components Relationship to MH Services How well are MH services working? Are Performance consumers improving in their recovery? How often do we see improvements in recovery? Reliability How consistent are the outcomes across consumers? How long does the consumer retain the Durability recovery-supportive skills and tools taught through MH services? How does the consumer perceive our ability to Perceived Quality support MH recovery? Community? Conformance Are we meeting program fidelity standards? to Standards 42Rocky Mountain INFORMS, March 17, 2011
  43. 43. Quality Control in Mental Health  Allocate and reallocate clinical resources more efficiently  Improve and maintain clinical program fidelity  Reduce length of treatment, while sustaining same level of recovery and recovery supportive factors  Increase the number of consumers served, while decreasing burden on case managers/therapists  Identify most effective programs based upon consumer needs 43Rocky Mountain INFORMS, March 17, 2011
  44. 44. Mental Health Recovery  Concept of Recovery has taken root around the world  Working Definition (MHCD): “A non-linear process of growth by which people move from lower to higher levels of fulfillment in the areas of hope, safety, level of symptom interference, social networks, and activity.”  Federal Grant (SAMHSA) for Transformation to Recovery-Oriented Mental Health Systems  For information on the Recovery Transformation Summit, see http://www.gmhcn.org/files/RRecovery_Newsletter_Fall2010.pdf 44Rocky Mountain INFORMS, March 17, 2011
  45. 45. Mental Health Recovery Outcomes  MHCD has developed 3 consumer specific recovery outcomes  Consumer Recovery Measure – (Consumer Perspective) Hope, Safety, Activity, Level of Symptom Management, Social Networks  Recovery Marker Inventory – (Clinician Perspective) Housing, Employment, Education, Active Growth, Participation, and Symptom Management  Recovery Needs Level – (Clinical Algorithm) Provides for one of 5 levels of treatment based upon clinical criteria  The examples in this presentation will utilize the Consumer Recovery Measure. 45Rocky Mountain INFORMS, March 17, 2011
  46. 46. 46Rocky Mountain INFORMS, March 17, 2011
  47. 47. 47Rocky Mountain INFORMS, March 17, 2011
  48. 48. Relationship among Recovery Outcomes (1) Recovery Marker Inventory (RMI) (Longitudinal data to support clinical decision making) To what degree is RECOVERY (4) Recovery happening for Needs Level consumers at MHCD (RNL) (Formative and summative (Appropriate level of services) evaluation of recovery) (2) Promoting Recovery (3) Consumer Recovery in Organizations (PRO) Measure (CRM) (Consumer’s perceptions of how well (Consumer’s perception of their specific programs and staff are own recovery) promoting recovery) 48Rocky Mountain INFORMS, March 17, 2011
  49. 49. Consumer Recovery Measure v3.0  The CRM V3.0 includes the 15 items listed below: 2. Lately I feel like I’ve been making important contributions (active-growth) 4. I have hope for the future (hope) 5. I am reaching my goals (active growth) 7. I have this feeling things are going to be just fine (hope) 8. Recently my life has felt meaningful (hope) 9. Recently, I have been motivated to try new things (active-growth) 11. There are some people who cause me a lot of fear (safety) 12. I get a lot of support during the hard times (social network) 14. In most situations, I feel totally safe (safety) 15. My life is often disrupted by my symptoms (symptom interference) 16. Sometimes I’m afraid someone might hurt me (safety) 17. I have people in my life I can really count on (social network) 18. Life’s pressures lead me to lose control (symptom interference) 19. I have friends or family I really like (social network) 20. My symptoms interfere less and less with my life (symptom interference) 21. When my symptoms occur, I am able to manage them without falling apart (symptom interference) 49Rocky Mountain INFORMS, March 17, 2011
  50. 50. Quality Control Issues in Recovery  Multiple sources of variability  Measurement  Consumer  System  Changing environmental, treatment, and consumer specific factors affect outcome measurements.  Difficulty in detection of small changes due to large variability within and among consumers 50Rocky Mountain INFORMS, March 17, 2011
  51. 51. Multilevel Modeling and Recovery  Multilevel modeling allows for the partitioning of variance among multiple levels of nesting, i.e. measures within consumers within therapists  Allows for regression based correction of expected outcomes for any unit at any level, i.e. conditional estimates based upon consumer characteristics in environment or treatment.  Can be used to simultaneously monitor multiple aspects of the system from measurements to clinical sites.  Based upon Mixed-Effects ANOVA design 51Rocky Mountain INFORMS, March 17, 2011
  52. 52. Example of Multilevel modeling concepts Consumer Level Effect Typical SLR Model System Level Effect Intercept Intercept = + ACT Tx Intake = + Intercept Mood = + CRM + Disorder ACT Tx = Scores Intercept Intercept = + Time ACT Tx = + in Tx Intercept Mood = + Disorder ACT Tx Higher Level Effects 52Rocky Mountain INFORMS, March 17, 2011
  53. 53. Multilevel Regression Corrected Control Charts  CUSUM for Consumers (between consumer comparisons)  Allows for determination of a consumer’s progress as compared to peers in same treatment with environmental and demographic similarities 53Rocky Mountain INFORMS, March 17, 2011
  54. 54. Example MRC-CUSUM Self Comparison 54Rocky Mountain INFORMS, March 17, 2011
  55. 55. Example MRC-CUSUM Peer Comparison 55Rocky Mountain INFORMS, March 17, 2011
  56. 56. Utilization of MRC-CUSUM  Improved allocation of resources – by allowing consumer comparison to peers  Identification of factors that may promote/inhibit recovery  Provide feedback regarding progress and relapse more quickly to clinicians 56Rocky Mountain INFORMS, March 17, 2011
  57. 57. Multivariate Control Chart  Bivariate Control Chart for plotting of regression parameters (intercept and slopes)  Corrections may be made based upon environmental, treatment, and demographic characteristics 57Rocky Mountain INFORMS, March 17, 2011
  58. 58. I II III IV 58 58Rocky Mountain INFORMS, March 17, 2011
  59. 59. Recovery  Intercept BELOW  ABOVE  AVG. AVG. Decreasing Increasing Recovery Slope I II IV III NOTE: ANY Outlier within a quadrant indicates it is farther away from the  average than would be expected under typical circumstances. 59Rocky Mountain INFORMS, March 17, 2011
  60. 60. Utilization of Bivariate Control Chart  Identify “outlying” consumers to help determine aspects of a program that promote self-perceived recovery, and those aspects that may be a deterrent to improvement in self-perceived recovery.  Allow for identification of consumers who may need further resources or different treatment.  Allows for overview of consumer progress, where comparisons over time may allow for evaluation of process changes and overall consumer effect. 60Rocky Mountain INFORMS, March 17, 2011
  61. 61. Summary of Benefits  Allow for more efficient allocation of treatment and resources.  Identify program aspects that promote or deter improvement in self-perceived recovery.  Identify consumer in need of additional treatment or resources.  Allow for the identification of consumer and system factors that affect or interact with consumer outcomes and program effectiveness.  Being able to cater to differing needs of the wide variety of consumers served.  Identification of Episodes of Care 61Rocky Mountain INFORMS, March 17, 2011
  62. 62. Moving forward in recovery models to drive quality improvement Statistical Models Information Technology Knowledge Building & Dissemination: Learning Collaboratives Staff Involvement,Training 62Rocky Mountain INFORMS, March 17, 2011
  63. 63. Future Directions to Drive Recovery System Improvement  Identify clinically significant patterns  Expand to other recovery measures and aspects.  Coordinate with data mining to identify relationships between services and recovery outcomes  Automate quality control process  Integrate fully into clinical quality review processes  Develop accessible reporting and dashboard systems for clinicians and managers 63Rocky Mountain INFORMS, March 17, 2011
  64. 64. More information  If you would like to see more information concerning MHCD’s research and work with Recovery please visit: http://www.outcomesmhcd.com/ http://www.reachingrecovery.org/ Or contact Christopher.McKinney@mhcd.org 64Rocky Mountain INFORMS, March 17, 2011
  65. 65. Extra slides that were mentioned but not presented on 3/17/11 due to time limitations From Mayo Clinic Conference on Operations Research & Systems Engineering in Healthcare  Lean Options for Walk-In, Open Access, and Traditional Appointment Scheduling in Outpatient Health Care Clinics (LaGanga & Lawrence, 2009)  Includes further development to appointment scheduling models to include metaheuristic optimization of overbooking levels  Comparison of traditional scheduling, open-access, and walk-in policies  Lean process improvement to reduce no-shows and expand intake capacity.  Condensed slide set. See http://www.outcomesmhcd.com/Pubs.htm for complete, original presentation.  Driving Clinical Quality Improvement Through Mental Health Recovery Control Charts (McKinney, Olmos, McLean, LaGanga, 2010)  Poster presentation  First Place Award 65 Rocky Mountain INFORMS, March 17, 2011
  66. 66. Lean Options for Walk-In, Open Access, and Traditional Appointment Scheduling in Outpatient Health Care Clinics Mayo Clinic Conference on Systems Engineering & Operations Research in Health Care Rochester, Minnesota – August 17, 2009 Linda R. LaGanga, Ph.D. Stephen R. Lawrence, Ph.D. Director of Quality Systems Leeds School of Business Mental Health Center of Denver University of Colorado Denver, CO USA Boulder, CO USAAdditional information available at: http://www.outcomesmhcd.com/Pubs.htm 66 Rocky Mountain INFORMS, March 17, 2011 © 2008 – Linda LaGanga and Stephen Lawrence
  67. 67. 1. Background on Appointment Scheduling 67Rocky Mountain INFORMS, March 17, 2011
  68. 68. Motivation Healthcare Capacity  Funding restrictions  Demand exceeds supply  Serve more people with limited resources Manufacturing Scheduling  Resource utilization  Maximize throughput Healthcare Scheduling as the point of access Maximize appointment yield 68Rocky Mountain INFORMS, March 17, 2011
  69. 69. 2. Lean Approaches Rapid Improvement Capacity Expansion (RICE) Team January, 2008 Article in press, Journal of Operations Management (2010). Available at http://dx.doi.org/10.1016/j.jom.2010.12.005 69Rocky Mountain INFORMS, March 17, 2011
  70. 70. Lean Approaches  Reducing Waste  Underutilization  Overtime  No-shows  Patient Wait time  Customer Service  Choice  Service Quality  Outcomes 70Rocky Mountain INFORMS, March 17, 2011
  71. 71. Lean Process Improvement in Healthcare Documented success in hospitals  ThedaCare, Wisconsin  Prairie Lakes, South Dakota  Virginia Mason, Seattle  University of Pittsburgh Medical Center  Denver Health Medical Center Influences  Toyota Production System  Ritz Carleton  Disney Hospitals to Outpatient  Clinics run by hospitals  Collaborating outpatient systems 71Rocky Mountain INFORMS, March 17, 2011
  72. 72. Lean Process Improvement: One Year After Rapid Improvement Capacity Expansion RICE Results Analysis of the1,726 intake appointments for the one year before and the full year after the lean project 27% increase in service capacity  from 703 to 890 kept appointments) to intake new consumers 12% reduction in the no-show rate  from 14% to 2% no-show Capacity increase of 187 additional people who were able to access needed services, without increasing staff or other expenses for these services 93 fewer no-shows for intake appointments during the first full year of RICE improved operations. 72Rocky Mountain INFORMS, March 17, 2011
  73. 73. Lean Process Improvement: RICE Project System Transformation Appointments Scheduled and No-Show Rates 450 20% 400Appointments 350 15% 300 250 10% 200 150 100 5% 50 0 0% Mon Tue Wed Thu Fri Mon Tue Wed Thu Fri Year Before Year After Lean Improvement Lean Improvement Appointments No-Show Rate 73Rocky Mountain INFORMS, March 17, 2011
  74. 74. How was this shift accomplished?  Day of the week: shifted and added  Tuesdays and Thursdays  Welcome call the day before  Transportation and other information  Time lag eliminated  Orientation to Intake Assessment  Group intakes  Overbooking  Flexible capacity 74Rocky Mountain INFORMS, March 17, 2011
  75. 75. Lean Scheduling Challenge  Choice versus Certainty  Variability versus Predictability  Sources of Uncertainty / Variability  No-shows  Service duration  Customer (patients’) Demand  Time is a significant factor  Airline booking models? 75Rocky Mountain INFORMS, March 17, 2011
  76. 76. 3. Response to Overbooking 76Rocky Mountain INFORMS, March 17, 2011
  77. 77. Sample Responses  Campus reporter’s visit to student health center  “Not now and never will”  Patient waits 15 – 20 minutes  New administration, new interests  Morning News Radio  “Overbooking leading to increased patient satisfaction? That just doesn’t make any sense!”  Public Radio Interviewer  Benefits of increased access at lower cost 77Rocky Mountain INFORMS, March 17, 2011
  78. 78. Other Responses  Practitioners  Dentists  General medicine  Child advocacy  How should we overbook?  Other options  Lean Approaches  Open Access (Advanced Access)  Walk-ins 78Rocky Mountain INFORMS, March 17, 2011
  79. 79. 4. Enhanced Appointment Scheduling Model 20% 15% Probability 10% 5% 0% 0 1 2 3 4 5 6 7 8 9 10 11 12 Number Waiting (k) 79Rocky Mountain INFORMS, March 17, 2011
  80. 80. Objectives of Research  Optimize patient flow in health-care clinics  Traditionally scheduled (TS) clinic  Some patients do not “show” for scheduled appointments  TS clinic wishes to increase scheduling flexibility  Some capacity allocated to “open access” (OA) appointments, OR  Some capacity allocated to “walk-in” traffic  Balance needs of clinic, providers, and patients 80Rocky Mountain INFORMS, March 17, 2011
  81. 81. Objectives of Research  Study impact of open access and walk-in traffic  When is open access or walk-in traffic beneficial?  What mix of TS, OA, and WI traffic is best?  What are trade-offs of TS, OA, and WI on clinic performance? 81Rocky Mountain INFORMS, March 17, 2011
  82. 82. Relative Benefits and Penalties   = Benefit of seeing additional client   = Penalty for client waiting   = Penalty for clinic overtime  Numéraire of , , and  doesn’t matter  Ratios (relative importance) are important  Allow linear, quadratic, and mixed (linear + quadratic) costs 82Rocky Mountain INFORMS, March 17, 2011
  83. 83. Linear & Quadratic Objectives  Linear Utility Function  N k ˆ  S    A    k    i  1  N 1,     k Patient waiting Patient kwaiting N 1,k U ˆ jk ˆ Benefit from A  j 1 k k i 1  k Clinic overtime penalties during penalties during patients served penalties normal clinic ops clinic overtime  Quadratic Utility Function  N k ˆ  S    A     2k  1     i  12  U ˆ jk N 1, k     k 2 N 1,k ˆ A j 1 k k i 1  k 83Rocky Mountain INFORMS, March 17, 2011
  84. 84. Heuristic Solution Methodology 1. Gradient search  Increment/decrement appts scheduled in each slot  Choose the single change with greatest utility  Iterate until no further improvement found 2. Pairwise interchange  Exchange appts scheduled in all slot pairs  Choose the single swap with greatest utility  Iterate until no further improvement found 3. Iterate (1) and (2) while utility improves 4. Prior research: Optimality not guaranteed, but almost always obtained 84Rocky Mountain INFORMS, March 17, 2011
  85. 85. How does Open Access contribute to leaner scheduling? 1. It provides a more reliable method of overbooking. 2. It eliminates the uncertainty of demand for same-day appointments. 3. It guarantees that patients will be seen when they want. 4. It reduces uncertainty caused by no-shows. 5. It eliminates waste caused by unfilled appointments. 85Rocky Mountain INFORMS, March 17, 2011
  86. 86. How does Open Access contribute to leaner scheduling? 1. It provides a more reliable method of overbooking. 2. It eliminates the uncertainty of demand for same-day appointments. 3. It guarantees that patients will be seen when they want. 4. It reduces uncertainty caused by no-shows. 5. It eliminates waste caused by unfilled appointments. 86Rocky Mountain INFORMS, March 17, 2011
  87. 87. 5. Computational Results 10 10 9 9 Net Utility per Provider 8 Net Utility per Provider 8 7 7 6 6 5 5 4 Walk-ins 4 Walk-ins 3 3 Open Access 2 Open Access 2 1 -6.19 1 -6.19 0 0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Open Access (OA) Traffic (% of capacity) Open Access (OA) Traffic (% of capacity) 87Rocky Mountain INFORMS, March 17, 2011
  88. 88. Computational Trials 44 sample problems solved Session size N = 12 Appointment show rate  = 70% Number of providers P = {1, 2, 4, 8} OA call-in rate  = {0%, 10%, …100%} capacity  With P = 4 and N = 12, then  = 24 is 50% of capacity Walk-in rate  = {0%, 10%, …100%} of capacity  With P = 4, then  = 2 is 50% of capacity Quadratic costs  Parameters  =1.0,  =1.0,  =1.5 88Rocky Mountain INFORMS, March 17, 2011
  89. 89. Patients Seen 12 12 Patients Seen per Provider Patients Seen per Provider Walk-ins Walk-ins Open Access Open Access 11 11 2 Providers (P=2) 2 Providers (P=2) 10 10 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% OA or WI Traffic (% of capacity) OA or WI Traffic (% of capacity) N=12, P=1,  =0.7,  =1.0, =1.0,  =1.0,  =1.5 89Rocky Mountain INFORMS, March 17, 2011
  90. 90. Patient Waiting Time 1.0 1.0 Expected Waiting Time / Patient Expected Waiting Time / Patient 0.9 Walk-ins 0.9 Walk-ins 0.8 Open Access 0.8 Open Access 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% OA or WI Traffic (% of capacity) OA or WI Traffic (% of capacity) N=12, P=1,  =0.7,  =1.0, =1.0,  =1.0,  =1.5 90Rocky Mountain INFORMS, March 17, 2011
  91. 91. Clinic Overtime 2.5 2.5 Expected Provider Overtime Expected Provider Overtime 2.0 2.0 Walk-ins (d time units) Walk-ins (d time units) 1.5 Open Access 1.5 Open Access 1.0 1.0 0.5 0.5 0.0 0.0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% OA or WI Traffic (% of capacity) OA or WI Traffic (% of capacity) N=12, P=1,  =0.7,  =1.0, =1.0,  =1.0,  =1.5 91Rocky Mountain INFORMS, March 17, 2011
  92. 92. Provider Utilization 90% 90% Expected Provider Utilization 85% Expected Provider Utilization 85% 80% 80% 75% 75% 70% 70% Walk-Ins Walk-Ins 65% Open Acess 65% Open Acess 60% 60% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% OA or WI Traffic (% of capacity) OA or WI Traffic (% of capacity) N=12, P=1,  =0.7,  =1.0, =1.0,  =1.0,  =1.5 92Rocky Mountain INFORMS, March 17, 2011
  93. 93. Net Utility 10 10 9 9 Net Utility per Provider 8 Net Utility per Provider 8 7 7 6 6 5 5 4 Walk-ins 4 Walk-ins 3 3 Open Access Open Access 2 2 1 -6.19 1 -6.19 0 0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Open Access (OA) Traffic (% of capacity) Open Access (OA) Traffic (% of capacity) N=12, P=1,  =0.7,  =1.0, =1.0,  =1.0,  =1.5 93Rocky Mountain INFORMS, March 17, 2011
  94. 94. 6. Insights and Recommendations 94Rocky Mountain INFORMS, March 17, 2011
  95. 95. Managerial Implications TS appointments provide better clinic utility than does WI traffic or OA call-ins  Any WI or OA traffic causes some decline in utility An all-WI or all-OA clinic performs worse than any clinic with some TS appointments  Even a relatively small percentage of scheduled appointments can significantly improve clinic utility  Degree of improvement depends on number of providers A mix of TS appointments with some OA or WI traffic does not greatly reduce clinic performance (utility) 95Rocky Mountain INFORMS, March 17, 2011
  96. 96. Insights from the Model  Loss of utility with WI traffic is due to the long right-tail of Poisson distribution  Excessive patient waiting & clinic overtime  Loss of utility with OA traffic is due to uncertainty about number of OA call-ins  TS appts reduce patient waiting and clinic overtime  Binomial distribution has truncated right tail  Multiple providers improves clinic utility  Portfolio effect – variance reduction 96Rocky Mountain INFORMS, March 17, 2011
  97. 97. Lean Options for Walk-In, Open Access, and Traditional Appointment Scheduling in Outpatient Health Care Clinics Mayo Clinic Conference on Systems Engineering & Operations Research in Health Care Rochester, Minnesota – August 17, 2009 Linda R. LaGanga, Ph.D. Stephen R. Lawrence, Ph.D. Director of Quality Systems Leeds School of Business Mental Health Center of Denver University of Colorado Denver, CO USA Boulder, CO USAQuestions and comments? linda.laganga@mhcd.org(laganga@colorado.edu), stephen.lawrence@colorado.edu.Further information at http://www.outcomesmhcd.com/Pubs.htmRocky Mountain INFORMS, March 17, 2011 97 © 2008 – Linda LaGanga and Stephen Lawrence
  98. 98. Driving Clinical Quality Improvement Through Mental Health Recovery Control Charts C.J. McKinney, Pablo A. Olmos, Cathie McLean, Linda R. LaGanga, Division of Quality Systems, Mental Health Center of Denver, Denver, COPresented at the Mayo Clinic Conference on Systems Engineering & Operations Research in Health Care (August 2010), Rochester, MN. Awarded First Place for Best Poster Presentation. INTRODUCTION RECOVERY ASSESSMENT continued QUALITY CONTROL CHARTS continued Every community mental health center focuses on clinical quality. Benefits of effective service  delivery support quality through: Recovery Needs Level 3.  The Utilization Review Process: When a consumer is “flagged” by the Change Chart they will be an  The  Recovery Needs Level is a series of indicators  that through an objective algorithm  assigns the  automatic candidate for a utilization management review. This review is done by other clinicians  • optimize resource allocation, consumer to an appropriate clinical service level.  The  RNL is  completed by the clinician every six months  reviewing a consumer’s medical record to determine if a gap in services has occurred and if other  • increase consistency in consumer outcomes,  and as needed.   The measure consist of 15 different dimensions such as the GAF, Residence, Case  services should be considered. The recommendations from this review are forwarded to the program  • increase service fidelity,  Management, Substance Abuse, and Service Engagement. manager for further review and implementation. • decrease administrative load on clinicians, and  • increase access to consumer services. Utilization Review Form This poster presents our development of a set of reliable and valid mental health recovery  Promoting Recovery in Organizations Qualitative Identification of  measures, which we combine for a multi‐perspective assessment of recovery progress, which  The   PRO survey is completed by the consumer, and consists of 7 sections covering  all major  service  anchors an objective clinical quality control system.   positions at MHCD, i.e. front desk, nursing/medical, case management,  and rehabilitation.   This data is  Service Outliers collected annually through a random sampling of consumers.  The survey summaries are then utilized to  determine how well the teams and system are promoting recovery ideals. RECOVERY ASSESSMENTMHCD consistently collects, reviews, and analyzes data across all consumers on four different  recovery‐oriented outcome measurement tools.  The combined data from these assessments  provide multi‐perspective viewpoints for a more comprehensive picture of the consumer’s  recovery experience and what factors may be impacting their recovery.  It also provides  supporting information to ensure the consumer is placed at a level of care that appropriately  reflects their needs.Recovery Marker Inventory – Clinician AssessmentAssessments are recorded on seven factors associated with recovery:  Employment,  Learning/Education, Activity/Growth Orientation, Symptom  Interference, Participation in  Services, Housing, and Substance Use.Documentation of this data provides the clinician with a longitudinal perspective – from both an  overall standpoint, as well as more specific recovery dimensions.  These observations can then be  CONCLUSION & FUTURE DIRECTIONS used to help guide clinical discussion with the consumer, and indicate treatment focus. Consistent with continuous quality improvement, integration of these tools  into the clinical workflow is a constantly evolving process.  We feel the  QUALITY CONTROL CHARTS following are basic needs to meet, and opportunities for operational  The Recovery Outcome Tools have enabled us to develop a quality review system to monitor individual  enhancement: consumer outcomes and recommend review in cases where the consumer may not be progressing as  expected. We are able to do this in three ways: • Education of Clinical staff, Executive Management, Consumers, and other stakeholders as to the value of outcomes data collection and analysis and integration into the clinical  1.The Consumer Recovery Profile provides a snapshot of a person’s current mental health recovery  progress. It demonstrates through graphs and tables the current status of a consumer to aid in service  practice planning. • Technological ability to “communicate” with the Electronic Medical Record ‐ the  Recovery Profile is connected to the Electronic Medical Record, so it can be easily  accessed by clinicians by bringing the information to them, without having to log in or  open other data storage sites Consumer Recovery Measure – Consumer Assessment • Integration into the daily clinical work flow – clinicians can review outcomes  With the Consumer Recovery Measure, the consumer rates agreement or disagreement with  information with consumers during individual sessions, so as to make the information  statements regarding  their current recovery experience.  These responses gauge consumer  more meaningful; it is employed as part of the Peer Review process; and can be used  perspective on five dimensions of recovery:  Symptom Management, Sense of Safety, Sense of  Growth, Sense of Hope, and Social Activity. during six month case reviews • Automation of Quality Review process – control charts “flag” concerning outcomes  Graphic representation of this data is shared with the consumer to initiate clinical discussion about  changes in these areas, what  the consumer attributes the changes to, and possible relationships  outliers and identify them for Utilization Management Review, so as to address and  between categories.  This promotes insight, and empowers the consumer to share their story in a  redirect treatment inefficiencies in a timely manner new and different way. • Exploration of “super performer” characteristics to identify benchmarks for  2. The Recovery Change Chart automatically identifies consumers needing further review by flagging those  with substantial change in their recovery outcomes. A flag occurs whenever a consumer deviates from  teams/programs their expected outcomes for an extended period of time or if the deviations are large.  • Consumer Recovery Portal – consumers will have access to their outcomes data for  increased engagement in the recovery process Self‐Comparing Control Chart Peer‐Comparing Control Chart •Integrate physical and mental healthcare •Maximize outcomes to improve human lives! mental For more information about research or health recovery at MHCD, please view conference presentations on our website: 98 www.outcomesmhcd.com Rocky Mountain INFORMS, March 17, 2011

×