1st Project - Health Systems.
Day after day, health is becoming an increasingly hot issue in our daily life. Particularly, ageing can be thought as one of the primary causes for such an increasing demand and expense in health services. Therefore, it’s not surprising a larger fraction of the countries’ domestic gross product is being allocated to improve care, provided by health authorities, as well as public services, guaranteeing a pleasurable and safe coexistence among people.
One way of achieving such goals, without excessive expenditure, is using decision support models. In one hand, it’s true that forecasting [Request 3], linear programming [Request 4] or a mere construction of a decision tree [Request 2] entails some costs. But, at the end, countries or health services that better apply these mathematical techniques are achieving better results with the same or lower costs.
IST - 4th Year - 2nd Semester - Biomedical Engineering.
1. Technical University of Lisbon, University of Lisbon, PT
Advising Healthcare Organizations
ANTÓNIO CALÇADA1
, GONÇALO FRAZÃO2
, LUÍS RITA3
AND SEBASTIÃO BARROS4
1
79630, antoniocalcada@hotmail.com
2
78136, goncalo.frazao@tecnico.ulisboa.pt
3
78680, luis.rita@tecnico.ulisboa.pt
4
78478, sebastiao.barros@tecnico.ulisboa.pt
Introduction
Day after day, health is becoming an increasingly hot issue in our daily life. Particularly, ageing can be
thought as one of the primary causes for such an increasing demand and expense in health services.
Therefore, it’s not surprising a larger fraction of the countries’ domestic gross product is being allocated to
improve care, provided by health authorities, as well as public services, guaranteeing a pleasurable and
safe coexistence among people.
One way of achieving such goals, without excessive expenditure, is using decision support models. In one
hand, it’s true that forecasting [Request 3], linear programming [Request 4] or a mere construction of a
decision tree [Request 2] entails some costs. But, at the end, countries or health services that better apply
these mathematical techniques are achieving better results with the same or lower costs.
This is just one side of the coin. The other is more related to the quality of the provided care. Facing an
exponential growth of new medical devices and an expansion of new diagnostic techniques and treatments,
it’s becoming hard for the health providers to keep the same step. Thus, it’s important to invest in appraisal
services, in order to be able to present doctors and other health professionals, the most important and
updated guidelines they should adopt. This is very importantly related to EBM (Evidence Based Medicine)
[Request 1]. Which pursuit the objective of providing the right healthcare based on the most recent medical
guidelines and in the patient’s will. N.B.: A key point to obtain quickly and trustworthy information is by
applying the right search protocol. This other perspective is also strongly correlated with cost containment,
once e.g. a more efficient treatment, many times induces a quicker recover and a decreased use of (limited)
resources.
2. 2
Request 1
Evidence Behind Clinical Protocol
A 55 years old patient, with a past history of acute myocardial infarction (which occurred 2 years ago), poor
residual ventricular contractility (ejection fraction under 40%) presenting with symptoms and signs of heart
failure, visits the outpatient clinic of a cardiology department, seeking for specific treatment. The clinical
protocol in use in the department proposes the use of beta blockers (carvedilol or bisoprolol) in this
situation.
Which are the main sources of evidence behind the recommendation? To support your answer, formulate
a clinical question and apply a search protocol.
We came to the conclusion that there is strong evidence supporting the effectiveness of beta blockers in
reducing mortality in patients with heart failure with reduced ejection fraction (HFrEF), when comparing to
placebo or other standard treatments. Some of the results will be presented below.
In our search for medical evidence supporting the proposed treatment (administration of beta blockers to
treat heart failure) we started with the first step in Evidence Based Medicine, which is the formulation of a
clinical question. To do so, we identified each component of the P-I-C-O method to formulate a clinical
question:
P – Patient/Population/Problem – patients with heart failure
I – Intervention – treatment with beta blockers
C – Comparison – no treatment
O – Outcome – prolong survival
We finally reached the following question:
In patients with heart failure, do beta blockers, compared to no treatment, prolong survival?
To search for the best evidence to answer our question we resorted to PubMed[1]
, a database from the
National Center for Biotechnology Information (NCBI), U.S. National Library of Medicine. Since we were
looking for evidence already pre-appraised by experts in the subject, we started by looking for existing
meta-analysis or systematic reviews (the studies of studies standing in the top of the hierarchy of evidence
3. 3
pyramid) covering the results of beta blockers in heart failure treatment. In May 23, 2017, we searched
using the following keywords:
(("meta analysis"[Publication Type] OR "review"[Publication Type]) AND ("adrenergic beta-antagonists"
[Pharmacological Action] OR "adrenergic beta-antagonists"[MeSH Terms] OR ("adrenergic"[All Fields] AND
"beta-antagonists"[All Fields]) OR "adrenergic beta-antagonists"[All Fields] OR ("beta"[All Fields] AND
"blockers"[All Fields]) OR "beta blockers"[All Fields])) AND "heart failure"[All Fields]
We obtained 2956 results. Since we didn´t have the time to go through such a large pool of results we
decided to adapt the clinical question, to make it more specific to our clinical case. We changed the P
component to:
P – person with heart failure with reduced ejection fraction (HFrEF)
And reformulated the question:
In patients with heart failure with reduced ejection fraction, do beta blockers, compared to no treatment,
prolong survival?
After running a new search, using the same set of keywords but substituting "heart failure"[All Fields] by
"reduced ejection fraction"[All Fields] we obtained 62 results. Next, we confined the articles publishing
dates for the last 10 years, reducing to 55 results, and then we filtered the dates for the last 5 years,
obtaining 46 results. At this point we decided to go through the set of results and discard the ones in which
beta blockers and reduced ejection fraction weren´t present in the abstract.
Above we present a summary list of the Meta-Analysis we found most relevant for our clinical case:
Thirty Years of Evidence on the Efficacy of Drug Treatments for Chronic Heart Failure With Reduced Ejection
Fraction: A Network Meta-Analysis.[2]
A systematic literature review through 57 randomized controlled trials published between 1987 and 2015,
concluded that (1) β blockers are better than placebo, reducing all-cause mortalities and (2) the
combination of β blockers with two other substances was the treatment resulting in the greatest mortality
reduction in patients with HFrEF.
Effect of age and sex on efficacy and tolerability of β blockers in patients with heart failure with reduced
ejection fraction: individual patient data meta-analysis.[3]
4. 4
The study gathered data from 13 833 patients with HFrEF from 11 trials and concluded that, irrespective of
age or sex, β blockers reduced mortality and hospital admission for heart failure, when compared to
placebo.
Varying effects of recommended treatments for heart failure with reduced ejection fraction: meta-analysis
of randomized controlled trials in the ESC and ACCF/AHA guidelines.[4]
A study comparing the treatments recommended for HFrEF, after evaluating 47 randomized controlled
trials prior to 2013, concluded that (1) β blocker was the best treatment (out of the recommended four) to
reduce the risk of death and (2) also reduced hospitalization.
Benefits of β blockers in patients with heart failure and reduced ejection fraction: network meta-analysis.[5]
A study assessing 21 randomized trials comparing β blockers with other β blockers or other treatments,
revealed (1) a benefit of β blockers in mortality, in comparison with placebo or standard treatments, (2) no
relevant difference in the effect of the six (including carvedilol and bisoprolol) β blockers studied.
Network Meta-Analysis to Assess Comparative Effectiveness of Beta-Blockers in Patients with Heart Failure
and Reduced Ejection Fraction.[6]
A study assessing 21 randomized trials, in 23 122 patients, comparing the efficacy of 8 β blockers (including
carvedilol and bisoprolol) and placebo in the treatment of HFrEF, concluded that (1) all β blockers are more
effective reducing mortality than placebo, (2) carvedilol was the better agent for reducing mortality while
bisoprolol ranked third.
Regarding the validity of the findings, we think they are not generalizable
In summary, we found that there is strong evidence supporting the effectiveness of beta blockers in
reducing mortality and hospital admission in patients with HFrEF, when comparing both to placebo or other
standard treatments. Thus, we recommend the clinic to maintain their prescription of beta blockers
(carvedilol or bisoprolol) to patients with HFrEF.
Evidence Based Medicine main goal is to assist the decision making regarding the care of individual patient.
Thus, we remember that is up to the doctor to take into account its own expertise and the patient’s values
and circumstances, when applying the evidence-supported practice.
7. 7
Similarly, by progressively increasing and decreasing the costs for Buy and Make option (Keeping
Make Option Open), we were able to realize that a small variation of 6% in Make costs was needed to get
a new best decision strategy (Make Units, without keeping make option). And 13% to turn Buy Units
(without keeping make option) our best option. This is a feasible scenario, once the prices tend to increase
over time (assuming a time lapse between Keep Make Option Open and Make or Buy decision) due to
inflation or other specific events, like natural or human caused events. Moreover, we varied Keep Make
Option Open decision value ($ 17 500) and checked that for this parameter the model is significantly stable,
since it was needed to change it by 86% to get a different best solution. Generally, this allowed us to
conclude that the model is very sensitive to the parameters, namely: predicted costs and percentages.
All these results were carefully calculated and presented in an Excel file attached to this document
(see Request 2 sheet).
8. 8
Request 3
Surgical Service Performance
Forecasting is the technique of making anticipations of the future, based on evidence from the past and
present through a trend analysis. Risk and uncertainty are central to forecasting. Considering several
forecasting techniques, the one with the lowest degree of uncertainty which fits the data should be used.
In order to obtain a good forecast (timely, reliable, accurate, meaningful units, easy to use), the following
steps must be made: (1) Identify the goal of the forecast; (2) Establish a time horizon; (3) Select a forecasting
technique; (4) Conduct the forecast; (5) Determine its accuracy; (6) Monitor the forecast. This process is
independent of the approach used to obtain a forecast.
In order to forecast the number of surgeries that will be performed in the hospital after 2014, a
non-judgmental approach should be used. The non-judgmental approach we will use is the Time series.
With this approach, the behavior of the series is identified utilizing factor such as seasonality, cycles,
irregular variations, and random variations. The techniques available for this approach are: (1) Techniques
for averaging, which apply Naïve forecast, moving averages and exponential smoothing; (2) Techniques for
trend, which apply linear equations using regression and trend adjusted exponential smoothing; (3)
Techniques for seasonality, which apply seasonal variations and indices techniques. The technique to
forecast will depend on the behavior of the data.
In all of the following forecasts, the goal is to forecast the number of surgeries for 2015. The
forecasting technique will depend on the type of surgery, since each surgery type has its own particular
behavior across the year.
The forecasting technique for trend are not used in any surgery type. The behavior of the number
of surgeries across the years present no trend, demonstrating high variation pattern (the behavior of each
surgery type is plotted afterwards).
The forecasting technique for averaging, specifically the Naïve approach, is not used because of its
simplicity. This method uses last period's data as the following period's forecast, without adjusting them or
attempting to establish causal factors. This method should be applied if the data to be forecasted is not
sensible (data forecast can afford to have error without serous consequences and the budget is short.
The exponential smooth technique is not used, as well, because this technique requires a continuous
input of data. The forecast projection is made according to the last period’s forecast and the error between the
9. 9
forecasted values and the actual values. In our type of data, the period considered is one week. Hence, we
cannot forecast 2015 (only the first week, where the error from the last week of 2014 is used).
The average technique used, moving average technique, does not consider weights: We have no
information of any event that singularizes some of the periods, nor can we identify one from the data plots. Also,
we do not privilege the more recent information, because 2013 and 2014 data seem very similar to us.
Urgent surgeries, due to its nature, are very difficult to forecast. By analyzing the plot, we cannot
identify a pattern for the distribution of the number of surgeries performed across a period of one year:
For the year of 2013, the average number of surgeries performed was 51 and the highest variation
occurred in week 53, where only 10 surgeries were performed. The standard deviation corresponding to
this data is 24,42%. For the year of 2014, the average number of surgeries performed was 48 and the
highest variation occurred in week 29, where only 14 surgeries were performed. The standard deviation
corresponding to this data is 26,63%. With this numerical analysis, the process of selecting the technique
for forecasting is made easier: For the period of 2 years, the average number of surgeries performed was
almost equal and the percentage of deviation is also similar. Hence, an averaging technique for forecasting
will better fit the data, namely, moving average.
Regarding the moving average technique, two approaches were made as an attempt to optimize the
method: using 3 and 5 weeks as the period for averaging. The following forecast was obtained:
0
50
100
0 10 20 30 40 50 60
Number of surgeries
Week
Number of Urgent surgeries
Urgent surgeries 2013 Urgent series 2014
12. 12
The three periods occur in the beginning of the year, corresponding to January; mid-year,
corresponding to the summer months, accentuated in August; and the end of the year, corresponding to
December. These” low periods” coincide with vacation season, where the number of available surgeons is
decreased.
With this behavior, we forecast that the number of surgeries of 2015 will behave accordingly. Thus,
the technique for seasonality will better fit the data resulting in the most accurate forecast. With the
technique for seasonality, the following forecast was obtained for 2015: In order to apply this technique,
we average the number of surgeries from 2013 and 2014, to input more values into the technique to
provide a more accurate forecast. The weekly indexes are calculated and the adjusted forecast is plotted
forwardly:
As expected, the forecast presents 3 events where the number of surgeries decreases. In order to assess
the accuracy of this method, a forecast for 2014 was made using data from 2013:
0
100
200
300
400
500
600
700
0 10 20 30 40 50 60
Number of surgeries
Week
Number of Elective surgeries - Seasonality
Elective surgeries 2013 Elective surgeries 2014 Elective surgeries 2015 (Forecast)
16. 16
MAPE, mean absolute percent error, avoids the problem of interpreting the measure of accuracy relative
to the magnitudes of the actual and the forecast values:
𝑀𝐴𝑃𝐸 =
𝐴𝑐𝑡𝑢𝑎𝑙 − 𝐹𝑜𝑟𝑒𝑐𝑎𝑠𝑡
𝐴𝑐𝑡𝑢𝑎𝑙
As said before, we seek the lowest MAD and MAPE for a given set of data.
Table 1 - Values of MAD and MAPE for the forecasting techniques applied
Urgent Surgeries Elective Surgeries Elective Surgeries
MA – 3W MA – 5W Seasonal MA – 3W MA – 5W
MAD 10.94 12.188 61.986 16.02 15.875
MAPE 0.231 0.261 0.138 0.658 0.636
An extra accuracy control method was used, the tracking signal, that is obtained by the following equation:
𝑇𝑟𝑎𝑐𝑘𝑖𝑛𝑔 𝑠𝑖𝑔𝑛𝑎𝑙 =
𝐴𝑐𝑡𝑢𝑎𝑙 − 𝐹𝑜𝑟𝑒𝑐𝑎𝑠𝑡
𝑀𝐴𝐷
This method states that if the tracking signal is between the upper and lower limit, established by
literature, the model is assumed to be working correctly. This method also provides information about
the forecast behavior: if the tracking signal is positive, the actual value is greater than the forecast; if the
tracking signal is negative, the opposite behavior occurs.
In order test if the forecasted values are inside our desirable range: we defined the desirable deviation,
MAPE, for each type of surgery, and plot the tracking signals for each surgery type and verify if the
method is viable.
For the urgent surgeries and additional surgeries, we assumed an ideal MAPE of 25%, since the behavior
of the number of surgeries performed presents no pattern (random behavior), with considerable
standard deviation percentages. For the elective surgeries, the behavior of the number of surgeries
performed is more patterned, so we assumed an ideal MAPE of 15%. The required value of MAD to trace
19. 19
Request 4
Hiring Medical Staff
Provide advice to the hospital’s board taking into account that the hospital wants to maximize revenue.
Comment on your presented solution and on the available resources.
To maximize the estimated revenue, we would advise the Hospital’s Board to hire 5 Surgeons OR, 1
Anesthesiologist OR, 1 Physician ED and 5 Nurses ED. This solution was obtained using the Solver tool from
Excel 2016, in which we used LP Simplex model to maximize the estimated revenue,
150 ∗ 𝑆𝑢𝑟𝑔𝑒𝑜𝑛 + 200 ∗ 𝐴𝑛𝑒𝑠𝑡ℎ𝑒𝑠𝑖𝑜𝑙𝑜𝑔𝑖𝑠𝑡 + 150 ∗ 𝑃ℎ𝑦𝑠𝑖𝑐𝑖𝑎𝑛 + 50 ∗ 𝑁𝑢𝑟𝑠𝑒,
subjected to the following constraints:
𝐶1: −3 ∗ 𝑃ℎ𝑦𝑠𝑖𝑐𝑖𝑎𝑛 + 𝑁𝑢𝑟𝑠𝑒 ≥ 0
𝐶2: 𝑆𝑢𝑟𝑔𝑒𝑜𝑛 + 𝐴𝑛𝑒𝑠𝑡ℎ𝑒𝑠𝑖𝑜𝑙𝑜 𝑖𝑠𝑡 − 𝑃ℎ𝑦𝑠𝑖𝑐𝑖𝑎𝑛 − 𝑁𝑢𝑟𝑠𝑒 = 0
𝐶3: 5 ∗ 𝐴𝑛𝑒𝑠𝑡ℎ𝑒𝑠𝑖𝑜𝑙𝑜𝑔𝑖𝑠𝑡 − 𝑆𝑢𝑟𝑔𝑒𝑜𝑛 = 0
𝐶4: 100 ∗ 𝑆𝑢𝑟𝑔𝑒𝑜𝑛 + 100 ∗ 𝐴𝑛𝑒𝑠𝑡ℎ𝑒𝑠𝑖𝑜𝑙𝑜𝑔𝑖𝑠𝑡 + 80 ∗ 𝑃ℎ𝑦𝑠𝑖𝑐𝑖𝑎𝑛 + 50 ∗ 𝑁𝑢𝑟𝑠𝑒 ≤ 1000
Considering the optimal solution obtained (in which all the constraints were satisfied), the estimated
revenue will be 1350, 93% of the budget will be used (930 out of 1000) yielding a profit of 420 and serving
12 arbitrary units of patients. We considered that each individual new hired staff will serve the same
number of patients (1 arbitrary unit). Looking at the ratio between the estimated revenue and the cost, it
would be expected to hire more Anesthesiologists; however, we have only 1 due to the constrains.
Consider that the hospital’s board wants to maximize the additional number of patients served in the
hospital as a result of the new staff hired. Would this change your advice?
To maximize the number of additional patients served by the new hired staff, using the Solver tool from
Excel 2016, applying LP Simplex model, we reached the same solution of 5 Surgeons OR, 1 Anesthesiologist
OR, 1 Physician ED and 5 Nurses ED (subjected to the same constraints).
21. 21
However, we can distinguish some limitations in this modelling approach:
• Complex to determine the objective function;
• It isn’t easy to identify the constraints that best fit in the problem;
• Given a Specified objective and a set of constraints it is feasible that the constraints may not be
directly expressible as linear inequalities;
• A major problem in LP is to estimate the relevant values of the constant coefficients;
• It assumes that the relations between inputs and outputs are linear (inputs can be added, divided
and multiplied), which not always happen in real life;
• It is based on the hypothesis of constant returns, which does not always happen in real life due to
fluctuations;
• It is a highly mathematical and complex technique, that requires an abundance of mathematical
calculations. LP models present trial and error solutions, which makes it very difficult to find the
optimal solution.
Regarding the context of our problem, we also identified some issues associated to the formulation of the
problem when compared to a real Hospital’s situation:
• The estimated revenue and costs might differ from the actual revenue.
• We considered that each individual new hired staff will serve the same number of patients,
however this might not be realistic.
• With the used model, it is not possible to maximize two different values. Ideally, we want to
maximize both profit (revenue – costs) and the number of patients served. Nonetheless, after
maximizing each individual value, we found that the profit optimal solution already maximizes the
number of patients served (in question a) and b) with a budget of 1000).
• Low and unrealistic Budget in a hospital (1000)
The first two issues might be considered flexible since the values given to the variables depend on the
Hospital’s board side.
22. 22
Conclusion
With the above analysis, the hospital's management can improve greatly, allowing for e.g. resource
planning and staff hiring. This will result in a decrease in overall costs and improved efficiency, productivity
and profitability.