NY Prostate Cancer Conference - M.W. Kattan - Debate 1: Do I need a nomogram to make good decisions? (The answer is yes)


Published on

Published in: Health & Medicine, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

NY Prostate Cancer Conference - M.W. Kattan - Debate 1: Do I need a nomogram to make good decisions? (The answer is yes)

  1. 1. Do I need a nomogram to make good decisions?The answer is Yes.<br />Michael W. Kattan, Ph.D.<br />Professor of Medicine, Epidemiology and Biostatistics, Cleveland Clinic Lerner College of Medicine of Case Western Reserve University<br />Chairman, Department of Quantitative Health Sciences, Cleveland Clinic<br />
  2. 2. The reason you need prediction models<br />Is not because any model is perfect<br />But a prediction model is better than any alternative<br />
  3. 3. Alternative #1<br />KEEP THE PATIENT IN THE DARK<br />Avoid giving any estimate of prognosis or outcome<br />Just treat the patient as you see fit<br />No need to involve the patient in the decision<br />
  4. 4. Problems with keeping the patient in the dark<br />Screening and treatment might do more harm than good<br />About 25% of patients with clinically localized disease regret their treatment choice<br />Not conveying risks of harm is illegal in the United States<br />Taking advantage of a helpless patient is unethical and punishable by prison (e.g., Tuskegee case in U.S.).<br />
  5. 5. Belmont reportU.S. law on Human Research Ethics<br />… so-called risk/benefit assessments are concerned with the probabilities and magnitudes of possible harm and anticipated benefits. Many kinds of possible harms and benefits need to be taken into account. There are, for example, risks of psychological harm, physical harm, legal harm, social harm and economic harm and the corresponding benefits. <br />It should also be determined whether an investigator's estimates of the probability are reasonable… <br />
  6. 6. Alternative 2<br />Give the same predictions to all patients<br />Perhaps tell all patients that all treatment outcomes are 50%.<br />Or, have different predictions across treatments or outcomes, but give same predictions to all patients (no tailoring).<br />
  7. 7. Problems with same predictions for all patients<br />No discriminatory ability<br />AUC = 0.5 (coin toss)<br />Worse than any nomogram you would consider<br />
  8. 8. Alternative 3<br />Use risk groups (or count risk factors)<br />Assign patient to a risk group (e.g., low, intermediate, or high)<br />Or, count how many bad risk factors patient has<br />Convey risk<br />
  9. 9. Problems with risk groups<br />Lousy discrimination (AUC not much better than 0.5)<br />Heterogeneity within a group<br />
  10. 10. CaPSURE Heterogeneity within Risk Groups<br />Nomogram Values by Prostate Cancer Risk Group<br />1.0<br />0.9<br />0.8<br />0.7<br />Preoperative NomogramPredicted Probability<br />0.6<br />0.5<br />0.4<br />0.3<br />0.2<br />0.1<br />0.0<br />Low<br />Intermediate<br />High<br />Risk Group<br />J Urol. 2005 Apr;173(4):1126-31<br />
  11. 11. Why Nomograms Matter: A Particular Example<br />Mr. X, from the Cleveland Clinic:<br />PSA=6, clinical stage = T2c, biopsy Gleason sum=9, planned dose of 66.6 Gy without neoadjuvant hormones<br /><ul><li> Shipley risk stratification: 81% @ 5 yr.
  12. 12. Surgery nomogram: 68% @ 5yr.
  13. 13. Radiation therapy nomogram: 24% @ 5yr.</li></ul>Kattan MW, et al., J Clin. Oncol., 2000.<br />
  14. 14. Alternative 4<br />Use clinical judgment to estimate risks of benefits and harms<br />You know more about the patient than the nomogram does<br />Nomogram looks flawed anyway<br />
  15. 15. Problems with your clinical judgment<br />Your ability to calculate an accurate predicted probability is lousy<br />
  16. 16. Urologists vs. Preoperative Nomogram<br />10 case descriptions from 1994 MSKCC patients presented to 17 urologists<br />In addition to PSA, biopsy Gleason grades, and clinical stage, urologists were provided with patient age, systematic biopsy details, previous biopsy results, and PSA history.<br />Preoperative nomogram was provided.<br />Urologists were asked to make their own predictions of 5 year progression-free probabilities with or without use of the preoperative nomogram.<br />Concordance indices:<br />Nomogram = 0.67<br />Urologists = 0.55, p<0.05<br />Ross P et al., Semin Urol Oncol, 2002.<br />
  17. 17. Nomogram for predicting the likelihood of additional nodal metastases in breast cancer patients with a positive sentinel node biopsy <br />Vanzee K, et al., Ann Surg Oncol., 2003.<br />
  18. 18. Breast Cancer Prediction: 17 Clinicians vs. Nomogram on 33 Patients<br />Areas<br />0.75<br />0.72 Nomogram<br />0.68<br />0.65<br />0.65<br />0.63<br />0.59<br />0.58<br />0.55<br />0.55<br />0.53<br />0.52<br />0.50<br />0.49<br />0.47<br />0.43<br />0.42<br />0.40<br />Nomogram<br />AUC 0.72<br />Clinician<br />AUC 0.54<br />Specht et al., Ann Surg Oncol., 2005.<br />
  19. 19. Biases in Human Prediction<br /> Feedback<br /> recall, overconfident, hindsight bias, chance<br />adapted from Hogarth, 1988<br />
  20. 20. Treatment decision making worksheet<br />
  21. 21. 0<br />10<br />20<br />30<br />40<br />50<br />60<br />70<br /> 80<br />90<br />100<br />Points<br />PSA<br />4<br />20<br />0.1<br />1<br />2<br />3<br />6<br />8<br />9<br />10<br />12<br />16<br />30<br />45<br />70<br />110<br />7<br />T2a<br />T2c<br />T3a<br />ClinicalStage<br />T1c<br />T1ab<br />T2b<br /> 2+3<br /> 4+ ?<br />3+  2<br />Biopsy Gleason Grade<br /> 2+  2<br />3+3<br /> 3+ 4<br />Total Points<br /> 0<br />20<br />40<br />60<br />80<br />100<br />120<br />140<br />160<br />180<br />200<br />60MonthRec. Free Prob.<br />.96<br />.93<br />.9<br />.85<br />.8<br />.7<br />.6<br />.5<br />.4<br />.3<br />.2<br />.1<br />.05<br />Nomograms for clinical trial design<br />Example: CALGB 90203, preoperative therapy for patients at high risk of failure following surgery for prostate cancer <br />< 60%<br />Eastham et al., Urology, 2004.<br />
  22. 22. rcalc.ccf.org<br />
  23. 23. Invalid reasons not to use nomograms<br />Too difficult<br />Use online: www.nomograms.org, or rcalc.ccf.org<br />Based on retrospective data<br />Prospectively collected, retrospective data<br />Tomorrow’s outcomes will be better than yesterday’s<br />Some tools adjust for year of treatment<br />Unclear to what degree this is an issue<br />No way to really know, and alternative to nomogram is worse<br />Missing an input variable you think is important<br />Might not be important<br />Splitting a risk group is not the same as improving a nomogram<br />An axis looks wrong<br />Axis doesn’t mean what you think it does<br />
  24. 24. Conclusions<br />Patients need, and deserve, predictions of treatment outcomes.<br />Formulas, such as those behind nomograms and risk calculators, are the most accurate choice for predictions because they are tailored to the patient.<br />Nomogram predictions assist in:<br />Patient counseling<br />Shared decision-making<br />Informed consent<br />Clinical trial design and analysis<br />Not using nomograms:<br />Leads to inferior decision making by you and your patients<br />Causes patients to regret their treatment choices and what you did to them<br />Is unethical because it departs from evidence based medicine<br />