0
Is surgery an anachronism in an evidence based age? Richard Smith Editor, BMJ www.bmj.com/talks
 
 
What I want to talk about <ul><li>“ The history of surgery: my contribution” </li></ul><ul><li>Surgery: historical and lit...
“ The history of surgery: my contribution”
 
 
My role as emotional lightning conductor --hanging onto the second retractor <ul><li>“ Married are you, Richard?” </li></u...
 
My vasectomy <ul><li>Twenty years later I had a similar conversation, only this time I was under the knife. I was having a...
Surgery: historical and literary reflections
 
Hemicorporectomy <ul><li>Hemicorporectomy or translumbar amputation is probably the most mutilating operation ever to be d...
 
Nemesis <ul><li>Nemesis is the goddess of divine justice and vengeance. Her anger is directed toward human transgression o...
 
Sir William Arbuthnot Lane (1853-1946) <ul><li>Performed total colectomies for the treatment of “auto-intoxication,” somet...
Ironic quote from Arbuthnot Lane <ul><li>&quot;If everyone believes a thing it is probably untrue!&quot; </li></ul><ul><li...
 
Monologue by Sir Patrick Ridgeon <ul><li>I tell you, Colly, chloroform has done a lot of mischief. It's enabled every fool...
Cutler Walpole’s Nuciform Sac <ul><li>Cutler Walpole made his fortune from removing the nuciform sac from the fashionable ...
GBS on operations <ul><li>The large range of operations which consist of amputating limbs and extirpating organs admits of...
GBS on doctors sticking together <ul><li>The only evidence that can decide a case of malpractice is expert evidence: that ...
GBS on EBM <ul><li>It does happen exceptionally that a practising doctor makes a contribution to science; but it happens m...
John Rowan Wilson on surgeons <ul><li>Old Fred had always tended to discount the surgeons.  He visualised them as a pack o...
The state of evidence in surgery
Useless operations <ul><li>Ligation of the internal mammary artery for angina </li></ul><ul><li>Extracranial/intracranial ...
State of the evidence <ul><li>“Unless assessments of surgical procedures are seen to be unbiased, properly randomised, and...
Large, randomised trials are needed <ul><li>Benefits are often small--so large trials are needed so as not to miss small b...
The safety and efficacy of new interventional procedures: published evidence and clinical guidance Tom Dent, Sally Wortley...
NICE review of evidence: methods  <ul><li>245 procedures </li></ul><ul><li>84 are in the Safety and Efficacy Register of N...
NICE review of evidence: results <ul><li>262 studies (an average of 5.2 per procedure)  </li></ul><ul><li>178 (68%) were c...
NICE review of evidence: results <ul><li>A fifth of studies supporting “safe and efficacious” procedures by randomised tri...
NICE review of evidence: conclusions <ul><li>“ Many interventional procedures enter clinical practice with limited publish...
State of the evidence <ul><li>Few systematic reviews </li></ul><ul><li>Few randomised trials </li></ul><ul><li>Those that ...
A snapshot of the evidence <ul><li>RCTs declined from 14% of research articles in the  British Journal   of Surgery  in 19...
Article in two issues of the British Journal of Surgery <ul><li>Case series  29(51%) </li></ul><ul><li>Animal studies 7 (1...
Analysis of general practice studies from the BMJ <ul><li>Questionnaire survey 6 (27%) </li></ul><ul><li>RCT 5 (23%) </li>...
The evidence patients and doctors need on a new treatment <ul><li>A large, well conducted RCT comparing the new treatment ...
A placebo controlled trial of athroscopy for osteoarthritis of the knee <ul><li>185 patients randomise to arthroscopic déb...
Moseley, J. B. et al. N Engl J Med 2002;347:81-88 Mean Values (and 95 Percent Confidence Intervals) on the Knee-Specific P...
Causes of lack of evidence and solutions <ul><li>Much of what I have to say comes from the following article </li></ul><ul...
Causes of lack of evidence and solutions <ul><li>History : Many operations developed before RCTs were expected </li></ul><...
Causes of lack of evidence and solutions <ul><li>Commercial competition and personal prestige : many financial drivers in ...
Causes of lack of evidence and solutions <ul><li>Solution: Move towards constant collection of data on outcomes of surgery...
Causes of lack of evidence and solutions <ul><li>Surgeon’s equipoise : Surgeons are (attractively) decisive and used to ac...
Causes of lack of evidence and solutions <ul><li>Lack of funding, infrastructure, and experience of data collection   </li...
Causes of lack of evidence and solutions <ul><li>Lack of education in clinical epidemiology.  Surgeons' knowledge of clini...
Causes of lack of evidence and solutions <ul><li>The learning curve : Should RCTs be done on new treatments or once surgeo...
Causes of lack of evidence and solutions <ul><li>The quality of the surgery : It must be similar for all interventions bei...
Causes of lack of evidence and solutions <ul><li>Development versus research : much innovation comes from small sequential...
Causes of lack of evidence and solutions <ul><li>Patient equipoise : This equipoise is really more important than the surg...
Causes of lack of evidence and solutions <ul><li>Blinding : clearly very difficult in surgical trials </li></ul><ul><li>So...
High quality databases can supplement or even provide an alternative to RCTs <ul><li>Must include individual data on all c...
NICE (National Institute of Clinical Excellence) on interventions <ul><li>All new interventions must be notified--by gover...
NICE on interventions <ul><li>IPAC rules </li></ul><ul><ul><li>“ Safe and efficacious” </li></ul></ul><ul><ul><li>Uncertai...
 
Comments <ul><li>This is progress </li></ul><ul><li>Or is it bureaucracy? </li></ul><ul><li>It looks like a compromise. Is...
Conclusions <ul><li>Surgery has been far from evidence based, which has led to some horrible excesses </li></ul><ul><li>Th...
Conclusions <ul><li>High quality databases should be very helpful </li></ul><ul><li>Various countries--including Australia...
Final thoughts
A good surgeon <ul><li>“ Good surgeons know how to operate, better surgeons when to operate, and the best when not to oper...
Is there anybody less evidence based than surgeons? Yes Politicians Managers Editors
Upcoming SlideShare
Loading in...5
×

Is Surgery An Anachronism In An Evidence Based Age

307

Published on

Published in: Health & Medicine, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
307
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Is Surgery An Anachronism In An Evidence Based Age"

  1. 1. Is surgery an anachronism in an evidence based age? Richard Smith Editor, BMJ www.bmj.com/talks
  2. 4. What I want to talk about <ul><li>“ The history of surgery: my contribution” </li></ul><ul><li>Surgery: historical and literary reflections </li></ul><ul><li>The state of evidence in surgery </li></ul><ul><li>Improving the evidence base </li></ul>
  3. 5. “ The history of surgery: my contribution”
  4. 8. My role as emotional lightning conductor --hanging onto the second retractor <ul><li>“ Married are you, Richard?” </li></ul><ul><li>“ No.” </li></ul><ul><li>“ Good idea. The only point I can see in being married is that it saves you having to find somebody to go on holiday with.” </li></ul><ul><li>“ Do you have horses?” </li></ul><ul><li>“ No. I live in a flat.” </li></ul><ul><li>“ Good chap. Super weather, isn’t it?” </li></ul><ul><li>And so it went until he cut something he shouldn’t have. “Bloody hell. Will you pull on that retractor, Smith? You are a bloody fool. Don’t you know anything about surgery? For God’s sake, pull harder.” </li></ul>
  5. 10. My vasectomy <ul><li>Twenty years later I had a similar conversation, only this time I was under the knife. I was having a vasectomy. The surgeon, who did 20 vasectomies a day five days a week, had already cut my right vas when he asked me what I did. </li></ul><ul><li>“ I’m a sort of journalist.” </li></ul><ul><li>“ What sort?” </li></ul><ul><li>“ A medical journalist.” </li></ul><ul><li>“ Oh. Where do you work?” </li></ul><ul><li>“ At the BMJ.” </li></ul><ul><li>“ What exactly do you do there?” </li></ul><ul><li>“ I’m the editor.” </li></ul><ul><li>Suddenly the surgeon began to sweat. It had taken him two minutes to find and cut one vas. He now took 20 minutes to find the second. “Oh God,” he said, “I suppose I’m going to read about this in next week’s journal.” </li></ul>
  6. 11. Surgery: historical and literary reflections
  7. 13. Hemicorporectomy <ul><li>Hemicorporectomy or translumbar amputation is probably the most mutilating operation ever to be described in surgical literature. </li></ul><ul><li>Treatment for pelvic malignancy </li></ul><ul><li>The procedure involves removal of the bony pelvis, both lower limbs, the external genitalia, the bladder, rectum and anus. </li></ul><ul><li>First proposed in 1951 and performed in 1960 </li></ul><ul><li>Two of the first three patients died within days of oedema; one survived 19 years </li></ul><ul><li>A series of 10 cases reported from New York in 1982 </li></ul>
  8. 15. Nemesis <ul><li>Nemesis is the goddess of divine justice and vengeance. Her anger is directed toward human transgression of the natural, right order of things and of the arrogance causing it. Nemesis pursues the insolent and the wicked with inflexible vengeance. </li></ul>
  9. 17. Sir William Arbuthnot Lane (1853-1946) <ul><li>Performed total colectomies for the treatment of “auto-intoxication,” something like chronic fatigue syndrome </li></ul><ul><li>Operated on many of London’s fashionable set </li></ul><ul><li>Operation had a 10% mortality </li></ul><ul><li>Dreadful side effects </li></ul><ul><li>The placebo effect guarantees that some will be “cured” </li></ul>
  10. 18. Ironic quote from Arbuthnot Lane <ul><li>&quot;If everyone believes a thing it is probably untrue!&quot; </li></ul><ul><li>Quoted by W. E. Tanner in Sir W. Arbuthnot Lane, &quot;Genesis&quot; </li></ul>
  11. 20. Monologue by Sir Patrick Ridgeon <ul><li>I tell you, Colly, chloroform has done a lot of mischief. It's enabled every fool to be a surgeon. I know your Cutler Walpoles and their like. They've found out that a man's body is full of bits and scraps of old organs he has no mortal use for. Thanks to chloroform, you can cut half a dozen of them out without leaving him any the worse, except for the illness and the guineas it costs him. I knew the Walpoles fifteen years ago. The father used to snip off the ends of people's uvulas for fifty guineas, and paint throats with caustic every day for a year at two guineas a time. His brother-in-law extirpated tonsils for two hundred guineas until he took up women's cases at double the fees. Cutler himself worked hard at anatomy to find something fresh to operate on; and at last he got hold of something he calls the nuciform sac, which he's made quite the fashion. People pay him five hundred guineas to cut it out. They might as well get their hair cut for all the difference it makes; but I suppose they feel important after it. You can't go out to dinner now without your neighbor bragging to you of some useless operation or other. </li></ul>
  12. 21. Cutler Walpole’s Nuciform Sac <ul><li>Cutler Walpole made his fortune from removing the nuciform sac from the fashionable in London </li></ul><ul><li>“ And have you had your nuciform sac removed, Mr Walpole.” </li></ul><ul><li>“ I’m one of the lucky and unusual people who was born without one.” </li></ul>
  13. 22. GBS on operations <ul><li>The large range of operations which consist of amputating limbs and extirpating organs admits of no direct verification of their necessity. There is a fashion in operations as there is in sleeves and skirts. ... There are men and women whom the operating table seems to fascinate: half-alive people who through vanity, or hypochondria, or a craving to be the constant objects of anxious attention or what not, lose such feeble sense as they ever had of the value of their own organs and limbs. </li></ul>
  14. 23. GBS on doctors sticking together <ul><li>The only evidence that can decide a case of malpractice is expert evidence: that is, the evidence of other doctors; and every doctor will allow a colleague to decimate a whole countryside sooner that violate the bond of professional etiquette by giving him away. </li></ul>
  15. 24. GBS on EBM <ul><li>It does happen exceptionally that a practising doctor makes a contribution to science; but it happens much oftener that he draws disastrous conclusions from his clinical experience because he has no conception of scientific method, and believes, like any rustic, that the handling of evidence and statistics needs no expertness. </li></ul>
  16. 25. John Rowan Wilson on surgeons <ul><li>Old Fred had always tended to discount the surgeons. He visualised them as a pack of blood and thunder, cut and thrust, bombastic dunderheads, too lacking in intelligence to constitute a danger to his own system. In his eyes they were the cavalry officers of medicine, dashing and romantic, useful to impress the simple hearted, but totally unimportant when there was any serious business to be done. He had outwitted the surgeons all his life and regarded them with scant respect. </li></ul>
  17. 26. The state of evidence in surgery
  18. 27. Useless operations <ul><li>Ligation of the internal mammary artery for angina </li></ul><ul><li>Extracranial/intracranial artery connection </li></ul><ul><li>Radical mastectomy </li></ul><ul><li>Sympathectomy for peripheral vascular disease </li></ul><ul><li>Spinal fusion </li></ul>
  19. 28. State of the evidence <ul><li>“Unless assessments of surgical procedures are seen to be unbiased, properly randomised, and with objective assessment of outcomes they will continue to lack credibility.” </li></ul><ul><li>A G Johnson and J Michael Dixon Removing bias in surgical trials BMJ, Mar 1997; 314: 916. </li></ul>
  20. 29. Large, randomised trials are needed <ul><li>Benefits are often small--so large trials are needed so as not to miss small but still important effects. Large trials are also better at detecting harms, which are often rare. </li></ul><ul><li>Randomisation is needed to avoid the bias that is all pervasive </li></ul><ul><li>The higher the quality of the trial the smaller the beneficial effect </li></ul>
  21. 30. The safety and efficacy of new interventional procedures: published evidence and clinical guidance Tom Dent, Sally Wortley, Bruce Campbell
  22. 31. NICE review of evidence: methods <ul><li>245 procedures </li></ul><ul><li>84 are in the Safety and Efficacy Register of New Interventional Procedures (SERNIP) </li></ul><ul><li>Authors randomly selected 25 procedures considered “safe and efficacious enough for routine use” and 25 of uncertain safety and efficacy </li></ul><ul><li>Two authors independently reviewed the evidence </li></ul>
  23. 32. NICE review of evidence: results <ul><li>262 studies (an average of 5.2 per procedure) </li></ul><ul><li>178 (68%) were case series </li></ul><ul><li>43 (16%) non-randomised comparisons </li></ul><ul><li>33 (13%) randomised trials </li></ul><ul><li>8 (3%) systematic reviews </li></ul>
  24. 33. NICE review of evidence: results <ul><li>A fifth of studies supporting “safe and efficacious” procedures by randomised trials--in contrast to 3% of the studies supporting uncertain procedures </li></ul><ul><li>Median number of patients in the studies supporting each procedure 583 (range 3 to 2873) </li></ul><ul><li>Follow up poorly reported </li></ul>
  25. 34. NICE review of evidence: conclusions <ul><li>“ Many interventional procedures enter clinical practice with limited published evaluation of their safety and efficacy” </li></ul><ul><li>“ The nature of the available studies means that the validity and durability of benefit often cannot be adequately assessed and important risks cannot be excluded” </li></ul>
  26. 35. State of the evidence <ul><li>Few systematic reviews </li></ul><ul><li>Few randomised trials </li></ul><ul><li>Those that exist are often of poor quality--which is true of all trials </li></ul><ul><li>Very few placebo controlled trials </li></ul><ul><li>Too many case series, which are ultimately uninterpretable </li></ul>
  27. 36. A snapshot of the evidence <ul><li>RCTs declined from 14% of research articles in the British Journal of Surgery in 1985 to 5% in 1992 </li></ul><ul><li>Treatments in general surgery are half as likely to be based on RCT evidence as treatments in internal medicine </li></ul><ul><li>75% of “surgical trials” are actually of medical treatments in surgical patients </li></ul><ul><li>Only a third of surgical trials have adequate blinding </li></ul><ul><li>In a study of 10 international journals from 1988 to 1994, Hall et al found that, of the few randomised controlled trials that were published, less than half included objective methods for assessing outcome </li></ul>
  28. 37. Article in two issues of the British Journal of Surgery <ul><li>Case series 29(51%) </li></ul><ul><li>Animal studies 7 (12%) </li></ul><ul><li>Non-systematic reviews 6 (11%) </li></ul><ul><li>RCTs 4 (7%) </li></ul><ul><li>Audit 3 (5%) </li></ul><ul><li>Four other designs 8 (14%) </li></ul>
  29. 38. Analysis of general practice studies from the BMJ <ul><li>Questionnaire survey 6 (27%) </li></ul><ul><li>RCT 5 (23%) </li></ul><ul><li>Database study 2 (9%) </li></ul><ul><li>Case control study 2 (9%) </li></ul><ul><li>Five other designs 7 (42%) </li></ul>
  30. 39. The evidence patients and doctors need on a new treatment <ul><li>A large, well conducted RCT comparing the new treatment with established evidence based treatments </li></ul><ul><li>To be “evidence based” a treatment will at some time have had to be tested against placebo </li></ul><ul><li>An RCT against placebo only if no evidence based treatment exists </li></ul>
  31. 40. A placebo controlled trial of athroscopy for osteoarthritis of the knee <ul><li>185 patients randomise to arthroscopic débridement, arthroscopic lavage, or placebo surgery </li></ul><ul><li>Patients in the placebo group received skin incisions and underwent a simulated débridement without insertion of the arthroscope </li></ul><ul><li>Blind assessment of outcome </li></ul><ul><li>Followed up for two years </li></ul><ul><li>Several outcome measures </li></ul><ul><li>165 patients completed the trial </li></ul><ul><li>Moseley JB, O'Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002; 347: 81-88 [ </li></ul>
  32. 41. Moseley, J. B. et al. N Engl J Med 2002;347:81-88 Mean Values (and 95 Percent Confidence Intervals) on the Knee-Specific Pain Scale
  33. 42. Causes of lack of evidence and solutions <ul><li>Much of what I have to say comes from the following article </li></ul><ul><li>Randomised trials in surgery: problems and possible solutions. </li></ul><ul><ul><li>Peter McCulloch, Irving Taylor, Mitsuru Sasako, Bryony Lovett, and Damian Griffin BMJ 2002 324: 1448-1451. </li></ul></ul>
  34. 43. Causes of lack of evidence and solutions <ul><li>History : Many operations developed before RCTs were expected </li></ul><ul><li>Solution: Comprehensively review surgical evidence. Be clear what we know and don’t know. </li></ul>
  35. 44. Causes of lack of evidence and solutions <ul><li>Commercial competition and personal prestige : many financial drivers in surgery </li></ul><ul><li>Hard to test objectively “your operation” </li></ul><ul><li>“ Surgeons' eagerness to learn the operation [laparascopic cholecystectomy] seemed related more to commercial concerns than to concern for patients.” </li></ul>
  36. 45. Causes of lack of evidence and solutions <ul><li>Solution: Move towards constant collection of data on outcomes of surgery and use statistical systems of continuous quality control to accumulate evidence of effectiveness and harms and decide when RCTs are necessary </li></ul>
  37. 46. Causes of lack of evidence and solutions <ul><li>Surgeon’s equipoise : Surgeons are (attractively) decisive and used to acting in uncertain circumstances. They may thus be less comfortable with accepting the uncertainty and recognising that they have the “equipoise” necessary for a trial. </li></ul><ul><li>Solution: include parallel, non-randomised, preference arms alongside RCTs. </li></ul>
  38. 47. Causes of lack of evidence and solutions <ul><li>Lack of funding, infrastructure, and experience of data collection </li></ul><ul><li>Solution: change to a culture of cooperation rather than competition. Form large groups to perform specific trials </li></ul>
  39. 48. Causes of lack of evidence and solutions <ul><li>Lack of education in clinical epidemiology. Surgeons' knowledge of clinical epidemiology remains poor </li></ul><ul><li>Solution: Include training in routine surgical training. (Clinical epidemiology is the basic science of clinical research.) More advanced training for some. </li></ul>
  40. 49. Causes of lack of evidence and solutions <ul><li>The learning curve : Should RCTs be done on new treatments or once surgeons have learnt to do them? Obvious dangers in comparing a new with a familiar operation. Solution: statistical techniques can help. Determine whether the top of the curve has been reached before analysing data. </li></ul>
  41. 50. Causes of lack of evidence and solutions <ul><li>The quality of the surgery : It must be similar for all interventions being tested </li></ul><ul><li>Solution: quality control must be part of the trial. Video evidence might be collected. </li></ul>
  42. 51. Causes of lack of evidence and solutions <ul><li>Development versus research : much innovation comes from small sequential changes not large leaps. When to do an RCT? </li></ul><ul><li>Solution: Use statistical quality improvement techniques. Ask independent third parties to decide when an RCT is necessary </li></ul>
  43. 52. Causes of lack of evidence and solutions <ul><li>Patient equipoise : This equipoise is really more important than the surgeon’s equipoise--and allocation of treatment by chance is especially hard when it’s a medical versus a surgical treatment </li></ul><ul><li>Solution: Not easy. Decision analysis techniques may help. </li></ul>
  44. 53. Causes of lack of evidence and solutions <ul><li>Blinding : clearly very difficult in surgical trials </li></ul><ul><li>Solution: Blinded observers should be used routinely for assessing outcome </li></ul>
  45. 54. High quality databases can supplement or even provide an alternative to RCTs <ul><li>Must include individual data on all consecutive cases </li></ul><ul><li>Must use standard definitions of conditions and outcomes </li></ul><ul><li>Must ensure data are complete and accurate </li></ul><ul><li>Must include data on all known patient characteristics that affect outcome </li></ul><ul><li>Users must know how to risk adjust </li></ul>
  46. 55. NICE (National Institute of Clinical Excellence) on interventions <ul><li>All new interventions must be notified--by government decree; always doubt on what is a new procedure </li></ul><ul><li>IPAC (Interventional Procedures Advisory Committee) prepares an overview (not a systematic review) of “the literature” </li></ul><ul><li>IPAC also consults specialist advisers--people “held in high regard by their colleagues; they will not necessarily be enthusiasts, with the bias which can accompany such enthusiasm” </li></ul>
  47. 56. NICE on interventions <ul><li>IPAC rules </li></ul><ul><ul><li>“ Safe and efficacious” </li></ul></ul><ul><ul><li>Uncertainty </li></ul></ul><ul><ul><li>Unsafe </li></ul></ul><ul><li>Uncertain procedures can be undertaken “judiciously” but patients must be informed of the uncertainty; data must be gathered </li></ul><ul><li>All subject to consultation </li></ul><ul><li>Guidance is published </li></ul><ul><li>Existing procedures have been reviewed </li></ul>
  48. 58. Comments <ul><li>This is progress </li></ul><ul><li>Or is it bureaucracy? </li></ul><ul><li>It looks like a compromise. Is this a “practical, sensible way forward” or a “cop out”? Such a system wouldn’t be acceptable for drugs </li></ul><ul><li>Can surgery ever be more evidence based? </li></ul>
  49. 59. Conclusions <ul><li>Surgery has been far from evidence based, which has led to some horrible excesses </li></ul><ul><li>The state of evidence in surgery is poor </li></ul><ul><li>It’s much harder to do RCTs in surgery than in medicine </li></ul><ul><li>But the problems can be overcome--and should be </li></ul>
  50. 60. Conclusions <ul><li>High quality databases should be very helpful </li></ul><ul><li>Various countries--including Australia and Britain--are developing ways of regulating new interventions </li></ul><ul><li>These are very much at the beginning and have a long way to develop </li></ul>
  51. 61. Final thoughts
  52. 62. A good surgeon <ul><li>“ Good surgeons know how to operate, better surgeons when to operate, and the best when not to operate.” </li></ul>
  53. 63. Is there anybody less evidence based than surgeons? Yes Politicians Managers Editors
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×