Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Rhetorical Moves and Audience
Considerations in the Discussion
Sections of Randomized Controlled Trials
of Health Interven...
Disclaimer
The views and opinions expressed do not
necessarily state or reflect those of the U.S.
Government, and they may...
Why study scientific papers?
• “The interpretive face of science is
characterized by controversy,
disagreement, dissensus....
Why study medical papers?
• Large potential impact on human health
• Input to evidence-based practice
• Well-elaborated st...
Hierachy of Evidence
Hierachy of Evidence
Randomized Controlled Trial
(RCT)
• Medical RCTs test the
efficacy of medical
treatments.
• Features:
– Control group (pla...
Corpus
• 37 research articles reporting on medical
trials, mainly RCTs
• Full-text available from PubMed Central
• Chosen ...
Other ongoing study on this corpus
• Understand citation structure of Discussion
sections of Randomized Clinical Trials
• ...
Corpus characteristics
• Various conditions, e.g.
– Brain injury
– Degenerative eye disease
– Malaria
– Postpartum back pa...
Corpus characteristics
• Various locales, e.g.
– Madagascar
– Malawi
– Netherlands
– Uganda
– United Kingdom
– United Stat...
Focus: 37 discussion sections
http://europepmc.org/
articles/PMC1420299
Rhetorical Moves in RCT
Discussion Sections
Rhetorical Moves in RCT
Discussion Sections
• Novelty
• Gaps/opportunities
• Contextualization
• Comparability to previous...
Novelty
• “We report the first safety data collected
in the setting of a randomized, controlled
study for patients sustain...
Gaps/opportunities
• "At the start of this study, little was known about
the specific effect of psychosocial determinants ...
Contextualization
• “This study is part of a series to explore effective xylitol
delivery vehicles that can be used in sch...
Comparability to previous results
• "However, none of these 11 studies were
performed on pediatric populations, making
fur...
Validity of methods
"Algometry is used to measure the sensitivity of pain or pressure [4].
Algometer instrumentation can i...
Validity of methods
"The technique of using the
consumption of morphine during
PCA treatment of postoperative
pain, as a m...
Anticipating objections
"It might be argued that these observations of blood
stage immunity (reduction in growth rates, pr...
Pointers to ongoing work
• "An ongoing phase III study is likely to
provide a better evaluation of safety in this
patient ...
Acknowledging limitations
• "Among the limitations of this study is the
screening strategy for the recruitment of
the pati...
Future work
• "New studies should be carried out, with
longer follow-up times and larger
samples, to evaluate the effects ...
Summaries
• "Based on present evidence BTX cannot be
recommended as treatment for neck pain in
chronic whiplash patients."...
Rhetorical Moves in RCT
Discussion Sections
• Novelty
• Gaps/opportunities
• Contextualization
• Comparability to previous...
Audience
Multiple audiences
“Audiences of specialists…do not comprise
the only audiences who respond to scientific
or other profess...
Interested audiences include:
• Specialists responsible for “authorizing the
claims to knowledge” (Prelli,1989, p26)
• Non...
Fulfilling audience expectations
“To succeed rhetorically, any rhetor must ground a
position in what the addressed audienc...
Audience-specific sections from
discussion sections of reports of
Randomized Controlled Trials
Researchers are the expected
audience for RCTs
Mellis, Craig. "Evidence-based Medicine: What Has Happened in the
Past 50 Y...
Audiences we find
• Specialist researchers in the same area
• Clinicians
• Clinical practice managers
• Grant funders
• He...
Specialist researchers in the same
area
• “The most plausible explanation for the
similar trends in BV reduction in both a...
Clinicians
• “caution should be exercised in applying these findings
to the general management of dental caries without
be...
Clinical practice managers
• “Our findings are particularly pertinent
to smaller clinical practices that might
not be alig...
Grant funders
• “Additional research is needed to
determine whether these improvements
translate into even longer-term red...
Grant funders
"Admittedly, overall complexity of the issue addressed by the
present study clearly merits pursuit of furthe...
Health systems
• “Given the rising costs and growing burden of diabetes on the
health-care system [57], strategies that ai...
Audiences we find
• Specialist researchers in the same area
• Clinicians
• Clinical practice managers
• Grant funders
• He...
Rhetorical Moves in RCT
Discussion Sections
• Novelty
• Gaps/opportunities
• Contextualization
• Comparability to previous...
Future work
• Systematically code rhetorical moves and
audiences – likely to lead to revisions
• Compare to previous work ...
Citation examples from
discussion sections of reports of
Randomized Controlled Trials
“This” work agrees with…
• “This is in accordance with earlier studies
in the ambulatory surgical setting [3]” -
PMC1637100
Definitions and background info
• “Self-efficacy, which may relate to
motivation, is the perceived confidence in
one's abi...
Presenting a range of evidence
• “Except in one study [20], short-term
administration of GH transiently worsened
insulin r...
Clause-level changes in meaning
• “Two of four randomised clinical trials
…have found a difference in admission
rate [12,1...
A single citation can support a
whole paragraph
• Dutton and colleagues [8] described a series
of 81 coagulopathic trauma ...
Discussing treatments, outcomes,
other authors’ conclusions
• Dutton and colleagues [8] described a series
of 81 coagulopa...
Sometimes several parallel
paragraphs.
• Dutton and colleagues [8] described a
series of 81 …patients treated with rFVIIa”...
Multiple citations in a paragraph
• “Berger et al. [42] compared the efficacy of
hypertonic saline and mannitol to reduce ...
Avoiding a 1-sentence paragraph?
• “Berger et al. [42] compared the efficacy of
hypertonic saline and mannitol to reduce I...
Rhetorical moves and audience considerations in the discussion sections of randomized controlled trials of health interven...
Upcoming SlideShare
Loading in …5
×

Rhetorical moves and audience considerations in the discussion sections of randomized controlled trials of health interventions--ECA--2017-06-22

396 views

Published on

European Conference on Argumentation talk

Jodi Schneider, Graciela Rosemblat, Shabnam Tafreshi and Halil Kilicoglu “Rhetorical moves and audience considerations in the discussion sections of Randomized Controlled Trials of health interventions” [Conference Panel Presentation], 2nd European Conference on Argumentation: Argumentation and Inference, Fribourg, Switzerland, June 20-23

1 of 3 talks in Jodi Schneider and Sally Jackson, organizers, “Innovations in Reasoning and Arguing about Health ”[Conference Panel], 2nd European Conference on Argumentation: Argumentation and Inference, Fribourg, Switzerland, June 20-23.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Rhetorical moves and audience considerations in the discussion sections of randomized controlled trials of health interventions--ECA--2017-06-22

  1. 1. Rhetorical Moves and Audience Considerations in the Discussion Sections of Randomized Controlled Trials of Health Interventions Jodi Schneider (University of Illinois at Urbana Champaign, USA) jschneider@pobox.com Graciela Rosemblat (National Library of Medicine, USA) Shabnam Tafreshi (The George Washington University, USA) Halil Kilicoglu (National Library of Medicine, USA) European Conference on Argumentation, Fribourg, Switzerland 2017-06-22
  2. 2. Disclaimer The views and opinions expressed do not necessarily state or reflect those of the U.S. Government, and they may not be used for advertising or product endorsement purposes.
  3. 3. Why study scientific papers? • “The interpretive face of science is characterized by controversy, disagreement, dissensus.” (Prelli, “A Rhetoric of Science”, 1989, p4) • “Scientific publications are documentary representations of defeasible arguments, supported by data and repeatable methods.” (Clark, Ciccarese, Goble 2014) • Prelli, Lawrence J. "A rhetoric of science: Inventing scientific discourse." (1991). • Clark, Tim, Paolo N. Ciccarese, and Carole A. Goble. "Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications." Journal of biomedical semantics 5.1 (2014): 28.
  4. 4. Why study medical papers? • Large potential impact on human health • Input to evidence-based practice • Well-elaborated study of evidence Figure credit: Duke University Medical Center Library. Introduction to Evidence-based Practice. What is Evidence-Based Practice (EBP)? http://guides.mclibrary.duke.edu/c.php?g=158201&p=1036021
  5. 5. Hierachy of Evidence
  6. 6. Hierachy of Evidence
  7. 7. Randomized Controlled Trial (RCT) • Medical RCTs test the efficacy of medical treatments. • Features: – Control group (placebo or standard treatment) – Random assignment – Neither doctors nor patients know which group is which (“double blind”) Image credit: Ancker, Jessica S & Quynh Pham. "Beyond the RCT Evaluating innovations in the Learning Health System” AMIA 2016 Tutorial
  8. 8. Corpus • 37 research articles reporting on medical trials, mainly RCTs • Full-text available from PubMed Central • Chosen for an ongoing citation annotation study using Xu et al.’s corpus Xu, Jun, Yaoyun Zhang, Yonghui Wu, Jingqi Wang, Xiao Dong, and Hua Xu. "Citation sentiment analysis in clinical trial papers." In AMIA Annual Symposium Proceedings, vol. 2015, p. 1334. American Medical Informatics Association, 2015.
  9. 9. Other ongoing study on this corpus • Understand citation structure of Discussion sections of Randomized Clinical Trials • Annotate two aspects of citations: – Citation scope – Citation matter • Began summer 2016 at U.S. National Library of Medicine, led by Halil Kilicoglu, with Graciela Rosemblat leading annotation. • Ongoing study in biomedical informatics
  10. 10. Corpus characteristics • Various conditions, e.g. – Brain injury – Degenerative eye disease – Malaria – Postpartum back pain • Various settings, e.g. – Critical care – Community/Ambulatory care – Internet-based treatment – Laboratory treatment – School-based treatment
  11. 11. Corpus characteristics • Various locales, e.g. – Madagascar – Malawi – Netherlands – Uganda – United Kingdom – United States • Single site and Multisite • Industry-sponsored and Researcher-focused • Various stages of clinical trial
  12. 12. Focus: 37 discussion sections http://europepmc.org/ articles/PMC1420299
  13. 13. Rhetorical Moves in RCT Discussion Sections
  14. 14. Rhetorical Moves in RCT Discussion Sections • Novelty • Gaps/opportunities • Contextualization • Comparability to previous results • Validity of methods • Anticipating objections • Pointers to ongoing work • Acknowledging limitations • Future work • Summaries
  15. 15. Novelty • “We report the first safety data collected in the setting of a randomized, controlled study for patients sustaining TBI with polytrauma.” (2206502) • “This is the first double blind prospective study designed to evaluate the effects of fruit juice feedings during diarrheal disease in young children.” (1217327)
  16. 16. Gaps/opportunities • "At the start of this study, little was known about the specific effect of psychosocial determinants in SCs, although interventions aiming to modify these determinants, such as cognitive behavioural programs were considered to be promising for musculoskeletal pain in general [4-8]. We developed the EAP to fill a gap in UC, which focuses mainly on biomedical determinants." (2211478) • "We have already studied the stimulation of periodontal tissue regeneration by FGF-2 in animal models and believe that the protein represents a major candidate for a periodontal tissue-regenerating agent." (2432040)
  17. 17. Contextualization • “This study is part of a series to explore effective xylitol delivery vehicles that can be used in school programs in the U.S. Results from the xylitol dose study [3] showed...” (2527560) • “In general we did not deviate from the original study design, which was described independently of the study results [15].” (1420299) • "The first report on this study showed that the intra- operative use of TA significantly decreased the incidence of intra-operative complications associated with PPV, and no adverse events related to surgery were found over a 3-month observation period [13]." (2413124)
  18. 18. Comparability to previous results • "However, none of these 11 studies were performed on pediatric populations, making further inferences or comparisons problematic" (1876598) • "This is in line with previous studies of internet-driven CBT for posttraumatic stress reactions [9], complicated grief [30] and CBT interventions in face-to-face studies [10,31]." (1885249)
  19. 19. Validity of methods "Algometry is used to measure the sensitivity of pain or pressure [4]. Algometer instrumentation can include manual and electric models. The use of any experimental instrument including the PGA must be tested for validity and reliability between examiners and between performances of the same examiner. The PGA used in this study has been tested against itself, palpation, pressure plates with reliable results [17,18]. However, the above is based on the assumption that the tester is trained in the application of the PGA otherwise issues associated with the rate of pressure application [45- 47] the determination of an end point based on a verbal patient response [45,46] and the possible sensitisation of a selected landmark based on repeated measures may all alter the accuracy of the outcome." (2427032)
  20. 20. Validity of methods "The technique of using the consumption of morphine during PCA treatment of postoperative pain, as a measure of the effect of the analgesic regime under study, has been used in several other studies of this kind [5,6]." (1637100)
  21. 21. Anticipating objections "It might be argued that these observations of blood stage immunity (reduction in growth rates, presence of crisis forms etc.) may rather be related to the potential pre-erythrocytic action of PEV3A. However, in previous challenge studies, the number of parasites emerging from the liver of unvaccinated control volunteers has been shown to vary as much as five fold [7]. Despite this variation, rates of parasite growth in these volunteers were similar. Equally, crisis forms have never been observed historically in our studies of pre-erythrocytic vaccine candidates. Indeed, following their observation in this study, the same slide reader went back to examine a selection of blood films from vaccinated and control volunteers in two previous studies where some evidence of pre-erythrocytic efficacy has been observed (VAC021 [28] and VAC023 [29]). The slide reader was blinded to the group allocation of the volunteers. In total 72 slides were selected for re- examination (6 slides each from 6 volunteers from each study, 3 vaccinees plus 3 controls) and no crisis forms were observed. It seems likely therefore, that the differences observed here are indeed related to vaccine induced blood stage immunity." (2204057)
  22. 22. Pointers to ongoing work • "An ongoing phase III study is likely to provide a better evaluation of safety in this patient population." (2206502) • Further analysis of the 18 months follow-up data and the examination of other potentially relevant moderators such as posttraumatic growth [Maercker & Knaevelsrud, in preparation] will hopefully enhance our understanding of online therapeutic processes." (188529)
  23. 23. Acknowledging limitations • "Among the limitations of this study is the screening strategy for the recruitment of the patients. We deliberately handled strict exclusion criteria for participation in this study. We excluded 72% (n = 253) of the patients who wanted treatment but did not meet the inclusion criteria. This might limit the generalizability of our results.” (1885249)
  24. 24. Future work • "New studies should be carried out, with longer follow-up times and larger samples, to evaluate the effects of NPPV on the quality of life of patients on weaning ventilation support and to understand how the cause of ARF could affect the results of different weaning ventilation methods. Cost evaluation should also be included in these studies." (2447605) • "Future studies directed on possible central mechanisms of this complicated chronic pain syndrome are warranted."(1815612)
  25. 25. Summaries • "Based on present evidence BTX cannot be recommended as treatment for neck pain in chronic whiplash patients."(1815612) • "The evidence from this study is that Hall PMCs allow a reliable, low-maintenance seal to be achieved by GDPs." (2265270) • "Therefore, the mouthwash clearly demonstrated an anti-malodor effect on morning breath potential without any measurable side effects in healthy subjects." (2637235)
  26. 26. Rhetorical Moves in RCT Discussion Sections • Novelty • Gaps/opportunities • Contextualization • Comparability to previous results • Validity of methods • Anticipating objections • Pointers to ongoing work • Acknowledging limitations • Future work • Summaries
  27. 27. Audience
  28. 28. Multiple audiences “Audiences of specialists…do not comprise the only audiences who respond to scientific or other professional discourse. There may be a number of interested audiences for any particular discourse.” (Prelli,1989, p26)
  29. 29. Interested audiences include: • Specialists responsible for “authorizing the claims to knowledge” (Prelli,1989, p26) • Non-specialists who might “judge the work as interested witnesses” especially if there are “broad social implications” (Prelli,1989, p27)
  30. 30. Fulfilling audience expectations “To succeed rhetorically, any rhetor must ground a position in what the addressed audience considers reasonable”(Prelli,1989, p27-8) “…[draw] on standard or conventional patterns of thinking that will be employed by one’s specific audience”(Prelli,1989, p37) “…choose purposes and contents likely to be deemed pertinent and conventionally appropriate by the specifically situated thought groups that comprise the audience.”(Prelli,1989, p38)
  31. 31. Audience-specific sections from discussion sections of reports of Randomized Controlled Trials
  32. 32. Researchers are the expected audience for RCTs Mellis, Craig. "Evidence-based Medicine: What Has Happened in the Past 50 Years?" J Paediatr Child Health 51, no. 1 (2015): doi:10.1111/jpc.12800.
  33. 33. Audiences we find • Specialist researchers in the same area • Clinicians • Clinical practice managers • Grant funders • Health systems
  34. 34. Specialist researchers in the same area • “The most plausible explanation for the similar trends in BV reduction in both arms of this study is the inherent activity of the placebo formulation.” (1851729)
  35. 35. Clinicians • “caution should be exercised in applying these findings to the general management of dental caries without being aware of potential pit falls.”(2265270) • “none of the results of this particular clinical trial suggest any clinical problems concerning the safety of administering FGF-2 to patients with periodontitis. One more piece of evidence supporting the high safety of FGF-2 applied locally to periodontal tissue is that this therapy has already been used for more than 5 years as a remedy for intractable ulcers (Fiblast spray).” (2432040)
  36. 36. Clinical practice managers • “Our findings are particularly pertinent to smaller clinical practices that might not be aligned with larger organizations, and who will have to choose EMR systems and DSSs on their own.”(1876598)
  37. 37. Grant funders • “Additional research is needed to determine whether these improvements translate into even longer-term reductions of health-related costs. Although the sample was heterogeneous on demographic factors such as income, the group was relatively ethnically and racially homogenous, consisting primarily of white women. Future replications with diverse populations are indicated to further evaluate the promise and translation potential of the MLP.” (1783667)
  38. 38. Grant funders "Admittedly, overall complexity of the issue addressed by the present study clearly merits pursuit of further studies of a closely similar, multi-factorial intervention design, though preferably conducted over much longer time span, as well as best targeting the over 60s as the population standing by far the best chance of a successful outcome, in order to verify whether the present findings, clearly encouraging as they appear, might actually have sufficient potential to develop into a trend of indisputable clinical significance, especially in terms of possible application in a comprehensively designed, nationwide programme specifically aimed at addressing the problem of a steadily growing proportion of the frail seniors dependent for their activities of daily living." (2637855)
  39. 39. Health systems • “Given the rising costs and growing burden of diabetes on the health-care system [57], strategies that aim to prevent or delay the onset of this disease might be beneficial [58]” (1865086) • “AQ+AS (0.51 US$) [28] is currently four times more expensive than AQ+SP (0.13 US$). In Madagascar, with a reported 2,114,400 cases of suspected malaria (2003), the use of AQ+SP instead of AQ+AS would reduce the annual antimalarial treatments costs by 800,000 US$. In view of the recent report by Tagbor and colleagues [29] there is hope that AQ+SP could serve as a safe and effective alternative for malaria treatment in pregnancy and for intermittent preventive treatment until the safety of ACTs for pregnant women has been established.” (1887535)
  40. 40. Audiences we find • Specialist researchers in the same area • Clinicians • Clinical practice managers • Grant funders • Health systems
  41. 41. Rhetorical Moves in RCT Discussion Sections • Novelty • Gaps/opportunities • Contextualization • Comparability to previous results • Validity of methods • Anticipating objections • Pointers to ongoing work • Acknowledging limitations • Future work • Summaries
  42. 42. Future work • Systematically code rhetorical moves and audiences – likely to lead to revisions • Compare to previous work in English for Specific Purposes for medical and general scientific work • Identify key suggestions for writing effective RCT reports, to reach their intended audiences. • Develop Argumentation Mining algorithms to automatically detect rhetorical moves. – Could be used to automatically extract discussion topics for evidence synthesis of RCTs. – Could be used to give authors fast, automatic feedback about readability.
  43. 43. Citation examples from discussion sections of reports of Randomized Controlled Trials
  44. 44. “This” work agrees with… • “This is in accordance with earlier studies in the ambulatory surgical setting [3]” - PMC1637100
  45. 45. Definitions and background info • “Self-efficacy, which may relate to motivation, is the perceived confidence in one's ability to accomplish a specific task [19].” - PMC2194735
  46. 46. Presenting a range of evidence • “Except in one study [20], short-term administration of GH transiently worsened insulin resistance [19,53] and increased fasting glucose levels [53].” - PMC1865086
  47. 47. Clause-level changes in meaning • “Two of four randomised clinical trials …have found a difference in admission rate [12,19] and two have not [22,23].” - PMC1142326
  48. 48. A single citation can support a whole paragraph • Dutton and colleagues [8] described a series of 81 coagulopathic trauma patients treated with rFVIIa. Of these, 20 received rFVIIa for treatment of coagulopathy related to TBI. Six of these patients had additional polytrauma. The outcome of these patients was poor and 15 of 20 patients died. The authors attributed this high mortality rate to the severity of brain injury. None of the 81 trauma patients in this series had any clinical indication of TE events.”
  49. 49. Discussing treatments, outcomes, other authors’ conclusions • Dutton and colleagues [8] described a series of 81 coagulopathic trauma patients treated with rFVIIa. Of these, 20 received rFVIIa for treatment of coagulopathy related to TBI. Six of these patients had additional polytrauma. The outcome of these patients was poor and 15 of 20 patients died. The authors attributed this high mortality rate to the severity of brain injury. None of the 81 trauma patients in this series had any clinical indication of TE events.”
  50. 50. Sometimes several parallel paragraphs. • Dutton and colleagues [8] described a series of 81 …patients treated with rFVIIa” • “Zaaroor and Bar-Lavie [23] reported the first series of five patients …” • “Morenski and colleagues [24] described …three pediatric … cases”
  51. 51. Multiple citations in a paragraph • “Berger et al. [42] compared the efficacy of hypertonic saline and mannitol to reduce ICP after a combination of two different neuronal injuries. Initially, ….The authors demonstrated that …After …. It is remarkable that … An accumulation …These different effects … [42]. Furthermore, Prough et al. observed a higher regional cerebral blood flow in dogs with induced intracerebral hemorrhage after hypertonic saline without any increase of the CPP [43].” - PMC1297608
  52. 52. Avoiding a 1-sentence paragraph? • “Berger et al. [42] compared the efficacy of hypertonic saline and mannitol to reduce ICP after a combination of two different neuronal injuries. Initially, ….The authors demonstrated that …After …. It is remarkable that … An accumulation …These different effects … [42]. Furthermore, Prough et al. observed a higher regional cerebral blood flow in dogs with induced intracerebral hemorrhage after hypertonic saline without any increase of the CPP [43].” - PMC1297608

×