European Conference on Argumentation talk Jodi Schneider, Graciela Rosemblat, Shabnam Tafreshi and Halil Kilicoglu “Rhetorical moves and audience considerations in the discussion sections of Randomized Controlled Trials of health interventions” [Conference Panel Presentation], 2nd European Conference on Argumentation: Argumentation and Inference, Fribourg, Switzerland, June 20-23 1 of 3 talks in Jodi Schneider and Sally Jackson, organizers, “Innovations in Reasoning and Arguing about Health ”[Conference Panel], 2nd European Conference on Argumentation: Argumentation and Inference, Fribourg, Switzerland, June 20-23.

1. Rhetorical Moves and Audience
Considerations in the Discussion
Sections of Randomized Controlled Trials
of Health Interventions
Jodi Schneider (University of Illinois at Urbana Champaign, USA)
jschneider@pobox.com
Graciela Rosemblat (National Library of Medicine, USA)
Shabnam Tafreshi (The George Washington University, USA)
Halil Kilicoglu (National Library of Medicine, USA)
European Conference on Argumentation, Fribourg, Switzerland
2017-06-22

2. Disclaimer
The views and opinions expressed do not
necessarily state or reflect those of the U.S.
Government, and they may not be used for
advertising or product endorsement
purposes.

3. Why study scientific papers?
• “The interpretive face of science is
characterized by controversy,
disagreement, dissensus.” (Prelli, “A
Rhetoric of Science”, 1989, p4)
• “Scientific publications are documentary
representations of defeasible arguments,
supported by data and repeatable
methods.” (Clark, Ciccarese, Goble 2014)
• Prelli, Lawrence J. "A rhetoric of science: Inventing scientific discourse." (1991).
• Clark, Tim, Paolo N. Ciccarese, and Carole A. Goble. "Micropublications: a
semantic model for claims, evidence, arguments and annotations in biomedical
communications." Journal of biomedical semantics 5.1 (2014): 28.

4. Why study medical papers?
• Large potential impact on human health
• Input to evidence-based practice
• Well-elaborated study of evidence
Figure credit: Duke University Medical Center Library. Introduction to
Evidence-based Practice. What is Evidence-Based Practice (EBP)?
http://guides.mclibrary.duke.edu/c.php?g=158201&p=1036021

5. Hierachy of Evidence

6. Hierachy of Evidence

7. Randomized Controlled Trial
(RCT)
• Medical RCTs test the
efficacy of medical
treatments.
• Features:
– Control group (placebo or
standard treatment)
– Random assignment
– Neither doctors nor patients
know which group is which
(“double blind”)
Image credit: Ancker, Jessica S & Quynh Pham. "Beyond the RCT
Evaluating innovations in the Learning Health System”
AMIA 2016 Tutorial

8. Corpus
• 37 research articles reporting on medical
trials, mainly RCTs
• Full-text available from PubMed Central
• Chosen for an ongoing citation annotation
study using Xu et al.’s corpus
Xu, Jun, Yaoyun Zhang, Yonghui Wu, Jingqi Wang, Xiao Dong, and Hua Xu.
"Citation sentiment analysis in clinical trial papers." In AMIA Annual
Symposium Proceedings, vol. 2015, p. 1334. American Medical
Informatics Association, 2015.

9. Other ongoing study on this corpus
• Understand citation structure of Discussion
sections of Randomized Clinical Trials
• Annotate two aspects of citations:
– Citation scope
– Citation matter
• Began summer 2016 at U.S. National Library
of Medicine, led by Halil Kilicoglu, with
Graciela Rosemblat leading annotation.
• Ongoing study in biomedical informatics

10. Corpus characteristics
• Various conditions, e.g.
– Brain injury
– Degenerative eye disease
– Malaria
– Postpartum back pain
• Various settings, e.g.
– Critical care
– Community/Ambulatory care
– Internet-based treatment
– Laboratory treatment
– School-based treatment

11. Corpus characteristics
• Various locales, e.g.
– Madagascar
– Malawi
– Netherlands
– Uganda
– United Kingdom
– United States
• Single site and Multisite
• Industry-sponsored and Researcher-focused
• Various stages of clinical trial

12. Focus: 37 discussion sections
http://europepmc.org/
articles/PMC1420299

13. Rhetorical Moves in RCT
Discussion Sections

14. Rhetorical Moves in RCT
Discussion Sections
• Novelty
• Gaps/opportunities
• Contextualization
• Comparability to previous results
• Validity of methods
• Anticipating objections
• Pointers to ongoing work
• Acknowledging limitations
• Future work
• Summaries

15. Novelty
• “We report the first safety data collected
in the setting of a randomized, controlled
study for patients sustaining TBI with
polytrauma.” (2206502)
• “This is the first double blind
prospective study designed to evaluate
the effects of fruit juice feedings during
diarrheal disease in young children.”
(1217327)

16. Gaps/opportunities
• "At the start of this study, little was known about
the specific effect of psychosocial determinants in
SCs, although interventions aiming to modify these
determinants, such as cognitive behavioural programs
were considered to be promising for musculoskeletal
pain in general [4-8]. We developed the EAP to fill a
gap in UC, which focuses mainly on biomedical
determinants." (2211478)
• "We have already studied the stimulation of
periodontal tissue regeneration by FGF-2 in animal
models and believe that the protein represents a
major candidate for a periodontal tissue-regenerating
agent." (2432040)

17. Contextualization
• “This study is part of a series to explore effective xylitol
delivery vehicles that can be used in school programs
in the U.S. Results from the xylitol dose study [3]
showed...” (2527560)
• “In general we did not deviate from the original study
design, which was described independently of the
study results [15].” (1420299)
• "The first report on this study showed that the intra-
operative use of TA significantly decreased the
incidence of intra-operative complications associated
with PPV, and no adverse events related to surgery
were found over a 3-month observation period [13]."
(2413124)

18. Comparability to previous results
• "However, none of these 11 studies were
performed on pediatric populations, making
further inferences or comparisons
problematic" (1876598)
• "This is in line with previous studies of
internet-driven CBT for posttraumatic stress
reactions [9], complicated grief [30] and CBT
interventions in face-to-face studies [10,31]."
(1885249)

19. Validity of methods
"Algometry is used to measure the sensitivity of pain or pressure [4].
Algometer instrumentation can include manual and electric models.
The use of any experimental instrument including the PGA must
be tested for validity and reliability between examiners and
between performances of the same examiner. The PGA used in
this study has been tested against itself, palpation, pressure
plates with reliable results [17,18]. However, the above is based on
the assumption that the tester is trained in the application of the PGA
otherwise issues associated with the rate of pressure application [45-
47] the determination of an end point based on a verbal patient
response [45,46] and the possible sensitisation of a selected landmark
based on repeated measures may all alter the accuracy of the
outcome." (2427032)

20. Validity of methods
"The technique of using the
consumption of morphine during
PCA treatment of postoperative
pain, as a measure of the effect
of the analgesic regime under
study, has been used in several
other studies of this kind [5,6]."
(1637100)

21. Anticipating objections
"It might be argued that these observations of blood
stage immunity (reduction in growth rates, presence
of crisis forms etc.) may rather be related to the
potential pre-erythrocytic action of PEV3A. However, in
previous challenge studies, the number of parasites emerging from the liver of unvaccinated
control volunteers has been shown to vary as much as five fold [7]. Despite this variation, rates
of parasite growth in these volunteers were similar. Equally, crisis forms have never been
observed historically in our studies of pre-erythrocytic vaccine candidates. Indeed, following
their observation in this study, the same slide reader went back to examine a selection of blood
films from vaccinated and control volunteers in two previous studies where some evidence of
pre-erythrocytic efficacy has been observed (VAC021 [28] and VAC023 [29]). The slide reader
was blinded to the group allocation of the volunteers. In total 72 slides were selected for re-
examination (6 slides each from 6 volunteers from each study, 3 vaccinees plus 3 controls) and
no crisis forms were observed. It seems likely therefore, that the
differences observed here are indeed related to
vaccine induced blood stage immunity." (2204057)

22. Pointers to ongoing work
• "An ongoing phase III study is likely to
provide a better evaluation of safety in this
patient population." (2206502)
• Further analysis of the 18 months follow-up
data and the examination of other potentially
relevant moderators such as posttraumatic
growth [Maercker & Knaevelsrud, in
preparation] will hopefully enhance our
understanding of online therapeutic
processes." (188529)

23. Acknowledging limitations
• "Among the limitations of this study is the
screening strategy for the recruitment of
the patients. We deliberately handled strict
exclusion criteria for participation in this
study. We excluded 72% (n = 253) of the
patients who wanted treatment but did not
meet the inclusion criteria. This might limit
the generalizability of our results.”
(1885249)

24. Future work
• "New studies should be carried out, with
longer follow-up times and larger
samples, to evaluate the effects of NPPV on the quality of life of
patients on weaning ventilation support and to understand how the
cause of ARF could affect the results of different weaning ventilation
methods. Cost evaluation should also be
included in these studies." (2447605)
• "Future studies directed on possible central
mechanisms of this complicated chronic pain
syndrome are warranted."(1815612)

25. Summaries
• "Based on present evidence BTX cannot be
recommended as treatment for neck pain in
chronic whiplash patients."(1815612)
• "The evidence from this study is that Hall PMCs
allow a reliable, low-maintenance seal to be
achieved by GDPs." (2265270)
• "Therefore, the mouthwash clearly demonstrated
an anti-malodor effect on morning breath potential
without any measurable side effects in healthy
subjects." (2637235)

26. Rhetorical Moves in RCT
Discussion Sections
• Novelty
• Gaps/opportunities
• Contextualization
• Comparability to previous results
• Validity of methods
• Anticipating objections
• Pointers to ongoing work
• Acknowledging limitations
• Future work
• Summaries

27. Audience

28. Multiple audiences
“Audiences of specialists…do not comprise
the only audiences who respond to scientific
or other professional discourse. There may
be a number of interested audiences for any
particular discourse.” (Prelli,1989, p26)

29. Interested audiences include:
• Specialists responsible for “authorizing the
claims to knowledge” (Prelli,1989, p26)
• Non-specialists who might “judge the work
as interested witnesses” especially if there
are “broad social implications” (Prelli,1989,
p27)

30. Fulfilling audience expectations
“To succeed rhetorically, any rhetor must ground a
position in what the addressed audience considers
reasonable”(Prelli,1989, p27-8)
“…[draw] on standard or conventional patterns of
thinking that will be employed by one’s specific
audience”(Prelli,1989, p37)
“…choose purposes and contents likely to be
deemed pertinent and conventionally appropriate
by the specifically situated thought groups that
comprise the audience.”(Prelli,1989, p38)

31. Audience-specific sections from
discussion sections of reports of
Randomized Controlled Trials

32. Researchers are the expected
audience for RCTs
Mellis, Craig. "Evidence-based Medicine: What Has Happened in the
Past 50 Years?" J Paediatr Child Health 51, no. 1 (2015):
doi:10.1111/jpc.12800.

33. Audiences we find
• Specialist researchers in the same area
• Clinicians
• Clinical practice managers
• Grant funders
• Health systems

34. Specialist researchers in the same
area
• “The most plausible explanation for the
similar trends in BV reduction in both arms
of this study is the inherent activity of the
placebo formulation.” (1851729)

35. Clinicians
• “caution should be exercised in applying these findings
to the general management of dental caries without
being aware of potential pit falls.”(2265270)
• “none of the results of this particular clinical trial suggest
any clinical problems concerning the safety of
administering FGF-2 to patients with periodontitis. One
more piece of evidence supporting the high safety of
FGF-2 applied locally to periodontal tissue is that this
therapy has already been used for more than 5 years as
a remedy for intractable ulcers (Fiblast spray).”
(2432040)

36. Clinical practice managers
• “Our findings are particularly pertinent
to smaller clinical practices that might
not be aligned with larger organizations,
and who will have to choose EMR systems
and DSSs on their own.”(1876598)

37. Grant funders
• “Additional research is needed to
determine whether these improvements
translate into even longer-term reductions
of health-related costs. Although the
sample was heterogeneous on demographic
factors such as income, the group was
relatively ethnically and racially homogenous,
consisting primarily of white women. Future
replications with diverse populations are
indicated to further evaluate the promise and
translation potential of the MLP.” (1783667)

38. Grant funders
"Admittedly, overall complexity of the issue addressed by the
present study clearly merits pursuit of further studies of a closely
similar, multi-factorial intervention design, though preferably
conducted over much longer time span, as well as best targeting
the over 60s as the population standing by far the best chance of
a successful outcome, in order to verify whether the present
findings, clearly encouraging as they appear, might actually
have sufficient potential to develop into a trend of
indisputable clinical significance, especially in terms of
possible application in a comprehensively designed,
nationwide programme specifically aimed at addressing the
problem of a steadily growing proportion of the frail seniors
dependent for their activities of daily living." (2637855)

39. Health systems
• “Given the rising costs and growing burden of diabetes on the
health-care system [57], strategies that aim to prevent or
delay the onset of this disease might be beneficial [58]”
(1865086)
• “AQ+AS (0.51 US$) [28] is currently four times more
expensive than AQ+SP (0.13 US$). In Madagascar, with a
reported 2,114,400 cases of suspected malaria (2003), the
use of AQ+SP instead of AQ+AS would reduce the annual
antimalarial treatments costs by 800,000 US$. In view of
the recent report by Tagbor and colleagues [29] there is hope
that AQ+SP could serve as a safe and effective alternative for
malaria treatment in pregnancy and for intermittent preventive
treatment until the safety of ACTs for pregnant women has
been established.” (1887535)

40. Audiences we find
• Specialist researchers in the same area
• Clinicians
• Clinical practice managers
• Grant funders
• Health systems

41. Rhetorical Moves in RCT
Discussion Sections
• Novelty
• Gaps/opportunities
• Contextualization
• Comparability to previous results
• Validity of methods
• Anticipating objections
• Pointers to ongoing work
• Acknowledging limitations
• Future work
• Summaries

42. Future work
• Systematically code rhetorical moves and
audiences – likely to lead to revisions
• Compare to previous work in English for Specific
Purposes for medical and general scientific work
• Identify key suggestions for writing effective RCT
reports, to reach their intended audiences.
• Develop Argumentation Mining algorithms to
automatically detect rhetorical moves.
– Could be used to automatically extract discussion
topics for evidence synthesis of RCTs.
– Could be used to give authors fast, automatic
feedback about readability.

44. Citation examples from
discussion sections of reports of
Randomized Controlled Trials

45. “This” work agrees with…
• “This is in accordance with earlier studies
in the ambulatory surgical setting [3]” -
PMC1637100

46. Definitions and background info
• “Self-efficacy, which may relate to
motivation, is the perceived confidence in
one's ability to accomplish a specific task
[19].” - PMC2194735

47. Presenting a range of evidence
• “Except in one study [20], short-term
administration of GH transiently worsened
insulin resistance [19,53] and increased
fasting glucose levels [53].” - PMC1865086

48. Clause-level changes in meaning
• “Two of four randomised clinical trials
…have found a difference in admission
rate [12,19] and two have not [22,23].” -
PMC1142326

49. A single citation can support a
whole paragraph
• Dutton and colleagues [8] described a series
of 81 coagulopathic trauma patients treated
with rFVIIa. Of these, 20 received rFVIIa for
treatment of coagulopathy related to TBI. Six
of these patients had additional polytrauma.
The outcome of these patients was poor and
15 of 20 patients died. The authors attributed
this high mortality rate to the severity of brain
injury. None of the 81 trauma patients in this
series had any clinical indication of TE
events.”

50. Discussing treatments, outcomes,
other authors’ conclusions
• Dutton and colleagues [8] described a series
of 81 coagulopathic trauma patients treated
with rFVIIa. Of these, 20 received rFVIIa for
treatment of coagulopathy related to TBI. Six
of these patients had additional polytrauma.
The outcome of these patients was poor and
15 of 20 patients died. The authors
attributed this high mortality rate to the
severity of brain injury. None of the 81 trauma
patients in this series had any clinical
indication of TE events.”

51. Sometimes several parallel
paragraphs.
• Dutton and colleagues [8] described a
series of 81 …patients treated with rFVIIa”
• “Zaaroor and Bar-Lavie [23] reported the
first series of five patients …”
• “Morenski and colleagues [24] described
…three pediatric … cases”

52. Multiple citations in a paragraph
• “Berger et al. [42] compared the efficacy of
hypertonic saline and mannitol to reduce ICP after
a combination of two different neuronal injuries.
Initially, ….The authors demonstrated that …After
…. It is remarkable that … An accumulation
…These different effects … [42]. Furthermore,
Prough et al. observed a higher regional cerebral
blood flow in dogs with induced intracerebral
hemorrhage after hypertonic saline without any
increase of the CPP [43].” - PMC1297608

53. Avoiding a 1-sentence paragraph?
• “Berger et al. [42] compared the efficacy of
hypertonic saline and mannitol to reduce ICP after
a combination of two different neuronal injuries.
Initially, ….The authors demonstrated that …After
…. It is remarkable that … An accumulation
…These different effects … [42]. Furthermore,
Prough et al. observed a higher regional cerebral
blood flow in dogs with induced intracerebral
hemorrhage after hypertonic saline without any
increase of the CPP [43].” - PMC1297608