The document discusses the critical importance of monitoring bibliographies by publishers, emphasizing the repercussions of citing retracted research papers. It highlights how faulty citations can mislead and harm public health, using examples of retracted clinical trials and the ongoing citations they receive. Key recommendations include creating databases for retractions and citations, and ensuring authors and editors are alerted about retracted works to prevent misuse of flawed literature.

1. TroubleattheAcademy:
ProblemCitations
Jodi Schneider
@jschneider
jschneider@pobox.com
ISSA
CrossRef LIVE18
Toronto, Canada
2018-11-13

2. My metadata hats
• Librarian
• Teacher of future librarians &
information managers
• Researcher
scholarly communication, science of
science, ontologies…
• Journal editor/founder
co-founder of The Code4Lib Journal
• Metadata user

3. Bibliographies are important.

4. Bibliographies are important.

5. Publishers:
Take an active role in
monitoring and tracking
bibliographies.

6. “[Y]ou can transform a fact into fiction or
a fiction into fact just by adding or
subtracting references”
- Bruno Latour

7. Science builds on the past
By Biochem1 (Own work) CC BY-SA 3.0 via Wikimedia Commons https://commons.wikimedia.org/wiki/File:‫.جنگا_ست_یک‬JPG

8. Science builds on the past
By Biochem1 (Own work) CC BY-SA 3.0 via Wikimedia Commons https://commons.wikimedia.org/wiki/File:‫.جنگا_ست_یک‬JPG
{{Existing “facts”}} + {{New “facts”}}

9. Science builds on the past
By Biochem1 (Own work) CC BY-SA 3.0 via Wikimedia Commons https://commons.wikimedia.org/wiki/File:‫.جنگا_ست_یک‬JPG
{{Existing “facts”}} + {{New “facts”}}
{{Stuff you cite}}

10. “Once upon a time…scientists…were
builders who constructed edifices, called
explanations or laws, by assembling bricks
called facts. When the bricks were sound
and were assembled properly, the edifice
was useful and durable and brought
pleasure, and sometimes reward, to the
builder. If the bricks were faulty or if they
were assembled badly, the edifice would
crumble, and this kind of disaster could
be very dangerous to innocent users of
the edifice as well as to the builder, who
sometimes was destroyed by the bricks.”
- Bernard K. Forscher

11. Faulty Bricks: Retracted Articles
• Articles are retracted due to:
• Error
• Plagiarism
• Fraud
• Duplicate publication
• Failure to replicate
• Ethics
• Author dispute…
• Reasons for retraction are often deliberately
vague or unstated!

12. Faulty Bricks: Not always
retracted
United States Office of Research Integrity
findings of fraud:
• Only 1/3 retract and mention
misconduct
• 1/3 retract but claim error, loss of data,
replication failure
• About 1/3 of are not followed by any
retraction or correction of tainted
studies.Resnik DB, Dinse GE. Scientific retractions and corrections related to misconduct findings. J Med Ethics. 2013 Jan;39(1):46-50.

13. Retracted research treated as
“bricks”
• New citations continue for years after
retraction. (Korpela 2010)
• May take 2 years or more for retraction
to occur. (Redman et al 2008)
• Citations are generally positive and
ignore known problems with the article.
(Bar-Ilan, J., & Halevi, G. 2017).
• Citations are generally positive not only
in Introduction but also in Methods,
Results, Discussion sections. (Van Der Vet & Nijveen 2016)

14. Citing faulty bricks harms people
A paper about a clinical trial for renal disease was retracted because:
“‘the trial had not been approved by the ethics committee, the involvement of a
statistician could not be verified, [and] the trial was not a double-blind study,
because Dr Nakao knew the treatment allocation’.”
“Nevertheless, the COOPERATE study was cited by 173 review articles
and 58 secondary clinical studies that enrolled a total of 35,929
patients.”
“The harm done by COOPERATE is thus 4-fold:
• patients were enrolled in an experimental therapy for a condition which
already had an accepted therapy;
• time, energy and money were wasted by patients and investigators;
• false information pervaded the literature;
• and combination therapy was accepted more quickly and used more widely
than it might have been otherwise.”
Steen, R. G. (2011). Retractions in the medical literature: how many patients are put at risk by flawed
research?. Journal of Medical Ethics, 37(11), 688-692.

15. https://publicationethics.org/files/retraction%20guidelines_0.pdf

16. Why should publishers care about
retraction?
• “Science is self-correcting” ?!
• Harm to Public Health – and moral liability
– Wakefield’s 1998 report in Lancet associated MMR
vaccination with autism. Firestorm of public
skepticism ever since. Partial retraction in 2004,
but only fully retracted in 2010, 12 years after
publication!
– Journal editors asked: “What do we still know?”
after prominent cases in anesthesiology affected
patient care
• Opportunities to show value: stewarding,
caretaking bibliographies

17. Heavily cited retracted papers
Retraction Watch, from Top 10 most highly cited retracted papers
Article
Year of
retraction
Citing
Articles
before
retraction
Citing Articles
after
retraction
1. Primary Prevention of
Cardiovascular Disease with
a Mediterranean Diet.
N Engl J Med, 2013
2018 1792 79
2. Visfatin: A protein
secreted by visceral fat that
mimics the effects of
insulin. SCIENCE, 2005
2007 224 977
3. Ileal-lymphoid-nodular
hyperplasia, non-specific
colitis, and pervasive
developmental disorder in
children. LANCET, 1998
2010 647 512

18. 134 citations to a retracted clinical
trial with faked data
Schneider & Ye in preparation,
updating Fulton et al 2015

19. 2715 citations of citations
to clinical trial with faked data
Schneider & Ye in preparation,
updating Fulton et al 2015

20. What is needed to address
citation of retracted literature?
• A database of retractions
• A database of citations
• Alert authors/editors before they finalize a
bibliography:
– Do you know this paper is retracted?
• Don’t cite, or mention the retraction
• When a paper is newly retracted:
– Notify authors/editors who already cited it

21. Database of Retractions:
Retraction Watch Database

22. Database of Retractions:
Why not Crossref?
• Deposit your metadata!
• Only 169 Crossref members have ever
registered retractions. Are YOUR
retractions registered?
https://www.crossref.org/dashboard/

23. Works well with Crossmark – but just
depositing the metadata helps!

24. Database of Citations: Open
Citations

25. Database of Citations:
Are you contributing?
https://www.crossref.org/reference-distribution/

26. “Participation reports give a clear picture for anyone to see the metadata Crossref
has”
https://www.crossref.org/participation/
You can check whether your favourite publisher release open references – and, if
it does not, please write it an email asking to release them!
https://www.crossref.org/members/prep/
Slide Credit: Silvio Peroni
Database of Citations:
Are you contributing? Check Crossref
Participation Reports

27. If you are a publisher that already submits article metadata to Crossref as a
participant in their Cited-by​ service, opening your reference data can be achieved
in a matter of days, either:
1. by contacting Crossref by e-mail (support@crossref.org), asking them to turn
on reference distribution for all of the relevant DOI prefixes, or
2. by setting the <reference_distribution_opt> metadata element to
"any" for each DOI deposit
If not already a participant in Cited-by, a Crossref member can register for this
service free-of-charge.
Slide Credit: Silvio Peroni
Database of Citations: Set
YOUR citations to OPEN

28. Make metadata a priority, not an
afterthought!
• Join the Metadata 2020 conversations
• Talk about metadata quality internally
• Use Crossref participation reports data
for leverage with your vendors.

29. Publishers:
Take an active role in
monitoring and tracking
bibliographies.

30. Monitor and track bibliographies
(1) Clearly mark retracted papers.
(2) Before publishing an article, check its
bibliography for retracted papers.
(3) Alert authors who have already cited a
retracted paper.

31. Bar-Ilan J, Halevi G. Post retraction citations in context: a case study.
Scientometrics. 2017 Oct 1;113(1):547-65.
Forscher BK. Chaos in the brickyard. Science. 1963 Oct 18;142(3590):339.
Fulton AS, Coates AM, Williams MT, Howe PR, Hill AM. Persistent
citation of the only published randomised controlled trial of omega-3
supplementation in chronic obstructive pulmonary disease six years
after its retraction. Publications. 2015 Feb 11;3(1):17-26.
Korpela KM. How long does it take for the scientific literature to purge
itself of fraudulent material?: the Breuning case revisited. Curr Med Res
Opin. 2010;26(4):843-7.
Redman B K, Yarandi HN , and Merz JF. Empirical Developments in
Retraction."J Med Ethics 2008; 34(11), 807-809.
Resnik DB, Dinse GE. Scientific retractions and corrections related to
misconduct findings. J Med Ethics. 2013 Jan;39(1):46-50.
Steen RG. Retractions in the medical literature: how many patients are put
at risk by flawed research?. J Med Ethics. 2011 Jan 1:jme-2011.
Van Der Vet PE, Nijveen H. Propagation of errors in citation networks: a
study involving the entire citation network of a widely cited paper
published in, and later retracted from, the journal Nature. Res Integr
Peer Rev. 2016 Dec;1(1):3.

32. Retraction Detector
Schneider & Kansara
Input reference strings
Visfatin: A protein secreted by visceral fat that mimics the
effects of insulin. SCIENCE, JAN 21 2005
Renear, A. H., & Palmer, C. L. (2009). Strategic reading,
ontologies, and the future of scientific publishing. Science,
325(5942), 828-832.
Outputs retracted papers (in PubMed)