The RSS has published a report tackling statistical bias in criminal trials where healthcare professionals are accused of murdering patients. Following several high-profile cases where statistical evidence has been misused, the Society calls for all parties in such cases to consult with professional statisticians and use only expert witnesses who are appropriately qualified. The report, ‘Healthcare serial killer or coincidence?’, is produced by the RSS’s Statistics and the Law Section. The group evolved from a working group of the same name set up in early 2000s after the Society wrote to the Lord Chancellor and made a statement setting out concerns around the use of statistical evidence in the case of Sally Clark. According to the report, suspicions about medical murder often arise due to a surprising or unexpected series of events, such as an unusual number of deaths among patients under the care of a particular professional. The RSS has major concerns about use of this kind of evidence in a criminal investigation: first, over the analysis and interpretation of such data, and secondly over whether it can be guaranteed that the data have been compiled in an objective and unbiased manner.

1. Richard D. Gill, Leiden University, 23 October 2023
A tale of two Lucy’s
Statistical issues in the investigation of suspected
serial killer nurses
Left: Lucia de Berk reading letters, (c) Carole Edrich
Right: Lucy Letby with friends at Coldplay concert at the Etihad Stadium Manchester, 10 June 2012 (?)
1

2. 4, 4, 9, 8, 4, 2
What set this off
(among other things)
2

3. What convinced the jurors: item 1 (*)
(*) Of course, we can’t know. But three “smoking guns” convinced media and public
3

4. What convinced the jurors: item 2
4

5. Item 2, a reconstruction Mycroft on 'X'
@ChrisJClarkEsq
“This is the breakdown of
the infamous `green
confession’. It is quite
clearly nothing of the sort.
I have broken down every
word (except those that are
indecipherable). The order
in which to read the note is
by colour.”
5

6. What convinced the jurors: item 3
Insulin ~ C-peptide ratio, babies F and L
Preparation:
Concentrations can be measured in many different units
1 mole = 6.02214076 × 10^23
molecules.
p (pico) = 10^-12; n (nano) = 10^-9;
µ (micro) = 10^-6; m (milli) = 10^-3
Hence: pmol, nmol, µmol, mmol
1 international unit, 1 “U” or “IU” or
“SU”, was originally defined as the
amount of insulin required to cause
convulsive hypoglycaemia in a
fasted 2kg rabbit
Insulin is a hormone which controls your blood sugar level.
Hypo-, hyper-glycaemia: too little, too much glucose in your blood
6

7. Baby F: a test showed that
the insulin concentration was
4657/pmol/L, the C-peptide
was 169 pmol/L and the
glucose was 1.3 mmol/L.
Within two hours of the blood
test the baby’s blood sugar
level was ~4 mmol/L, which is
normal
Baby L: similar, but less
extreme
http://pathlabs.rlbuht.nhs.uk/insulin.pdf
http://pathlabs.rlbuht.nhs.uk/seafrm.htm )
What convinced the jurors:
item 3
Insulin ~ C-peptide ratio,
babies F and L
7

8. • 2015, 2016: conflicts at CoCH between management, medical
consultants, and nurses
• Everyone had noticed alarming rise in death rate in NICU
• Incidents seemed often to happen in Lucy Letby’s shifts
• The RCPCH was charged with an investigation
• Late 2016, confidential report submitted
• Early 2017, redacted report made public
• May 2017, Hospital asks Cheshire Constabulary to investigate cause of
deaths
Obviously, not. The trial lasted 10 months
Was that all? (1/4)
8

9. • May 2017, police announce investigation, newspapers report
• June 2017, retired paediatrician Dewi Evans turns up at Cheshire
police station offering his services
• Police give him medical notes of 32 incidents, half of which were
deaths, the other half collapses with successful resuscitation
• Many deaths and collapses concern same infants and many concern
members of twins, triplets; many were very premature
• 2018, Lucy Letby taken in for questioning and then released
• 2019, Lucy Letby taken in for questioning and then released
• 2020, Lucy Letby arrested [TV vans in the street before police arrive]
Obviously, not. The trial lasted 10 months
Was that all? (2/4)
9

10. • 32 charges, of which 8 murders and the rest attempted
murder
• October 2022, trial starts
• June 2023, trial ends, jury retires to consider verdict
• July and August: jury returns verdicts; judge announces
sentence
• Verdict: 7 murders and 15 murder attempts
• Sentence: full life
Obviously, not. The trial lasted 10 months
Was that all? (3/4)
10

11. • A mountain of evidence was brought in by the prosecution
• All of it is circumstantial
• Lucy was never seen to do anything wrong
• Every event has a natural explanation
• The medical evidence says babies F and L were not
administered a large unauthorised amount of insulin
• The rise in number of deaths has natural explanations
• Some events are hard to explain (no proper investigation
done at the time, no samples retained)
Obviously, not. The trial lasted 10 months
Was that all? (4/4)
11

12. • Police believe more babies were murdered or harmed by Lucy in earlier
years, and a new investigation is proceeding, studying medical records
of another 1000 babies
• Prosecution has appealed against not guilty verdict for baby K;
trial scheduled for July 2024. Reporting restrictions have been imposed
• A public enquiry (the Thirlwall enquiry) will take place to find out why
hospital directors stopped medical consultants from going to the police
• Lucy has asked for an appeal (she has a right to ask, but it may not be
granted). Three judges will decide
• A trial is being prepared for the crime of “corporate manslaughter”
(suspect = CoCH management)
The prosecution and the public want more
What next?
12

13. • The 32 incidents were compiled by senior medical consultants (“gang of 4”)
• They include all 16 deaths from Jan 2015 to June 2016, exactly the period that
Lucy Letby worked on the ward, fully qualified
• They include 16 collapses in same period, all when Lucy was on duty
• Dewi Evans found something odd in most events which happened when Lucy
was there and in each case, he fantasized a murderous attack which might have
caused that anomaly. He claims that he was the one who told the police to look
the nursing roster and that he was the one who spotted Lucy’s presence at event
after event. Police claim they had expected he would find evidence for some bug
• He did not spot the anomalous insulin results & did not have babies F or L in his
list. Dr Brearey found them in 2018
• Doctors Brearey, Gibbs, Jayaram, and NN (CoCH), and Evans (retired) seem all to
have lied. There is evidence of connections between the gang of four and Evans.
Reconstruction by an amateur (Mark Mayes)
What happened in 2017?
13

14. •The defence did not find any experts
•Neither prosecution nor defence used statistical or
epidemiological evidence
•The single defence witness was the plumber
•The defence agreed that the insulin evidence meant there was a
murderer on the ward
•The great British public has never been so uneducated
The defence; the plumber; the tabloids; social media;
the jury system; changes to the rules of the last 20 years
How could it go so wrong?
14

15. • A few brave souls are starting to speak out
• UK media labels them as crackpots and conspiracy theorists
• At least four qualified scientists run “pro Lucy” websites arguing
the trial was unfair
• Something will happen (CPS appeal / Lucy appeal / Enquiry /
CPS “corporate manslaughter” charge / earlier deaths
investigation)
• I think that the tide is turning
Who knows
What will happen next?
15

16. • Lucia: management and doctors worked together; Lucy: they were
fighting
• Adversarial vs. inquisitorial system; case law vs. codified law;
difference in culture; time (20 yrs apart); media & social media
• Lucia’s defence did their best to defend her; Lucy’s seemed not too
• UK: political issues around NHS; systematic NHS failings; social
inequality; poor education; systematic erosion of right to a fair trial
over several decennia
• In both cases, failures of top medical doctors plus failures of
management caused unnecessary loss of lives, a notable nurse got
the blame
• No objective definition of “unexpected, unexplained, collapse”
My answer: yes and no. The same but far worse.
And is either case only about statistics, only about medicine, or more complex?
Is Lucy’s case essentially the same as Lucia’s?
16

17. • Overdispersion (the cluster includes two triplets)
• Public misunderstanding and distrust of statistical science
• Misunderstanding and distrust within science
• The magical power of numbers
• Lies, damned lies, and statistics
• Police investigate crimes. Doctors cure patients. Psychiatrists
associate criminal behaviour with psychiatric disorders.
• In the hospital situation, people die “normally” all the time.
See the report of the Royal Statistical Society!
Appendix: statistical issues (1/6)
https://rss.org.uk/news-publication/news-publications/2022/section-group-reports/rss-publishes-report-on-dealing-with-uncertainty-i/
17

18. • The UK used to have the best forensic science institute in the world but
it was disbanded for political reasons (privatization frenzy)
• Instead, CoCH allowed doctors to control a police investigation
• There should have been a comparison of collapses when Lucy was
present with those when she was not present
• There should have been a comparison of the death rate when Lucy was
present with the death rate when other similar nurses were present
• Advanced statistical technology allows taking account of confounding
factors (and this is rocket science, not just common sense. But it should
be common sense that it is necessary!)
RCPCH recommended a forensic investigation of those
few cases which they did not understand
Appendix: statistical issues (2/6)
https://rss.org.uk/news-publication/news-publications/2022/section-group-reports/rss-publishes-report-on-dealing-with-uncertainty-i/
18

19. Suspicions about medical murder often arise due to a surprising or unexpected series of
events, such as an unusual number of deaths among patients under the care of a
particular professional.
The RSS has major concerns about use of this kind of evidence in a criminal
investigation: first, over the analysis and interpretation of such data, and secondly over
whether it can be guaranteed that the data have been compiled in an objective and
unbiased manner.
When interpreting such data, investigators need to consider:
A: Could the deaths have occurred for reasons other than murder?
B: If murder was the cause, is the person under suspicion responsible?
The report also calls for more care to be taken by experts to avoid drawing erroneous
inferences from such data, by properly controlling for plausible causal factors.
The RSS’s concerns about the compilation of the data used in such investigations are
that attention is rarely given to ensuring that unconscious bias has not influenced the
selection of cases. Such innocent cognitive biases are prevalent throughout society and
control of these needs active steps such as blinding. For medical misconduct cases, the
report recommends that investigations should be supervised by expert panels
independent of both the suspect and their employer.
We have seen cases across the world of people wrongfully charged based on incorrect
statistical analysis. We’re calling for better collaboration between the legal and statistical
communities to prevent such miscarriages of justice happening in the future.
19

20. Healthcare serial killer or coincidence?
Statistical issues in investigation of
suspected medical misconduct
Summary Report
by the RSS Statistics and the Law Section
September 2022
Healthcare serial killer or coincidence?
Statistical issues in investigation of suspected
medical misconduct
by the RSS Statistics and the Law Section
September 2022
64 pp.
15 pp.
READ IT!
Read me!
20

21. Recommendation 4: Investigations should be guided by panels representing
all relevant areas of expertise but independent of both the suspect and the
employing institution.
Recommendation 5: To the maximum extent practicable, experts informing an
investigation, such as DNA specialists,
fi
ngerprint examiners, toxicologists,
and pathologists should be kept “blind” to all aspects of the case irrelevant to
the question they are being asked to answer. Blinding is a key tool in
minimising prejudicial subjective e
ff
ects such as unconscious bias.
Recommendation 6: It is vital that investigators appreciate the truth of the
well-known aphorism “correlation is not causation”, and the fact that the
connection between them is well-studied, and that in
fi
elds such as medical
diagnosis there are accepted criteria to guide the valid drawing of conclusions
in observational studies. Possible confounding factors must be identi
fi
ed, and
their e
ff
ect quanti
fi
ed, before attributing causes to observed e
ff
ects.
Recommendation 7: When courts must evaluate the results of problematic
investigations, it is particularly important that they consider reports and expert
testimony from independent statisticians. If investigative bias is a signi
fi
cant
concern, lawyers and courts should also consider seeking evaluations from
experts of cognitive bias and factors associated with the accuracy of expert
judgment. 21

22. https://www.science.org/content/article/unlucky-numbers-fighting-murder-convictions-rest-shoddy-stats
Unlucky numbers
22

23. A murderous nurse at work—
or just coincidence?
Cognitive biases can easily
lead an investigation astray
and have drastic effects on
how suspicious a cluster of
deaths seems. In this
imaginary example drawn from
real-world errors, a doctor
reports that many deaths
seem to occur while Nurse X is
on duty. The hospital launches
an investigation, reexamining
deaths at Nurse X’s ward over
the past 2 years. A simple
statistical test* compares the
rate of suspicious deaths when
Nurse X is on duty with the
rate when she is off. It then
calculates the probability of
seeing this pattern purely as a
result of random chance.**
The outcome depends greatly
on the type of investigation. 23

24. • Science on Trial (Sarrita Adams)
• Chimpinvestor blog (Peter Elston)
• Law, health, technology substack (Scott McLachlan)
• My collaborator, Engel Wichmann (my blog, and our “Foundation”,
Science 4 Justice - NL)
• Mark Mayes’ YouTube films
• Mycroft on 'X' @ChrisJClarkEsq; The Secret Solicitor
@AlanBra15552677
• So many other great people on Twitter and Facebook ♥ ♥ ♥ ♥ ♥
So much great work by so many people!
Resources
24