The importance of measurements of uncertainty is explained and illustrated with the help of a famous experiment in nutrition from 1930 and a complex cross-over trial from the 1990s,

1. What is the point of
point estimates?
Stephen Senn
Consultant Statistician
Edinburgh
(C)
Stephen
Senn
2022
1
“Superficially, point estimation may
seem a simpler problem to discuss
than that of interval estimation”
Cox and Hinckley, p250

2. Apologies
• This was originally submitted to the
Communicating Statistical Methods session
• But it was not accepted 
• Due to a cancellation from the Cluster Trials
session I am able to present it here
• I have departed from the abstract somewhat
to introduce a cluster trial example
• By doing this I shall annoy two sets of people
• Those who wanted to hear what was in the
abstract
• Those who want to hear about up-to-date
research on cluster trials
(C) Stephen Senn 2022 2

3. Being befuddled about balance
• A matter that often befuddles amateur (non-statistical) commentaries
on randomised studies is the role of balance
• If factors are balanced, adjusting for them (in the linear case) won’t
change the estimate so what’s the point?
• However, the statistical point of view is more or less the opposite.
• If you have balanced for a factor you ought to have the factor in the
model
• We don’t analyse a matched pairs design like a completely randomised one
• Why?
(C) Stephen Senn 2022 3

4. As we all know
• The reason is to do with the standard error
• Balancing by a factor eliminates its effect on the estimate
• But it does not, of its own, eliminate its effect on the standard error
• In fact it increases it slightly
• To estimate how well you have done, you need to remove its effect on
the standard error
• This requires putting it in the model
• We all know this but one still regularly encounters arguments that
show we have forgotten
• In my opinion this is because we obsess about point estimates
(C) Stephen Senn 2022 4

5. An incomplete block design in asthma
Two formulations of formoterol given at different doses
• Details need not concern us
• Some patients received
• ISF24 and MTA6
• Some patients received
• ISF24 but not MTA6
• Some patients received
• MTA6 but not ISF24
• I can calculate both within subject
and between subject estimates of
the effect ISF24-MTA6
• This is not the way I would analyse
these data but I could (and have)
(C) Stephen Senn 2022 5
Data from Senn et
al, 1997

6. More or Less
More Less
(C) Stephen Senn 2022 6

7. The Lanarkshire Milk Experiment
An incomplete block cluster allocated trial from 1930
• 67 Schools enrolled
• In some schools children were
allocated 1:1 either to act as
controls or receive raw milk
• In other schools children were
allocated 1:1 either to act as
controls or receive pasteurised milk
• Just over 18,000 schoolchildren
studied
• Four months of treatment
(C) Stephen Senn 2022 7

8. Analysis
What the authors did
• Analysed data on weight and height
• Pooled all controls
• Not a good idea
• But analysed each of 14 = 2 sex by 7
age groups separately
• Calculated ‘probable errors’
• But it is not clear how
• Concluded that milk was beneficial
but that one could not decide
whether raw or pasteurised was
better
(C) Stephen Senn 2022 8
Prominent commentators
Fisher Student

9. What I have done
• Developed algebraic expressions for variances of contrasts for six different
types of analysis
• Four that might be reasonable
• Two that the authors carried out
• For each of the six analyses there are two variances
• The true variance
• What one might naively estimate
• Simulated to check my formulae
• Everything looks fine
• This means
• Either it is fine
• Or I made the same mistakes in my simulation as in the theory
(C) Stephen Senn 2022 9

10. What I have not done
• I have not got hold of the original data
• In fact we don’t even know how many schools of each type there
were
• Therefore I have had to speculate (postulate, guess, fake) what the
variance components might be
• The key issue as all those interested in cluster allocate trials know is
the ratio of the between to the within cluster variance
• I don’t like ICCs
(C) Stephen Senn 2022 10

11. Milking some
data
Parameter settings are
identical for all six cases
Heights in inches are
considered
The formulae are for a
given sex by age group and
n=10 per group per school
33 schools per milk type
in a & b the school effect is
fixed
in e & f the school effect is
random
in c & d the school effect is
ignored

12. Sacred cow
The TARGET study
Target study
• One of the largest studies ever run in
osteoarthritis
• 18,000 patients
• Randomisation took place in two sub-
studies of equal size
• Lumiracoxib versus ibuprofen
• Lumiracoxib versus naproxen
• Practical considerations dictated design
• Purpose to investigate CV and GI
tolerability of lumiracoxib
• Sub-study effect explicitly dealt with in
analysis
Lanarkshire Milk Study
• At the time one of the largest nutritional
studies
• 18,000 school children
• Randomisation took place in two sub-
studies of equal size
• No milk versus raw milk
• No milk versus pasteurised milk
• Practical considerations dictated design
• Purpose to investigate effect of milk on
height and weight
• Sub-study effect ignored in analysis
(C) Stephen Senn 2022 12
(c) Stephen Senn 2012

13. Baseline Demographics
Sub-Study 1 Sub Study 2
Demographic
Characteristic
Lumiracoxib
n = 4376
Ibuprofen
n = 4397
Lumiracoxib
n = 4741
Naproxen
n = 4730
Use of low-dose
aspirin
975 (22.3) 966 (22.0) 1195 (25.1) 1193 (25.2)
History of
vascular disease
393 (9.0) 340 (7.7) 588 (12.4) 559 (11.8)
Cerebro-
vascular disease
69 (1.6) 65 (1.5) 108 (2.3) 107 (2.3)
Dyslipidaemias 1030 (23.5) 1025 (23.3) 799 (16.9) 809 (17.1)
Nitrate use 105 (2.4) 79 (1.8) 181 (3.8) 165 (3.5)
(C) Stephen Senn 2022 13
If this looks familiar it ought to.
Bergen 2012

14. Baseline Deviances
Model Term
Demographic
Characteristic
Sub-study
(DF=1)
Treatment
given Sub-
study
(DF=2)
Treatment
(DF=2)
Use of low-dose
aspirin
23.57 0.13 13.40
History of
vascular disease
70.14 5.23 47.41
Cerebro-
vascular disease
13.54 0.14 7.75
Dyslipidaemias 117.98 0.17 54.72
Nitrate use 39.83 4.62 29.17
(C) Stephen Senn 2022 14

15. Some final words
• The very impressive and interesting causal
inference school seems to be promoting
identifiability rather than estimability
• The former considers whether what happens
asymptotically is correct
• It is not always obvious what has to go to infinity
for an asymptote to be reached
• Schools/centres or pupils/patients
• I think we are in danger of loosing a valuable
insight from experimental design theory
• How treatments are varied across the block
structure matters
• The lessons of the Rothamsted School should
be heeded
(C) Stephen Senn 2022 15
stephen@senns.uk
@stephensenn
http://www.senns.uk/Blogs.html
“..the calculation of standard errors is
idle and misleading, if the method of
arrangement adopted fails to guarantee
their validity…”
RA Fisher, The Design of Experiments
section 34
Fisher Yates Nelder