The document discusses the importance of ensuring independence in experimental design, detailing how to choose and analyze samples appropriately to avoid bias and confounding variables. It provides examples from human, animal, and cell culture studies about how to properly assign and justify experimental units while considering relationships between subjects. Additionally, it emphasizes the need for randomization in group assignments and controlling for position and order effects to enhance the validity of results.

1. Identifying independence and
assigning samples to groups
I declare
independence!
Not quite Conro!
Experimental
design: part4

2. At the end of the
experiment you will
perform statistical
analyses on your data
And you will plot
graphs showing your
results But what
constitutes a
data point?
The key
considerations is
independence here
How related are the
samples to one
another? Where are
the sources of
variability
Before we can decide
how many samples to
collect, we need to decide
what those samples are

3. Let’s start with
human studies
The biggest source of
variability is between
individuals
Most of the time you
would consider each
individual person as a
separate data point
If you take multiple measurements of
the same thing in the same person
then those data would make you
more confident about the value you
got for that individual but wouldn’t
tell you more about the population

4. You will use the mean
or median of the
multiple
measurements in your
final statistical analysis
But ultimately 1 person
= 1 data point

5. For example: if you had a pair of
twins in your study, you might
expect them to respond to your
treatment in similar ways. It
might be inappropriate to
consider them as indepedent
data points
The challenge in relation to
independence comes when
there are connections
between the participants
Same idea with people
who live together, go to the
same school, work
together or whatever!

6. There aren’t one size
fits all rules here. You
have to be the one
that decides
Consider what the
harshest critic of your
work would say
You have to ask, what
connections are
relevant to your
study?
Whatever you decide,
you will have to justify
the decision

7. Let’s move on to
animal studies
Mice, rats, cows etc
are social animals,
they are housed
together
Your treatment,
whatever it is, may
cause other animals
that share an
enclosure to respond
Squeak Squeak
Squeak
Squeak

8. If the differences in
response between cages
are not the same as the
differences in response
for animals within the
same cage
Then your
experimental unit
should be the cage
Note that the within
cage effects happen a
lot, it is safer to
assume that it will for
your study!

9. In this set up if all the
brown mice got
treatment 1, and the
white mice treatment
2
The sample size would be
4, for the 4 cages, rather
than 8 for the 8 mice
getting treated
Isn’t that a waste
of animals?

10. Isn’t that a waste
of animals?
Yes, it could be!
It could also be a waste of
time, money and resources.
This is why you should think
about these things before
your experiment!
It’s ethically
unforgivable for poor
experimental design to
waste experimental
units

11. One more example:
let’s look at cell
culture-type
experiments
A standard sort of
experimental set up might
involve taking one flask of cells
and splitting it into multiple
wells of a single multi-well dish

12. Treatments would then be
applied to the different
wells in the dish
Although you would get 6
readings per coloured
treatment, each reading
isn’t independent
They’ve been cultured
together, treated
together, exposed to the
same environmental
conditions etc
The six values will be
conflated to 1 per
treatment in your
final analysis

13. The six replicates here
would all contribute to
the one data point
So why would
you do 6 at all?
If you made a mistake in
any one well, or got some
other spurious result, it
would have a smaller
impact on the overall
findings
In this sort of set up,
people often talk about
“biological” or
“experimental” repeats”
and “technical repeats”

14. The biological repeat
refers to the whole
experiment: 1 flask to one
set of data
Technical repeat are the
data from the individual
wells or equivalent (mice
of the same cage)

15. What one researcher defines
as a “biological” repeat may
not be the same as another
person doing the same
experiment
So, it is important that you
can justify your choice and
when you write up your work
you make it clear what you
mean
For example: one stock
flask of cells split into 3
sub flasks and each sub-
flask gives one 24 well
plate
For some experiments you
might justify saying this is 3
biological repeats, but for
others it would just count as 1.
The answer depends on where
the variability is.

16. Let’s talk about one more
situation where identifying
potential for problems of
replicates not actually being
independent
Specifically we are going to
look at taking
measurements through
time
If you are considering an
experiment where you take the
same samples and treat twice
you need to be sure that the
first treatment doesn’t affect
the second measurement

17. A
+
Let’s look at the same
experiment we discussed
before:
Reading 1
Time (washout
period)
B
+ Reading 2
You really don’t
want there to be
an order effect
Ideally your pilot
data will let you
know how long to
leave between
treatments
However, if you can’t
remove the effect of the
first treatment, then you
might not be able to use
paired analysis

18. Randomisation and assigning
groups
I see an issue with
your group
assignments
Leeburt’s group Conro’s group
What? You told me
to be random
And all the young
women ended up in
your study group? yeah, pure
coincidence

19. You should be able to
defend your group choices
to your harshest critic
Let’s look at ways you can
go about assigning
experimental units to
different treatment groups

20. Fully Randomised
Probably the most obvious way
is to completely randomise
which group you assign your
experimental units to
A random number generator
should be used rather than
manually trying this, as humans
will always introduce patterns
subconsciously

21. Fully Randomised
Why wouldn’t you
fully randomise?
Although full randomisation
removes researcher bias it
doesn’t allow you any chance
to control for confounding
variables

22. Stratified,
then randomised
A common approach is to
split your population into
groups and then randomly
assign sub-groups afterward

23. Stratified,
then randomised
For example, we might
consider gender to be a
confounding variable and so
we split the group first based
on gender
Doing this will help
ensure that the gender
balance is roughly equal
Then assign groups
from the sub
populations

24. Stratified,
then randomised
It might be appropriate to
stratify on age as well as
gender
If you had 2 study populations
from this cohort. Then each would
have 1 x older women, 3 or 4
older men, 7 younger men and 2
or 3 younger women

25. Stratified,
then randomised
How much stratification you do
depends on the overall study
population size and what you think
matters to the interpretation
Stratification should be
prioritised to control for the
confounder that will have the
biggest effect

26. Non-equal groups
Usually you will be able to
use stronger statistical tests
when your group sizes are
the same
However, there may be practical
or ethical reasons why you might
not want to have uneven groups
So, try to balance in terms of
numbers whenever you can
Can you think of any
reason why?
Treatment Group Control Group

27. Non-equal groups
Treatment Group
Control Group
If the treatment
might cause harmful
side-effects
Yes, unequal groups may
not be ideal in terms of
stats but they can be the
correct thing to do ethically
Less animals / humans will
suffer harm and you will still
be able to answer your
question

28. Position and order effects
It’s not just who is in which
group that matters
For example, surgeons will get
better at a surgical technique
the more they perform it
Or might be better at the
beginning of their shift
compared with the end
You also need to consider
the experimenter involved
in the research

29. Position and order effects
You may also need to control for
which scientist is doing the
research.
This is particularly true wherever
interaction with the participants
is involved such as in collecting
patient details
Or if there is a
subjective
measurement
involved

30. Position and order effects
Position and order effects are
also relevant in lab work
Setting up your plate like
this might make it easier
to set up
But you won’t know if the
effect is due to the treatment
or whether it’s he location or
order matter

31. Position and order effects
Experiment 1
Experiment 2
Experiment 3
In addition, to randomisation
within experiments, the set up
should be different between
each experiment

32. Part 4 Recap.
Make sure your samples are truly independent
Randomize everything you can, every time you can.
Position, order, experimenter.
Use groupings as a way to remove the effect of confounding
variables from your interpretation

33. Videos
Don’t worry Liam, the
next video will make the
process much easier!
Umar John Liam
Conro