Dr. Michael Bull examines the implications of air pollution on health, particularly through the lens of the six cities study, which demonstrated a correlation between fine particle exposure and increased mortality. He critiques the assumptions behind air quality measurement and model verification, arguing that inherent variability and biases compromise their accuracy. Ultimately, he questions whether the established scientific beliefs adequately represent causation in the context of air pollution and its effects.

1. What do we really know?
Dr Michael Bull

2. Examination of three AQ related topics under a slightly philosophical lens
• Does air pollution really harm our health?
• What does an air pollution measurement tell us? And
• Is model verification an appropriate approach?

3. Does air pollution really harm our health?

4. The six cities study
• Observational study across six
different cities in the USA
• Examined mortality over ~16
year period from 1974
• Correlated rates of mortality
with exposure to air pollutants
• Credited with being the catalyst
for new air quality standards in
the USA – particularly for long
term exposure

5. Concluded exposure to fine particles increased risk of mortality

6. Six cities study – An Observational Study
• Observational study is where you compare two or more population samples and
attempt to show the effect of one single variable on a particular outcome.
• Idea is that you attempt to select the samples so that every other potential cause
of the outcome is the same in each sample and only the study factor allowed to
vary.
𝑻𝒐𝒕𝒂𝒍 𝑹𝒊𝒔𝒌 = 𝑹𝒊𝒔𝒌𝑨𝒊𝒓 𝑷𝒐𝒍𝒍𝒖𝒕𝒊𝒐𝒏𝟏 + σ𝟎
𝑵
𝑹𝒊𝒔𝒌 𝑨𝒍𝒍 𝑶𝒕𝒉𝒆𝒓 𝑪𝒂𝒖𝒔𝒆𝒔
𝑻𝒐𝒕𝒂𝒍 𝑹𝒊𝒔𝒌 = 𝑹𝒊𝒔𝒌𝑨𝒊𝒓 𝑷𝒐𝒍𝒍𝒖𝒕𝒊𝒐𝒏𝟐 + σ𝟎
𝑵
𝑹𝒊𝒔𝒌 𝑨𝒍𝒍 𝑶𝒕𝒉𝒆𝒓 𝑪𝒂𝒖𝒔𝒆𝒔

7. Six cities study – An Observational Study
• Observational study is where you compare two or more population samples and
attempt to show the effect of one single variable on a particular outcome.
• Idea is that you attempt to select the samples so that every other potential cause of the
outcome is the same in each sample and only the study factor allowed to vary.
𝑻𝒐𝒕𝒂𝒍 𝑹𝒊𝒔𝒌 = 𝑹𝒊𝒔𝒌𝑨𝒊𝒓 𝑷𝒐𝒍𝒍𝒖𝒕𝒊𝒐𝒏𝟏 + σ𝟎
𝑵
𝑹𝒊𝒔𝒌 𝑨𝒍𝒍 𝑶𝒕𝒉𝒆𝒓 𝑪𝒂𝒖𝒔𝒆𝒔
𝑻𝒐𝒕𝒂𝒍 𝑹𝒊𝒔𝒌 = 𝑹𝒊𝒔𝒌𝑨𝒊𝒓 𝑷𝒐𝒍𝒍𝒖𝒕𝒊𝒐𝒏𝟐 + σ𝟎
𝑵
𝑹𝒊𝒔𝒌 𝑨𝒍𝒍 𝑶𝒕𝒉𝒆𝒓 𝑪𝒂𝒖𝒔𝒆𝒔
Must be the same

8. Six cities study – An Observational Study
• Observational study is where you compare two or more population samples and
attempt to show the effect of one single variable on a particular outcome.
• Idea is that you attempt to select the samples so that every other potential cause of the
outcome is the same in each sample and only the study factor allowed to vary.
𝑻𝒐𝒕𝒂𝒍 𝑹𝒊𝒔𝒌 = 𝑹𝒊𝒔𝒌𝑨𝒊𝒓 𝑷𝒐𝒍𝒍𝒖𝒕𝒊𝒐𝒏𝟏 + σ𝟎
𝑵
𝑹𝒊𝒔𝒌 𝑨𝒍𝒍 𝑶𝒕𝒉𝒆𝒓 𝑪𝒂𝒖𝒔𝒆𝒔
𝑻𝒐𝒕𝒂𝒍 𝑹𝒊𝒔𝒌 = 𝑹𝒊𝒔𝒌𝑨𝒊𝒓 𝑷𝒐𝒍𝒍𝒖𝒕𝒊𝒐𝒏𝟐 + σ𝟎
𝑵
𝑹𝒊𝒔𝒌 𝑨𝒍𝒍 𝑶𝒕𝒉𝒆𝒓 𝑪𝒂𝒖𝒔𝒆𝒔
Also assumes these have been
assessed in the same manner

9. Potential Issues
• The six cities are very diverse – Portage population ~11,000 and 25km2
compared with St Louis population, 171 km2, >2 million (images from Google maps)

10. Potential issues
• Exposure assessed using a single pollution monitor in each city
• Large measured differences in life expectancy nationwide
• Socioeconomic factors a large contributor and there are large differences in
the economic status of each city
• The experimental population was not representative of the general
population
• Is the observed causality a representation of the strengths of the beliefs of
scientists rather than a true feature of the world
• Correlation is not causation and are we measuring the “cause”

11. The Problem of Common Cause

12. Correlates well with NO2 exposure (and all measured
combustion related pollutants)

13. Selective population …..

14. Selective population …..

15. Ultimately we can never “know”

16. Air quality monitoring
• Use diffusion tubes as the more extreme example although similar (but
potentially smaller uncertainties remain with more detailed measurements)
• Have stated accuracy ±20% - not found definitive explanation of this but
I’ve seen this as the RSD is 20% - around 90% of measurements within two
standard deviations.

17. Say true concentration is 30µg/m3 – what could a diffusion
tube measure?

18. So what does a measurement of 30µg/m3 mean?

19. Are we attempting to impose order on an inherently chaotic
or random phenomena ?
• We attempt to make our measurements more accurate using bias
adjustment, but does this really make our answer more accurate or
certain?
• For this to work, (a) a systematic laboratory error must exist and (b) this
must dominant all other causes
• But – we know that the method is not exact and hence there will always be
variability from the actual value
𝑻𝒐𝒕𝒂𝒍 𝑬𝒓𝒓𝒐𝒓 = 𝑬𝒓𝒓𝒐𝒓𝑳𝒂𝒃 + σ𝟎
𝑵
𝑳𝒂𝒃 𝑨𝒍𝒍 𝑶𝒕𝒉𝒆𝒓 𝑪𝒂𝒖𝒔𝒆𝒔

20. Normal distribution with 20% RSD

21. Is the reality that bias adjustment simply a reflection of the inherent error?

22. Is there a visible lab based error?
Look at one single laboratory result

23. • Examine individual results from a labs – there’s no apparent “lab related”
error
• You’ll also find that bias adjustment makes error worse in around 30% of
cases and generally only helps for the remainder is because diffusion tubes
appear to overestimate concentrations (and hence any adjustment under
1.0 will make it better)
• Are we simply applying an adjustment factor that is based on the random
and uncertain nature of the measurement method?
• Do we really know what the concentration is?!

24. Model verification

25. Causes of model error
• Input data errors – traffic, emission data, meteorology
• NOx to NO2 conversion
• Monitoring data errors
• Local site characteristics
• Local site weather conditions
• Background concentrations
• Model performance when predicting close to the source
• Model errors

26. Is model verification another example of imposing “order”
on a chaotic system?
• We attempt to make our modelling more accurate using model adjustment,
but does this really make our answer more accurate or certain?
• Model verification imposes a linear correction on a system that is clearly
more likely to be non-linear. There is no reason to believe this is correct.
• Model verification will work even if you use a random number generator
instead of a model and removes all authority from the model itself.
• Allows gross errors to be hidden and air quality specialists to claim they
have an accurate modelling approach.
𝑻𝒐𝒕𝒂𝒍 𝑬𝒓𝒓𝒐𝒓 = σ𝟎
𝑵
𝑬𝒓𝒓𝒐𝒓 𝑨𝒍𝒍 𝑪𝒂𝒖𝒔𝒆𝒔

27. Are experts right or are they
members of expert groups that
simply believe the same things?

28. Back to the Rhino

29. Michael Bull
Michael Bull and Associates Ltd
16 Mount Pleasant Road,
Tunbridge Wells,
TN1 1QU
mb@michaelbullassociates.com
michael.bull@odourconsultant.co.uk