Statistics
for Journalists

      Adrian MacLeod
 adrian@writethinking.co.uk
Percentage starting points

Wheat in millions of tonnes
             Produce    Consume
China           99.7       115.4
E...
Percentage starting points

      Wheat in millions of tonnes
                Produce     Consume
      China         99.7...
Rates and decimals


          In scientific notation:

      10n means 1 followed by n 0s

  2.1 e 10n means 2.1 followed...
Rounding


Too much detail confuses so rounding help
        the reader understand.

   3,123,918 is better written as 3.1...
Probability

If 2 out of 10 cars has a defect, the probability
       that any given car has a defect is

                ...
http://www.guardian.co.uk/society/2003/mar/27/medicineandhealth.lifeandhealth
      Skin cancer kills thousands of Britons...
Are you more likely to die in the UK?



                  Population            Number of             Chance of
         ...
Averages

Mean or average = sum of values/number of values

Median = Rank values and find middle

Mode = Value that occurs...
Normal distribution




Mathematical
      curve




          Real bell-shaped result
Standard deviation

Measures the spread of data.

s, sn, sn-1, sn and sn-1 are all different ways of
calculating the sprea...
Standard deviation

For a normal distribution:

1s 67% of the data is within one standard deviation of the mean

2s 95% is...
Standard deviation example


If a survey shows that average height of athletes on a
team is 175cm with a standard deviatio...
Graphing

                Pie charts good for showing how a whole
                is made up of parts. NB Groups should be...
Estimating from a sample


Using a sample of a population to predict something
about the whole population

Estimates can n...
Estimating from a sample

  Sample                 Margin of error
     size             (95% confidence)


      50      ...
Estimating a mean


  The margin of error of the mean can be calculated
  thus:



Margin of error at           2(sample’s...
How scientists work


Observational (epidemiological)
research


Experimental studies


Peer review


Systematic review (m...
Inference


To draw conclusions from data, you need to have
confidence in that data.

The statistical measure of confidenc...
Correlation r




The correlation of two factors is calculated with the
correlation coefficient (r) ranging from 0 to 1. N...
Correlation based on a sample


Correlation based on a sample with p= 0.05

For a sample of 10 if r is greater than 0.63 t...
Coefficient of determination

The value of r2 for two characteristics gives the %
variation in one quantity that is explai...
Correlation does NOT imply causation
                                       Facebook fuelling divorce, research claims



...
Surveys

Sample should be random

High response rate

Don’t depend on volunteered responses

Careful in constructing quest...
Experiment

Treatment = the thing you are testing

Control group = group not treated for comparison

Random assignment = p...
'Seafarers' disease' scurvy on rise among
 children due to lack of vitamin C in diet
                       Daily Mail 7 N...
'Seafarers' disease' scurvy on rise among
 children due to lack of vitamin C in diet
                                  . ....
'Seafarers' disease' scurvy on rise among
 children due to lack of vitamin C in diet

                                    ...
Media studies graduates

The Press Gazette reports that Media Studies graduates
    are among the most successful at findi...
Where graduates are after six months


                              All subjects
  Employed (73.6%)


             Study ...
4%
                                             8%
                                                  12%
                 ...
Apples & oranges


  August 2000 US Justice Policy Institute’s claimed that


   “more African American men are incarcerat...
The new phenomena phenomenon



                               Actual figures

                               Perception
C...
Care with comparisons

A new study claims that Autism has risen ten-fold in
the US in the past decade.


Most media source...
Dangerous conclusions




                                                Burglary
                                      R...
Sampling


26 February 2002 former US Secretary of Health,
Education and Welfare Joseph Califano’s claimed that
under-age ...
Time periods

11 June 2002 The New York Times claimed the
average temperature in Alaska “has risen about seven
degrees ove...
2002
             2000
             1995
             1990
Time slots




             1980
             1973
            ...
Massaged figures

A US government task force on college drinking
claimed that 1,400 students are killed and 600,000
are as...
Fear from simplifying
Lancet: "fear of breast cancer is so pervasive among US
women that it is causing them to ignore far ...
Breast cancer figures




                        80+ 1:8
                        80 1:26
                        60 1:56
...
Questions
WHO?   Reputable statistician or hopeful charity?
       Political spin doctor or civil service department.
    ...
Checklist
Where possible, go back to original numbers and check
assumptions and spin
Ask yourself whether the numbers you ...
Upcoming SlideShare
Loading in …5
×

Statistics for Journalists

1,090 views
1,028 views

Published on

Published in: News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,090
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Statistics for Journalists

  1. 1. Statistics for Journalists Adrian MacLeod adrian@writethinking.co.uk
  2. 2. Percentage starting points Wheat in millions of tonnes Produce Consume China 99.7 115.4 EU 104.4 91.2 Change %= x 100 Starting value
  3. 3. Percentage starting points Wheat in millions of tonnes Produce Consume China 99.7 115.4 EU 104.4 91.2 EU produces 4.7% more wheat than China China produces 4.5% less wheat than the EU China consumes 26.5% more than the EU The EU consumes 21% less than China The EU produces 14.5% more than it consumes The EU consumes 12.6% less than it produces
  4. 4. Rates and decimals In scientific notation: 10n means 1 followed by n 0s 2.1 e 10n means 2.1 followed by n 0s 2.1 e 10-n means 2.1 with the point moved n places to the right 2.1 e 102 = 210 2.1 e 10-2 = 0.021 2.1 e 10-5 = 0.000021 or 2.1 out of 105 (105 = 100,000)
  5. 5. Rounding Too much detail confuses so rounding help the reader understand. 3,123,918 is better written as 3.1m If the part you are throwing away begins with 4 or less, just through it away. If it begins with a 5 or more, increase the final digit of your rounded number by one. 3,138,487 is 3.1m 3,176,918 is 3.2m Beware rounding errors (percentages may add up to 99% or 101%)
  6. 6. Probability If 2 out of 10 cars has a defect, the probability that any given car has a defect is 2/10 or 0.2 or 20% Don’t confuse probability with quantity More does not necessarily mean more likely
  7. 7. http://www.guardian.co.uk/society/2003/mar/27/medicineandhealth.lifeandhealth Skin cancer kills thousands of Britons Guardian Thursday 27 March 2003 Thousands more Britons than Australians die from skin cancer even though more cases of the disease are diagnosed in Australia, new figures showed today. Cancer Research UK, Britain's biggest charity, and the government today launched a nationwide campaign to encourage more Britons to protect themselves and their children from the sun's harmful rays. Figures released as part of the Sun Smart campaign show that in the last five years there have been 8,100 British deaths from malignant melanoma compared to 4,900 in Australia. The figures showed that nearly 8,000 cases of malignant melanomas are diagnosed in Australia each year, and nearly 6,000 in the UK. Yet 600 more people die from the disease each year in Britain than in Australia. Dermatologists, from CRUK said that a lack of public awareness about skin cancer is leading to needless deaths. Many patients failed to use proper protection in the sun and others did not spot early symptoms of malignant melanomas, they added. People should seek medical advice if they notice that a mole changes shape, gets bigger, alters in colour (particularly getting darker or multi-shaded) bleeds or becomes itchy or painful. The campaign stresses that pre-cancerous moles are easy to treat and are usually removed under local anaesthetic. An early melanoma can be cured in this way, but if left, the disease can spread. The main risk factor for malignant melanoma is ultraviolet light from the sun or sun beds. People are considered more at risk if they have lots of moles, are fair skinned with blue eyes, tend to sunburn easily or have freckles. Dr Charlotte Proby, consultant dermatologist for CRUK said: "Malignant melanoma is a preventable cancer. We need the public to be aware of what they can do to help prevent the disease." She added: "The success of sun awareness campaigns in Australia is self evident."
  8. 8. Are you more likely to die in the UK? Population Number of Chance of deaths* dying UK 63 million 8,100 1: 7,700 Australia 20 million 4,900 1: 4000 * Deaths over five years due to malignant melanoma. Source: Cancer Research UK. Impression given: twice as likely to die from skin cancer in the UK. Truth: half as likely to die from skin cancer in the UK.
  9. 9. Averages Mean or average = sum of values/number of values Median = Rank values and find middle Mode = Value that occurs most frequently Mean useful for comparing groups of figures which have a normal distribution Median useful for comparing groups of figures which do not have a normal distribution Warnings: Do NOT average averages Average does not necessarily mean typical
  10. 10. Normal distribution Mathematical curve Real bell-shaped result
  11. 11. Standard deviation Measures the spread of data. s, sn, sn-1, sn and sn-1 are all different ways of calculating the spread but give more or less the same answer provided the number of values in the data sample is more than 20 Large SD Small SD
  12. 12. Standard deviation For a normal distribution: 1s 67% of the data is within one standard deviation of the mean 2s 95% is within two standard deviations of the mean 3s 99% is within three standard deviations of the mean 1 SD 1 SD 2 SDs 2 SDs 3 SDs 3 SDs Mean
  13. 13. Standard deviation example If a survey shows that average height of athletes on a team is 175cm with a standard deviation of 10cm then it is reasonable to estimate that: 67% of athletes are between 165 and 185cm 95% of athletes are between 155 and 195cm
  14. 14. Graphing Pie charts good for showing how a whole is made up of parts. NB Groups should be distinct and separate. Bar charts for comparing. NB the impact of bar charts can be distorted if the origin is not zero or if 3-D columns are used. Line graphs are good as showing how things change over time. Y origin should be 0. Dates should be evenly spaced
  15. 15. Estimating from a sample Using a sample of a population to predict something about the whole population Estimates can never be 100% accurate so they should come with a margin of error Margin of error at 90% confidence is 0.82/ n Margin of error at 95% confidence is 0.98/ n Margin of error at 99% confidence is 1.29/ n ** Provided the whole population is a large number, the margin of error is NOT dependent on population size **
  16. 16. Estimating from a sample Sample Margin of error size (95% confidence) 50 ±13.9%* 100 ±9.8%* 200 ±6.9%* 300 ±5.7%* 400 ±4.9%* 500 ±4.4%* 750 ±3.6%* 1000 ±3.1%* 2000 ±2.2%* * % is actually percentage points
  17. 17. Estimating a mean The margin of error of the mean can be calculated thus: Margin of error at 2(sample’s standard deviation) = 95% confidence number in sample Margin of error at 1.7(sample’s standard deviation) = 90% confidence number in sample Use this formula only for sample sizes greater than 30
  18. 18. How scientists work Observational (epidemiological) research Experimental studies Peer review Systematic review (meta analysis) Risk factor based on converging data 50:1
  19. 19. Inference To draw conclusions from data, you need to have confidence in that data. The statistical measure of confidence is p. The smaller the number the better. p is a probability between 0 and 1 In social science the p value should be at most 0.05 In medicine the p value should be at most 0.005 Often, all you have to do is check that the p-value is appropriately small.
  20. 20. Correlation r The correlation of two factors is calculated with the correlation coefficient (r) ranging from 0 to 1. Near 0 means little correlation. Near 1 means closely correlated. r=-1.0 means an inverse correlation. EG: The weight of people in a sample is likely to be correlated with height (tall people are heavier) and inversely correlated with life expectancy (heavier people die younger)
  21. 21. Correlation based on a sample Correlation based on a sample with p= 0.05 For a sample of 10 if r is greater than 0.63 then there is a (non-trivial) correlation For a sample of 30 if r is greater than 0.36 then there is a (non-trivial) correlation For a sample of 100 if r is greater than 0.2 then there is a (non-trivial) correlation
  22. 22. Coefficient of determination The value of r2 for two characteristics gives the % variation in one quantity that is explained by another EG: If the correlation coefficient between weight and 2 height is r = 0.07 then r = 0.49 which means that about 49% of the variation in weight is explained by height. 51% will be explained by other factors NB: Correlation does NOT imply causation If correlation between X and Y is high then we can say X is a good predictor of Y If r is v small and confidence is high (p is also small) then may be statistically significant but not significant isn the real world
  23. 23. Correlation does NOT imply causation Facebook fuelling divorce, research claims n io Telegraph 21 December 2009 at The social networking site, which connects old friends and allows users to make us new ones online, is being blamed for an increasing number of marital breakdowns. ca Divorce lawyers claim the explosion in the popularity of websites such as Facebook and Bebo is tempting to people to cheat on their partners. http://www.telegraph.co.uk/technology/facebook/6857918/ y pl Suspicious spouses have also used the websites to find evidence of flirting and even affairs which have led to divorce. im One law firm, which specialises in divorce, claimed almost one in five petitions they processed cited Facebook. Facebook-fuelling-divorce-research-claims.html T Mark Keenan, Managing Director of Divorce-Online said: "I had heard from my NO staff that there were a lot of people saying they had found out things about their partners on Facebook and I decided to see how prevalent it was I was really surprised to see 20 per cent of all the petitions containing references to Facebook. es do "The most common reason seemed to be people having inappropriate sexual chats with people they were not supposed to." n Flirty emails and messages found on Facebook pages are increasingly being io cited as evidence of unreasonable behaviour. at ... The UK’s divorce rate has fallen in recent years, but two in five marriages are el still failing according the latest statistics. rr Mr Keenan believes that the general divorce rate will rocket in 2010 with the Co recession taking the blame.
  24. 24. Surveys Sample should be random High response rate Don’t depend on volunteered responses Careful in constructing questions: not good if you are interpreting the answers; if the answers are ambiguous; if the phrasing of questions pushes people towards a particular answer Always state who conducted survey and give margin of error
  25. 25. Experiment Treatment = the thing you are testing Control group = group not treated for comparison Random assignment = people are assigned to control group or treatment group randomly Placebo = fake treatment given to control group so they do not realise they are the control group Double blind = neither those being experimented upon nor those giving the treatment know who is getting the the treatment and who the placebo.
  26. 26. 'Seafarers' disease' scurvy on rise among children due to lack of vitamin C in diet Daily Mail 7 November 2009 Scurvy is making a comeback among England's children. Caused by a lack of vitamin C, the potentially fatal disease was a scourge of Seafarers-disease-Scurvy-rise-children-lack-vitamin-C-diet.html pirates and sailors in the heyday of the British Empire, but was thought to be largely a thing of the past. However, newly released statistics show that the number of children admitted http://www.dailymail.co.uk/health/article-1225905/ to hospital with scurvy soared by over 50 per cent in the past three years. Released following a parliamentary question, the figures show that in 2004/05 there were 61 children admitted with scurvy in England. But by 2007/08, the latest year for which figures are available, there were no fewer than 94 cases: up 54 per cent in three years. Because the figures cover only those admitted to hospital with scurvy as a primary or secondary diagnosis, the actual numbers with the disease will be far higher as many will not get further than their GP. Others may be listed under the wider term of 'malnutrition'. Scurvy occurs if people do not eat enough foods containing vitamin C such as fruits, tomatoes, potatoes, liver and oysters. Scurvy leads to spots on the skin, particularly the legs, as capillaries break down. There is cracking and bleeding of the lips, nostrils and ears. Gums go spongy and teeth fall out. ... continues
  27. 27. 'Seafarers' disease' scurvy on rise among children due to lack of vitamin C in diet . . . continued Wounds cannot heal properly, and old scars reappear. There is internal haemorrhaging and left untreated, victims will die. Seafarers-disease-Scurvy-rise-children-lack-vitamin-C-diet.html Conservative health spokesman Stephen O'Brien, who uncovered the figures, said: 'It is shocking that this disease of 17th-century pirates is on the rise again in 21st-century England.' http://www.dailymail.co.uk/health/article-1225905/ Ursula Arens, of the British Dietetic Association, said it was not possible to say how the children were getting scurvy: whether it was from a poor diet, or as a by-product of other diseases such as cancer. 'There may be examples of children just living on bread and jam and nothing else because of poverty,' she said. 'It is such an unusual thing now that perhaps it is something that many GPs would not be able to diagnose.' A spokesman for the Department of Health said: 'Families in lower income groups tend to consume less vitamin C in their diet. 'The Department of Health promotes consumption through its "five a day" campaign and Healthy Start, which provides free vitamin supplements for beneficiaries.'
  28. 28. 'Seafarers' disease' scurvy on rise among children due to lack of vitamin C in diet Scurvy cases Data on which Mail story was based 2007-08 94 showed primary and secondary 2006-07 101 hospital diagnosis of absorbic acid 2005-06 68 deficiency (scurvy). 2004-05 61 2003-04 72 Problems: 1) Showed all patients not just children 2) Based on small numbers with large year-year variations (noise) 3) Picked two years with biggest difference (longer time period and window average would have been more convincing) 4) Data contains no info about CAUSES of scurvy. Actually more likely to be caused by increased cancer survival.
  29. 29. Media studies graduates The Press Gazette reports that Media Studies graduates are among the most successful at finding jobs. It is true that 73.6% are in employment compared with 66.9% figure for all graduates. The implication is that you are less likely to be unemployed. But PG has excluded the figures for people who stay in education. Actually the figure for those who are unemployed is more revealling: 10.1% for media studies compared with 6.9% for all subjects. Perhaps a more significant story is that only 11.5% found jobs in the media. Those ending up as secretaries, shop assistants and bar staff adds up to a far higher figure.
  30. 30. Where graduates are after six months All subjects Employed (73.6%) Study (9.1%) Employed (66.9%) Study (18.7%) Unemployed (10.1%) Unemployed Media studies (6.9%)
  31. 31. 4% 8% 12% 16% Marketing, sales, PR advertising Commercial, public sector managers Scientific research, analysis Engineering Health Teaching All subjects Business & Finance Media studies Information technology Nursing & health associate Business & finance associate Media, literary, design, sport Other porfessional & technical Types of work Numerical clerks & cashiers Other clerical & secretarial Retail assts, waiting & bar staff Health & childcare Armed forces & public protection Other Unknown
  32. 32. Apples & oranges August 2000 US Justice Policy Institute’s claimed that “more African American men are incarcerated than enrolled in college.” You can go to prison at any age, for any length of time, but most people go to college for only a few years during their late teens and twenties.
  33. 33. The new phenomena phenomenon Actual figures Perception Cases of AIDS 55 60 65 70 75 80 85 90 95 00 05 10 15 Year
  34. 34. Care with comparisons A new study claims that Autism has risen ten-fold in the US in the past decade. Most media sources reported this as proof that a worrying increase in the condition has gone unnoticed, while others raised the possibility of a link to the substance thimerosal, an ingredient in childhood vaccinations. In fact, the increase was probably due to a massive increase in the definition of autistic disorders over the past 10 years.
  35. 35. Dangerous conclusions Burglary Robbery International Crime Assault Victimisation Survey 2000 UK 6.0% 1.2% 2.8% South Africa 5.8% 4.6% 6.3% Conclusion: UK nearly as dangerous as South Africa? Murder rate (1997 figures): UK 1.4 per 100,000 SA 58.4 per 100,000
  36. 36. Sampling 26 February 2002 former US Secretary of Health, Education and Welfare Joseph Califano’s claimed that under-age drinkers consume a quarter of all alcohol drunk in the United States. The survey over-sampled teenagers. As The New York Times conceded in a subsequent correction, the actual figure is only 11%.
  37. 37. Time periods 11 June 2002 The New York Times claimed the average temperature in Alaska “has risen about seven degrees over the last 30 years.” This surprised experts at the Alaska Climate Research Center, whose figures show an increase of only 2.5 degrees in the same period. The Times corrected its mistake on 11 July, but still claimed an increase of 5.4 degrees, which it justified by using the period from 1966 to 1995, rather than the “last 30 years” from 1973 to 2002.
  38. 38. 2002 2000 1995 1990 Time slots 1980 1973 1970 1966 1962
  39. 39. Massaged figures A US government task force on college drinking claimed that 1,400 students are killed and 600,000 are assaulted each year because of alcohol. That figure would represent two thirds of all the assaults in the US, according to the FBI. But the “assaults” included everyone who said they were “pushed or hit” by anyone as a result of someone drinking. And the deaths included fatal auto and other accidents in which anyone (not only drivers) tested positive for any amount of alcohol.
  40. 40. Fear from simplifying Lancet: "fear of breast cancer is so pervasive among US women that it is causing them to ignore far more serious health threats." The commonly reported figure that 1 in 8 figure women will develop breast cancer only applies to those already in their 80s Office of National Statistics shows that the risk for women under age 35 is 1:625, rising to 1:56 by age 50. Before the age of 50 only one woman out of 136 dies of breast cancer. By the age of 60 this is one out of 65. And by the age of 80 only one woman in 26 dies of breast cancer, which represents the lifetime risk for most women - that means the other 25 will die of something else.
  41. 41. Breast cancer figures 80+ 1:8 80 1:26 60 1:56 50 1:136 35 1:625
  42. 42. Questions WHO? Reputable statistician or hopeful charity? Political spin doctor or civil service department. Ask yourself how reliable you think the organisation compiling the statistics is and rate the results accordingly. WHY? Do they have a vested interest in producing a particular result? If so is the way they compiled the data reasonable? Did they ask the right questions? Have they focused on the most helpful result to them and distorted the truth in the process? HOW? Was the sample representative? Was it big enough? Was there bias built into the way they conducted the survey?
  43. 43. Checklist Where possible, go back to original numbers and check assumptions and spin Ask yourself whether the numbers you are reporting tell the whole story Like any other aspect of journalism, check sources Watch time periods Take care with percentages and other rates With surveys, find out what questions were asked Question the conclusions that you and others derive from figures Watch out for spin - figures selected or presented for one party’s advantage Check that comparisons are valid

×