‘A lottery of treatment and care – MS services across the UK’ report by the MS Society:
who does it represent?
Paul J. B...
came from a large survey of all members of the MS Soc over the age of 18 years who had
claimed on their application form...
the report in that the basic contemporary demographic character of the population of
PwMS in the UK is unknown because t...
community of PwMS; a position that becomes even more serious when it is asked if the
results could even be regarded as r...
claimed this proportion was as high as 60% - a ‘fact’ that was enthusiastically reported in
the local Colchester Gazette...
gave hard-working health-care professionals and volunteers alike more work, to explain to
PwMS that their situation was ...
Dorning H, Luck G and Holloway E (2013a) A lottery of treatment and care – MS services
across the UK. Technic...
Dr. Paul J. Bull: Brief biography
Born 1953, married, one daughter, three grandchildren.
BA Hons in Geography (Sheffield...
Upcoming SlideShare
Loading in …5

Bull ms lottery of care 2013


Published on

Published in: Health & Medicine, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Bull ms lottery of care 2013

  1. 1. 1 ‘A lottery of treatment and care – MS services across the UK’ report by the MS Society: who does it represent? Paul J. Bull Hon Visiting Fellow, Anglia Ruskin University and Chairman, Colchester Branch, MS Society Abstract This paper challenges the findings of the report by the MS Society ‘A lottery of treatment and care’ by arguing that they cannot be supported by the research methods used to produce them. In particular, it shows that the sample of members of the MS Society with MS used for the research cannot be regarded as representative either of all people with MS in the UK or of all members of the MS Society with MS. Furthermore, it cannot be thought to offer meaningful results for either of these two cohorts of people at sub-national spatial scales either. It is therefore suggested that much more rigorous scrutiny of future projects by the MS Society must be undertaken to stop such tawdry research being repeated. Introduction In April 2013 the MS Society of Great Britain and Northern Ireland (MS Soc) published a report entitled ‘A lottery of treatment and care – MS services across the UK’ along with an accompanying technical report (Dorning et al. 2013a) and a poster presentation at the MS Soc’s biennial ‘MS Frontiers’ research conference, ‘My MS, my needs: do feelings about income relate to getting what you need from health and social care services?‘ (Dorning et al. 2013b). This suite of work organised in-house and written by the MS Soc’s own research staff, presented evidence on four important components of the lives of people with MS (PwMS) in the UK: access to medicines, care and support, financial security, physical and mental wellbeing. In the main report it claimed, inter alia, that PwMS in the UK were being denied access to Disease Modifying Drugs (DMDs) with only six out of every ten eligible PwMS taking them (MS Soc 2013, 10); the degree of use of many of the newer symptom relieving therapies was ‘abysmally low’ (MS Soc 2013, 12); standards of social care ranged from excellent to ‘criminally cruel’ and most was provided informally by relatives and friends (MS Soc 2013, 14); only a quarter of PwMS of working age were employed compared to 75% generally in the UK but only a quarter of those who wanted to get back into work could access appropriate support (MS Soc 2013, 18). In addition, the poster presentation (Dorning et al. 2013b) suggested that access to many treatments and support was directly related to ability to pay. The latter appeared to be a new and especially stark finding that the financially worst off were the PwMS least likely to get the care and treatments they needed; the poorest baring the greatest burden of their illness! This project was concerned with the experiences of PwMS in the UK as a whole and for a number of smaller spatial units from the four individual countries of the nation to Primary Care Trusts and Local Authorities. The evidence used to represent PwMS for the report
  2. 2. 2 came from a large survey of all members of the MS Soc over the age of 18 years who had claimed on their application form they had MS (MwMS). However, almost without exception and justification, the report was written as if the two were identical, that PwMS were the same as MwMS. In doing so this research raises a number of important methodological and conceptual concerns of a general nature for the study of PwMS in the UK and for an understanding of exactly what this putative research was revealing. This review will consider the following three: 1. Can MwMS be satisfactorily used to represent PwMS in the UK? 2. Accepting that they can, were they properly represented in this project? 3. Again, accepting that they were, does it also hold true for sub-national spatial units too? It will then conclude with recommendations for future in-house research carried out by the MS Society. Representative of all PwMS in the UK? Because there is no national consolidated list of PwMS in the UK, or a complete national register of people with the illness, researchers must use a proxy as a sampling frame or contact list. As in the research above therefore membership lists and magazine circulation lists of the main UK MS charities have often been employed for this task. But, is such a choice acceptable? To what extent can MwMS really be thought of as representative of all PwMS in the UK? Clearly the authors of the recent MS Soc’s report believed they could, dismissing the issue in a few disappointingly vague sentences. “The demographics of those who responded are as we would expect. In the UK around three times as many women as men have MS, and our survey respondents reflect this. Our respondents were slightly older than the MS population, reflecting our membership, but people of all ages and all types of MS responded.” (MS Soc 2013, 7) “The response rate, absolute numbers of respondents and the broad demographic distribution of respondents give us confidence that our findings present an accurate picture of treatment and care across the UK, and provide a clear mandate for action.” (MS Soc 2013, 7) “The final sample consisted of 7,669 (75%) women and 2,619 (25%) men, which is representative of people with MS in the UK. The average age of respondents was between 51 and 65 years.” (Dorning et al. 2013a, 6) The last sentence above on the average age of the respondents to this survey is unfortunately all too representative of the casual approach taken in this research. Exactly what average, or measure of central tendency – mean, mode or median, do the authors mean (sic)? There is, of course, a very good reason for the lack of clarity in this element of
  3. 3. 3 the report in that the basic contemporary demographic character of the population of PwMS in the UK is unknown because there have been few attempts in the recent past at a complete regional ascertainment of all cases of MS from available data sources. Indeed, since the year 2000 there have been only four as in table 1. Table 1. Regional epidemiological studies in the UK since 2000 Region and date Female % Mean age Plymouth (2001) 73 52 North Northern Ireland (2004) 67 50 South Glamorgan (2005) 71 51 Aberdeen City, Orkney and Shetland (2009) 71 53 Sources: Fox et al. (2004), Gray et al. (2008), Hirst et al. (2009) and Visser et al. (2012). Curiously this includes one study from each of the four countries of the UK although not ones one could regard as representative of them. Nevertheless, at least in terms of three of these regional studies the findings on the two quoted demographic attributes are similar to those noted by the MS Society’s report with perhaps the exception of the one in Northern Ireland. But, this refers to a small and unusual part of the province that includes only the four districts of Coleraine, Moyle, Ballymoney and Ballymena. However, if representativeness is to be based on demographic similarity, then there are a number of other such attributes that may be of at least equal if not more importance than the two noted in the MS Soc’s report in affecting access to care, treatments, financial security and mental health such as ethnicity, family and household composition, marital status, employment, income and wealth. For example, a disabled, unmarried, unemployed man with MS living alone may find it much more difficult to secure the care and support he needs than a disabled married woman living with her family. Much more evidence therefore is required in this analysis to justify the Inferences made about the similarity between MwMS and PwMS.. Furthermore, there is at least one other sense in which MwMS are definitely different from PwMS generally, an area therefore in which the evidence in this report is knowingly biased, namely their membership of a support charity for which they made a conscious decision to join. The reasons for such a decision will have been varied and complex ranging from an altruistic desire to help others to perhaps a hope of helping themselves through better information and health-care contacts. Nevertheless, no matter how such behaviour may impact on their access to health care and financial and psychological health, it knowingly biases the results and makes them unsafe to be regarded as representative of the wider
  4. 4. 4 community of PwMS; a position that becomes even more serious when it is asked if the results could even be regarded as representative of MwMS themselves? Representative of MwMS in the UK? The point of a sample is to represent as closely as possible the population from which it was drawn and of which it is part in an abbreviated form by keeping all possible elements of bias to a minimum. There are well tried and tested methods for carrying out such a task of which the most obvious is through a random sampling technique. This was not carried out in this case. Instead a full census trawl was attempted by sending a postal questionnaire to all MwMS in the UK. Yet in order to discover equivalent evidence a random sample survey, maybe stratified by the four countries of the UK, of no more than 15% of the cases would have been required; an approach that would have been much more efficient and cheaper by allowing much speedier analysis through the use of powerful inferential statistical techniques. Instead, a very large self-selected sample was secured biased in unknown ways. Thus, it is simply not safe to assume that the evidence used is representative of even MwMS let alone the wider community of PwMS in the UK; the methodology adopted simply did not allow it! Furthermore, the knowingly biased way in which the sample was collected precludes the use of any statistical techniques based on the linear model. Why was such a wide-bore, expensive, blunderbuss approach adopted when a far more focussed and cheaper approach could have produced safer results? Why were scarce charitable research resources wasted in this way? It may have been reasonable to expect the separately published technical report (Dorning et al. 2013a) to have provided an argument for their methodological approach. It did not. Perhaps it was undertaken in the mistaken belief that it was a useful way of collecting evidence on MwMS both at the local and national scales at the same time. As will be shown in the next section, although such evidence was collected, it seriously increased the chances of the results for smaller spatial units misrepresenting what was actually happening in them. Such a serendipitous approach should never have been countenanced. If the research question necessitated data for small areas then an approach should have been used that collected this evidence directly in a way that was representative of each area separately probably best via random sampling. They could have then been added together to form a larger random sample for a bigger geographical unit if desired. But, dividing an almost useless biased national sample into samples for smaller spatial units just created a series of smaller equally useless biased samples! Representative of MwMS in sub-national spatial units? A good example of how the approach used for this report produced biased false results can be found in the evidence released for North East Essex. Furthermore, because this survey definitely produced incorrect results in this one area, it cannot therefore be claimed to be generally applicable for all sub-national areas in the UK. For NE Essex the MS Society issued a press release that announced that eligible PwMS were not receiving the DMDs they deserved and needed. Indeed, in NE Essex from the 88 questionnaires returned it was
  5. 5. 5 claimed this proportion was as high as 60% - a ‘fact’ that was enthusiastically reported in the local Colchester Gazette (30/04/2013) set in the almost obligatory press medical hyperbole of ‘post-code lottery of care’ and ‘fighting for expensive medicines’. Yet, nothing could have been further from the truth. According to both the consultant neurologist and the MS Specialist Nurse for the area, if a person with MS met the nationally agreed criteria for the prescribing of a DMD, namely two clinically significant relapses in a 24 month time period (MS Society website, accessed 01/07/13), they would have received a DMD. However, such a report in the local newspaper worried a large number of PwMS and their families that they may be being denied life-changing medication to the point that the local consultant neurologist had to write to the Colchester Gazette (published 03/05/2013) affirming that no person with MS was missing out on the drugs to which they were entitled. Now, it may well be that the MS Society believed the criteria for prescribing DMDs have been and still are too severe and ambiguous, but that is a national matter and not a local one. There was no reason to berate local health care professionals and health authorities for following national guidelines. To play out such at a political game locally was wholly inappropriate and counter-productive. It may have produced a modicum of local publicity for the MS cause but, at the same time it worried many PwMS about their medication and, worse, raised the real possibility of souring the relationships between local MS-charity workers – professional and volunteers alike – and the local health-care sector. If the MS Society had really wished to know about the experiences of PwMS at the local scale then they should have attempted one or both of two different research approaches. First, separate local random samples of MwMS should have been carried out. Such an approach may have seemed too arduous in both time and expense for all the areas in which they had an interest, but at least it would have generated samples from which meaningful inferences could have been drawn. And, these samples need not to have been very large. The 88 in NE Essex, or 12% of the people in the area known to have MS, would have been more than enough, if only it had been known they were representative of the area. Unfortunately, they were not in any sense. The second approach would have been to ask the people who work for the MS Society in each locality either as employees such as the Regional Support and Service Development Officers or as branch committee volunteers. These are the people who understand the local situation best, have contacts with local health-care providers and are often MwMS themselves with good relations with other PwMS in their areas. Not to have included such individuals in this research project was an inept over-sight for three reasons. First, that at least in one locality – NE Essex, when the press began to try to solicit comment on the report it put the local MS Soc branch committee members in the awkward position of being expected to support conclusions that were wrong. Second, it angered many people who had worked for the MS cause for many years for not being consulted about the research and for not giving them any reliable evidence they could use to argue for better MS support. And, the third reason was that it
  6. 6. 6 gave hard-working health-care professionals and volunteers alike more work, to explain to PwMS that their situation was not as dire as the local press had stated. Conclusion When a research project is being designed great care must be taken over the methodology adopted; that is on the logic of the project. Get it right and the research will produce reliable evidence on which future action can be based. Get it wrong, as in this case, and the whole exercise is undermined, a waste of time and effort on which no future action at all can be based reliably. In this case the methodology did not allow the sample of MwMW, no matter how large, to be representative of PwMS in the UK. Nor did it even allow the sample to be representative of MwMS themselves. As a result, the evidence presented in this MS Soc report simply described what had happened to a large group of MwMS and cannot logically be thought to relate to any wider constituency. Furthermore, the chosen methodology compromised anything useful to be said about smaller geographical units as demonstrated by the NE Essex example. Clearly, the MS Society, that is supposed to stand for excellence in research, must do a great deal better in the future if it is to keep such an appellation. When a research grant application Is made to the MS Society it must satisfy a number of severe tests before it can be funded. These include both peer and lay reviews and the need to demonstrate a working knowledge of cognate research and publications. Within such a rubric it is difficult to believe a proposal for the self-indulgent research described above would have been commissioned without serious revision. In particular, there is very little evidence shown of other published relevant research; a point that became starkly obvious in the introduction written by the Acting Chief Executive. In this section she asserted the report to be an ‘unprecedented’ piece of research (MS Society 2013, 5). It was not! In 2002 the MS Society itself published “Are we being served? Health care experiences of people with MS” an account that methodologically was even worse, that again attempted separate regional profiles and was based on a huge self-selected questionnaire survey that claimed an even bigger response of 16,400 (MS Society 2002). It would appear these MS Soc researchers did not even know what their own organisation had already published! Clearly, therefore, in order to prevent such tawdry research from being produced again the MS Society must impose a much stricter regime of scrutiny and begin to judge both its own in- house research activities and external research grant applications identically; a future research prophylaxis to prevent muddled methodologies, impotent findings and flaccid scholarship. Conflict of interest disclosure. Paul Bull has received a small research grant from the MS Society and was a member of their Research Strategy Committee.
  7. 7. 7 References Dorning H, Luck G and Holloway E (2013a) A lottery of treatment and care – MS services across the UK. Technical report. MS Society, London. Available on-line at www.mssociety.org.uk. Dorning H, Luck G and Holloway E (2013b) My MS, My needs: do feelings about income relate to getting what you need from health and social care services? Poster presentation at the MS Frontiers conference, London. Fox CM, Bensa S, Bray I and Zajicek JP (2004) The epidemiology of MS in Devon: a comparison of the new and old classification criteria. J. Neurol. Neurosurg and Psychiatry 75, 56-60. Gray OM, McDonnell GV and Hawkins SA (2008) Factors in the rising prevalence of MS in the North-East of Ireland. Mult. Scler. 14, 880-886. Hirst C, Ingram G, Pickersgill T, Swingler R and Compston DAS (2009) Increasing prevalence and incidence of MS in South East Wales. J Neurol Neurosurg Psychiatry 80, 386-91. MS Society (2002) Are we being served? Health care experiences of people with MS. MS Society’ London. MS Society (2013) A lottery of treatment and care – MS services across the UK. MS Society, London. Available on-line at www.mssociety.org.uk. Visser EM, Wilde K, Wilson JF, Yong KK and Counsell CE (2012) A new prevalence study of multiple sclerosis in Orkney, Shetland and Aberdeen city. J Neurol Neurosurg Psychiatry 83, 719-724. Correspondence: email: paul.bull1@btopenworld.com
  8. 8. 8 Dr. Paul J. Bull: Brief biography Born 1953, married, one daughter, three grandchildren. BA Hons in Geography (Sheffield), PhD (Cambridge). University lectureships in Geography held at Queens Belfast 1977-86 and Birkbeck College University of London 1986-2000. Retired in 2000 due to ill health caused by MS. Fellow of the Royal Geographical Society since 1989. Honorary Research Fellowships at University of Exeter 2000-03, the School for Health and Related Research at the University of Sheffield 2007-10 and Anglia Ruskin University since 2012. MS career: first symptom in 1976, diagnosed in 1981, needed a wheelchair in 2001. Charity work: MS Society: organiser of Getting to Grips courses at my local branch (Colchester) 1996- 98, Regional Committee member 1999-2001, member of the MS UK Science and Development Board 2002-6, Chair of the UK MS Register Steering Group 2005-7, member of the Research Strategy Committee for the MS Network 2004-12, member of the partnership committee with the James Lind Alliance on MS research priorities 2012-13, Chairman Colchester Branch of MS Society from 2012. John Grooms charity 2003-6: trustee and member of various committees.