Advantages and disadvantages of the ‘distance’ that quantitative research establishes between the researcher and the researched
Advantages and Disadvantages of the ‘Distance’ that Quantitative Research Establishes Between the Researcher and the Researched? By Dr. Awais e Siraj Managing Director Genzee Solutions Islamabad Pakistan In 2003, first Beta Interferon was launched in Pakistan used for the treatment of Multiple Sclerosis. Till that time, Multiple Sclerosis was not taken very seriously by physicians as well as patients. The common belief was that it does not exist in this part of the world and even if it does, it is not possible to offer a solution. There was no market data as the ‘market’ did not exist. There were no formal studies of incidence and prevalence, only stories and anecdotes. One story goes like this: “Yasmin was a great student of arts and textile designing. Since her childhood she was also active in sports and always won medals in 100 meters and 400 meters events. She represented her college in many competitions and brought name and fame for herself, institution and her parents. On 21 August 1997, she started feeling numbness and loss of power in her left hand. She just ignored it. Then in the evening, when she started her daily routine of studies, she could see only half of the page. The other half was blurred. Next day, her father took her to a GP who prescribed certain multivitamins and described it as ‘exam pressure, you know’. They came back and she recouped. After a week, right on the dinner table she fainted and had to be immediately rushed to the hospital. This time, she was lucky to be seen by a ‘specialist’ who performed a series of tests and also got an MRI scan. He immediately put her on steroids for four days. On the third day, he announced much to the anxiety and ignorance of her bamboozled parents that she has some signs of ‘demyelination’ in her brain and that she is suffering from a disease called Multiple Sclerosis which is rare in Pakistan but still exists. She will have to use some kind of injections called Beta Interferons for the rest of her life. “These injections are currently not available in Pakistan and you will have to get them from somewhere. They are very expensive”, informed their well reputed and well informed neurologist. “Yasmin’s parents belonged to a middle class family of an underdeveloped country. Her father was an administration manager in a small semi government organization and mother was a house wife. They had five children including Yasmin, all in their school going age. They had no other source of income and healthcare reimbursement was never heard of in Pakistan. Beta Interferons were very expensive even by international standards and still did not offer complete cure. Her father decided to sell off their only property, a small house worth a few thousand dollars to buy the drug that could save the life of his dear daughter only to find out later that the disease is not going away and that the entire family is now bankrupt. Yasmin is completely paralyzed and on bed for the last one year.”
The head of marketing of this company had some point to start with, at least. But one story is not going to hammer the point home, he thought. “I need some concrete evidence and solid data to prove that the disease exists/does not exist in Pakistan. Moreover, I have to base my company decisions on some data which has to be more than just one or two stories”. He had to convince the health authorities as well as physicians that this disease exists in Pakistan. He also thought of raising funds for patients who could not afford his drug through philanthropist activity but everyone seemed to ask for ‘numbers’ and not just stories. After spending a very hard time he was able to motivate a few neurologists to collect some meaningful data and put it together in the shape of a publication. An instrument (questionnaire) was designed after much deliberations and data was collected from different neurology clinics, radiology centers and hospitals. They could only identify 142 cases but this time we they also had some numbers to play with like mean age, male to female ratio, disease progression, disease stage etc.etc. The following paragraph is an abstract of the publication. “We describe retrospective data from the largest series of patients (n_/142) with multiple sclerosis (MS) from Pakistan. Mean age at onset was 27 years, with a female to male ratio of 1.45:1. The disease onset was polysymptomatic in 75% patients. Motor weakness was the most common onset symptom (70%), followed by sensory symptoms (45%). Optico-‐spinal type of MS was seen in only 3% of patients The course was relapsing-‐remitting (RR) in 81%, primary progressive (PP) in 21%, and secondary progressive (SP) in 4% of patients. Almost three-‐fourths of the patients were moderately (45%) or severely (31%) disabled at the time of evaluation. Two-‐thirds of patients with severe disability had a mean disease duration of only 5.2 years. In conclusion, MS is not uncommon in Pakistan, and many patients were found to have severe disability despite short disease duration.” M Wasay, S Ali, IA Khatri, A Hassan, M Asif, N Zakiullah, A Ahmed, A Malik, B Khealani, A Haq and S Fredrikson Multiple Sclerosis 2007; 00: 000_000. http://msj.sagepub.com Yasmin was one of the 142 cases mentioned in the abstract given above. It was only after the publication of this data that all stakeholders started seriously thinking about Multiple Sclerosis in Pakistan. Debates started generating later, building on this and stories became more and more available when individual explorers started looking beyond numbers into the lives of patients and their families. This study became a building block for further studies and currently an extensive ‘genetic’ study on Multiple Sclerosis is under process. At the same time, philanthropists and NGO’s are beginning to believe in its existence and treatment strategies are under immense discussion so are the treatment guidelines specific for Pakistan where healthcare reimbursement is almost non – existent. The data presented by Wasay et al is “natural science” and advocates of quantitative research would emphasize that all research carried out in social sciences should be similar. Furthermore, a quantitative data on social aspects of this study would be possible like the ‘affordability’ ‘disability’ ‘financial impact on the family’ and ‘financial burden on the family’ and ultimately some development economist would also be able to calculate the Disability Adjusted Life Years
and Economic Burden of this disease on society some day. But all of this started with one story of Yasmin. Without the story, no researcher will ever be able to understand what was really going on in the mind of Yasmin and her family. It is sometimes a story of chicken and egg. When this study was being conducted there was no data to understand Multiple Sclerosis in Pakistan. Only such stories as that of Yasmin came along and no one really cared about it. It was only after collection, collation and publication of this data that led many to recognize that Multiple Sclerosis needs to be taken seriously as it is in rest of the world. Then they started digging out stories again and reaching individual patients and their families of these 142 (and more as they were diagnosed) and trying to find similarities and differences, later extrapolating and generalizing for ‘masses’ and society. The ‘distance’ between story of Yasmin and M Wasay et al was created by the difference between the methodologies of research. Through quantitative research, Wasay and his colleagues could reach wider range of population, get some hard data and make it meaningful for those who were associated to this disease somehow. However, the story of Yasmin per se takes us into a different world altogether, a world of social systems and beliefs, of families and their thinking, of individuals and their response to different calamities of life, of developing world and their healthcare systems. When the data was shared with patients and their families, there was a big ‘so what’ question mark in their eyes. “What is in it for me?” “I have a story to tell and I am looking for a solution to my problems” was a standard response. They wanted the researchers to see the situation through their eyes. Some were even keen to take researchers to their homes and shared their life with them. In the beginning of research carried out by Wasay et al, there was a clear ‘research strategy’. The theory was “Multiple Sclerosis is a known and established disease of the world”. Hypothesis to be tested was “If it exists in the rest of the world, it must exist in Pakistan”. The research design thus revolved around finding the true “incidence” and “prevalence” of this disease but this was not possible in the available resources and time. So, a shortcut was designed by selecting specific research sites where the chances of finding such patients was thought maximum. A simple instrument was designed that could record all the ‘measurable’ information. Data collected was processed, analyzed, written down and published. During this entire process, if the researchers were asked to describe the ‘feelings’ of subjects under study, they would simply say “this was not the design of research”. Thus the researchers could not distinguish people and social institutions from ‘the world of nature’. The researchers believed that ‘the principles of scientific method can and should be applied to all phenomenon that are the focus of investigation. The researchers in this case turned a ‘blind eye’ to the difference between the social and natural world. In the words of Schutz, “it means ignoring and riding roughshod over the fact that people interpret the world around them, whereas this capacity for self – reflection cannot be found among the objects of natural sciences”.
Another aspect that created the ‘distance’ between the researchers and subjects in this case was a sense of precision and accuracy in the research design. While the researchers had a lot of ‘yes’, ‘no’ in their questionnaire, the subjects being humans, had a tendency to talk more or try to describe what a ‘yes’ or ‘no’ would mean to them but since they were limited by the questions with fixed choice answers, important information could have been missed out. The story of Yasmin came without reliance on instruments and procedures and therefore the connection between research and everyday life remained intact. Usually the undue reliance of quantitative research on administering research instruments to subjects or on controlling situations to determine their effects may lead to incorrect information. For example if a question was put to Yasmin “Would you like to share your health status with public at large through media?” she may immediately say “yes” but in reality, talking about Multiple Sclerosis would mean that she would probably have a difficult social and marital life or may never get married altogether for the rest of her life as people around her would consider her a life long liability. The point here is that subjects sometimes do not fully understand the questions and their implications in a structured setting whereas in an open setting, the same issues can be explored and put into perspective. The abstract of the article published by Wasay et al looks very ‘scientific’, very ‘natural’ yet very ‘static’. It seems as if they are talking about molecules, atoms, chemicals, metals and electrons and not about human individuals. The denotation of events to humans is ignored. The connection of findings to everyday contexts is not known. It has in fact created a ‘static social world’ that is mutually exclusive of people who are its constituents. Advocates of qualitative research or antagonists of quantitative research put forward the concept of empathy or ‘take the role of the other’. In other words one has to ‘stand in the shoes’ of the researched to understand their concepts better. This is only possible when there is no distance between the researcher and researched. Again, the story of Yasmin and similar other stories would never have emerged if the researchers from the very beginning would have stayed at a ‘distance’ from the subject in order to follow a certain methodology of research instead of undertaking research per se. Another key advantage of quantitative research is its objectivity. However objectivity is too plain and utterly inhuman. In the above example, Wasay et al fail to recognize the cognitive and intellectual processes going on in the minds of their subjects while collecting this data. On the other hand subjective data will lead to emotional stories and description of the new and challenging social processes engulfing the lives of Multiple Sclerosis patients, their families, physicians and society. One of the reasons why quantitative research creates a distance between researcher and researched is the underpinning belief that quantitative research can be replicated whereas this is not the case with qualitative research. The core reason for this is the ‘disconnection’ between researcher and subjects on a human and emotional level. Researchers, being human themselves, are likely to be carried away while undertaking research on emotional issues like
the one described in case of Yasmin. It can therefore be argues that some distance is essentially required to collect meaningful information. However, similar stories revealed a lot of common areas of discussion and future course of action thereby bolstering the idea that qualitative research had a key role to play as well In conclusion, there is no right or wrong argument in favor of or against the ‘distance’ that quantitative research established between the researcher and the researched. It all depends on the nature of research. It is sometimes necessary to keep a distance while it is sometimes necessary to remove it. The process may go in full circle will repetitions of distance and no -‐ distance (Qualitative may follow quantitative or Vice Versa) as shown by example. Given below is a summary of advantages and disadvantages and the researcher has to choose a methodology that suits the requirements of research in a given time and space. Summary: Advantages Disadvantages 1. Collection of hard core objective data 1. Leave out human element 2. Data collected is close to ‘natural 2. Fail to understand and record feelings sciences experimentation’. 3. Emotionally involved researcher likely 3. Does not allow ‘drifting’ in emotion to misread important information 4. Data collected is verifiable 4. Replication is a question mark because 5. Replication possible of dependence on researcher 6. Data can be processed and analyzed observation and finding using sophisticated statistical tools and 5. Generalization is difficult with computer soft-‐wares like SPSS emotionally loaded responses. 7. Saves researched from information 6. Descriptive information is scanty that is overload deemed necessary while explaining 8. Extrapolation of data is acceptable findings from quantitative data. 9. Larger scale generalization of concepts
Section B 4a) Are the following variable names acceptable by SPSS? Provide explanations in your answers. i) var 1 ii) earningsaftertax iii) pt/job iv) var_0001 “Var 1”, “pt/job” and “Var_0001” are not acceptable by SPSS as it does not entertain spaces, signs or symbols between, before or after each variable. When these three were entered at variables in the “Variable view”, a window “variable contains an illegal character” appeared promptly. Variable “earningsaftertax” was acceptable as a variable. 4b) If you are given the task to code up a set of survey data which consists of mainly attitudinal and open-‐ended responses, what cautions would you bear in mind when you carry out the task? Coding for open – ended questions is complicated. Answers to open – ended or attitudinal response should be summarized into a number of different categories or groups before entering data in SPSS. Groups are easier to make once the researcher has gone through or enlisted all the responses to a specific question from the respondents. These categories or groups then need to be assigned numbers for entry into SPSS and record must be maintained so that it can be referred to during explanation. Take this question as an example: Q. What do you think is the root cause of terrorism in Afghanistan? The responses can be many but mostly would fall into: a) Local Culture/Fragmented Society b) Religious Extremism c) Poverty d) Lack of Education e) Lack of employment f) Geostrategic influence
Almost 80 to 90% of responses would fall into either of the above but there would be certain answers that fall into neither. For those, another category (Others) can be made and coded as following: a) Local Culture/Fragmented Society = 1 b) Religious Extremism = 2 c) Poverty = 3 d) Lack of Education = 4 e) Lack of employment = 5 f) Geostrategic influence = 6 g) Others = 99 This data can then be entered in SPSS and analyzed further. 4c) When do we use Cronbach’s Alpha? What can Cronbach’s Alpha tell us? We use Cronbach’s Alpha to ascertain the degree to which the items that make up a scale are coherent with one another. Cronbachs alpha is a test of internal reliability or consistency and is used to measure how well a set of items (or variables) measures a single unidimensional latent construct. It calculates the average of all possible split – half reliability coefficients. Cronbach’s Alpha ranges between 0.00 to 1.00. A reliability coefficient of .70 or higher is considered "acceptable" in most social science research situations(DeVellis 2003) however Bryman (Alan Bryman 2008: 151) advocates a figure of 0.80 to be ‘typically employed as a rule of thumb to denote an acceptable level of internal reliability’. A minimum level of 0.60 has been described as ‘good’ by Berthoud (2000b: 169). When data have a multidimensional structure, Cronbachs alpha will usually be low. Briggs and Cheek (1986) have recommended inter – item correlation of 0.2 to 0.4 as an optimal range. Cronbachs alpha can be written as a function of the number of test items AND the average inter-‐correlation among the items. The formula for the standardized Cronbachs alpha is: where N is equal to the number of items, c-‐bar is the average inter-‐item covariance among the items and v-‐bar equals the average variance. The formula indicates that if we increase the number of items, Cronbachs alpha will increase. Alpha will be low if the average inter-‐item correlation is low. Cronbach’s alpha increases if the average inter-‐item correlation increases.
If the inter-‐item correlations are high, then there is evidence that the items are measuring the same underlying construct. This is really what is meant by "high" or "good" reliability when a reference is made to a unidimensional latent construct. Cronbachs alpha will generally be low for all items in case of a multi-‐dimensional data. For such cases, it is suggested that a factor analysis be carried out to see which items load highest on which dimensions and then take the alpha of each subset of items separately. Bibliography: 1. Acton C., Miller R., SPSS for Social Scientists, Pelgrave, Second Edition, 2009 2. Bezerra R. F., Jalloh, S. and Stevenson, J. (1998) “Formulating Hypothesis Graphically in Social Research’, Quality and Quantity 32(4):327 – 353 3. Bryman A., Social Research Methods, Oxford University Press, 3rd Edition, 2008. 4. Bryman, A. and Cramer, D. (1990) ‘Concepts and Their Measurement’, in Quantitative Data Analysis for Social Scientists, pp.61-‐74. London: Routledge 5. CLMS, (Version No. 2) Doctorate in Social Sciences, Module 1, Unit 3 6. De Vaus, D.A. (1996) “Developing Indicators for Concepts”, in Surveys in Social Research, (4th Edition), pp.47 – 59. London: UCL Press. 7. Dijkstra, W., Smit, J.H. and Comijs, H. C. (2001) ‘Using Social Desirability Scales the Research among the Elderly’, Quality and Quantity 35(1):107-‐115 8. Drennan, J. (2003) Cognitive Interviewing: Verbal Data in the Design and Pre – Testing of Questionnaires’, Journal of Advanced Nursing Methodological Issues in Nursing Research 42(1): 57 – 63 9. Kraus, L. and Augustin, R. (2001) ‘Research Report – Measuring Alcohol Consumption and Alcohol Related Problems: Comparison of Responses from Self – Administered Questionnaires and Telephone Interviews’, Addiction 96(3):459 – 471 10. Palant J., (2007) SPSS Survival Manual, Third Edition, McGraw Hill, Open University Press 11. Schutz, A. (1962), Collected Papers I, The Problem of Social Reality (The Hague: Martinus Nijhof) 12. Wasay, M. et al, Multiple Sclerosis in Pakistan, Multiple Sclerosis, Vol. 13, No. 5, 668-‐669 (2007) 13. Wright Mills, C. (2000) ‘Abstracted Empiricism’, in The Social Imagination, (40th Anniversary Edition), pp. 50 – 75. New York; Oxford: Oxford University Press