11.vol 0005www.iiste.org call for paper_no 1-2_ pp. 25-64
Issues in Social and Environmental AccountingVol. 5, No. 1/2 December 2011Pp 25-64 Quality in Qualitative Studies: The Case of Validity, Reliability and Generalizability*) Azham Md. Ali1**) Hamidah Yusof Faculty of Management and Economics Universiti Pendidikan Sultan Idris MalaysiaAbstractThe concern over the subject of quality judgement of qualitative research has led to the emer-gence of several varied positions. Whichever labels used in describing the positions, each has adistinctive character. To attempt to an understanding of the meaning of the quality criteria ofvalidity, reliability and generalizability in qualitative research, this paper provides a review ofrelevant literature of recent years. To check on the extent of their reporting in research papers, atotal of 15 accounting research recently published in top-ranked accounting journals have beenselected. Except for one paper, the rest has failed to provide detailed discussion on the validity,reliability and generalizability of their research. But this would not necessarily mean that theconcerned researchers have disregarded the aspects of validity, reliability and generalizabilityduring the conduct of their research. Perhaps the only way to find out the truth is by asking theresearchers themselves the relevant questions.Keywords: qualitative research, quality, validity, reliability, generalizability, accounting1 Azham Md. Ali is currently a Professor of Accounting and Finance at the Faculty of Management and Economics atthe Universiti Pendidikan Sultan Idris, Tanjung Malim Perak. Prior to his current appointment, he was attached to theUniversiti Utara Malaysia, Sintok Kedah for which he had served for close to 25 years. His areas of interest are audit-ing, international accounting, accounting history and qualitative research. To date, he has published in numerous localand international journals and presented papers in various conferences and seminars.*) A slightly different version of the paper with the same title was presented as Invited Paper II for The Second Na-tional Seminar on Qualitative Research, 4-5 December 2006 at Kompleks Dewan Kuliah Sains, Fakulti Sains, Uni-versiti Malaya. The seminar’s organizers are the Qualitative Research Association of Malaysia (QRAM) and InstitutPengurusan Penyelidikan dan Perundingan, Universiti Malaya.**) The following people from the Faculty of Accountancy, Universiti Utara Malaysia (UUM) have been quite suppor-tive in different ways over the successful completion of the original version of the paper (which was presented as In-vited Paper II for The Second National Seminar on Qualitative Research, 4-5 December 2006 organized by the Qualita-tive Research Association of Malaysia and the Institute of Research Management and Monitoring, University of Ma-laya): Noor Yusni Md. Yusuf, Azharudin Ali, Dr. Nor Aziah Abd. Manaf and Dr. Mohamad Hisyam Selamat. Thus, themain author is very much appreciative of their support. Since the opportunity to work on the paper has come from Dr.Siti Nabiha Abdul Khalid from Universiti Sains Malaysia (USM) and Dr. Aini Hassan from Universiti Malaya (UM),the main author is also very much grateful to them. Finally, a lot of gratitude is for the author’s former PhD supervisorProf. Stuart Turley from the University of Manchester, UK who in 1995 had been quite understanding enough in allow-ing him to change the research approach (following the change in the research question raised) from quantitative toqualitative and which has led him to come to know the very interesting world of qualitative inquiry.
26 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 At present, there is an observable “validity” and “reliability” or trend that qualitative research other concepts like “consistency” attracts students as well as practi- or “adequacy of meaning” or tioners who are not at ease with even others. demanding quantitative proce- dures and who expect that in Eberle (2005, p. 4) qualitative research they can sub- stitute methodological sophistica- 1. Introduction tion with common-sense. The more fashionable qualitative re- Quality concerns play a central role search has become in different throughout all steps of a research proc- fields, like social work, business ess from the inception of a research administration, marketing and question and data collection, to the consulting, the more we face a analysis and presentation of research quality problem. Any investiga- findings. Following this line of thought, tion which does not make use of there is no question that quality is an statistical procedures is called integral and explicit part in qualitative “qualitative” nowadays, as if this inquiry. However, the concern over the were a quality label in itself. As subject of quality judgement of qualita- many have already called for: we tive research has led to the emergence of need to define criteria by which several varied positions. See Table 1 we can distinguish “good” from below. “bad” qualitative research, be it Table 1 Quality Judgement PositionsLiterature Quality PositionDenzin (1995) positivism, postpositivism, postmodern and postfoundationalOnwuegbuzie (2002) positivists, postpositivists, poststructuralists and postmodernistsLazaraton (2003) foundationlism (positivism/rationalism), quasi-foundationalism (postpositivism, constructivism) and nonfoundationalism (postmodernism)In sentence form, these quality positions modifying quantitative researchmay be understood as the followings: criteria to produce criteria fit for qualitative research context.y There is only one way to judge the y The way to judge the quality of quality of qualitative studies qualitative studies is by develop- which is the same for any type of ing a unique set of criteria. scientific inquiry: the criteria of reliability, internal and external Other than these four quality positions, a validity and objectivity. qualitative study may also be judged ony There is no way to judge the qual- its quality through ity of qualitative studies. y the conduct of a deductive quanti-y The way to judge the quality of tative study to test out the emerg- qualitative studies is through ing theory.
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 27This paper is an attempt to understand plausibility, validity, validation, andthe meaning of the quality criteria of credibility. Various authors havevalidity, reliability and generalizability constructed diverse typologies ofin qualitative research. Following that validity (e.g., Maxwell’s five types,attempt, the paper focuses on the report- 1992; Lather’s four frames, 1993;ing of these and other quality criteria in and Schwandt’s four positions,published studies. In order to come to 1997). It is little wonder that Don-some understanding (or perhaps utter moyer (1996), who wrote and edito-confusion?) of what these three terms rial on validity in the Educationalstand for, there is the review of selected Researcher, commented on the di-recent literature. The review is done verse perspectives of validity… Nov-with no intention whatsoever to provide ice researchers, in particular, cananything new or different in regard to become increasingly perplexed inthe meaning of validity, reliability and attempting to understand the notiongeneralizability of qualitative studies as of validity in qualitative inquiry.compared to those offered by variousparties to date. To name just four of So, while the paper unlike so many oth-these parties, these would be Davies and ers such as Whittemore et al. (2001)Dodd (2002), Golafshani (2003), Kvale does not provide new or different mean-(1995), and Whittermore, Chase and ings to what is currently understood toMandle (2001). be validity, reliability and generalizabil- ity in qualitative inquiry, it is on theHence, unlike Davies and Dodd (2002), other hand pretty much like Morse, Bar-Golafshani (2003) and Kvale (1995), the rett, Mayan, Olson and Spiers (2002)paper does not take a critical focus on which propounds the implementation ofthe inadequacy of applying a quantita- the so called verification strategies dur-tive concept of rigour to evaluate quali- ing the conduct of a qualitative inquirytative research. In short, it does not at- to ensure the attainment of rigor or qual-tempt to redefine the concepts of reli- ity in such work. That said, the paperability, validity and triangulation. Also, does not make the same kind of stancethe paper does not see the need to add to as that of Morse et al. (2002): a return toor subtract from the present plethora of terminology for ensuring rigor in quali-meanings of validity, reliability and gen- tative inquiry that is used by mainstreameralizability of qualitative studies. This science, i.e., validity and reliability. Inis done so that the paper is never in dan- the final analysis, the ultimate aim of theger of introducing yet more confusion paper is as mentioned succinctly byinto the already turbulent waters of the Seale (1999, p. 465):meaning of quality in qualitative studies!Related to this, note the following com- A lot of effort has been expended bying from Creswell and Miller (2000, p. methodologists over the years, try-124): ing to give some guidance to quali- tative researchers in improving or … readers are treated to a confusing judging the quality of qualitative array of terms for validity, including research. You could say that all authenticity, goodness, verisimili- methodological writing is ultimately tude, adequacy, trustworthiness, directed at such a goal, because the
28 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 idea of writing about how one can Later, and in a more recent time, Irvine do research is presumably aimed at and Gaffikin (2006) had raised this mat- giving other people some good ideas ter by having it placed in a bigger con- on how they might proceed with text. They said (Irvine and Gaffikin, their own studies. 2006, p. 115): “While much qualitative research has been undertaken within theApart from reviewing selected works on discipline of accounting, little or no in-the subject of quality criteria of validity, tention has been paid to the way inreliability and generalizability which which that research has been con-provides the picture that there is much ducted.”confusion of what these terms actuallymean in qualitative inquiry, the paper Aside from the analysis done on a selec-gives focus to the reporting of these tion of qualitative accounting papers, aquality criteria in a number of published total of twelve qualitative studies thatresearch papers. One set of papers is are concerned with various other fieldscomprised of qualitative accounting re- in social sciences have been analyzed insearch recently published in top-ranked terms of their reporting of the qualityaccounting journals. It appears that the criteria. Out of these twelve, five arepaper provides the only attempt to date PhD thesis while an additional three areover such analysis of qualitative studies journal writings whose discussion on thein accounting. variety of quality criteria are concerned specifically with the use of specific re-Perhaps, this is not surprising since over search methods such as convergent in-the last two decades there appears to be terviews for data collection or analysis.mere handful of published papers around These published works are chosen forwhich give focus on the subject of qual- analysis since they would provide someity of qualitative studies in accounting. of the best portrayal of such reporting.These would include Ahrens and Chap- Hence, they can be the models for thoseman (2006), Atkinson and Shaffir researchers who are looking for the ac-(1998), Lillis (2006), McKinnon (1988) ceptable ways in reporting the qualityand Modell (2005). In fact, in what ap- criteria which may or may not be thepears to be one of the earliest writings validity, reliability and generalizabilityon the subject matter of validity and reli- that are the focus of the paper.ability in qualitative accounting studies,the late McKinnon mentioned over two By having this analysis done togetherdecades ago the following (McKinnon, with that for a selected few accounting1988, p. 34): papers, it is hoped that the paper shall be able to contribute to improved practices … field studies are frequently sub- in qualitative studies for those new in jected to common and global criti- the field including doctoral candidates. cisms of their apparent inability to For the doctoral students in particular attend to such research criteria as they need no further reminder that dem- validity and reliability … many pub- onstrating the trustworthiness of one’s lished field studies in accounting do thesis is a requirement. not report how issues of validity and reliability are addressed. By and large, by focusing on the report-
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 29ing of quality criteria in a total of 27 is concerned, particular craft skills suchpublished studies so that budding re- as member checking, accounting forsearchers in qualitative inquiry may be negative instances, analytic induction,assisted in their work, the paper attempts the uses of numbers, using low inferenceto be on the side propounded in Seale descriptors, the grounding of theory,(1999) as opposed to those mentioned deconstructive approaches, reflexiveearlier as the varied positions on quality accounting and new textual forms ofin qualitative studies. Specifically, Seale reporting do not have to be linked inex-(1999) emphasizes the need for re- tricably to particular philosophical orsearchers to gain the so called paradigm positions. In the final analysis,“apprenticeship experiences” as opposed he considers a major threat to quality isto “intense methodological awareness”. the idea that research must be carried outHe wrote (Seale, 1999, pp. 475-476): under the burden of fulfilling some phi- losophical or methodological scheme. Methodological writing is of limited Instead, in his view, what should be the use to practicing social researchers, case is simply this (Seale, 1999, p. 472): who are pursuing a craft occupation, “Practicing social researchers can learn in large part learned “on the job,” to do good work from a variety of exam- through apprenticeship, experience, ples, done within different “moments,” trial, and error rather than by study- without needing to resolve methodologi- ing general accounts of method … cal disputes before beginning their Intense methodological awareness, work.” if engaged in too seriously, can cre- ate anxieties that hinder practice … Finally, it perhaps needs to be noted that people learn how to do research the analysis done on the reporting of through apprenticeship experiences, validity, reliability and generalizability fortunately possible to have by read- in qualitative accounting papers is con- ing others’ work rather than actually cerned with only a few of the relatively going and sitting at their feet large number of these papers. A differ- (although this also can be useful). ent sample might have given a some- Any contemplation of other people’s what different picture. Also, it perhaps research work, if it involves thinking needs to be stated out that the rudimen- seriously about its strengths and tary analysis done for this paper on the weaknesses, can be this kind of vi- reporting of quality criteria in fifteen carious apprenticeship experience. qualitative studies from the accounting field and twelve others from variousAll in all, Seale disagrees with the idea other fields in the arena of social sci-that philosophical, political, or theoreti- ences could very well move to the nextcal positions ought to determine the de- level with the focus on the specific re-cisions that social researchers make “on search strategies undertaken. Such fur-the ground” so that quality is underwrit- ther study with perhaps a much largerten by adherence to a particular position. sample of published studies could followInstead, in his view research practice in the footsteps of Barusch, Gringeri andshould be conceived as relatively George (2011) for the field of socialautonomous from such abstract and gen- work and Suter (2010) for personal rela-eral considerations. In short, as far as he tionship.
30 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64In Barusch et al. (2011), they use Cres- validity, the generalizability of the re-well’s (2007) eight strategies as the sults; reliability, the extent to which thebenchmark for rigour in assessing a ran- findings can be replicated; objectivity,dom sample of 100 qualitative social the extent to which the findings are freework articles drawn from selected social from bias. As perhaps to be expected,work journals. As for Suter (2010), the those who do not consider themselvesstrategies for validity that the so called positivists oppose such views.“validity processes” are judged againstcome from various methodological writ- Janesick (1994, p. 217) challenges theings in the field. A further study in the notion that the “trinity of validity, gener-manner of Barusch et al. (2011) or that alisability and reliability”, terms usuallyof Suter (2010) in a field such as ac- synonymous with the quantitative para-counting should be able to deepen un- digm, should be strictly applied to allderstanding on the extent of rigour or research. A decade later, Morgan andquality in qualitative studies reported in Drury (2003) who agree with her explainpublished works. why: in qualitative research, the re- searcher is more interested in question-The rest of the paper is divided into ing and understanding the meaning andthree sections. The next section covers interpretation of phenomena. But thethe varied understanding of validity, reli- issue involving these quality criteriaability and generalizability in qualitative cannot easily be pushed away with suchresearch – together and separately. The remark. This can be seen in the efforts ofsection which comes right after focuses others working in qualitative inquiry inon specific references made to these coming out with new terms to replacequality criteria in recent published pa- validity, reliability and other terms usedpers in the accounting field. The last in quantitative inquiry. Among the nota-section is the discussion and conclu- ble ones are Guba and Lincoln (1981),sions. As part of this third and last sec- Lincoln and Guba (1985), and Whitte-tion, there is the inclusion of the analysis more et al. (2001).done on the reporting of quality criteriaby a total of twelve published works in In the case of Guba and Lincoln (1981),social sciences. they propose that the criteria to reach the goal of trustworthiness in qualitative inquiry are credibility, fittingness,2. Validity, Reliability and Gener- auditability and confirmability. Thesealizability are as opposed to the criteria internal validity, external validity, reliability andAs mentioned in the introduction, there objectivity to reach the goal of rigour inare varied positions over quality consid- quantitative inquiry. A few years later,eration for qualitative research. For they suggest the criteria to now be credi-many positivists, they feel that if a re- bility, transferability, dependability andsearch does not satisfy several criteria, confirmability (Lincoln and Guba,then it is not true research. These criteria 1985). Not every one agrees howeverare (Guba and Lincoln, 1994): internal with these ideas of theirs. See for exam-validity, the degree to which the results ple Hammersley (1992), Kuzel andcan be attributed to treatment; external Engel (2001) and Yin (1994). Perhaps in
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 31regard to Yin (1994) the opposition is positivism; while we have nothingnot surprising. This is because he de- against coining new terms, particu-scribes trustworthiness as a criterion to larly if this would avoid the concep-test the quality of research design and tual baggage that may be attached tonot a goal of the research as proposed by a certain terminology, we believe thatGuba and Lincoln (1981) and Lincoln we may want to examine existingand Guba (1985). tools before adding new terms to po- tentially similar concepts.Also, over two decades later, there is aclear opposition among some parties in Later at the end of their writing, theythe use of these terms to describe the mention the following (Bergman andquality criteria in qualitative inquiry. Coxon, 2005, p. 13):Note the following coming from Morseet al. (2002, p. 3): “… the broad and ab- Whether or not researchers coin theirstract concepts of validity and reliability own terminology because they rejectcan be applied to all research because constructs that may have emergedthe goal of finding plausible and credible from another epistemological tradi-outcome explanations is central to all tion, or whether they begin theirresearch.” Later, in the same work, the quality considerations by adoptingfollowing is mentioned (Morse et al., the existing terminology is not im-2002, p. 14): “Our argument is based on portant at this point. Instead, it is thethe premise that the concepts of reliabil- accountability of research practicesity and validity as overarching con- through explicit description of re-structs can be appropriately used in all search steps, which allow an audi-scientific paradigms because, as Kvale ence to judge the plausibility of a(1989) states, to validate is to investi- particular study and its findings.gate, to check, to question, and to theo- (Emphasis added.)rise. All of these activities are integralcomponents of qualitative inquiry thatensure rigor.” ValidityAlso note the following which is men- To understand what validity is in a re-tioned more recently by Bergman and search inquiry, one only needs to refer toCoxon (2005, p. 3): “Quality considera- the experts. But it seems that is not suchtions in empirical research tend to be a good idea as far as qualitative studiesaddressed by the concepts “validity” and are concerned! For at least two main“reliability”… We will critically exam- reasons. First, the experts themselvesine some possibilities of these concepts have failed to be consistent. Second,…” Next, in the attached footnote num- validity in qualitative inquiry comesber 3, the following is what they say: about in so many different ways of un- derstanding. When it concerns the first It has often been suggested that these reason, two fine examples are Harry terms are inappropriate since they Wolcott and David Silverman. As for have emerged from a positivistic tra- the second reason, the recent works by dition. However, we argue that con- Spencer, Ritchie, Lewis and Dillon cerns about data quality transcend (2003) and Onwuegbuzie (2002) are
32 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64proof enough. beginning writing early and sharing ideas with others in the setting; lettingThe first of two examples of experts in readers ‘see for themselves’; reportingqualitative field who fail to be helpful fully; being candid; seeking feedback;over the subject of validity is Harry trying to achieve a balance through rig-Wolcott who for over three decades, as orous subjectivity; and, writing accu-the master ethnographer, has chartered rately.an interpretive, postpositivist approachto the anthropology of educational prac- In the case of Silverman, as compared totices (Denzin, 1995, pp. 181-182). But Wolcott, he provides a more recent ex-with his 1994 publication, calling for ample of resistance and confusion overmethods and texts that produce under- the issue of validity in qualitative re-standing, not validity, he states (Wolcott, search. This is as far as Silverman1994, p. 369): “I do not accept validity (2001) is concerned that is comprised ofas a valid criterion for guiding or judg- among others a chapter on validity, reli-ing my work.” In his 1990 publication ability and generalization. As Kalekin-earlier (as found in Onwuegbuzie, 2002, Fishman in her review of this work putsp. 9), he in fact raises the question as to it (Kalekin-Fishman, 2001, p. 3):whether validity is appropriate, legiti-mate or useful in qualitative studies. In After acknowledging and detail-his view, it seems validity interferes ing the weaknesses of the concep-with his goal of understanding the un- tualization of reliability and valid-derlying phenomenon. This has led On- ity on the home ground of quanti-wuegbuzie (2002, p. 9), in commenting tative research, Silverman insistsover this stance of Wolcott, to mention on applying these same terms forthe following: “According to Wolcott, deciding the quality of every kindunderstanding is a more fundamental of research. He is, for example,concept for qualitative research than is adamant about testing for validity,validity. Consequently, he attempts to i.e., for the “truthfulness” of evi-understand what is occurring rather than dence, even though he agrees withto convince his audience.” most qualitative theorists that this is highly dubious goal.Nonetheless, Wolcott seems unable todismiss validity outright. As pointed out Indeed, if a check is made on the sug-by Spencer et al. (2003, p. 59), Wolcott gestions he made in the book regardingis “[p]erhaps the most frequently cited the attainment of validity in qualitativeexample of someone who apparently studies, one can find the followings: arejects validity while retaining its under- reliance on theoretical models to escapelying concerns …” (Emphasis is in the cultural bias and methods such as ana-original.) Next, they specify that Wol- lytic induction, constant comparativecott (after saying that he cannot see any method and deviant case analysis. Suchplace for validity in his work) has in fact ideas have led Kalekin-Fishman to saymade suggestions on how to produce the following (Kalekin-Fishman, 2001,valid qualitative works. These are p. 3):(Spencer et al., 2003, p. 59): listeningmore than talking; recording accurately; By insisting that these methods as-
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 33 sure “validity”, Silverman is defend- y interpretive validity (Maxwell, ing the positivistic postulate that 1992; Miller and Fredericks, every type of social science has to be 1995) committed to discovering the truth, y validity as incitement to discourse even though throughout the book he (Lather, 1995) defends a constructionist perspective. y reflexive validity (Stiles, 1993) or (Emphasis in the original.) substantive validation (Angen, 2000)With towering personalities in qualita- y dialectic validity (Waterman,tive inquiry themselves showing evi- 1998)dence of being conflicted over validity, y critical validity (Waterman, 1998)there is perhaps little hope that minions y pragmatic validity (Kvale, 1996)in the field can be crystal clear of what y catalytic or emancipatory validityvalidity is and is not. Certainly the two (Stiles, 1993)writings referred to next could not be ofmuch help either? Spencer et al. (2003, pp. 61-62) have also concluded that the different notionsFirst, from Spencer et al. (2003) who of validity in the literature may be cate-conduct a study on quality assessment of gorised into the following areas: methodqualitative research for the British gov- or research process, status of the find-ernment, their literature search has led to ings, quality of relations with partici-more than ten kinds of validity – and pants and impact of contribution of themany of them are interchangeable with inquiry. This literature finding of theirsother terms. For example, for descriptive is in contrast to their findings from thevalidity (Maxwell, 1992; Miller and 29 in-depth interviews with government-Fredericks, 1995), the other terms pro- based commissioners and managers ofviding the same meaning are descriptive research and policy makers, other fun-adequacy (Hammersley, 1991) and va- ders of evaluation research, academicslidity at the individual level (Sykes, and practitioners involved in conducting1990). The validity which these terms qualitative research and writing aboutrefer to is concerned with researchers quality. Most of the discussion on valid-actually capturing what they intended to ity that these interviewees were con-study and accurately reporting what they cerned with is limited to the validity ofhave seen or heard. Besides descriptive interpretations and conclusions (Spencervalidity, others pointed out and given the et al., 2003, p. 64). Spencer et al. (2003,definitions by Spencer et al. (2003, p. pp. 63-64) have also found the existence61) include: of different notions of validity together with the different usage of validity termsy validity of data generation among the research participants. It (Mason, 2002) or procedural trust- seems some research participants can be worthiness (Stiles, 1993) grouped as the positivists while others,y validity of interpretation (Mason, post-positivists. 2002) or theoretical validity (Maxwell, 1992; Miller and Besides Spencer et al. (2003), another Fredericks, 1995) or explanatory recent effort which concerns validity adequacy (Hammersley, 1991) criterion is by Onwuegbuzie (2002). To
34 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64develop the so-called Qualitative Legiti- Reliabilitymation Model which attempts to inte-grate many of the types of validity iden- Participants in the research conducted bytified by qualitative researchers, he pro- Spencer et al. (2003, p. 65) mentionedvides an interesting discussion on valid- earlier view reliability in qualitative re-ity. He mentions that validity in qualita- search in the following manners: thetive research has been operationalized in reassurance that another researcher in-a myriad of ways with to date no one vestigating the same issue or workingdefinition of validity represents a he- with the same data set would derive thegemony in qualitative research. The fol- same findings. But there are researcherslowings are the definitions of validity: who have different views. Stenbacka (2001) argues that since reliability issuey consensual validity (Eisner, 1992) relates to measurements it has thus noy catalytic validity (Lather, 1986) relevance in qualitative research. Simi-y ironic legitimation (Lather, 1993) larly, Golafshani (2003, p. 601) writesy paralogical legitimation (Lather, that the concept of reliability is irrele- 1993) vant in qualitative research. He nonethe-y rihizomatic legitimation (Lather, less next mentions the following (p. 1993) 601): “To ensure reliability in qualitativey voluptous legitimation (Lather, 1993) research, examination of trustworthinessy communicative validity (Kvale, is crucial.” 1995)y action validity (Kvale, 1995) The fact that there exist conflictingy investigation validity (Kvale, 1995) views over the quality criterion of reli-y descriptive validity (Maxwell, 1992) ability in qualitative inquiry have how-y theoretical validity (Maxwell, 1992) ever failed to stop Morgan and Druryy interpretive validity (Maxwell, 1992) (2003, p. 6) in detailing out in one longy evaluative validity (Maxwell, 1992) paragraph how qualitative research can attain an appropriate level of researchFinally, as if the situation over validity is reliability. They write:not troublesome enough for the fact thatthe experts themselves have failed to be This can be achieved by explainingconsistent and that there appears to be the methodological framework andtoo many facets of validity in qualitative the range of strategies that have beenresearch, another dimension of validity used within the study. The rationaleconfusion has arisen over the years with for the way in which participantsmany researchers generating or adopting were selected to take part should alsowhat they consider to be more appropri- be described, as should the re-ate terms to describe the qualifying searcher’s role and their perceivedcheck or measure for their research. So, relationship to those participants. Itinstead of the term validity, as noted will be necessary to document ana-Winter (2000, p. 6), these researchers lytic constructs and meanings, whichrefer to ‘trustworthiness’, ‘worth’, derive from data, alongside the meth-‘relevant’, ‘plausible’, ‘confirmable’, odological approach and procedurescredible or representative’. that were used for producing data. This would include providing de-
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 35 scriptions of phenomena with appro- Generalizability priate narrative of the social context in which they occurred, particularly Typically, the word ‘generalizability’ is in terms of persons, places and defined as the degree to which the find- events. Theoretical propositions also ings can be generalized from the study need to be fully explained in terms of sample to the entire population (Polit how constructs have been formed and Hungler, 1991, p. 645 as found in through detailed procedures. Myers, 2000, p. 2). Since qualitative studies have found it difficult to achieveIn short, what they are saying is this: to this, these studies have continued to beattain reliability in research, there is a criticized for their lack of generalizabil-need for the qualitative researcher to ity. This is despite the many positivedocument the succession of moves aspects of qualitative research and thatthrough the stages of data production, there are other types of generalizabilityanalysis and interpretation. It appears which qualitative research may still sat-however that this explanation by Mor- isfy.gan and Drury (2003) is concerned withthe type called external reliability as op- In regard to the latter, Spencer et al.posed to internal reliability (Spencer et (2003, pp. 67-69) have listed them out asal. 2003, pp. 64-65). follows:Indeed, as far as Spencer et al. (2003) y representational generalization orare concerned, there are more than one generalization within a caseor two notions of reliability. So, besides (Lewis and Ritchie, 2003);internal and external reliability which y analytical or theoretical generali-are concerned with the notion of consis- zation (Glaser and Strauss, 1967;tency, there are: reliability as replication, Strauss and Corbin, 1998);inter-coder reliability and auditability, y empirical or inferential generali-dependability or reflexivity. To achieve zation (Stake, 1978; Lincoln andthe different notions of reliability, there Guba, 1985; Hammersley, 1992).are various steps which a researcherneeds to undertake. For internal reliabil- All in all, just like the criteria of validityity, for example, the researcher may un- and reliability, there is more than onedertake five different steps including type of qualitative generalization. But itusing a team of researchers and peer ex- seems to some parties these other typesamination. This is as opposed to the ex- do not exist or are not quite so signifi-ternal reliability where the researcher cance as their favourite one. For them,needs to clearly document in the writing there is only one kind of generalizabil-another five different matters. These ity, and it is this very type which qualita-matters include those quoted earlier tive studies would invariably fail to sat-coming from Morgan and Drury (2003, isfy and which relegates these studies top. 6). be among those which they consider to be lacking in rigour.
36 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-643. Validity, Reliability and Gener- close in the writing with or without spe-alizability in Accounting Research cific reference made to the quality crite- ria of interest. Such analysis needs firstIn order to see the relevance of the qual- of all the preparation of a table of re-ity criteria of validity, reliability and search strategies and the quality criteriageneralizability in accounting research, a which they satisfy. Next, there is a needtotal of fifteen papers published in recent for a production of a checklist of appro-years in the top ranked accounting jour- priate research strategies. Finally, thenals is gathered and analysed. See Ap- checklist is used as a basis for compari-pendix A. From the outset, it can be eas- son with details of research strategiesily seen that nearly half or seven of them disclosed in those four papers particu-does not even bother to make any spe- larly in their research method section.cific reference to these quality criteria. Table 2 provides an example of a set ofAs for the rest, most actually have also research strategies which need to be im-failed to say much. The exceptional one plemented for establishing rigour inis Sweeney and Pierce (2004). But even qualitative research. This list is preparedthis one has failed to refer to the quality based upon discussion in Baxter andcriterion of generalizability. Eyles (1997, pp. 506-510).Nonetheless, out of the fifteen, aside Once the comparison is made and thefrom Sweeney and Pierce (2004), four – result is known, a conclusion may thusBeattie, Fearnley and Brandt (2004); be made in regard to the application ofGendron, (2002); Gendron and Bedard, any quality criteria. That said, the lack(2006); Kwok and Sharp (2005) - man- of disclosure of the research strategiesage to provide much details on their sec- for rigour or quality in a research papertion of the research method. This is in or report may not necessarily mean thatcontrast to two - Herbach (2005), Ritten- the study has failed to implement themberg and Covaleski (2001) - which pro- during the research process. Or, there isvide quite minimal discussion on re- the possibility that the researchers andsearch method. All in all, it is just one journal editors are those who uphold thepaper - Sweeney and Pierce (2004) – quality judgement position that there iswhich may be said to have attained the no way to judge the quality of qualitativelevel of excellence regarding the quality studies!criteria. Another four papers – Beattie etal. (2004); Gendron (2002); Gendron It is just perhaps due to space limitationand Bedard (2006); Kwok and Sharp that leads to their failure in accounting(2005) – may or may not be at the same for these research strategies in the re-level. search writing. Nonetheless, for the good of everyone involves in qualitativeIn other to find out the truth for these research from authors to editors to read-four papers, there is a need for a more ers, it is perhaps worth considering thedetailed analysis over research strategies following coming from Welsh (2002, p.or practices which a qualitative re- 3):searcher would normally undertake inorder to attain rigour or quality in his or Debate on the usefulness of theher work and which he or she may dis- concepts of validity and reliability
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 37 Table 2 Strategies for Attaining Qualitative Rigour Strategies MeaningRationale for methodol- Qualitative methods are argued to be the most (or only) appropri-ogy ate way to address the research questionMultiple methods More than one method used for studying the problem (e.g. in- depth interviews plus participant observation plus textual analy- sis)Respondent A description of the group(s) of respondents (e.g. number and gender ratio is given)Interview quotations The words of the respondent may be read or the presentation of verbatim quotationsInterview practices Details of how interviews were conducted (e.g. use of interview schedules are provided)Procedures for analysis A description of how data were converted or condensed into theoretical constructs is givenImmersion of lengthy It is argued that long fieldwork develop rapport with respondentsfieldwork and / or enable deep understandings of the research situationRevisits Revisits to respondents are made usually to clarify meanings and build rapportVerification by respon- Respondents were contacted to verify interpretations or meaningsdentsAppeals to interpretive An existing theory is supported or refuted by the findings, i.e.,community there is more than reference to the literatureRationale for verification Rationale for showing that there is agreement between constructs or interpretations and the meanings held by respondents is pro- vided in qualitative research has been carried out in a thorough and undertaken for many years …. transparent manner …. However, Some researchers suggest that in most published research it is whilst these terms are inappropri- unusual to find accounts of ex- ate in qualitative research, prefer- actly how researchers analysed ring to use terms such as their data and it is partly because “trustworthiness”, “rigorousness”, of this missing information that or “quality” of the data, it is nev- this research tradition has been ertheless important that qualita- open to allegations of tive research and data analysis are “unthorough” research practices.
38 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64That Welsh (2002) is not alone in having may be said for the chart (p. 73) in Shen-such view should not perhaps be surpris- ton (2004). See Appendix B for a sectioning. A more recent writing by Sinkovics, of this Table II and the chart.Penz and Ghauri (2005, p. 32) seems to It is also notable that Baxter and Eylespoint to the same direction: (1997) have provided a list of eight questions which they referred to in the Qualitative research methodology latter half discussion of their paper. It … is often criticised for high lev- seems an alternative move available in els of subjectivity and low reli- case a list shown in Table 2 earlier ap- ability and validity. On a substan- pear wanting to some whose qualitative tive level this criticism is unfair studies use interviews as the main because qualitative research of- method of data collection. Specifically, fers holistic perspectives on phe- it involves answering these eight ques- nomena which cannot be tions ranging from what was the natural achieved otherwise. However, history of the research to what results criticism is often due to a low are presented to how transferable are the quality of documentation and re- findings (Baxter and Eyles, 1997, pp. porting of the findings cannot be 511-520). See Appendix C for the full ignored. While quantitative stud- listing of the questions. The following is ies follow a rigorous organisation what Baxter and Eyles (1997, p. 520) and presentation in how results say regarding these questions: “It serves are presented, qualitative studies not only as a guide for what to look for are often reported in a descriptive but reveals where gaps exist in reporting and narrative way. information necessary for ascertaining rigour.” It is also interesting to note that Baxter and Eyles (1997, p. 520) have4. Discussion and Conclusions further proposed that the four trustwor- thy criteria and the corresponding strate-With or without specific references gies developed by Lincoln and Gubamade to quality criteria such as validity, (1985) be incorporated into the researchreliability and generalizability in the re- process as a basis for answering theseporting or publication of a piece of questions.qualitative research, a conscientious re-searcher who are looking for rigour or Other than Baxter and Eyles (1997),quality in their work would be in no Greenhalgh and Taylor (1997) and Maysdoubt see the need for the implementa- and Pope (2000) are other examples oftion of the necessary research strategies those who produce lists of questionsor practices. Lincoln and Guba (1985), which may be asked over a piece ofBaxter and Eyles (1997), Morse et al. qualitative research. Greenhalgh and(2002), Horsburgh (2003) and Shenton Taylor’s (1997) list of nine questions is(2004) are some of the writings which however more appropriate for those inprovide extensive discussion of these medical field. As for Mays and Pope’sresearch strategies and the quality crite- (2000) list, check out what they say (p.ria which they satisfy. In particular, for 52): “We list some questions to ask forBaxter and Eyles (1997), their Table II any piece of qualitative research (box);(p. 512) is quite revealing. The same the questions emphasise criteria of rele-
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 39vance and validity. They could also be Finally, despite the various positions onused by researchers at different times quality judgement and that different setsduring the life of a particular research of quality criteria appear to exist for dif-project to improve its quality.” Mays ferent kinds of qualitative studies, thereand Pope’s list may be found in Appen- are around more than a few qualitativedix C. studies which can be considered quite enlightening for the fact that they areThough the presence of checklists able to balance the reporting of the storyshould make it easy for researchers to with that of the research process under-identify which steps to be taken prior to taken. In reporting the research process,the undertaking of a research or which these papers have also made specificsteps are yet to be implemented while references to quality criteria such as va-conducting the research, several writers lidity, relevance and generalizability tohave pointed out the problems that they varying degrees. Table 3 provides a listmay create for the same researchers. See of some of these works and their rele-Barbour (2001) and Chapple and Rogers vant details.(1998) on what these problems are. Not-withstanding these problems, it seems At least two other types of writing aretheir presence cannot be worse than that also available which make specific refer-of different sets of quality criteria for ences to various quality criterion includ-different types of qualitative studies ing validity, reliability and generalizabil-which some parties have suggested to ity in their discussion. The first is con-exist. See Chapple and Rogers (1998), cerned the conduct of qualitative re-Klein and Myers (1999), Healy and search by students doing PhD. The sec-Perry (2000) and Lilford, Edward, ond concerns the application of specificBraunholtz, Jackson, Thornton and types of research method for data collec-Hewison (2001). This myriad of quality tion and analysis in qualitative research.criteria could very well be one of the Table 4 and Table 5 provide details ofreasons leading to researchers failing to the specific examples of the respectivemake specific references to validity, reli- types of writings.ability, generalizability or any other cri-teria in their writings. And these re-searchers may include Beattie et al. All in all, it may safely be said that the(2004), Gendron (2002), Gendron and qualitative writings listed in Table 3,Bedard (2006) and Kwok and Sharp Table 4 and Table 5 are some of the best(2005) mentioned earlier. It is just per- examples as far as the incorporation ofhaps too troublesome for them to refer to specific references on quality criteria inthese criteria in their papers when what the reporting of qualitative research isis important is that they conduct their concerned.studies as expected for rigour and thatthe research strategies performed are All is apparently not lost!reported to the extent possible.
40 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 Table 3 Enlightened Qualitative StudiesStudy / Scope Field Type of Primary Methods Quality Crite- Study for Data Collection ria Referred / Data Analysis To (number of para.)Cregan (2005) / Labour Induction Postal Survey / ValidityAustralia Content Analysis, Reliability Hierarchical Cluster (several Analysis, Textual pages!) AnalysisEfinger, Maldonado, PhD Phenomenol- Structured, Open- 3 Trustworthi-McArdle (2004) / courses ogy Ended Questionnaire ness*United States - Construc- / Content Analysis 5 Authentic- tivist ity** (5)Waldman, Lituchy, Quality Multiple Open-Ended AccuracyGopalakrishnan, La- Case-Study Interviews/ Objectivityframboise, Galperin Pattern-Matching Reliability and Kaltsounakis Validity (3)(1998) / UnitedStates and CanadaRiley (1995) / Tourism Situational- Long Interviews / Credibility,United States ism Grounded Theory Dependability, Confirmability (3) Note: * Dependability, Tranferability and Credibility ** Fairness, Ontological Authenticity, Educative Authencity, Catalytic Authencity and Technical Authencity
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 41 Table 4 The Conduct of Qualitative Research for a PhDStudy / Scope Research Strat- Primary Methods Quality Criteria egy for Referred To Data Collection / (number of para.) AnalysisBowen (2005) / Exploratory study In-Depth, Open- 4 TrustworthinessUnited States Ended (7) Interviews / Grounded TheoryPeterson and Higgs Hermeneutics Conversational Inter- Credibility*(2005) / views / RigorTransnational Hermeneutics Ethical (10 )de Weerd-Nederhot Case Study Interviews/Within – Variety!(2001) / And Cross – Case (several pages!)The Netherlands Analysis ala Miles and Huberman (1994)McCotter (2001) / Deconstruction Variety Validity (4)United StatesCarter (1999) / Multiple case Semi-Structured In- 4 TrustworthinessCanada study terviews / (5) Grounded TheoryNote: *Authenticity, Plausibility, Trustworthiness Table 5 The Use of Specific Research Methods for Data Collection / Analysis in Qualitative StudiesStudy / Scope Field of Study Primary Quality Criteria Methods for Referred To Data Collec- (number of para.) tion / AnalysisRao and Perry (2003) / Marketing/ Convergent Construct ValidityAustralia Internet Interviews Internal Validity External Validity Reliability (7)Callahan and Elliot Behavioural Free Narrative Variety!(1996) / Economics (several pages!)United StatesYeung (1995) / International Qualitative ValidityHong Kong and ASEAN Business Personal Reliability Interviews (several pages!)
42 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64References ging the dog? British Medical Journal, 322, 1115-1117. Barusch, A., Gringeri, C., & George, M.Adams, C.A., & Harte, G. (1998). The (2011). Rigor in qualitative so- changing portrayal of the employ- cial work research: A review of ment of women in British banks’ strategies used in published arti- and retail companies’ corporate cles. Retrieved June 02, 2011 annual reports. Accounting Or- from http://www.faqs.org/ peri- ganizations and Society, 23(8), odicals/201103/2302649611.htm 781-812 l#ixzz1Nl0kfxpMAhrens, T., & Chapman, C.S. (2006). Baxter. J., & Eyles, J. (1997). Evaluating Doing qualitative field research in qualitative research in social ge- management accounting: Posi- ography: Establishing ‘rigour’ in tioning data to contribute to the- interview analysis. Transactions ory. Accounting, Organizations of the Institute of British Geogra- and Society, 31, 819-841. phers, 22(4), 505-525.Anderson-Gough, F., Grey, C., & Beattie, V., Fearnley, S., & Brandt, R. Robson, K. (2001). Tests of time: (2004). A grounded theory model organizational time-reckoning and of auditor-client negotiations. In- the making of accountants in two ternational Journal of Auditing, 8 multi-national accounting firms. (1), 1-19. Accounting, Organizations and __________ (2001). Behind closed Society, 26(2), 99-122. doors: What company audit is__________ (2005). “Helping them to really about. Basingstoke, Hamp- forget..”: The organizational em- shire: Palgrave. bedding of gender relations in __________ (2000). "Behind the audit public audit firms. Accounting, report: a descriptive study of dis- Organizations and Society, 30(5), cussion and negotiations between 469-490. auditors and directors", Interna-Angen, M.J. (2000). Evaluating interpre- tional Journal of Auditing, 4(2), tive inquiry: Reviewing the valid- 177-202. ity debate and opening the dia- Bergman, M.M., & Coxon, A.P.M. logue. Qualitative Health Re- (2005). The quality in qualitative search, 10(3), 378-395. methods. Forum Qualitative So-Arena, M., Arnaboldi, M., & Azzone, G. cial Research [On-line Journal], (2006). Internal audit in Italian 6(2). Retrieved September 03, organizations: A multiple case 2006, from http://www.qualitative study. Managerial Auditing Jour- -research.net/fqs-texte/2-05/05-2- nal, 21(3), 275-292. 34-e.htmAtkinson, A.A., & Shaffir, W. (1998). Bowen, G.A. (2005). Preparing a quali- Standards for field research in tative research-based dissertation: management accounting. Jour- Lessons learned. The Qualitative nal of Management Accounting Report, 10(2), 208-222. Retrieved Research, 10, 41-68. September 03, 2006, from http:// Barbour, R.S. (2001). Checklists for www.nova.edu/ssss/QR/QR10-2/ improving rigour in qualitative bowen.pdf research: A case of the tail wag-
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 43Brown, S.M. (1992). Cognitive mapping product development systems. and repertory grids for qualitative Management Decision, 39(7), 513 survey research: Some compara- -538. tive observations. Journal of Denzin, N.K. (1978). The research act: Management Studies, 29(3), 287- A theoretical introduction to so- 307. ciological methods (2nd ed.). NewCallahan, C., & Elliot, C.S. (1996). Lis- York: McGraw-Hill. tening: A narrative approach to _________ (1995). The experiential everyday understandings and be- text and the limits of visual under- havior. Journal of Economic Psy- standing. Educational Theory, 45 chology, 17(1), 79-114. (1), 7-18.Carter, M. (1999). A profile of service- __________ & Lincoln, Y. S. (Eds.). learning programs in South Caro- (1994). Handbook of qualitative lina and their responsiveness to research. Thousand Oaks, CA: the national priorities. Columbus, Sage Publications. Ohio: Bell & Howell Company. Eberle, T.S. (2005). Promoting qualita-Chapple, A., & Rogers, A. (1998). Ex- tive research in Switzerland. Fo- plicit guidelines for qualitative rum Qualitative Social Research research: A step in the right direc- [On-line Journal], 6(2). Retrieved tion, a defence of the ‘soft option’ September 03, 2006, from http:// or a form of sociological imperial- www.qualitative-research.net/fqs- ism? Family Practice, 15, 556- texte/2-05/05-2-31-e.htm 561. Efinger, J., Maldonado, J., & McArdle,Cregan, C. (2005). Can organizing G. (2004). PhD students’ percep- work? An inductive analysis of tions of the relationship between individual attitudes toward union philosophy and research: A quali- membership. Industrial and La- tative investigation. The Qualita- bour Relations Review 58(2), 282- tive Report, 9(4), 732-759. Re- 304. trieved September 03, 2006, fromCreswell, J.W. (2007). Qualitative in- http://www.nova.edu/ssss/QR/ quiry and research design: QR9-4/efinger.pdf Choosing among five traditions. Eisenhardt, K.M. (1989). Building theo- Thousand Oaks, CA: Sage Pub- ries from case study research. lications. Academy of Management Review,Creswell, J.W., & Miller, D.L. (2000). 14(4), 532-550. Determining validity in qualita- Eisner, E. (1992). Objectivity in educa- tive inquiry. Theory into Prac- tional research. Curriculum In- tice, 39(3), 124-131. quiry, 22(1), 9-15.Davies, D., & Dodd, J. (2002). Qualita- Gendron, Y. (2002). On the role of the tive research and the question of organization in auditors’ client- rigor. Qualitative Health Re- acceptance decisions. Accounting, search, 12, 279-289. Organizations and Society, 27(7),de Weerd-Nederhot, P.C. (2001). Quali- 659-684. tative case study research: The __________ , & Bedard, J. (2006). On case of a PhD research project on the constitution of audit commit- organising and managing new tee effectiveness. Accounting, Or-
44 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 ganizations and Society, 31(3), auditor behaviour and the psycho- 211-239. logical contract. European Ac-__________ ., Cooper, D.J., & Townley, counting Review, 10(4), 787–802. B. (2007). The construction of Horsburgh, D. (2003). Evaluation of auditing expertise in measuring qualitative research. Journal of government performance. Ac- Clinical Nursing, 12(2), 307-312. counting, Organizations and Soci- Huberman, M., & Miles, M.B. (1991). ety, 32(1/2), 101-129. Analyse des données qualitatives:Glaser, B., & Strauss, A. (1967). The Recueil de nouvelles discovery of grounded theory: Methods. Brussels: De Boeck- Strategies of qualitative research. Wesmael. London: Wiedenfeld and Nichol- Irvine, H.J., & Gaffikin, M. (2006). Get- son. ting in, getting on and getting out:Golafshani, N. (2003). Understanding Reflections on a qualitative re- reliability and validity in qualita- search project. University of Wol- tive research. The Qualitative Re- longong Research Online. Re- port, 8(4), 597-607. trieved June 02, 2011 http://Greenhalgh, T. & Taylor, R. (1997). ro.uow.edu.au/cgi/ viewcon- How to read a paper: Papers that tent.cgi? article=1127&context= go beyond numbers (qualitative commpapers&sei- research). British Medical Jour- redir=1#search=% 22Ir- nal, 315(7110), 740.-743. vine+H,+Gaffikin+M,+%Grey, C., Robson, K., & Anderson, F. 282006% (1997). Professional socialization 29+Getting+in,+getting+on+and+ in chartered accountancy prac- getting+ out:%22 tices: Making up accountants. Janesick, V.J. (1994). The dance of London: ICAEW. qualitative research design: Meta-Guba, E.G., & Lincoln, Y.S. (1981). phor, methodolatry, and meaning. Effective evaluation. San Fran- In Denzin, N.K. & Lincoln, Y.S. cisco: Jossey-Bass Publishers. (Eds.), Handbook of qualitativeGuba, E.G., & Lincoln, Y.S. (1994). research (pp. 209-235). Thousand Handbook of qualitative research. Oaks, CA: Sage Publications. Thousand Oaks, CA: Sage Publi- Kalekin-Fishman, D. (2001). Review: cations. David Silverman (2001). Inter-Hammersley, M. (1991). Reading ethno- preting qualitative data: Methods graphic research. London: Long- for analysing talk, text and inter- man. action. Forum Qualitative Social__________ (1992). What’s wrong with Research, 2(3). Retrieved Septem- ethnography? London: Routledge. ber 03, 2006, from http://nbn-Healy, M., & Perry, C. (2000). Compre- resolving.de/urn:nbn:de:0114- hensive criteria to judge validity fqs010363 and reliability of qualitative re- King, N. (1994). The qualitative re- search within the realism para- search interview. In Cassell, C., & digm. Qualitative Market Re- Symon, G. (Eds.), Qualitative search, 3(3), 118-126. methods in organizational re-Herrbach, O. (2001). Audit quality, search: A practical guide (pp. 14-
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 45 36). London: Sage Publications. for qualitative research in appliedKlein, H.K., & Myers, M.D. (1999). In- linguistics: Whose criteria and vestigating information systems whose research? Modern Lan- with ethnographic research. Com- guage Journal, 87, 1-12. munications of the Association for Lewis, J., & Ritchie, J. (Eds.). (2003). Information System, 2(23), 1-20. Qualitative research practice - AKuzel, A. & Engel, J. (2001). Some guide for social science students pragmatic thought on evaluating and researchers. London: Sage qualitative health research. In Publications. Morse, J., Swanson, J., & Kuzel, Lilford, R.J., Edwards, S., Braunholtz, A. (Eds.). The nature of qualita- D.A., Jackson, J., Thornton, J., & tive evidence (pp. 114-138). Hewison, J. (2001). Ethical issues Thousand Oaks, CA: Sage Publi- in the design and conduct of ran- cations. domised controlled trials. In Ste-Kvale, S. (1989). Issues of validity in vens, A.J.H., Abrams, K., Brazier, qualitative research. Lund, Swe- J., Fitzpatrick, R., Lilford, R.J. den: Chartwell Bratt. (Eds.). Advanced Handbook of______ (1995). The social construction Methods in Evidenced Based of validity. Qualitative Inquiry, 1, Healthcare (pp. 11-24). Thousand 19-40. Oaks, CA: Sage Publications.______ (1996). Interviews: An introduc- Lillis, A.M (2006). Reliability and valid- tion to qualitative research. Thou- ity in field study research. In sands Oaks, London, New Delhi: Hoque, Z. (Ed.) Methodological Sage Publications. issues in accounting research:Kwok, Chee Chiu, & Sharp, D. (2005). Theories and methods (pp. 461- Power and international account- 475). London: Spiramus. ing standard setting: Evidence Lincoln, Y.S., & Guba, E.G. (1985). from segment reporting and intan- Naturalistic inquiry. Beverly gible assets projects. Accounting, Hills, CA: Sage Publications. Auditing and Accountability Jour- Mason, O.J. (2002). Teaching qualitative nal, 18(1), 74-99. research methods: Some innova-Lather, P. (1986). Issues of validity in tions and reflections on practice. openly ideological research: Be- Psychology Teaching Review, 10 tween a rock and a soft place. In- (1), 68-75. terchange, 17(4), 63-84. Maxwell, J.A. (1992). Understanding______ (1993). Fertile obsession: Valid- and validity in qualitative re- ity after poststructuralism. The search. Harvard Educational Re- Sociological Quarterly, 34(4), 673 view, 62(3), 279-300. -93. __________ (2004). Qualitative re-______ (1995). Post-critical pedagogies: search design: An interactive A feminist reading. In McLaren, approach (2nd ed.). Beverly P. (Ed.) Postmodernism, postcolo- Hills, CA: Sage Publications. nialism and pedagogy (pp. 167- Mays, N. & Pope, C. (2000). Qualita- 186). Albert Park: James Nicholas tive research in health care: As- Publishers. sessing quality in qualitativeLazaraton, A. (2003). Evaluative criteria research. British Medical Jour-
46 A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 nal. 320(7226), 50-52. Qualitative Report, 4(3/4). Re-McCotter, S.S. (2001). The journey of a trieved September 03, 2006, from beginning researcher. The Quali- http://www.nova.edu/ssss/QR/ tative Report, 6(2). Retrieved Sep- QR4-3/myers.html tember 03, 2006, from http:// Neimark, M.D. (1992). The hidden di- www.nova.edu/ssss/QR/QR6-2/ mensions of annual reports. mccotter.html London: Paul Chapman.McKinnon, J. (1988). Reliability and Onwuegbuzie, A.J. (2002). Positivists, validity in field research: some post-positivists, post- strategies and tactics. Account- structuralists, and post- ing, Auditing and Accountabil- modernists: Why can’t we all get ity, 1, 34-54. along? Towards a framework forMiles, M.B. & Huberman, A.M. (1994), unifying research paradigms. Edu- Qualitative data analysis: An ex- cation, 122(3), 518–530. panded sourcebook (2nd ed.). Lon- Patton, M.Q. (1990), Qualitative evalua- don: Sage Publications. tion and research methods (2ndMiller, S., & Fredericks, M. (1995). The ed.). Newbury Park: Sage Publi- nature of ‘evidence’ in qualitative cations. research methods. Qualitative Peterson, M., & Higgs, J. (2005). Using Health Research, 5, 388-397. hermeneutics as a qualitative re-Modell, S. (2005). Triangulation be- search approach in professional tween case study and survey practice. The Qualitative Report, methods in management ac- 10(2), 339-357. counting research: An assess- Polit, D., & Hungler, B. (1991). Nursing ment of validity implications. research: Principles and methods. Management Accounting Re- New York: JB Lippincott. search, 16, 231-254. Rao, S., & Perry, C. (2003). ConvergentMorgan, A.K., & Drury, V.B. (2003). interviewing to build a theory in Legitimising the subjectivity of under-researched areas: Principles human reality through qualitative and an example investigation of research method. The Qualitative internet usage in inter-firm rela- Report, 8(1). Retrieved September tionships. Qualitative Market Re- 03, 2006, from http:// search: An International Journal, www.nova.edu/ssss/QR/QR8-1/ 6(4), 236-247. morgan.html Riley, M. (1995). Tourism development under close control: The case ofMorse J.M., Barrett M., Mayan M., Ol- the Falkland Islands. Tourism son K. & Spiers J. (2002). Verifi- Management, 16(6), 471-474. cation strategies for establishing Rittenberg, L.E., & Covaleski, M.A. reliability and validity in qualita- (2001). Internalization versus ex- tive research. International Jour- ternalization of the internal audit nal of Qualitative Methods, 1(2), function: An examination of pro- 1-19. fessional and organizational im-Myers, M. (2000). Qualitative research peratives. Accounting, Organiza- and the generalizability question: tions and Society 26, 617–641. Standing firm with proteus. The Rubin, H., & Rubin, I. (1995). Qualita-
A. Md. Ali, H. Yusof / Issues in Social and Environmental Accounting 1/2 (2011) 25-64 47 tive interviewing: The art of search requires quality concepts hearing data. London: Sage of its own. Management Decision, Publications. 39(7), 551-555.Sarens, G., & De Beelde, I. (2006), Stiles, W.B. (1993). Quality control in Building a research model for in- qualitative research. Clinical Psy- ternal auditing: Insights from lit- chology Review, 13, 593–618. erature and theory specification Strauss, A., & Corbin, J. (1990). Basics cases. International Journal of of qualitative research: Grounded Accounting, Auditing and Per- theory procedures and techniques. formance Evaluation, 3(4), 452- Newbury Park, CA: Sage Publica- 470. tions.Seale, C. (1999). Quality in qualitative Strauss, A., & Corbin, J. (1998). Basics research. Qualitative Inquiry, 5 of qualitative research: Tech- (4), 465-478. niques and procedures for devel-Shenton, A.K. (2004). The analysis of oping grounded theory (2nd ed.). qualitative data in LIS research London: Sage Publications. projects: A possible approach. Suter, E.A. (2010). Validity in qualita- Education for Information, 22, tive research on personal rela- 143-162. tionships. Kentucky Journal ofSilverman, D. (2001). Interpreting Communication special issue on qualitative data: Methods for ana- the future of relational commu- lysing talk, text and interaction nication. Retrieved June 02, (2nd ed.). London: Sage Publica- 2011 from https:// tions. portfolio.du.edu/pc/port?Sinkovics, R.R., Penz, E., & Ghauri, portfolio=esuter P.N., (2005). Analysing textual Sweeney, B., & Pierce, B. (2004). Man- data in international marketing agement control in audit firms: A research. Qualitative Marketing qualitative examination. Account- Research: An International Jour- ing, Auditing and Accountability nal, 8(1), 9–38. Journal, 17(5), 779-812.Siti-Nabiha, A.K., & Scapens, R.W. Sykes, W. (1990).Validity and reliability (2005). Stability and change: An in qualitative market research: A institutionalist study of manage- review of the literature. Journal of ment accounting change. Account- the Market Research Society, 32 ing, Auditing and Accountability (3), 289-328. Journal, 18(1), 44-73. Thompson, J.B. (1988). Critical Herme-Spencer, L., Ritchie, J., Lewis, J., & Dil- neutics (4th ed.). Cambridge: lon, L. (2003). Quality in qualita- Cambridge University Press. tive evaluation: A framework for Unerman, J. (2000). Reflections on assessing research evidence. Lon- quantification in corporate social don: National Centre for Social reporting content analysis. Ac- Research. counting, Auditing and Account-Stake, R. (1978). The case study method ability Journal, 13(5), 667-680. in social inquiry. Educational Re- Unerman, J. (2003). Enhancing organ- searcher, 7(2), 5-8. izational global hegemony withStenbacka, C. (2001). Qualitative re- narrative accounting disclosures: