This document discusses a systematic analytical framework for analyzing qualitative interview data in management accounting research, emphasizing the importance of rigorous study design and trustworthiness. The paper outlines the method choices, study design, and analytical protocols used in a field study of performance measurement in manufacturing firms. It highlights the need for clear communication in qualitative research and demonstrates how qualitative and quantitative data can be integrated for reliable findings.

1. A framework for the analysis of interview data
from multiple field research sites
Anne M. Lillis
Afzaal Ali
Business School, University of International
Business and Economics, Beijng, China.

2.  Anne researches and teaches in Management Accounting. Her
research interests focus on field studies of the design and behavioural
influence of performance management and control systems. Her
teaching interests are in Management Accounting, with a particular
focus on capstone and case-based teaching.
Anne Lillis is a Professor of Management
Accounting and Head Department of Accounting
and Finance, University of Melbourne, Melbourne,
Victoria, Australia.
BCom, University of Melbourne 1980
MCom, University of Melbourne 1990
PhD, University of Melbourne 1998
Anne is an Associate Editor of Journal of Management Accounting
& Research,
and a member of the editorial boards of Accounting,
Organizations and Society, Contemporary Accounting Research,
Management Accounting Research, Behavioral Research in
Accounting, Accounting and Finance, Abacus and the
European Accounting Review.

3. Summary of the article:
Quantitative and qualitative research are commonly considered
to differ fundamentally.
In this paper, the author basically tried to focus the issue with
qualitative data.
In general, papers reporting the results of research studies
disclose little detail regarding attributes of study design,
analytical processes and methods actually used by
researchers.
The major focus of the paper is the application of a systematic
analytical protocol designed to encourage completeness and
impartiality in the collection and analysis of qualitative data.

4. Conti…
This paper describes in some depth the method choices and
analytical protocol used in a field study project.
To overcome this issue, the author follows a systematic
analytical protocol proposed by Miles and Huberman (1994) in
their book “Qualitative data analysis: An expanded
sourcebook”.

5. Background of the problem:
Over the past 15 years, in the Accounting (Management
Accounting) literature researchers have been urged to shift their
study of the role and functioning of accounting to its natural
contexts i.e. case/ field study research.
While the call for fieldwork has been persistent, the penetration of
this research into top-ranking journals, particularly in the United
States, has been very limited.
Shields (1997) attributes the lack of publication of case/ field study
research to several factors, with lack of knowledge about how to
do good studies' leading the list.
Similarly Foster and Young (1997) argue that few studies meet the
criteria of high quality field research applied in other disciplines.

6. Conti…
It may be that some of the problem lies with the way field study
papers are written.
Compared with surveys and laboratory studies, qualitative
research, at least in accounting, lacks an accepted language in
which to communicate study design parameters and the critical
links between design and credible research outcomes.
It seems that at least part of the difficulty in publishing field
research lies in convincing reviewers that the study is not only
relevant and interesting, but also trustworthy.

7.  Section-01: Already discussed
 Second-02: The description of the field study protocol
commences in this study with a discussion of the general
research question and implications for method and design
choices.
 Section-03: and Section-04: Provide a brief overview of the
study design and method and an outline of the population and
sample.
 Section-05: Outlines the general design characteristics of the
interview guide.
Sections of the paper:

8.  Section-06: Details the analytical method applied to the
qualitative data.
 Section-07: Describes the way qualitative and quantitative
data are linked in this study to provide tests of convergent
validity at several levels.
Conti:

9. Objectives of the paper:
This paper describes the application of qualitative study
design features that can enhance trustworthiness.
The paper examines the design of a systematic field study
protocol.
Rather than an abstract discussion of method, the protocol is
described within the context of a particular study in order to
link explicitly the research question, research design and
analytical protocol.
Thus this paper focuses primarily on the design of an
analytical protocol that can support an audit trail from field
data to the testing or development of propositions.
An audit trail is a transparent description of the research steps taken from the
start of a research project to the development and reporting of findings.

10.  This paper describes part of a larger study of strategy and
performance measurement in manufacturing.
 While the larger study involved the collection and analysis of
both quantitative and qualitative data.
 This paper reports on the qualitative elements of the study.
 The study was motivated from contingency- type studies
linking strategy and management control system design and
from the more recent management accounting literature on
strategic performance measurement.
2. The motivation for field research:

11.  Two issues are particularly relevant to the design of the
qualitative elements of the study:
 1st one:-
 The findings of contingency studies linking strategy with the
use and characteristics of management accounting controls
have been notoriously conflicting (Langfield-Smith, 1997).
 Past literature does not support predictions about the roles of
any specific performance measurement techniques in
particular strategic contexts.
Conti…

12.  In the 1980s, the manufacturing side witnessed radical
change in context of competitiveness.
 In this situation, traditional performance measurement
systems were proved to be insufficient and to some extent
become irrelevant.
 As a response, some new solutions emerged in the form of
the Balanced Scorecard (Kaplan and Norton, 1992; 1996)
and Integrated Performance Measurement (Dixon et al.,
1992).
 However, the emphasis shifted to promoting the
development of non-financial dimensions of performance
such as quality and customer responsiveness etc.
2nd one:-

13. Examples include:
Profitability
Revenue
Return on investment (ROI)
Cash flow
Standard deviation or Variance
Coefficient of variation
Rate of return or return on investment
Working capital
Financial Performance Measures:
Examples include:
Customer satisfaction
Employee satisfaction
Quality
Market share
Number of new products
Non- Financial Performance Measures:

14. Both of these issues suggest the need for an exploration of
competitive strategy
and its implications for accounting performance
measurement in practice.
Moreover, there is an apparent need to gain greater insight
into the empirical experience of variables
such as competitive strategy and the role of accounting
performance measures (Langfield-Smith, 1997).

15. 3. Overview of study design and method:
Data were collected using a semi-structured interview in combination
with a structured questionnaire administered to 50 managers in 36
manufacturing firms in Victoria, Australia.
The use of a semi-structured interview schedule in combination with a
structured questionnaire aimed to derive the benefits of quantitative
and qualitative methods, and to apply appropriate methods to the
questions of interest.
Conventional statistical analyses were applied to data collected using
the structured questionnaire.
This paper focuses on the qualitative data collected during the semi-
structured interviews.

16. Conti…
 The researcher used several approaches to limit biasness both
during the interview and in the analysis of transcripts.
 First, researcher designed an interview guide with the aim of
ensuring complete and consistent coverage in each interview
session.
 Secondly, researcher in advanced developed a list of neutral
questions and queries in order to minimize interference.
 Fourth, a systematic auditable process developed by Miles and
Huberman (1994) was used.
 Finally, the use of structured questions provided quantitative
response from the respondents on established scales.

17.  Time and cost considerations limited the sample of participant
firms to the population of manufacturers in the State of
Victoria, Australia.
 The population was further restricted to relatively large
manufacturing firms with more than 200 employees.
 Researcher only consider firms that would have formal
performance measurement systems in place.
 A listing of 340 Victorian manufacturing firms with more than
200 employees was obtained in May 1994, from the Victorian
State Department of Business and Employment.
4. The population:

18. The sample:
 Several features of the study design determined the sample
size.
 The sample was required to be sufficiently large to be able
to observe patterns across firms.
 To conduct simple inferential statistical evaluation of
results.
 However, the sample size was to be limited because the
study design required semi-structured interviews and a
structured questionnaire to be administered by a single
researcher.

19. A sample size of 30-40 manufacturing firms was determined
to be appropriate in this context.
The firms were selected across industry categories with the
aim of obtaining the participation of firms of comparable size,
and with a variety of strategic orientations. .
In total, researcher contacted 55 profit centre managers. .
Out of 55, finally 36 managers were available to give
feedback. While 19 managers refused to participate in this
study, and final response rate was 65 percent.
Conti..

20. Follow-up sites visits:
In order to confirm and find possible contradiction in the data
collected from profit centre managers, researcher selected
12 manufacturing managers from sample firms.
Researcher formally got permission from all of the profit
centre managers to pursue a follow-up interview with a
manufacturing manager.
The final sample consisted of 36 profit centre managers and
12 manufacturing managers.
The interviews were conducted between June 1994 and
January 1995.

22. 5. Instrument design- the interview guide:
The study was conducted with a semi-structured interview and
structured questionnaire.
The interview guide used in this study is divided into four
sections. For example, the performance measures used, the
constitution of cost benchmarks and competitive priorities
represent three separate themes.
The interview protocol was developed from methods
documented by McCracken (1988) and Brenner, Brown and
Canter (1985) based on their qualitative research experiences.

28. In addition to the potential interviewer-induced bias in the
collection of qualitative data, the analysis of qualitative data
is subject potentially to significant bias as it relies on
interpretations and classifications imposed by the researcher.
Among the reasons why readers experience `lack of trust' in
qualitative studies lies in the absence of established
techniques for ensuring that data analysis is both complete
and impartial.
On the other hand, the analysis of qualitative data is less
structured.
6. The qualitative analysis protocol:

29. Readers are left questioning the extent to which propositions
raised by the researcher are supported by the data; whether
potential alternative propositions have been `overlooked‘;
whether all cases were systematically evaluated before
propositions were raised; whether the researcher saw only what
they set out to see in the data; and whether the process is
reproducible.
The use of a systematic analytical protocol such as the Miles
and Huberman (1994) data display approach used here,
enhances trust in the results of qualitative analysis.
Conti…

30. To analyzed and interpret qualitative data, researcher
followed a four steps qualitative analysis protocol.

31. The variable constitution of performance benchmarks was operationalized in
the qualitative data, and this variable was hypothesized to be related to
strategy. The structure of the matrix reflects this theoretical proposition.

32. Step 1: Transcripts were coded using the qualitative analysis package
QSR NUDIST. NUDIST was used in this study to code all of the raw,
unsummarised interview transcripts, by associating the sentences in the
transcript with one or more themes defined in a hierarchical coding
structure.
Each sentence was given a text-unit number in NUDIST. In this way, each
text unit is identified as relating to one or more pre- determined thematic
codes.

33. NUDIST is an acronym for Non-numerical Unstructured
Data Indexing, Searching and Theorizing. .
It is a qualitative analysis software package distributed by
Qualitative solutions and Research Pty Ltd, La Trobe
University, Bundoora, Victoria, Australia.

34. Step 2: A data sheet was prepared manually for each transcript. The text in
the transcripts was reconfigured and reported under conceptual headings
using NUDIST.
The reorganized transcripts were then summarized manually under similar
conceptual headings to those used in the coding scheme.

35. Step 3: The data sheets prepared for each transcript were then
combined in a multi-case matrix format.
For this study, these matrices and the analyses they supported
were focused on the variables to be intercorrelated.

36. Step 4: The content of the matrix was then built up case by case.
Each case was classified initially as `predominantly cost
minimisation', `mixed' or `predominantly differentiation' based on
scores on the scaled strategy variable.

38. 7. Linking qualitative and quantitative data:
 Thus, the findings were triangulated against the quantitative
analysis of participant-rated data on variables similar to those
explored in the qualitative data.
 For example, scaled responses were collected on strategy
using Porter‘s (1980) typology in a form similar to that used in
survey-based research (Govindarajan and Fisher, 1990).
 The scaled responses reflected an over- whelming importance
of differentiation strategies in the sample. This quantitative
analysis was compared with the elaborated responses on the
firms' competitive edge and strategic priorities.
 These quantitative and qualitative analyses `converged' to
reflect a similar strategic profile across the sample.
Data triangulation validates data and research by cross verifying the same
information

39. The proposition emerged that firms experience greater difficulty
integrating performance criteria relating to customer
responsiveness with efficiency criteria than they did integrating
quality and efficiency criteria.
Proposition:

40. Retrospective evaluation of the method adopted:
 First, the study design achieved its purpose successfully.
 Beyond such re-examination of traditional contingency
models, the findings from this study raised new
unhypothesised relations that formed the foundation for
theory building.
 The protocol used here establishes a disciplined approach
to data extraction and analysis that at least promotes
completeness and impartiality.
A retrospective evaluation may assess the degree to which goals were met.

41. Conti…
 She mentioned some researcher-related problems
specifically in qualitative study.
 There are practical matters such as scheduling too
many interviews too close together, not gathering
sufficient archival and other data while on site, getting
too tired, occasionally losing enthusiasm and
concentration and thus finding loose threads in
interviews which should have been followed up.

42. THANKS