A Three-Stage Framework For Teaching Literature Reviews A New Approach
1. International Journal of Management Education 9(4), 2011 1
A three-stage framework for teaching literature reviews:
A new approach
Teresa Smallbone and Sarah Quinton, Oxford Brookes University, England
DOI:10.3794/ijme.94.337
Received: July 2010
Revised: February 2011, May 2011
Accepted: June 2011
Introduction
In this paper we set out to discuss the ways in which reviewing academic literature has evolved and then
outline a new approach to teaching literature reviews via a three-stage framework. A literature review is a
requirement in assessed pieces of written work in management studies for many courses and institutions at
both undergraduate and postgraduate level. Electronic search engines and greater access to internet-based
academic secondary sources, coupled with the drive by national institutions and government requirements for
increased academic output, has led to a huge expansion in the number of published academic journal
articles. The task of reviewing the published literature in a particular field is in theory simpler, as much more
is readily accessible through search engines, but it is also more complex as the task becomes ever larger, and
the review itself more difficult to organise, write and contain.
In an academic context, literature reviews should have a defined purpose and an identified audience. They
must contain an in-depth analysis of past research and from it create a summary, evaluation or synthesis to
build an argument and make a contribution to knowledge (see Bruce, 1994; Hart, 1998; Baker, 2000).
However, writing a literature review is fraught with problems for students because they have difficulty
understanding what is expected of the outcome. In our view, literature reviews are inherently difficult to
write and the results are often of poor quality. Students must develop a critical standpoint in order to write a
successful literature review but it may be very difficult for students to understand and engage in a critique of
academic literature due to their prior educational experience or cultural expectations.
Nevertheless, it is important that management students write literature reviews because it enables them to
learn about rigour in research, particularly when they are unable to collect primary data. Gaining access to
organisations and specific groups in the population is increasingly problematic in many discipline areas,
including business and management. Changing attitudes to data privacy, reinforced by legislation, mean, for
example, that simple sampling frames used in past research, such as internal company address books and
telephone lists, are no longer freely available to researchers.
Writing a literature review helps students to learn how to sort and categorise large amounts of disparate
information drawn from many sources and re-frame it for a given purpose. None of the skills necessary for
these transformations of the data is innate or can be assumed to be present in the undergraduate and
postgraduate student populations. There is little consensus about how this fundamental aspect of research
may be taught. In this paper we outline a three-stage framework to better prepare students for writing their
Abstract
Writing a literature review yields many academic benefits. It is an appropriate route for management students to learn
academic skills, such as how to search databases and to search off line, and to improve practical and theoretical
knowledge. It enables theory development unimpeded by the practical obstacles of gaining access to people and
organisations to collect data. It requires the development of expertise in research methods, numeracy, attention to
detail, and in the analysis and interpretation of data. Despite these benefits, the pedagogic literature has little to say
about the best means of teaching students how to research and write literature reviews.
This paper develops a three-stage framework for teaching literature reviews which gives explicit guidance for teachers
and simplifies the process for students. The framework comprises a means of learning how to carry out a systematically
informed search for relevant literature, demonstrated through examples; an approach to learning how to read and
deconstruct a text in a critically informed way, through using a template with a questioning approach; and a way
explaining how to reconstruct the material, using a simple metaphor to demonstrate how this is done.
Keywords: literature reviews; teaching framework; academic skills; synthesis
Teresa Smallbone is a senior lecturer in marketing with a particular interest in consumer behaviour, research
methods and effective strategies for the development of teaching and learning.
Sarah Quinton is a senior lecturer in marketing with a particular interest in the use of technology for
marketing by SMEs, research methods and the development of critical writing skills for students.
2. International Journal of Management Education 9(4), 2011
2
own literature reviews.
The case for literature reviews
The benefits of literature reviews in an academic context are manifold. Literature reviews can help students
to identify trends that may have emerged in a subject (Hart, 1998; Saunders, Lewis, & Thornhill, 2007) and
gain a new perspective (Hart, 1998). They enable students to display their knowledge of a chosen field and
what has gone before, to develop their search abilities and their ability to critique a subject area. A
literature review may also be used to create an analytical framework to analyse primary data (Jankowicz,
2000). Literature reviews from one subject area can assist in delineating the boundaries of that subject, and
researchers working across subjects can map different literature reviews to find an overlap. Writing literature
reviews overcomes the increasing problems of access to organisations and falling public participation in
surveys (Betts & Lound, 2010), which is already forcing academics to re-think their research strategies.
The discipline required to read for and construct a literature review hones intellectual as well as writing skills
(Leki, 1992), enabling students to demonstrate their understanding not only of the subject but also of
research methods. Creating a literature review yields both practical knowledge, such as how to physically
perform searches and categorise search outputs; and conceptual knowledge, such as the deconstruction of
past research, transformational thinking and synthesis leading to the creation of new knowledge. The whole
process involves moving to higher order critical thinking: from knowledge and comprehension to analysis and
evaluation (Bloom, 1956). As a result, it helps students to develop the ability to synthesise information from a
variety of sources, a skill vital for management students once in business life, while also developing their
writing skills.
Literature reviews in management research form a central role in developing ideas and argument, and enable
management students to develop their thinking and critical abilities. The quality of the reasoning must be
based on good background knowledge relevant to the context, a sound knowledge of concepts in the
discipline, and of the method of argumentation in inquiry (Bailin, Case, Coombs, & Daniels, 1999). This
implies that the studentsâ approach should be concerned with uncovering complexity, richness and
controversy, so that their conclusions are contextually sound and based on multiple perspectives.
There are clear pedagogic benefits in applying a more systematic approach. Following a rigorous and
replicable searching process through a library or resource centre, and using key search strings while searching
online, can help students structure their thinking. As research supervisors in management research now
frequently question the validity and reliability of reviews, a well-documented, verifiable and logical process
may reduce accusations of plagiarism and enhance the credibility of their work.
It is thus an essential skill for any business and management student to be able to perform a structured
search of academic literature, evaluate texts to critique the material, then construct an argument. A
management graduate should be able to evaluate the quality and rigour, and determine the value of
information, when reaching a management decision. An ability to write a report, underpinned by an
understanding and appropriate use of relevant supporting information, is a prerequisite for a successful
management career and one that is recognised by educators and employers.
Thus the Quality Assurance Agency (QAA, 2007) requirements for general business and management honours
degrees include demonstration of the following skills:
Cognitive skills of critical thinking, analysis and synthesis. This includes the capability to identify
assumptions, evaluate statements in terms of evidence, to detect false logic or reasoning, to identify
implicit values, to define terms adequately and generalise appropriately. (p. 3).
Similarly, employers value problem solving very highly as a skill for a business graduate (Institute of Directors,
2007). In research carried out for the Institute of Directors, 294 of the 500 company directors who responded
(59%) stated that problem solving (involving thinking and analysing information) was very important.
The evolution of literature reviews
In recent years, the methodological approach to researching and writing literature reviews has come under
increasing scrutiny. In the social sciences, including management, there has been much discussion about
applying a more systematic approach to gathering material for literature reviews, specifically about adopting
the procedures developed in medical sciences for systematic literature reviews. Recent authors have covered
topics such as how to construct searches on electronic databases and issues such as which search strings to
use; advice on the use of search filters (Deurenberg, Vlayent, Guillo, Oliver, Fervers, & Burgers, 2007); how
to develop inclusion and exclusion criteria in the health and medical science subject areas (McNally & Alborz,
2004); and a description of the searching strategies employed by teachers (Haig & Dozier, 2003).
In terms of the approach, as far back as 1989, Cooper suggested, in the preface to the first edition of his
3. International Journal of Management Education 9(4), 2011 3
book, that there was a systematic, objective alternative to what he described as the standard intuitive,
narrative and subjective approach to reviewing existing literature, and that this new approach was gaining
rapid acceptance in the social science field. The methodology of systematic literature reviews was
subsequently developed in medical science. It aims to synthesise past research in a transparent and
reproducible way, as a means to inform decision-making about the effectiveness of healthcare interventions
(Cook, Mulrow, & Haynes, 1997; Pope, 2003; Sackett, Rosenberg, Muir, Gray, Haynes, & Richardson, 1996). A
systematic literature review is rule-driven and evidence-based, and relies on electronic searches conducted
in an exhaustive manner, using explicit criteria for locating studies and deciding whether to include them.
Transparency is key: reviews make clear how the researchers have searched for and evaluated the available
literature so that readers can form a judgement as to the basis on which they drew their conclusions (Oakley,
2006).
Authors such as Pawson (2001), Boaz, Ashby, and Young (2002) and Dixon-Woods et al. (2006) suggest that the
methodology of systematic review can be combined with a more qualitative approach and adopted for the
social sciences. There have been strong arguments both in favour and against applying it to the social
sciences (see, for example, Hammersley, 2001; Clegg, 2005; Oakley, Gough, Oliver, & Thomas, 2005).
Both education and management research pose particular problems for applying the methodology of
systematic literature reviews. Ogawa and Malen (1991) described the field of education as having an
abundance of âmultivocal literaturesâ (p. 265), and proposed the use of exploratory case study method as a
means to engender greater rigour when writing reviews. Davies (2000) found that much educational research
was not robust, with a wide range of research methodologies, a fragmented research community and few
systematic reviews on which to base policy. A similar critique has been applied to management research
(Tranfield, Denyer, & Smart, 2003) where there is also a âdislocation of research from practiceâ (Denyer &
Tranfield, 2006, p. 213). At Cranfield University, where David Tranfield is based, systematic review is
nevertheless now regarded as a key tool for management researchers, and students on the PhD programme
spend much of their first year writing a systematic literature review.
Systematic searches of academic literature, and the classification and extraction of relevant data for
synthesising into a review, require both analytic and interpretive skills, and a high level of numeracy and
attention to detail, as well as expertise in a range of research methods (Oakley et al., 2005). The problem is
that students do not necessarily have the knowledge base; the challenge for teachers is to make it explicit
and accessible.
Many authors have drawn attention to the lack of explicit teaching of students about how to tackle the
writing of a literature review. Jackson aired the issue in 1980, Cooper in 1989, and Bruce in 1994. It was
raised again by Foese, Gantz and Henry in 1998; Boote and Beile in 2005; and Zorn and Campbell in 2006.
Recent authors who have begun addressing the pedagogic issues include Granello (2001), who used Bloomâs
taxonomy as a way of helping students to move up the hierarchy of thinking: from the lower levels of
knowledge, comprehension and application, to higher order analysis, synthesis and finally evaluation. Harris
(2006) developed a teaching model to improve graduate studentsâ writing. Both Criollo (2003) and Liu and You
(2008) considered the impact of prior educational experience and national culture on the way university
students write literature reviews. Overall, however, given its importance in academic practice and
assessment, remarkably little has been written in the academic pedagogic literature about how to teach
students to write good literature reviews.
The challenge of teaching how to write a literature review
In our experience, there are general challenges that are presented by literature reviews. When literature
reviews are discussed in a classroom, it is often done in an abstract way and as a stand-alone topic, which
makes it more difficult to understand. Writing literature reviews is not necessarily part of most studentsâ
skills set. It does not directly relate to their everyday experiences, which adds to their perception of
literature reviews as difficult. Relating the concept to current assignments could have more impact and make
the subject more concrete. In order to create a literature review, students need to spend time exploring
subject alleys that may turn out to be dead ends, but this is a necessary part of learning what will not go into
a literature review. Thus literature reviews take time to create, posing another challenge for the increasingly
time-pressured students.
Students may not have been taught to question what they read during their secondary education. This may be
particularly true of students from other educational cultures where reproduction of information is strongly
encouraged (Liu & You, 2008). Helping develop these students and change their approach can be a challenge.
Most students at both undergraduate and postgraduate level find it difficult to believe that they have their
own authorâs voice, and that their voice is valuable. At some stage in the literature review the author will
need to express a well-informed opinion. Yet students frequently doubt that they can provide any original
insight into the literature. This doubt can lead to hesitancy in putting pen to paper. It is a further challenge
4. that must be overcome when teaching students to write literature reviews.
Teachers face other challenges: teaching how to create a literature review is particularly difficult to do if you
have never written one yourself. The experience of trying to weave published literature together to form a
coherent critical discussion takes practice, and if the teacher has not recently tried to do it the teaching can
become detached and unrealistic. Thus teachers might have to face their own weaknesses in their
understanding of what is required. This may be more likely in certain practitioner-focused subjects including
management, which does not have a long-established tradition of scholarly writing (Hawkins, 1984;
Varndarajan, 2003).
Having become aware of the challenges through their teaching, the authors set out to provide simple tools
that would help both students and teachers overcome this problem.
Method
The authors have taught both undergraduates and postgraduates research methods in a university business
school for many years. They supervise undergraduate, masters and doctoral dissertations and theses. In 2007
and 2008 they attended a series of workshops run by the Advanced Institute of Management (AIM) on
systematic reviews and writing literature reviews. This paper reflects the learning and experience gained
during those workshops, and the structured reflection on this subject that both preceded them and has
continued subsequently. As teachers of research methods and dissertation supervisors, we have considered
how to facilitate studentsâ understanding and improve their ability to write literature reviews at both
undergraduate and postgraduate level. The proposed three-stage approach to teaching literature reviews (a
structured search, the deconstruction and formal evaluation of a text, and the reconstruction and creation of
an original contribution) offers a summation of this expertise. It is based on learning from the workshops, a
literature search of the limited published pedagogic material on the subject, and collective experience of
grappling with how to teach âdoing a literature reviewâ to a wide range of undergraduate and postgraduate
students.
The three stages of a literature review
We see writing literature reviews as falling into three distinct stages. The first is the identification and
collection of literature as part of a more rigorous, systematised approach to literature reviews (see Tranfield
et al., 2003). We accept the critique, discussed earlier, developed by Cooper (1989), Tranfield et al. (2003)
and others, of the traditional narrative approach to literature reviews, and agree that there is a need to
develop a more systematic and transparent approach. The second involves the de-construction of academic
literature. It is achieved largely by asking questions of the literature (November, 2002; Quinton & Smallbone,
2006; Walllace & Wray, 2006; Smallbone, Hartley, & Quinton, 2007). The third is the formation of argument
and discussion, and its reconstruction and transformation into a body of new work that benefits the subject
area. This paperâs contribution to knowledge lies in suggesting a framework that gives explicit guidance to
the three stages involved.
A structured search
A systematically informed literature review involves the painstaking search for relevant literature. However,
this is only a preliminary to the filtration of that literature into a smaller subset that is based on reliable and
relevant research. Oakley et al. (2005) described the lessons learned while constructing a methodology to
support systematic reviews in the social sciences. These include the need to search large amounts of
literature to find small subsets of studies that are deemed reliable. Systematic searches invariably uncover
many more studies in a particular area than might have been suspected, but may also omit key pieces of
data. As a result, a literature search may involve a mapping exercise to establish the extent of the literature;
an exercise that is both transparent in its method and is reproducible. The next step focuses on reducing the
number of studies to a subset that answers a particular research question.
A search and mapping exercise is described below; used to demonstrate to students both how to conduct and
how to document an initial search. The design of the example was based on a systematically informed
approach to reviewing literature, as suggested by Tranfield et al. (2003). A systematically informed approach,
as shown in this example, helps identify and clarify the selection of the literature and minimise bias in the
process of selecting the literature (Cook et al., 1997). In the example we use, two large electronic databases
were searched, one broad based (EBSCO) and one a specialised education based database (ERIC, Educational
Resources Information Center). Keywords and search strings were generated using a thesaurus and other
keywords used by pedagogic researchers in their published works (Table 1). The researchers were interested
to find any links in the literature between the specific motivations of gifted students and their performance
at university in order to write a paper that would offer guidance to academics on how to get the most from
this type of student. The aim was to identify literature that would answer the research question âwhat
motivates gifted students to higher performance at university?â
4 International Journal of Management Education 9(4), 2011
5. The initial key word search used âgifted studentsâ and gave a large but mostly irrelevant set of US results
(279 hits) on the US-based ERIC database. A class discussion can be built around a live demonstration of this
search and could highlight what needs to be considered in the first mapping stage. Topics for this discussion
might include decisions about the relevance of search results. For example, the search found potentially
relevant papers such as Phillips and Lindsay (2006), which investigates motivation and achievement in gifted
students, but the students in that study and in many others were at school rather than at university
(Alexander, Murphy, & Guan, 1998; Neber, Finsterwald, & Urban, 2001; Rigby, 2005; Philips & Lindsay, 2006)
and thus were not of interest to the research question.
The next step is to develop a set of criteria by which to judge the identified literature, and a class discussion
can be focused around what the relevant criteria might be. The criteria eventually adopted were that papers
must: be peer reviewed; have focused specifically on university students; concentrate on motivation for
studying and achievement at university; be dated from 1998 onwards in the ERIC database; not be duplicates
held in other databases; be written in English. International studies were permitted owing to the lack of UK-
based material. From these early stage discussions, the students build up an understanding of how to search,
select and perform an initial evaluation of search results, and the need to document the process.
Table 1: Summary of exemplar search strings and number of articles found
Subsequent discussions can be centred on the potential limitations of a systematically informed approach to a
literature review. For example, finding articles is always dependent on the quality of indexing and
cataloguing of articles in electronic databases and is often reliant on the software. As English language usage
is not standardised, the key words used to describe what is being searched for change, making searching
more complicated and reducing the chance of finding all the potential literature. A non-systematic route,
such as following colleaguesâ recommendations and tracing references used in articles, may yield highly
relevant supplementary material. In the current example, these suggestions led to the John Hopkins
University Center for Talented Youth, a source of much relevant research and many earlier papers. The key
point to be made here is the need for students to keep meticulous records so that the whole process is
transparent and reproducible.
Deconstruction
Having identified the relevant literature, students need to digest the content, to decide how much data to
Key search strings used Database - ERIC Number meeting
criteria
Database - EBSCO Number meeting
criteria
Gifted students 279 search too
broad
0 60 11
Gifted students and
university
19 8 16 0
Gifted students and higher
education
Duplicate results as
above search term
Duplicate results as
above search term
10 0
Motivated students and
university
6 1 60 2
Successful students and
university
7 2 129 3
Engaged students and
university
3 0 136 0
High performing students
and university
0 0 147 3
Keen students and
university
0 0 12 0
Higher achieving students 13 3 199 0
Higher achieving students
and higher education
3 3 98 3
Higher achieving students
and university
0 0 53 1
Receptive students and
university
0 0 8 0
Brights students and
university
0 0 12 0
Total usable 17 23
5
International Journal of Management Education 9(4), 2011
6. Initial questions Response
What is my purpose in reading this material? To identify what motivates gifted students to higher
performance at university
When was the research reported on actually done? Between 2003-2006
What type of literature is this? An academic, peer reviewed journal article in the
learning and teaching subject area.
How extensive are the range of sources? A fairly wide range of mostly academic journal
articles including leading authors in the subject,
could have included more practitioner based subject
material.
Is the approach inductive or deductive? Deductive
Data collection methods: what was actually done? 3 year data collection using sample of 127
undergraduate business students on 2nd year
compulsory module looking for patterns in
attendance, characteristics of student and academic
achievement. Data collection details not provided.
Who is the intended audience? Other academics with an interest in the
achievement of undergraduate students.
Style
Is it constructed clearly? Yes
Is this an opinion piece, a conceptual paper, an
empirical paper, a research agenda, a systemised
review?
This is a piece of empirical research
Is it written clearly with key terms well defined? Yes
To what extent can you follow the argument through
a logical development?
There is a central theme and the research is
developed from relevant literature findings.
Does the use of tables, diagrams, and charts add
value to the explanation or the conclusions?
Some figures provide more insight than others,
tables of regression values have minimal value other
than to prove the regression work has been done.
extract and then to assess its worth. As a first step, many students need to improve their use of reading,
which involves an understanding of how argument works in text and the comprehension of the material in
hand (Du Boulay, 1999; Browne & Freeman, 2000). They need to be taught how to develop the skills of
appraisal and critique (Chamberlain & Burrough, 1985; Du Boulay, 1999). In our view, students need to learn
to regard academic journal articles and books not as repositories of facts, but as a series of arguments that
can be analysed and dissected. An ability to deconstruct argument is important, but only part of what is
required for learning. New knowledge is formed on the basis of previous understanding, but there has to be
some reflection as well (Lundquist, 1999).
In the course of teaching undergraduates and postgraduates we have developed with them a template. This is
shown, with a brief example of how it can be used, in Table 2, It contains a series of questions to encourage
students to read critically and to record their views. We introduce it as part of a classroom exercise when
students read a journal article they have been given in advance of the class, using the template. It has been
developed from a wide range of pedagogic sources and with students themselves (see, for example,
Greenhalgh, 2001; November, 2002; Metcalfe, 2006; Quinton & Smallbone, 2006; Wallace & Wray, 2006).
The questions in the template are designed to prompt students to think more deeply about the content,
message and structure of the articles they read and, crucially, the authorâs perspective, approach and
ontology. Their answers should provide them with a means of entry into the authorâs thinking, as well as a
useful source of comments for the later stages of writing up the literature review. Many of the questions are
designed to encourage students to be reflective and self-reflective: the intention is to encourage them to
think rather than report. At the same time, they are also prompted to think about the clarity of the authorâs
style and use of argument, and the effectiveness of the use of tables, diagrams and charts. This is to help
them develop an understanding of what makes for good academic writing and to give pointers on how to
develop their own style.
The core questions in the analysis section of the template are to aid students in the deconstruction of the
authorâs material and argument. They deal with issues such as the robustness of the research methods used,
the extent to which the results can be generalised to other cases and other situations, and the nature of the
evidence that supports such assumptions. The issue of author bias, especially cultural bias, is also raised
here. This is an interesting area for student discussion and reflection.
6 International Journal of Management Education 9(4), 2011
7. Analysis
What is the central issue dealt with in the paper? Is
it an important issue? Why?
The issue is whether higher academic achievement
can be linked to certain variables and if so what
might be the implications for British university
educators. It is an important area, as there exists a
desire to raise achievement level at British
universities in order to remain competitive with
other countries in the longer term.
Is there a particular bias, intellectual or cultural or
methodological?
There is a cultural bias as it was a British based
piece of research appearing in a British based
journal. Though the author is British, he works at a
Norwegian university.
What assumptions have been made e.g. about the
âgeneralisabilityâ of the results?
It is suggested that the findings may be transferable
across other modules and courses, and by inference,
across other universities.
What is the evidence supporting these assumptions?
How explicit are the conceptual frameworks?
No evidence provided. No conceptual framework
produced for future research but that was not an
aim of the paper.
What level of certainty do the authors show when
making any claims?
In places they cite other authorsâ results to support
their own findings and discussion. However many of
these stem from research conducted in other
educational cultures and countries.
To what extent are othersâ work which may be
contradictory included?
Only once or twice in passing.
How can you verify the results? As the details of the data collection were not given,
not easily.
Reflection
How do you respond to what the author is saying? The results are logical and not surprising.
How convincing is this piece of work?
How do you rate this article?
Although interesting, it is limited owing to its
sample size and narrowness. On a scale of 1-10, a 6.
How does it relate to other concepts you have come
across?
The paper highlights concepts that occur elsewhere
in the literature, for example; culture, attendance,
predisposition to study.
To what extent are the claims made consistent with
your own experience and knowledge?
The claims made as to relevant work experience on
which to draw, maturity and A level entry are not
surprising and are consistent with own experience
and reading.
Does it point to further research in a particular
direction?
Yes, but only broadly, this aspect could have been
given more detail and depth.
How might it be relevant to your current work? It provides insight into factors influencing British
undergraduate achievement which can be fed into a
wider literature review on key motivations of
students to higher performance at university.
Table 2: Worked example of question template for deconstructing academic material using Halpernâs 2007 paper
We encourage students to use the template with all the academic texts they might read. As they read and
make notes on an article they can fill in the template, which will provide them with an aide memoire for
each article. Students can collect a bank of structured notes from which to start building the third stage of a
literature review, the synthesis and original contribution to knowledge.
Reconstruction and synthesis
The third stage involves teaching students how to reconstruct material into a new pattern. This in itself is
complex. There is little pedagogic guidance on how to lay bare the thought processes involved. Although
many business research methods text books contain a chapter on âwriting upâ or even on âwriting your
literature reviewâ, they skip over the whyâs and the howâs of synthesising an argument from a disparate body
of academic literature (see, for example, Collis & Hussey, 2009; Ghauri & Grønhaug, 2005; Hair, Money,
Samouel, & Page, 2007).
A synthesis, as Suri and Clarke (2009) suggested, transforms data and or concepts. It takes what has been
reported or described in the original studies, adds the researcherâs interpretation, and then recasts it into
new findings and interpretations. Research synthesis is itself an emerging area of research methods
literature. It encompasses the range of methods of synthesising qualitative research data, as described in the
standard texts on qualitative data analysis. Transparency about the process is essential as it enhances
accountability, authenticity and credibility; three key aspects of validity in qualitative data analysis (Quinton
7
International Journal of Management Education 9(4), 2011
8. & Smallbone, 2005). Key authors include Yin (1989), and Denzin and Lincoln (1989), who discuss the synthesis
of multiple cases or exemplars; Miles and Huberman (2002), who discuss cross-case analysis; and Noblit and
Hare (1988), who developed the term meta-ethnography in the context of educational research, since
extended to medicine (Campbell et al., 2003).
In our teaching, we emphasise the staged approach to creating literature reviews (outlined in Figure 1) and
rely on the use of metaphor (Metcalfe, 2006) to underline and illustrate the deconstruction and final
reconstruction process. The use of a simple metaphor to help students reconstruct and create something of
value has, in our experience, been easily understood, and may be applied by students from diverse
educational and cultural backgrounds at both undergraduate and postgraduate levels. We emphasise that the
approach we follow is similar to the process of analysing qualitative primary data. In other words, the results
of the literature review should be treated as a data set (Ogawa & Malen, 1991) that requires further
transformation. We suggest that students review the contents and organisation of their wardrobes as a means
to encourage their ability to re-interpret information. This exercise can be carried out in the imagination.
They are asked to suggest how their clothes could be organised by, for example, arranging all jackets
together on hangers; then trousers, pullovers, shirts, t-shirts, and so forth; and by keeping other items such
as shoes together in one place. The wardrobe itself signifies the literature and prior research on the subject
area. The students are then encouraged to develop some selection criteria, such as which clothes they wear
most, or that they would least like to lose, or those destined for the charity shop. When applied to academic
literature these selection criteria would be equally explicit, such as relevance to the research question;
robustness of the original data; date of research and publication; and its cultural context. It is important to
stress at this stage that the selection criteria are not arbitrarily created, but can be defended and justified.
Once identified, the criteria for selection must be rigorously applied. The garments selected will remain in
their sub categories (all the shirts, all the trousers, etc) until they are then re-sorted into as many different
new categories as possible. For clothing, new categories might be by colour, occasion for wear, style, type of
fabric, season or garment care; mimicking the de-construction process in the literature review. The final step
is to create new categories that make for a coherent and transcendent categorisation that includes all the
remaining clothes. This could arise, for example, by creating outfits that work together or that will be
appropriate for particular situations, such as working in an office or going to a party. Of course, it is
emphasised that when the students write their reviews, these new categories or concepts should relate to
the research objectives. A possible next step is to encourage students to discuss how they will categorise and
sort the key data from the academic literature they have reviewed. Qualitative data analysis or concept
mapping software may provide helpful support to the process of data deconstruction and re-assembly. For
example, NVivo software can be used to tag, sort and categorise information from articles into small groups.
Alternatively, Atlas ti software can create maps of information from articles into linked clusters. This can
enable students to create new insights and to perceive the links between these groups.
Figure 1: A framework for teaching literature reviews
Conclusion
This paper has provided a concrete, three-stage approach to teaching literature reviews. It bridges the gap
between the need for management students to produce a literature review and the current lack of specific
tools for teachers to guide students through the process. Writing a literature review involves higher order
thinking and processing skills, and requires critical thought, all of which are hard to achieve. The approach
8 International Journal of Management Education 9(4), 2011
9. outlined is intended to make explicit how to approach an aspect of academic endeavour that it is often
wrongly assumed that students know how to undertake. Simplifying the complexity of creating literature
reviews is beneficial to both student and teacher.
Unfortunately, it is beyond the range of this article to demonstrate the efficacy of the approach suggested in
the third stage of the framework and to incorporate empirical data. This is a limitation. Possible further
research might be to adopt the whole approach and collect feedback from students on how useful and helpful
they found this framework to support their writing of literature reviews.
Most students are familiar with the use of databases to identify relevant literature. The computerisation and
application of technology to academic literature searching has led to an explosion in the amount of data
available. A systematically informed search of electronic databases, supplemented by hand searches, with all
activities carefully documented, enhances the rigour of studentsâ data collection, and the validity and
reliability of their later analysis and evaluation. It also helps to develop their ability to critique material and
their confidence in handling data. The ability to create a system to document the search findings and form
criteria for considering literature for inclusion, by filtering and filleting the material that has been collected,
is a necessary and important research skill in itself. Students need to be taught how to make this stage
transparent. In itself, this stage assists in establishing the rigour of the review at the outset, and it improves
the credibility of the work for supervisors and examiners.
The use of a question template to investigate each text or journal article eases the deconstruction of
complex material. The application of a simple proforma makes investigating a piece of academic literature
less intimidating; using the question template as outlined helps to give students a method of accessing the
meaning and value of the material they have read. Used in isolation, it could encourage a rather formulaic
approach to reading academic literature but, as part of a three stage approach, we have found that it
engages students who find this process difficult.
The third stage uses a simple metaphor as a tool that can readily be translated across cultures and
educational backgrounds to demonstrate the principles of reconstruction and synthesis. Students learn to
appreciate the potential to create something new and valuable from the elements of what had existed
before. In this stage, the studentâs learning may be transformed as the explicitness of the process assists
learning. The techniques can then be embedded through practice. By using the wardrobe metaphor to
practise the reconstruction of ideas in different configurations, students become more experienced in how to
embark on the process of synthesising ideas, concepts and theories. Students should then be able to
demonstrate how they may add value and contribute to knowledge through applying the principle to their
own management research work.
The writing of literature reviews is complex but the process does not need to be shrouded in mystery.
Practising systematically informed searches for academic literature and sifting and filtering the results to
prune the available material develops studentsâ analytical skills, as does the re-construction of argument and
narrative. The development of a nascent authorâs voice through a management studentâs expression of their
own ideas based on their own interpretation of the literature is one which teachers would wish to encourage
and this paper provides tools to support this process.
References
Alexander, P., Murphy, K., & Guan, J. (1998). Differences among low, average and high achieving college students on learning
and study strategies. Educational Psychology, 18, 391-407. doi:10.1080/0144341980180403
Bailin, S., Case R., Coombs J. R, & Daniels, L. B. (1999). Common misconceptions of critical thinking. Journal of Curriculum
Studies, 31, 269-283. doi:10.1080/002202799183124
Baker, M. J. (2000). Writing a literature review. The Marketing Review, 1, 219-247. doi:10.1362/1469347002529189.
Betts, P., & Lound, C. (2010). The application of alternative modes of data collection in UK Government social surveys, a
report for the Government Statistical Service. Office for National Statistics. Retrieved June 11, 2010, from
http://www.ons.gov.uk/reports-and-publications
Bloom, B. S. (1956). A taxonomy of educational objectives: The classification of educational goals. New York: McKay.
Boaz, A., Ashby, D., & Young, K. (2002). Systematic reviews: what have they got to offer evidence based policy and practice?
London: ESRC UK Centre for Evidence Policy and Practice.
Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in
research preparation. Educational Researcher, 34(6), 3-15.
Browne, M. N., & Freeman, K. (2000). Distinguishing features of critical thinking classrooms. Teaching in Higher Education, 5,
301-309. doi:10.1080/13562510050084703
Bruce, C. S. (1994). Research studentsâ early experience of the dissertation literature review. Studies in Higher Education,
19, 217-229.
Campbell, R., Pound, D., Pope, C., Britten, N., Pill, R., Morgan, M., & Donovan, J., (2003). Evaluating meta-ethnography: A
synthesis of qualitative research on lay experiences of diabetes and diabetes care. Social Science and Medicine, 56,
671-684. doi:10.1016/S0277-9536(02)00064-3
Chamberlain, K., & Burrough, S. (1985). Techniques for teaching critical reading. Teaching of Psychology, 12, 213-215.
doi:10.1207/s15328023top1204_8
9
International Journal of Management Education 9(4), 2011
10. International Journal of Management Education 9(4), 2011
10
Clegg, S. (2005). Evidence-based practice in educational research: A critical realist critique of systematic review. British
Journal of Sociology of Education, 26, 415-428. doi:10.1080/01425690500128932
Collis, J., & Hussey, R. (2009). Business research: A practical guide for undergraduate and postgraduate students (3rd ed.).
Basingstoke, England: Palgrave Macmillan.
Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997). Systematic reviews: Synthesis of best evidence for clinical decisions.
Annals of Internal Medicine, 126, 376-380.
Cooper, H. M. (1989). Integrating research: A guide for literature reviews (2nd ed.). Newbury Park, CA: Sage.
Criollo, R. (2003). Teaching TESOL undergraduates to organize and write literature reviews. The Internet TESL Journal, 9(4).
Retrieved May 19, 2010, from http://iteslj.org/Techniques/Criollo-LitReview.html
Davies, P. (2000). The relevance of systematic reviews to educational policy and practice. Oxford Review of Education, 26,
365-378. doi:10.1080/713688543
Denyer, D., & Tranfield, D. (2006). Using qualitative research synthesis to build an actionable knowledge base. Management
Decision, 44, 213-227. doi:10.1108/00251740610650201
Denzin, N. K., & Lincoln, Y. S. (Eds.). (1989). The landscape of qualitative research, theories and issues. Thousand Oaks, CA:
Sage.
Deurenberg, R., Vlayent, J., Guillo, S., Oliver, T., Fervers, B., & Burgers, J. (2007). Standardization of search methods for
guideline development: An international survey of evidence-based guideline development groups. Health Information
and Libraries Journal, 25, 23-30. doi:10.1111/j.1471-1842.2007.00732.x
Dixon-Woods, M., Bonas, S., Booth, A., Jones, D., Miller, T., Sutton, A., et al. (2006). How can systematic reviews incorporate
qualitative research? A critical perspective. Qualitative Research, 6, 27-44. doi:10.1177/1468794106058867
Du Boulay, D. (1999). Argument in reading: What does it involve and how can students become better critical readers?
Teaching in Higher Education, 4, 147-162. doi:10.1080/1356251990040201
Foese, A., Gantz, B., & Henry, A. (1998). Teaching students to write literature reviews: A meta-analytic model. Teaching of
Psychology, 25(2), 102-105. doi:10.1207/s15328023top2502_4
Ghauri, P., & Grønhaug, K. (2005). Research methods in business studies: A practical guide (3rd ed.). Harlow, England: FT
Prentice Hall.
Granello, D. H. (2001). Promoting cognitive complexity in graduate written work: Using Bloomâs taxonomy as a pedagogical
tool to improve literature reviews. Counselor Education and Supervision, 40, 292-307.
Greenhalgh, T. (2001). How to read a paper: The basics of evidence based medicine (2nd ed.). London: BMJ Books.
Haig, A., & Dozier, M. (2003). Systematic searching for evidence in medical education - Part 2: Constructing searches. Medical
Teacher, 25, 463-484. doi:10.1080/01421590310001608667
Hair, J. F., Money, A. H., Samouel, P. & Page, M. (2007). Research methods for business. Chichester, England: John Wiley.
Halpern, N. (2007). The impact of attendance and student characteristics on academic achievement: Findings from an
undergraduate business management module. Journal of Further and Higher Education, 31, 335-349.
doi:10.1080/03098770701626017
Hammersley, M. (2001). On âsystematicâ reviews of research literatures: A ânarrativeâ reply to Evans & Benefield. British
Educational Research Journal, 27, 543-554. doi:10.1080/01411920120095726
Harris, M. J. (2006). Three steps to teaching abstract and critique writing. International Journal of Teaching and Learning in
Higher Education, 17, 136-146.
Hart, C. (1998). Doing a literature review: Releasing the social science research imagination. Thousand Oaks, CA: Sage.
Hawkins, R. G. (1984). International business in academia: The state of the field. Journal of International Business Studies,
15(3), 13-18. doi:10.1057/palgrave.jibs.8490491
Institute of Directors. (2007). Institute of Directors skills briefing â December 2007: Graduatesâ employability skills.
Retrieved June 9, 2010, from http://www.IoD.com
Jackson, G. B. (1980). Methods for integrative reviews. Review of Educational Research, 50, 438-460.
Jankowicz, A. D. (2000). Business research projects (3rd ed.). London: International Thomson Business Press.
Leki, I. (1992). Building expertise through sequenced writing assignments. TESOL Journal, 1(2), 19-23.
Liu. Y., & You, X. (2008). Negotiating into academic discourse: Taiwanese and US college students in research writing.
International Journal of English Studies, 8(2), 152-172.
Lundquist, R. (1999). Critical thinking and the art of making good mistakes. Teaching in Higher Education, 4, 523-530.
doi:10.1080/1356251990040408
McNally, R., & Alborz, A. (2004). Developing methods for systematic reviewing in health services delivery and organization:
An example from a review of access to health care for people with learning disabilities. Part 1: Identifying the literature.
Health Information and Libraries Journal, 21,182-192. doi:10.1111/j.1471-1842.2004.00512.x
Metcalfe, M. (2006). Reading critically at university. London: Sage.
Miles, M. B., & Huberman, A. M. (2002). The qualitative researcherâs companion. Thousand Oaks, CA: Sage.
Neber, H., Finsterwald, M., & Urban, N. (2001). Cooperative learning with gifted and high-achieving students: A review and
meta-analysis of 12 studies. High Ability Studies, 12, 199-214. doi:10.1080/13598130120084339
Noblit, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. Newbury Park, CA: Sage.
November, P. (2002). Teaching marketing theory: A hermeneutic approach. Marketing Theory, 2, 115-132.
doi:10.1177/1470593102002001646
Oakley, A., Gough, D., Oliver, S., & Thomas, J. (2005). The politics of evidence and methodology: Lessons from the EPPI-
Centre. Evidence & Policy: A Journal of Research, Debate and Practice, 1, 5-32. doi:10.1332/1744264052703168
Oakley, A. (2006). Resistances to ânewâ technologies of evaluation: Education research in the UK as a case study. Evidence &
Policy: A Journal of Research, Debate and Practice, 2, 63-87. doi:10.1332/174426406775249741
Ogawa, R. T., & Malen, B. (1991). Towards rigor in reviews of multivocal literature: Applying the exploratory case method.
Review of Educational Research, 61, 265-286.
Pawson, R. (2001). Evidence based policy: II The promise of realist synthesis. London: ESRC Centre for Evidence Based Policy
and Practice.
Phillips, N., & Lindsay, G. (2006). Motivation in gifted students. High Ability Studies, 7, 57-73.
doi:10.1080/13598130600947119
Pope, C. (2003). Resisting evidence: The study of evidence-based medicine as a contemporary social movement. Health, 7,
267-282. doi:10.1177/1363459303007003002
11. International Journal of Management Education 9(4), 2011 11
Quality Assurance Agency. (2007). QAA subject benchmark statements: General business and management. Retrieved June
18, 2010, from http://www.qaa.ac.uk/academicinfrastructure/benchmark/statements/GBM.pdf
Quinton, S., & Smallbone, T. (2005). The troublesome triplets: Issues in teaching reliability, validity and generalisation to
business students. Teaching in Higher Education, 10, 299-311. doi:10.1080/13562510500122099
Quinton, S., & Smallbone, T. (2006). Postgraduate research in business: A critical guide. London: Sage.
Rigby, K. (2005). âRocky Mountain Talent Searchâ at the University of Denver. High Ability Studies, 16, 71-75.
doi:10.1080/13598130500115270
Sackett, D. L., Rosenberg, W. M. C., Muir, D., Gray, J. A., Haynes, R. B., & Richardson, W. S. (1996). Evidence based
medicine: What it is and what it isn't. British Medical Journal, 312, 71-72.
Saunders, M., Lewis, P., & Thornhill, A, (2007). Research methods for business students (4th ed.). Harlow, England: Pearson
Education.
Smallbone, T., Hartley, J., & Quinton, S., (2007, May). Critical reading: Developing an approach that will engage students.
Paper presented at the Annual BMAF Conference 2007: Meeting the challenges facing learning and teaching in a
globalised world, Aston University, England.
Suri, H., & Clarke, D. (2009). Advancements in research synthesis methods: From a methodologically inclusive perspective.
Review of Educational Research, 79, 395-426. doi:10.3102/0034654308326349
Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management
knowledge by means of systematic review. British Journal of Management, 14, 207-222. doi:10.1111/1467-8551.00375
Varndarajan, P. R. (2003). Musings on relevance and rigor of scholarly research in marketing. Journal of the Academy of
Marketing Science, 31, 368-376.
Wallace, M., & Wray, A. (2006). Critical reading and writing for postgraduates. London: Sage.
Yin, R. K. (1989). Case study research: Design and methods. Thousand Oaks, CA: Sage.
Zorn, T., & Campbell, N. (2006). Improving the writing of literature reviews through a literature integration exercise.
Business Communication Quarterly, 69, 172-183. doi:10.1177/1080569906287960