A Delphi Toolkit For Ecotourism Research

This article was downloaded by: [ University of Florida] , [ Holly Donohoe]
On: 04 January 2012, At: 08: 26
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Ecotourism
Publication details, including instructions for authors and
subscription information:
http:/ / www.tandfonline.com/ loi/ reco20
A Delphi toolkit for ecotourism
research
Holly M. Donohoe
a
a
Department of Tourism, Recreation and Sport Management,
University of Florida, Gainesville, FL, USA
Available online: 15 Jan 2011
To cite this article: Holly M. Donohoe (2011): A Delphi toolkit for ecotourism research, Journal of
Ecotourism, 10:1, 1-20
To link to this article: http:/ / dx.doi.org/ 10.1080/ 14724040903418897
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http: / / www.tandfonline.com/ page/ terms-and-
conditions
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation
that the contents will be complete or accurate or up to date. The accuracy of any
instructions, formulae, and drug doses should be independently verified with primary
sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand, or costs or damages whatsoever or howsoever caused arising directly or
indirectly in connection with or arising out of the use of this material.

A Delphi toolkit for ecotourism research
Holly M. Donohoe∗
Department of Tourism, Recreation and Sport Management, University of Florida, Gainesville,
FL, USA
(Received 5 September 2008; final version received 13 October 2009)
The Delphi is considered a legitimate and valuable research technique in a range of
research fields. Its value is based on its utility for forecasting, issue identification or
prioritisation, and concept or framework development. However, systematic guidelines
for designing and executing a Delphi are not widely available. Concomitantly, a
plethora of methodological interpretations are proving a source of confusion for the
Delphi architect. This paper explores the utility of a proposed generic Delphi toolkit
(GDT) for designing and implementing a Delphi study. An ecotourism research
example is introduced as a basis for testing the GDT and for a critical review of its
benefits and challenges for ecotourism research.
Keywords: ecotourism; Delphi technique; research design
Introduction
It is now approaching 60 years since the first Delphi was conducted and more than 30 since it
was first applied by tourism researchers. During this time, the Delphi has broadened its fields
and frequency of application. Despite the method’s growing popularity, the Delphi literature
hosts two compelling concerns that serve as the foundation for this paper. The first concern is
related to methodological refinement. Methodological critique and evaluation are necessary
for identifying opportunities and constraints and for encouraging methodological refinement
and best practice (Alberts, 2007; Brown, 2007). In its place, the Delphi literature is dominated
by one-off case studies that lack sufficient detail about their design and execution to support
methodological evaluation (Powell, 2003). Mullen (2003) and Rowe and Wright (1999)
argued that our knowledge about the structure and potential of the Delphi is poor as a
result. This trend is inexorably linked to a second concern related to Delphi architecture.
The literature has come to suggest that there is no reference standard for Delphi architects
and there has been difficulty in drafting a ‘generic’ or ‘universal’ description of the
method and its procedures (Linstone & Turoff, 2002). As a consequence, there exists real
potential for confusion and ad hoc Delphi design and this raises serious questions about
the method’s validity and reliability. Given that guidelines are essential for Delphi design
and administration, that a lack of guidelines is contributing to confusion and that confusion
is impeding methodological review, the development and evaluation of a ‘generic’ description
of the method is required for methodological refinement.
ISSN 1472-4049 print/ISSN 1747-7638 online
# 2011 Taylor & Francis
DOI: 10.1080/14724040903418897
http://www.informaworld.com
∗
Email: hdonohoe@hhp.ufl.edu
Journal of Ecotourism
Vol. 10, No. 1, March 2011, 1–20
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

Day and Bobeva (2005) proposed a generic Delphi toolkit (GDT) for providing guidance
for Delphi research. If the GDT is substantiated as a well-proven and robust instrument, the
authors claim that the Delphi could be applied in a wider range of disciplines and methodologi-
cal confusion could be reduced. The validity of the GDT has yet to be tested; therefore, the
remainder of this paper addresses the call for Delphi research concerned with methodological
testing and refinement by testing the GDT. In doing so, this paper offers several contributions
towards addressing the aforementioned concerns. First, a critical review of the Delphi tech-
nique is provided and the GDT is introduced. Second, a case study example – a research
agenda concerned with cultural sensitivity and ecotourism – is presented in order to test the
utility of the GDT for guiding Delphi architecture and administration. By testing the validity
and appropriateness of GDT, this paper seeks to consolidate understanding of the Delphi
technique and to offer insight into its application based on a ecotourism research case.
The Delphi technique
The Delphi is a technique used to systematically combine expert knowledge and opinion to
arrive at an informed group consensus (Moëller & Shafer, 1994). The Delphi technique was
originally developed as a tool for soliciting opinion from a group of experts in order to
inform a forecasting process and it is now accepted as a valid technique for a variety of
research purposes (Landeta, 2006). Accordingly, the Delphi technique has been applied
in a breadth of fields that include: environmental management (Gokhale, 2001; Wright,
2006), business marketing and management (Chevron, 1998; Hayes, 2007), recreation
(Austin, Lee, & Getz, 2008; Vaugeois et al., 2005), and health (de Meyrick, 2003). In
the tourism case, the Delphi has been applied for the purposes of long-range travel and
tourism planning (Moëller & Shafer, 1994), to predict change in the tourism industry
(Lloyd, La Lopa, & Braunlich, 2000; Yeong, Keng, & Leng, 1989), to develop ecotourism
and sustainable tourism management and evaluation frameworks (Miller, 2001; Spenceley,
2005), and to develop and/or refine ecotourism definitions (Garrod, 2003).
The Delphi technique is a structured communication process that facilitates (but does
not force) consensus (Linstone & Turoff, 1975). It is founded on the use of techniques
that aim to develop from a group of experts, an agreed view or shared interpretation on a
topic or issue where there is contradiction, controversy, or conflict (Day & Bobeva,
2005). Individual expert contributions are collected through iterative survey rounds,
group opinion is assessed, and opportunity is provided for individuals to revise their con-
tributions (controlled feedback) (Linstone & Turoff, 1975). Evaluation of the Delphi tech-
nique reveals a set of key Delphi attributes and benefits (Donohoe & Needham, 2009):
(1) Legitimacy: An established research technique.
(2) Suitability: Well suited to complex problems where the contributions of experts
would contribute to advancing understanding and knowledge about the problem.
(3) Proximity: ‘Virtual laboratory’ where physical meetings are not required.
(4) Reflexivity: By design, participants think through concepts and questions – so that
quality and objectivity of the data collected and the credibility of the study’s find-
ings may be enhanced.
(5) Flexibility: Methodological adaptation facilitates a comprehensive understanding
of the research problem.
(6) Repetition: Multiple iterations move the group towards an informed judgement
about a complex problem.
(7) Anonymity: Reduces the risk(s) for group dynamics to influence outcomes.
2 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

Delphi proponents advocate anonymity as a means for facilitating rational and reflexive
individual contribution and for reducing the influence of dominant participants, the power
of persuasion, and the bandwagon effect that is associated with other group methods
(Duboff, 2007; Linstone & Turoff, 2002; Rowe & Wright, 1999). Sunstein (2006)
argued that anonymity is what distinguishes the Delphi from other group techniques and
on this basis, Wolfers and Zitzwitz (2004) and Rowe and Wright (1999) have found the
Delphi to be more accurate than other group techniques. However, anonymity does not
rule out the possibility of researchers giving undue credence to the opinion of select
experts, the tendency of the individuals to be influenced by the statistical measure of
group judgement, or the comments of panelists reported in summary reports (Wright,
2006). Other potential problems associated with the Delphi have been comprehensively
reviewed elsewhere. Representative examples include critiques and discussion related to
expert selection (Alberts, 2007; Phillimore & Goodson, 2004), questionnaire ambiguity
(Sackman, 1975; Seely, Iglarsh, & Edgell, 1980), false consensus (Linstone & Turoff,
1975), panel stability, the substantive quality of panelists contributions (Alberts, 2007;
Mullen, 2003; Wright, 2006), and the validity of the results (Landeta, 2006; Powell,
2003). Antecedent research suggests that the Delphi technique, like other methods, is not
free from flaws, bias, and other potential pitfalls or dangers (Briedenhann & Butts, 2006;
Murray, 1979). It is recommended that Delphi administrators recognise and address
potential constraints and the importance of professional judgement in control of the
exercise (Delbecq, Van de Ven, & Gustafson, 1975; Powell, 2003). Furthermore, care
must be afforded the final justification for the technique on the basis of its utility, its
suitability for the research problem, its potential advantages, and the measures available
for mitigating potential problems. Day and Bobeva (2005) contended that the GDT could
provide a basis for such justification and the guidance for doing so. As reminder, this
has yet to be substantiated in practice.
The GDT
Day and Bobeva’s (2005) GDT is a three-stage model for designing and implementing a
Delphi study. The model is informed by a comprehensive analysis of the Delphi literature.
Commonalities between Delphi techniques used by other researchers are identified and syn-
thesised into a set of generic Delphi guidelines. In this regard, the GDT is a reference stan-
dard for the design, adaptation, and implementation of a Delphi study. The first stage,
‘Exploration’, demands rigorous preparation where care is afforded the basis for consensus
(purpose statement), the establishment of participant selection criteria, and the creation of
an expert panel; the design of a data collection and analysis instrument; the identification of
an initial set of issues to be tested through Delphi implementation and/or pilot testing. Day
and Bobeva (2005) presented a set of generic design criteria and common options to help
guide Delphi architects through this preparatory phase (Table 1). For the purposes of this
paper, the term ‘Exploration’ has been changed to ‘Preparation’ to simplify the language
and make clear the stage objectives.
The second stage, referred to as ‘Distillation’, typically involves three rounds of surveys
that are circulated to a predetermined expert panel. The literature indicates that a scoping
round (Round 1), that is, a preliminary or general survey round, is commonly used
during the preparation phase as a means to circulate and solicit reactions to an introduction
package (initial problem and position statement) and/or pilot test surveys (Green, Hunter, &
Moore, 1990; Hurd & McLean, 2004; Weber & Ladkin, 2003). This scoping round is
optional, though it is recommended as it may help to identify ambiguities, it allows
Journal of Ecotourism 3
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

some degree of early validity testing and methodological refinement, improves the feasi-
bility of administration, and allows the researcher to narrow the study’s focus (Garrod &
Fyall, 2005; Powell, 2003). Following the optional scoping round, the Delphi moves
into the ‘Distillation’ stage where a survey is developed on the basis of the purpose state-
ment and the outcomes of the preparatory phase. The survey is circulated to the expert panel
and responses are collected (Round 2). The results are synthesised into a summary report
that is circulated to participants with the subsequent round. The report may not only
include simple statistical analysis (frequency tabulation, median, etc.), but it may also
include representative comments or criticisms from participants (anonymous) and a
consensus status update. The preparation of the report allows the Delphi administrators
the opportunity to analyse interim results and to monitor progress (or lack thereof)
towards consensus while the distribution of the report allows participants the opportunity
to reflect on their responses in light of the group’s judgement (Gordon, 1994; Rowe &
Wright, 1999). Subsequent rounds are essentially the same as Round 2, with surveys
(developed on the basis of the previous round) and summary reports provided to partici-
pants. The purpose of the iterative rounds, and particularly the feedback element that
comprise the ‘Distillation’ stage of the Delphi, is to move the group towards a convergence
of opinion (Day & Bobeva, 2005).
In the final round, a synthesis and/or evolving consensus statement is distributed and a
final judgement is requested from participants (Gordon, 1994). It is important to note that
additional rounds may be necessary if the Delphi has not yet reached a critical point for
study termination. To complete this assessment, there are two recommended approaches:
extensive statistical testing and simple statistical reporting. Examples of extensive statistical
testing for consensus include Okoli and Pawlowski’s (2004) use of Kendall’s W coefficient
of concordance and Schmidt’s (1997) consensus ratio. Others prefer simple statistics for
measuring the degree of convergence (Day & Bobeva, 2005). For example, the use of
the mean/median to measure the control tendency and the standard deviation as a determi-
nant of convergence is common in the tourism literature (Garrod & Fyall, 2005; Green
et al., 1990; Kaynak & Macauley, 1984; Miller, 2001). Holey, Feeley, Dixon, and Whittaker
(2007, p. 52) concur, adding that the evolution of consensus can also be ascertained by
descriptive analysis of group trends such as an increase in agreement percentages, conver-
gence of range with standard deviations, and as a decrease in the number of comments
made. When sufficient convergence has been achieved, the Delphi rounds cease and the
group judgement is used to inform the final report and the initial research problem
(Stage Three). ‘Convergence’ is commonly used in the literature to describe Delphi activi-
ties and objectives during this stage; thus, it replaces ‘Distillation’ in the remainder of this
paper.
Table 1. Generic design tool: Delphi criteria and options for the preparation phase.
Generic criteria Design options
1. Purpose of the study Building, exploration, testing, evaluation
2. Participants Homogeneous or heterogeneous groups
3. Anonymity of panel Full or partial
4. Number of rounds Between 2 and 10
5. Concurrency of rounds Sequential set of rounds
6. Mode of operation Face-to-face or remote access
7. Communication media Paper-and-pen, Internet-based, telephone, fax, email
Adapted after Day and Bobeva (2005).
4 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

The third stage of the Delphi, named the ‘Utilisation’ stage, includes the development
and dissemination of the final Delphi report. The results from the ‘Convergence’ phase are
integrated into a final consensus judgement and a report is developed on this basis. As an
option for Delphi administrators, a draft may be circulated to Delphi participants for one last
review. Although a group judgement has been achieved at this point, circulation of the draft
provides participants with a final opportunity to reflect on their responses and comment on
the Delphi outcome. Commentary is then integrated into the final report. Consideration
must also be given to the impacts of the consensus judgement on the initial research
problem during this stage of the Delphi. That is, the final report, and particularly the con-
sensus statement, is to be applied to the original research purpose so that the central research
question may be addressed. Dissemination is a distinctive feature of this stage and dissemi-
nation activities may be classified as both ‘short term’ and ‘long term’ in nature (Day &
Bobeva, 2005). In the short term, the final report is distributed to Delphi participants and
others who may have assisted in the Delphi exercise and/or have an interest in its
outcome (e.g. supporting organisations, research assistants, etc.). Long-term knowledge
dissemination activities include related publications that address both the substance and
the structure of the study. Although, ‘Utilisation’ aptly describes some of the activities at
this stage, it does not describe the essential Delphi product. Therefore, ‘Consensus’ is
used for the remainder of this paper to describe the third stage of a Delphi.
Delphi design and implementation: an ecotourism example
Using the GDT, the following sections critically assess the utility of the model for Delphi
architecture and administration. Each of the generic stages is explored through a Delphi
study concerned with ecotourism and cultural sensitivity. The decisions involved in its
design and the critical issues revealed by its implementation serve as focus. The ambition
is to bring the GDT to life in order to test its utility for guiding Delphi architecture generally,
and ecotourism research specifically.
Purpose of the study
The purpose of the research is to develop a cultural sensitivity definition and to test its
validity among a group of international ecotourism experts (Table 2). The Delphi technique
is applied in order to identify and describe the definitional elements as well as the
opportunities and barriers to culturally sensitive ecotourism. The results are to inform an
examination of the extent to which progress towards making ecotourism sensitive to
Table 2. Delphi response rates and convergence measures.
Round one Round two Round three
Response ratea
60% 84% 77%
Convergence measures Mean scoreb
4.2 4.26 4.34
Standard deviation 0.97 0.87 0.63
a
The research was completed between January 2008 and July 2008. As the research moved into the summer months
and the busy ecotourism season, it was expected that participation would decline. Therefore, a larger initial group
was sought in order to mitigate a potentially high attrition rate. This proved to be a useful strategy.
b
Definitional appropriateness scores measured on a five-point Likert-type scale where: 5 ¼ very appropriate, 4 ¼
appropriate, 3 ¼ somewhat appropriate, 2 ¼ not very appropriate, 1 ¼ not appropriate.
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

cultural differences (and similarities) in a global community is to proceed. In completing
these final steps, the research is to offer a practical product – a definition – that contributes
to ecotourism discourse related to definitions, cultural sensitivity, and management
guidelines.
The reason for this research focus is directly linked to a critical development in the eco-
tourism research and the recent identification of an ecotourism problem: the lack of cultural
sensitivity in ecotourism theory and praxis. Since 2000, a growing number of ecotourism
researchers are suggesting that a set of values – embedded in ‘Western’ understandings
of human/environment relationship – are being superimposed on ‘other’ or ‘non-
Western’ ecotourism destinations and their value systems (Backman & Morais, 2001;
Braden & Prudnikova, 2008; Carrier & Macleod, 2005; Cater, 2006; de la Barre, 2005;
Jamal, Borges, & Stronza, 2006; Stark, 2002; Weaver & Lawton, 2007). A lack of sensi-
tivity to the cultural context of ecotourism presents perils related to goal achievement.
The literature suggests that the real danger exists for model transference failure when a
single ‘ecotourism mould’ is used; there exists potential for benefits to be replaced with
insecurity, resentment, conflict, ecological degradation, and economic loss (Vivanco,
2002). Cater (2006, p. 36) asserted that ‘if we uncritically accept Western-constructed
ecotourism as the be-all-to-end-all, we do so at our, and others’ peril’. In response to
these concerns, ecotourism researchers and practitioners are calling for increased cultural
sensitivity, increased reflexivity in research and praxis, and a re-thinking of the so-called
‘universal’ ecotourism knowledge. Sofeild (2007, p. 158) concurred: ‘where different
world views and different aesthetics are involved, a greater degree of acceptance and
understanding for difference is imperative’.
For ecotourism to truly exemplify sustainable development and the core ecotourism
tenets, the ‘mould’ must be sensitive to culture (Cater, 2006; de la Barre, 2005; Honey,
1999; Humberstone, 2004; McCool & Moisey, 2001; Stark, 2002; Vivanco, 2002). In
the ideal case, cultural sensitivity is a key ecotourism attribute as culture has the potential
to affect ecotourism opportunities, experiences, and management (Jamal, Borges, &
Stronza, 2006). Perceptions, attitudes, and values may also have the potential to affect
opportunities, experiences, and management, and this cultural dimension has the capacity
to affect ecotourism’s social relevancy (Nyiri, 2006; Sofeild, 2007). Therefore, the attribute
denotes the sensitivity level of ecotourism to cultural differences and similarities and it
ultimately dictates the adaptation investments (e.g. policy development, training
modules, regulations, programmes, research agendas) required of ecotourism stakeholders
and leaders to make more acceptable the sharing of core definitions, planning and manage-
ment tools and expertise, and international standards in the cultural mosaic of our global
human landscape. To achieve this culturally sensitive ideal, multiple (and overlooked)
ecotourism realities need to be identified and the influences, potential conflicts, and out-
comes of such phenomena must be understood. The latter requires an epistemology and
methodology that highlights and prioritises cultural sensitivity and that is counter-
hegemonic by design (Denzin & Lincoln, 2005; Kincheloe & McLaren, 2005; Saukko,
2005). This includes definitions, standards, research models, visions, and strategies that
are sensitive to the diversity of cultural values and identities that exist in the ecotourism
landscape (de la Barre, 2005). Given the contemporary ecotourism context and the
rapid development of ecotourism markets in emergent destinations – such as Bhutan,
Cambodia, China, Thailand, and Vietnam, a response to this call – as Sofield suggests,
is imperative.
This research problem and purpose statement is the foundation for all other design
decisions during the ‘Preparation’ stage of the Delphi.
6 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

Methodology
The Delphi technique was selected over traditional group methods such as the group inter-
view or the focus group, as a valid method for eliciting knowledge and opinion from a
group of international ecotourism experts. The method was judged to be a best-fit on the
basis of its suitability for tourism research (Donohoe & Needham, 2009; Garrod & Fyall,
2005; Green et al., 1990) and its demonstrated utility for informing a definition develop-
ment process (Garrod, 2003). Delphi attributes (described earlier in the text) may also be
considered advantages that are congruent with the research purpose and problem. The struc-
ture, utility, and purpose of the Delphi technique are congruent with the work of de la Barre
(2005) and Stark (2002) who suggest a need for a ‘constructed space’ for consensus build-
ing. Given that ecotourism has historically been advanced by non-inclusive knowledge
systems and that this process finds us now in a potentially calamitous predicament, it
makes the most sense to apply a ‘counter-hegemonic’ methodology that supports Haber-
mas’s (1990) principle of ‘universalisability’ and the achievement of inclusive consensus.
In this way, the product of the Delphi study is to reflect multiple interests, values, and exper-
tise while eliminating the risk of furthering Cater’s (2006) ‘hegemonic sphere’ (an exclusive
or Western-derived model). Instead, the Delphi method offers a constructed space for con-
sensus building. It facilitates reflexivity and it allows for alternative viewpoints and/or
voices of the ‘other’ to be heard.
It is important to acknowledge that the Delphi exercise is to produce a definition that is
positioned in both time and space. This is an important consideration because the conditions
that are framing the exercise, including the global popularity of ecotourism and the recent
discourse related to cultural sensitivity, are likely to affect the outcome. The experts who are
to comprise the Delphi panel are also representative of a diversity of truths located in time
and space, and their opinions and contributions are likely to reflect this fact. Therefore, it is
recognised that the outcome of the research is contextual and subjective.
Implementing the Delphi
The preparation stage
The literature establishes that Delphi study objective achievement is intrinsically linked to a
carefully designed and executed Delphi plan (Day & Bobeva, 2005; Miller, 2001; Philli-
more & Goodson, 2004). Based on the GDT, a three-stage Delphi implementation plan
was developed to guide the study (Figure 1).
In the preparation stage, the expert panel design is identified as the most important
design decision (Needham & de Loë, 1990). Therefore, a Delphi study requires that the
declaration of selection criteria, the selection of experts, and the management of an
expert panel (Green et al., 1990; Linstone & Turoff, 2002) be carefully managed. A
Delphi study does not rely on a statistically representative sample of participants to
achieve valid results (Okoli & Pawlowski, 2004; Powell, 2003). Guidance suggests that
design decisions related to Delphi panel size, characteristics, and composition must ‘be
governed by the purpose of the investigation’, the scope of the problem, and the resources
available (Cantrill, Sibbald, & Buetow, 1996, p. 69). The expertise represented on the panel
must therefore be congruent with the research topic and issues.
For this study, a panel of international ecotourism experts was desired so that the Delphi
captures a diversity of experience, knowledge, skills, and cultural perspectives. Guided by
the GDT, a seven-phase purposive sampling strategy was used for identifying, recruiting
and selecting participants. It is detailed here because in application, a number of Delphi
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

design challenges were revealed and the decisions made to mitigate such challenges may
help to shed light on the technique’s strengths and weaknesses.
Phase 1 – panel size: The GDT suggests that there is variation in the requisite size for a
Delphi panel, but limited guidance is provided for making panel size decisions (panel size
that will best serve the research agenda). Thus, the researcher consulted the Delphi literature,
and particularly the tourism literature, for guidance in this regard. Linstone (1978) asserted
that accuracy deteriorates with smaller panel sizes and improves with larger numbers. For
Delphi studies that are composed of a mixed group of experts (heterogeneous group), Brie-
denhann and Butts (2006) recommend a larger group. The tourism literature contains
examples of panels in the range of 7–100 participants (Choi & Sirakaya, 2006; Green
et al., 1990; Kaynak & Marandu, 2006; Kuo & Chiu, 2006; Lloyd, La Lopa, & Braunlich,
2000; Miller, 2001; O’Connor & Frew, 2004; Tsaur, Lin, & Lin, 2006; Weber & Ladkin,
2003; Yeong, Keng, & Leng, 1989). The challenge in constructing a heterogeneous panel
of international ecotourism experts is in ensuring that the panel includes a diversity of cultural
backgrounds, perspectives, and experience. Given the global context of the research and its
cultural theme, this balance is imperative for producing a relevant and inclusive consensus.
Thus a goal of 75–125 participants was judged to be congruent both with the aims of the
study and Delphi research standards. Also, the goal was purposefully broad to facilitate
Figure 1. Three-stage Delphi approach.
8 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

inclusion and to mitigate attrition between rounds (see Phase 5). A minimum panel size of 20
participants was also determined on the basis of the research objectives as well as recommen-
dations found in the literature (Archer, 1980; Linstone & Turoff, 2002). The minimum estab-
lishes a critical level under which the Delphi panel can no longer be considered stable and the
Delphi can no longer be considered a rigorous exercise (Garrod, 2003). Thus, a safeguard
(minimum panel size and attrition rate/critical level monitoring) is in place in the case of
high attrition rates during the convergence stage. A balance between academic and pro-
fessional experts as well as their global distribution was desired and relatively equal represen-
tation throughout the Delphi exercise was maintained.
Phase 2 – panel expertise: A purposive sampling method was employed to select the
relevant experts concerning the study topic: (1) ecotourism professionals from government,
private industry and non-governmental organisations and (2) academics engaged in eco-
tourism research and education. It is assumed that these groups, together (heterogeneous
panel), will contribute rich and varied data and information for the Delphi study. The
inclusion of both professional and academic experts is substantiated by Alberts (2007),
Briedenhann and Butts (2006), Donohoe and Needham (2009), Sunstein (2006), and
Vaugeois et al. (2005) as a means to achieve a balance between differing approaches to
and perspectives on ‘knowledge’, to mitigate the existent divide between research and
practitioner communities (knowledge sharing, communications, priorities, epistemology,
etc.), and to achieve an inclusive Delphi outcome. An intriguing dichotomy is present,
and it was expected that consultations with these groups would provide meaningful
results. For this study, the ‘expert’ must be proﬁcient in English and they must satisfy a
minimum of one of the selection criteria (parameters for participant recruitment and
selection).
(1) Current or previous experience in the public or governmental sector related to eco-
tourism and/or nature-based tourism activities (minimum 5 years)
(2) Current or previous experience in the private sector related to ecotourism and/or
nature-based tourism activities (minimum 5 years)
(3) Evidence of professional productivity in terms of peer-reviewed or professional
publications and research and/or participation in academic or industry symposia
(4) Teaching portfolio that includes courses dedicated to tourism and/or ecotourism
(minimum 5 years)
(5) Other – potential participants are to be afforded the opportunity to provide insight
into ecotourism expert qualities and/or attributes that may have been overlooked
and that should be considered for participant selection.
The researcher recognises that the language requirement is a very important element in
this study as it introduced a potential bias – particularly because of the desired diversity of
‘voices’. However, English is the primary language of the researcher and for this reason, as
well as logistical reasons such as time and budget, multiple language editions of the survey
were not developed. Instead, English served as the unifying language for the panel and
where possible, language assistance was provided. For example, support was provided to
English-as-a-second-language participants in locations such as Canada, Brazil, China,
Greece, and Mexico so to avoid unnecessary language-based exclusion or attrition. The
irony of only using English in a study of cultural sensitivity is acknowledged.
Phase 3 – expert recruitment: Portals through which ecotourism experts were identiﬁed
and recruited include international peer-reviewed publications and organisations where
ecotourism experts publish and/or seek membership. Peer-reviewed journals provide
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

publication portals where ecotourism experts publish their research findings. For example,
the Journal of Ecotourism, the Journal of Sustainable Tourism, the International Journal of
Tourism Research, and Tourism Management are established publications where ecotour-
ism research is frequently published. Expertise is established through a rigorous peer-
review process. In addition to the selection criteria, peer-review serves as an additional
measure of expertise. International organisations where ecotourism experts seek member-
ship provide complementary portals for recruitment. The International Ecotourism
Society (TIES) and the International Ecotourism Club (ECOCLUB) are global organis-
ations where ecotourism experts (both academics and professionals) seek membership.
Unique to these organisations is a strong representation of ecotourism professionals from
a diversity of locations and cultural backgrounds. While academic experts are also rep-
resented in the membership ledgers, the [total] diversity of ecotourism expert members is
the strength and utility of these portals. In total, six portals are included as they offer unpar-
alleled access to ecotourism experts generally, and potential research participants specifi-
cally. In the absence of these select portals, the international pool of ecotourism experts
would be difficult to access and the challenge of constructing the desired Delphi panel
would be magnified.
Phase 4 – recruitment strategies: Using the expert portals, direct and assisted strategies
were used to identify potential participants and to create a potential participants database.
First, individuals with ecotourism research experience are identified through a strategic
review of papers published in the aforementioned journals within the last 5 years (2002–
2007). This direct approach was expected to produce 100 individuals but it produced
160. By way of the TIES portal, additional ecotourism experts were identified from the
membership and expert lists found on the organisations’ public website. Individuals and
their contact information were harvested and entered into the potential research participant
database. In addition to these direct methods, TIES and ECOCLUB assisted by circulating a
‘call for participants’ to their membership through electronic or print newsletters and/or
posting the ‘call’ on their website. Interested individuals were invited to submit an
‘expression of interest’ to the researcher. This assisted approach was expected and did
produce a pool of approximately 600 potential participants for the Delphi exercise. The
lists were cross-referenced to remove duplications and the final pool included a total of
706 individuals.
Phase 5 – panelist screening and selection: Potential participants were contacted by
email with an invitation to participate. The invitation contained a description of the research
and the participant selection criteria as well as a link to an Internet-based survey. The survey
required potential participants to indicate which, if any, of the expert selection criteria they
satisfy, to indicate their English language proficiency, to identify their nationality, to self-
identify as an academic, professional, or combination thereof, and to provide their contact
information. Respondents were provided with a 14-day period in which to respond. A
total of 159 individuals responded while 157 satisfied the expert selection criteria.
Phase 6 – panel composition: While creating a panel that is statistically representative
of the World’s population of cultures is likely to be laborious and time consuming, it was
not likely to add great(er) value to the study. Instead, the decision was taken to develop a
panel comprising a diversity of individuals from a variety of locations. To establish diver-
sity, two measures were employed. First, the number of nationalities, organised by conti-
nent, provided a simple diversity measure (Figure 2). A diversity of nationalities was
desired (i.e. at least 25) while a lack of diversity (i.e. few nationalities or a western predica-
tion) was to serve as indication that cultural richness had not been achieved. This was not
the case and no additional recruitment was required. While this measure may be considered
10 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

subjective by some, statistical sampling is acknowledged by the GDT as unnecessary for
establishing validity (Day & Bobeva, 2005). Instead, representativeness and validity is
established by other rules of thumb that are best described on the basis of their appropriate-
ness to the research at hand (Linstone & Turoff, 1975; Miller, 2001; Spenceley, 2005). The
panelist’s locations provide a secondary measure of panel composition. The locations were
marked on an Internet-based map and the link was shared with participants. For both the
researcher and the participant group, the map provided an interactive and public view of
the ‘laboratory’. A map with markers across the globe, while a very basic indication,
was meant to complement the aforementioned diversity measure.
Phase 7 – initiate Delphi rounds: Once the participants were selected and the desired
balance was achieved, the participants were sent a formal email invitation to participate. A
link to the round one Internet-based survey was provided in the letter. The Internet
comprised the primary communication medium in the preparatory stage. The Delphi invita-
tions were administered by email, and the round one survey was administered using an
Internet-based survey software provider. The advantage of using the Internet is that it is
a convenient, economical, and quick method for transmitting communications and
collecting data (Denzin, 2004; Donohoe, 2008). It is an increasingly attractive option for
Delphi administrators who struggle to mitigate the traditionally long waits between
Delphi iterations when surface mail is the primary communication method (Day &
Bobeva, 2005; Donohoe, 2009). While surface mail was offered as an alternative to
participants who do not have the Internet access or who prefer the traditional paper-and-
pen approach to surveys, this offer was not accepted by any of the Delphi panelists.
Therefore, a critical design decision was made to continue Internet-based communications
and e-surveys through subsequent rounds.
As the expert panel construction is considered the most important Delphi design feature,
this very detailed and strategic approach was considered methodologically imperative.
The care to detail and process afforded in this stage, as guided by the GDT, produced a
Delphi panel consisting of 157 ecotourism experts (47 professional, 22 academic, 25
both academic and professional) from over 40 countries (Figure 2).
It is important to note that the time required to complete the preparation stage varies
considerably across Delphi studies. Deﬁning the research problem and purpose can take
Figure 2. Delphi panel composition.
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

months of preparation, review, and preliminary investigation. The recruitment process can
also vary depending on the challenges that may present when identifying, approaching, and
inviting potential participants to join the panel. A standard time frame for the preparation
stage is therefore not defined by the GDT or elsewhere in the literature. In this case, the
total time for the preparation stage was 1 year with the majority of this period dedicated
to defining the research problem and process. Once the panel recruitment process was
defined, it took approximately 2 months to complete the seven-phase process described
here. This would have been much longer if traditional communications methods were
used; the use of the Internet expedited communications and recruitment. A Delphi study
requires more time and effort than many of the methodological alternatives available to
the ecotourism researcher. Therefore, it is recommended that the research architect
budget an appropriate amount of time for the design decisions and activities required
during the Delphi study preparation stage and that the Internet be considered as a communi-
cation and time management tool.
The convergence stage
The GDT recognises that preparatory effort is necessary before initiating the convergence
stage, but it does not distinguish this work as a separate stage (Day & Bobeva, 2005).
Instead, the GDT suggests that the initial base of knowledge be ‘generated’ through the
first round of the study and that it be complimented with a synthesis of the key issues ident-
ified in the literature. Therefore, the first round was a scoping exercise designed to get
participants reflecting about the relationship between cultural sensitivity and ecotourism.
Round one asked: What are the three most important issues surrounding contemporary
ecotourism?; How important is it for ecotourism to be explicitly sensitive to culture and
cultural differences around the globe?; How appropriate is the working definition of cultural
sensitivity for ecotourism (definition provided)?; and What cultural variables or attributes
need to find expression in the definition? Participants were provided with a one-month
period in which to respond. A response rate of 60% was achieved (Table 2) and the
results revealed that a majority agree (81%) that cultural sensitivity is a very important eco-
tourism issue that is highly relevant to the contemporary context but neglected in theory and
praxis. The working definition for culturally sensitive ecotourism provided to the panelists
was purposefully broad and loosely structured so as to allow the group to inform the shape
and content of the definition over the course of the Delphi exercise. As expected, the first
definition resulted in a flurry of valuable comments, suggestions, and critiques that were
used to enhance the definition and inform the subsequent Delphi rounds.
The GDT suggests that subsequent rounds may be considered ‘evaluation and exten-
sion’ of the results produced in the first ‘scoping’ round. Therefore, the focus of the remain-
der of the study was on developing a group consensus definition for culturally sensitive
ecotourism. This included two survey rounds that successfully captured hundreds of com-
ments and suggestions from the Delphi panel. In each round, an enhanced definition was
introduced and feedback was collected regarding its appropriateness. In order to assess
the level of convergence between rounds, simple mean scores and standard deviation
were calculated (Table 2). This approach is substantiated by the GDT as well as throughout
the Delphi literature and in particular, the Delphi research within the tourism domain (Green
et al., 1990; Moëller & Shafer, 1994). As the definition was enhanced, the research revealed
a convergence of group opinion and an increase in convergence between rounds. Between
rounds two and three, the definitional enhancements were minimal, no significant change in
the mean scores was observed, but the increase in convergence (reduction in standard
12 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

deviation value) was noteworthy. Thus, it was decided that consensus had been reached and
that further rounds would not produce additional convergence of opinion.
It is important to note that the design of the data collection instrument evolved between
rounds. The GDT suggests that ‘creativity’ should guide any data collection mechanism and
that different ways of conceptualising the issue and structuring questions must be tried so as
to match the expertise of the panel and their preferred communication mode. The GDT
provides standard guidance on structuring the survey questions in accordance with well-
proven survey design practices. Moreover, the GDT suggests that the key is formulating
clear, concise, and unambiguous questions, together with clear instructions for the panelists.
It is also suggested that the researcher assume a commonsense approach to survey design
where care is afforded an aesthetically pleasing and a standard and technically easy format
so as to facilitate participation and mitigate attrition. For example, they suggest including
questions and scales that are easily identifiable by the participant and where the participant
does not require additional training in order to complete the survey. As recommended by the
GDT, this study employed a combination of open and closed-ended questions and the
majority of consensus building activities and measures were based on a five-point
Likert-type scale where panelists judged the appropriateness of the definition from ‘five’
for very appropriate to ‘one’ not appropriate (Table 2).
It is also important to note that the time required to complete the convergence stage
varies across Delphi studies. Contributing factors include the time between rounds, the
number of survey rounds required to achieve consensus, administration challenges, and
attrition. While the literature contains reports of Delphi’s that required several months to
several years (Brown, 2007; Hanafin et al., 2007; Hurd & McLean, 2004; Vaugeois
et al., 2005), 1 year appears to be the average length of time required for Delphi adminis-
tration (Landeta, 2006). In this case, the survey rounds were administered between January
and July 2008. The total time required to complete the convergence stage was 6 months.
The consensus stage
The convergence stage produced a definition that is considered the most appropriate to an
international panel of ecotourism experts. The process assures that the definition has under-
gone testing and that a consensus judgement has been rendered regarding its value and
appropriateness. Before declaring ‘consensus’, a draft final report was circulated to the
Delphi panelists with a request for final comments. A few panelists submitted final rec-
ommendations and these were integrated into the final consensus definition (Figure 3). A
final Delphi report was then prepared and disseminated to the panelists, supporting organ-
isations, and other interested stakeholders. The final result is a definition for culturally
sensitive ecotourism.
Discussion: critical issues for Delphi administration
The Delphi is a legitimate research technique that is proving useful in a growing range of
disciplines and fields. In the ecotourism case, it has been shown that the Delphi is an effec-
tive technique for the generation of opinion from a geographically dispersed group of
ecotourism experts (Garrod & Fyall, 2005; Miller, 2001). The GDT introduced by Day
and Bobeva (2005) can be a valuable source of guidance for Delphi architects in all
stages of ecotourism research administration. In this case, the GDT has provided a useful
instrument for moving through each of the Delphi stages and the researcher was able to
carefully consider and operationalise the utility, structure, and substance of a Delphi for
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

the study of cultural sensitivity and ecotourism. In doing so, the exercise revealed a number
of GDT benefits. First, the GDT provides a comprehensive summary of the critical issues
that must be considered when designing a Delphi study. The ‘checklist’ format of the GDT
facilitates critical reflection on the research problem, the choice of methodology, and the
associated research design and administration decisions. It offers a range of options as
well as a set of recommendations which have proven to be particularly useful for the
first-time Delphi architect. Second, the GDT describes a set of ‘optional’ iterations
where researchers can validate the research purpose (scoping round) and the research pro-
ducts (draft final report circulation). These options provide researchers with an additional
set of tools that can ensure that the methodology is a ‘best-fit’ for the research, that the
method will produce meaningful data and by extension, and that the results of the
Delphi study can be substantiated. As such, the GDT provides researchers with a reference
standard for designing, adapting, and implementing a Delphi study.
In addition to the panel design, instrument design, communication method, time
requirements, and convergence measures, there are other design and implementation
issues that have been revealed by the implementation of the Delphi study. Four issues
were selected for discussion here as they are judged to be highly relevant to the ecotourism
research context and for the methodogical refinement process.
Issue 1: analysis of the Delphi results
The GDT warns about the potential data analysis problems that can arise. However, the dis-
cussion is limited to a specific data collection instrument and little guidance is provided for
those using a breadth of instruments and techniques (as is recommended by the GDT). This
study revealed the complexity of processing a large, rich, and varied data set comprising
numerical and non-numerical data. At this time, there are few tools available for processing
data of this kind and this presents the Delphi administrator with additional time constraints
Figure 3. Definitional enhancement process.
14 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

(Day & Bobeva, 2005). As it is recommended that the time between rounds be kept to a
minimum – less than 2 months, the researcher maintained strict deadlines so as to avoid
attrition between rounds, to avoid ‘survey fatigue’, and to mitigate temporal delay.
However, the data and recommended timelines presented the researcher with considerable
challenges in maintaining the research schedule and delivering the analysis reports to the
panelist with the subsequent round. Although deadline adherence served to mitigate attri-
tion and to expedite the Delphi process, additional time or flexibility could have allowed
for greater data analysis creativity and the identification of multiple perspectives. In this
regard, the GDT provided limited guidance on time management.
Issue 2: timing of the Delphi surveys
The GDT recommends that the surveys be dispatched so as to capture the maximum avail-
ability of the panelists and to achieve an acceptable response rate. This approach has been
substantiated in the literature, and the work of Briedenhann and Butts (2006) confirmed the
importance of knowing the situational context so as to avoid distractions and attrition. Their
study of rural tourism in Africa coincided with the World Forum on Sustainable Develop-
ment in Johannesburg and this caused a decline in participation as panelists abandoned the
Delphi to attend the event. For this study, care was afforded the timing of the surveys so that
they did not coincide with the active ecotourism season for professionals and the active
research or vacation seasons for academics. The surveys were initiated in January and a
high response rate was achieved and maintained throughout. Although attrition did
occur, the response rate improved and attrition rates decreased as the rounds progressed.
This may be evidence of the panelists level of commitment to the study and/or the cultural
sensitivity/ecotourism issue. It may also be the result of the efforts to inform the panelists of
the Delphi process, goals, and timelines at the outset so as to mitigate confusion and disil-
lusionment. It is therefore recommended that researchers become informed of the context
for administration and that the Delphi be timed to avoid known distractions so as to
achieve a desirable response rate. It is also recommended that panel include individuals
who are interested in the research topic and that they are well informed so as to prevent frus-
tration with the process and loss of interest over the course of the exercise.
Issue 3: communications
The GDT recommends that the researcher maintain a high level of communication for a
number of reasons: it supports participant commitment, it enables an adequate level of
response, and it mitigates attrition. For this study, significant time and effort were dedicated
to communications. Through all research stages, the Internet served as the primary com-
munication medium. Potential Delphi participants were identified through Internet
resources (online journals and databases), and all communications were conducted
through email. This made communications with a global network of experts possible,
rapid, and efficient while recruitment costs were reduced as postage and travel were not
required. A website was also created to serve as the central research communication
portal. The interactive online map and the survey summary reports were posted and
shared through this page. Participants frequented the webpage and they reported that
added value as a very useful communication portal for the study. The Delphi surveys
were administered through a ‘user-friendly’ online survey provider. Postage costs for
survey delivery and return were eliminated and data collection and analysis costs were
minimal. While the time required for a Delphi study is substantial, the GDT suggests
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

that the time required can be decreased through the use of e-surveys and internet-based
tools. Although this Delphi study was completed in a 6-month period, the reliance on
Internet-based communications revealed several challenges – both expected and unex-
pected. Technological challenges presented the most significant barrier to participant
recruitment and data collection. Select participants reported difficulties such as an inability
to open survey links, to download the survey, or to submit their survey responses. In most
cases, difficulty was the result of user error and administrator assistance served to resolve
the issue. In other cases, computer hardware such as operating systems, Internet connec-
tions, and technical failures were the source for difficulty. For some, Internet access was
not available (infrastructure limited in remote locations and developing countries); it was
interrupted by electricity failures or unreliable connections (or in one case, cables
damaged by terrorists). Technological failures at the research centre also caused delay
when the server collapsed and the network remained inaccessible for 7 days. Finally,
some email addresses were inactive and this resulted in a significant amount of returned
emails. These challenges are not unique to the ecotourism case, but given the nature of
ecotourism operations and research, it is often the case that experts are located in areas
where Internet communications are not always possible. To maximise communications
and time savings as well as to mitigate technological challenges, it is recommended that
the researcher offer a pen-and-paper survey alternative, invest in developing clear directions
for survey completion, and pilot test online surveys and communications and that they
make themselves or an assistant available for technical support throughout the duration
of an Internet-based Delphi study.
Issue 4: panel management
The GDT identifies panel stability as a significant challenge for Delphi administrators.
Panel stability is understood to represent minimal attrition rates, preferred panel size man-
agement, and preferred panel composition or balance (Garrod & Fyall, 2005). A lack of
panel stability presents a considerable barrier to maintaining the expert group and achieving
the Delphi’s objectives (Garrod, 2003). In this case, the researcher consulted the Delphi lit-
erature for a more comprehensive set of recommendations. The addition of new members,
in order to replace withdrawals, is not a recommended mitigation measure. Murray (1979,
p. 155) suggested that it damages the ‘very core of the Delphi procedure’ and ‘the results
that emerge must be suspect’. Instead, it is recommended that Delphi architects develop an
initial expert sample list that reflects the predetermined expert selection criteria, establish a
minimum requisite group size and/or a preferred group size, and develop a panel stability
management plan accordingly (Choi & Sirakaya, 2006; Gordon, 1994). Garrod and Fyall
(2005) recommended that panel stability be assessed periodically throughout the process
with a quality control measure based on predetermined panel composition preferences
and criteria. Based on their experiences, Pan, Vega, Vella, Archer, and Parlett (1995,
p. 32) advocated that the ‘sample size should be as large as possible to allow for subsequent
drop-outs, yet small enough to ensure the respondents are all experts in their fields’. In the
worst case, Garrod (2003) recommended Delphi termination if the desired panel compo-
sition and size are compromised by attrition between rounds. Given the nature of the
research, the importance of the panel composition, and the established panel size
minimum, the latter is particularly pertinent. These recommendations complemented the
guidance provided by the GDT and were therefore integrated into the Delphi study
described here. For ecotourism researchers, a similar path would serve to avoid confusion
and study compromise.
16 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

Conclusion
The review of the design and implementation of this Delphi study has illuminated the
potential utility of the GDT for ecotourism research. It makes possible the sharing of
values across a broad ecotourism stakeholder spectrum, the convergence of group
opinion on contemporary and relevant ecotourism issues, and the delivery of outcomes
that may have important implications for ecotourism management. Furthermore, bringing
together ecotourism experts is a particularly arduous task given the nature of the environ-
ments in which ecotourism activities occur. Experts are spread across the globe and are
often located in remote areas. Bringing these individuals together to discuss pertinent
issues presents logistical and financial challenges. The Delphi offers a constructed space
for examining ecotourism free from the aforementioned challenges. While the examination
of the GDT has illuminated other potential challenges associated with Delphi design and
implementation, the researcher is optimistic that the guidance provided by the GDT and
complimented by a literature review, that the level of care afforded the research design,
and that study implementation controls will mitigate potential challenges and maximise
potential opportunities. The GDT discussed in this paper has been proposed by Day and
Bobeva (2005) as a Delphi guide. Its utility as a guide has been assessed and it has been
found to be practical and user-friendly. Furthermore, its utility for research design may
be enhanced, its limitations can be mitigated, and its potential challenges can be addressed
when additional resources are consulted. For these reasons, it is suggested that ecotourism
researchers consult the GDT for guidance when considering, designing, or conducting a
Delphi study.
As the ecotourism research domain moves through its ‘coming of age’ (Weaver &
Lawton, 2007, p. 1168), it is important to consider not only emergent themes and knowl-
edge voids, but also its methodological evolution. In Backman and Morais’s (2001)
review of the methodological approaches used in the ecotourism literature, they conclude
that the current methodological state of the domain is relatively immature and narrowly
focused. The Delphi technique, unlike the commonly applied interview, participant obser-
vation, or content analysis methods, is unique because it has not been the focus of signifi-
cant attention or critical review in the ecotourism literature. In fact, Garrod (2003) reported
that the ecotourism literature is very lean in this regard. This paper represents an attempt to
address this void and to contribute to ecotourism’s methodological ‘coming of age’. It is
hoped that it will ignite interest in the Delphi method, critical discourse related to its suit-
ability for ecotourism research and that it will inspire other researchers to benefit from its
advantages, attributes, and applications.
Acknowledgements
The author wishes to express gratitude to Delphi participants for their expertise and contributions and
to TIES and the ECOCLUB for their support.
References
Alberts, D.J. (2007). Stakeholders or subject matter experts, who should be consulted? Energy Policy,
35, 2336–2236.
Archer, B. (1980). Forecasting demand: Quantitative and intuitive techniques. Tourism Management,
1(1), 5–12.
Austin, D.R., Lee, Y., & Getz, D.A. (2008). A Delphi study of trends in special and inclusive
recreation. Leisure/Loisir, 32(1), 163–182.
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

Backman, K.F., & Morais, D.B. (2001). Methodological approaches used in the literature. In D.B.
Weaver (Ed.), The encyclopedia of ecotourism (pp. 597–610). Wallingford, Oxon: CABI Publishing.
Braden, K., & Prudnikova, N. (2008). The challenge of ecotourism development in the Altay region of
Russia. Tourism Geographies, 10(1), 1–21.
Briedenhann, J., & Butts, S. (2006). The application of the Delphi technique to rural tourism project
evaluation. Current Issues in Tourism, 9(2), 171–190.
Brown, C.A. (2007). The opt-in/opt-out feature in a multi-stage Delphi method study. International
Journal of Social Research Methodology, 10(2), 135–144.
Cantrill, J.A., Sibbald, B., & Buetow, S. (1996). The Delphi and nominal group techniques in health
services research. International Journal of Pharmacy Practice, 4(2), 67–74.
Carrier, J.G., & Macleod, D.V.L. (2005). Bursting the bubble: The socio-cultural context of ecotour-
ism. Journal of the Royal Anthropological Institute, 11, 315–334.
Cater, E. (2006). Ecotourism as a Western construct. Journal of Ecotourism, 5(1&2), 23–39.
Chevron, J.R. (1998). The Delphi process: A strategic branding methodology. Journal of Consumer
Marketing, 15(3), 254–264.
Choi, H.C., & Sirakaya, E. (2006). Sustainability indicators for managing community tourism.
Tourism Management, 27, 1274–1289.
Day, J., & Bobeva, M. (2005). A generic toolkit for the successful management of Delphi studies. The
Electronic Journal of Business Research Methodology, 3(2), 103–116.
Delbecq, A.L., Van de Ven, A.H., & Gustafson, D.H. (1975). Group techniques for program plan-
ning: A guide to nominal and Delphi processes. Glenview, IL: Scott, Foresman and Co.
de la Barre, S. (2005). Not ‘ecotourism’?: Wilderness tourism in Canada’s Yukon territory. Journal of
Ecotourism, 4(2), 92–107.
de Meyrick, J. (2003). The Delphi method and health research. Health Education, 103(1), 7–16.
Denzin, N.K. (2004). Prologue: Online environments and interpretive social research. In M.D. Johns,
S.-L.S. Chen, & G.J. Hall (Eds.), Online social research: Methods, issues, and ethics (pp. 1–12).
New York: Peter Lang.
Denzin, N.K., & Lincoln, Y.S. (2005). Paradigms and perspectives in contention. In N.K. Denzin &
Y.S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 183–190). Thousand
Oaks, CA: Sage Publications, Inc.
Donohoe, H. (2009). Internet-based Delphi administration: The evolution of methodological best-
practices for tourism research. Travel and Tourism Research Association 40th Annual
Conference Proceedings, Honolulu, HI.
Donohoe, H.M. (2008). Internet-based leisure research: Opportunities and constraints (Pesquisa No
Lazer Associados à Internet: Oportunidades e Entraves). Motriz. Revista de Educação Fı́sica,
UNESP, 14(1), 1–8.
Donohoe, H.M., & Needham, R.D. (2009). Moving best practice forward: Delphi characteristics,
advantages, potential problems, and solutions. International Journal of Tourism Research,
11(5), 415–437.
Duboff, R.S. (2007, September 28). The wisdom of (expert) crowds. Harvard Business Review.
Retrieved January 3, 2008, from http://hbr.org/2007/09/the-wisdom-of-expert-crowds/ar/1
Garrod, B. (2003). Defining marine ecotourism: A Delphi study. In B. Garrod & J.C. Wilson (Eds.),
Marine ecotourism: Issues and experiences (pp. 17–36). Clevedon: Channel View Publications.
Garrod, B., & Fyall, A. (2005). Revisiting Delphi: The Delphi technique in tourism research. In B.W.
Ritchie, P. Burns, & C. Palmer (Eds.), Tourism research methods: Integrating theory and practice
(pp. 85–98). Wallingford: CAB International.
Gokhale, A.A. (2001). Environmental initiative prioritization with a Delphi approach: A case study.
Environmental Management, 28(2), 187–193.
Gordon, T.J. (1994). The Delphi method. AC/UNU millennium project: Futures research method-
ology. Retrieved November 23, 2007, from http://www.futurovenezuela.org/_curso/5-delphi.pdf.
Green, H., Hunter, C., & Moore, B. (1990). The application of the Delphi technique in tourism. Annals
of Tourism Research, 17, 270–279.
Habermas, J. (1990). Discourse ethics on a program of philosophical justification. In C. Lenhardt &
S.W. Nicholsen (Eds.), Moral consciousness and communicative action. Cambridge, MA: MIT
Press.
Hanafin, S., Brooks, A.-M., Carroll, E., Fitzgerald, E., Gabhainn, S.N., & Sixsmith, J. (2007).
Achieving consensus in developing a national set of child well-being indicators. Social
Indicators Research, 80, 79–104.
18 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

Hayes, T. (2007). Delphi study of the future of marketing of higher education. Journal of Business
Research, 60, 927–931.
Holey, E.A., Feeley, J.L., Dixon, J., & Whittaker, V.J. (2007). An exploration of the use of simple
statistics to measure consensus and stability in Delphi studies. BMC Medical Research
Methodology, 7, 1–6.
Honey, M. (1999). Ecotourism and sustainable development: Who owns paradise? Washington, DC:
Island Press.
Humberstone, B. (2004). Standpoint research: Multiple versions of reality in tourism theorising and
research. In J. Phillimore & L. Goodson (Eds.), Qualitative research in tourism: Ontologies,
epistemologies and methodologies (pp. 119–136). London: Routledge.
Hurd, A., & McLean, D. (2004). An analysis of the perceived competencies of CEOs in public park
and recreation agencies. Managing Leisure, 9, 96–110.
Jamal, T., Borges, M., & Stronza, A. (2006). The institutionalisation of ecotourism: certiﬁcation,
cultural equity and praxis. Journal of Ecotourism, 5(3), 145–175.
Kaynak, E., & Macauley, J.A. (1984). The Delphi technique in the measurement of tourism market
potential: The case of Nova Scotia. Tourism Management, 5(2), 87–101.
Kaynak, E., & Marandu, E.E. (2006). Tourism market potential analysis in Botswana: A Delphi study.
Journal of Travel Research, 45, 227–237.
Kincheloe, J.L., & McLaren, P. (2005). Rethinking critical theory and qualitative research. In N.K.
Denzin & Y.S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed.,
pp. 303–342). Thousand Oaks, CA: Sage Publications, Inc.
Kuo, N.-W., & Chiu, T. (2006). The assessment of agritourism policy based on SEA combination with
HIA. Land Use Policy, 23, 560–570.
Landeta, J. (2006). Current validity of the Delphi method in social sciences. Technological
Forecasting and Social Change, 73, 467–482.
Linstone, H.A. (1978). The Delphi technique. In J. Fowles (Ed.), Handbook of futures research.
London: Greenwood Place.
Linstone, H.A., & Turoff, M. (1975). The Delphi method: Techniques and applications. London:
Addison-Wesley Publishing Company, Inc.
Linstone, H.A., & Turoff, M. (2002). General applications: Introduction. In M. Turoff & H.A.
Linstone (Eds.), The Delphi approach: Techniques and applications (pp. 71–79). Reading,
MA: Addison-Wesley Publishing Company Inc. Retrieved November 11, 2007, from http://
www.is.njit.edu/pubs/delphibook.
Lloyd, J., La Lopa, J.M., & Braunlich, C.G. (2000). Predicting changes in Hong Kong’s hotel industry
given the change in sovereignty from Britain to China in 1997. Journal of Travel Research, 38,
405–410.
McCool, S.F., & Moisey, R.N. (Eds.). (2001). Tourism, recreation and sustainability: Linking culture
and the environment. Wallingford: CABI Publishing.
Miller, G. (2001). The development of indicators for sustainable tourism: Results of a Delphi survey
of tourism researchers. Tourism Management, 22, 351–362.
Moëller, G.H., & Shafer, E.L. (1994). The Delphi technique: A tool for long-range travel and tourism
planning. In J.R.B. Ritchie & C.R. Goeldner (Eds.), Travel, tourism and hospitality research: A
handbook for managers and researchers (2nd ed., pp. 473–480). New York: John Wiley & Sons,
Inc.
Mullen, P.M. (2003). Delphi myths and reality. Journal of Health Organization and Management,
17(1), 37–52.
Murray, T.J. (1979). Delphi methodologies: A review and critique. Urban Systems, 4, 153–158.
Needham, R.D., & de Loë, R. (1990). The policy Delphi: Purpose, structure, and application. The
Canadian Geographer, 34(2), 133–142.
Nyiri, P. (2006). Scenic spots: Chinese tourism, the state, and cultural authority. Seattle, WA:
University of Washington Press.
O’Connor, P., & Frew, A.J. (2004). An evaluation of methodology for hotel electronic channels of
distribution. Hospitality Management, 23, 179–199.
Okoli, C., & Pawlowski, S.D. (2004). The Delphi method as a research tool: An example, design
considerations and applications. Information and Management Systems, 42, 15–29.
Pan, S.Q., Vega, M., Vella, A.J., Archer, B.H., & Parlett, G.R. (1995). Mini-Delphi approach: An
improvement on single round techniques. Progress in Tourism and Hospitality Research, 2(1),
27–39.
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

Phillimore, J., & Goodson, L. (2004). Progress in qualitative research in tourism. In J. Phillimore & L.
Goodson (Eds.), Qualitative research in tourism: Ontologies, epistemologies and methodologies
(pp. 3–29). London: Routledge.
Powell, C. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing, 41(4),
376–382.
Rowe, G., & Wright, G. (1999). The Delphi technique as forecasting tool: Issues and analysis.
International Journal of Forecasting, 15, 353–375.
Sackman, H. (1975). Delphi assessment, expert opinion, forecasting, and group processes. Santa
Monica, CA: Rand Corporation.
Saukko, L. (2005). Methodologies for cultural studies. In N.K. Denzin & Y.S. Lincoln (Eds.), The
Sage handbook of qualitative research (3rd ed., pp. 343–356). Thousand Oaks, CA: Sage
Publications, Inc.
Schmidt, R. (1997). Managing Delphi surveys using non-parametric statistical techniques. Decision
Sciences, 28(3), 763–774.
Seely, R.L., Iglarsh, H., & Edgell, D. (1980). Utilising the Delphi technique at international confer-
ences: A method for forecasting international tourism conditions. Travel Research Journal, 1,
30–35.
Sofeild, T. (2007). Tourism and natural landscapes: Universal principles or culturally determined
values? In H.M. Donohoe & W.J. Li (Eds.), Leisure in context: Implications for human develop-
ment and well-being (pp. 150–159). Hangzhou: Zhejiang University Press.
Spenceley, A. (2005). Nature-based tourism and environmental sustainability in South Africa. Journal
of Sustainable Tourism, 13(2), 136–170.
Stark, J.C. (2002). Ethics and ecotourism: Connections and conﬂicts. Philosophy and Geography,
5(1), 101–113.
Sunstein, C.R. (2006). Infotopia: How many minds produce knowledge. Oxford: Oxford University
Press.
Tsaur, S., Lin, Y., & Lin, J. (2006). Evaluating ecotourism sustainability from the integrated perspec-
tive of resource, community and tourism. Tourism Management, 27, 640–653.
Vaugeois, N., Rollins, R., Hammer, K., Randall, C., Clark, B., & Balmer, K. (2005). Realizing
improved knowledge transfer in the leisure industry: A new agenda and strategies to link research-
ers and professionals, Travel and Tourism Research Association Canada Chapter Conference
Proceedings, Kelowna, BC, Canada.
Vivanco, L. (2002). Escaping from reality. The Ecologist, 32(2), 26–30.
Weaver, D.B., & Lawton, L.J. (2007). Twenty years on: The state of contemporary ecotourism
research. Tourism Management, 28, 1168–1179.
Weber, K., & Ladkin, A. (2003). The convention industry in Australia and the United Kingdom: Key
issues and competitive forces. Journal of Travel Research, 42, 125–132.
Wolfers, J., & Zitzwitz, E. (2004). Prediction markets. Journal of Economic Perspectives, 18(2),
107–126.
Wright, T.S.A. (2006). Giving ‘Teeth’ to an environmental policy: A Delphi study at Dalhousie
University. Journal of Cleaner Production, 14, 761–768.
Yeong, Y.W., Keng, K.A., & Leng, T.L. (1989). A Delphi forecast for the Singapore tourism industry:
Future scenario and marketing implications. European Journal of Marketing, 23(11), 15–26.
20 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012

A Delphi Toolkit For Ecotourism Research

More Related Content

Similar to A Delphi Toolkit For Ecotourism Research

More from Hannah Baker

Recently uploaded

A Delphi Toolkit For Ecotourism Research