The document discusses the Nominal Group Technique (NGT) and Delphi method, which are consensus methods used for problem-solving, idea generation, and determining priorities. The NGT is a structured face-to-face group process with stages of silent idea generation, round-robin feedback, discussion, and voting. The Delphi method uses questionnaires and feedback to gather anonymous experts' judgments to reach consensus over multiple rounds. Both methods aim to balance participation and minimize individual dominance to achieve group agreement.

1. 1/22
Nominal Group Technique and Delphi method
Presenter: Dr Girish J, M.D.
AIIMS, New Delhi

2. 2/22
Introduction
• The Nominal Group Technique (NGT) and Delphi Technique
are consensus methods used in research that is directed at
problem-solving, idea-generation, or determining priorities
• Aim= to achieve a general agreement or convergence of
opinion around a particular topic
1. McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm [Internet]. 2016 [cited 2020 Dec
27];38:655–62. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4909789/

3. 3/22
• The NGT is a highly structured face-to-face group
interaction, which empowers participants by providing an
opportunity to have their voices heard and opinions
considered by other members
• It was designed by Delbecq and Van de Ven and comprises
four key stages: silent generation, round robin, clarification
and voting (ranking or rating)
Nominal Group Technique
1. McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm [Internet]. 2016 [cited 2020 Dec
27];38:655–62. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4909789/

4. 4/22
FGD vs
Consensus
methods
• Focus groups are useful for investigating
an issue in-depth, including the
identification of problems, questions or
significant issues.
• Consensus methods= raise potential
solutions or answers to a question,
which can then be prioritised or agreed
upon.
• A key strength of consensus methods=
balanced participation from group
members
• In focus group- the facilitator must
control for, and minimise the risk of, a
dominant participant influencing the
discussion.

5. 5/221. McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm [Internet]. 2016 [cited 2020 Dec
27];38:655–62. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4909789/

6. 6/22
Nominal Group Technique
Silent generation of ideas in writing:
• Welcome statement – importance of the task and of each member’s
contribution
• Leader reads out the question – usually open-ended question that calls for
a list of items e.g. the elements of a specified problem
• Each member (including leader) take 5 minutes to write their ideas in
response to the question
• Discussion is not permitted
1. Survey methods: Abramson and Abramson

7. 7/22
Nominal Group Technique
‘Round-robin’ feedback of ideas:
• Round-robin – “taking turns”
• Leader asks each member to contribute one of the ideas they
have written (ideas not duplicated)
• Each idea is numbered and written on a board or flip pad
• Completed sheets of which are taped or pinned where they are
visible to all members
• Members encouraged to add ideas; they may ‘pass’ in one
round and contribute in a later one
• Process until no further ideas are forthcoming
• Discussion is not permitted during this stage
1. Survey methods: Abramson and Abramson

8. 8/22
Nominal Group Technique
Serial discussion of ideas:
• Each of the ideas listed on the board or flip pad is discussed in turn
• Discussion: whether any questions, clarification, needs explanation
of logic behind it, or express a view about its relative importance
• The object is to obtain clarity on points of view, but not to resolve
differences of opinion
• Modify the list if overlap present after the serial discussion
• Modest rewording may be undertaken if the group wishes to refine
the list
1. Survey methods: Abramson and Abramson

9. 9/22
Nominal Group Technique
Preliminary vote:
• Each participant is asked to select a specified number (five to nine)
of ‘most important’ items from the total list and copy them on cards.
• If six are to be chosen, each participant is asked to write ‘6’ on the
‘most important’ card, then ‘1’ on the least important.
• The leader also ranks the items.
• The cards are then collected and shuffled to maintain anonymity,
and the votes are read out and recorded on a tally-chart.
1. Survey methods: Abramson and Abramson

10. 10/22
Nominal Group Technique
Discussion of preliminary vote:
• Brief discussion of the voting pattern is now
permitted.
• Purpose of this discussion is additional clarification,
and not to pressure to change their votes.
Final vote:
• Voting is then repeated
Alternate to ranking: scale from 0 (unimportant) to 10
or 100 (very important)
1. Survey methods: Abramson and Abramson

11. 11/22
Nominal Group Technique
For many small groups:
• Voting done in small groups
• In break, group leaders meet to prepare a master list of items
(aggregated votes), including the top five to nine priorities
identified by each group
• Where necessary, items are reworded or combined
• All the participants then gather in a single large group and discuss
each item in the master list in turn, for clarification
• At any member’s request, items not included in the master list can
be added
• A final vote is then conducted

12. 12/22
Nominal Group Technique
Uses of NGT:
• Involve participants from different background
• Balances the influence of individuals
• Confront issues through constructive problem solving
• Prioritize ideas democratically
Disadvantages of NGT
• Requires preparation
• Lends itself only to a single-purpose and minimizes discussion

13. 13/22
Delphi method
• Delphi Technique are consensus methods used in
research that is directed at
 Problem-solving
 Idea-generation
 Determining priorities
• The Delphi method assumes that group judgments
are more valid than individual judgments
1. Survey methods: Abramson and Abramson

14. 14/22
Delphi method
• The Delphi technique uses a multistage self-completed questionnaire
with individual feedback, to determine consensus from a larger
group of expert’s
• Uses interactions between group (called panel) members via
questionnaires rather than face-to-face communication. This means
that it preserves participant anonymity, if that is relevant
• Delphi technique is commonly used to develop guidelines with health
professionals.
1. Fink-Hafner D, Dagen T, Doušak M, Novak M, Hafner-Fink M. Delphi Method: Strengths and Weaknesses.
2019 Nov 1 [cited 2021 Jan 6];2:1–19.

15. 15/22
First round questionnaire
Present a series of statements that the respondent is asked to rate on a clearly
defined Likert scale.
Respondents are asked both to rate the item and to write free-text comments
Second-round questionnaire
The responses to the first-round questionnaires collated
Reminders are sent to non-responders
Presents the same statements, the individual respondent’s rating and the
median rating from the entire panel. A selection of the free-text responses is
given, to represent the breadth of opinion.
After considering the group median and free-text comments, respondent’s re-rate the statements.
Reaching a consensus
1. Fink-Hafner D, Dagen T, Doušak M, Novak M, Hafner-Fink M. Delphi Method: Strengths and Weaknesses.
2019 Nov 1 [cited 2021 Jan 6];2:1–19.

16. 16/22
Three main variants of Delphi:
(a)“Conventional” Delphi - an open-ended exploratory
phase followed by multiple consensus-seeking rounds
(b)“real-time” Delphi, multiple rounds are temporally
compressed to a single meeting
(c)“Policy” Delphi - forum in which ideas are presented to
decision-makers, who choose a solution

17. 17/22
Example of delphi method
• A Delphi method of consensus development was used,
comprising three survey rounds in the context of obesity
research.
• In Round 1, participants were asked to rate
agreement/disagreement with 77 statements across seven
domains relating to definitions of, and approaches to, using
big data
• Participants were also asked to contribute further ideas in
relation to these topics, which were incorporated as new
statements (n = 8) in Round 2. In Rounds 2 and 3
participants re-appraised their ratings in view of the group
consensus

18. 18/22
Advantages of Delphi
• No proximity or a face-to-face meeting and thus
allows for expert’s independent thought
• There is less opportunity to (be forced to) conform
with the dominant view
• Through the feedback given in Delphi, an individual
expert may enrich his/her in-sight into empirical
factors or theoretical assumptions allowing them to
correct any misconceptions
1. Fink-Hafner D, Dagen T, Doušak M, Novak M, Hafner-Fink M. Delphi Method: Strengths and Weaknesses.
2019 Nov 1 [cited 2021 Jan 6];2:1–19.

19. 19/22
Advantages of Delphi (Contd..)
• Enables group communication free from geographical
constraints
• Saves money and time; it also reduces travel costs
• By straddling the divide between the qualitative and
quantitative methods it is able to provide a more complete
picture
1. Fink-Hafner D, Dagen T, Doušak M, Novak M, Hafner-Fink M. Delphi Method: Strengths and Weaknesses.
2019 Nov 1 [cited 2021 Jan 6];2:1–19.

20. 20/22
Disadvantages
• There is a lack of guidance and agreed standards on how to interpret and
analyse the results.
• Delphi is quite time-consuming work for both researchers and participants, so
vulnerable to drop-outs.
• Participants might also drop out due to the long process.
• Use of monetary payments or moral persuasion to convince the participants
may introduce bias into the results.
• Methodological limitations include the difficulty of generalising the results to
a wider population due to the sample size .
1. Fink-Hafner D, Dagen T, Doušak M, Novak M, Hafner-Fink M. Delphi Method: Strengths and Weaknesses.
2019 Nov 1 [cited 2021 Jan 6];2:1–19.

21. 21/22
• The response rate among the invited experts, limited views, the uneven spread
of expertise among the participants or the participants’ specific agenda and/or
geographic/cultural location.
• As the procedure depends on the quality of the feedback provided, the careful
analysis of the responses is a big responsibility of the re-searcher.
• It is difficult to determine what constitutes a sufficient consensus
• The experts’ responses in the Delphi process might not be truly independent,
especially when the experts involved are in contact with each other
• While in the case of e-Delphi, entering data into computer-based data screens
is an advantage, it might also be an inconvenience for some experts.
Disadvantages
1. Fink-Hafner D, Dagen T, Doušak M, Novak M, Hafner-Fink M. Delphi Method: Strengths and Weaknesses.
2019 Nov 1 [cited 2021 Jan 6];2:1–19.

22. 22/22
Applications of consensus methods
Health
systems
research
• Insufficient data to make decisions
• Prioritise health problems
• Opinions of other health professionals / lay persons /
carers
• Development of quality & safety indicators
Prescribing
research
• Guidelines development
• Enhance clinical decision making
• Include patient / carer perspectives in decisions
Qualitative
research
• Building theories
• Participatory action and community engaged research
Diagnostics • Tools or questionnaire development
Medical
education
• Comparison of new / revised curriculum
• Assessment tools development
• Development of educational resources

23. 23/22
THANK YOU

24. 24/22
History of delphi
• The Delphi Technique was developed by the Rand
Corporation in 1953
• The Delphi method was developed at the beginning of the
Cold War to forecast the impact of technology on warfare.
• In 1944, General Henry H. Arnold ordered the creation of
the report for the U.S. Army Air Corps on the future
technological capabilities that might be used by the
military.

25. 25/22
Consensus methods
Advantage:
• Communication with others – opportunity to reach a modified judgement
after appraising the opinions of others (unlike interviews or questionnaire
surveys)
Disadvantage of group techniques:
• Decisions may be heavily influenced by this interaction, unduly affected by:
• Chairperson’s bossiness or ineffectiveness,
• Dominance by forceful speakers
• Deference to authority, power, prestige or age, or friendships or
antagonisms between participants