Presentation given by Tunde Varga-Atkins and Jaye McIsaac at the University of Manchester, ALT-C 2012, Sep 11, 2012.

1. ID 164. Combatting survey fatigue:
uncovering the student learning
experience using the
Nominal Group Technique
Tunde Varga-Atkins, eLearning Unit
Jaye McIsaac, Educational Development
University of Liverpool
11-13 Sep 2012, ALT-C
Funded by ELESIG

2. Outcomes
•To consider the use of Nominal
Group Technique as a student
evaluation strategy
•To question the use (and concept)
of technology in relation to an
evaluation process

3. HE context: surveys

4. One solution: face-to-face Nominal
Group Technique sessions

5. What is the
Nominal Group Technique?
• Structured group activity
• One given topic
• Facilitates group
decision-making
• Immediate results
• Reducing researcher/participant bias
- Delbecq & Van de Ven (1971)

6. Nominal Group Technique Stages
Example question: ‘What to keep doing (feedback)?’
1) Individual responses 2) Clarification and 3) Ranking responses
consolidation
‘Nominal’ Group = only in name group, most activities are individual

7. What to keep doing? (feedback)
Stage 1: individual responses
1 2 3
Annotated Model Audio
,personalised answers feedback
feedback. giving
extensive
comments
4
Giving model 5
Audio 6
answers for feedback Drafts that
essay titles for must be
everything! submitted and
marked prior
final work

8. What to keep doing? (feedback)
Stage2 : clarification and consolidation
Model answers
4
2
Giving model
Model
answers for
answers
essay titles
Audio feedback
Personalised feedbac
3 Audio 6
5 Drafts that
feedback Audio must be 1
giving feedback submitted and Annotated
extensive for marked prior ,personalised
comments final work feedback.
everything!

9. What to keep doing? (feedback)
Stage3 : ranking responses
Item no. Item description
5 points 1 Annotated, personalised feedback
4 points 2 Giving model answers
3 points 4 Audio feedback
2 points 7 etc.
1 point 10 etc.

10. The top five qualities of good
feedback
1 Timing,
quick
Detailed,
on how to
2
improve
Applicable
to future
3 work
Personal
4 Consistent
bw
5 markers

11. Conclusion: The Nominal Group
Technique
is a useful student evaluation
technique

12. … (minor) problems of efficiency
Credit by Nathan T. Baker

13. Can technology improve
(time & resource) efficiency
of the NGT process?
Presumption!

14. No single technology
could replicate
all the 3 stages of
NGT.
Technologies we tried: ID275, Thu 9am

15. and…

16. Our assumption:
Technology can
improve (time)
efficiency of the NGT
process.

17. revisited…
learning +
(evaluation)
technology = efficiency ?
Problem of lack of theorising technology (Oliver 2012)

18. Paper Digital
• ubiquitous • text entry
• manipulabl • legibility
e • visibility
• overlay • scoring
• multi-items • manipulable
• legible • multiitems
• visible
(Luff et al 2007)

19. Conclusion: can technology enhance
efficiency of the process?
• ubiquitous
• manipulable
• overlay
• multi-items
• text entry
• legibility
• visibility
Image by Paul Duvall

20. Finally…thanks to
Contact:
–Tünde Varga-Atkins, tva@liv.ac.uk
–Jaye McIsaac, jayem@liv.ac.uk

21. References
• Delbecq, A., Van de Ven, Andrew, & Gustafson, D.
(1975). Group techniques for program planning a :
guide to nominal group and Delphi processes.
Glenview Ill.: Scott Foresman.
• Luff et al (2007) Augmented Paper: Developing
relationships between digital content and paper
• Oliver, M. (2012) Learning with technology as a
coordinated sociomaterial practice: digital literacies as
a site of praxiological study
• Varga-Atkins, McIsaac et al (2011) Using the nominal
group technique with clickers to research student
experiences of e-learning: a project report
[http://slidesha.re/xQlBCg ]
• Varga-Atkins, McIsaac et al (2011) The Nominal
Group Technique – a practical guide for facilitators
[http://slidesha.re/AmYOgv]

22. Guide to NGT & Project report
See References
On slideshare.net , search for Nominal Group Technique