• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Using focus groups for evaluating learner experiences by Tunde Varga-Atkins and  Luciane Vieira Mello Rigden, University of Liverpool
 

Using focus groups for evaluating learner experiences by Tunde Varga-Atkins and Luciane Vieira Mello Rigden, University of Liverpool

on

  • 425 views

This presentation was developed for participants on a PgDip programme - the remit was to offer insight into focus groups as a potential method of data collection for their research project.

This presentation was developed for participants on a PgDip programme - the remit was to offer insight into focus groups as a potential method of data collection for their research project.

Statistics

Views

Total Views
425
Views on SlideShare
408
Embed Views
17

Actions

Likes
1
Downloads
7
Comments
0

1 Embed 17

https://twitter.com 17

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Purpose: to talk about focus groups as method of data collection
  • Danger of not being neutral: defensive positions vs real ‘disinterested’ listening.Neutral is not equal to being unprepared or unknowing! Preparation is vital.
  • * May depend on research question and facilitation and group!
  • Nominal Group element!
  • Credibility, trust and maximum participationGroup management:Convergence and conformityAre we looking for consensus?Are participants trying to conform to emerging group norms? Are participants Closure (reporting) and thanks
  • Credibility, trust and maximum participationGroup management:Convergence and conformityAre we looking for consensus?Are participants trying to conform to emerging group norms? Are participants Closure (reporting) and thanks

Using focus groups for evaluating learner experiences by Tunde Varga-Atkins and  Luciane Vieira Mello Rigden, University of Liverpool Using focus groups for evaluating learner experiences by Tunde Varga-Atkins and Luciane Vieira Mello Rigden, University of Liverpool Presentation Transcript

  • Using focus groups for evaluating learner experiences Tünde Varga-Atkins - eLearning Unit, CLL Luciane Vieira de Mello Rigden – School of Life Sciences University of Liverpool 6th June 2013 Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • Outline 1. Why focus group? 2. What is a focus group? 3. Focus groups in context of other methods 4. Viewpoints: focus groups as a – Commissioner (stakeholder) – Observer – Facilitators 5. Focus group analysis and reporting 6. Alternatives / enhancements: – Nominal Group Technique – Visual and any other methods Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • Your experience • Have you had experiences of focus group, either as: – Commissioner? – Facilitator? – Observer? – Participant? What is your research question? Appropriate/Not to explore in a focus group? Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • 1. Why focus groups? Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • Rationale for focus groups “When people gather to talk about something, their contributions and understandings will be enriched by the group dynamic.” (Cousins 2009) “To display and discuss differences in a group. ...” (Lewis 2003) Focus groups are for sharing and comparing … and remembering Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • 2. What is a focus group? Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • Focus group process Neutral facilitator Size: 6-8 (4-12) participants Incentive Sample selection! Ethics: informed consent and reporting Audio recorder & other resources or stimulus material 1-2 hours an open group discussion with a focus Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • What is a focus group? Study skills International student experienceFocus: Question(s): English language Peer support Can you tell me about the support that you have received on campus? A focused but open discussion with a group of 6-8 participants, moderated by a neutral facilitator usually lasting 1-2 hours. The facilitator usually has a topic (or question) guide which may or may not be covered in sequence. Introduction Closure Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • The evaluation/research cycle* 1. Set out purpose 2. Agree what questions to ask 3. Agree on method (focus group) 5. Compiling draft report 6. Draft report sent to students for confirmation 10. Action & feedback to students 7. Production of final report 4. Conduct of focus groups 8. Circulation of report to staff 9. Presentation and discussion of report Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool. *of not just focus groups but any participant consultation!
  • 3. Focus groups in context of other methods Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • Some possible combinations Exploring depth survey survey focus group(s) focus group(s) Generate ideas focus group(s) a b c Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • Methods: from individual to group Surveys Focus groups Interviews Researcher present individual group Researcher not present Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • Methods: from individual to group Surveys Focus groups Interviews Researcher present individual group Researcher not present How many hours do you spend studying?* What makes you study (more)? Can you tell me a bit more about what you do when you study? Idea development aided by others? Participant motivation Bias by others? Shared experience? * BUT: in a focus group setting, participants can have a discussion about what constitutes study, which may make them adjust their own understanding of what study is.Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • Surveys Focus groups Interviews Bias? Useful when you don’t want others to bias your participant Potential of ‘group bias’ – participants say what they think you or other participants want to hear Useful when you don’t want others to bias your participant Shared experience? Participant relies on own experience – may be more difficult to recall. Useful to talk about shared experiences. ‘You bounce off ideas from one another’. Useful when you want to explore individual experiences. Idea development aided by others? Relies on participant able to formulate their ideas without prompts. FGs are useful when the development of ideas and understanding is aided by other group members. FGs are useful when the development of ideas and understanding is aided by other group members. Motivation/Enjoyment ? Non-response and quality of response as a problem. Participants generally enjoy the process.* They learn something new. Participants generally enjoy the process. They learn something new. BUT: feasibility and practicality may be the largest factors (of getting volunteers, organising them together etc.) Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • Sample questions from a UoL curriculum evaluation focus group • 1.What aspects of the X best facilitated your learning? • 2. What aspects of the X inhibited your learning ? • 3. What aspects of the X caused the most stress/anxiety and did this stress/anxiety worsen or lessen as the X progressed? • 4. Optional specific questions. • 5. Can you suggest one (two) different ways in which the X could be improved? Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • 4. Viewpoints Focus groups as a: commissioner observer facilitator Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • As a commissioner of a focus group • Experiences of Dr Mello (Lu): – Asking the right questions • After a questionnaire: realisation of non-properly covered aspects guided question formulation • Questions were discussed with the facilitators (but I would do it more extensively next time….) – Coverage of questions • Some really short answers from students • Group dynamic – What is important? • Have clear what you want • Take the first one as an experience – Expectations • If you have an open mind, you will surprise yourself!
  • As an observer of a focus group • Experiences of Dr Mello (Lu): – What is going on? • Got to know the questions just before meeting the students – Benefits • Better appreciation & understanding of the whole process • Confidence to plan another focus group • How I learned from the students!!! – Challenges • Hard to just observe! • Have a good chat with the facilitator in advance
  • As a facilitator of a focus group • Design • Introduction, scene setting • Questioning techniques • Managing group dynamics • Managing schedule / topics / coverage
  • Slides supporting focus group process (facilitation)
  • The schedule of a Focus Group 1. Introduction = setting scene and establishing trust: – purpose – ground rules (conformity / convergence) – ethics – reporting 1. Introduction 2. The ‘Focus’ = questions 3. Closure Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • 2. The ‘Focus’ = aka questions Low vs high moderation 2 questions 6 questions: exploring international student experience on campus 6 questions (Cousins 2009, p63) Note: Participants do not normally see the questions – the discussion is good if it seems to flow naturally. These will not necessarily be covered in order.
  • Anatomy of a good focus group question with ready probes (Cousins 2009, p63) Preparation and good question (and prompts, probes) design is very important! English language Peer support Probing for range Probing for depth Study skills Can you say something about further support..? Open question
  • 5. Analysis and reporting Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • Analysis Transcript Report Analytical process Various decisions and challenges! • Emphasis – key points • Focus • Agreement • Majority vs single voices Research question dependent
  • 6. Alternative methods Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • It doesn’t have to be just words! Tasks e.g. projection techniques Source: http://bit.ly/JZRz1T Visual techniquesStimulus Source http://bit.ly/Mc0kSu Bringing as many stimulus material related to the topic as possible to help participants recall e.g. bring assignment samples for a discussion on feedback. Asking participants to take photos, draw diagrams or drawings to elicit their experiences e.g. ask them to draw a timeline of their time at university Asking participants to undertake an activity, e.g. using projective techniques or write down their top five priority in the given topic etc. Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • The Nominal Group Technique • Good decision-making technique with no need for transcription! • Brings in a quantitative element through the ranking stages. • More useful for evaluation and less useful for ‘full’ research. – See references for more details. Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • The Nominal Group Technique: an alternative focus group method Surveys Focus groups Interviews individual group Nominal Group Technique The Nominal Group Technique is a useful technique that is structured in a way that although it is a group session, it does focus on individual experiences and opinions rather than group consensus. Participants at various points in the session are asked to give their individual opinions which are ranked at the end.
  • Summary • Focus groups are useful when: – exploring a shared experience. – participants (and the research!) benefits from the development of understanding and ideas in a group environment. – group bias or conformity is not likely/important. • Focus groups: – need to be facilitated by a neutral facilitator. – produce quality of the output (if focused and run well). – can be enriched by various techniques (visuals, stimulus) or tasks. • Analysis involves various decisions on what/how to report. • A useful alternative is the Nominal Group Technique. – Just writing up a combined ‘Nominal Focus Group’ as a useful method. Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • References • Focus groups – Cousin, G. (2009). Focus Group Research. IN: Researching learning in higher education : an introduction to contemporary methods and approaches. London: Routledge, pp.51-69. – Ritchie, J., & Lewis, J. (2003). Qualitative research practice : a guide for social science students and researchers. London; Thousand Oaks, Calif.: Sage Publications. Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.
  • References • Nominal Group Technique – Practical guide: Varga-Atkins, T., with contributions from Bunyan, N; McIsaac, J; Fewtrell J. (2011) The Nominal Group Technique: a practical guide for facilitators. Written for the ELESIG Small Grants Scheme. Liverpool: University of Liverpool. October. Version 1.0. Available at http://slidesha.re/s5KPUr – Project report: Varga-Atkins, T. with contributions from Bunyan, N; McIsaac, J and Fewtrell, R. (2011) Using the nominal group technique with clickers to research student experiences of e-learning. Project Report written for the ELESIG Small Grants Scheme. Liverpool: University of Liverpool. Available from slideshare.net at http://slidesha.re/sc8gwT Creative Commons: Attribution-NonCommercial-Sharealike By Tunde Varga-Atkins, University of Liverpool.