Faculty & Student Perceptions of Asynchronous Audio FeedbackDenise Nelson
PlanBackgroundAudio feedback processtechnicalcommenting strategiesResearch process & findings faculty trainingstudents/faculty perspectivesRecommendationsFuture Interest2
Who has experience with audio commenting of student coursework?3
BackgroundFramed by the Community of Inquiry theoretical frameworkDraws upon the work of Dr. Phil Ice on audio feedback, differences being population – mostly undergradslevel of cognition not analyzedfaculty perspectivessituated in SIAST - context of 3 online programsPractical Nursing ProgramPerioperative Nursing ProgramFaculty Certificate ProgramProject sponsored by Campus Saskatchewan and SIASTResearch conducted Fall 2009 – June 20104
5Garrison, D. R., Anderson, T., & Archer, W. (2000).
Community of Inquiry6Social presence is “the ability of participants to identify with the community (e.g., course of study), communicate purposefully in a trusting environment, and develop inter-personal relationships by way of projecting their individual personalities.” (Garrison, 2009) Teaching Presence  is the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes (Anderson, Rourke, Garrison & Archer,  2001). Cognitive Presence is the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse (Garrison,  Anderson &  Archer,  2001).
Focus on FeedbackStudents can cope without face-to-face teaching, but they cannot cope without regular feedback on assignments (Gibbs & Simpson, 2004).7
Audio Feedback TechniqueConvert student document to pdf.Configure Adobe Acrobat Professional with comment/mark-up tools.Use Adobe Acrobat Pro, microphone/headset and Microsoft Sound Recorder to embed comments. Listen to recording to ensure clarity.Inform students to double-clickProvide upfront note to students to contact faculty if difficulty in accessing audio.8
Technical Process9
Additional Hardware/SoftwareAdobe Acrobat ProHeadset/MicrophoneUSB, noise-canceling10
Read the entire student assignment.Decide which feedback audio/text mark-up.Give feedback like face-to-face.Use audio to elaborate details, summarize, give examples/references.Integrate and situate audio and text comments.Do not repeat the written word in audio.Situate mark in audio comment.11Commenting Strategies
…Commenting StrategiesUse text mark-up for specific grammar, punctuation etc.  Use mark-up tools - customized stamps, highlighting, callout, arrows...Can insert attachments.Include rubrics.Other uses - summarizing/weaving discussions, exam review12
Mark-Up Example13
Research14
Literature Review15
QuestionsIn what manner is perceived learning impacted by the use of audio feedback?16Do students learning in asynchronous learning environments believe audio or text-based student feedback is a more effective means of interaction with their faculty?
How does the use of audio feedback impact the     students’ sense of community in asynchronous learning networks?
What relationship exists between the use of audio feedback and student /faculty?MethodNested mixed methodEnd of course web surveyFocus groupFollow-up emailed interview questions17
SampleConvenienceNursing programs & Faculty Certificate Program (FCP) students and facultyParticipants:8 faculty respondents (all)14 student respondents – 8 FCP, 6 Practical Nursing Program18
Faculty Training19
Data CollectionProgram areas randomly selected assignments to include written feedback only or audio and written feedback.Nature of assignments ranged from paragraph responses to major papers.~ half of assignments used written feedback and half used written and audio feedback.Completion of course20
Findings21
Audio  vs. Text --Which is More  Effective?Qualitative – both students & faculty perceived that audio in conjunction with written is most effectiveStudents and faculty comfortable with written and reluctant to negate its valueStudents perceived audio more effective rePersonalizationRetention MotivationUnderstanding instructor’s intent  Feelings of involvement and instructor caring22
Student Perceptions of Auditory Compared to Written23
Student Voices24“Certainly tone of voice and inflection help to distinguish intended meaning more accurately than written text.”“I enjoyed the enhanced feedback (audio), and I think with it an instructor can be more detailed with his analysis. If [instructor] had to write out all his advice for my first paper it may have been longer than the paper itself.”“I was thoroughly impressed and motivated by the format (audio and written) selected…”
Student Follow-Up Email Responses Re useful feedback25“Audio comments could include references to other resources, good examples that could replace errors, and specific pieces of work that were good and why.”“the most important thing in my mind is to give something that isn’t as easy to write down—for example something a bit more lengthy you might want to say or a longer explanation etc. The things that go into the marking that you can’t just jot down really quickly.”
Audio Feedback Impact on Sense of Community?ConnectednessPersonalizationAuthenticityEnhances social presence26
Student Voices27“It was nice to have the audio of the instructors, it made me feel more personal and that you were more of a class.” “I thought it was easy to use and make you feel more connected to the instructor, even in a distance course.”“I enjoyed the auditory feedback because I felt more like the instructor was speaking directly to me, although I appreciated the written comments equally.”
Audio Feedback & Student Satisfaction?Based on open comments of web survey, 9 of the 11 students seemed satisfied. “did not always work”“annoying, a true conversation or written comments would be better”“It seemed the instructor was trying to sound neutral in the audio feedback which left a feeling of apathy.”voice inflection and nuance to grasp more meaningmore and elaborate feedback DetailsExamplesReferences28
Perceived Learning Impacted by Audio Feedback?Faculty and students perceive audio enhances learning.3/4 students who responded to email interview questions revealed that audio feedback supports learning when it is personable, clear and detailed as “opposed to a check mark.”29“I found that I retained the verbal feedback better than just reading about it.”“Sometimes it (audio feedback) was very informative. Hearing their voices made it stick more into my head. I took it more seriously for if you wanted to know why something was the way it was marked, you had to listen.”
Audio Feedback & Faculty Satisfaction?Perceive benefits of social, cognitive and teaching presence  Added value - more quantity and quality of feedback, even though it requires more faculty time to plan and createIncorporating a variety of media helps to project teaching presence and support student’s varied learning preferences.Liked informalityPreferred use for lengthy and complex assignments30
Faculty VoicesVoiceThread conversation31
General Research FindingsNo clear preference for written or audioMore time required to listen and/or provide audio commentsStudent populations differQuality of audio dependent on various factors: audio settings, equipment, noise32
More Student Voices33“Certainly tone of voice and inflection help to distinguish intended meaning more accurately than written text.”“This is my first experience with audio feedback and I think it is AWESOME. The insertion of text is also beneficial.”“option was easy to access”“I liked the audio because this instructor said more than ‘good job’.”“Listening to feedback as you went through a paper was very helpful, almost like a one on one with the instructor to hear their thoughts as they progressed through reading the paper.”“more personal, negative comments seen as more constructive. Increased perception of teacher engagement.”
Faculty ExampleExample of student paperExample34
Summary - FacultyAffords elaborationPersonalizes feedbackEasier to express oneself -“feel freer to expand on comments”Expect technical glitches√ complex assignments, research papers, critiquesAll recommend its use35
What are limitations of audio commenting?36
RecommendationsProvide portal of resources and have a resource person for inquiries/supportInvestigate further use –discussions/group work to evaluate impactInvestigate LMS which supports user-friendly audio embedInvestigate further use by/purchase of software for interested facultyInvestigate options for student use37

Faculty&student perceptionsofasynchronousaudiofeedbackcnie2011

  • 1.
    Faculty & StudentPerceptions of Asynchronous Audio FeedbackDenise Nelson
  • 2.
    PlanBackgroundAudio feedback processtechnicalcommentingstrategiesResearch process & findings faculty trainingstudents/faculty perspectivesRecommendationsFuture Interest2
  • 3.
    Who has experiencewith audio commenting of student coursework?3
  • 4.
    BackgroundFramed by theCommunity of Inquiry theoretical frameworkDraws upon the work of Dr. Phil Ice on audio feedback, differences being population – mostly undergradslevel of cognition not analyzedfaculty perspectivessituated in SIAST - context of 3 online programsPractical Nursing ProgramPerioperative Nursing ProgramFaculty Certificate ProgramProject sponsored by Campus Saskatchewan and SIASTResearch conducted Fall 2009 – June 20104
  • 5.
    5Garrison, D. R.,Anderson, T., & Archer, W. (2000).
  • 6.
    Community of Inquiry6Socialpresence is “the ability of participants to identify with the community (e.g., course of study), communicate purposefully in a trusting environment, and develop inter-personal relationships by way of projecting their individual personalities.” (Garrison, 2009) Teaching Presence  is the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes (Anderson, Rourke, Garrison & Archer, 2001). Cognitive Presence is the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse (Garrison, Anderson & Archer, 2001).
  • 7.
    Focus on FeedbackStudentscan cope without face-to-face teaching, but they cannot cope without regular feedback on assignments (Gibbs & Simpson, 2004).7
  • 8.
    Audio Feedback TechniqueConvertstudent document to pdf.Configure Adobe Acrobat Professional with comment/mark-up tools.Use Adobe Acrobat Pro, microphone/headset and Microsoft Sound Recorder to embed comments. Listen to recording to ensure clarity.Inform students to double-clickProvide upfront note to students to contact faculty if difficulty in accessing audio.8
  • 9.
  • 10.
    Additional Hardware/SoftwareAdobe AcrobatProHeadset/MicrophoneUSB, noise-canceling10
  • 11.
    Read the entirestudent assignment.Decide which feedback audio/text mark-up.Give feedback like face-to-face.Use audio to elaborate details, summarize, give examples/references.Integrate and situate audio and text comments.Do not repeat the written word in audio.Situate mark in audio comment.11Commenting Strategies
  • 12.
    …Commenting StrategiesUse textmark-up for specific grammar, punctuation etc. Use mark-up tools - customized stamps, highlighting, callout, arrows...Can insert attachments.Include rubrics.Other uses - summarizing/weaving discussions, exam review12
  • 13.
  • 14.
  • 15.
  • 16.
    QuestionsIn what manneris perceived learning impacted by the use of audio feedback?16Do students learning in asynchronous learning environments believe audio or text-based student feedback is a more effective means of interaction with their faculty?
  • 17.
    How does theuse of audio feedback impact the students’ sense of community in asynchronous learning networks?
  • 18.
    What relationship existsbetween the use of audio feedback and student /faculty?MethodNested mixed methodEnd of course web surveyFocus groupFollow-up emailed interview questions17
  • 19.
    SampleConvenienceNursing programs &Faculty Certificate Program (FCP) students and facultyParticipants:8 faculty respondents (all)14 student respondents – 8 FCP, 6 Practical Nursing Program18
  • 20.
  • 21.
    Data CollectionProgram areasrandomly selected assignments to include written feedback only or audio and written feedback.Nature of assignments ranged from paragraph responses to major papers.~ half of assignments used written feedback and half used written and audio feedback.Completion of course20
  • 22.
  • 23.
    Audio vs.Text --Which is More Effective?Qualitative – both students & faculty perceived that audio in conjunction with written is most effectiveStudents and faculty comfortable with written and reluctant to negate its valueStudents perceived audio more effective rePersonalizationRetention MotivationUnderstanding instructor’s intent Feelings of involvement and instructor caring22
  • 24.
    Student Perceptions ofAuditory Compared to Written23
  • 25.
    Student Voices24“Certainly toneof voice and inflection help to distinguish intended meaning more accurately than written text.”“I enjoyed the enhanced feedback (audio), and I think with it an instructor can be more detailed with his analysis. If [instructor] had to write out all his advice for my first paper it may have been longer than the paper itself.”“I was thoroughly impressed and motivated by the format (audio and written) selected…”
  • 26.
    Student Follow-Up EmailResponses Re useful feedback25“Audio comments could include references to other resources, good examples that could replace errors, and specific pieces of work that were good and why.”“the most important thing in my mind is to give something that isn’t as easy to write down—for example something a bit more lengthy you might want to say or a longer explanation etc. The things that go into the marking that you can’t just jot down really quickly.”
  • 27.
    Audio Feedback Impacton Sense of Community?ConnectednessPersonalizationAuthenticityEnhances social presence26
  • 28.
    Student Voices27“It wasnice to have the audio of the instructors, it made me feel more personal and that you were more of a class.” “I thought it was easy to use and make you feel more connected to the instructor, even in a distance course.”“I enjoyed the auditory feedback because I felt more like the instructor was speaking directly to me, although I appreciated the written comments equally.”
  • 29.
    Audio Feedback &Student Satisfaction?Based on open comments of web survey, 9 of the 11 students seemed satisfied. “did not always work”“annoying, a true conversation or written comments would be better”“It seemed the instructor was trying to sound neutral in the audio feedback which left a feeling of apathy.”voice inflection and nuance to grasp more meaningmore and elaborate feedback DetailsExamplesReferences28
  • 30.
    Perceived Learning Impactedby Audio Feedback?Faculty and students perceive audio enhances learning.3/4 students who responded to email interview questions revealed that audio feedback supports learning when it is personable, clear and detailed as “opposed to a check mark.”29“I found that I retained the verbal feedback better than just reading about it.”“Sometimes it (audio feedback) was very informative. Hearing their voices made it stick more into my head. I took it more seriously for if you wanted to know why something was the way it was marked, you had to listen.”
  • 31.
    Audio Feedback &Faculty Satisfaction?Perceive benefits of social, cognitive and teaching presence Added value - more quantity and quality of feedback, even though it requires more faculty time to plan and createIncorporating a variety of media helps to project teaching presence and support student’s varied learning preferences.Liked informalityPreferred use for lengthy and complex assignments30
  • 32.
  • 33.
    General Research FindingsNoclear preference for written or audioMore time required to listen and/or provide audio commentsStudent populations differQuality of audio dependent on various factors: audio settings, equipment, noise32
  • 34.
    More Student Voices33“Certainlytone of voice and inflection help to distinguish intended meaning more accurately than written text.”“This is my first experience with audio feedback and I think it is AWESOME. The insertion of text is also beneficial.”“option was easy to access”“I liked the audio because this instructor said more than ‘good job’.”“Listening to feedback as you went through a paper was very helpful, almost like a one on one with the instructor to hear their thoughts as they progressed through reading the paper.”“more personal, negative comments seen as more constructive. Increased perception of teacher engagement.”
  • 35.
    Faculty ExampleExample ofstudent paperExample34
  • 36.
    Summary - FacultyAffordselaborationPersonalizes feedbackEasier to express oneself -“feel freer to expand on comments”Expect technical glitches√ complex assignments, research papers, critiquesAll recommend its use35
  • 37.
    What are limitationsof audio commenting?36
  • 38.
    RecommendationsProvide portal ofresources and have a resource person for inquiries/supportInvestigate further use –discussions/group work to evaluate impactInvestigate LMS which supports user-friendly audio embedInvestigate further use by/purchase of software for interested facultyInvestigate options for student use37

Editor's Notes

  • #5 3,000 grant from Campus SaskatchewanPractical Nursing Program – 5 online coursesPerioperative Nursing Program – 4 online coursesFaculty Certificate Program – 3 online coursesIce’s subjects – graduate and doctoral students
  • #9 Convert student document to pdf --can do several docs with one command.
  • #10 Provide up front note to students to contact faculty if difficulty in accessing audio
  • #11 Adobe Acrobat Pro 10 ~ $60 education pricingDynex USB headset with microphone–noise canceling ~$55.Other possibilities of software – interactive pdfs
  • #12 Give feedback informally as though you were sitting face-to-face with the students and discussing their coursework.Integrate and situate audio and text comments so students clearly understand their meaning and relevance.Some faculty situate mark in audio comment to increase possibility that student listens to it.Read the entire student assignment.Decide which feedback audio/text mark-up.Give feedback like face-to-face.Use audio to elaborate details, summarize, give examples/references.Integrate and situate audio and text comments.Do not repeat the written word in audio.Situate mark in audio comment.
  • #13 Note limitations for printing (e.g., sticky notes don’t print rather use text boxes)Use text mark-up for specific grammar, punctuation etc. Use mark-up tools such as customized stamps, highlighting, sticky notes, callout, arrows...Can insert attachments into student coursework (e.g., APA checklist)Consider including rubrics in pdfSuggest use for summarizing/weaving discussions
  • #16 Phil presented at CNIE 2008 on the Impact of Asynchronous Audio Feedback on Teaching, Social and Cognitive Presence.Writing comments on assignments major part of faculty workload in post-secondary ed (Gibbs & Simpson, 2004)Gibbs & Simpson (2004) describe conditions under which assessment supports students’ learningFeedback not understood by students (Lea & Street,1998)Feedback not read by students (Hounsell,1987)Audio element strengthens sense of community & faculty’s ability to influence more personalized communication with students (Ice, Curtis, Wells and Phillips, 2007; Oomen-Early, Bold, Wigington, Gallien & Anderson, 2008; Wilson, 2009)Ice et al. (2007) found that students perceived audio feedback to be more effective that text-based feedback for conveying nuance,associated with feelings of increased involvement and enhanced community interactions,associated with increased retention of contentperception that instructor cared more about the student.
  • #24 When using auditory feedback, inflection in the instructor’s voice made his/her intent clearAuditory comments made me feel more involved in the course than written commentsThe instructor’s intent was clearer when using auditory comments rather than written commentsAuditory comments are more personal than written commentsI retained auditory comments better than written commentsAuditory comments motivated me more than written comments
  • #30 Further investigation of student coursework analysis is warranted to the degree of increased cognition.
  • #33 No clear preference for written or audio feedback—valuable in combinationMore time required to listen and/or provide audio commentsStudent populations differ (undergraduate/professional faculty/graduate) – seems to be more valued by those with higher education levelQuality of audio dependent on various factors: audio settings and equipment (e.g., Dynex headset)
  • #34 Overall Positive Comments
  • #36 Affords elaboration of instructor comments (used for summation and/or specific feedback)Personalizes feedback (expression, inflection, humour) Easier to express oneself than formalized written comments (“feel freer to expand on comments”)Technical glitches when starting (audio setup; eliminate background noise)More valuable with complex assignments, research papers, critiques - effort to “feed forward”large than small class size – could also copy and paste generic clips,All faculty would recommend use of audio commenting to other faculty
  • #37 Orientation timeInstructor time to provide meaningful feedbackEquipment: USB noise cancelling headset & microphone (e.g., Dynex ~$53.)Now available Adobe Acrobat X Pro - $56.Considerations: Consider installing software on laptop for portability – providing feedback anywhereSuggest software on laptop for portabilityAsk audience????
  • #38 Provide portal of resources and have a resource person for inquiries/supportInvestigate further use in discussions and group work to evaluate impactInvestigate LMS which supports user-friendly audio embedInvestigate further use by/purchase of software for interested faculty (e.g., uptake by Psychiatric Nursing and Learning Assistance)Investigate options for student use
  • #39 Dr. Graham GibbsTransforming the Experience of Students through Assessment (TESTA) is a £200,000 National Teaching Fellowship project on programme assessment, funded by the Higher Education Academy, led by the University of Winchester (2009-2012).What is the central pedagogic problem TESTA addresses?Assessment innovations at the individual module level often fail to address assessment problems at the programme-level, some of which, such as too much summative assessment and not enough formative assessment, are a direct consequence of module-focused course design and innovation. QAA course specification requirements (of outcomes and criteria) at the module level have not succeeded in clarifying for students the goals and standards they should be orienting their overall effort towards. There needs to be more consistency between modules, across programmes, and a greater emphasis on progressively developing students’ internalisation of programme-level standards, over time, rather than relying on documentation to specify criteria at the level of assignments or modules. Co-ordinatedprogramme-wide assessment policy and practice is required to address both these issues.University of Winchester, University of Chichester, Bath Spa University, University of WorcesterDenise, yes, work on assessment is exactly what I do in my retirement! I am working with a project called TESTA which you can read about at http://www.testa.ac.uk/ The web site contains a video lecture (that I have now given many times) and all the resources you'd need to undertake the same kind of 'research and development' interventions with academic departments that TESTA does. It is expanding rapidly across and within universities and TESTA is now concentrating on building the capacity of others to use the TESTA methodology. The scale of change it is producing is exhilarating! Graham