Dr. Eloise Tan
                    &
                    Morag Munro
                    Learning
                    Innovati on Unit,
                    DCU

                    June 7 th , 201 2
                    CELT conference
                    National
                    Univer sity of
                    Ireland Galway




@t_eloise #celt12
REFLECTING ON TEACHING PRACTICE

 Module: Online assessment          Module: Post Graduate Tutor
       and feedback                       demonstrator
 Blended professional              Blended course for
  development course
 Participants across Dublin         postgraduate tutor
 One to face to face at the         demonstrators
  beginning and at the end of
  the module                        Participants from DCU
 Weekly                            Face to face delivery
  lectures/discussions
  through webinars                  Assessments online
  (Blackboard Collaborate)
 Discussion forum
 Online content


                          @t_eloise #celt12
“TO BEND OR TURN
BACKWARDS” (SALMON,
          P.381, 2002)




                                 Image
                                 credit:
                                 http://www
                                 .gilbertglas
                                 s.com/ser v
                                 ices
             @t_eloise #celt12
ASSESSING REFLECTION

Reflective writing is
 a common type of
 assessment for
 higher order learning
 across a variety of
 disciplines (Craf t, 2005;
 Das Gupta & Charon, 2004;
 Hoover, 1993).




                              Image credit:
   @t_eloise #celt12          http://www.gilbertglass.com/services
REFLECTION ONLINE

                                       Conflict between
                                        personal nature of
                                        reflective writing
                                        and the perceived
                                        impersonal nature of
                                        technology by some?
                                        (Weiss, 2000).




                                                     @t_eloise #celt12
Image credit:
http://www.gilbertglass.com/services
RESEARCH ON ONLINE REFLECTION

 Online reflective learning in graduate education ( Andrusyszyn
  & Davie, 1997)

 Online reflective learning in nursing (Morgan et al, 2006)

 Study of reflections in action for online learners (Salmon ,
  2002)

 Quan comparison between reflections in an online and a f2f
  MBA (Hay et al, 2004)




                           @t_eloise #celt12
REFLECTING ON TEACHING PRACTICE

 Module: Online assessment          Module: Post Graduate Tutor
       and feedback                       demonstrator
 Blended professional              Blended course for
  development course
 Participants across Dublin         postgraduate tutor
 One to face to face at the         demonstrators
  beginning and at the end of
  the module                        Participants from DCU
 Weekly                            Face to face delivery
  lectures/discussions
  through webinars                  Assessments online
  (Blackboard Collaborate)
 Discussion forum
 Online content


                          @t_eloise #celt12
REFLECTING LEARNING JOURNAL

   5 entries
   Models of reflection provided
   One to one
   Scaf folding
   Screencasting as general feedback to group correct / praise
   Feedback cycle (one week)
   Use of rubric
   Formative to summative assessment




                              @t_eloise #celt12
@t_eloise #celt12
SCAFFOLDING OF REFLECTION




Formative                       Summative


            @t_eloise #celt12
General feedback
                      Journal entry #1
      "My concerns / expectations for online assessment
              and feedback for my teaching"



@t_eloise #celt12
Reflective depth 0                1                 2                 3                 4                  5                  6

                No reflection     Only includes     Includes          Includes          Includes           Includes           Includes description of
                evident. The      descriptions of   description of    description of    description of     description of     events; demonstrates
                entry relies      events; no        events and        events,           events;            events;            analysis from a
                wholly on         reflection is     some analysis     analysis of       demonstrates       demonstrates       different perspective(s),
                citations from    demonstrated      behind the        events; and       analysis from a    analysis and       demonstrates ability to
                course content    beyond the        events; no        demonstrates      different          evaluation         think of alternative
                with no           descriptions      evidence of       analysis of the   perspective(s),    from a different   outcomes or
                reflection or                       using multiple    events from a     demonstrative      perspective(s),    approaches, writing
                sharing of                          perspectives in   different         s ability to       demonstrative      conveys an intention to
                experiences /                       analysing the     perspective(s)    think of           s ability to       transform teaching
                thoughts.                           issues                              alternative        think of           practices based on
                                                                                        outcomes or        alternative        reflective learning.
                                                                                        approaches         outcomes or
                                                                                                           approaches




Presentation    0                                                     1                                    2

                Chaotic in organization and presentation of ideas.    The writing lacked an organized      Writing is well-focused; arguments or
                                                                      flow and the ideas were hard to      perspectives are precisely defined and
                                                                      follow                               explained; coherent flow in developing an
                                                                                                           insightful idea demonstrated

Use of course   0                                                     1                                    2
content
                No reference to course content.                       Acquired knowledge from course       Acquired knowledge from course content is
           @t_eloise #celt12                                          content is linked to personal        linked to personal reflection in a meaningful
                                                                      reflection at a superficial level.   way that shows a deep understanding of
                                                                                                           content.
Common strengths
      O Sharing hopes / concerns
      O Honest
      O Identify individual goals for learning in this
         course




@t_eloise #celt12
Common challenges
      O Described many events, reflected a little (use
        your word count wisely)
      O Stream of consciousness affected
        organisation/presentation of ideas
      O Connections to content (pre-reading resource)
        missed




@t_eloise #celt12
For the next entry
O Try focusing in on fewer events to allow for greater
  reflection
O Question declarative statements such as ‘The best time I
  ran a module was…’ Best for whom? What criteria are
  you judging best by?
O Question use of adjectives such as ‘The anxious
  students…’ Could they have been confused? Bored? What
  led you to interpret them as anxious?



 @t_eloise #celt12
For the next entry
      O Go back and edit – especially if you wrote
        ‘stream of consciousness’
      O Think about what influences your perspective
        (your experiences, your theoretical standpoint,
        the constraints / opportunities in your
        situation)
      O Think about it from another’s point of view



@t_eloise #celt12
Common themes
      O Biggs (2003) model of Lecturer / Student
          perspectives on assessment
      O   Looking for efficiency but still effectiveness
      O   Changing perceptions of online assessment
      O   Institutional roadblocks / pressures
      O   Want to learn more about peer / self
          assessment


@t_eloise #celt12
UNINTENDED CONSEQUENCES


                                       Feedback for
                                       lecturer
                               Deepening
                               engagement

                                  Multimedia as
                                  reflection / building
Personal to                       digital literacy
 Dialogical
                                      Success beyond
                                      measure of rubric


                  @t_eloise #celt12
Image credit:
                    http://www.innovativelearning.com
                    /educational_psychology/developm
                    ent/zone-of-proximal-
@t_eloise #celt12   development.html
DIGITAL
                    LITERACY
                    -   Should I
                        assess it?
                    -   How?




@t_eloise #celt12
WHAT DID TECHNOLOGY FACILITATE?

 Ease of marking / transparency – Rubrics in Moodle 2

 Comment inline contributed to emerging „dialogical‟ nature of
  entries

 Content (and prompts) moved beyond text to demonstrate a
  digital literacy through meaningful engagement with
  multimedia

 Communicate common themes/successes/challenges –
  screencasting through Camtasia

 Manageable one to one sessions – Blackboard Collaborate


                           @t_eloise #celt12
LESSONS LEARNED




Expectations must    Account for digital      Create dialogue
be clear; signpost        literacy           through feedback




                     Scaffold rubric (?)


                         @t_eloise #celt12
MOVING FORWARD


How / if to assess multimedia as
reflection (look to digital literacy)


Audio reflections


        Peer to peer reflections

         @t_eloise #celt12
@t_eloise #celt12
IMAGE CREDITS

 Communication http:// rkb-lbc-
  09.blogspot.ie/2009/08/communication -fundamentals-par t-4.html
 Thinking outside the box http://www.zenlawyerseattle.com/thoughts -
  on-per sonal-injur y -law/think -outside-the-box/
 Multimedia http://www.canidae.org/multimedia -in-education /
 Student engagement http://campusclimatesolutions.org/student -
  engagement -in-climate-action-planning/
 Thermometer
 http://thermometer-clipar t.blogspot.ie/2011/08/thermometer.html
 Signpost
 http://read -out.net/
 Digital literacy http://www.onlinecollege.org/201 2/04/24/rethinking -
  digital-literacy/
 Dialogue http://trinities.org/blog/archives/2837




                                 @t_eloise #celt12

A case study of reflective learning online

  • 1.
    Dr. Eloise Tan & Morag Munro Learning Innovati on Unit, DCU June 7 th , 201 2 CELT conference National Univer sity of Ireland Galway @t_eloise #celt12
  • 2.
    REFLECTING ON TEACHINGPRACTICE Module: Online assessment Module: Post Graduate Tutor and feedback demonstrator  Blended professional  Blended course for development course  Participants across Dublin postgraduate tutor  One to face to face at the demonstrators beginning and at the end of the module  Participants from DCU  Weekly  Face to face delivery lectures/discussions through webinars  Assessments online (Blackboard Collaborate)  Discussion forum  Online content @t_eloise #celt12
  • 3.
    “TO BEND ORTURN BACKWARDS” (SALMON, P.381, 2002) Image credit: http://www .gilbertglas s.com/ser v ices @t_eloise #celt12
  • 4.
    ASSESSING REFLECTION Reflective writingis a common type of assessment for higher order learning across a variety of disciplines (Craf t, 2005; Das Gupta & Charon, 2004; Hoover, 1993). Image credit: @t_eloise #celt12 http://www.gilbertglass.com/services
  • 5.
    REFLECTION ONLINE Conflict between personal nature of reflective writing and the perceived impersonal nature of technology by some? (Weiss, 2000). @t_eloise #celt12 Image credit: http://www.gilbertglass.com/services
  • 6.
    RESEARCH ON ONLINEREFLECTION  Online reflective learning in graduate education ( Andrusyszyn & Davie, 1997)  Online reflective learning in nursing (Morgan et al, 2006)  Study of reflections in action for online learners (Salmon , 2002)  Quan comparison between reflections in an online and a f2f MBA (Hay et al, 2004) @t_eloise #celt12
  • 7.
    REFLECTING ON TEACHINGPRACTICE Module: Online assessment Module: Post Graduate Tutor and feedback demonstrator  Blended professional  Blended course for development course  Participants across Dublin postgraduate tutor  One to face to face at the demonstrators beginning and at the end of the module  Participants from DCU  Weekly  Face to face delivery lectures/discussions through webinars  Assessments online (Blackboard Collaborate)  Discussion forum  Online content @t_eloise #celt12
  • 8.
    REFLECTING LEARNING JOURNAL  5 entries  Models of reflection provided  One to one  Scaf folding  Screencasting as general feedback to group correct / praise  Feedback cycle (one week)  Use of rubric  Formative to summative assessment @t_eloise #celt12
  • 9.
  • 10.
    SCAFFOLDING OF REFLECTION Formative Summative @t_eloise #celt12
  • 11.
    General feedback Journal entry #1 "My concerns / expectations for online assessment and feedback for my teaching" @t_eloise #celt12
  • 12.
    Reflective depth 0 1 2 3 4 5 6 No reflection Only includes Includes Includes Includes Includes Includes description of evident. The descriptions of description of description of description of description of events; demonstrates entry relies events; no events and events, events; events; analysis from a wholly on reflection is some analysis analysis of demonstrates demonstrates different perspective(s), citations from demonstrated behind the events; and analysis from a analysis and demonstrates ability to course content beyond the events; no demonstrates different evaluation think of alternative with no descriptions evidence of analysis of the perspective(s), from a different outcomes or reflection or using multiple events from a demonstrative perspective(s), approaches, writing sharing of perspectives in different s ability to demonstrative conveys an intention to experiences / analysing the perspective(s) think of s ability to transform teaching thoughts. issues alternative think of practices based on outcomes or alternative reflective learning. approaches outcomes or approaches Presentation 0 1 2 Chaotic in organization and presentation of ideas. The writing lacked an organized Writing is well-focused; arguments or flow and the ideas were hard to perspectives are precisely defined and follow explained; coherent flow in developing an insightful idea demonstrated Use of course 0 1 2 content No reference to course content. Acquired knowledge from course Acquired knowledge from course content is @t_eloise #celt12 content is linked to personal linked to personal reflection in a meaningful reflection at a superficial level. way that shows a deep understanding of content.
  • 13.
    Common strengths O Sharing hopes / concerns O Honest O Identify individual goals for learning in this course @t_eloise #celt12
  • 14.
    Common challenges O Described many events, reflected a little (use your word count wisely) O Stream of consciousness affected organisation/presentation of ideas O Connections to content (pre-reading resource) missed @t_eloise #celt12
  • 15.
    For the nextentry O Try focusing in on fewer events to allow for greater reflection O Question declarative statements such as ‘The best time I ran a module was…’ Best for whom? What criteria are you judging best by? O Question use of adjectives such as ‘The anxious students…’ Could they have been confused? Bored? What led you to interpret them as anxious? @t_eloise #celt12
  • 16.
    For the nextentry O Go back and edit – especially if you wrote ‘stream of consciousness’ O Think about what influences your perspective (your experiences, your theoretical standpoint, the constraints / opportunities in your situation) O Think about it from another’s point of view @t_eloise #celt12
  • 17.
    Common themes O Biggs (2003) model of Lecturer / Student perspectives on assessment O Looking for efficiency but still effectiveness O Changing perceptions of online assessment O Institutional roadblocks / pressures O Want to learn more about peer / self assessment @t_eloise #celt12
  • 18.
    UNINTENDED CONSEQUENCES Feedback for lecturer Deepening engagement Multimedia as reflection / building Personal to digital literacy Dialogical Success beyond measure of rubric @t_eloise #celt12
  • 19.
    Image credit: http://www.innovativelearning.com /educational_psychology/developm ent/zone-of-proximal- @t_eloise #celt12 development.html
  • 20.
    DIGITAL LITERACY - Should I assess it? - How? @t_eloise #celt12
  • 21.
    WHAT DID TECHNOLOGYFACILITATE?  Ease of marking / transparency – Rubrics in Moodle 2  Comment inline contributed to emerging „dialogical‟ nature of entries  Content (and prompts) moved beyond text to demonstrate a digital literacy through meaningful engagement with multimedia  Communicate common themes/successes/challenges – screencasting through Camtasia  Manageable one to one sessions – Blackboard Collaborate @t_eloise #celt12
  • 22.
    LESSONS LEARNED Expectations must Account for digital Create dialogue be clear; signpost literacy through feedback Scaffold rubric (?) @t_eloise #celt12
  • 23.
    MOVING FORWARD How /if to assess multimedia as reflection (look to digital literacy) Audio reflections Peer to peer reflections @t_eloise #celt12
  • 24.
  • 25.
    IMAGE CREDITS  Communicationhttp:// rkb-lbc- 09.blogspot.ie/2009/08/communication -fundamentals-par t-4.html  Thinking outside the box http://www.zenlawyerseattle.com/thoughts - on-per sonal-injur y -law/think -outside-the-box/  Multimedia http://www.canidae.org/multimedia -in-education /  Student engagement http://campusclimatesolutions.org/student - engagement -in-climate-action-planning/  Thermometer  http://thermometer-clipar t.blogspot.ie/2011/08/thermometer.html  Signpost  http://read -out.net/  Digital literacy http://www.onlinecollege.org/201 2/04/24/rethinking - digital-literacy/  Dialogue http://trinities.org/blog/archives/2837 @t_eloise #celt12