Effects of a Scaffold on Quality
              of Goals Set By Undergraduate
              Students

              Elizabeth A. Webster, Mariel Miller, &
              Allyson F. Hadwin

              This research was funded by a SSHRC Standard Research Grant 410-2008-0700 (A.
              Hadwin) and a SSHRC Doctoral Fellowship (E. Webster).




University                                                                    Technology Integration &
of Victoria                      1                                            Evaluation Research Lab
Self-Regulated Learning
             Active regulation of cognitions, motivation, and
              behaviours to accomplish goals
             Successful self-regulated learners take control of
              their own learning by
               Setting task-specific goals
               Selectively using metacognitive, motivational, and
                learning strategies
               Adapting in the face of challenges




University                                                      Technology Integration &
of Victoria                             2                       Evaluation Research Lab
Winne & Hadwin’s Model of SRL
                                 Phase 1:
                                   Task
                                Perceptions




                  Phase 4:     Monitoring     Phase 2:
                                                Goal
                 Large Scale       &          Setting &
                 Adaptation
                               Evaluating     Planning




                                 Phase 3:
                                  Task
                                Enactment



University                                                Technology Integration &
of Victoria                         3                     Evaluation Research Lab
Properties of Effective Goals in SRL

             Specific
             Short-term
             Challenging yet attainable

             Learning vs. performance
             Self-set vs. assigned



University                                   Technology Integration &
of Victoria                          4       Evaluation Research Lab
Challenges with Goal Setting
             Setting vague, long-term goals rather than goals
              to guide specific study episodes


                                 Get started!

                                 Organize myself or my time.

                                 Stay on top of my work and
                                 catch up on all my readings.



University                                                      Technology Integration &
of Victoria                          5                          Evaluation Research Lab
TASC Goals
              imeframe
                 2 hours or less
                 Monitor progress frequently
              ctions
                 Cognitive processing
                 Guide strategy choice
              tandard
                 Concrete criterion
                 Evaluate progress
              ontent/concepts
                 Specific course material
                 Identify and focus on specific elements


University                                                  Technology Integration &
of Victoria                                     6           Evaluation Research Lab
Goal-Related Perceptions
             Goal efficacy
             Perceived goal difficulty
             Perceived goal attainment




University                                Technology Integration &
of Victoria                         7     Evaluation Research Lab
Supporting Goal Setting
             Our past research
               Goal   setting prompt


                            State one goal you have for
                            improving your learning next week.


             Problem: goal quality not high enough


University                                                       Technology Integration &
of Victoria                             8                        Evaluation Research Lab
Purpose & Research Questions
             Examine how students might be better
              supported in setting high quality studying goals

              1.   How does a scaffolded goal setting prompt
                   contribute to goal quality and goal perceptions?
              2.   Do goal quality and goal perceptions change over
                   time?
              3.   Are goal quality and goal perceptions related to
                   academic performance?


University                                                      Technology Integration &
of Victoria                             9                       Evaluation Research Lab
METHODS




University              Technology Integration &
of Victoria      10     Evaluation Research Lab
Research Setting
             ED-D 101: Learning Strategies for University
              Success
             Lecture
               Week   3: Goal setting module
             Lab (10-20 students)
               Weekly   goal setting




University                                               Technology Integration &
of Victoria                             11               Evaluation Research Lab
Personal Planning Tool (PPT)
             Weekly diary
             Cycle of planning & reflecting anchored in day-
              to-day studying practices
             Supports SRL and promotes transfer of SRL
              cycles across study episodes




University                                                Technology Integration &
of Victoria                         12                    Evaluation Research Lab
GOAL        PERCEIVED DIFFICULTY
                          1 = easy to 5 = extremely
                                 challenging

                                               GOAL EFFICACY
                                           1 = cannot do it at all to
                                          5 = highly certain I can do
                                                      it]



University                                                 Technology Integration &
of Victoria          13                                    Evaluation Research Lab
PERCEIVED ATTAINMENT
                   1 = did not attain it at all to
                       5 = fully attained it




University                                Technology Integration &
of Victoria   14                          Evaluation Research Lab
Conditions
              Open-Ended Condition (PPT A)          Scaffolded Condition (PPT B)
        For the task you selected above, set      My goal is to define
        one goal for this week. Remember a                      explain
        good goal statement includes                            analyze
        information about: what action, what
        course content or concepts, when (day,    What? (Course concepts or content)
        time, duration) & to what degree,         _______________________________
        amount, or standard.
        _______________________________           When? (day, time)
        _______________________________           _______________________________
        _______________________________
        _______________________________           To what degree, amount, or standard?
        _______________________________           _______________________________




University                                                                     Technology Integration &
of Victoria                                  15                                Evaluation Research Lab
97 Participants
                                     Open-Ended       Scaffolded

              n                      43               54

              Female                 60.5%            51.9%

              Age (SD)               19.1 yrs (2.6)   19.4 yrs (4.1)

              High school avg (SD)   81.1% (7.5)      80.9% (6.4)

              Term GPA (SD)          5.0/9.0 (1.7)    5.0/9.0 (1.3)

              1st year               59.5%            50.0%

              Faculty of social
                                     53.5%            44.4%
              science


University                                                             Technology Integration &
of Victoria                                     16                     Evaluation Research Lab
Timeline
                           Goal Quality Coding

                 Week 1         Week 2            Week 3
                 PPT A          PPT A             PPT A

                              Intervention
        Week 4   Week 5    Week 6        Week 7     Week 8     Week 9
        PPT A    PPT A     PPT A         PPT A      PPT A      PPT A
        PPT B    PPT B     PPT B         PPT B      PPT B      PPT B


                           Goal Quality Coding

                 Week 10       Week 11            Week 12
                  PPT A         PPT A              PPT A
University                                                   Technology Integration &
of Victoria                         17                       Evaluation Research Lab
Coding Goal Quality
             565 goals
             2 independent coders
             Coded each TASC criterion
                  0 (absent), 1 (weak), or 2 (strong)
             Overall quality score out of 8

             Grouped goals into
                  Weak (0-2)
                  Moderate (3-4)
                  Good (5-6)
                  Excellent (7-8)

             Discrepancies resolved through discussion



University                                                Technology Integration &
of Victoria                                         18    Evaluation Research Lab
Goal Quality
    Level of
                 %
    quality                        Description                                         Example
             (N = 565)
    (score)
                         •Few or no properties of TASC      My goal is to balance my work with my own spare time. I plan
      Weak               goals                              on reading ahead in both my Soc and Poli textbooks before
               14.0%                                        next class. I don't have a specific time for this task to be done
       (1)               •Often planning goals or too
                                                            as long as I strive myself to read ahead before the next class.
                         vague
                         •At least two properties of TASC   On monday my task is to go to the library and find useful
                         goals                              resources for my essay. The following day I will give myself an
   Moderate
               36.5%     •Often behavioural in nature       hour to read over my sources and choose which sources will
     (2)                                                    be useful. By thursday night my goal is to have an outline and
                         and/or to-do lists                 thesis statement for my essay.
                         •At least three properties of      For PSYC test 2 on Friday, re-create a table of brain structures
                         TASC goals                         (brain areas, hindbrain, midbrain and forebrain) and organize
      Good                                                  specific functions for each of those brains. Explain in my words
               34.3%     •Overall goal is good
       (3)                                                  how each function works and relate each other.
                         •Missing one property or weak in
                         two properties
                         •All four properties of TASC       Friday from 10:30 to 12:30, I will analyze the different DC
                         goals                              Motor Speed vs Torque plots obtained from the lab 3 data and
   Excellent
               15.2%     •Overall goal is strong            be able to explain (in-depth) the effects of using terminal
      (4)                                                   voltage, field current, and series resistance in speed control of
                                                            DC motors.




University                                                                                                    Technology Integration &
of Victoria                                              19                                                   Evaluation Research Lab
FINDINGS




University               Technology Integration &
of Victoria      20      Evaluation Research Lab
Q1: Did Scaffolding Affect Goal Quality
              and Goal Perceptions?
             No difference between open-ended and
              scaffolded conditions at end of semester

             RM MANOVA assessing change over time in goal
              quality, goal efficacy, perceived difficulty, perceived
              attainment
               Mean scores at beginning (Weeks 1/2/3) and end (Weeks
                10/11/12) of the semester
               No interaction
                     F(4, 91) = .83, p = .512, partial   2   = .04



University                                                            Technology Integration &
of Victoria                                      21                   Evaluation Research Lab
Q2: Did Goal Quality and Goal Perceptions
              Change Over Time?
                  Main        effect of time
                          F(4, 91) = 27.04, p < .001, partial                2   = .54
                                   Beginning                    End
                                                                                   Repeated-measures ANOVA
                                  (T1, T2, T3)            (T10, T11, T12)
                                                                                                             Partial
                                  M           SD            M           SD            F           p                 2

       Goal qualitya
                                2.23         0.60         2.81         0.76         74.84       <.001           .44

       Goal efficacyb
                                3.97         0.58         4.09         0.70          2.52        .116           .03

       Perceived
                                3.25         0.65         3.42         0.71          4.55        .036           .05
       difficultyb
       Perceived
                                3.74         0.77         4.03         0.87          8.09        .005           .08
       attainmentb
              Note. N = 96. aGoal quality scored on a scale from 1 to 4. bRated on a scale from 1 to 5.

University                                                                                                Technology Integration &
of Victoria                                                   22                                          Evaluation Research Lab
Distribution of Goal Quality at
              Beginning and End of Semester
                                   179
                156

                      124
                            97
                                         Weak/Moderate
                                         Good/Excellent




                Beginning        End



University                                            Technology Integration &
of Victoria                  23                       Evaluation Research Lab
Q3: Were Goal Quality and Goal Perceptions
              Related to Term GPA?
                                                 Term GPA
                                             r              p
                Goal quality
                 Beginning                  .40        <.001
                 End                        .37        <.001
                Goal efficacy
                 Beginning                  .24         .020
                 End                        .15         .152
                Perceived difficulty
                 Beginning                  -.02        .836
                 End                         .08        .425
                Perceived attainment
                 Beginning                  .26         .011
                 End                        .20         .046
                Note. N = 96.
University                                                      Technology Integration &
of Victoria                            24                       Evaluation Research Lab
DISCUSSION




University                 Technology Integration &
of Victoria       25       Evaluation Research Lab
Supporting Goal Setting
             Higher goal quality associated with higher term
              GPA
             Students can learn to set high quality goals
               Instruction
                          in effective goal setting
               Weekly practice



             Evolving goal setting prompt



University                                                Technology Integration &
of Victoria                            26                 Evaluation Research Lab
Original                                                         Revised
                            Prompt                                                           Prompt


   State one goal you have for improving                             For the task you selected above, set one
   your learning next week.                                          goal for this week. Remember a good
                                                                     goal statement includes information
                                                                     about: what action, what course content
                                                                     or concepts, when (day, time, duration)
                                                                     & to what degree, amount, or standard.
     This week while studying for my psychology midterm I
     will make a mind map of the different fields of
     psychology. I will cover their main ideas, main                 For this week my goal is to create a conecpt map for
     psychologists, and other key terms. This will help me to        Econ 205 making connections from chapters 8
     visually see the differences so that I can compare and          (monopolys) and 9 (oligopolies). Using the connections,
     contrast them during the test when the question comes           I will formulate a paragraph summarizing the chapters
     up. I will do this Friday morning for 2 hours. The mind         using at least 5 of the connections mentioned in the
     map will be colourful and contain pictures as well as           concept map. I will do this goal from 7-8PM Thursday
     words.                                                          night.



     My goal for this week is to work on my math practice             Friday from 10:30 to 12:30, I will analyze the different
     problems and readings for an hour and half every day in          DC Motor Speed vs Torque plots obtained from the lab
     order to begin to understand how to use matrices when            3 data and be able to explain (in-depth) the effects of
     solving algebraic problems, and i wish further my                using terminal voltage, field current, and series
     understanding of chapter 8 probability concepts.                 resistance in speed control of DC motors.

University                                                                                                     Technology Integration &
of Victoria                                                     27                                             Evaluation Research Lab
Current Goal Setting Prompt




University                             Technology Integration &
of Victoria               28           Evaluation Research Lab
THANK YOU!




University                 Technology Integration &
of Victoria       29       Evaluation Research Lab

Effects of a Scaffold on Quality of Goals Set By Undergraduate Students

  • 1.
    Effects of aScaffold on Quality of Goals Set By Undergraduate Students Elizabeth A. Webster, Mariel Miller, & Allyson F. Hadwin This research was funded by a SSHRC Standard Research Grant 410-2008-0700 (A. Hadwin) and a SSHRC Doctoral Fellowship (E. Webster). University Technology Integration & of Victoria 1 Evaluation Research Lab
  • 2.
    Self-Regulated Learning  Active regulation of cognitions, motivation, and behaviours to accomplish goals  Successful self-regulated learners take control of their own learning by  Setting task-specific goals  Selectively using metacognitive, motivational, and learning strategies  Adapting in the face of challenges University Technology Integration & of Victoria 2 Evaluation Research Lab
  • 3.
    Winne & Hadwin’sModel of SRL Phase 1: Task Perceptions Phase 4: Monitoring Phase 2: Goal Large Scale & Setting & Adaptation Evaluating Planning Phase 3: Task Enactment University Technology Integration & of Victoria 3 Evaluation Research Lab
  • 4.
    Properties of EffectiveGoals in SRL  Specific  Short-term  Challenging yet attainable  Learning vs. performance  Self-set vs. assigned University Technology Integration & of Victoria 4 Evaluation Research Lab
  • 5.
    Challenges with GoalSetting  Setting vague, long-term goals rather than goals to guide specific study episodes Get started! Organize myself or my time. Stay on top of my work and catch up on all my readings. University Technology Integration & of Victoria 5 Evaluation Research Lab
  • 6.
    TASC Goals imeframe  2 hours or less  Monitor progress frequently ctions  Cognitive processing  Guide strategy choice tandard  Concrete criterion  Evaluate progress ontent/concepts  Specific course material  Identify and focus on specific elements University Technology Integration & of Victoria 6 Evaluation Research Lab
  • 7.
    Goal-Related Perceptions  Goal efficacy  Perceived goal difficulty  Perceived goal attainment University Technology Integration & of Victoria 7 Evaluation Research Lab
  • 8.
    Supporting Goal Setting  Our past research  Goal setting prompt State one goal you have for improving your learning next week.  Problem: goal quality not high enough University Technology Integration & of Victoria 8 Evaluation Research Lab
  • 9.
    Purpose & ResearchQuestions  Examine how students might be better supported in setting high quality studying goals 1. How does a scaffolded goal setting prompt contribute to goal quality and goal perceptions? 2. Do goal quality and goal perceptions change over time? 3. Are goal quality and goal perceptions related to academic performance? University Technology Integration & of Victoria 9 Evaluation Research Lab
  • 10.
    METHODS University Technology Integration & of Victoria 10 Evaluation Research Lab
  • 11.
    Research Setting  ED-D 101: Learning Strategies for University Success  Lecture  Week 3: Goal setting module  Lab (10-20 students)  Weekly goal setting University Technology Integration & of Victoria 11 Evaluation Research Lab
  • 12.
    Personal Planning Tool(PPT)  Weekly diary  Cycle of planning & reflecting anchored in day- to-day studying practices  Supports SRL and promotes transfer of SRL cycles across study episodes University Technology Integration & of Victoria 12 Evaluation Research Lab
  • 13.
    GOAL PERCEIVED DIFFICULTY 1 = easy to 5 = extremely challenging GOAL EFFICACY 1 = cannot do it at all to 5 = highly certain I can do it] University Technology Integration & of Victoria 13 Evaluation Research Lab
  • 14.
    PERCEIVED ATTAINMENT 1 = did not attain it at all to 5 = fully attained it University Technology Integration & of Victoria 14 Evaluation Research Lab
  • 15.
    Conditions Open-Ended Condition (PPT A) Scaffolded Condition (PPT B) For the task you selected above, set My goal is to define one goal for this week. Remember a explain good goal statement includes analyze information about: what action, what course content or concepts, when (day, What? (Course concepts or content) time, duration) & to what degree, _______________________________ amount, or standard. _______________________________ When? (day, time) _______________________________ _______________________________ _______________________________ _______________________________ To what degree, amount, or standard? _______________________________ _______________________________ University Technology Integration & of Victoria 15 Evaluation Research Lab
  • 16.
    97 Participants Open-Ended Scaffolded n 43 54 Female 60.5% 51.9% Age (SD) 19.1 yrs (2.6) 19.4 yrs (4.1) High school avg (SD) 81.1% (7.5) 80.9% (6.4) Term GPA (SD) 5.0/9.0 (1.7) 5.0/9.0 (1.3) 1st year 59.5% 50.0% Faculty of social 53.5% 44.4% science University Technology Integration & of Victoria 16 Evaluation Research Lab
  • 17.
    Timeline Goal Quality Coding Week 1 Week 2 Week 3 PPT A PPT A PPT A Intervention Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 PPT A PPT A PPT A PPT A PPT A PPT A PPT B PPT B PPT B PPT B PPT B PPT B Goal Quality Coding Week 10 Week 11 Week 12 PPT A PPT A PPT A University Technology Integration & of Victoria 17 Evaluation Research Lab
  • 18.
    Coding Goal Quality  565 goals  2 independent coders  Coded each TASC criterion  0 (absent), 1 (weak), or 2 (strong)  Overall quality score out of 8  Grouped goals into  Weak (0-2)  Moderate (3-4)  Good (5-6)  Excellent (7-8)  Discrepancies resolved through discussion University Technology Integration & of Victoria 18 Evaluation Research Lab
  • 19.
    Goal Quality Level of % quality Description Example (N = 565) (score) •Few or no properties of TASC My goal is to balance my work with my own spare time. I plan Weak goals on reading ahead in both my Soc and Poli textbooks before 14.0% next class. I don't have a specific time for this task to be done (1) •Often planning goals or too as long as I strive myself to read ahead before the next class. vague •At least two properties of TASC On monday my task is to go to the library and find useful goals resources for my essay. The following day I will give myself an Moderate 36.5% •Often behavioural in nature hour to read over my sources and choose which sources will (2) be useful. By thursday night my goal is to have an outline and and/or to-do lists thesis statement for my essay. •At least three properties of For PSYC test 2 on Friday, re-create a table of brain structures TASC goals (brain areas, hindbrain, midbrain and forebrain) and organize Good specific functions for each of those brains. Explain in my words 34.3% •Overall goal is good (3) how each function works and relate each other. •Missing one property or weak in two properties •All four properties of TASC Friday from 10:30 to 12:30, I will analyze the different DC goals Motor Speed vs Torque plots obtained from the lab 3 data and Excellent 15.2% •Overall goal is strong be able to explain (in-depth) the effects of using terminal (4) voltage, field current, and series resistance in speed control of DC motors. University Technology Integration & of Victoria 19 Evaluation Research Lab
  • 20.
    FINDINGS University Technology Integration & of Victoria 20 Evaluation Research Lab
  • 21.
    Q1: Did ScaffoldingAffect Goal Quality and Goal Perceptions?  No difference between open-ended and scaffolded conditions at end of semester  RM MANOVA assessing change over time in goal quality, goal efficacy, perceived difficulty, perceived attainment  Mean scores at beginning (Weeks 1/2/3) and end (Weeks 10/11/12) of the semester  No interaction  F(4, 91) = .83, p = .512, partial 2 = .04 University Technology Integration & of Victoria 21 Evaluation Research Lab
  • 22.
    Q2: Did GoalQuality and Goal Perceptions Change Over Time?  Main effect of time  F(4, 91) = 27.04, p < .001, partial 2 = .54 Beginning End Repeated-measures ANOVA (T1, T2, T3) (T10, T11, T12) Partial M SD M SD F p 2 Goal qualitya 2.23 0.60 2.81 0.76 74.84 <.001 .44 Goal efficacyb 3.97 0.58 4.09 0.70 2.52 .116 .03 Perceived 3.25 0.65 3.42 0.71 4.55 .036 .05 difficultyb Perceived 3.74 0.77 4.03 0.87 8.09 .005 .08 attainmentb Note. N = 96. aGoal quality scored on a scale from 1 to 4. bRated on a scale from 1 to 5. University Technology Integration & of Victoria 22 Evaluation Research Lab
  • 23.
    Distribution of GoalQuality at Beginning and End of Semester 179 156 124 97 Weak/Moderate Good/Excellent Beginning End University Technology Integration & of Victoria 23 Evaluation Research Lab
  • 24.
    Q3: Were GoalQuality and Goal Perceptions Related to Term GPA? Term GPA r p Goal quality Beginning .40 <.001 End .37 <.001 Goal efficacy Beginning .24 .020 End .15 .152 Perceived difficulty Beginning -.02 .836 End .08 .425 Perceived attainment Beginning .26 .011 End .20 .046 Note. N = 96. University Technology Integration & of Victoria 24 Evaluation Research Lab
  • 25.
    DISCUSSION University Technology Integration & of Victoria 25 Evaluation Research Lab
  • 26.
    Supporting Goal Setting  Higher goal quality associated with higher term GPA  Students can learn to set high quality goals  Instruction in effective goal setting  Weekly practice  Evolving goal setting prompt University Technology Integration & of Victoria 26 Evaluation Research Lab
  • 27.
    Original Revised Prompt Prompt State one goal you have for improving For the task you selected above, set one your learning next week. goal for this week. Remember a good goal statement includes information about: what action, what course content or concepts, when (day, time, duration) & to what degree, amount, or standard. This week while studying for my psychology midterm I will make a mind map of the different fields of psychology. I will cover their main ideas, main For this week my goal is to create a conecpt map for psychologists, and other key terms. This will help me to Econ 205 making connections from chapters 8 visually see the differences so that I can compare and (monopolys) and 9 (oligopolies). Using the connections, contrast them during the test when the question comes I will formulate a paragraph summarizing the chapters up. I will do this Friday morning for 2 hours. The mind using at least 5 of the connections mentioned in the map will be colourful and contain pictures as well as concept map. I will do this goal from 7-8PM Thursday words. night. My goal for this week is to work on my math practice Friday from 10:30 to 12:30, I will analyze the different problems and readings for an hour and half every day in DC Motor Speed vs Torque plots obtained from the lab order to begin to understand how to use matrices when 3 data and be able to explain (in-depth) the effects of solving algebraic problems, and i wish further my using terminal voltage, field current, and series understanding of chapter 8 probability concepts. resistance in speed control of DC motors. University Technology Integration & of Victoria 27 Evaluation Research Lab
  • 28.
    Current Goal SettingPrompt University Technology Integration & of Victoria 28 Evaluation Research Lab
  • 29.
    THANK YOU! University Technology Integration & of Victoria 29 Evaluation Research Lab