CREATING EFFECTIVE
AND MEASURABLE
LEARNING OUTCOMES
Emily Magruder, Ph.D.
Director, Institute for Teaching and Learning
Opening Poll
Go to Café Learn Engagement Bootcamp.
>> Course Design
>> Measurable Learning Outcomes
Reflection Activity
• Did the results of the poll or any of the
examples surprise you?
• What can faculty do to keep teaching and
learning linked?
DESIGN
Approaches to Design
Content
Textbook
Outcomes
What we know about learning
1. Prior knowledge matters.
2. Novices are not experts.
3. Metacognition (thinking about thinking)
can help bridge the gap.
Bransford, Brown, Cocking, How People Learn (1999)
Backward Design
Types of Outcomes
•Cognitive
•Psychomotor
•Affective
Bloom’s Taxonomy:
Cognition
Dee Fink’s Framework:
Adds Affect for “Significant Learning”
Dee Fink and Associates
http://www.aacu.org/peerreview/pr-wi07/pr-wi07_analysis3.cfm
Preparing to Write Objectives
1. Ask dean: Are outcomes mandated by an accreditor?
By the institution? Can I modify them?
2. Research the history of the course: Why was it
proposed? What other courses should it prepare
students for? Who are the students?
3. Are there ILOs and PLOs?
4. What should students who have taken this course be
able to know/do/value at the end?
Effective Outcomes are “Mapped”:
Measurable
Upon completion of this course, students will be
able to [do something].
Begin with an action verb that describes
behavior that can be observed or evaluated.
Example: At the end of this course, students
will be able to deliver a carefully constructed
persuasive speech to a well-defined audience.
Effective Outcomes are “mApped”:
Appropriate
Outcomes should match what is taught and
the means of assessment.
Example: A multiple-choice quiz can
measure ability to recall a definition, but it
cannot measure ability to explain a concept.
Effective Outcomes are “mApped”:
Appropriate
• Level 3. Problem-Solving –
Evaluating and Creating
• Level 2. Interpretation –
Applying and Analyzing
• Level 1. Recall –
Remembering and
Understanding
Effective Outcomes are “maPped”:
Precise
Outcomes should specify the behavior, the
conditions under which it will be performed,
and the degree of mastery expected.
Examples
• On all written assignments and exams, students will use
correct grammar and spelling.
• Students will walk the entire length of a balance beam set
to standard height without falling off within a 15 second
time frame.
• Students will correctly write and balance chemistry
equations using chemical equations.
Backward Design
Ultimate
OutcomeMediating
Outcome
Foundational
Outcome
Practice
Write (or rewrite) an outcome for one lesson.
• Is it measurable?
• How will it be assessed?
• What foundation knowledge is needed?
• What process knowledge?
Wrapping It Up
• What’s one thing you learned today?
• What do you want to know more about?
• What’s one thing you will apply in your teaching?
“Learning results from what the
student does and thinks
and only from what the student
does and thinks.”
-Herbert Simon, 2001
References
Susan A. Ambrose, et al., How Learning Works: Seven
Research-Based Principles for Smart Teaching. Jossey-
Bass, 2010.
John D. Bransford, Ann L. Brown, & Rodney R. Cocking,
How People Learn: Brain, Mind, Experience and School.
National Academies Press, 2000.
Linda B. Nilson, Teaching at Its Best: A Research-Based
Resource for College Instructors. Jossey-Bass, 2010.

Effective and measurable learning outcomes

  • 1.
    CREATING EFFECTIVE AND MEASURABLE LEARNINGOUTCOMES Emily Magruder, Ph.D. Director, Institute for Teaching and Learning
  • 2.
    Opening Poll Go toCafé Learn Engagement Bootcamp. >> Course Design >> Measurable Learning Outcomes
  • 3.
    Reflection Activity • Didthe results of the poll or any of the examples surprise you? • What can faculty do to keep teaching and learning linked?
  • 4.
  • 5.
  • 6.
    What we knowabout learning 1. Prior knowledge matters. 2. Novices are not experts. 3. Metacognition (thinking about thinking) can help bridge the gap. Bransford, Brown, Cocking, How People Learn (1999)
  • 8.
  • 9.
  • 10.
  • 11.
    Dee Fink’s Framework: AddsAffect for “Significant Learning” Dee Fink and Associates http://www.aacu.org/peerreview/pr-wi07/pr-wi07_analysis3.cfm
  • 12.
    Preparing to WriteObjectives 1. Ask dean: Are outcomes mandated by an accreditor? By the institution? Can I modify them? 2. Research the history of the course: Why was it proposed? What other courses should it prepare students for? Who are the students? 3. Are there ILOs and PLOs? 4. What should students who have taken this course be able to know/do/value at the end?
  • 13.
    Effective Outcomes are“Mapped”: Measurable Upon completion of this course, students will be able to [do something]. Begin with an action verb that describes behavior that can be observed or evaluated. Example: At the end of this course, students will be able to deliver a carefully constructed persuasive speech to a well-defined audience.
  • 14.
    Effective Outcomes are“mApped”: Appropriate Outcomes should match what is taught and the means of assessment. Example: A multiple-choice quiz can measure ability to recall a definition, but it cannot measure ability to explain a concept.
  • 15.
    Effective Outcomes are“mApped”: Appropriate • Level 3. Problem-Solving – Evaluating and Creating • Level 2. Interpretation – Applying and Analyzing • Level 1. Recall – Remembering and Understanding
  • 16.
    Effective Outcomes are“maPped”: Precise Outcomes should specify the behavior, the conditions under which it will be performed, and the degree of mastery expected.
  • 17.
    Examples • On allwritten assignments and exams, students will use correct grammar and spelling. • Students will walk the entire length of a balance beam set to standard height without falling off within a 15 second time frame. • Students will correctly write and balance chemistry equations using chemical equations.
  • 18.
  • 19.
    Practice Write (or rewrite)an outcome for one lesson. • Is it measurable? • How will it be assessed? • What foundation knowledge is needed? • What process knowledge?
  • 20.
    Wrapping It Up •What’s one thing you learned today? • What do you want to know more about? • What’s one thing you will apply in your teaching?
  • 21.
    “Learning results fromwhat the student does and thinks and only from what the student does and thinks.” -Herbert Simon, 2001
  • 22.
    References Susan A. Ambrose,et al., How Learning Works: Seven Research-Based Principles for Smart Teaching. Jossey- Bass, 2010. John D. Bransford, Ann L. Brown, & Rodney R. Cocking, How People Learn: Brain, Mind, Experience and School. National Academies Press, 2000. Linda B. Nilson, Teaching at Its Best: A Research-Based Resource for College Instructors. Jossey-Bass, 2010.

Editor's Notes

  • #3 Let’s begin with an activity. “Learning can occur without teaching.” - Those who said TRUE: examples? Those who said FALSE: example? “Teaching can occur without learning.” - TRUE: examples? FALSE: examples? Linda Nilsen says that when learning occurs without teaching, NO HARM IS DONE … when teaching occurs without learning ….
  • #4 Take 2 minutes to reflect in writing on these two questions. Table Activity: Spend 2 minutes discussing what faculty can do to keep teaching and learning linked. Share: Appoint a spokesperson to share 1 thing your group discussed. Rule: you cannot share anything already contributed.
  • #5 Why do architects and engineers create blueprints? And details? What can we do to keep teaching and learning linked: DESIGN!
  • #7 If you are like me, you were trained to be an expert in your discipline. Were you trained to be an expert in teaching – which means knowing something about how people learn. Architects and engineers in California have to keep up with changing building codes – with each earthquake we learn more about the forces our physical structures have to withstand. Higher Ed can benefit from thinking like architects and engineers. As we learn more about how people learn, we must re-design our courses. Design opportunities for assessment: for students to know how they are doing and get feedback; for us to know how well we are doing so that we can continuously improve.
  • #8 I was first invited to think about the relationship between teaching and learning when I was a tenant organizer in New York City. Paulo Freire: “banking concept”: “knowledge is a gift bestowed by those who consider themselves knowledgeable upon those whom they consider know nothing.” Teacher knows everyting; students know nothing. Teacher teaches; students are taught. Teacher talks; students listen. Opposite of the “banking approach”: problem-posing education, teaching as practice of freedom.
  • #10 Cognitive: thinking Psychomotor: ability to manipulate objects Affective: involve emotion (receive, respond, value) * All of these things can be measured.
  • #11 Bloom as revised by Anderson and Kratwohl Based on hierarchy of cognitive processes Concrete/low level  abstract/high level Handy list of verbs. Point of education: get well beyond understand and remember to higher-order thinking. Students learn best when engaged in authentic activity. True cognitive level depends upon learning context. [Nilsen’s iambic pentameter example: turn this into an activity.]
  • #12 Cognition on the right side of the model; affect on the left. In contrast to Bloom’s taxonomy, which is hierarchical, Fink’s is cumulative and interactive. For Fink, when a course promotes all 6 kinds of learning, a “significant” experience occurs. Well-designed course positions learning and motivation as mutually reinforcing
  • #14 Action verbs: avoid “states of mind” like “understand” or “comprehend.” * Refer to one of the many lists that has been compiled of verbs corresponding to each level in Bloom’s taxonomy.
  • #15 Action verbs: avoid “states of mind” like “understand” or “comprehend.”
  • #16 Outcomes should be appropriate to the course’s level and purpose.
  • #17 Action verbs: avoid “states of mind” like “understand” or “comprehend.”
  • #18 Behavior? Conditions? Degree?
  • #19 What will the culminating assignment of your course – or of a module – be? Working backwards: What do students need to learn to do to be able to do that?
  • #20 Write an outcome. Then, in pairs, ask whether it is measurable, appropriate and precise. How will re-engineering outcomes transform your teaching – and your student’s learning?
  • #22 If we summed up 40 years of research on learning….