Learning Outcomes:
Blueprints for Teaching and Learning
Peter Newbury, Ph.D.
Director, Centre for Teaching and Learning, and
Senior Advisor for Learning Initiatives, UBC Okanagan
peter.newbury@ubc.ca peternewbury.org @polarisdotca
October 31, 2017
1
Goals Objectives
2
Flight Deck by
Wayan Volta
on flickr CC
Okay, but let’s look at
this checklist.
[trust and verify]
Do you understand
how to fly an airplane? Yes
Scholarly
approach to
teaching:
backwards
design
(Wiggins & McTighe)
3
What should
students
learn?learning
outcomes
assessment
that supports
learning
active
learning
What should
students
learn?
What are
students
learning?
What instructional
strategies
help students
learn?
(Wieman)
(NASA) 4
Introductory “Astro 101”
TRADITIONAL COURSE SYLLABUS
COURSE WITH
LEARNING OUTCOMES
5
This course covers Chapters
1. Mercury
2. Venus
…
8. Neptune
9. other objects
10. Formation of the Solar System
A. deduce from patterns in the
properties of the planets, moons,
asteroids and other bodies that the
Solar System had single formation
event.
B. provide notable examples of how
comets influenced history, art and
science
Introductory “Astro 101”
COURSE WITH
LEARNING OUTCOMES
6
 completes the phrase,
“By this end of this
lesson/unit/course, you
should be able to…”
A. deduce from patterns in the
properties of the planets, moons,
asteroids and other bodies that the
Solar System had single formation
event.
B. provide notable examples of how
comets influenced history, art and
science
 begins with an “action verb”
Learning outcomes are valuable to students
7
give big picture of the
next part of the course
reveal what the instructor is looking for: no more guessing
what “understand” means or what will be on the exam
allow students to check they’ve mastered
the concepts (especially when they’re studying)
allow students to monitor their
own progress (metacognition)
Learning outcomes are valuable to instructors
8
help the instructor select
instructional strategies and activities
crystallize what the instructor cares about
help the instructor create assessments
“help the instructor create assessments”
9
Astro 101 Final Exam
(10 marks) List 3 patterns of the Solar System as a
whole. Then, outline in some detail the current
model for the formation of the Solar System. In
particular, make sure you explain how the
observed patterns and regularities are related to
this theory of formation.
“help instructors select instructional activities”
10
ClassAction http://astro.unl.edu/classaction/
Does this peer instruction
(“clicker”) question support
the learning outcomes?
don’t
use it
use it
yesno
look for
another
question
look for
another
question
Woo-hoo!
move to
next topic!
A course should have
COURSE-LEVEL
LEARNING OUTCOMES
several outcomes giving big
picture, attitudes, behaviors:
“How you will be different by the
end of the course?”
likely can’t be assessed with a
single exam question
TOPIC-LEVEL
LEARNING OUTCOMES
many outcomes defining what it
means to “understand” at this
level (1st term, 4th term,
apprentice, etc.)
should be repeatedly assessed
in class, homework, exams
11
course-level and topic-level outcomes are related
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
Course-level LO #4
12
Course-level LO #2
Course-level LO #3Course-level LO #1
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
Topic-
level LO
Topic-
level LO
Topic-
level LO Topic-
level LO
Topic-
level LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
How are course-level and topic-level learning outcomes (LOs) related?
Can you see 2 rules or patterns? Discuss with others in your group.
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
Course-level LO #4
13
Course-level LO #2
Course-level LO #3Course-level LO #1
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
Topic-
level LO
Topic-
level LO
Topic-
level LO Topic-
level LO
Topic-
level LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
What’s wrong here?
(How do you fix it?)
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
Course-level LO #4
14
Course-level LO #2
Course-level LO #3Course-level LO #1
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
Topic-
level LO
Topic-
level LO
Topic-
level LO Topic-
level LO
Topic-
level LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
What’s wrong here?
(How do you fix it?)
Learning outcomes are valuable to programs
15
ensure program is
comprehensive, meets
accreditation standards
reduce risk of replicating content over and over
create logical, transparent
trajectory through program
identify concepts that need to be seen again and again
weave professional practices and
standards throughout program
Writing learning outcomes is hard
because you have to
16
recognize
(through your
expert blindness)
declare
(and be
accountable for)
what you want your students
to be capable of doing.
start
with the
action
verb
Bloom’s Taxonomy [3,4]
17
transform or combine ideas to create something new
think critically about and defend a position
break down concepts into parts
apply comprehension to unfamiliar situations
demonstrate understanding of ideas and concepts
remember and recall factual information
6 Create
5 Evaluate
4 Analyze
3 Apply
2 Understand
1 Remember
Bloom’s Taxonomy [3,4]
18
6 Create
5 Evaluate
4 Analyze
3 Apply
2 Understand
1 Remember
higher order thinking
lower order thinking
Bloom’s Taxonomy [3,4]
19
6 Create
5 Evaluate
4 Analyze
3 Apply
2 Understand
1 Remember
develop, create, propose, formulate, design, invent
judge, appraise, recommend, justify, defend, criticize, evaluate
compare, contrast, categorize, distinguish, identify, infer
apply, use, compute, solve, predict, construct, modify
describe, explain, summarize, interpret, illustrate
define, list, state, label, name, describe
20(NASA)
The wrong way
21
Astro 101 Learning Outcomes
A. understand how Saturn’s rings formed
Really?
 “understand”? That could be a sentence, Certificate, or degree.
 how does a student check they mastered the Saturn part of the course?
 how does a student demonstrate to you they “understand”
 what and how are you going to teach them about Saturn’s rings?
From course outline
22
Work your way through the list of topics and decide, “What do I want
students to be able to do, to demonstrate they ‘get’ this topic?”
Astro 101 Learning Outcomes - Saturn
1. give a detailed description of the size and structure of Saturn’s rings
2. trace the gravitational feedback cycle that keeps Saturn’s rings so thin
“Back-engineer” from exams
Use last year’s (or several years’) final exam. For each good question, ask
yourself
 What is the learning outcome I want students to demonstrate to
properly answer this question?
 Is this the outcome I want for my class?
23
24
What should
students
learn?learning
outcomes assessment
that supports
learning
active
learning
What should
students
learn?
What are
students
learning?
What instructional
strategies
help students
learn?
1. Write a learning
outcome that your
group’s question
assesses.
(Refer to Bloom’s
Taxonomy handout
for verbs.)
2. Think of a strategy
you could use to
teach it.

Driving School
(Wieman)
Bloom’s Taxonomy
25
6 Create
5 Evaluate
4 Analyze
3 Apply
2 Understand
1 Remember
develop, create, propose, formulate, design, invent
judge, appraise, recommend, justify, defend, criticize, evaluate
compare, contrast, categorize, distinguish, identify, infer
apply, use, compute, solve, predict, construct, modify
describe, explain, summarize, interpret, illustrate
define, list, state, label, name, describe
1. Write a learning outcome that your
group’s question assesses.
2. Think of a strategy you could use to
teach it.
Sample Class C Written Test
26
(California DMV)
Sample Class C Written Test
27
(California DMV)
Sample Class C Written Test
28
(California DMV)
Sample Class C Written Test
29
(California DMV)
Sample Class C Written Test
30
(California DMV)
Sample Class C Written Test
31
(California DMV)
Sample Class C Written Test
32
(California DMV)
Sample Class C Written Test
33
(California DMV)
Sample Class C Written Test
34
(California DMV)
Sample Class C Written Test
35
(California DMV)
9
8
36
3 6
10
7
5
4
2
1
recognize, interpret, and
obey all signs and signals
assess risk and select
actions to minimize harm
9
8
37
assess risk and select
actions to minimize harm
recognize, interpret, and
obey all signs and signals
3 6
10
7
5
4
2
1
safely operate a
motor vehicle
9
8
38
assess risk and select
actions to minimize harm
safely operate a
motor vehicle
recognize, interpret, and
obey all signs and signals
3 6
10
7
5
4
2
1
Topic-level
LOTopic-level
LO
Topic-level
LO
Topic-
level LO
Topic-
level LO Topic-
level LO
Topic-level
LO
Topic-
level LO
coffee break
39
Learning Outcomes:
Blueprints for Teaching and Learning
40
Part 2: Your Turn
1. Form groups of 2-3.
It really helps if you’re in similar disciplines and/or teach similar
courses because you have rich conceptual frameworks to share.
41
2. Identify one concept or skill you all agree is important.
3. Write a topic-level learning outcome for that concept or skill.
(Refer to Bloom’s Taxonomy hand-out of action verbs.)
4. Think about how you’d assess it and how you’d teach it.
5. Be prepared to share your outcome, and give feedback to others.
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
Course-level LO #4
42
Course-level LO #2
Course-level LO #3Course-level LO #1
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
Topic-
level LO
Topic-
level LO
Topic-
level LO Topic-
level LO
Topic-
level LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
Learning outcomes:
43
program
course
topic
44
1st term
Ensures safety of
personnel on board
Trains in safety, firefighting
and search and rescue Ensures safety of
ship's cargo
Manages the deck
department and crew
Chief Mate
2nd term
3rd term
Ensures ship's
maintenance
45
1st term
Ensures safety of
personnel on board
Ensures safety of
ship's cargo
Manages the deck
department and crew
Chief Mate
2nd term
3rd term
Ensures ship's
maintenance
Trains in safety, firefighting
and search and rescue
46
1st term
Ensures safety of
personnel on board
Ensures safety of
ship's cargo
Manages the deck
department and crew
Chief Mate
2nd term
3rd term
Ensures ship's
maintenance
Trains in safety, firefighting
and search and rescue
47
1st term
leadership
team work inclusion
career-ready
Airport Operations
2nd term
3rd term
accreditation
4th term
48
maintains
reconditions
rebuilds
overhauls
Transport Trailer
Technician
does diagnostic
troubleshooting
motorized commercial truck,
bus, and road transport
equipment
Industry Training Authority itabc.ca
Learning
Outcomes:
Blueprints for
Teaching and
Learning
Peter Newbury
peter.newbury@ubc.ca
49
What should
students
learn?
What should
students
learn?
What are
students
learning?
What instructional
strategies
help students
learn?
References
50
1. Wiggins, G. P., & McTighe, J. (2005). Understanding by design.
2. Excerpt from Wieman, C. (2007). Slides from the Wieman Learning Goals Workshop.
www.cwsei.ubc.ca/resources/learn_goals.htm
3. Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New
York: David McKay Co Inc.
4. Revised Bloom’s Taxonomy www.celt.iastate.edu/teaching/RevisedBlooms1.html
5. California DMV Sample Class C Written Test 5
www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm

Learning Outcomes: Blueprints for Teaching and Learning

  • 1.
    Learning Outcomes: Blueprints forTeaching and Learning Peter Newbury, Ph.D. Director, Centre for Teaching and Learning, and Senior Advisor for Learning Initiatives, UBC Okanagan peter.newbury@ubc.ca peternewbury.org @polarisdotca October 31, 2017 1 Goals Objectives
  • 2.
    2 Flight Deck by WayanVolta on flickr CC Okay, but let’s look at this checklist. [trust and verify] Do you understand how to fly an airplane? Yes
  • 3.
    Scholarly approach to teaching: backwards design (Wiggins &McTighe) 3 What should students learn?learning outcomes assessment that supports learning active learning What should students learn? What are students learning? What instructional strategies help students learn? (Wieman)
  • 4.
  • 5.
    Introductory “Astro 101” TRADITIONALCOURSE SYLLABUS COURSE WITH LEARNING OUTCOMES 5 This course covers Chapters 1. Mercury 2. Venus … 8. Neptune 9. other objects 10. Formation of the Solar System A. deduce from patterns in the properties of the planets, moons, asteroids and other bodies that the Solar System had single formation event. B. provide notable examples of how comets influenced history, art and science
  • 6.
    Introductory “Astro 101” COURSEWITH LEARNING OUTCOMES 6  completes the phrase, “By this end of this lesson/unit/course, you should be able to…” A. deduce from patterns in the properties of the planets, moons, asteroids and other bodies that the Solar System had single formation event. B. provide notable examples of how comets influenced history, art and science  begins with an “action verb”
  • 7.
    Learning outcomes arevaluable to students 7 give big picture of the next part of the course reveal what the instructor is looking for: no more guessing what “understand” means or what will be on the exam allow students to check they’ve mastered the concepts (especially when they’re studying) allow students to monitor their own progress (metacognition)
  • 8.
    Learning outcomes arevaluable to instructors 8 help the instructor select instructional strategies and activities crystallize what the instructor cares about help the instructor create assessments
  • 9.
    “help the instructorcreate assessments” 9 Astro 101 Final Exam (10 marks) List 3 patterns of the Solar System as a whole. Then, outline in some detail the current model for the formation of the Solar System. In particular, make sure you explain how the observed patterns and regularities are related to this theory of formation.
  • 10.
    “help instructors selectinstructional activities” 10 ClassAction http://astro.unl.edu/classaction/ Does this peer instruction (“clicker”) question support the learning outcomes? don’t use it use it yesno look for another question look for another question Woo-hoo! move to next topic!
  • 11.
    A course shouldhave COURSE-LEVEL LEARNING OUTCOMES several outcomes giving big picture, attitudes, behaviors: “How you will be different by the end of the course?” likely can’t be assessed with a single exam question TOPIC-LEVEL LEARNING OUTCOMES many outcomes defining what it means to “understand” at this level (1st term, 4th term, apprentice, etc.) should be repeatedly assessed in class, homework, exams 11 course-level and topic-level outcomes are related
  • 12.
    Topic-level LO Topic-level LO Topic-level LO Topic- level LO Course-level LO#4 12 Course-level LO #2 Course-level LO #3Course-level LO #1 Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic-level LO Topic-level LO Topic- level LO How are course-level and topic-level learning outcomes (LOs) related? Can you see 2 rules or patterns? Discuss with others in your group.
  • 13.
    Topic-level LO Topic-level LO Topic-level LO Topic- level LO Course-level LO#4 13 Course-level LO #2 Course-level LO #3Course-level LO #1 Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic-level LO Topic-level LO Topic- level LO What’s wrong here? (How do you fix it?)
  • 14.
    Topic-level LO Topic-level LO Topic-level LO Topic- level LO Course-level LO#4 14 Course-level LO #2 Course-level LO #3Course-level LO #1 Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic-level LO Topic-level LO Topic- level LO What’s wrong here? (How do you fix it?)
  • 15.
    Learning outcomes arevaluable to programs 15 ensure program is comprehensive, meets accreditation standards reduce risk of replicating content over and over create logical, transparent trajectory through program identify concepts that need to be seen again and again weave professional practices and standards throughout program
  • 16.
    Writing learning outcomesis hard because you have to 16 recognize (through your expert blindness) declare (and be accountable for) what you want your students to be capable of doing. start with the action verb
  • 17.
    Bloom’s Taxonomy [3,4] 17 transformor combine ideas to create something new think critically about and defend a position break down concepts into parts apply comprehension to unfamiliar situations demonstrate understanding of ideas and concepts remember and recall factual information 6 Create 5 Evaluate 4 Analyze 3 Apply 2 Understand 1 Remember
  • 18.
    Bloom’s Taxonomy [3,4] 18 6Create 5 Evaluate 4 Analyze 3 Apply 2 Understand 1 Remember higher order thinking lower order thinking
  • 19.
    Bloom’s Taxonomy [3,4] 19 6Create 5 Evaluate 4 Analyze 3 Apply 2 Understand 1 Remember develop, create, propose, formulate, design, invent judge, appraise, recommend, justify, defend, criticize, evaluate compare, contrast, categorize, distinguish, identify, infer apply, use, compute, solve, predict, construct, modify describe, explain, summarize, interpret, illustrate define, list, state, label, name, describe
  • 20.
  • 21.
    The wrong way 21 Astro101 Learning Outcomes A. understand how Saturn’s rings formed Really?  “understand”? That could be a sentence, Certificate, or degree.  how does a student check they mastered the Saturn part of the course?  how does a student demonstrate to you they “understand”  what and how are you going to teach them about Saturn’s rings?
  • 22.
    From course outline 22 Workyour way through the list of topics and decide, “What do I want students to be able to do, to demonstrate they ‘get’ this topic?” Astro 101 Learning Outcomes - Saturn 1. give a detailed description of the size and structure of Saturn’s rings 2. trace the gravitational feedback cycle that keeps Saturn’s rings so thin
  • 23.
    “Back-engineer” from exams Uselast year’s (or several years’) final exam. For each good question, ask yourself  What is the learning outcome I want students to demonstrate to properly answer this question?  Is this the outcome I want for my class? 23
  • 24.
    24 What should students learn?learning outcomes assessment thatsupports learning active learning What should students learn? What are students learning? What instructional strategies help students learn? 1. Write a learning outcome that your group’s question assesses. (Refer to Bloom’s Taxonomy handout for verbs.) 2. Think of a strategy you could use to teach it.  Driving School (Wieman)
  • 25.
    Bloom’s Taxonomy 25 6 Create 5Evaluate 4 Analyze 3 Apply 2 Understand 1 Remember develop, create, propose, formulate, design, invent judge, appraise, recommend, justify, defend, criticize, evaluate compare, contrast, categorize, distinguish, identify, infer apply, use, compute, solve, predict, construct, modify describe, explain, summarize, interpret, illustrate define, list, state, label, name, describe 1. Write a learning outcome that your group’s question assesses. 2. Think of a strategy you could use to teach it.
  • 26.
    Sample Class CWritten Test 26 (California DMV)
  • 27.
    Sample Class CWritten Test 27 (California DMV)
  • 28.
    Sample Class CWritten Test 28 (California DMV)
  • 29.
    Sample Class CWritten Test 29 (California DMV)
  • 30.
    Sample Class CWritten Test 30 (California DMV)
  • 31.
    Sample Class CWritten Test 31 (California DMV)
  • 32.
    Sample Class CWritten Test 32 (California DMV)
  • 33.
    Sample Class CWritten Test 33 (California DMV)
  • 34.
    Sample Class CWritten Test 34 (California DMV)
  • 35.
    Sample Class CWritten Test 35 (California DMV)
  • 36.
    9 8 36 3 6 10 7 5 4 2 1 recognize, interpret,and obey all signs and signals assess risk and select actions to minimize harm
  • 37.
    9 8 37 assess risk andselect actions to minimize harm recognize, interpret, and obey all signs and signals 3 6 10 7 5 4 2 1 safely operate a motor vehicle
  • 38.
    9 8 38 assess risk andselect actions to minimize harm safely operate a motor vehicle recognize, interpret, and obey all signs and signals 3 6 10 7 5 4 2 1 Topic-level LOTopic-level LO Topic-level LO Topic- level LO Topic- level LO Topic- level LO Topic-level LO Topic- level LO
  • 39.
  • 40.
    Learning Outcomes: Blueprints forTeaching and Learning 40 Part 2: Your Turn
  • 41.
    1. Form groupsof 2-3. It really helps if you’re in similar disciplines and/or teach similar courses because you have rich conceptual frameworks to share. 41 2. Identify one concept or skill you all agree is important. 3. Write a topic-level learning outcome for that concept or skill. (Refer to Bloom’s Taxonomy hand-out of action verbs.) 4. Think about how you’d assess it and how you’d teach it. 5. Be prepared to share your outcome, and give feedback to others.
  • 42.
    Topic-level LO Topic-level LO Topic-level LO Topic- level LO Course-level LO#4 42 Course-level LO #2 Course-level LO #3Course-level LO #1 Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic-level LO Topic-level LO Topic- level LO
  • 43.
  • 44.
    44 1st term Ensures safetyof personnel on board Trains in safety, firefighting and search and rescue Ensures safety of ship's cargo Manages the deck department and crew Chief Mate 2nd term 3rd term Ensures ship's maintenance
  • 45.
    45 1st term Ensures safetyof personnel on board Ensures safety of ship's cargo Manages the deck department and crew Chief Mate 2nd term 3rd term Ensures ship's maintenance Trains in safety, firefighting and search and rescue
  • 46.
    46 1st term Ensures safetyof personnel on board Ensures safety of ship's cargo Manages the deck department and crew Chief Mate 2nd term 3rd term Ensures ship's maintenance Trains in safety, firefighting and search and rescue
  • 47.
    47 1st term leadership team workinclusion career-ready Airport Operations 2nd term 3rd term accreditation 4th term
  • 48.
    48 maintains reconditions rebuilds overhauls Transport Trailer Technician does diagnostic troubleshooting motorizedcommercial truck, bus, and road transport equipment Industry Training Authority itabc.ca
  • 49.
    Learning Outcomes: Blueprints for Teaching and Learning PeterNewbury peter.newbury@ubc.ca 49 What should students learn? What should students learn? What are students learning? What instructional strategies help students learn?
  • 50.
    References 50 1. Wiggins, G.P., & McTighe, J. (2005). Understanding by design. 2. Excerpt from Wieman, C. (2007). Slides from the Wieman Learning Goals Workshop. www.cwsei.ubc.ca/resources/learn_goals.htm 3. Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc. 4. Revised Bloom’s Taxonomy www.celt.iastate.edu/teaching/RevisedBlooms1.html 5. California DMV Sample Class C Written Test 5 www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm