Learning Outcomes Peter Newbury Center for Teaching Development, UCSD ctd.ucsd.edu 27 November 2013

1. slides and resources: ctd.ucsd.edu/programs/fall-2013-weekly-workshops/
CTD WEEKLY WORKSHOPS:
LEARNING OUTCOMES
Peter Newbury
Center for Teaching Development,
University of California, San Diego
pnewbury@ucsd.edu
@polarisdotca
ctd.ucsd.edu
#ctducsd
Wednesday, November 27, 2013
12:00 – 12:50 pm Center Hall, Room 316

2. Scholarly approach to teaching:
What should
students
learn?
learning
outcomes
(goals, objectives)
What are
students
learning?
What instructional
approaches
help students
learn?
Carl Wieman
Science Education Initiative
cwsei.ubc.ca
2
Learning Outcomes
assessment
(Dec 4)
alt to lecture
(Oct 30)
peer instruction,
(Nov 13)

3. (Image: NASA)
3
Learning Outcomes

4. Introductory “Astro 101”
Traditional Course Syllabus
Course with Learning Outcomes
This course covers Chapters
 deduce from patterns in the
properties of the planets, moons,
asteroids and other bodies that the
Solar System had single formation
event.
1.
Mercury
2.
Venus
…
8.
Neptune
9.
other objects
10. Formation of the Solar System
 reconstruct the formation and
evolution of various bodies in the
Solar System by interpreting the
presence (and their appearance)
or absence of craters
 provide notable examples of how
comets influenced history, art and
science
4
Learning Outcomes

5. Learning outcomes
 deduce from patterns in the
properties of the planets, moons,
asteroids and other bodies that the
Solar System had single formation
event.
 completes the sentence, “By this end of this
lesson/unit/course, you will be able to…”
 begins with an action verb (“deduce”) (more below)
 tells the students what they must do to demonstrate
they “understand” the concept
5
Learning Outcomes

6. What is the Value of Course-Specific
Learning Outcomes?
Simon & Taylor [1] asked students to complete this
sentence:
For me, the use of learning goals in this course is…
They received 597 responses from students in computer
science and microbiology. Responses were put into
categories that emerged from the responses.
6
Learning Outcomes

7. 7
Learning Outcomes
Simon & Taylor (2009)

8. Learning outcomes are valuable to…
 the students



reveals what the instructor is looking for (no guessing
what “understand” means.)
big picture of the next part of the course
allows student to check that s/he has mastered the
concept (especially when studying later)
 the instructor


crystallizes what the instructor actually cares about
helps the instructor


8
choose clicker questions for peer instruction in class
write the final exam
Learning Outcomes

9. …choose clicker questions for PI
9
Learning Outcomes
ClassAction http://astro.unl.edu/classaction/

10. …write the final exam
(10 marks) List 3 patterns of the Solar System as a
whole. Then, outline in some detail the current model for
the formation of the Solar System. In particular, make
sure you explain how the observed patterns and
regularities are related to this theory of formation.
10
Learning Outcomes

11. Course-level LOs
11
Topic-level LOs
several LOs giving big
picture, attitudes,
behaviors
many LOs defining
what it means to
“understand” at this
level (freshman, etc.)
can be (should be)
repeatedly assessed on
HW, exams
 support one or more
course-level LOs
(if not, why not?)
(likely) can’t be
assessed with a single
exam question
supported by many
topic-level LOs
(if not, why not?)
Learning Outcomes

12. Course-level
learning outcome (LO) #1
Course-level LO #3
Course-level LO #2
Topic-level
TopicLO
Topic-level
Topic-level level LO
Topic-level
LO
LO
LO
Topic-level
TopicTopic-level
Topic-level
LO
level
learning outcomeLO TopicLO
level LO
TopicTopicTopic- Topic-level
level LO
level LO
level LO
LO
12
Learning Outcomes
sync your LOs
Course-level LO #4
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topiclevel LO
Topiclevel LO
see ASTR 310 handout

13. Writing topic-level LOs
Writing learning outcomes is hard because you have to
 recognize
 declare
 (admit)
what you want your students to be capable of doing.
A good start is picking the verb describing the action
the students will perform to demonstrate their mastery
of the concept.
13
Learning Outcomes

14. Bloom’s Taxonomy [2,3]
transform or combine ideas to create
something new
6
Create
5
Evaluate
think critically about and defend a position
4
Analyze
break down concepts into parts
3
Apply
apply comprehension to unfamiliar situations
2
1
14
Understand
demonstrate understanding of ideas
and concepts
Remember
remember and recall factual information
Learning Outcomes

15. Bloom’s Taxonomy [2,3]
6
5
Evaluate
4
Analyze
3
Apply
2
Understand
1
15
Create
Remember
Learning Outcomes
higher order thinking
lower order thinking

16. Bloom’s Taxonomy [2,3]
Create
develop, create, propose, formulate,
design, invent
Evaluate
judge, appraise, recommend, justify,
defend, criticize, evaluate
4
Analyze
compare, contrast, categorize, distinguish,
identify, infer
3
Apply
apply, demonstrate, use, compute, solve,
predict, construct, modify
2
Understand
describe, explain, summarize, interpret,
illustrate
1
Remember
define, list, state, label, name, describe
6
5
16
Learning Outcomes

17. Driver’s Ed 101: How to Drive in CA
The whiteboards on the tables are numbered. Your
group will concentrate on the DMV Test Question
matching your board’s number.
Task: Write a learning outcome
that your group’s question assesses.
(refer to Wieman handout
for Bloom’s Taxonomy verbs)
17
Learning Outcomes

18. 1. “Back-engineer” LOs from exams
Use last year’s (or several years’) final exam. For each
good question, ask yourself
 What is this question assessing? What is the learning
outcome I want students to demonstrate to properly
answer this question?
 Is that the outcome I want, or is it too low (or high)?
When you have a list of LOs,
 Does it cover everything I want for this course?
 Have I over- or under-represented any concepts?
18
Learning Outcomes

19. 2. Draft LOs from course outline
Work your way through the list of topics. For each topic,
decide
 What do I want students to be able to do, to
demonstrate they “get” this topic?
 Don’t worry about drafting many low-level LOs.
When you revise, you’ll start grouping them into
higher-level LOs.
(for example, see ASTR 310 Learning Goals)
19
Learning Outcomes

20. Share your LOs with your students
 After you’ve gone to the trouble of creating learning
outcomes, share them with your students


(good) publish them as a document along side your syllabus
(better) include relevant learning goals in your lecture slides
at the beginning of each topic, even each class.
 Be wary of reading them aloud: the students may not yet
have the knowledge to appreciate the LOs. The LOs will
be there when they study.
 Don’t worry about “spoon-feeding” them – so what if
students do exactly what you feel demonstrates
understanding?
20
Learning Outcomes

21. Scholarly approach to teaching:
What should
students
learn?
learning
outcomes
(goals, objectives)
What are
students
learning?
What instructional
approaches
help students
learn?
Carl Wieman
Science Education Initiative
cwsei.ubc.ca
21
Learning Outcomes
assessment
(Dec 4)
alt to lecture
(Oct 30)
peer instruction,
(Nov 13)

22. References
1.
Simon, B., & Taylor, J. (2009). What is the Value of Course-Specific Learning Goals? Journal
of College Science Teaching, 39, 2, 52-57. PDF available at
www.cwsei.ubc.ca/SEI_research/files/LifeSci/Simon_Taylor_ValueOfCourseSpecificLG.pdf
2.
Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain.
New York: David McKay Co Inc.
Adapted from Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for Learning.
Teaching, and assessing: A revision of bloom's taxonomy of educational objectives.
Revised Bloom’s Taxonomy www.celt.iastate.edu/teaching/RevisedBlooms1.html
3.
4.
22
Excerpt from Wieman, C. (2007). Slides from the Wieman Learning Goals Workshop.
www.cwsei.ubc.ca/resources/learn_goals.htm
California DMV Sample Class C Written Test 5
www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm
Learning Outcomes

23. Bloom’s Taxonomy of the Cognitive Domain
(Levels of Learning)
6 Create: transform and combine ideas to create something new
develop, create, propose, formulate, design, invent
5 Evaluate: think critically about and defend a position
judge, appraise, recommend, justify, defend, criticize, evaluate
4 Analyze:: break down concepts into parts
compare, contrast, categorize, distinguish, identify, infer
3 Apply: apply comprehension to unfamiliar situations
apply, demonstrate, use, compute, solve, predict, construct, modify
2 Understand: demonstrate understanding of ideas, concepts
describe, explain, summarize, interpret, illustrate
1 Remember: remember and recall factual knowledge
define, list, state, label, name, describe
23
Learning Outcomes
Adapted from Carl Wieman (2007)
www.cwsei.ubc.ca/resources/learn_goals.htm