Learning Outcomes: Blueprints for Teaching and Learning

1. Learning Outcomes: Blueprints for Teaching and Learning Peter Newbury, Ph.D. Director, Centre for Teaching and Learning, and Senior Advisor for Learning Initiatives, UBC Okanagan peter.newbury@ubc.ca peternewbury.org @polarisdotca October 31, 2017 1 Goals Objectives

2. 2 Flight Deck by Wayan Volta on flickr CC Okay, but let’s look at this checklist. [trust and verify] Do you understand how to fly an airplane? Yes

3. Scholarly approach to teaching: backwards design (Wiggins & McTighe) 3 What should students learn?learning outcomes assessment that supports learning active learning What should students learn? What are students learning? What instructional strategies help students learn? (Wieman)

4. (NASA) 4

5. Introductory “Astro 101” TRADITIONAL COURSE SYLLABUS COURSE WITH LEARNING OUTCOMES 5 This course covers Chapters 1. Mercury 2. Venus … 8. Neptune 9. other objects 10. Formation of the Solar System A. deduce from patterns in the properties of the planets, moons, asteroids and other bodies that the Solar System had single formation event. B. provide notable examples of how comets influenced history, art and science

6. Introductory “Astro 101” COURSE WITH LEARNING OUTCOMES 6  completes the phrase, “By this end of this lesson/unit/course, you should be able to…” A. deduce from patterns in the properties of the planets, moons, asteroids and other bodies that the Solar System had single formation event. B. provide notable examples of how comets influenced history, art and science  begins with an “action verb”

7. Learning outcomes are valuable to students 7 give big picture of the next part of the course reveal what the instructor is looking for: no more guessing what “understand” means or what will be on the exam allow students to check they’ve mastered the concepts (especially when they’re studying) allow students to monitor their own progress (metacognition)

8. Learning outcomes are valuable to instructors 8 help the instructor select instructional strategies and activities crystallize what the instructor cares about help the instructor create assessments

9. “help the instructor create assessments” 9 Astro 101 Final Exam (10 marks) List 3 patterns of the Solar System as a whole. Then, outline in some detail the current model for the formation of the Solar System. In particular, make sure you explain how the observed patterns and regularities are related to this theory of formation.

10. “help instructors select instructional activities” 10 ClassAction http://astro.unl.edu/classaction/ Does this peer instruction (“clicker”) question support the learning outcomes? don’t use it use it yesno look for another question look for another question Woo-hoo! move to next topic!

11. A course should have COURSE-LEVEL LEARNING OUTCOMES several outcomes giving big picture, attitudes, behaviors: “How you will be different by the end of the course?” likely can’t be assessed with a single exam question TOPIC-LEVEL LEARNING OUTCOMES many outcomes defining what it means to “understand” at this level (1st term, 4th term, apprentice, etc.) should be repeatedly assessed in class, homework, exams 11 course-level and topic-level outcomes are related

12. Topic-level LO Topic-level LO Topic-level LO Topic- level LO Course-level LO #4 12 Course-level LO #2 Course-level LO #3Course-level LO #1 Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic-level LO Topic-level LO Topic- level LO How are course-level and topic-level learning outcomes (LOs) related? Can you see 2 rules or patterns? Discuss with others in your group.

13. Topic-level LO Topic-level LO Topic-level LO Topic- level LO Course-level LO #4 13 Course-level LO #2 Course-level LO #3Course-level LO #1 Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic-level LO Topic-level LO Topic- level LO What’s wrong here? (How do you fix it?)

14. Topic-level LO Topic-level LO Topic-level LO Topic- level LO Course-level LO #4 14 Course-level LO #2 Course-level LO #3Course-level LO #1 Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic-level LO Topic-level LO Topic- level LO What’s wrong here? (How do you fix it?)

15. Learning outcomes are valuable to programs 15 ensure program is comprehensive, meets accreditation standards reduce risk of replicating content over and over create logical, transparent trajectory through program identify concepts that need to be seen again and again weave professional practices and standards throughout program

16. Writing learning outcomes is hard because you have to 16 recognize (through your expert blindness) declare (and be accountable for) what you want your students to be capable of doing. start with the action verb

17. Bloom’s Taxonomy [3,4] 17 transform or combine ideas to create something new think critically about and defend a position break down concepts into parts apply comprehension to unfamiliar situations demonstrate understanding of ideas and concepts remember and recall factual information 6 Create 5 Evaluate 4 Analyze 3 Apply 2 Understand 1 Remember

18. Bloom’s Taxonomy [3,4] 18 6 Create 5 Evaluate 4 Analyze 3 Apply 2 Understand 1 Remember higher order thinking lower order thinking

19. Bloom’s Taxonomy [3,4] 19 6 Create 5 Evaluate 4 Analyze 3 Apply 2 Understand 1 Remember develop, create, propose, formulate, design, invent judge, appraise, recommend, justify, defend, criticize, evaluate compare, contrast, categorize, distinguish, identify, infer apply, use, compute, solve, predict, construct, modify describe, explain, summarize, interpret, illustrate define, list, state, label, name, describe

20. 20(NASA)

21. The wrong way 21 Astro 101 Learning Outcomes A. understand how Saturn’s rings formed Really?  “understand”? That could be a sentence, Certificate, or degree.  how does a student check they mastered the Saturn part of the course?  how does a student demonstrate to you they “understand”  what and how are you going to teach them about Saturn’s rings?

22. From course outline 22 Work your way through the list of topics and decide, “What do I want students to be able to do, to demonstrate they ‘get’ this topic?” Astro 101 Learning Outcomes - Saturn 1. give a detailed description of the size and structure of Saturn’s rings 2. trace the gravitational feedback cycle that keeps Saturn’s rings so thin

23. “Back-engineer” from exams Use last year’s (or several years’) final exam. For each good question, ask yourself  What is the learning outcome I want students to demonstrate to properly answer this question?  Is this the outcome I want for my class? 23

24. 24 What should students learn?learning outcomes assessment that supports learning active learning What should students learn? What are students learning? What instructional strategies help students learn? 1. Write a learning outcome that your group’s question assesses. (Refer to Bloom’s Taxonomy handout for verbs.) 2. Think of a strategy you could use to teach it.  Driving School (Wieman)

25. Bloom’s Taxonomy 25 6 Create 5 Evaluate 4 Analyze 3 Apply 2 Understand 1 Remember develop, create, propose, formulate, design, invent judge, appraise, recommend, justify, defend, criticize, evaluate compare, contrast, categorize, distinguish, identify, infer apply, use, compute, solve, predict, construct, modify describe, explain, summarize, interpret, illustrate define, list, state, label, name, describe 1. Write a learning outcome that your group’s question assesses. 2. Think of a strategy you could use to teach it.

26. Sample Class C Written Test 26 (California DMV)

36. 9 8 36 3 6 10 7 5 4 2 1 recognize, interpret, and obey all signs and signals assess risk and select actions to minimize harm

37. 9 8 37 assess risk and select actions to minimize harm recognize, interpret, and obey all signs and signals 3 6 10 7 5 4 2 1 safely operate a motor vehicle

38. 9 8 38 assess risk and select actions to minimize harm safely operate a motor vehicle recognize, interpret, and obey all signs and signals 3 6 10 7 5 4 2 1 Topic-level LOTopic-level LO Topic-level LO Topic- level LO Topic- level LO Topic- level LO Topic-level LO Topic- level LO

39. coffee break 39

40. Learning Outcomes: Blueprints for Teaching and Learning 40 Part 2: Your Turn

41. 1. Form groups of 2-3. It really helps if you’re in similar disciplines and/or teach similar courses because you have rich conceptual frameworks to share. 41 2. Identify one concept or skill you all agree is important. 3. Write a topic-level learning outcome for that concept or skill. (Refer to Bloom’s Taxonomy hand-out of action verbs.) 4. Think about how you’d assess it and how you’d teach it. 5. Be prepared to share your outcome, and give feedback to others.

42. Topic-level LO Topic-level LO Topic-level LO Topic- level LO Course-level LO #4 42 Course-level LO #2 Course-level LO #3Course-level LO #1 Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic-level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic- level LO Topic-level LO Topic-level LO Topic- level LO

43. Learning outcomes: 43 program course topic

44. 44 1st term Ensures safety of personnel on board Trains in safety, firefighting and search and rescue Ensures safety of ship's cargo Manages the deck department and crew Chief Mate 2nd term 3rd term Ensures ship's maintenance

45. 45 1st term Ensures safety of personnel on board Ensures safety of ship's cargo Manages the deck department and crew Chief Mate 2nd term 3rd term Ensures ship's maintenance Trains in safety, firefighting and search and rescue

46. 46 1st term Ensures safety of personnel on board Ensures safety of ship's cargo Manages the deck department and crew Chief Mate 2nd term 3rd term Ensures ship's maintenance Trains in safety, firefighting and search and rescue

47. 47 1st term leadership team work inclusion career-ready Airport Operations 2nd term 3rd term accreditation 4th term

48. 48 maintains reconditions rebuilds overhauls Transport Trailer Technician does diagnostic troubleshooting motorized commercial truck, bus, and road transport equipment Industry Training Authority itabc.ca

49. Learning Outcomes: Blueprints for Teaching and Learning Peter Newbury peter.newbury@ubc.ca 49 What should students learn? What should students learn? What are students learning? What instructional strategies help students learn?

50. References 50 1. Wiggins, G. P., & McTighe, J. (2005). Understanding by design. 2. Excerpt from Wieman, C. (2007). Slides from the Wieman Learning Goals Workshop. www.cwsei.ubc.ca/resources/learn_goals.htm 3. Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc. 4. Revised Bloom’s Taxonomy www.celt.iastate.edu/teaching/RevisedBlooms1.html 5. California DMV Sample Class C Written Test 5 www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm