While the departments designed the proposals, the SEI project constrained the sorts of activities the funding could support. SEI allowed work focusing on course transformation, targeting 3 areas. First, identifying what students should learn? This involves establishing well-defined, faculty-consensus learning goals for courses … that is, identifying what students should be able to *do* at the end of a course. The process often involves successive meetings of faculty working groups and faculty interviews. The second area involves examining what students are actually learning using research and assessment methodologies, including student interviews, observations, and surveys. Information from these 2 areas are then used to inform identifying which instructional approaches improve student learning – specifically achievement of the key learning goals identified by faculty. Here faculty incorporate new proven pedagogical approaches such as interactive lectures, modified HW and group learning. A key component is measuring progress – that is measuring achievement of the learning goals, and using the information on student learning to inform future improvements! Faculty time is already squeezed and so the SEI funding provides the resources to hire the Science Teaching Fellows who partner with faculty on this effort.
Some of your objectives may be content-independent (ie, skills!)
Discussion – are exam questions addressing range of goals? Are learning goals that narrow? Or are assessments not reflecting the goals?
Discussion – are exam questions addressing range of goals? Are learning goals that narrow? Or are assessments not reflecting the goals?
Summer = course scale.Weekly = topic scale. Lecture scale = selected from topic scale. Put on slide. Minimize content for that day.
Model each one of these. What are some ways to ask questions? One is to ask rhetorically.Class, what’s another way to ask a question? Target the whole class.John, what’s another way? Target someone else.Are there other ways to ask a question? Let’s think about it. Target class: verbally, clickers, other waysTarget someone in particular: randomly, in seating order, call on particular personWait and then…. Call on volunteers, call on someone who hasn’t volunteered, answer own question
During each section, ask people for examples of questions that they wrote that fall into this category. Give clicker booklet for responding.Point out the handout where each one is detailed more.
One of themes of this year’s workshop: implementation
What do you want them to learn today? Learning goals and formative assessment
What do you want them to learn today? WRITING LEARNING GOALS TO DRIVE INSTRUCTION & ASSESSMENT Dr. Stephanie V. Chasteen Dr. Kathy Perkins Physics Department & Science Education Initiative Univ. of Colorado at Boulder http://colorado.edu/seiWeb and blog: http://sciencegeekgirl.comEmail: firstname.lastname@example.org
At the end of this workshopYou will be able to… Develop and communicate your learning goals clearly for a given topic Characterize learning goals or assessments using Bloom’s Taxonomy Recognize the value of aligning assessments with goals
Let’s get our brains on topicCASE STUDY: Frustrated student 5 minutesThink – pair – shareWhat issues might be contributing to this situation?Do the assessments give the student any feedbackabout what they understand while they are learningabout this topic?Does the student appear to understand what isexpected of her?What suggestions do you have for the professor?Have you faced a similar challenge?
Outcomes should drive assessment & instruction Where Where you at? you goin’ to? “Backwards design”Atkin, Black, & Coffey 2001; Otero & Nathan 2008
But how do we measure outcomes? How do you know when you know something? How do you know when your students know something? How do your students know when they know something? Buzzword: Metacognition
People often don’t know what they don’t know AND/OR Think they know something but don’t!MISCONCEPTIONS Private Universe (http://www.learner.org/resources/series28.html)
Misconceptions can drive instruction If a camera crew making a documentary on studentmisconceptions were to question your students at the end of your course orthe end of your degree program, what would you be most embarrassed to find out that they didn’t know? These should be your top goals
Learning goals• Definition: What students should be able to do after completing a course• Requirement: Must be measurable assessment and goals tightly linked Your goals should reflect what you value in student learning Often, students never know what your goals are!
Learning Goals are different than a syllabusSyllabus/ Topic List Learning goals:• Material covered (and time Outcome and student oriented:spent) • Identifies what students will be able to do as a result of learning • Defines what students are expected to learn Learning Goals (for a whole course) can be broad. At the topic or lecture level, the learning objectives should be more specific
Example Learning Goal 13Journalism:Students should be able to contrast press freedom inthe United States with that of other countries aroundthe world.GeneticsStudents should be able to calculate the probabilitythat an individual in a pedigree has a particulargenotype. Courtesy Sandra Fish and Michelle Smith
Goals at different levels 14 Course: Students should be able to Course-scale learning goals contrast press freedom in the United to 10 per course) States with that of other countries around the world. Topic-scale learning goals Topic: Students should be able to2-5 per topic) calculate the probability that an individual in a pedigree has a particular genotype. Class-scale learning goalsper class period) Consistent & aligned
Human Genetics for non-majors Course-level learning goal Specific learning goal Content: Demonstrate how Predict the probability of meiosis leads to diversity in generating sperm and egg cells the next generation with specific chromosomal Skills: Become better problem makeup, and explain how these solvers cells are produced13
Intro Astronomy Course-level learning goal Class-scale learning goal Content: Explain the role of Analyze the phases of the natural forces in the moon by using computer universe simulations and constructing a Skills: Interpret simulations model. and data14
But what does “understanding” mean?How do we define goals? 1. What are the different types of knowledge we want students to have? 2. At what level do we want that knowledge to be?
#1: Types of knowledge (learning goals)FACTS: What type of understanding do you want them to gain? Terminology, information, detailsCONCEPTS Classifications, categories, principles, models, reasoning. Analyze, explain, and predict the world around youPROCEDURES: Skills, techniques, methods, problem-solving. Thinking like a scientist: Use alternative representations, compare and contrast, strategize, justify, design an experiment, create a graph.METACOGNITIVE Self-awareness about what helps you learn; studying & learning strategies.AFFECTIVE (attitudes & beliefs): Appreciate, enjoy, value. Recognize that the behavior of the world around you is not magical and mysterious, but rather can be understood and predicted using certain fundamental principles.)Handout
EXERCISE #2: Exam Dissection Use 3-5 questions on 4 minutes one of your exams Work alone or in pairs First, determine the Type of Knowledge 1 Cognitive Process Level (Bloom’s) 2 3 4 5 6 Knowledge Type of Remember Understan Apply Analyze Evaluate Synthesize d A. Factual the question knowledge B. Mark the question Conceptual knowledge number in the C. Procedural knowledge (skills) appropriate box D. Meta- cognitive knowledge E. Attitudes and beliefs
#2: Levels of knowledge Bloom’s Taxonomy, 1956What level of understanding do you want them to gain? Higher-level cognitive skills Higher cognitive orders Lower cognitive orders ( Lower-level cognitive skills
EXERCISE #2 (continued) Exam Dissection For those same 3-5 4 minutes questions Now determine the Bloom’s Level (within Type of Knowledge Cognitive Process Level (Bloom’s) 2 levels) of those same 1 2 3 4 5 6 Remember Understan Apply Analyze Evaluate Synthesize d questions. A. Factual knowledge For each question, B. Conceptual knowledge mark the question C. Procedural knowledge number in the (skills) D. Meta- appropriate box cognitive knowledge E. Attitudes and beliefs
Check-list for creating class-scale learning goals: Is goal expressed in terms of what the student will achieve or be able to do? Is the Bloom’s level of the goal aligned with your actual expectations? Is the goal well-defined? Is it clear how you would measure achievement? Do chosen verbs have a clear meaning? Is terminology familiar/common? If not, is the terminology a goal? Is it relevant and useful to students? (e.g. connected to their everyday life OR does it represent a useful application of the ideas). Handout
Intro Physics Original L.G. Understand how energy, frequency and wavelength are related.New L.G. AdvantagesCompare and contrast Higher level. Defineselectromagnetic waves understanding. Encourages(e.g., gamma and radio) in critical thinking as well asterms of energy, memorization.wavelength, frequency,and relevant applications. Image: Michael Ströck (mstroeck)
Goals can be revised for clarity or to address highergoalJournalism Learning Goal Understand the essential features of a newspaper article.Revised Learning Goal AdvantagesAnalyze and contrast the Operationalized. Higherstructures of a newspaper level goal. Encouragesarticle, a news broadcast, comparison. Requiresand an online news site application of knowledge.Higher level of Bloom’s: Write a mission statement for your career as a journalist.15 Learning Goal credit: Sandra Fish. Image by Tkgd2007 on Wikimedia
Work on your learning goals with your partner 15 minutes Individually, using one exam question that you brought with you, write a topic-level learning goal that this question would assess. (Keep a copy of this first try). Share your learning goal with your partner and/or facilitators, and use your white boards to work on revising each LG. Compare the current wording of the LG to the guidelines we have provided (identify the “level” of this LG, and whether it is too broad or too narrow). Discuss how the LG could be rewritten to better state your true goal. Then, for the topic you have been discussing, write a LG that is one or more level(s) higher on Bloom’s. Note: You can use the verbs and model questions on the “Bloom’s Taxonomy17 handout to help you.
Share your learning goals with another group 18 10 minutes Do the learning goals make sense to someone who didn’t write them? Do they meet the checklist guidelines? How might the learning goals be improved? From the LGs you discuss, pick one that underwent improvement to share with whole group
Share the process of creating learning goals with your group• What was the exam question you started with?• What was your original LG?• How did the goal change through discussion?• What was difficult about this process and what did you learn through your discussion?• Questions? 19
Now you’ve got learning goals. How do youintegrate this with your course?• When would you write your learning goals?• When do you refer to your written learning goals?• How does this help you decide what to do in your class time?• How does this help you decide what to put on your homework? 19
How do we align goals and assessment? Outcomes should drive assessment & instruction Where Are we Where you at? you there goin’ to? yet?Atkin, Black, & Coffey 2001; Otero & Nathan 2008
What assessments should I use?To find the answer to that, you need to consider•What are assessmentsthat align with mylearning goals and keyconcepts?•What kinds ofoutcomes can Imeasure? Image: Gabriel Pollard
Why care about assessment?Assessment is more than grades, it is feedback for students and instructors and it drives student learning (National Institute for Science Education, 1999) Ongoing assessment plays a key role – possibly the most important role – in shaping classroom standards and increased learning gains” – Black and Wiliam, 1998
When can we assess students? Course-scale: Before or after a course. Class-scale: Before, during, or at the end of a class A bit of Jargon: Formative vs. Summative Assessment
What are some assessment methods? 35FORMATIVE, IN-CLASS SUMMATIVE EVALUATION Concept Tests / Clickers Minute Papers Quizzes Just in Time Teaching Exams Listening to student discussion in class Oral presentations Weekly / Daily Surveys Poster symposia White-boarding activities In-class work / Tutorials SUMMATIVE FEEDBACK Case studies Conceptual surveys Ranking / ordering tasks Attitude surveys Think-pair-share. Student-designed reading assessmentsOTHERS FORMATIVE, OUT OF CLASS Homework Discussion boards
Alignment– an example:Broader Learning goals:• Use graphs as part of thought processes• Recognize equilibrium points & determine if stable Formative Summative Measurable Assessment Assessment Outcomes (Instruction) (Exams)Interpret graphs Tutorial with Which of theseof potential topographic points isenergy maps. Where stable? Why? will dislodgedPredict behavior boulder roll? Relate to U.
How do we align goals and assessment? Instruction & Form. Assess.Formative Summative &assessment Learning goals formative assessment Where Are we Where you at? you there goin’ to? yet?Atkin, Black, & Coffey 2001; Otero & Nathan 2008
Assessments communicate your intent If you test them on facts, then that is what they will studyDoes this process change how you think about your exams? ?
Questions?Please fill out action plan and evaluationNote the “take-home” exercises in packet Much more at: per.colorado.edu stem.colorado.edu www.colorado.edu/sei phet.colorado.edu www.colorado.edu/istem