2009-10 End of Year Event Talks Overview of CWSEI progress (lots of data!)– Carl Wieman Improving Student Study Habits: results of interventions Sara Harris & Louis Deslauriers Interactive Engagement: examples from UBC classes (video) Sarah Gilbert & department members Poster session 11‐1:30 room 101 Details on everything being done and learnedWorkshop & Discussion 1:30 – 3:00pm, room 101 – How to Most Effectively Measure the Learning that Matters (workshop led by Carl Wieman) g ( p y )3:15 – 4:30pm, room 101 – Incorporating Writing in the Science Curriculum; what and how? (discussion)
CWSEI “Trinity” for each course What should What are students students1st: Learning goals. (what learn? learning?should students be able to do?)should students be able to do?)2nd: Good assessment Which instructional approaches h(validated tests) improve student learning?3rd: Improved teaching methods : Improved teaching methods (research based, improve learning)Materials, assessment tools, homework, notes … Materials assessment tools homework notessaved, reused, improved. Making teaching more effective, and more rewarding f M ki t hi ff ti d di for faculty and students
Carl Wieman Science Education InitiativeStarted 3 years ago ⇒ widespread impro ement in science ears idespread improvementeducation.Departments at various scales and levels of maturityLarge scale mature-- Earth and Ocean SciencesLarge scale younger-- Physics and Astronomy Computer Science MathSmaller scale programs -- Chemistry Statistics Life Chemistry, Statistics,Sciences$2 M gift from David Cheriton for math and comp sci comp. sci.
Today--focus on data1. How many courses/faculty transformed?2.2 How much better is the learning? a. learning b. engagement c. innovative problem solving3. But does it stay learned? (retention)4. Reaching all students. Turning low performers into high5. Blizzard of data on improvement from across thedepartments (appetizer for posters)
1. How widespread is the change-- EOS, most mature,full 3 year effort effort.24 courses transformed.18 with formal CWSEI support pp6 with strong informal and moral support~ 26 faculty involvedtypical new things•clearly articulated learning goals for students andfaculty•pre-reading assignments & quizzes•clicker questions and peer discussion•worksheets & in-class group activities•group exams•team projects team•pre-post testing to measure learning, ...much more active learning and feedback,
EOSC 111: Laboratory ‐ Completed ‐ All hands on, Exploration of Planet Earth ‐ First yr Lab ‐ lots of group work, Sara HarrisS i ‐ approx 100 per semester ‐ i di id l d 00 individual and group quizzesEOSC 112: The Fluid Earth: ‐ been through about 2 ‐ clickers Atmosphere and Ocean years of EOS‐SEI ‐ online quizzes Sara Harris, Roger , g ‐ service course for anyone ‐ article readings, quizzed, y g ,q ,Francois, William Hsieh at UBC with feedback (rubrics) ‐ about 350/year (split between 2 sections)EOSC 114: Natural ‐ Completed June’08 ClickersDisasters st year exploratory ‐ 1 year exploratory on‐line assignments on line assignmentsR. Stull and many others course ‐ over 1000 stu. per yearEOSC 210: Earth Science ‐ Completed Clicker Qs, in each lecture. for Engineers for Engineers ‐ Lecture and lab Lecture and lab Activities and discussions Activities and discussionsErik Eberhardt, Uli Mayer, ‐ 230 each September in most lectures. Labs with Stuart Sutherland group work and hands on activitiesEOSC 211: Computer ‐ Second teaching term In‐class worksheets, pair‐Methods in the Earth, Sept. 2010 programming, name‐Ocean and Atmospheric ‐ 2nd year programming sticks, pair and small group Sciences course lecture/lab discussions, class Richard Pawlowicz, ‐ 55 students enrolled last discussionsCatherine JohnsonCatherine Johnson term
EOSC 212: Topics in Earth ‐ Completed June ‘09 ‐ team‐based quizzes and and Planetary Sciences ‐ 2nd year “science inclass activities & M. Bostock, M. Jellinek thinking” course discussions ‐ 20 to 40 students per p ‐ article reading and g year question posing workshop style classes ‐ peer assessed presentations & postersEOSC 220: Introduction to Complete EOSC 220: Introduction to ‐ Complete 3x5 cards used to answer 3x5 cards used to answerMineralogy ‐ mandatory intro. lab questions in class, in‐class Mary Lou Bevier course for EOS students activities, class discussion, ‐ 120 students enrolled labs have group work and group quizzesEOSC 221: Petrology ‐ Completed Wake up exercises Maya Kopylova ‐ Lecture and lab (integrating activities into ‐ 100 each January each lecture), some 3x5 cards, labs with group work and hands on, some work and hands on some "authentic activity" labsEOSC 223: Field ‐ Minor support summer lectures have regular techniques 2009 activities and 3x5 cards to Mary Lou Bevier ‐ Lectures and Field get feedback, component Field activitiesEOSC 252: Physics of ‐ First teach term ‐ lab exercisesgeologic materials completed ‐ in‐class demonstrations F. Herrmann ‐ 2nd yr “physics” course with worksheets ‐ 20 – 30 students each 20 30 students each ‐ aiming for interactive aiming for interactive year lecturing next yr. etc. for 3 more pages
2. But do these changes improve student outcomes?(learning, engagement, ...)Hard to tell in most courses because no pre-transformdata.dataData from example courses where similar transformations,and good pre t d d transform and post t f d t transform data. f d tLouis Deslauriers and Ellen Schelew (physics)--- cleanestcomparison study of teaching methods ever done.Will be landmark in science education research(as soon as they write it up for publication)
new-- Louis Deslauriers (PD) and Ellen Schelew (grad std) Perfect comparison of teaching methods: identical sections (260 each), intro phys. 153, same material & time. ___I___________ _____II_________ Experienced highly rated Experienced highly rated instructor instructor-- trad. lecture & ~2 cl. 2 instructor trad. instructor--trad. lecture & ~2 cl. 2 questions questions same preparation same attendance same attendance wk 1-11wk 1-11 same engagement same midterm 1 & 2 grades Wk 12-- competition elect-mag waves elect-mag elect mag waves Louis and Ellen (inexper.) regular instructor research based teaching intently prepared lecture common exam on EM waves
transformed section•pre class reading assignments with quizzes•pre-class•in-class small group activities•clicker questions with student-student discussion•targeted instructor feedback guided by observationsof student thinking
Results II. Trad I. Transformed. 1. Attendance pre 58% 58% (wk 10 & 11) during 58 % 81% 2. Engagement pre 50% 50% (wk 10 & 11) (back ½ room) during 50 % 85% 3. Learning (test) 41(1)% 74(1) % above guess (23%) g ( ) 18% 51% S. D. = 13% trad trans trad. ⇒0.58 x 0.5 = 29% engaged ad ⇒0 58 0 5 9% e gaged 18 51 for above average instructor trans. ⇒0.81 x 0.85 = 69% engagedother things practiced: scientific discourse, critiquing scientificarguments, sense-making, collaboration.
But how did students feel about it? “Q1 I really enjoyed the interactive teaching technique during Q1.the three lectures on E&M waves (Ch32).” 70 63 57 ents 60 umber of stude 50 40 30 20 12 10 Nu 2 0 0 strongly Strongly Agree Neutral Disagree Strongly agree disagree agree Q2 I feel I would have learned more if the whole phys153 course would have been taught in this highly interactive style. 80 70 67 mber of students 60 50 40 36 30 21 20 Num 8 10 2 0 Strongly Agree Neutral Disagree Strongly agree disagree
Q6 I found the pre-reading to be very helpful to my learning: 70 66 60 Numb of students s 50 40 35 28 30 ber 20 10 3 1 0 gy Strongly Agree g Neutral Disagree g Strongly gy agree disagree Q5 What contributed most to my learning during these three lecture on E&M waves: 60 51 Number of students 50 39 40 30 22 r 20 13 8 10 0 clicker in-class Trying to figure Trying to work instruc. pre- The instructor The pre-reading The pre-reading pre-read out the answer out the answers explanation to quiz quest. to clicker to the in-class explan. activity l the clicker reading d quiz questions activities to c.q. or in- questions or class activities activ.
Q8 In class, the group discussions with my neighbors were veryhelpful to my learning: 80 70 70 60 Numb os students s 50 40 33 ber 30 22 20 10 7 0 0 Strongly agree Agree Neutral Disagree Strongly disagree
What does such a class l k l k h d h l look like?See upcoming video clips session. p g p
Measuring student (dis)engagement. Erin LaneWatch random sample group (10-15 students). Checkagainst li t of di i t list f disengagement b h i t behaviors each 2 min. h i time (minutes)
What about advanced upper division courses? Physics 408-- d Ph i 408 advanced optics d ti Taught by same instructor for several years-- continually working t i ti ll ki to improve. He radically transformed this year. Ended up covering same material in less time. Midterm exam grades: Pre transformation (lecture) P t f ti (l t ) 56 +/ 3 1% +/-3.1% Post transformation 77 %(Exams different, but three experts did blind rating of the differentexams. All concluded post transformation exam more difficult)
What about learning to think more innovatively? g y Learning to solve challenging novel problems Jared Taylor and George Spiegelman “Invention activities”-- practice coming up with Invention activities mechanisms to solve a complex novel problem. Analogous to mechanism in cell.2008-9-- randomly chosen groups of 30, 8 hours ofinvention activities.This year, run in lecture with 300 students. 8 timesper term. (video clip)
Plausible mechanisms for biological process student never encountered before 6.0 Average Number 5.0 olutions 4.0 mber of So 3.0Num 2.0 1.0 0.0 Control Structured Inventions (Outside Inventions (During Problems (tutorial) of Lecture) Lecture)
Average Time to First Solution Thread 14.0 12.0 10.0 n)Time (min 8.0 6.0 4.0 2.0 0.0 Control C t l SPSA (O t id of (Outside f IA (O t id of (Outside f IA (D i L t ) (During Lecture) Lecture) Lecture)
3. So research based teaching achieves much betterlearning & much greater engagement.Does it stay learned? y(retention)
3. Mastery of quantum mechanics concepts-short& long term Deslauriers & Wieman to be published score on quantum mech. concept survey q p y 100 90 88 %) 85 ncept Surve Score (% 80 interactive engagement/practice 70 68 ey superb traditional 65 60 lecturer 50 Con 40 30 0 5 10 15 20 Retention interval (Months)
Deslauriers, Lane,4. Bringing up the bottom of the distribution Harris, Wieman “What do I do with the weakest students? Are they just hopeless, or is there anything I can do to make a difference?” a. To get such big improvements in average, have to impact entire distribution b. Data on how to transform lowest performing students into medium and high. Intervened with bottom 25% of students after midterm 1. •Phys250 ( gp y program, high selective and demanding), y (engphys p g , g g), bottom 25% averaged +20% improvement on midterm 2! •EOS climate science course. Very broad range of students. • Averaged +30% improvement!
nonintervention intervention100 100 90 80 70 score student exam scores 60 50 40 M1 30 MIDTER 20 M 2 100 %0 0 20 40 60 80 100 120 140 160 180 student number student number midterm 1 score What magic does this? ,& X, midterm 2 score Listen to next talk. •~All UBC science students can b successful All i t d t be f l •A little help on how to learn goes a long way
Large scale survey (~ 600) and interviews on factors that UBC science students perceive as affecting academic performance Ashley WelshAn early finding•Students overwhelmingly recognize they do not know how tostudy effectively Is seen as major barrier to success but find effectively. success,little help in learning how to study.
masses of other datawill overwhelm you with blizzard of infoGo to posters to get details and more results
Math 152 - Assessment of Matlab “for” loop mastery70% Correct60% Incorrect50% Blank40%30%20%10% 0% 2008 2009
Math 184--intro calculus “workshops” part of course. workshops courseLast year collected data on how they were functioning,(observations, surveys, examine correlation of studentmarks with numerous factors.)This yea , made changes based o t e data s year, ade c a ges on the data.
Math 184 Workshops – Correlation between workshop attendance and course grades g Pearson Correlation Coeffficient 0.60 0.50 relevant # is 0.40 (corr. coeff.)2 0.30r 0.20 0.10 0.00 2008 2009
Math 184 Workshops Student Survey The workshop problems …. "… provide useful practice for p p "… are related to material covered … solving problems on tests" in class"50% 2008 50% 200840% 2009 40% 200930% 30%20% 20%10% 10%0% 0% Strongly Agree Neutral Disagree Strongly Strongly Agree Neutral Disagree Strongly Agree Disagree Agree Disagree
EOSC 211: Computer Methods in the Earth SciencesIntroduced technique of “Pair-Programming”from comp sci ed research:Compared to previous year:• Labs are completed about 15% faster p• Lab marks are about 10% higher• Students are MUCH happier with the transformed course t f d 31
EOS Impact of TA training program 6. I consider myself to be an effective teacher. 12 10 8 6 Pre2008 4 Post 2 0 Agree Neutral Disagree 6. I consider myself to be an y effective teacher. 90 80 702009 60 50 Pre 40 Post 30 20 10 0 Agree Neutral Disagree
EOS 212-- using model based reasoning Pre & post‐test scores pre 12 ts er of student 10 post 8 6 Numbe 4 2 0 90‐100 80‐89 70‐79 60‐69 50‐59 <49 score (%) bins ( )100% 90% 80% 70% MUCH more time on EOSC212 EOS collecting data 60% 50% a LITTLE more time on on time students 40% EOSC212 30% Roughly equal time on studying in courses. 20% EOSC212. 10% 0% a LITTLE less time on EOSC212. Relative amount of MUCH less time on EOSC212. time for different courses across sci.i
Phys 109 & Sci 1 Intro physics lab“invention” activity to develop scientific reasoning invention before activityy after activity 1.0 ction of students s use histograms use standard s deviation frac 0 student characterization of data
see posters to learn more about these and many more Conclusions1. It is possible to make widespread transformationin UBC science teaching-- many courses, many faculty.2. CWSEI transformations lead to•much greater engagement,•much greater learning, learning•happier students. Looking forward to great progress in coming year
third year quantum mechanics course--Common questions on QM spinpre transformpre-transform 2009 final exam 68%+/-3% 68%+/ 3%2010 midterm 76%+/-2%(spent half as much ti( t h lf h time on t i ) topic)
physics lab diagnostic measurements showing improvements, but more work needed