Expert Guided Crowd Sourced Learning Content

1,433 views

Published on

Keynote presented at the Improving University Teaching Conference (http://www.iutconference.com/) Vancouver July 2014

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,433
On SlideShare
0
From Embeds
0
Number of Embeds
916
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Expert Guided Crowd Sourced Learning Content

  1. 1. A pilot study in a large enrolment introductory science course Expert-guided crowdsourced learning content Simon Bates simon.bates@ubc.ca @simonpbates bit.ly/batestalks
  2. 2. ~2010 ~2006 2013
  3. 3. Context - flipped classrooms CC BY-NC 2.0 https://flic.kr/p/f3ynHx Derek Bruff: Class time reconsidered http://prezi.com/donq036eunko/class-time-reconsidered/
  4. 4. Weekly rhythm for our 1A class Poh,  M.Z.,  Swenson,  N.C.,  Picard,  R.W.,  "A  Wearable  Sensor  for  Unobtrusive,  Long-­‐term  Assessment  of  Electrodermal  AcEvity,"  IEEE  TransacEons  on  Biomedical  Engineering,  vol.57,  no.5,  pp.1243-­‐1252,     May  2010.  doi:  10.1109/TBME.2009.2038487  
  5. 5. Weekly rhythm for our 1A class
  6. 6. Weekly rhythm for our 1A class
  7. 7. Learning gains on PI
  8. 8. Learning gains on PI
  9. 9. 1 2 3 A ball initially at rest is thrown upwards, comes back down & is caught ! Which of the following is a plausible graph of the acceleration of the ball with time?
  10. 10. Freeman et al PNAS www.pnas.org/cgi/doi/10.1073/pnas.1319030111 Wieman commentary PNAS http://www.pnas.org/cgi/doi/10.1073/pnas.1407304111 “This meta-analysis makes a powerful case that any college or university that is teaching its STEM courses by traditional lectures is providing an inferior education to its students”
  11. 11. ~2010 ~2006 2013
  12. 12. a"web&based"MCQ"repository"created"by"students"
  13. 13. Ins$tu$ons((signing(up(per(year:( ! 2009:! ! !22( 2010:! ! !66( 2011:! ! !204( 2012:! ! !266( 2013!(Jan,Jun):! !214( Growing(content(repository:( ! Courses:! !2,500( Logins/month:! !75,000( Ques<ons:! !600,000( Answers:! !12,000,000( Answers (20,000,000) Questions (1,000,000)
  14. 14. Student'ownership'over' learning'resource' Student'familiarity'with' social'so7ware' Leveraging'student' energy'and'crea9vity'
  15. 15. Badges' Points' Leaderboards'
  16. 16. Typical implementation Minimum participation requirements for each of two assessment exercises (PW1, PW2) Write 1 Answer 5 Rate / comment 3 5% course credit Physics 101, Energy & Waves Winter Semester: 3 sections, 791 students
  17. 17. Scaffolding TOCCLT'.' - QestIsct How 1o... q/t4osTer Submit ond onswer questions on topics in lhe torget region, just obove lhe physics you hove olreody mostered. Dislroclers ) l*"*"ò rNrrltng Ih s region corrk:ins llte plrysics knowledgo ond colrco¡ls you connol leorn yel becouse the louncalions crrc nol in plocc PHYSICS TOPICS IN YOUR TARG T REGION buo'5ot't1 'Den¡¡lu B¿:ovvì 1 J F:rcq - YOUR CHOSEN TOPIC à.. auC'q^ x@ bôo'osro pn ro, d logroo, o 9tS COMMON MISCONCEPÎIONS AND ERRORS (Sec hHÞ:/jÞhy¡.udofo¡.cdu/CJP/trêconcoÞllo¡t.pdt tor o llil ol common mhconcepllons) ¿K N(,qht %cce Açxs no exi¡þ , c.rrrçrJ On oblec'-t 5 õÞYJrnr cìü?À I in rtuiò. ü lrh'brot4ont eprç,e .I¡1enÀs crq den-s,{^ c>Ç c.,þ¡ec , cìo+ .*¿n:r oÇ Çtu,8, fr- btrrqon! $crCe c¿cs Àoc¡:'cr fr¿lt{e- r *hcrn ú? Trr¿e. $ììfifflsz hrrr: ¿lÕoolr-g lnnS (**= tcpo þ/rnl {otr-* qrÇ bq.:. 0" Ooz Physics knowledge ond conccpluol underslonding you lrove olrcody conslrucleC ¡n your heod ¡,<'t.lerJ rt)^,¿z! Þ.rrp r- rr't¡ ,l? Chcck lhol yovr on3Íêr ¡t rêo!ànoblê ðnd potrlble 'f<,,cc Photo by Seth Casteel http://www.littlefriendsphoto.com Permission to use agreed Introduced in tutorials ! Extensive scaffolding exercises ! Revisited in subsequent tutorials ! Tutorials delivered by 24 TAs
  18. 18. Engagement with PeerWise Number Multiplier Number Multiplier Questions 1105 [1.7] (1.4) 998 [1.6] (1.3) Answers 11393 [17.2] (14.4) 11807 [18.7] (15.0) Comments 4901 [7.4] (6.2) 5509 [8.7] (5.0) PW 1 PW 2
  19. 19. Engagement with PeerWise
  20. 20. Engagement with PeerWise PW1 PW2 Contributed and met minimum requirements Contributed but did not meet mininum requirements Did not participate 0.84 0.80 Total: 791
  21. 21. Question/Explanation Quality Bloom’s Taxonomy of levels in the cognitive domain Score Level Description 1 Remember Factual knowledge, trivial plugging in of numbers 2 Understand Basic understanding of content 3 Apply Implement, calculate / determine. Typically one-stage problem 4 Analyze Typical multi-step problem; requires identification of strategy Evaluate Compare &assess various option possibilities; often conceptual Synthesize Ideas and topics from disparate course sections combined. Significantly challenging problem. Text
  22. 22. 1 2 3 4 1 2 3 4 0 20 40 60 Numberofquestions Assessment 1 Assessment 2 Bloom's Taxonomy: Question Quality Textp>0.05, NS 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 6 Taxonomic Category PercentageofSubmittedQuestions First semester N = 350 Second semester N = 252 Bates et al, PRST PER, in press 2014 http://arxiv.org/abs/1308.2202
  23. 23. Question/Explanation Quality Description of explanation quality Score Level Description 0 Missing No explanation provided or explanation incoherent/ irrelevant 1 Inadequate Wrong reasoning and/or answer; trivial or flippant 2 Minimal Correct answer but with insufficient explanation/ justification/ Some aspects may be unclear/incorrect/ confused. 3 Good Clear and detailed exposition of correct method & ans 4 Excellent Thorough description of relevant physics and solution strategy. Plausibility of all answers considered. Beyond normal expectation for a correct solution
  24. 24. 0 1 2 3 4 0 1 2 3 4 0 20 40 60 Numberofquestions Assessment 1 Assessment 2 Explanation Quality p=0.02
  25. 25. ~2010 ~2006 2013
  26. 26. PHYS101: Energy and Waves
  27. 27. Logistics ! ! Cohort split into 4 groups ! Each week one group tasked with creating LOs ! Each submission counts for 2.5% of final grade ! Repeat cycle twice per Semester ! Students can submit >2 LOs & receive grade for best 2 ! Short survey on submission ! Students encourage to apply CC licenses
  28. 28. Results: engagement 0 100 200 300 LO 1 LO 2 LO 3 LO 4 LO 5 LO 6 LO 7 LO 8 Number of students Assigned Optional
  29. 29. 0 10 20 30 40 50 60 70 80 90 100 0 100 200 300 Grade (%) Numberofstudents LO Grade Distribution
  30. 30. 0 10 20 30 40 50 60 70 80 90 100 0 100 200 300 Grade (%) Numberofstudents LO Grade Distribution These students didn’t even try! 0 10 20 30 40 50 60 70 80 90 100 0 100 200 300 Grade (%) Numberofstudents LO Grade Distribution
  31. 31. Which students aren’t submitting LOs? Non-participants are from all walks of life! 0 10 20 30 40 50 60 70 80 90 100 0 5 10 15 20 25 Final Exam (%) Numberofstudents Final Exam marks of students that did not participate in LOs
  32. 32. Results: types of Los PPT DOC Prezi Ted-Ed PDF Image Other
  33. 33. Results: time on task 0 100 200 300 400 Less than 0.5h 0.5 to 1 h 1 to 2h 2 to 3h 3 to 4h 4 to 5h More than 5h Number of students
  34. 34. Results: self reported change in understanding 0 200 400 600 800 None Little Moderate Good Excellent Number of students 0200400600800 Number of students
  35. 35. PHYS101: Energy and Waves 3. Successes http://youtu.be/BObyt_NJYrE
  36. 36. Sample 2 - Standing Wave in a bowl
  37. 37. Sample 3 - Colour Loss Underwater
  38. 38. Student generated exam content 3. Successes
  39. 39. simon.bates@ubc.ca @simonpbates bit.ly/batestalks Relevant publications: ! Scaffolding student engagement via online peer learning - European Journal of Physics 35 (4), 045002 (2014) ! Student-Generated Content: Enhancing learning through sharing multiple-choice questions. International Journal of Science Education, 1-15 (2014). ! Assessing the quality of a student-generated question repository - Phys Rev ST PER (2014) in press ! Student-generated assessment - Education in Chemistry (2013) 13 1
  40. 40. Image credits Crowd http://www.flickr.com/photos/18378655@N00/613445810/

×