0
Fine Tuning Assessment for Teaching and Learning  Roy Williams Michael McCabe
Context  <ul><li>Mathematics student numbers 2002 – 2010: </li></ul><ul><li>30 to 150 in first year  (plus 2 nd  year?).  ...
<ul><li>Questionmark Graphics  </li></ul>
2010 Semester 2 <ul><li>Staffing Changes </li></ul><ul><li>New Responsibilities </li></ul><ul><li>Short notice for new del...
QMark:  Outdated for Mathematics     <ul><li>No algorithmic-randomised questions </li></ul><ul><li>Technical, authoring an...
Development  <ul><li>Meeting with Wiley Plus within month </li></ul><ul><li>? Redevelopment in MapleTA? </li></ul><ul><li>...
 
Advantages <ul><li>E-book provided for all  </li></ul><ul><li>Large banks of pre-prepared questions </li></ul><ul><li>Link...
Disadvantages <ul><li>Learning rather than examination focus </li></ul><ul><li>Pre-prepared ppts – formulaic </li></ul><ul...
Assessment <ul><li>Formative </li></ul><ul><li>Pre-prepared </li></ul><ul><li>Selected formative (10, mixed difficulty) </...
Assessment <ul><li>Summative </li></ul><ul><li>Replaced repeated assessments with repeated questions  </li></ul><ul><li>Cu...
Disadvantages <ul><li>No password facility </li></ul><ul><li>Administrative overheads and work-arounds </li></ul><ul><li>L...
Evaluation   <ul><li>Victory  </li></ul><ul><li>One-stop shop for information </li></ul><ul><li>Hand-written lecture notes...
Evaluation   <ul><li>MapleTA  </li></ul><ul><li>Ease of Use </li></ul><ul><li>Many practice questions  </li></ul><ul><li>B...
Evaluation   <ul><li>Wiley Plus  </li></ul><ul><li>Multiple attempts at questions </li></ul><ul><li>Cumulative scoring  </...
Evaluation   <ul><li>Useability  </li></ul><ul><li>MapleTA very good </li></ul><ul><li>Wiley good </li></ul><ul><li>Victor...
Evaluation   <ul><li>Method  </li></ul><ul><li>Survey on 6 point scale (e.g. essential to learning >> Of no use at all </l...
Questions Arising <ul><li>To ASP or not to ASP, if so, 100% or not?  </li></ul><ul><li>Alternative philosophies (OS, Comme...
Upcoming SlideShare
Loading in...5
×

Fine tuning assessment for teaching and learning

216

Published on

Can we customise e-assessment so that it supports learning, rather than just summative certification? Presentation at Portsmouth University

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
216
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Fine tuning assessment for teaching and learning"

  1. 1. Fine Tuning Assessment for Teaching and Learning Roy Williams Michael McCabe
  2. 2. Context <ul><li>Mathematics student numbers 2002 – 2010: </li></ul><ul><li>30 to 150 in first year (plus 2 nd year?). </li></ul><ul><li>Negligible staff increases </li></ul><ul><li>Standard Wiley textbook </li></ul><ul><li>In-house assessments (Qmark) </li></ul><ul><li>PRS feedback (good question bank) </li></ul><ul><li>Link to personal tutors (most staff) </li></ul><ul><li>Support from Maths Cafe </li></ul>
  3. 3. <ul><li>Questionmark Graphics </li></ul>
  4. 4. 2010 Semester 2 <ul><li>Staffing Changes </li></ul><ul><li>New Responsibilities </li></ul><ul><li>Short notice for new delivery of course </li></ul><ul><li>Regional Maths Seminar (Sigma-Hub) </li></ul><ul><li>Awareness of Wiley Plus resources </li></ul><ul><li>Opportunity to rethink course delivery </li></ul>
  5. 5. QMark: Outdated for Mathematics <ul><li>No algorithmic-randomised questions </li></ul><ul><li>Technical, authoring and display issues </li></ul><ul><li>Answering in mathematical notation </li></ul><ul><li>(Limits questions formats) </li></ul><ul><li>Cumbersome editing and graphics updating </li></ul>
  6. 6. Development <ul><li>Meeting with Wiley Plus within month </li></ul><ul><li>? Redevelopment in MapleTA? </li></ul><ul><li>Decision to do ‘live pilot’ – book plus </li></ul><ul><li>e-assessment, e-book, etc. </li></ul><ul><li>Two week start-up </li></ul><ul><li>Wiley support and resources </li></ul><ul><li>Wiley meeting with staff </li></ul><ul><li>Wiley meeting with students </li></ul><ul><li>Wiley assistance at launch </li></ul>
  7. 8. Advantages <ul><li>E-book provided for all </li></ul><ul><li>Large banks of pre-prepared questions </li></ul><ul><li>Links from assessment to full-text, hints, solutions, tutorials </li></ul><ul><li>Learning design underlying the structure </li></ul><ul><li>Pre-prepared lecturer ppts: </li></ul><ul><ul><li>Supplementary PRS questions </li></ul></ul><ul><ul><li>PPT Summary (2 screen options) </li></ul></ul><ul><ul><li>Applets – interactive activities </li></ul></ul><ul><ul><li>Instructor guidance and resources </li></ul></ul><ul><ul><li>Exportable MCQ’s >> Victory </li></ul></ul>
  8. 9. Disadvantages <ul><li>Learning rather than examination focus </li></ul><ul><li>Pre-prepared ppts – formulaic </li></ul><ul><li>Limitations on customisation, and in-house Q’s </li></ul><ul><li>(MapleTA underlies Wiley Plus) </li></ul><ul><li>Error reporting, not correction </li></ul><ul><li>3/7 tests in MapleTA – (different text book) </li></ul>
  9. 10. Assessment <ul><li>Formative </li></ul><ul><li>Pre-prepared </li></ul><ul><li>Selected formative (10, mixed difficulty) </li></ul><ul><li>Unlimited attempts on algorithmic questions </li></ul><ul><li>Varied question types, including interactive graphics </li></ul>
  10. 11. Assessment <ul><li>Summative </li></ul><ul><li>Replaced repeated assessments with repeated questions </li></ul><ul><li>Cumulative scoring </li></ul><ul><ul><li>Builds on success </li></ul></ul><ul><ul><li>Promotes confidence </li></ul></ul><ul><ul><li>Avoids repetition </li></ul></ul><ul><li>Q. Classification </li></ul><ul><ul><li>Easy/ medium/ hard. </li></ul></ul><ul><ul><li>By learning objective </li></ul></ul><ul><li>Varied Q. types, including interactive graphics </li></ul>
  11. 12. Disadvantages <ul><li>No password facility </li></ul><ul><li>Administrative overheads and work-arounds </li></ul><ul><li>Long-term continuity, control, stability </li></ul><ul><ul><li>Year-on-year use of same tests </li></ul></ul><ul><ul><li>Upward compatibility of software </li></ul></ul><ul><li>Non-uniformity/ comparability of questions </li></ul><ul><ul><li>Scoring </li></ul></ul><ul><ul><li>Length </li></ul></ul><ul><ul><li>Amount of work required </li></ul></ul><ul><ul><li>Recommended time-on-task, and marks </li></ul></ul><ul><li>Discriminating between formative and summative </li></ul>
  12. 13. Evaluation <ul><li>Victory </li></ul><ul><li>One-stop shop for information </li></ul><ul><li>Hand-written lecture notes </li></ul><ul><ul><li>Lectures as process? </li></ul></ul><ul><ul><li>Lectures as mathematical thinking? </li></ul></ul><ul><ul><li>Digestible summary of contents </li></ul></ul><ul><li>But </li></ul><ul><li>Unreliability and GUI Issues </li></ul><ul><li>Mathematical limitations </li></ul>
  13. 14. Evaluation <ul><li>MapleTA </li></ul><ul><li>Ease of Use </li></ul><ul><li>Many practice questions </li></ul><ul><li>But </li></ul><ul><li>Input format ? </li></ul><ul><li>Lack of partial credit, inflexible marking </li></ul>
  14. 15. Evaluation <ul><li>Wiley Plus </li></ul><ul><li>Multiple attempts at questions </li></ul><ul><li>Cumulative scoring </li></ul><ul><li>Many practice questions </li></ul><ul><li>Feedback and links: solutions, hints, tutorials. </li></ul><ul><li>Invigilators calm and helpful </li></ul><ul><li>But </li></ul><ul><li>? Input format </li></ul><ul><li>GUI? </li></ul>
  15. 16. Evaluation <ul><li>Useability </li></ul><ul><li>MapleTA very good </li></ul><ul><li>Wiley good </li></ul><ul><li>Victory: issues </li></ul><ul><li>MTH 204 </li></ul><ul><li>Confirms findings, broadly </li></ul>
  16. 17. Evaluation <ul><li>Method </li></ul><ul><li>Survey on 6 point scale (e.g. essential to learning >> Of no use at all </li></ul><ul><li>Open-text questions (positive and negative) </li></ul><ul><li>MTH 155: 68/150 </li></ul><ul><li>MTH 204: 11/120 </li></ul>
  17. 18. Questions Arising <ul><li>To ASP or not to ASP, if so, 100% or not? </li></ul><ul><li>Alternative philosophies (OS, Commercial) </li></ul><ul><li>Sustainability </li></ul><ul><li>Viable hybrid model? </li></ul><ul><li>Learning (L&T model – TEL&T)? </li></ul><ul><li>Scaffolding – lecturer, interactive resource, both? </li></ul><ul><li>Educational benefits </li></ul><ul><ul><li>Repeat assessments or questions? </li></ul></ul><ul><ul><li>Integrated multi-mode resources? </li></ul></ul><ul><li>Trade-off: security and admin. load. </li></ul><ul><li>Customisation of resources </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×