assessment and rubrics ecp ind mar 2012

1,805 views

Published on

Presentation from the Teaching and Learning Induction Worshop held in March 2012 at Mont Fleur.

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,805
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
116
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Module design and assessment issues: e.g. University policy, variation in faculty expectations, taking the ECP reality into account (“killer courses”), how to be fair but maintain/ set standards, timing and collaboration between ECP staff Formative and summative, formal and informal Role of marks in assessment – ECP vs mainstream. Weighting of marks for different activities must not simply follow the mainstream models but prepare students for them – scaffolded to independent What is valuable ? Grad attributes and critical skills – role of ECP in preparing students for success How can ECP make assessment more useful to students yet give a good indication of readiness? Alternatives to traditional forms. Transparency and student induction into “secrets” of academic operations (what kind of support does not affect standards negatively) Standards – even or uneven – different results – balance of pass /fail (good/bad) Rubrics – multi-purpose
  • Grad attributes are assessable and need to be built in to module design and assessment .
  • Think carefully about what the point of the assessment is: is it for learning or of learning?
  • E.g. if you use only MCQ’s, are we tapping in to only one kind of thinking and learning? Is assessment about making our tasks easier or is it about measuring a range of student competencies and attitudes, like flexibility, tolerance, innovative thinking? Are we going beyond testing/measuring only the base-line knowledge or are we going for acceptable mediocrity?
  • The words we chose for our outcomes must relate directly to whether or not we can assess/ measure them, and so speaks to assessment tasks appropriacy.
  • Assessment strategy: includes both the form of assessment, which outcomes are being assessed, and the purpose of the assessment (why am I doing it? What do I want to know?) Other questions to answer: Is this within the capability of students? Have I taught this (content and/or skill)? What conditions will affect their success/failure? What are my assumptions? Do I cater for all kinds of learners (visual, aesthetic, etc)?
  • Rather refer to page 114 in reader. Provide Assessment table for check-listing against outcomes. Activity: Take your outcomes and now use the table to fill in two different assessments you might use and check them against outcomes in table provided. How does assessment relate to concepts?
  • See differences in Knight pp. 117/118
  • Think about feedback in assessment …
  • Difference between Norm and Criterion-Referenced assessments - See differences in Knight pp. 125 - 127 Normative : students are ranked according to marks – expected range of ability – a way of treating marks rather than awarding them. Implies that teaching and learning has little effect. Criterion-referenced is a way of awarding marks – provide criteria and then measure attainment; i.e. all can fail or all can pass. Assessment is then based on criteria (level descriptors) and if achieved and how well. These are inherently complex (Knight) and takes time and requires accurate description to capture. See advantages p.127
  • Rubrics are an efficient and effective form of authentic assessment; they measure both process and product according to real-life criteria. They show students where they are in relation to where they need to be and provide the scaffolding necessary to improve the quality of work. The students become involved in both peer and self-assessment. Teachers and students are clear about what makes a good final product and why. Rubrics improve students’ end products and therefore increase learning.
  • A task description is framed by you as instructor and involves the performance required by the student to complete the task. It can apply to the overall behaviour required. You can usually cut and paste the task from your course outline and put it at the top of the rubric. This will allow you to communicate your expectations in relation to the assignment to the students. You would also need to include a heading or a title for the task as well as the description of the task. Scale: this shows how well or poorly a particular student has done. You should try to used positive, active verbs e.g exemplary, proficient, partially proficient, incomplete or not yet proficient or exemplary, good, satisfactory, needs improvement or exemplary, accomplished, developing, beginning. These could also correspond to a range of marks e.g. Exemplary 75+; very good 70-74; good 0-69; satisfactory 50-59; needs improvement 40-49 What you should first do is to construct three levels of performance for the rubric, then expand it to five. It is much easier to refine the descriptions of the assignment and create more levels after marking some of the students work or seeing what they are actually able to do The more levels there are the more difficult it is to differentiate between them and develop criteria for the scales. Dimensions give clarity on the parts of the task and which of these components is the most important, how much weight is given to each aspect of the assignment – you can add points or percentages to each dimension. Dimensions show the type of skills that students would need to be competent or to successfully complete a scholarly work. Dimensions clearly show the components of the task. They enable the instructor to provide feedback on specific parts of the assignment and how well or poorly they were done. When dimensions are well done, they will show once a piece has been marked what the student’s strengths and weaknesses at a glance. Dimensions don’t include anything concerning the quality of the performance. Descriptions of the dimensions a rubric should contain at least the highest level of performance in that dimension (a rubric that contains only the description of the highest level of performance is called ‘a scoring guide’ rubric. It allows for greater flexibility and personal input but increases time for written feedback). The rubric should, however, contain at least 3 scales and a description of the most common ways in which the students meet or fail to meet the highest level of expectations.
  • Do individual rubric on assessment task brought – see example & handout or use examples in readings.
  • Module design and assessment issues: e.g. University policy, variation in faculty expectations, taking the ECP reality into account (“killer courses”), how to be fair but maintain/ set standards, timing and collaboration between ECP staff Formative and summative, formal and informal Role of marks in assessment – ECP vs mainstream. Weighting of marks for different activities must not simply follow the mainstream models but prepare students for them – scaffolded to independent What is valuable ? Grad attributes and critical skills – role of ECP in preparing students for success How can ECP make assessment more useful to students yet give a good indication of readiness? Alternatives to traditional forms. Transparency and student induction into “secrets” of academic operations (what kind of support does not affect standards negatively) Standards – even or uneven – different results – balance of pass /fail (good/bad) Rubrics – multi-purpose
  • assessment and rubrics ecp ind mar 2012

    1. 1. WORKSHOP ON TEACHING AND LEARNINGASSESSMENT & RUBRICS MONT FLEUR 18 - 20 MARCH 2012
    2. 2. OutcomesParticipants should be able to:2. align assessment with outcomes3. select a range of assessment forms available (and approved by UWC)4. justify the choice of assessment forms5. explain and defend marks and weightings6. meet the criteria for reliability and validity7. create appropriate rubrics
    3. 3. Assessment Assessment must be aligned with learning outcomes and modular content. (Constructive alignment – Biggs) Assessment provides students with opportunities to show that they can do what is set out in the learning outcomes.
    4. 4. The Purpose of Assessment‘Assessment defines what students regard asimportant, how they spend their time and how theycome to see themselves as individuals’(Brown, 2001 in Irons 2008:11)‘assessment is seen to exert a profound influenceon student learning: on what students focus theirattention on, on how much they study, on theirquality of engagement with learning tasks, andthrough feedback, on their understanding andfuture learning’.(Gibbs & Simpson, 2004 in Irons, 2008:11)
    5. 5. Biggs: Assessment Tasks (ATs) provide students the opportunity to demonstrate whether or not they have achieved the ILOs and what level their performance is in those ILOs should be appropriately designed or selected to address all the ILOs that we want to assess use different assessment methods (tasks) addressdifferent ILOs. There should, therefore, be several kinds oftask. provide the evidence allowing teachers to make a judgment about the level of a student’s performance against the ILOs and to award a final grade.
    6. 6. Common ILOs Possible Assessment Tasks(Bi Describe essay question, exam, oral presentation (peer assessment) Explain assignment, essay question exam, oral, letter-to-a-friend Integrate project, assignment Analyse case study, assignment Apply project, case study, experiment Solve problem case study, project, experiment Design, create project, experiment Reflect reflective diary, portfolio, self-assessment Communicate a range of oral, writing or listening tasks, e.g. presentation, debate, role play, reporting,assignment, précis, paraphrasing,answering questions etc.
    7. 7. Alignment Graduate attributes (What are the overarching attitudes, skills and dispositions for UWC students?) ↕ Learning Outcomes (What do I want my students to be able to do?) ↕ Assessment Criteria (What do I need to see to know they can do it?) ↕ Teaching and Learning Activities(What will they be able to do, to know, how will their thinking and behaviour change as a result of the teaching/learning experience?) ↕ Assessment Tasks (How can I get them to show me those things?) ↕ Assessment Strategy (How can all my tasks be ‘combined’ to fit the time and cover all outcomes?) 22/07/12
    8. 8.  Start with the outcomes we intend students to learn (ILO), and align teaching and assessment to those outcomes. Outcome statements contain a learning activity (a verb) that students must perform to best achieve the outcome. Learning is constructed by what students do, not what we teachers do. Assessment concerns how well they achieve the intended outcomes, not how well they report back to us what we have told them.
    9. 9. (Source: Knight, 2001www.heacademy.ac.uk)
    10. 10. Summative assessment This usually happens at end – making a final judgement about learning in relation to outcome It usually has a mark associated. Assessment of learning.
    11. 11. Formative Assessment Assessment for learningused for diagnosis, growth and improvementin student learning takes place during learning and can beongoingfeedback is important – time consuming soshould promote student responsiveness.Electronic feedback sometimes more effectiveas it cannot be ignored (Winberg, 2008)
    12. 12. Think about feedback in assessmentAs a teacher: What type of feedback do you give? What happens with it? What would you like to see happen with it?Cast your mind back to when you were a student … What types of feedback did you get as a student? What did you find helpful? What did you do with it? Vardi, I (2009) Improving Student Writing with Effective Feedback 2nd Annual SoTL Commons Conferenceconference March 2009
    13. 13. Getting feedback… (Winberg, 2008)
    14. 14. Good feedback (Winberg, 2008) Facilitates the development of self-assessment in learning (reflection) Encourages teacher-peer dialogue around learning Helps clarify what good performance is (goals, criteria, expected standards) Provides opportunities to close the gap between current and desired performance Delivers high quality information to students about their learning (individual or general) Encourages positive motivational beliefs and self-esteem Provides information to teachers that can be used to shape the teaching (from HEA guide Enhancing student learning through effective formative feedback, p. 2)
    15. 15. Formal and informal assessment Formal - structure and awareness of both lecturer and student that assessment is occurring and how results are documented. Usually more structured and planned. Informal - spontaneous and not necessarily documented e.g. suggestions for improvement or revision, further drafts, learning activities to elucidate if students grasp principles.
    16. 16. Reliability and validity Inter-rater reliability and intra-rater reliability Content validity – how closely do questions or assessment tasks relate to content/concepts and learning outcomes being assessed?
    17. 17. Traditional and alternative methods Traditional methods focus on the products of learning rather than the process and is high stakes; cumulative and summative; lecturer is assessor Alternative methods use authentic situations; focus on processes; use artifacts as evidence of student thinking (e.g. portfolios, blogs, podcasts, journals); are metacognitive; learning progress and growth over extended period; use self and peer assessment and feedback (critical friends).
    18. 18. Norm- and Criterion-ReferencedApproaches to Assessment Norm – comparing students to others in a group – bell curve
    19. 19. The UWC Assessment Policy Supporting student-centred learning – encouragement of student responsibility for own learning, within reason (e.g. workload) Assessment committees in each faculty Collegial engagement on assessment tasks Follow principles of good assessment practices: transparency, clear ILO’s, etc. Use of formative and summative assessment Integrated assessment – theory and practice
    20. 20. Assessing by grading with rubricsFor: Student’s performance is appropriately assessed against what they are intended to learn( - criterion-referenced. The final grade tells students what they have achieved and what they need for a better grade.Against: Requires a different mind set for some teachers. Initially more work in designing ILOs, suitable assessment tasks and rubrics, but once established is no more extra work than marking.
    21. 21. What is a rubric? It is a scoring guide or a set of expectations used to judge student performance. A rubric is a scoring guide or a set of expectations used to judge student performance. Char acteri stics are arran It shows students how well they have performed on an ged in level s, indic ating the degre e to whic ha stand ard has been met. assignment. It breaks the assignment into parts, using criteria and Rubri cs are espe cially usefu l for asse ssing comp lex and subje ctive subje cts. levels of performance required for the assignment. They provi It can be used for a wide range of tasks (essays, de worki ng guide s for both teach ers and stude nts and are usual ly hand ed out befor e the assig nmen t so that stude nts research projects, oral presentations, portfolios, etc.) know what is expe cted. It is especially useful for assessing complex and subjective subjects. 21
    22. 22. Why use rubrics? Learners know exactly what is expected; there are clear targets and expectations. Rubrics protect against evaluator bias because they are consistent. Rubrics evaluate on the sum of a full range of criteria rather than a single numerical score. Rubrics empower students because they can use them to develop their abilities. Rubrics can be created for any content area and can be modified easily for various grade levels.
    23. 23. Parts of the rubric A rubric is a grid made up of four basic parts:a task description (the actual assignment whichinvolves performance expected of the student)a scale of the levels of achievement (marks ordescriptions of levels of achievement)the dimensions of the assignment (a breakdown ofthe skills/knowledge involved in the assignment)descriptions of what constitutes each level ofachievement (specific feedback) 07/22/12 Induction workshop for UWC staff 10 & 11 23 March 2010
    24. 24. TitleTask Description Scale Level 1 Scale level 2 Scale level 3Dimension 1Dimension 2Dimension 3Dimension 4 07/22/12 Induction workshop for UWC staff 10 & 11 24 March 2010
    25. 25. Checklist for a good rubric:____ Rubric Do the categories reflect the major learning Categories objectives?____ Levels Are there distinct levels which are assigned names and mark values?____ Criteria Are the descriptions clear? Are they on a continuum and allowing for student growth?____ Student-friendly Is the language clear and easy for students to understand?____ Teacher-friendly Is it easy for the teacher to use?____ Validity Can the rubric be used to evaluate the work? Can it be used for assessing needs? Can students easily identify growth areas needed?
    26. 26. References Butcher, C., Davies, C. & Highton, M. (2006). Designing Learning: From Module Outline to Effective Teaching. London & New York: Routledge. Chapter 6, pp.93-129. Driscoll, A. & Wood, S. (2007) Developing Outcomes-based Assessment for Learner- centred Education: A Faculty Introduction.Virginia:Stylus. Chapter 4 Irons, A. (2008). Enhancing Learning Through Formative Assessment and Feedback. London and New York: Routledge Knight, P. (2001). A briefing on key concepts: Formative and summative, criterion and norm-referenced assessment. Assessment Series No. 7. Generic Network: Learning and Teaching Support Centre Maki, P.L. (2004). Assessing for Learning:Building a Sustainable Commitment Across the Institution. Sterling: Stylus. Chapter 5 119-152. McNamara, J. & Burton, K. (2009). Assessment of Online Discussion Forums for Law Students. Journal of Teaching and Learning Practice.6(2): http://ro.uow.edu.au/jutlp (accessed 18 August, 2010). O’Donavon, B., Price, M. & Rust, C. (2004) Know what I mean? Enhancing student understanding of assessment and criteria. Teaching in Higher Education, 9(3):325-335 Stevens, D.D. & Levi, A.J. (2005). Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning. Sterling: Stylus. Chapter 3, pp. 29-46. 07/22/12 Induction workshop for UWC staff 10 & 11 26 March 2010
    27. 27. Do we have a sense of1. Assessment and alignment2. Forms of assessment3. Assessments and marks4. Reliability and validity5. Traditional and alternative assessment6. Norm and criterion-referenced assessment7. Rubrics?

    ×