Selectingn course evaluation
•Download as PPTX, PDF•
0 likes•507 views
This document discusses course evaluation and grading. It emphasizes aligning evaluation measures with course learning outcomes, using valid measures, and clearly describing assignments and grading policies to students. Examples are provided to illustrate aligning evaluations with outcomes in various subjects. Issues with non-alignment are also discussed. The document concludes with next steps for developing strong evaluation measures, such as creating clear assignment descriptions, rubrics, point scales, and grade cutoffs.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Selectingn course evaluation
Similar to Selectingn course evaluation (20)
Recently uploaded
Recently uploaded (20)
Selectingn course evaluation
- 1. Course Evaluation/Grading Mary Hazzard, Ph.D. Assessment Fellow
- 2. Learning Outcomes The learner will be able to: Align course learning activities/grading with Course Learning Outcomes (CLO) Determine if a measure is valid. Develop course evaluation/grading measures
- 3. Purpose of Evaluation Improve student learning (formative) Assign grades to students that reflect degrees of learning (summative)
- 4. Course evaluation/grading Types of assignments selected are valid measures of the stated CLO. Grading policy is stated clearly. Rubrics are used to evaluate student work and available to students. Instruments selected are varied, sequenced, and valid for the content being evaluated. A signature assignment may be included in the course.
- 5. Validity Degree to which an instrument accurately evaluates the specific concept(s) that the individual is attempting to measure. Degree to which an instrument accurately evaluates the specific CLO or unit learning outcome.
- 6. Validity Types Face Validity Criterion related validity Construct Validity Content Validity
- 7. Face Validity How a measure or procedure appears. Does it seem like a reasonable way to gain the information the persons are attempting to obtain? Does it seem well designed? Does it seem as though it will work reliably?
- 8. Criterion related validity Used to demonstrate the accuracy of a measure or procedure by comparing it with another measure or procedure which has been demonstrated to be valid.
- 9. Construct Validity Agreement between a theoretical concept and a specific measuring device or procedure.
- 10. Content Validity Extent to which a measurement reflects the specific intended domain of content
- 11. Direct Measures for Grading This is not meant to be a complete listing. Group or individual project Group or individual presentation Debate Threaded discussion Exams (Weekly, Midterm, final, pretest, posttest) Film or videotape Multimedia presentation Second Life assignment Written assignment(s) Role Play Panel discussion Case study
- 12. Direct Measures continued Demonstration Web search and evaluation Create models, designs, plans, etc. Evaluation report Research project Dissection report Laboratory work Journal Community service learning Fieldwork Practicum
- 13. Indirect Measures May or may not be included in the course grade Reflection journal (generally graded) Survey Interview
- 14. Alignment: Example - Biology CLO: Describe fundamental concepts in chemistry and biochemistry, including the properties of water, important organic molecules, acids and bases. Evaluation: Midterm and comprehensive final (multiple choice) Threaded Discussion Research paper on important organic molecules
- 15. Alignment: Example - Accounting CLO: Analyze one or more cases that involve accounting fraud, litigation and auditor liability. Evaluation: Final exam (case analysis) Case analysis written assignment Threaded discussion analyzing a case
- 16. Alignment: Example - Education CLO: Integrate knowledge and skills in creating and maintaining effective learning environments for all P12 students. Evaluation: Supervisor evaluation in practicum Oral presentation in P12 classroom. Classroom management plan
- 17. Alignment: Example - Education CLO: Design and build a learning environment (simulation) in a virtual world Evaluation: Design and build a learning environment in Second Life. Oral presentation of the learning environment in a virtual world.
- 18. Problems with non Alignment No clear evidence of students ability to attain the CLO. May underestimate the effectiveness of instruction. Students may be engaged in the activities but may not achieve the intended results.
- 19. Next Steps Develop the clear assignment description for student. Develop rubrics to use in grading assignment. Determine the number of points for each assignment. Determine if plus and minus will be used in grading. Determine how the points correspond to a specific grade e.g. how many points for an A, B, or C.
- 20. Summary Align evaluation measures with the CLO and/or unit learning outcomes Identify valid evaluation measures Describe the assignment Develop evaluation rubrics Determine points for evaluation measures. Correlate points with specific grades.
Editor's Notes
- This module includes content related to course evaluation and grading.
- Learning outcomes for this module include the ability to:Align course learning activities/grading with Course Learning Outcomes (CLO)Determine if a measure is valid.Develop course evaluation/grading measures
- There are two main purposes of evaluation in a course. To improve student learning is a formative measure to increase student learning. Assignment of grades or summative evaluation involves assigning grades to the student that reflect the degree of learning If the instruments are not aligned with the objectives then they do not provide clear evidence of intended student learning.
- The types of assignments selected must be a valid measure of the stated CLO’s for the course. In addition, the grading policy must be clearly stated and rubrics must be used to evaluate student work and this must be available to the student. Any instruments/assignments selected in a course should be varied. Sequenced, and valid for the content being evaluated. Using a variety of assignments allows students with different types of learning to be able to succeed. Not all students are great test takers nor are all students good at oral presentations or writing. A signature assignment may be included in the course and if this occurs then refer to the module on signature assignment. The signature assignment is deigned to measure a PLO and may also be used in evaluation/grading in the course.
- Validity is the degree to which an instrument accurately evaluates the specific concept(s) that the individual is attempting to measure. In terms of course development it is the degree to which an instrument accurately evaluates the specific CLO or unit learning outcome.
- Types of validity include: face validity, criterion related validity, construct validity, and content validity. We will briefly look at each of these types of validity.
- Face validity is concerned with how a measure or procedure appears. Does it seem like a reasonable way to gain the information the persons are attempting to obtain? Does it seem well designed? Does it seem as though it will work reliably? Does it seem to accurately measure what the CLO indicates the student is to be able to do as a result of the class. One thing to look at is the action verb in the CLO and does the assignment ask the student to do what the CLO indicates.
- Criterion related validity is used to demonstrate the accuracy of a measure or procedure by comparing it with another measure or procedure which has been demonstrated to be valid. This is not used as frequently in determining validity of the CLO measure as it might be for a PLO if there is a test the graduate must take for example to become licensed.
- Construct validity is the agreement between a theoretical concept and a specific measuring device or procedure.
- Content validity is the extent to which a measurement reflects the specific intended domain of content. This type of validity may be used to determine validity of CLO measures.
- Some examples of direct measures that can be used for grading include:Group or individual projectGroup or individual presentationDebateThreaded discussionExams (Weekly, Midterm, final, pretest, posttest)Film or videotapeMultimedia presentationSecond Life assignmentWritten assignment(s)Role PlayPanel discussionCase study.
- Additional direct measures include:DemonstrationWeb search and evaluationCreate models, designs, plans, etc. Evaluation reportResearch projectDissection reportLaboratory workCommunity service learningFieldworkPracticumThis list is not meant to be all inclusive and you may think of other creative ways to evaluate a stduent’s learning.
- Some faculty also like to use indirect measures and these may or may not be included in the course grade. The reflective journal is generally graded but a survey or interview may or may not be graded.
- Alignment of the CLO and evaluation method is very important and several examples will be provided. The first is from biology and the CLO is:Describe fundamental concepts in chemistry and biochemistry, including the properties of water, important organic molecules, acids and bases. Some examples of evaluation methods that might be used include: Midterm and comprehensive final (multiple choice)Threaded DiscussionResearch paper on important organic moleculesYou will note that the key verb is describe so the methods used should allow the student to describe. The methods listed all allow the student to describe, discuss, or select the best description such as in a multiple choice exam.
- The second example is a CLO from accounting. The CLO is: Analyze one or more cases that involve accounting fraud, litigation and auditor liability. The key word is analyze so the evaluation methods should allow the student to analyze cases that include fraud, litigation or auditor liability. Some evaluation measures might include: Final exam (case analysis)Case analysis written assignmentThreaded discussion analyzing a case
- The third example is from education and the CLO is: Integrate knowledge and skills in creating and maintaining effective learning environments for all P12 students. Here you have to look carefully at the wording of the CLO. The student is actually being asked to create and maintain a learning environment integrating the knowledge and skills. Some examples of evaluation then might include: Supervisor evaluation in the practicum when the student is actually student teachingOral presentation in P12 classroom. This might be observed by the university faculty and evaluated.Classroom management plan. The student might have to present a written classroom management plan prior to actually implementing the plan.
- The last example is also taken from education and the CLO is: Design and build a learning environment (simulation) in a virtual world. Here the key words are design and build and some examples of evaluation measures might include: Design and build a learning environment in Second Life, and an oral presentation of the learning environment in a virtual world.
- Problems that can occur if the CLO are not aligned with the evaluation measures include:No clear evidence of students ability to attain the CLO.May underestimate the effectiveness of instruction.Students may be engaged in the activities but may not achieve the intended results.
- Once the evaluation measures have been aligned with the CLO then it is time to develop the written directions for the student to follow to achieve the assignment. In addition, develop the rubrics that will be used to evaluate the assignment. Refer to the module on rubrics for more information on this. Both the assignment description and the rubric should be available in the course for the student to review. This allows the student to self-evaluate their work and to know precisely what is required for a specific grade.You also will need to determine the number of points for each assignment and if you will be using plus and minus grades such as B+, B, B-. Determine how the points will correspond to a specific grade e.g. how many points for an A, B, or C.
- In summary:Align evaluation measures with the CLO and/or unit learning outcomesIdentify valid evaluation measuresDescribe the assignmentDevelop evaluation rubricsDetermine points for evaluation measures.Correlate points with specific grades.