Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Creating Assessments


Published on

This file accompanies the "Creating Assessments" session at the Academic Impressions conference titled "A Comprehensive Approach to Designing Online Courses", Dec 3-4, 2007, Austin TX

Published in: Business, Education
  • Be the first to comment

  • Be the first to like this

Creating Assessments

  1. 1. A Comprehensive Approach to Designing Online Courses Creating Assessments Christina Sax ACADEMIC ■ IMPRESSIONS
  2. 2. <ul><li>Appreciate the process, importance, and utility of assessment </li></ul><ul><li>Understand the differences between assessment, evaluation, testing, grading </li></ul><ul><li>Explore the various factors that influence the creation of assessments </li></ul><ul><li>Review the basic principles of creating assessments </li></ul><ul><li>Explore various aspects of assessment in the online environment </li></ul>Session Goals
  3. 3. <ul><li>What is assessment? </li></ul><ul><li>Why is it important? </li></ul>Understanding Assessment Activity Sheet 1
  4. 4. <ul><li>An ongoing systematic process that: </li></ul><ul><li>Sets clear expectations of student learning in the form of objectives </li></ul><ul><li>Provides sufficient opportunities for students to achieve these expectations </li></ul><ul><li>Gathers evidence of student learning </li></ul><ul><li>Uses this information to improve teaching and learning </li></ul>Assessment (verb) – A Dynamic Process Handout 1
  5. 5. <ul><li>An instrument used to: </li></ul><ul><li>Identify what students have learned </li></ul><ul><li>Measure the match between the learning objectives and students’ attainment of those objectives </li></ul>Assessment (noun) – Strategies & Tools Handout 1
  6. 6. <ul><li>Provides feedback to students </li></ul><ul><ul><li>identify strengths/weaknesses </li></ul></ul><ul><ul><li>adjust learning practices & strategies </li></ul></ul><ul><li>Documents individual student achievement & progress </li></ul><ul><li>Impacts decision making of prospective students </li></ul>Importance to Students
  7. 7. <ul><li>Provides feedback for improvement of teaching, support systems, curriculum </li></ul><ul><li>Ensures pre-reqs in course sequencing </li></ul><ul><li>Documents success of curriculum </li></ul><ul><li>Supports strategic planning/decision making (funding, resource allocation, etc) </li></ul><ul><li>Supports reaccreditation </li></ul><ul><li>Addresses external call for accountability </li></ul>Importance to Faculty and Institutions
  8. 8. <ul><li>Learning Objective </li></ul><ul><li>(what students should be able to do) </li></ul><ul><li>?? ?? Assessment </li></ul><ul><li>Learning Outcome </li></ul><ul><li>(what students can actually do) </li></ul>Measuring the Gap
  9. 9. <ul><li>Assessment </li></ul><ul><li>Evaluation </li></ul><ul><li>Testing </li></ul><ul><li>Grading </li></ul>What’s the Difference? Activity Sheet 2
  10. 10. <ul><li>Evaluation = Judgment </li></ul><ul><ul><li>Interpreting assessment results & using them to make a decision </li></ul></ul><ul><ul><li>Investigating the quality or worth of an entity to make a decision </li></ul></ul><ul><li>Evaluation = Assessment </li></ul><ul><ul><li>Determining the match between objectives and outcomes </li></ul></ul>Assessment vs Evaluation
  11. 11. <ul><li>Testing is … </li></ul><ul><li>a systematic measurement of knowledge, skills, abilities </li></ul><ul><li>a part of the assessment process </li></ul>Assessment vs Testing
  12. 12. <ul><li>Part of the assessment process </li></ul><ul><li>Grades influenced by: </li></ul><ul><ul><li>student learning, attendance, participation, timely submission of work </li></ul></ul><ul><li>A grade may indicate overall level of understanding, but does not indicate of which specific knowledge, skills, abilities </li></ul>Assessment vs Grading
  13. 13. <ul><li>Focus on learning objectives </li></ul><ul><li>Consider best practices </li></ul><ul><li>Define the purpose </li></ul><ul><li>Select assessment strategies </li></ul><ul><li>Consider the end results </li></ul><ul><li>Link to knowledge/cognition taxonomies </li></ul><ul><li>Select & design assessment tool </li></ul><ul><li>Select & design grading criteria/tools </li></ul>Steps to Creating Assessments
  14. 14. Step 1 – Learning Objectives
  15. 15. <ul><li>Align assessments, objectives, questions </li></ul><ul><li>Seek useful information </li></ul><ul><li>Conduct on an ongoing basis </li></ul><ul><li>Use a variety of methods </li></ul><ul><li>Communicate purpose, expectations, grading criteria, feedback loops </li></ul>Step 2 - Best Practices in Assessment
  16. 16. <ul><li>Determine pre-req knowledge </li></ul><ul><li>Provide learning opportunity </li></ul><ul><li>Provide feedback to students </li></ul><ul><li>Student self-assessment </li></ul><ul><li>Document progress </li></ul><ul><li>Document outcomes </li></ul><ul><li>Feedback to faculty </li></ul><ul><li>Mid-course correction </li></ul>Step 3 - Define the Purpose
  17. 17. <ul><li>Formative vs summative </li></ul><ul><li>Direct vs indirect evidence </li></ul><ul><li>Learning processes vs outcomes </li></ul><ul><li>Objective vs subjective </li></ul><ul><li>Quantitative vs qualitative </li></ul><ul><li>Performance vs traditional </li></ul><ul><li>Local vs commercial/published </li></ul><ul><li>Embedded vs add-on </li></ul>Step 4 - Assessment Strategies
  18. 18. <ul><li>Formative </li></ul><ul><ul><li>Occurs while student learning is taking place </li></ul></ul><ul><ul><li>Purpose is to improve teaching & learning </li></ul></ul><ul><ul><li>Students receive feedback beyond a grade on strengths & weaknesses </li></ul></ul><ul><li>Summative </li></ul><ul><ul><li>Occurs at the end of a course (or program) </li></ul></ul><ul><ul><li>Purpose to document student learning </li></ul></ul><ul><ul><li>Students may or may not receive feedback other than a grade </li></ul></ul>Formative & Summative Assessment
  19. 19. <ul><li>Objective </li></ul><ul><ul><li>Usually one correct answer </li></ul></ul><ul><ul><li>Can be lower order and higher order processing </li></ul></ul><ul><ul><li>Quickly provide information on broad range or targeted content areas </li></ul></ul><ul><ul><li>No professional judgment needed to score </li></ul></ul><ul><ul><li>Do not require lots of time to grade </li></ul></ul><ul><ul><li>Results easily summarized by a single number </li></ul></ul><ul><li>Subjective </li></ul><ul><ul><li>May be multiple possible answers </li></ul></ul><ul><ul><li>Assess many important skills at once </li></ul></ul><ul><ul><li>Assess skills d irectly </li></ul></ul><ul><ul><li>Are more active & promote deep, lasting learning </li></ul></ul><ul><ul><li>Students can provide lots of information on broad range of objectives at once </li></ul></ul><ul><ul><li>Requires professional judgment to score </li></ul></ul><ul><ul><li>Allows for scoring nuances </li></ul></ul><ul><ul><li>Require lots of time to score </li></ul></ul><ul><ul><li>Summary of results requires narrative </li></ul></ul>Objective & Subjective – Pros & Cons
  20. 20. <ul><li>Standards-based </li></ul><ul><ul><li>Are students meeting your standards? </li></ul></ul><ul><li>Benchmarking </li></ul><ul><ul><li>How do student compare to peers? </li></ul></ul><ul><li>Best Practice </li></ul><ul><ul><li>How do students compare to best of their peers? </li></ul></ul><ul><li>Value-Added </li></ul><ul><ul><li>Are students improving? </li></ul></ul><ul><li>Longitudinal </li></ul><ul><ul><li>Is program improving? </li></ul></ul><ul><li>Capability </li></ul><ul><ul><li>Are students doing as well as they can? </li></ul></ul>Step 5 - Consider the End Results
  21. 21. <ul><li>Consider level of knowledge and processing </li></ul><ul><ul><li>Lower order vs higher order </li></ul></ul><ul><li>Align objectives (verbs) and assessment </li></ul><ul><li>(Developing Objectives session – Handout 3) </li></ul>Step 6 - Link to Knowledge & Cognition Taxonomies
  22. 22. <ul><li>Objective: Create a sterile environment. </li></ul><ul><li>Are these assessments aligned? Why/why not? </li></ul><ul><ul><li>Write a definition of the phrase “sterile environment”. </li></ul></ul><ul><ul><li>Arrange these ten steps in the proper order to ensure a sterile environment is achieved. </li></ul></ul><ul><ul><li>Create an instruction manual titled “Sterile Environment Procedures”. </li></ul></ul><ul><ul><li>Write an essay describing the importance of sterile conditions in surgical settings. </li></ul></ul><ul><ul><li>Write an essay describing how you would create a sterile environment under suboptimal conditions. </li></ul></ul>Align Objectives, Verbs, Assessments
  23. 23. <ul><li>The toolbox includes WAY MORE </li></ul><ul><li>than exam questions !!!! </li></ul>Step 7 - Assessment Tools Handout 2
  24. 24. <ul><li>Examples </li></ul><ul><ul><li>List of criteria & points </li></ul></ul><ul><ul><li>Rubrics </li></ul></ul><ul><li>Uses </li></ul><ul><ul><li>States clear expectations; guide for students </li></ul></ul><ul><ul><li>Guide to faculty for fair and accurate scoring </li></ul></ul><ul><ul><li>Vehicle for feedback to students </li></ul></ul>Step 8 - Grading Criteria & Tools
  25. 25. <ul><li>Use to plan, design, and create assessments </li></ul><ul><li>Based in link to specific learning objectives </li></ul><ul><li>State clear and thorough directions and expectations for students </li></ul><ul><li>State grading standards, criteria, rubrics </li></ul>Assessment Mapping Grid Handout 3
  26. 26. <ul><li>Work in teams </li></ul><ul><li>Use Activity Sheet 4 </li></ul><ul><li>Review the NURS246 course description & learning objective </li></ul><ul><li>Use the Assessment Mapping Grid to plan, select, and design an online assessment </li></ul><ul><li>Group reports (designate your spokesperson) </li></ul>Activity – Create an Online Assessment
  27. 27. <ul><li>Describe the contents of your Assessment Mapping Grid. </li></ul><ul><li>What challenges did you encounter? </li></ul><ul><li>How can you overcome these challenges? </li></ul>Activity – Group Reports
  28. 28. <ul><li>Well suited to the online environment </li></ul><ul><li>Not well suited to the online environment </li></ul>Characteristics of Online Assessments … Activity Sheet 3B
  29. 29. <ul><li>Efficient management, collection, transfer of assessment information </li></ul><ul><li>Possibility of reduced grading time </li></ul><ul><li>Increased opportunities for students: </li></ul><ul><ul><li>Self-assessment, ongoing & rapid feedback, knowledge of overall status through online gradebook </li></ul></ul><ul><li>Course management system tools better facilitate some assessment practices </li></ul>Online Assessment – Benefits of Technology
  30. 30. <ul><li>Database to track, monitor, document </li></ul><ul><li>Easy access to development & deployment of multiple assessment tools with automatic & instant feedback </li></ul><ul><li>Multiple communication tools to </li></ul><ul><ul><li>Document & track progress, formally & informally </li></ul></ul><ul><ul><li>Provide/receive continuous feedback from faculty & student peers </li></ul></ul><ul><li>Content tools to develop projects & problem solving tasks, independently & in teams </li></ul>Course Management System Tools
  31. 31. <ul><li>Discussions are important form of ongoing informal & formal assessment </li></ul><ul><li>Greater opportunity for full involvement online vs f2f (# students, depth of discussion) </li></ul><ul><li>Easy to track, document, grade student discussion online vs f2f </li></ul><ul><li>Easy to model/provide examples of “quality” responses and dialogue </li></ul><ul><li>Easy to conduct multiple conversations at once </li></ul>Discussion Boards
  32. 32. <ul><li>Common online text </li></ul><ul><li>Reconstructive writing prompt sends reader back to text </li></ul><ul><li>Constructive writing prompt promotes thinking beyond </li></ul><ul><li>Inquiry prompt </li></ul><ul><li>Directions to interact with fellow students </li></ul><ul><li>Expectations for students’ work </li></ul><ul><li>Directions for mechanical steps to accomplish tasks </li></ul><ul><li>Personal/human feel </li></ul>Discussion Board Model (Hofmeister & Thomas, 2005)
  33. 33. <ul><li>Course management system provides tools to structure & facilitate group work online vs f2f </li></ul><ul><li>Allows faculty to monitor progress of group work, and individual contributions to group </li></ul><ul><li>Allows for three forms of feedback – faculty, self, peer </li></ul>Group/Team Work
  34. 34. <ul><li>Automated grading of objective question types </li></ul><ul><li>allows for: </li></ul><ul><ul><li>Ongoing formal assessment </li></ul></ul><ul><ul><li>Ongoing informal self-assessment </li></ul></ul><ul><ul><li>“ Best attempt” assessment </li></ul></ul><ul><ul><li>Varied assessment (question types) </li></ul></ul><ul><ul><li>Rapid feedback </li></ul></ul><ul><ul><li>Adaptation to individual student situations, starting and progress levels </li></ul></ul><ul><ul><li>Reduced grading time </li></ul></ul>Quizzing/Testing Tools - Objective
  35. 35. <ul><li>Technology allows for: </li></ul><ul><ul><li>Reduced grading time & faster feedback </li></ul></ul><ul><ul><ul><li>Cut & paste common comments, rather than handwritten for each student </li></ul></ul></ul><ul><ul><li>Multiple forms of feedback </li></ul></ul><ul><ul><ul><li>Extensive typed text </li></ul></ul></ul><ul><ul><ul><li>Hand mark-ups </li></ul></ul></ul><ul><ul><ul><li>Audio clips </li></ul></ul></ul>Quizzing/Testing Tools - Subjective
  36. 36. <ul><li>What do you want students learn? </li></ul><ul><li>What do you want to know about the learning? </li></ul><ul><li>Why do you want to know this? </li></ul><ul><li>How do you determine this? </li></ul><ul><li>How will you use this information? </li></ul><ul><li>Assessment requires time, </li></ul><ul><li>reflection, and planning. </li></ul>Assessment Wrap-Up