The Development and Usability Evaluation of a Standards-Based Grading Tool for Faculty


Published on

The Development and Usability Evaluation of a Standards-Based Grading Tool for Faculty

Published in: Education, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Development and Usability Evaluation of a Standards-Based Grading Tool for Faculty

  1. 1. The Development and Usability Evaluation of a Standards-Based Grading Tool for Faculty<br />Alaa Sadik, Sultan Qaboos University (OMAN)<br />
  2. 2. Introduction<br />Grades are the common method for reporting student performance across universities.<br />Today, universities tend to use standards-based assessment approach to assess student performance.<br />
  3. 3. Introduction<br />Sultan Qaboos University(SQU):<br />a cross-cultural organization, <br />brings together hundreds of instructors from around the world.<br />the largest academic community, consists of 9 colleges,<br />awide variance with regard to grading practices.<br />
  4. 4. Problem of the Study<br />Many instructors believe that:<br />Preparing, scoring, grading, and reporting student academic performance are each extremely exhausting and time-consuming tasks.<br />It is difficult to perform standards-based assessment according to the university quality assurance policy and requirements.<br />
  5. 5. The Need<br />Therefore,<br />the need was emerged to: <br />Use/develop a standards-based assessment tool that allows for more accurate and relevant grading and reporting.<br />
  6. 6. Research Questions<br />How effective (useful) is the assessment tool in documenting, grading and reporting student performance?<br />How efficient (easy to use) is the assessment tool as perceived by users?<br />What is the overall satisfaction of users toward the use of the assessment tool?<br />What individual difference variables influence instructors’ perceived usability of the assessment tool?<br />
  7. 7. Purpose of the Study<br />The main purpose of this study was to design, develop and evaluate a standards-based assessment tool for instructors at SQU.<br />
  8. 8. Method<br />Development of the assessment tool<br /> A review of the literature <br />A series of interviews and focus groups <br />were conducted, to determine the gap between the existing grading practices of faculty and those that are needed for the assessment tool.<br />
  9. 9. Method<br />The assessment tool should<br />Support the University grading scale and generate students’ grades automatically; <br />Enable qualitative assessment of student performance;<br />Attach content and performance standards to grade sheets; <br />Have a built-in e-mail function for communicating grades with students;<br />Facilitate total point and percent weight of scores and assignments; and<br />Generate course statistics for distributions, correlations and variances.<br />
  10. 10. Method<br />
  11. 11. Method<br />
  12. 12. Method<br />
  13. 13. Method<br />
  14. 14. Method<br />Sample<br />116 faculty members <br />from 4 colleges at SQU.<br />
  15. 15. Method<br />Procedures<br />Participants were asked to perform a series of specific tasks in RealGrade, each of which had several subtasks.<br />
  16. 16. Method<br />Instruments<br />Grading tool usability questionnaire<br />To solicit participants’ perceptions toward the effectiveness, efficiency and acceptance of the tool. <br />Instructor interviews<br />To gain a thorough understanding of the use of RealGrade and to provide rich detail and insights into instructors’ experiences.<br />
  17. 17. Results: Questionnaire<br />Effectiveness<br />Participants strongly agreed or agreed (m=3.93) that RealGrade :<br /> Is effective in facilitating the process of standards-based assessment, <br /> Has the tools needed to assess student performance and communicate grades with students, and <br /> Increases their productivity.<br />
  18. 18. Results: Questionnaire<br />Efficiency<br />Participants indicated that (m=3.34):<br /> Learning to use RealGrade takes a short time, <br /> The user interface menus and dialogue boxes are favorable, <br /> Weighting, assessing, and managing student assignments based on performance standards are very easy tasks.<br />
  19. 19. Results: Questionnaire<br />Satisfaction<br />The majority of participants (m=4.19) reported that they liked and felt comfortable with RealGrade as a tool to assess student performance.<br />
  20. 20. Results: Questionnaire<br />
  21. 21. Results: Questionnaire<br /> Participants with greater degrees of computer experience had higher perceptions of effectiveness, efficiency, and satisfaction with RealGrade.<br /> Participants who had less than five years of teaching experience found RealGrade more effective, efficient and satisfactory than those who had more than five or ten years of experience.<br />
  22. 22. Results: Interview<br />
  23. 23. Discussion and Conclusions<br />Differences among instructors are vital to the eventual acceptance and implementation of the standards-based assessment tool.<br />Further research is needed to develop report card to help instructors and students understand the standards-based assessment information included and to make it more comprehensible.<br />
  24. 24. Thank You<br />More information<br /><br />Contact <br /><br />