More Related Content
What's hot
What's hot (14)
Viewers also liked
Viewers also liked (7)
Similar to Tests and Quizzes (Samigo) 2011
Similar to Tests and Quizzes (Samigo) 2011 (20)
Recently uploaded
Recently uploaded (20)
Tests and Quizzes (Samigo) 2011
- 1. Tests & Quizzes (aka SAMigo) Christine Doherty, Stanford University June 16, 2011
- 2. Agenda Future of T&Q: Development Dilemmas Community Contribution 2.8 Highlights 2.9 Plans Q&A
- 3. Future of T&Q: Development Dilemmas
- 4. Guiding Principles Flexibility/Freedom Reliability/Error-Free
- 5. Development Focus Improve UI/Interaction Add New Features
- 6. Project Management Stanford as Gatekeeper Modularize/Distribute
- 8. New Release Process http://confluence.sakaiproject.org/confluence/display/SAM
- 9. Rationale User experience reviews are critical: New features can cause clutter in UI, confusion, bad user experience New features can clash with existing or planned features and lead to unexpected behavior Stanford only has one designer to tackle the workload, so we need help
- 10. Requirements for Contributions Clear specs are a must: Rationale for new feature/enhancement Description of expected behavior Documentation of all affected screens Screen shots desirable
- 11. Timeline
- 12. 2.8 Highlights
- 13. RespondusIntegration before 2.8 Respondus Sakai – T&Q Question Types Changed Missing Metadata
- 14. RespondusIntegration in 2.8 Sakai – T&Q Respondus QTI XML or Zip file from Respondus Imports to Sakai Tests & Quizzes
- 17. Respondus Integration Respondus Sakai – T&Q Respondus Sakai – T&Q
- 18. Record Average Score New Feature: ability to record average score for multiple submissions Redesigned UI for student’s Submitted Assessments list to accommodate displaying average scores Contributed by Rice University
- 28. Release Scores Only Based on feedback from instructors wanting to show students their scores without allowing them to see questions again (since questions are reused in subsequent tests) Solution is to separate out scores from rest of feedback options
- 29. Release Scores Only to Students
- 32. Send zero scores to Gradebookfor non-submission records Clash of scoring rules in Tests & Quizzes and Gradebook caused inability to set zero scores for non-submission records for inclusion in course grade calculation Solution is to allow instructors to explicitly enter a zero score for a non-submission record that gets sent to gradebook Patch contributed by Longsight
- 33. Send zero scores to Gradebookfor non-submission records
- 34. Send zero scores to Gradebookfor non-submission records
- 35. Send zero scores to Gradebookfor non-submission records
- 36. Send zero scores to Gradebook
- 37. Send zero scores to Gradebookfor non-submission records
- 38. Send zero scores to Gradebookfor non-submission records
- 39. Send zero scores to Gradebookfor non-submission records
- 40. Rich Text Entry for Short Answer/Essay Instructors from several institutions requested a way for students to enter rich text in essay questions for equations, foreign characters, or just general formatting Solution is to present students with the ability to use the FCKeditorin Short Answer/Essay questions. Contributed by Longsight
- 41. Rich Text Entry for Short Answer/Essay
- 42. Rich-text for Short Answer
- 43. Rich-text for Short Answer
- 44. Publish Action Based on feedback that Settings and Publish actions were redundant, both leading to Settings page Solution is to link Publish action to Publish Assessment page where settings are verified and notification added (if desired)
- 45. Publish Action
- 46. Publish Action
- 47. Publish Action
- 48. Publish Action
- 49. Remove Assessment Types Assessment Types often prove to be confusing to instructors, since they may not know what the labels mean and what settings go with them One solution is to remove any trace of assessment types with a sakai property (samigo.showAssessmentTypes=False) Contributed by Longsight
- 53. 2.9 Plans
- 54. Stanford contributions for 2.9 Restyling and rewriting of warning messages Reset selection and warning messages redraw at current question, instead of top of page Reworking automatic submission logic Validation for numeric response questions Count Characters button for Short Answer/Essay questions Render part information and images in Print view
- 55. Community contributions for 2.9 Redesign of main Assessments page (Longsight) Calculated question type (University of Dayton) Survey – Matrix of Choices question type (Rutgers) Extended Matching Item question type (University of Cape Town)
- 56. Redesign of Main Assessments Page
- 57. Redesign of Main Assessments Page
- 58. Contacts Questions and feedback: samigo-dev@lists.stanford.edu Lydia Li, Stanford University Project Manager lydial@stanford.edu Karen Tsao Samigo Developer ktsao@stanford.edu Homepage: http://confluence.sakaiproject.org/confluence/display/SAM
- 59. Online Final Exams Using Sakai at Stanford University Jamie Tsui 9:45 AM - 10:15 AM in BeaudryB
- 60. Questions?
Editor's Notes
- Since tools is growing more complex and the list of new feature requests is also growing, we need to step back and look at the big picture, decide what will guide our future decisions, how we want to focus our efforts, and how we will sustain the amount of work required to keep this tool going and expanding.
- Example of good spec on the confluence page for the new release process
- Proposed timeline; since some dates already passed, we can still consider contributions if work is done or nearly done.Adjust timeline (in future releases) if you feel your feature will require more than 4-6 weeks of coding.Best to have design specs before coding, so usability review can make needed changes early in the process.
- Prior to 2.8, an assessment exported from Respondus would not import correctly into T&Q.Question types would change. Metadata disappeared.
- In 2.8, an assessment exported from Respondus in QTI XML or Zip format will import correctly in T&Q.
- Use Import Assessment button in main page.
- New in Import page: select between QTI file from Sakai system and QTI file from Respondus
- Export from T&Q will not import correctly into Respondus, because format of tags in T&Q doesn’t match Respondus.We do not have plans to do this work. If another institution wants to implement this functionality, we will apply patch.
- New average score option appears in Grading section of Settings page, along with Highest Score and Last Score optionsThis option only works with if multiple submissions are allowed. If limited to one submission, a warning message will appear when settings are saved.
- New average score option appears in Grading section of Settings page, along with Highest Score and Last Score optionsThis option only works with if multiple submissions are allowed. If limited to one submission, a warning message will appear when settings are saved.
- All Submissions page looks the same
- Average Submission page (default view in Total Scores) drops a lot of columns that apply to individual submissions:Link to student’s responses, Submit Date column, Adjustment column, Comment column
- Average score will be sent to gradebook, if that option is selected.
- Submitted Assessments display redesigned to enable students to view average score, while still being able to see scores and feedback for each submission.Previously, each submission record was an independent row in the table.Score type is shown in parenthesis after the score value in the Recorded Score column.
- (Previous design of Submitted Assessments list)Submitted Assessments display redesigned to enable students to view average score, while still being able to see scores and feedback for each submission.Previously, each submission record was an independent row in the table.Score type is shown in parenthesis after the score value in the Recorded Score column.
- Submitted Assessments list when collapsed to show only recorded scores
- Submitted Assessments list for an assessment with no feedback or scores released.
- In Feedback section of Settings, releasing scores has been separated from other feedback (questions, responses, etc.), but still considered a feedback component.The Release Scores option has to be combined with an option to release feedback. If ‘No Feedback’ selected, the scores will not display either.Previously, the only way to release scores viewable in the Tests & Quizzes tool was to release feedback that included the questions by default.The other way was to turn off feedback and send the scores to the Gradebook and have students find scores there.
- While working on the feedback settings, we took the opportunity to clarify some settings whose meaning has been unclear for users.Delivery Dates – Due Date vs. Retract Date often proved confusing for instructors. There were different interpretations of what retraction means.Late Submission – Clarified this is a ‘one last chance to submit’ after the due date, not allowing students to submit multiple times after due date.
- Prior to 2.8, T&Q shows zero score for non-submission records, but sent a null score to Gradebook (displaying as dash) to differentiate students who hadn’t taken assessment from those who really got zero points from taking assessment.In 2.5, Gradebook changed their logic to not include null scores in the calculation of Course Grade.There was no way for instructors to send a zero to the Gradebook, as the T&Q Scores page already said zero and T&Q scores can’t be edited within Gradebook.
- Prior to 2.8, T&Q shows zero score for non-submission records, but sent a null score to Gradebook (displaying as dash) to differentiate students who hadn’t taken assessment from those who really got zero points from taking assessment.In 2.5, Gradebook changed their logic to not include null scores in the calculation of Course Grade.There was no way for instructors to send a zero to the Gradebook, as the T&Q Scores page already said zero and T&Q scores can’t be edited within Gradebook.
- Now, Adjustment column shows dash.
- Instructor can enter zero in that column and click Update to send a zero to the Gradebook.
- That zero score will be reflected in the Gradebook.
- Although issue has a technical fix, the UI could be more clear to let instructors know they need to enter a zero score.We will be adding an additional patch as part of 2.8 maintenance branch to have dashes appear in Score and Final Score as well for non-submission records.Example from Stanford instance, CourseWork. Tests & Quizzes called Assignments at Stanford
- When instructor enters a zero in Adjustment column and updates, the Final Score will change to zero.Score column will stay as dash as reminder there is no actual submission and, therefore, no original score.
- The same ‘Show/Hide Rich-Text Editor’ toggle used in assessment authoring now appears in delivery for short answer questions.The reason not to load the editor at first is that loading several editors on the same page (if several short answer questions on a page) would cause slow performance.
- FCKeditor can be toggled on and off as long as there is no formatting entered yet.
- Once any formatting exists, which includes line breaks, and the data has been saved by navigation buttons or links, the page will always have the editor loaded when revisited.The toggle disappears.
- Originally, we did not have a Publish action. Only way to publish was to go to Settings. This was not intuitive enough for users, so we made a Publish action.Publish button was still on Settings page, however, making actions redundant.
- Originally, we did not have a Publish action. Only way to publish was to go to Settings. This was not intuitive enough for users, so we made a Publish action.Publish button was still on Settings page, however, making actions redundant.
- Originally, we did not have a Publish action. Only way to publish was to go to Settings. This was not intuitive enough for users, so we made a Publish action.Publish button was still on Settings page, however, making actions redundant.
- Now, Publish action goes right to Publish Assessment page. Settings are expressed in prose form. Instructors can verify these and continue to publish or edit the settings if changes are needed.
- This patch will remove the Assessment Types link (and page) from the top navigation, the assessment type drop-down for creating a new assessment, and the Assessment Type Information section of the Settings page.
- This patch will remove the Assessment Types link (and page) from the top navigation, the assessment type drop-down for creating a new assessment, and the Assessment Type Information section of the Settings page.
- This slide shows what the Assessment Types page looks like for system admins (members of Administration Workspace).Some things to consider when deciding to enable this property:It is possible to just remove all the pre-defined assessment types (system admins) and that will apply to all sites. This will make the assessment type drop-down disappear from the Create a New Assessment part of main page. System admins will be able to adjust the default selections in Default Assessment Type, which affects all new assessments if type is not used. Easier to make changes than having to edit database. Keeping the Assessment Types page also allows instructors to create their own types, only visible to them.
- Survey Matrix – allows rating items against same set of answersEMI – Extended Matching Item, involves a list of items and a set of questions about the list. used for medical education
- Confusion about Pending vs. Published assessments, renamed to working copies and published copies, explained what that meansJavascript tabbed display makes greater visual clarity between two functional areas, saves screen real estateCreate an assessment organization also improved
- Filters to allow for showing All, Active, Inactive (with explanations)Default sorting has active assessments on top