Tests and Quizzes (Samigo) 2011
Upcoming SlideShare
Loading in...5
×
 

Tests and Quizzes (Samigo) 2011

on

  • 1,256 views

 

Statistics

Views

Total Views
1,256
Views on SlideShare
1,256
Embed Views
0

Actions

Likes
0
Downloads
9
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Since tools is growing more complex and the list of new feature requests is also growing, we need to step back and look at the big picture, decide what will guide our future decisions, how we want to focus our efforts, and how we will sustain the amount of work required to keep this tool going and expanding.
  • Example of good spec on the confluence page for the new release process
  • Proposed timeline; since some dates already passed, we can still consider contributions if work is done or nearly done.Adjust timeline (in future releases) if you feel your feature will require more than 4-6 weeks of coding.Best to have design specs before coding, so usability review can make needed changes early in the process.
  • Prior to 2.8, an assessment exported from Respondus would not import correctly into T&Q.Question types would change. Metadata disappeared.
  • In 2.8, an assessment exported from Respondus in QTI XML or Zip format will import correctly in T&Q.
  • Use Import Assessment button in main page.
  • New in Import page: select between QTI file from Sakai system and QTI file from Respondus
  • Export from T&Q will not import correctly into Respondus, because format of tags in T&Q doesn’t match Respondus.We do not have plans to do this work. If another institution wants to implement this functionality, we will apply patch.
  • New average score option appears in Grading section of Settings page, along with Highest Score and Last Score optionsThis option only works with if multiple submissions are allowed. If limited to one submission, a warning message will appear when settings are saved.
  • New average score option appears in Grading section of Settings page, along with Highest Score and Last Score optionsThis option only works with if multiple submissions are allowed. If limited to one submission, a warning message will appear when settings are saved.
  • All Submissions page looks the same
  • Average Submission page (default view in Total Scores) drops a lot of columns that apply to individual submissions:Link to student’s responses, Submit Date column, Adjustment column, Comment column
  • Average score will be sent to gradebook, if that option is selected.
  • Submitted Assessments display redesigned to enable students to view average score, while still being able to see scores and feedback for each submission.Previously, each submission record was an independent row in the table.Score type is shown in parenthesis after the score value in the Recorded Score column.
  • (Previous design of Submitted Assessments list)Submitted Assessments display redesigned to enable students to view average score, while still being able to see scores and feedback for each submission.Previously, each submission record was an independent row in the table.Score type is shown in parenthesis after the score value in the Recorded Score column.
  • Submitted Assessments list when collapsed to show only recorded scores
  • Submitted Assessments list for an assessment with no feedback or scores released.
  • In Feedback section of Settings, releasing scores has been separated from other feedback (questions, responses, etc.), but still considered a feedback component.The Release Scores option has to be combined with an option to release feedback. If ‘No Feedback’ selected, the scores will not display either.Previously, the only way to release scores viewable in the Tests & Quizzes tool was to release feedback that included the questions by default.The other way was to turn off feedback and send the scores to the Gradebook and have students find scores there.
  • While working on the feedback settings, we took the opportunity to clarify some settings whose meaning has been unclear for users.Delivery Dates – Due Date vs. Retract Date often proved confusing for instructors. There were different interpretations of what retraction means.Late Submission – Clarified this is a ‘one last chance to submit’ after the due date, not allowing students to submit multiple times after due date.
  • Prior to 2.8, T&Q shows zero score for non-submission records, but sent a null score to Gradebook (displaying as dash) to differentiate students who hadn’t taken assessment from those who really got zero points from taking assessment.In 2.5, Gradebook changed their logic to not include null scores in the calculation of Course Grade.There was no way for instructors to send a zero to the Gradebook, as the T&Q Scores page already said zero and T&Q scores can’t be edited within Gradebook.
  • Prior to 2.8, T&Q shows zero score for non-submission records, but sent a null score to Gradebook (displaying as dash) to differentiate students who hadn’t taken assessment from those who really got zero points from taking assessment.In 2.5, Gradebook changed their logic to not include null scores in the calculation of Course Grade.There was no way for instructors to send a zero to the Gradebook, as the T&Q Scores page already said zero and T&Q scores can’t be edited within Gradebook.
  • Now, Adjustment column shows dash.
  • Instructor can enter zero in that column and click Update to send a zero to the Gradebook.
  • That zero score will be reflected in the Gradebook.
  • Although issue has a technical fix, the UI could be more clear to let instructors know they need to enter a zero score.We will be adding an additional patch as part of 2.8 maintenance branch to have dashes appear in Score and Final Score as well for non-submission records.Example from Stanford instance, CourseWork. Tests & Quizzes called Assignments at Stanford
  • When instructor enters a zero in Adjustment column and updates, the Final Score will change to zero.Score column will stay as dash as reminder there is no actual submission and, therefore, no original score.
  • The same ‘Show/Hide Rich-Text Editor’ toggle used in assessment authoring now appears in delivery for short answer questions.The reason not to load the editor at first is that loading several editors on the same page (if several short answer questions on a page) would cause slow performance.
  • FCKeditor can be toggled on and off as long as there is no formatting entered yet.
  • Once any formatting exists, which includes line breaks, and the data has been saved by navigation buttons or links, the page will always have the editor loaded when revisited.The toggle disappears.
  • Originally, we did not have a Publish action. Only way to publish was to go to Settings. This was not intuitive enough for users, so we made a Publish action.Publish button was still on Settings page, however, making actions redundant.
  • Originally, we did not have a Publish action. Only way to publish was to go to Settings. This was not intuitive enough for users, so we made a Publish action.Publish button was still on Settings page, however, making actions redundant.
  • Originally, we did not have a Publish action. Only way to publish was to go to Settings. This was not intuitive enough for users, so we made a Publish action.Publish button was still on Settings page, however, making actions redundant.
  • Now, Publish action goes right to Publish Assessment page. Settings are expressed in prose form. Instructors can verify these and continue to publish or edit the settings if changes are needed.
  • This patch will remove the Assessment Types link (and page) from the top navigation, the assessment type drop-down for creating a new assessment, and the Assessment Type Information section of the Settings page.
  • This patch will remove the Assessment Types link (and page) from the top navigation, the assessment type drop-down for creating a new assessment, and the Assessment Type Information section of the Settings page.
  • This slide shows what the Assessment Types page looks like for system admins (members of Administration Workspace).Some things to consider when deciding to enable this property:It is possible to just remove all the pre-defined assessment types (system admins) and that will apply to all sites. This will make the assessment type drop-down disappear from the Create a New Assessment part of main page. System admins will be able to adjust the default selections in Default Assessment Type, which affects all new assessments if type is not used. Easier to make changes than having to edit database. Keeping the Assessment Types page also allows instructors to create their own types, only visible to them.
  • Survey Matrix – allows rating items against same set of answersEMI – Extended Matching Item, involves a list of items and a set of questions about the list. used for medical education
  • Confusion about Pending vs. Published assessments, renamed to working copies and published copies, explained what that meansJavascript tabbed display makes greater visual clarity between two functional areas, saves screen real estateCreate an assessment organization also improved
  • Filters to allow for showing All, Active, Inactive (with explanations)Default sorting has active assessments on top

Tests and Quizzes (Samigo) 2011 Tests and Quizzes (Samigo) 2011 Presentation Transcript

  • Tests & Quizzes (aka SAMigo)
    Christine Doherty, Stanford University
    June 16, 2011
  • Agenda
    Future of T&Q: Development Dilemmas
    Community Contribution
    2.8 Highlights
    2.9 Plans
    Q&A
  • Future of T&Q:
    Development Dilemmas
  • Guiding Principles
    Flexibility/Freedom
    Reliability/Error-Free
  • Development Focus
    Improve UI/Interaction
    Add New Features
  • Project Management
    Stanford as Gatekeeper
    Modularize/Distribute
  • Community Contribution
  • New Release Process
    http://confluence.sakaiproject.org/confluence/display/SAM
  • Rationale
    User experience reviews are critical:
    New features can cause clutter in UI, confusion, bad user experience
    New features can clash with existing or planned features and lead to unexpected behavior
    Stanford only has one designer to tackle the workload, so we need help
  • Requirements for Contributions
    Clear specs are a must:
    Rationale for new feature/enhancement
    Description of expected behavior
    Documentation of all affected screens
    Screen shots desirable
  • Timeline
  • 2.8 Highlights
  • RespondusIntegration before 2.8
    Respondus
    Sakai – T&Q
    Question Types Changed
    Missing Metadata
  • RespondusIntegration in 2.8
    Sakai – T&Q
    Respondus
    QTI XML or Zip file from Respondus
    Imports to Sakai Tests & Quizzes
  • Respondus Integration
  • Respondus Integration
  • Respondus Integration
    Respondus
    Sakai – T&Q
    Respondus
    Sakai – T&Q
  • Record Average Score
    New Feature: ability to record average score for multiple submissions
    Redesigned UI for student’s Submitted Assessments list to accommodate displaying average scores
    Contributed by Rice University
  • Record Average Score
  • Record Average Score
  • Record Average Score
  • Record Average Score
  • Record Average Score
  • Record Average Score
  • Record Average Score
  • Record Average Score
  • Record Average Score
  • Release Scores Only
    Based on feedback from instructors wanting to show students their scores without allowing them to see questions again (since questions are reused in subsequent tests)
    Solution is to separate out scores from rest of feedback options
  • Release Scores Only to Students
  • Release Scores Only
  • Clarification of Settings
  • Send zero scores to Gradebookfor non-submission records
    Clash of scoring rules in Tests & Quizzes and Gradebook caused inability to set zero scores for non-submission records for inclusion in course grade calculation
    Solution is to allow instructors to explicitly enter a zero score for a non-submission record that gets sent to gradebook
    Patch contributed by Longsight
  • Send zero scores to Gradebookfor non-submission records
  • Send zero scores to Gradebookfor non-submission records
  • Send zero scores to Gradebookfor non-submission records
  • Send zero scores to Gradebook
  • Send zero scores to Gradebookfor non-submission records
  • Send zero scores to Gradebookfor non-submission records
  • Send zero scores to Gradebookfor non-submission records
  • Rich Text Entry for Short Answer/Essay
    Instructors from several institutions requested a way for students to enter rich text in essay questions for equations, foreign characters, or just general formatting
    Solution is to present students with the ability to use the FCKeditorin Short Answer/Essay questions.
    Contributed by Longsight
  • Rich Text Entry for Short Answer/Essay
  • Rich-text for Short Answer
  • Rich-text for Short Answer
  • Publish Action
    Based on feedback that Settings and Publish actions were redundant, both leading to Settings page
    Solution is to link Publish action to Publish Assessment page where settings are verified and notification added (if desired)
  • Publish Action
  • Publish Action
  • Publish Action
  • Publish Action
  • Remove Assessment Types
    Assessment Types often prove to be confusing to instructors, since they may not know what the labels mean and what settings go with them
    One solution is to remove any trace of assessment types with a sakai property (samigo.showAssessmentTypes=False)
    Contributed by Longsight
  • Remove Assessment Types
  • Remove Assessment Types
  • Remove Assessment Types
  • 2.9 Plans
  • Stanford contributions for 2.9
    Restyling and rewriting of warning messages
    Reset selection and warning messages redraw at current question, instead of top of page
    Reworking automatic submission logic
    Validation for numeric response questions
    Count Characters button for Short Answer/Essay questions
    Render part information and images in Print view
  • Community contributions for 2.9
    Redesign of main Assessments page (Longsight)
    Calculated question type (University of Dayton)
    Survey – Matrix of Choices question type (Rutgers)
    Extended Matching Item question type (University of Cape Town)
  • Redesign of Main Assessments Page
  • Redesign of Main Assessments Page
  • Contacts
    Questions and feedback: samigo-dev@lists.stanford.edu
    Lydia Li, Stanford University
    Project Manager
    lydial@stanford.edu
    Karen Tsao
    Samigo Developer
    ktsao@stanford.edu
    Homepage:
    http://confluence.sakaiproject.org/confluence/display/SAM
  • Online Final Exams Using Sakai
    at Stanford University Jamie Tsui
    9:45 AM - 10:15 AM in BeaudryB
  • Questions?