3. Assessment Approach
Central concept:
Questions are weighted:
• Some things are bigger
obstacles than others.
• Users evaluate
competing statements
• 14 questions currently
• Modeled after
placement assessment
for first-year English
course
4. Interpreting the Score
• Script keeps running tally of
the scores
• Individual column totals are
what’s important, not column
comparison
• Currently “17” in right column
is our threshold for potential
problems
• “Magic number” is
subjective; changes with
number of questions
• Distribution of 3 and 4-
weight questions not
calculated, but remains
important
5. Assessment Deployment
Positioned on semester class schedule, as well as inside
course sites
Pre-registration access theoretically prevents enroll-drop-
lockout problem
Link automatically placed in schedule based on course delivery
method (fully online & hybrid online)
Faculty consulted on assessment content but not schedule
placement
One faculty member raised concern about OIT inserting
“unauthorized” information into their course listing
6. Assessment Usage
Daily visits from April 2010 – November 2011
• 30-40 visits per day in peak times of the academic year
• Referrer data indicates 90%+ visits coming from within csustan.edu
Source: Google Analytics
7. Everybody Likes Statistics
61 respondents from survey of Fall 2011 online
courses
5% had previously dropped because of online
format
20% aware of our assessment
~50% of those had taken it (11.5% of total)
100% reported understanding their score
No one shied away from an online course after
seeing their score
66% said the assessment confirmed what they
already felt they knew about their readiness
80% would recommend it to others
8. Under the Hood
The Question Interface
• Radio buttons trigger a javascript
• Payload includes question
number, the button’s answer, and
the question weight
• “question2storage” is dealt with in
the script
9. Under the Hood
Stepping Through the Script
1. Gather the current scores in the
“yes” and “no” columns
2. Turn them into integers so
additional math can be done to
them.
10. Under the Hood
Stepping Through the Script
• What is the current selection
for this question, if any?
• We need to know this in case
the user is changing their
answer.
• If so, we’ll need to add or
subtract differently.
11. Under the Hood
Stepping Through the Script
Evaluate.
1. Add the question weight to the current total for the column and
dynamically write out the new total onto the page.
1. If we’re changing the answer, change the other column in the
opposite direction and write the new total.
2. Store the latest answer in case the user changes their mind
again.
12. Under the Hood
Stepping Through the Script
Evaluate.
1. Add the question weight to the current total for the column and
dynamically write out the new total onto the page.
1. If we’re changing the answer, change the other column in the
opposite direction and write the new total.
2. Store the latest answer in case the user changes their mind
again.
Editor's Notes
Positioned on semester class schedule, as well as inside course sitesPre-registration access theoretically prevents enroll-drop-lockout problemCoordinated with university scheduler and OIT programmer for automatic placement based on course delivery methodFaculty consulted on assessment content but not schedule placementOne faculty member raised concern about OIT inserting “unauthorized” information into their course listingFaculty are encouraged to post the link inside the course, but we don’t know at this time how many of them do that.
Assessment was built to fill an important gap in our services, and been publicized to online faculty but not consistently spotlighted as perhaps it should be.The tactic of placing it in the course schedule was thought to accomplish a good deal of the awareness efforts – we hoped students would simply see it in the schedule and be curious enough to take it.A few reasons why students might not have taken it even though the link is right there:If you’ve taken an online course before, there’s no need to retake the assessmentNot reading carefully the text in front of you is a feature of the Internet age…Students feel confident in their understanding of what’s entailed in taking an online courseTo get some sense of its use, we did survey the current Fall 2011 population of online students (contacted via their instructors)