Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sakai_EvalSys_VT&Tufts

415 views

Published on

Published in: Education
  • Be the first to comment

Sakai_EvalSys_VT&Tufts

  1. 1. Will Humphries, Senior Developer @ Tufts Brian Broniak, Director of Online Learning & Collaboration Services (OLCS) @ Virginia Tech June 10-15, 2012Growing Community;Growing Possibilities
  2. 2.  Background information ◦ Both schools and Sakai The VT story from idea through full production The Tufts story from mandate to current state Goal: Share ideas for a successful implementation of EvalSys at your university 2012 Jasig Sakai Conference 2
  3. 3.  VT is in Blacksburg, VA – a college town 30,000 students globally, 7000 courses Most students in B’burg, but students global 4 semesters each year – F/S, Summer I and II Very decentralized, hard to do “enterprise” Sungard Banner as our SIS Sakai 2.8.1 as of Spring 2012  Scholar Sakai school fully since 2010, informally for much longer 2012 Jasig Sakai Conference 3
  4. 4.  Tufts is around Boston, MA 10,000 students, approx. 5,000 undergrad Sakai for 1 year, 3 of 8 colleges ◦ it’s called Trunk Moving to PeopleSoft Campus Solutions 9.0 Sakai 2.8.1, eval 1.4 2012 Jasig Sakai Conference 4
  5. 5.  Using paper OpScan forms since 70s ColdFusion based system, first try online ◦ Decided it would not meet needs at scale so started a tool for Sakai  EvalSys today Task Force(s) on the process and form in parallel to work on the system Another system for distance learning (IDDL) at VT (developed in-house) 2012 Jasig Sakai Conference 5
  6. 6.  Decentralized – some online, some paper A&S and Eng. motivated to leave paper ◦ Cost savings from centralized service Evals for all Tufts Trunk adopters Fall 2012 ◦ 30,000 potential responses from 2,500 courses Evals in LMS a selling point for Trunk project 2012 Jasig Sakai Conference 6
  7. 7.  Provide “one” system to unify this data – replace ColdFusion, paper, IDDL system Provide robust reporting Garner 70%+ response rates sustained Provide a “good” student and instructor experience Our approach: ◦ Ramp up slowly to allow for departments to adjust (a 5 semester ramp-up) ◦ Communicate, communicate, and communicate 2012 Jasig Sakai Conference 7
  8. 8.  EvalSys 1.3 with local changes We use the hierarchy with 3 levels – VT-wide, college, and department Email reminders each course section every 2 days All scaled questions on 6 point scale We have an external data warehouse and Jaspersoft reporting interface We purge the data each semester after putting copies of reports into Scholar personal workspaces and in our warehouse 2012 Jasig Sakai Conference 8
  9. 9.  In Spring 2012 we finally ran this VT wide! ◦ 4500 courses, 125,000 requests with 85,000 completed = 69% response rate Students like it far better, faculty feedback has been mixed Response rates in high 67-70% range each semester + a definite “reminder effect” (see chart) 2012 Jasig Sakai Conference 9
  10. 10.  Asked two questions to random sampling of students over all 5 semesters ◦ “Compared to the evaluation form used in the past, the items on this form allow me to provide a better evaluation of this course/instructor.” 22,697 responses from students here – with 87% favorable (somewhat agree, agree, or strongly agree), 61% = agree or strongly agree ◦ “I prefer filling out the evaluation form on-line rather than filling out the paper-and- pencil form in class.” 23,017 responses from students here – with 87% favorable (somewhat agree, agree, or strongly agree), 77% = agree or strongly agree 2012 Jasig Sakai Conference 10
  11. 11.  Need to get chart here from data Steve is working on 2012 Jasig Sakai Conference 11
  12. 12.  Major org challenges, especially with decentralized universities ◦ Get an “owner” outside of IT – our Office of Assessment and Evaluation has been a great partner Focus on the form and go ahead and change scales earlier rather than later Ramp up slowly, gives you time to refine system and everyone to adjust Build a warehouse (lots of valuable data) – do your best to gather report requirements Work through change curve – myths, rationale, etc 2012 Jasig Sakai Conference 12
  13. 13.  Centralized administration Identify edge cases early ◦ Cross-listed ◦ Team taught ◦ TA’s / non-official instructors ◦ Studentless/instructorless courses External reporting? Still in the works 2012 Jasig Sakai Conference 13
  14. 14.  Gradual pilot vs. rapid deployment CM API (evalgroup) vs. Site-based (evalproviders) One template vs. several Separate instance vs. primary LMS instance 2012 Jasig Sakai Conference 14
  15. 15.  Consolidated emails Exposing response rates real-time Deal with edge cases – non-traditional schedules and GAs Get back to the Sakai baseline 2012 Jasig Sakai Conference 15
  16. 16. Contact information: Brian Broniak – bbroniak@vt.edu Will Humphries – will.d.humphries@gmail.com June 10-15, 2012Growing Community;Growing Possibilities

×