Nurturing Insight to Assessing Online Teaching


Published on

WCET 2008 Conference presentation

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Discuss when online program started – 1999 – two programs When I was hired 2001 When the Koehler Center was full-time Only form of assessment was SPOT Began finding solutions for more assessment beyond student assessment.
  • Discuss first assessment tool – too long, too much work
  • eCollege component – provides various reporting features – this example shows two courses that fall below MLA standards of 87% retention. These would be pulled and
  • How these tools help with faculty development
  • Nurturing Insight to Assessing Online Teaching

    1. 1. Nurturing Insight to Assessing Online Teaching Romana Hughes Texas Christian University
    2. 2. History <ul><li>Online Graduate Programs 1999 </li></ul><ul><li>eLearning 2001 </li></ul><ul><li>Koehler Center for Teaching Excellence 2002 </li></ul><ul><li>Standards and Procedures Document (2003) </li></ul>
    3. 3. Standards and Procedures for Online Courses at TCU <ul><li>Written fall 2002 </li></ul><ul><li>Approved by Provost Council April 2003 </li></ul><ul><ul><li>Requires an online student evaluation </li></ul></ul><ul><ul><ul><li>Launched spring semester 2004 </li></ul></ul></ul><ul><ul><li>Requires assessment of instructional design </li></ul></ul><ul><ul><ul><li>First attempt launched spring semester 2005 </li></ul></ul></ul><ul><ul><ul><li>Not adopted by Nursing faculty – too much work </li></ul></ul></ul><ul><ul><li>Requires online faculty to attend training </li></ul></ul><ul><ul><li>Requires departmental review of all assessment to make recommendations for continued quality and improvement </li></ul></ul>
    4. 4. Faculty Requirements for online assessment of courses <ul><li>Must be a simple format </li></ul><ul><li>Must be one page </li></ul><ul><li>Must be online </li></ul><ul><li>Must have a report tool </li></ul><ul><li>Must be easy to learn and available for peer review </li></ul>
    5. 5. Online Self Assessment Tool – OSAT Created by Clarkson College <ul><li>Systematically review courses and determine what areas of instructional design could be improved </li></ul><ul><li>Develop clearer perspective on course design </li></ul><ul><li>To easily identify areas of strength and weaknesses for future development </li></ul><ul><li>Four level rubric </li></ul><ul><li>Copy of TCU OSAT is found at back of handout. </li></ul>
    6. 6. Pilot Spring 07 <ul><li>Used Clarkson’s OSAT to assess the spring 2006 MLA online courses </li></ul><ul><li>Modified the layout to fit the needs of TCU policies and procedures </li></ul><ul><li>Faculty approved tool April 2007 </li></ul>
    7. 7. Where is OSAT now? <ul><li>August 2007 completely online in TCU Portal </li></ul><ul><li>Available for peer review and personal review of online courses </li></ul><ul><li>Reporting tool created September 2007 </li></ul><ul><li>October 2007 Incorporated Program Intelligence Manager (eCollege tool)– PIM - results with OSAT results </li></ul>
    8. 8. Course Home/Syllabus Section OSAT online
    9. 9. Content Levels Section OSAT Online
    10. 10. Presentation Levels Section OSAT Online
    11. 11. Current Report Options <ul><li>By College </li></ul><ul><li>By Date </li></ul><ul><li>By Term </li></ul>
    12. 12. Example of One Report Flags to watch - Score below 2.0
    13. 13. Program Intelligence Manager PIM <ul><li>PIM enables data-driven decision-making to </li></ul><ul><li>Assist in more successful student outcomes and better institutional performance </li></ul><ul><li>Allows schools to determine what factors contribute to the success (or lack of success) of their online courses </li></ul><ul><li>To improve and maintain quality course offerings online </li></ul>
    14. 14. Program Intelligence Manager – PIM Report Example Flags to watch – retention below 87% Set by department 100.00% 100.00% 100.00% NURS70030 Fall 2006 eCourse 100.00% 100.00% 100.00% NURS60713 Fall 2006 eCourse 100.00% 100.00% 100.00% NURS60482 Fall 2006 eCourse 100.00% 100.00% 100.00% NURS60413 Fall 2006 eCourse 90.91% 100.00% 90.91% NURS60043 Fall 2006 eCourse 94.12% 88.89% 100.00% NURS60003 Fall 2006 eCourse 62.50% 50.00% 100.00% NURS4303 Fall 2006 eCourse 88.46% 89.29% 96.15% NURS40632 Fall 2006 eCourse 83.33% 80.95% 100.00% NURS30543 Fall 2006 eCourse 87.50% 87.50% 87.50% MALA70973 Fall 2006 eCourse 70.00% 88.89% 80.00% MALA61293 Fall 2006 eCourse 62.50% 100.00% 62.50% MALA61203 Fall 2006 eCourse 93.33% 100.00% 93.33% MALA61173 Fall 2006 eCourse Course Start to Course End Completion Rate Census to Course End Completion Rate Course Start to Census Completion Rate Dev101 Fall 2006 eCourse Course Completion Rates: Course Completion Rate Analysis - eCourse Terms
    15. 15. Faculty Development <ul><li>OSAT Score should be 2.0 or greater </li></ul><ul><li>PIM retention target is 87% </li></ul><ul><li>Deans and/or Department Chairs review reports with Koehler staff </li></ul><ul><li>Faculty Development plans are created with Dean and/or Department Chair and faculty for scores that are lower than set standard </li></ul><ul><li>Faculty sign up for workshops based on FD plan or request one-to-one training </li></ul>
    16. 16. Contacts Romana Hughes [email_address] 817-257-6631