Cengage Learning Webinar, Course Redesign, Assessing Learning & Cost in Course Redesign
Upcoming SlideShare
Loading in...5
×
 

Cengage Learning Webinar, Course Redesign, Assessing Learning & Cost in Course Redesign

on

  • 854 views

Building on the overview of the course redesign provided in our webinar in April entitled "Improving Outcomes and Reducing Costs: The Case for Redesign," Dr. Carolyn Jarmon of the National Center for ...

Building on the overview of the course redesign provided in our webinar in April entitled "Improving Outcomes and Reducing Costs: The Case for Redesign," Dr. Carolyn Jarmon of the National Center for Academic Transformation described the methods institutions have used to measure the impact of course redesign on student learning and instructional costs.

Providing specific examples from institutions that have used these methods to prove their success, she also shared the tools that these institutions have used to demonstrate increased learning and reduced costs, so that other institutions seeking to validate their success with course redesign are equipped to do so.

Statistics

Views

Total Views
854
Views on SlideShare
604
Embed Views
250

Actions

Likes
0
Downloads
8
Comments
0

1 Embed 250

http://blog.cengage.com 250

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • For those of you who would like to learn more about the course redesign process, we will be hosting a follow-up webinar on May 7th titled “Assessing Learning and Cost in Course Redesign.” Building on the overview of the course redesign movement provided today, Carolyn will describe the methods institutions have used to measure the impact of course redesign on student learning and instructional costs. She will provide specific examples from institutions that have used these methods to prove their success. She will also share the tools that these institutions used to demonstrate increased learning and reduced costs, so that other institutions seeking to validate their success with course redesign are equipped to do so.

Cengage Learning Webinar, Course Redesign, Assessing Learning & Cost in Course Redesign Cengage Learning Webinar, Course Redesign, Assessing Learning & Cost in Course Redesign Presentation Transcript

  • Assessing Learning and Cost in CourseRedesignwith Dr. Carolyn Jarmon, Vice President forThe National Center for Academic Transformation
  • REDESIGNING STUDENT LEARNINGENVIRONMENTS:Measuring Cost and Quality
  • POLL #1What stage is course redesign at yourinstitution?See the poll on the right of your screen…
  • WHAT IS THE MOST IMPORTANT THING THATWE HAVE LEARNED ABOUT QUALITY ANDCOST?The factors that lead toincreased studentlearning and increasedstudent retention are thesame as those that leadto reduced instructionalcosts!
  • FIVE PRINCIPLES OF SUCCESSFUL COURSEREDESIGN#1: Redesign the whole course– Quality: Eliminate “course drift”; greatercourse coherence and quality control– Cost: Eliminate duplicate effort; createopportunities for alternate staffing
  • FIVE PRINCIPLES OF SUCCESSFUL COURSEREDESIGN#1: Redesign the whole course– Quality: Eliminate “course drift”; greatercourse coherence and quality control– Cost: Eliminate duplicate effort; createopportunities for alternate staffing
  • FIVE PRINCIPLES OF SUCCESSFUL COURSEREDESIGN#2: Encourage active learning– Quality: “Learning is not a spectatorsport.”– Cost: Reduce faculty preparation andpresentation time; reduce grading time(e.g., interactive software, peer learningteams)
  • FIVE PRINCIPLES OF SUCCESSFUL COURSEREDESIGN#3: Provide students with individualizedassistance– Quality: Students get help when they are“stuck” and stay on task rather than givingup: software tutorials, F2F in labs or helprooms, “beep a tutor,” SMARTHINKING– Cost: Apply the right level of humanintervention: peer tutors, course assistants
  • FIVE PRINCIPLES OF SUCCESSFUL COURSEREDESIGN#4: Build in ongoing assessment and prompt(automated) feedback– Quality: Enables practice, diagnosticfeedback, focused time on task– Cost: Good pedagogy with large numbers ofstudents; individual and group assessment;faculty spend time on what students don’tunderstand
  • FIVE PRINCIPLES OF SUCCESSFUL COURSEREDESIGN#5: Ensure sufficient time on task and monitorstudent progress– Quality: Self-pacing vs. milestones forcompletion; points for engagement– Cost: Course management systems canreduce costs while increasing oversight
  • READINESS CRITERIA• What does it mean to be “ready” to do a major course redesign?• Is your institution ready?• Which courses are “ready”—i.e., are good candidates for a comprehensiveredesign?• What decisions needto be made?
  • READINESS CRITERION #1Course Choice• What impact wouldredesigning the coursehave on thecurriculum, onstudents and on theinstitution—i.e., whydo you want toredesign this course?
  • POLL #2Using the chat function, what course (orcourses) is your institution considering foran upcoming redesign?Write the course title in the chat box.
  • FACTORS TO CONSIDER WHEN THINKINGABOUT HIGH IMPACT• High drop-failure-withdrawal rates• Student performance in subsequent courses• Students on waiting lists• Student complaints• Other departmental complaints• Lack of consistency in multiple sections• Difficulty finding qualified adjuncts
  • READINESS CRITERION #2Redesign Model• Which redesign model do you thinkwould be most appropriate for yourredesign? Why?• What aspects fit your particulardiscipline and your particularstudents?
  • REDESIGN MODELS• Supplemental – Add to the current structure and/or changethe content• Replacement – Blend face-to-face with onlineactivities• Emporium – Move all classes to a lab settingFully online – Conduct all (most)learning activities online• Buffet – Mix and match accordingto student preferences• Linked Workshop – JIT workshopslinked to college level course
  • READINESS CRITERION #3Assessment Plan• Which assessmentmodel do you thinkwould be mostappropriate for yourredesign? Why?
  • ASSESSMENT GOALTo establish thedegree to whichimproved learninghas been achievedas a result of thecourse redesign.
  • ASSESSMENT PLANNINGStep 1. Establish the method ofobtaining data.Step 2. Choose the measurementmethod.
  • ESTABLISH THE METHOD OFOBTAINING DATA• Baseline “Before”(traditional) and “After”(redesign)• Parallel Sections –Compare traditionalsections and redesignedsections
  • CHOOSE THE MEASUREMENTMETHOD: FOUR MODELSA. Comparisons of Final ExamsB. Comparisons of Common ContentItems Selected from ExamsC. Comparisons of Pre- and Post- TestsD. Comparisons of Student Work usingCommon Rubrics
  • COMMON EXAMSUniversity of MassachusettsIn spite of moredifficult questions,student scores onexams in a redesignedbiology courseaveraged 73% vs. 61%in the traditionalcourse.
  • COMMON ITEMSCarnegie Mellon University• Redesign students showedsignificant improvement based oncomparing exam questions.• In solving problems, redesignstudents made fewer than oneerror per problem, whereastraditional students made aboutsix errors per problem.
  • PRE/POST TESTSUniversity of Central FloridaStudents in thetraditional formatposted a 1.6-pointimprovement on acontent exam, whereasat 2.9, the mean changefor students in theredesigned course wasalmost double thatamount.
  • COMMON RUBLICSTallahassee Community CollegeStudents in a redesignedEnglish composition coursescored significantly higheron final essays, with anaverage score of 8.34compared to 7.33 fortraditional students.
  • More Is Not Better!• Performance in follow-on courses• Student attitude toward subject matter• Student interest in pursuing furthercoursework in the discipline• Differences in performance among studentsubpopulations• Deep vs. superficial learning
  • GRADES ARE NOT A SUFFICIENT MEASUREOF STUDENT LEARNING• Lack of consistency• Different coverage• Different tests andexams• Curving• InflationUse only for course completion!
  • READINESS CRITERION #4Cost Savings Plan• Which cost savingsstrategy do you thinkwould be mostappropriate for yourredesign? Why?
  • COST SAVINGS GOALCreate cost savings that canbe used to sustainongoing redesign, to fundfuture operations and tofree up resources forprogram and/orinstitutional priorities.
  • STEP 1: WHAT’S YOURENROLLMENT SITUATION?• Is your enrollmentgrowing or projectedto grow?• Is your enrollmentstable or declining?
  • STEP 2: CHOOSE LABOR-SAVINGTACTICS• Coordinated development and shared instructionaltasks for individual development and delivery.• Software for face-to-face.• Automated grading of homework, quizzes, examsfor hand grading.• Course management software for humanmonitoring and course administration.• Interaction with other personnel for one-to-onefaculty/student interaction.
  • STEP 3: CHOOSE A COSTREDUCTION STRATEGY• Each instructor carries more studentsby– Increasing section size.– Increasing the number of sections carried.• Change the mix of personnel teachingthe course.• Do both simultaneously.
  • COST REDUCTIONEXAMPLETraditional• Instructor load = 4sections• 150 minutes/week3 50-minute or 2 75-minute meetings• Time spent = 600minutes per weekRedesign• Instructor load = 6sections• 150 minutes/week2 50-minute meetings• Time spent = 600 minutesper week• Option: Reduce # of sectionsfrom 12 to 8 per semester,saving an additional $32,000
  • COST REDUCTIONEXAMPLETraditional• Each instructorteaches 1 section• Section size = 25• Time spent = 200hoursRedesign• Time spent = 100hours• Options:– Each instructor = 2sections of 25– Each instructor = 1section of 50
  • GENERAL PSYCHOLOGYFrostburg State UniversityTraditional• 2 meetings/week• Sections of 50• 9 FT and 9 PTfaculty• $89 cost-per-studentRedesign• 1 meeting/week• Sections of 150• 1 FT and 3 PT faculty;ULAs in lab• Required lab time• $26 cost-per-student
  • ENGINEERING PHYSICSAuburn UniversityTraditional• 750 students annually• Lecture, lab, recitationstructure• 14 GTAs• High DFW rates• Passively unengagedstudents• $390 cost-per-student• Redesign• Online activities completedbefore class• Physics activity session(combined lab and recitation)• 9 GTAs• Online: pre-tests, post-tests,problems, conceptualinventories• $219 cost-per-student
  • STATICSMississippi State UniversityTraditional• 3 hours of lecture• Students do homeworkand problem-solvingoutside of class (thetoughest part!)• 5 FT and 5 PT facultyteach 11 sections of ~35students• $299 cost-per-studentRedesign• 3-hour laptop lab &hands-on experiments• Peer-to-peer and GTAassistance• Lectures streamed forstudents to watch andreview on their own• One instructor teachesall sections supportedby GTAs and ULAs• $252 cost-per-student
  • FAIRFIELD UNIVERSITYGeneral BiologyTraditional• 7 sections (~35)• 7 faculty• 100% wet labs• $131,610• $506 cost-per-studentRedesign• 2 sections (~140)• 4 faculty• 50% wet, 50% virtual• $98,033• $350 cost-per-student
  • COURSE PLANNING TOOLA decision-making toolthat enablesinstitutions tocompare the“before” activitiesand costs (thetraditional course)and the “after”activities and costs(the redesignedcourse)
  • Instructional Costs per Hour
  • TRADITIONAL COURSETraditional Course Tenure-Track Non-Tenure- Adjunct/Faculty Track Faculty TAs/GAs PT FacultyHourly rate = Hourly rate = Hourly rate = Hourly rate =# of Hours Total Cost# of Hours Total Cost# of HoursTotal Cost# of Hours Total CostI. Course PreparationA. Curriculum Development $0 $0 $0 $0B. Materials Acquisition $0 $0 $0 $0C. Materials Development1. Lectures/presentations $0 $0 $0 $02. Learning materials/software $0 $0 $0 $03. Diagnostic assessments $0 $0 $0 $04. Assignments $0 $0 $0 $05. Tests/evaluations $0 $0 $0 $0Sub-Total 0 $0 0 $0 0 $0 0 $0D. Faculty/TA Development/Training1. Orientation $0 $0 $0 $02. Staff meetings $0 $0 $0 $03. Attend lectures $0 $0 $0 $0Sub-Total 0 $0 0 $0 0 $0 0 $0Total Preparation 0 0 0 0 0 0 0 0II. Course DeliveryA. Instruction1. Diagnose skill/knowledge $0 $0 $0 $02. Presentation $0 $0 $0 $03. Interaction $0 $0 $0 $04. Progress monitoring $0 $0 $0 $0Sub-Total 0 $0 0 $0 0 $0 0 $0B. Evaluation1. Test proctoring $0 $0 $0 $02. Tests/evaluation $0 $0 $0 $0Sub-Total 0 $0 0 $0 0 $0 0 $0Total Delivery 0 $0 0 $0 0 $0 0 $0TOTAL 0 $0 0 $0 0 $0 0 $0
  • CPT TASKS• Calculations you have completed– All personnel costs expressed as an hourly rate (page1)– The specific tasks associated with offering thetraditional course and how much time each personspends on each of the tasks (page 2)• Calculations you need to complete– Tasks and hours in the redesign (page 3)– Total instructional costs for the traditional course(page 4)– Total instructional costs for the redesign (page 4)
  • REDESIGNED COURSERedesigned Course Tenure-Track Non-Tenure- TAs/GAs Adjunct/Faculty Track Faculty PT Faculty# of Hours # of Hours # of Hours # of HoursI. Course PreparationA. Curriculum DevelopmentB. Materials AcquisitionC. Materials Development1. Lectures/presentations2. Learning materials/software3. Diagnostic assessments4. Assignments5. Tests/evaluationsSub-Total 0 0 0 0D. Faculty/TA Development/Training1. Orientation2. Staff meetings3. Attend lecturesSub-Total 0 0 0 0Total Preparation 0 0 0 0II. Course DeliveryA. Instruction1. Diagnose skill/knowledge2. Presentation3. Interaction4. Progress monitoringSub-Total 0 0 0 0B. Evaluation1. Test proctoring2. Tests/evaluationSub-Total 0 0 0 0Total Delivery 0 0 0 0TOTAL 0
  • Annual Cost ComparisonInstitution: NOTE If you insert or delete columns, you must adjust the formulas that calculate totals.Course:Tenure-Track Non-Tenure- Adjunct/Faculty Track Faculty TAs/GAs PT FacultySections Sections Sections SectionsANNUAL COST OF THE TRADITIONAL COURSE# of sections taught in fall and springCost of one sectionCost of instruction by type #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! Total cost of direct instructionTotal cost of course coordinationGRAND TOTAL #VALUE! Total cost of other personnelTotal # of studentsCost per student #VALUE!ANNUAL COST OF THE REDESIGNED COURSE# of sections taught in fall and springCost of one sectionCost of instruction by type #VALUE! #VALUE! #VALUE! #VALUE! #VALUE! Total cost of direct instructionTotal cost of course coordinationGRAND TOTAL #VALUE! Total cost of other personnelTotal # of studentsCost per student #VALUE!
  • THE CPT – WHY DO IT?• Provides a structure for the teamto think about activities andcosts• Allows consideration of changesin specific instructional tasks• Permits visualization ofduplication and waste• Enables cost/benefit analysis re:type of personnel per taskTEAM!!!
  • READINESS CRITERION #5Learning Materials• Are the faculty able andwilling to incorporateexisting curricularmaterials in order tofocus work on redesignissues rather thanmaterials creation?
  • READINESS CRITERION #6Active Learning• Do the faculty membershave an understandingof and some experiencewith integratingelements of computer-based instruction intoexisting courses?
  • READINESS CRITERION #7Collective Commitment• Describe the members ofyour team, the skills theybring to the project andwhat their roles will be inboth the planning andimplementation phasesof the project.
  • WHY INSTITUTIONAL TEAMS?• Faculty experts• Administrators• Technologyprofessionals• Assessment experts
  • READINESS CRITERIA1. Course Choice2. Redesign Model3. Assessment Plan4. Cost Savings Plan5. Learning Materials6. Active Learning7. Collective Commitment
  • NCAT’S REDESIGN METHODOLOGY“A Menu of Redesign Options”• Six Models for Course Redesign• Five Principles of Successful CourseRedesign• Four Models for Assessing StudentLearning• Cost Reduction Strategies• Readiness Criteria• Course Planning Tool• Five Critical Implementation Issues• Planning Checklist
  • REDESIGNING STUDENT LEARNINGENVIRONMENTS :Measuring Cost and QualityCarolyn Jarmon, Ph.D.cjarmon@theNCAT.orgwww.theNCAT.org
  • Consultation and Training• Provide an orientation to course redesign• Help you develop a set of learning outcomes• Help you determine which method of course redesign will help you achieve yourdesired outcomesFaculty Development• Help faculty develop a deeper understanding of the selected redesign model• Train faculty on strategies for implementing the redesigned curriculum• Work with faculty to determine the key assignments• Help construct an assessment plan around both your desired outcomes and keyassignmentsOngoing Consulting• Ongoing telephone and web-based consulting for the duration of the redesign
  • Additional Services Data Collection and Analysis Review and evaluation of existing course includingsyllabus, course description and learning outcomes. Map Cengage Learning print and/or digital content toyour specific standards or learning outcomes. Development of supporting course documents Development of media assets and assessment tools