Use of LMS Tools to Evaluate Learning Effectiveness

Terry L. Heiney, Ph.D. Vice President – Sales1

A Few Points to Note This webinar is being recorded Slides and video will be available at CommLab India website within a week(well notify you by email) During the webinar presentation your phones are set to mute in order to provide you the best possible sound quality without background noise. If you have any question during the presentation, please use the chat functionality of WebEx and type your question any time. We will answer your questions during the Q&A session. 2

Focus of Presentation – What the LMS Can Tell Us Is our instruction effective from an instructional design perspective? What are the affective impacts of our instruction? Are there hidden learner messages in the data? 3

Common LMS Data Review and final exam question responses Course evaluations Learner data  enrollments and completions  completion times  time on page, page views, and related learner/course interactions 4

Sites Considered in this Presentation General continuing education on public Internet Company proprietary continuing education on private intranet Company proprietary training on private intranet 5

Reviews and Final Exams Useful for all types of content 2008 2010 Question Users Attempts Correct Average Users Attempts Correct Average Review 2 321 411 367 89.29% 185 212 187 88.21% Review 4 293 354 296 83.62% 167 180 171 95.00% Review 5 290 334 314 94.01% 176 188 177 94.15% Review 7 294 348 266 76.44% 179 200 151 75.50% Review 8 310 405 332 81.98% 176 201 174 86.57% Review 9 309 404 274 67.82% 176 200 146 73.00% Review 10 310 406 271 66.75% 175 199 168 84.42% Review 12 327 435 276 63.45% 188 218 140 64.22% 6

Course Evaluation – SampleCourse Evaluations Summary ReportQuestion (Response 1 to 5) Question (Response 0 or 1) This course was valuable to me for the ContinuingThe course objectives were met. Education Credit. This course was valuable to me for the Relevance toThis course was comprehensive and organized. My Practice. This course was valuable to me for the Timeliness ofThe content was current and technically accurate the Subject Matter. This course was valuable to me because of theThis course was timely and relevant. Reputation of the Author.The prerequisite requirements were appropriate. This course was valuable to me because of the Price.This course was a valuable learning experience. This course was valuable to me for Other Reasons.The course completion time was appropriate. 7

Course Evaluation – Public Internet 2008 2010 Course 1 2 3 1 2 3 Question (Response 1 to 5) Averages The course objectives were met. 4.09 4.13 4.23 3.99 4.02 4.09 This course was comprehensive and organized. 4.1 4.15 4.19 3.97 4.01 4.05 The content was current and technically accurate. 4.08 4.15 4.22 4.07 4.03 4.08 This course was timely and relevant. 4.06 4.12 4.21 4.04 4.06 4.09 The prerequisite requirements were appropriate. 4.04 4.09 4.16 3.99 4.01 4.03 This course was a valuable learning experience. 4.07 4.15 4.23 3.98 4.07 4.11 The course completion time was appropriate. 4.01 3.95 4.11 3.89 3.9 4.06 Nothing changes! 8

Course Evaluation – Public Internet 2008 2010 1 2 3 1 2 3Question (Response 0 or 1) AveragesThis course was valuable to me for the Continuing 0.7 0.7 0.71 0.75 0.73 0.72Education Credit.This course was valuable to me for the Relevance to My 0.46 0.46 0.5 0.42 0.49 0.47Practice.This course was valuable to me for the Timeliness of the 0.05 0.05 0.08 0.1 0.09 0.15Subject Matter.This course was valuable to me because of the 0.01 0.01 0.02 0.03 0.02 0.02Reputation of the Author.This course was valuable to me because of the Price. 0.02 0.02 0.03 0.05 0.07 0.08This course was valuable to me for Other Reasons. 0.01 0 0.01 0.01 0.01 0 9

Course Evaluation – Proprietary 10

Completion Data – Public Internet Courses Total Successful Unsuccessful No Attempt % Not % Completed 2010 Series Enrollments Completions Completion on Final AttemptedPart 1 756 564 75% 5 187 25%Part 2 617 501 81% 4 112 18%Part 3 500 411 82% 0 89 18% Courses Total Successful Unsuccessful No Attempt % Not % Completed 2008 Series Enrollments Completions Completion on Final AttemptedPart 1 588 447 76% 17 122 21%Part 2 437 331 76% 17 89 20%Part 3 359 304 85% 2 53 15% 11

Completion Data – Proprietary Content Course Students Completed Students Enrolled % Completed 2005 2085 96.16% 1500 1642 91.35% 1459 1459 100.00% 1111 1410 78.79% 1295 1407 92.04% 1343 1405 95.59% 825 1100 75.00% 918 1060 86.60% 1023 1023 100.00% 923 1004 91.93% 723 875 82.63% 640 827 77.39% 652 735 88.71% 588 651 90.32% 582 626 92.97% 453 558 81.18% 12

Other Data – Time on PageSession ID Date Start Time Page Duration Type ScoreT2YBwDLG3s 11/18/2008 15:33:16 11/18/2008 16:26:01 1 0:52:45 default 11/18/2008 16:26:01 1 0:00:00 default 11/18/2008 16:26:11 6 0:00:10 default 11/18/2008 16:26:11 6 0:00:00 default 11/18/2008 16:26:17 27 0:00:06 default 11/18/2008 16:26:24 6 0:00:07 default 11/18/2008 16:26:24 6 0:00:00 default 11/18/2008 16:26:37 7 0:00:13 default 11/18/2008 16:26:39 7 0:00:02 default 11/18/2008 16:26:41 8 0:00:02 default 11/18/2008 16:26:42 8 0:00:01 default 11/18/2008 16:26:43 9 0:00:01 default 11/18/2008 16:26:45 10 0:00:02 default 11/18/2008 16:26:48 11 0:00:03 default 11/18/2008 16:26:49 12 0:00:01 default 11/18/2008 16:26:50 12 0:00:01 default 11/18/2008 16:26:51 13 0:00:01 default 11/18/2008 16:27:14 13 0:00:23 WhoMustQuiz 5-May 100% passed 11/18/2008 18:30:10 72 0:03:59 finalExam 15/15 100% passed 11/18/2008 18:30:17 73 0:00:07 default 11/18/2008 18:30:58 73 0:00:41 CCHsurvey 1-Jan 100% passed 11/18/2008 18:30:58 74 0:00:00 default 11/18/2008 18:31:01 74 0:00:03 defaultSubtotals: 92 2:57:45Totals: 92 2:57:45 13

Other Data – Page Attempts # Timestamp Session ID Page External Page Duration 1 12/07/2010 10:20 AM 85l8rrd2gefm9694ar429sbk57 1 00:00:00 2 12/07/2010 10:20 AM 85l8rrd2gefm9694ar429sbk57 1 00:00:00 3 12/29/2010 11:35 AM 61nfhjngo9tl250s6auev058o1 1 00:00:00 4 12/29/2010 11:35 AM 61nfhjngo9tl250s6auev058o1 1 00:00:00 5 12/29/2010 12:48 PM 61nfhjngo9tl250s6auev058o1 1 00:00:00 6 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 8 00:00:00 7 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 9 00:00:00 8 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 8 00:00:00 9 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 9 00:00:00 10 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 10 00:00:00 11 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 11 00:00:00 12 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 12 00:00:00 13 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 12 00:00:00 14 12/29/2010 01:09 PM 61nfhjngo9tl250s6auev058o1 33 00:00:00 15 12/29/2010 01:12 PM 61nfhjngo9tl250s6auev058o1 33 00:00:00 16 12/29/2010 01:13 PM 61nfhjngo9tl250s6auev058o1 35 00:00:00 14

Other Data – Time on PageSession ID Date Start Time Page Duration Type ScoreT2YBwDLG3s 11/18/2008 15:33:16 11/18/2008 16:26:01 1 0:52:45 default 11/18/2008 16:26:01 1 0:00:00 default 11/18/2008 16:26:11 6 0:00:10 default 11/18/2008 16:26:11 6 0:00:00 default 11/18/2008 16:26:17 27 0:00:06 default 11/18/2008 16:26:24 6 0:00:07 default 11/18/2008 16:26:24 6 0:00:00 default 11/18/2008 16:26:37 7 0:00:13 default 11/18/2008 16:26:39 7 0:00:02 default 11/18/2008 16:26:41 8 0:00:02 default 11/18/2008 16:26:42 8 0:00:01 default 11/18/2008 16:26:43 9 0:00:01 default 11/18/2008 16:26:45 10 0:00:02 default 11/18/2008 16:26:48 11 0:00:03 default 11/18/2008 16:26:49 12 0:00:01 default 11/18/2008 16:26:50 12 0:00:01 default 11/18/2008 16:26:51 13 0:00:01 default 11/18/2008 16:27:14 13 0:00:23 WhoMustQuiz 5-May 100% passed 11/18/2008 18:30:10 72 0:03:59 finalExam 15/15 100% passed 11/18/2008 18:30:17 73 0:00:07 default 11/18/2008 18:30:58 73 0:00:41 CCHsurvey 1-Jan 100% passed 11/18/2008 18:30:58 74 0:00:00 default 11/18/2008 18:31:01 74 0:00:03 defaultSubtotals: 92 2:57:45Totals: 92 2:57:45 15

Other Data – Page Attempts # Timestamp Session ID Page External Page Duration 1 12/07/2010 10:20 AM 85l8rrd2gefm9694ar429sbk57 1 00:00:00 2 12/07/2010 10:20 AM 85l8rrd2gefm9694ar429sbk57 1 00:00:00 3 12/29/2010 11:35 AM 61nfhjngo9tl250s6auev058o1 1 00:00:00 4 12/29/2010 11:35 AM 61nfhjngo9tl250s6auev058o1 1 00:00:00 5 12/29/2010 12:48 PM 61nfhjngo9tl250s6auev058o1 1 00:00:00 6 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 8 00:00:00 7 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 9 00:00:00 8 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 8 00:00:00 9 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 9 00:00:00 10 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 10 00:00:00 11 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 11 00:00:00 12 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 12 00:00:00 13 12/29/2010 12:49 PM 61nfhjngo9tl250s6auev058o1 12 00:00:00 14 12/29/2010 01:09 PM 61nfhjngo9tl250s6auev058o1 33 00:00:00 15 12/29/2010 01:12 PM 61nfhjngo9tl250s6auev058o1 33 00:00:00 16 12/29/2010 01:13 PM 61nfhjngo9tl250s6auev058o1 35 00:00:00 16

Other Data – Individual Test Results Test Details Question Response Correct Response Where an IRA is payable to a QTIP trust and provides that the IRA’s annual income is to be distributed during the surviving spouse’s lifetime in one or more installments to a QTIP trust, a QTIP election can be made. TRUE TRUE A husband dies and his estate makes a Property must pass to a surviving QTIP election for spouse who is a U.S. citizen property passing to his wife. The wife (unless the property passed to a The value of the property on the dies. Which of the following statements is non-U.S. citizen-spouse in the wife’s valuation date is includible in correct? form of a QDOT). the wife’s estate. If the surviving spouse disposes In figuring the marital deduction, the total of QTIP property during his or her value of the lifetime, it is a taxable event property passing to a surviving spouse for gift tax purposes under Code must be reduced by: Sec. 2519. Both a. and b. If the taxable event is the surviving spouse’s death, then the surviving spouse’s estate can The maximum deduction for a charitable recover any transfer tax from the The residuary estate reduced by bequest of the residuary estate is: recipient of the property. taxes and administration expenses If the estate residuary is a charitable bequest the charitable deduction must be The property must be held in reduced by: trust. Both a. and b. 17

Conclusions To make course evaluations more valuable we need to think much more critically about the questions we ask. Monitoring completion rates increases completion rates. Learners will not complete review/reinforcement exercises if the interactions are optional. Authors need to improve questions with poor response rates in subsequent editions of a course. Exercises and exam items should be based on the learning objectives for a course. 18

Open House Your phones are set mute in order to provide you best possible hearing experience without background noise. Please ask your questions by using the chat functionality of Webex 19

Next Steps Slides and webinar recording will be posted at Elearning resources: http://www.commlabindia.com/elearning-resources/lms-to-eval Please download complimentary Sample Storyboard templates, Project Plan templates, Sample feedback forms and more from: http://www.commlabindia.com/downloads/elearning-webinars/s Contact me for your eLearning needs at: ( terry.heiney@commlabusa.com) 20

THANK YOU 21

Use of LMS Tools to Evaluate Learning Effectiveness

by CommLab India

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics

Use of LMS Tools to Evaluate Learning Effectiveness Presentation Transcript