This details a successful data-driven redesign of Math 215, an online statistics concepts course at Franklin University. The redesigned course incorporated new interactive educational multimedia. This new design resulted in improved student retention, better student performance, and better satisfaction with the course.  

1. Dr. Nimet Alpay & Dr. Natalya Koehler.
Successful Redesign of Online Statistical
Concepts Course: Improved Student
Retention, Performance, and Satisfaction
with the Course.

2. THE NEED

3. College Graduation
1 According to the 2011 report from Complete College America
Part-time students have a 24.3% chance
of graduating with a 4-year bachelor’s
degree within 8 years and adult students
are far less likely to complete their
degrees than their traditional peers.1

4. Student Population
Statistics of Undergraduate Population Attending Franklin University
Category Percentage/Years
Average Age 33.27 years
Race Caucasians – 66.7%
African Americans – 21.6%
Others – 11.7%
Student Status Full time – 33.6%
Part time – 66.4%
Non-traditional – 80%
Financial Aid Recipients 70%
Gender Males – 43.5%
Females – 56.5%

5. MATH 215 Student Population
Gateway Course
General Education Course
FF and online formats
Pre-requisite
Levels of students’ motivation (the
course serves as a pre-requisite or not)

6. MATH 215 Course Pre-requisites
• Learning Strategies
• Introduction to Excel
Spreadsheets
• Intermediate Algebra or
Introduction to Quantitative
Reasoning (new course)

7. Student Performance and Attrition in Fall 2013

8. Redesign Goals
To increase
Student Retention Performance Graduation Rates Instructional Support for
Online Learners

9. Topics Coverage Based on ODHE Requirements
Below are the course topics based on ODHE requirements for MATH 215:
• sampling techniques
• data types
• experiments
• measures of central tendency
• measures of dispersion
• graphical displays of data
• basic probability concepts
• binomial and normal probability distributions
• sampling distributions and Central Limit Theorem
• confidence intervals
• hypothesis tests of a mean, or a proportion for one or two populations
• hypothesis tests for qualitative data
• linear regression
Topics that are excluded from the old course
based on ODHE requirements:
• ANOVA
• Regression Analysis

10. THE ANALYSIS BEFORE REDESIGN

11. Data-driven Course Redesign
The course
redesign was
based on the
analysis of:
Student course evaluation surveys
Instructor interviews
Program and department chair interviews
Student use of optional student academic support services
Data about student performance and attrition

12. Factors Impacting Student Retention
at the Course Level
Effective Instruction Effective Course Design
Student Retention

13. We Identified Need for
Systematic Instruction (Fall 2013)
Class 1 – High Attrition Rate (44%)
1. “Professor X seemed to genuinely want me
to succeed.“
1. “I feel that this course was extremely
demanding. Between the homework,
quizzes, group assignments, and test, it felt
nearly impossible for someone who isn’t
strong in math (like myself) to keep up”.
Class 2 – Low Attrition Rate (16%)
1. “He did an excellent job.”
2. Professor X was great, he always took the
time to talk to all of us provided several
examples that related the information to
everyone.
1. I thought the program worked very well
with my learning style.
• The instructor was helpful in providing “just in
time” help.
• Students find the course very difficult.
• The instructor provided systematic instruction.
• Students do not have negative comments about
the course.

14. Student Course Evaluation Survey (Fall 2013,
old course design)

15. THE REDESIGN

16. The Redesign Elements
Weekly
Assignment &
Assessment
types
Textbook
change
Web-based
component
change
Systematic
instruction
through
interactive
multimedia
lectures

17. Textbook Change
Old Textbook
• Triola, M. (2014) Elementary Statistics Using
the TI-83/84 Plus Calculator (3rd edition).
Pearson.
New Textbook
• Navidi W. & Monk B. (2014). Essential
Statistics. (Franklin University Custom
Edition). McGraw-Hill
“Extremely confusing textbook.”
-Rep. Student Comment
“Textbook is easy to understand.”
-Rep. Student Comment
“I liked the book a lot as it broke
down the concepts in a clear and
concise manner.”
-Rep. Student Comment
“Content explanations and exercises
don’t seem connected to each other”
-Rep. Instructor Comment

18. Web-Based Component Change
• SmartBook – adaptive learning technology
• Connect Math – assignment and assessment platform with integrated
media-rich ebook.
*Both products are integrated under ConnectMath
SmartBook Connect Math

19. Weekly Assignment & Assessment Types

20. Systematic Instruction – Interactive
Multimedia Lectures
28 interactive multimedia lectures to help students
understand different statistical concepts without
frustration.
The lectures include the following components:
• New information delivery
• Includes examples
• Includes both visual and auditory information
• Information summary
• “Check your learning” comprehension question
• Multiple choice
• Drag and drop
• Fill in the blank
• Enter answer
• Other
http://cs.franklin.edu/~
reid21/showcase/story.
html
Click here
to see the
demo of
multimedia
lectures Click on the button to see the demo
of multimedia lectures

21. THE RESULTS

22. Data Collection Instruments
Attrition & Performance Data Evaluations
• Course evaluation
• Multimedia evaluation
Instructor Feedback
• Course design feedback
• Multimedia feedback
Academic Support Services
• Attendance Rates
Triangulation

23. Results: D Below Rates & Attrition Rates
35.29%
64.71%
Fall 2014 Attrition Rate: 14.31%
36.49%
63.51%
MATH 215 Student Performance – Fall 2014
33.93%
66.07%
35%
65%
Overall Performance Face-to-face Classes Online Classes

24. Results: Student Performance Rates
FALL 13 FALL 14
Total # of Students 335 255
Online Students 221 (66%) 127 (50%)
FF Students 114 (34%) 128 (50%)
Grade A 12.2% 21.2%
Grades: A – C 49.8% 64.3%
Grades: D & Below 50.2% 35.6%
Attrition Rates 26.11% 14.31%

25. Continuing Effect of Course Redesign
FALL 13 FALL 14 WINTER 15 SUMMER 15 FALL 15
Total # of Students 335 255 279 147 236
Online Students 221 (66%) 127 (50%) 170 (61%) 83 (56%) 147 (62%)
FF Students 114 (34%) 128 (50%) 109 (39%) 65 (44%) 89 (38%)
Grade A 12.2% 21.2% 19.4% 20% 23%
Grades: A – C 49.8% 64.3% 66.7% 68% 66%
Grades: D & Below 50.2% 35.6% 31.4% 32% 33%
Attrition Rates 26.11% 14.31% 11% 13% 17%

26. Course Recommend Rates
TERMS
Percent Course
Recommend
Percent Faculty
Recommend
Fall 13 55.8% 82%
Winter 14 57.3% 88%
Summer 14 53% 76%
Fall 14 71% 88%
Winter 15 69% 85%
Summer 15 71% 92%
Fall 15 77% 88%

27. Results: Course Survey
Themes from student comments about effectiveness of the new design of the course over Fall 14 – Sum 15.
# The themes from students’ comments (composite
responses) -- FALL 2014
Number of users
commenting on the theme
1. The course is challenging and overwhelming. 33
2. The course is well designed and can be self-led. 19
4. The educational value of ConnectMath auto-graded learning
activities is excellent because it contains examples and
guided solutions.
13
4. LearnSmart auto-graded learning activities are redundant. 10
5. Need for additional resources for learning (Student Learning
Center).
8

28. Use of Academic Support Services
• The number of students who used optional support services in the
Fall 2013 – Fall 2015.
Types of optional support
services Fall 2013
Winter
2014
Summer
2014 Fall 2014
Winter
2015
Summer
2015 Fall 2015
Tutoring Services
(OL, FF, W-In)
37.8% * 9.50% 16.00% 12.90% 9.40% 16.20% 8.3%
Test Preparation Workshops
22.40% 15.40% 23.00% 13.70% 20.20% 12.20% 17.40%
Structural Learning Assistance
(SLA)
13.40% 13.20% 21.30% 9.40% 15.20% 14.20% 12.20%

29. Informal Student Feedback via Email
-Tina
I felt pretty good taking the
test and am pleased with my
final score! I must say that I am
extremely impressed with the
way Franklin does this class. I
know Statistics is typically a
universally feared class, but it
hasn’t been too bad so far.
-Brandon
I enjoyed the class even with
all of the work. Near the end,
everything has fit together
and it all makes sense. I have
a better understanding of how
this type of math helps
businesses. All of the
information from this class will
help me with my future
business.
-Dianne

30. THE RECENT UPDATES

31. Introduced a New Course as a Pre-requisite
Math 115:
Introduction to
Quantitative
Reasoning
It includes topics from
Beginning Algebra and
some concepts from
Descriptive Statistics
and Simple
Probability.
Application
based.
Excel is used for
tech
assignments.
This is the pre-
requisite for
students who will
NOT take College
Algebra for their
programs.

32. Conducting a Research Study on the Use of
Interactive Multimedia
Winter 15 term: Too few participants
• incentive: raffle for Ipad Mini
Fall 15 term: 86 students participated
• incentive: bonus points
(2% of the available points in the course)
Winter 16 term: study still going on.

33. Average Student Ratings of the Interactive Multimedia Lectures
(86 participants; rating scale of 5; 5-strongly agree, 1-strongly disagree)
Survey Questions Average ratings
1. I liked the layout of the sections in the lecture. 4.7
2. I liked the navigation in the sections in the lecture. 4.7
3. I liked the explanation and the examples used in the informational slides of the lecture sections. 4.6
4. I liked the summary slides used thought the sections. 4.5
5. I found the transcript option useful. 3.6
6. The sections helped me understand this weeks’ material. 4.6
7. The section design helped me retain the new information. 4.4
8. The section design helped me maintain my attention. 4.5
9. I found the self-assessment feature (Check Your Learning questions) helpful 4.5
10. The answer feedback to the Check Your Learning questions (explanations of the correct
answers) were helpful.
4.5
11. The answer feedback to the Check Your Learning questions was sufficient. 4.3
12. I liked the supplemental content (if any), such as calculator tips and topic videos. 3.9
13. The pace of the content was good. 4.6
14. I enjoyed my experience. 4.5

34. Themes from Student Comments about
Effectiveness of Multimedia Design

35. Representing Student Comments about
Effectiveness of Multimedia Design
“Narrator spoke slowly and clearly.”
“In the randomized experiment section, the colors of the coupons and
the colors of flyers were color coordinated to make the student (me)
visualize how to think about the percentages of the different groups of
people. This demonstration was a real-world scenario that I could
relate to and therefore made the material easy to understand and
increased me chances of retaining the information.”
“What I liked most about this lecture was the
flow. The flow moved from the description of
percentiles and quartiles into showing
examples of how to calculate these items.”
“I appreciate the fact that If I stopped a section
I could resume where a left off.”
“It was also helpful having the tutorial to be long enough to explain the information, but not too
long to lose interest.”

36. Themes from Student Comments about
Helpfulness of Software Features

37. Representing Student Comments about
Effectiveness of Software Features
“I did access the supplemental materials and it was
helpful by providing some more in depth information.”
“I think the use of visual aids and animation help the learner adjust to
the new material more quickly than the text.”
“The Summary is also important because it
reiterates the key elements, which can also
help the viewer retain the information.”
“I really liked the transcript option so you could
keep volume off or go a little quicker if you read
through a familiar topic.”
“I feel like the more I test myself on the subject material, the more comfortable I get with
determining the correct answer and confirming my understanding.”
“it is nice to have graphs and tables involved in an
online class.”

38. Effectiveness of Support for Cognitive Processing of
Information Provided by the Multimedia Lectures
Sub category Number of
Commends
Helped students remember the material 37
Helped students understand the material 32
Helped students maintain their attention 24
Helped students organize information for encoding into long-term
memory
13

39. Representing Student Comments about Cognitive
Support for Information Processing
“Useful tool to use for learning, definitely helped understand the
material.”
“The interactive lectures during the beginning of a
week to watch will most certainly help make you
study and remember the material in the long run.”
“The in-depth discussion about the material is what made me pay attention and want to take
down notes.”
“By using this as a review I feel I was
able to better retain the
information.”
“The explanations break the material down in terms I
can understand and relate to therefore making it
easier to retain the information.”

40. Overall Satisfaction with the Multimedia Lectures
# Theme Number of Comments
1 Reduces phobia of math 7
2 Recommendation to build similar lectures for other courses 10
“Overall, I plan to continue to use the interactive
multimedia lecture until the course ends. The material
and discussion is very clear and concise and to the
point.”
“I had a terrible time in previous math classes and
dreaded taking this one. As it turns out this class has
not been bad at all. The interactive lectures make a
difference for me.”
“I have enjoyed these interactive multimedia lectures since I
began using them at the start of the course. I think they would be
very useful in other classes where topics tend to need more
explanation than just written text.”

41. Instructor Responses to Multimedia

42. THANK YOU