sistem evaluasion: measuring student acceptance and satisfaction

1. Measuring Student
Acceptance and Satisfaction
An Online Distance
Education Class Case Study

2. Researchers
Ralph Gabbard, Indiana State University
and Indiana University School of Library
and Information Science
Andrew Dillon, Indiana University School of
Library and Information Science
Julia Newcome, Community College of
Beaver County

3. Presentation Overview
 Brief overview of research Environment
 Identification of first and second research
questions
 Discussion of method and preliminary results
 Identification of third research question
 Discussion of method and preliminary results
 Study limitations
 Further Research

4. Research Background
 Setting
– Community College of Beaver County
 Two English Composition Classes
– Traditional class
– Online class

5. Web Site

6. Class Dialogues
 These dialogues are extremely
important and form the basis of the
online experience.
 They were sent out as emails as well as
posted on the Web site.
 Example

7. First and Second Research
Questions
 What kinds of attitudes and perceptions
do students’ have about online classes?
 How do students’ attitudes and
perceptions change over the course of
their learning experience?

8. Technology Acceptance
Model (TAM)
 The developers of TAM were Fred D.
Davis, Richard Bagozzi and Paul
Warshaw
 TAM is based on Ajzen and Fishbein’s
Theory of Reasoned Action (TRA)

9. TAM Continued

10. TAM Questions (1)
 Perceived Ease of Use
– I would find learning online easy
– Learning online would be easy for me
– I would find it easy to learn online
– It would be easy for me to become skillful
at learning online

11. TAM Questions (2)
 Perceived Usefulness
– Enrolling in a distance class would improve
my performance in college
– Enrolling in a distance education class is
a(n) _____
– Enrolling in a distance education class
would increase my productivity in college
– I would find distance education useful in
college

12. TAM Questions (3)
 Attitude Toward Using
– Enrolling in a distance education class is
a(n) _____ idea
– A distance education class would enhance
my effectiveness in college
– I ____ the idea of distance education
– Learning online would be _____

13. TAM Procedure
 TAM-based questionnaire was given 3
times to online students

14. TAMS Preliminary Data
Analysis
5.736
6.062 6.111
6.111
4.968
5.517
6.055
5.906
5.321
0
1
2
3
4
5
6
7
1 2 3
Questionnaire Number
Mean
Value
peu
pu
atu

15. TAM Validity and Reliability
 Morris and Dillon report that in the TAM
variable Perceived Usefulness has a
reliability of .92; the variable Perceived
Ease of Use with a reliability of .90; and
Attitude toward Using with a reliability of
.85

16. Third Research Question
 How do student attitudes about the
effectiveness of the instructor and the
course compare between traditional
classes and online classes?

17. Student Disconfirmation
Model (SDM)
 Two goals
– To satisfy and delight the customer
 Delight is defined as the most extreme
form of satisfaction
 Disconfirmation is defined as the
difference between perceived and
expected service quality

18. SDM Components
 Divided the instructional experience into three
main components:
– Instructor
 Knowledge, Organization, Fairness; Courteousness,
interesting,Enthusiasm
– Content/text/notes
 Text, Examples, Handouts, Lecture (dialogues)
– Grading
 Difficulty of tests, Material on tests, Writing assignments

19. SDM Questions (1)
 Instructor Component
– The instructor’s knowledge of the subject matter
was:
– The instructor's fairness was:
– The instructor's courteousness was:
– The instructor's ability to make the course
interesting was:
– The instructor's enthusiasm was:
– The course material organization was:

20. SDM Questions (2)
 Content Component
– The readability of the tests, quizzes, and
worksheets was:
– The usefulness of class examples was:
– The usefulness of handouts was:
– The usefulness of the lecture(dialogue) material
was:
– The amount of material covered in the course
was:

21. SDM Questions (3)
 Grading Component
– The level of difficulty of tests, quizzes,
and worksheets was:
– The difficulty of the writing assignments
was:
– The type of material covered on the tests
was:

22. SDM Procedure
 SDM-based questionnaire was given 3
times to both online and traditional class
students.

23. SDM Scoring
Levelof
Disconfirmation
Response Satisfied Delighted
Much better than
expected
1 1
About as expected 1 0
Much worse than
expected
0 0

24. SDM Preliminary Data
Analysis
Comparison of Instructor Component
0 20 40 60 80 100
%Satisfied
%Delighted
%Satisfied
%Delighted
%Satisfied
%Delighted
Questionnaire
One
Questionnaire
Two
Questionnaire
Three
Percentage
Traditional
Online

25. SDM Preliminary Data
Analysis
0 20 40 60 80 100
Percentage
%Satisfied
%Delighted
%Satisfied
%Delighted
%Satisfied
%Delighted
Questionnaire
One
Questionnaire
Two
Questionnaire
Three
Comparsion of Content Component
Traditional
Online

26. SDM Preliminary Data
Analysis
0 20 40 60 80
Percentage
%Satisfied
%Delighted
%Satisfied
%Delighted
%Satisfied
%Delighted
Questionnaire
One
Questionnaire
Two
Questionnaire
Three
Comparison of Grading Component
Traditional
Online

27. SDM Preliminary Data
Analysis
0 20 40 60 80 100
Percentage
%Satisfied
%Delighted
%Satisfied
%Delighted
%Satisfied
%Delighted
Questionnaire
One
Questionnaire
Two
Questionnaire
Three
Online Components
Grading
Content
Instructor

28. SDM Preliminary Data
Analysis
0 20 40 60 80 100
Percentage
%Satisfied
%Delighted
%Satisfied
%Delighted
%Satisfied
%Delighted
Questionnaire
One
Questionnaire
Two
Questionnaire
Three
Traditional Components
Grading
Content
Instructor

29. Study Limitations
 No randomization; students self
selected
 TAM may be too simplistic
 Questions about the validity of SDM

30. Further Research
 More analysis of TAM data and
comparisons between semesters
 More analysis of SDM data

31.  Presentation can be found at
http://odin.indstate.edu/level1.dir/cml/ALN/aln.html
 Examples of questionnaires can be found at
http://odin.indstate.edu/level1.dir/cml/ALN/tam.html
http://odin.indstate.edu/level1.dir/cml/ALN/sdm.html
 Bibliography of Sources can be found at
http://odin.indstate.edu/level1.dir/cml/ALN/bib.pdf