Learning Management Systems Evaluation based on Neutrosophic sets

1. Doctoral Program of Information Systems
PhD Research Proposal
<E-Learning>
Prof. Dr. M. Badr Senousy
Prof of Computer and Information Systems
Learning Management Systems Evaluation
Prof. Dr. Alaa El Din M. Riad
Prof of Computer and Information Systems
Submitted by:
Nouran M. Radwan
Faculty of Computer and information Sciences _Mansoura University
2018
Supervised by:
2/22/20181

2. In the name of Allah, the Most Gracious and the Most Merciful
 Special appreciation goes to my supervisor Prof. Dr. Alaa Riad for his supervision
and constant support. I also owe Prof. Dr. M Badr Senousy great thanks for his
constructive comments and guidance throughout the dissertation work.
 My Thanks to Prof. Dr. Ibrahim El Henawy and Prof. Dr. Atef Ghalwash for
agreeing to be a discussant for my thesis.
 I am most grateful to Prof. Dr. Florentine Smarandache [Department of
Mathematics, University of New Mexico, USA] and Prof. Dr. Ahmed Salama
[Department of Mathematics and Computer Science, Faculty of Sciences, Port
Said University, Egypt] for lending me their expertise and helping me to
understand neutrosophic approach.
2
Acknowledgment

3.  "A new expert system for learning management systems evaluation
based on neutrosophic sets." Expert Systems 33.6 (2016): 548-558.
Indexed in ISI/Thomson Reuters, IF:1.18
 "Neutrosophic AHP Multi Criteria Decision Making Method Applied on
the Selection of Learning Management System". International Journal
of Advancements in Computing Technology 8.5 (2016).
Indexed in Scimagojr/SJR, IF:0.115
 "Approaches for Managing Uncertainty in Learning Management
Systems." Egyptian Computer Science Journal 40.2 (2016): 1-7.
 "Neutrosophic Logic Approach for Evaluating Learning Management
Systems." Neutrosophic Sets and Systems 11(2016):3-7.
3
Publications
2/22/2018

4. Agenda
4
Introduction : Problem Statement , Contribution
Related Work
Research Methodology
Learning Management Systems Evaluation
Results
Conclusion and Future Work
2/22/2018

5. Agenda
5
Introduction : Problem Statement , Contributions
Related Work
Research Methodology
Learning Management Systems Evaluation
Results
Conclusion and Future Work
2/22/2018

6. 2/22/20186
Problem Statement
 Previous studies conducted to evaluate LMSs
– unilateral view.
– no comprehensive model.
– full information availability condition.
 LMSs described by uncertainty terms.

7. Contributions
 Proposing a comprehensive model for LMSs evaluation
under uncertainty.
2/22/20187
Software
Selection
• Neutrosophic
Analytical
Hierarchy Decision
Making Method
Quality
Assessment
• Neutrosophic
Expert System
Success
Measurement
• Neutrosophic
Success
Measurement
Model

8. 2/22/20188
Introduction
Data or
Information
Certainty
True
False
Uncertainty
Vagueness
Imprecision
Ambiguity
Inconsistency
Fig 1. Certain and Uncertain Data or Information

9. 9 2/22/2018
Introduction
Main
Uncertainties
Types
Meaning Example
Vagueness when available information is
normally graded.
The boy is nearly tall.
Imprecision when the obtainable information is
not defined.
The temperature of the machine is
between 88-92 °C.
Ambiguity when information leads to several
possible interpretations.
The flower color may be red or
yellow.
Inconsistency when available information is
contradicted and cannot be true at
the same time.
The chance of raining tomorrow is
70%, the chance of not raining is 50%.
Table 1. Main Uncertainties Types

10. 10
1
0
X
Y
MembershipFunction
Fig 2. Fuzzy Set
1
0 Y
MembershipFunction
X
Fig 3. Type 2 Fuzzy Set
1
0 Y
MembershipFunction
X
Fig 4. Intuitionistic Fuzzy Set
MembershipFunction
Y
X
Fig 5. Neutrosophic Set
Introduction
0
0
0
1
1
1

11. • Weather Forecasting
– Precipitation
– Windspeed
11
Introduction
-Temperature
-Humidity
???
Fuzzy
P: 70%
W: 60%
 Raining 70%
Type 2 Fuzzy
P: 50-70%
W: 45-65%
 Raining 50%-65%
Intuitionistic
Fuzzy
P: 50%, P’:20%
W: 50%, W’:30%
 Raining 50%,
Not Raining 20%
Neutrosophic
P:50%, P’:50%
W:70%, W’:40%
 Raining 50%,
Not Raining 30%,
Indeterminacy 30%

12. 12
Multivalued Logic
Models
Uncertainty Data types
Vagueness Imprecision Ambiguity Inconsistency
Fuzzy  ------- ------- -------
Type 2 fuzzy   ------- -------
Intuitionistic
Fuzzy
  ------- -------
Neutrosophic    
Table 2. Multivalued Logic Models and Uncertainty Data Types
2/22/2018
Introduction

13.  Neutrosophic logic is proposed by Smarandache in which
variable is described by
₋ t is the degree of truth
₋ f is the degree of false
₋ i is the degree of indeterminacy.
2/22/201813
Introduction

14. 2/22/201814
Introduction
Examples of Indeterminacy

15. 2/22/201815
Introduction
Medical
Domain
• Diagnosis
and
prognosis
• Drug test
reliability
Data Mining
• Clustering
• Forecasting
Image
Processing
• Image
Retrieval
• Image
denoising
GIS Topology
• Spatial
regions with
ambiguity
boundary
Decision
Making
• Recruitment
• Investment
Neutrosophic Applications

16. Agenda
16
Introduction : Problem Statement , Contribution
Related Work
Research Methodology
Learning Management Systems Evaluation
Results
Conclusion and Future Work
2/22/2018

17. 17 2/22/2018
Year Author Title Concern Technique
2013
Jayakumar
and
Banbehari
Website Quality Assessment Model for
Developing Efficient E-Learning
Framework
Quality
Assessment
Questionnaire
2013
Valdez-
Silva et al.
Expert System for Evaluating Learning
Management Systems Based on
Traceability
Quality
Assessment
Traceability
Model
2014
Mtebe et
al.
A Model For Assessing Learning
Management System Success In
Higher Education In Sub-Saharan
Countries
Success
Measurement
Questionnaire
2014
Teng-chiao
Lin et al.
Evaluation Model For Applying An E-
learning System In A Course: An Analytic
Hierarchy Process–multi-choice Goal
Programming Approach
Selection
Process
Analytic
Hierarchy
Process
Related Work

18. 18 2/22/2018
Year Author Title Concern Technique
2014
Bhuiyan
and Kundu
Developing and evaluating a desktop-
based learning management system
Quality
Assessment
Questionnaire
2016 Sahid et al.
Evaluation and measurement of
Learning Management System based on
user experience
Quality
Assessment
Questionnaire
2016
M. Attia et
al.
A Model For Assessing And Enhancing
Efficiency Of E-learning Systems
Success
Measurement
Dempster-
Shafer theory
2017
Kennedy
Hadullo et
al.
A Model For Evaluating E-learning
Systems Quality In Higher Education in
Developing Countries
Success
Measurement
Questionnaire
Related Work

19. Agenda
19
Introduction : Problem Statement , Contribution
Related Work
Research Methodology
Learning Management Systems Evaluation
Results
Conclusion and Future Work
2/22/2018

20. 20 2/22/2018
Fig 6. Research steps of the study
Research Methodology
Literature Review
Identifying LMSs evaluation
process
Semi-Structured
Questionnaire
Understanding neutrosophic
concept and discussing LMSs
selection criteria, quality and
success factors
Expert Questionnaire
Implementing and validating
the proposed LMSs evaluation
model

21. Agenda
21
Introduction : Problem Statement , Contribution
Related Work
Research Methodology
Learning Management Systems Evaluation
Results
Conclusion and Future Work
2/22/2018

22. 22 2/22/2018Fig 7. LMSs Evaluation Proposed Model
Phase 1: Software
Selection
Phase 2: Quality
Assessment
Phase 3: Success
Measurement
Identify LMSs criteria
requirements and
alternatives
Compare alternatives
under each criteria
Ranks the overall weights
and select the best
alternative
Determine LMSs Quality
factors and attributes
Defining memberships
for LMSs quality factors
Obtaining and validating
quality results
Define Factors affecting
LMSs success
Extracts each factor
Importance
Develop a comprehensive
model for success
measurement
Learning Management Systems Evaluation

23. 23
 LMSs selection is a multi criteria decision making process .
 The AHP (Analytical Hierarchy Process) method provides
help decision makers in analyzing LMSs criteria.
 The main limitation of AHP is
₋ its incapability of reflecting uncertain data
Learning Management Systems Evaluation
Phase 1 : LMSs Software Selection
2/22/2018

24. 24
The procedures of the NAHP are as follows:
2/22/2018
1 • Define criteria, sub criteria and alternatives of the problem.
2 • Construct the hierarchy of the considered problem.
3 • Identify the neutrosophic numbers that correspond to the 1–9 Saaty scale.
4 • Collect neutrosophic preference via the pairwise comparison from experts
5 • Check the consistency of each pairwise comparison.
6 • Calculates the relative weight of criteria.
7 • Compares the alternatives under each criterion or sub criterion.
8 • Ranks the overall weights, and a choice is made of the best alternative.
Learning Management Systems Evaluation
Phase 1 : LMSs Software Selection

25. 25
Fig 8. Decision Hierarchy Model of the LMS
SelectingtheMostAppropriateLMS Cost
Evaluative
Tools
Student Tracking
Exam Pool
Compatibility
Platform
Content Developing
Tools
Support
Documentation
Technical
Sustainability
Moodle
Sakai
Atutor
ILIAS
Dokeos
2/22/2018
Learning Management Systems Evaluation
Phase 1 : LMSs Software Selection

26. 26
Neutrosophic Expert System Development
2/22/2018
1
• Determine system requirements represented in inputs, rules and outputs.
2 • Experts define inputs, rules of knowledge base and output of the system.
3 • Inputs are presented in truth, falsity and indeterminacy membership functions.
4 • Inference engine create the rules which are stored in the neutrosophic inference.
5 • Neutrosophic sets are converted into a single crisp value which has triplet format
6 • Implementing the neutrosophic expert system by using inputs, rules and output
7 • Validate the system to ensure that the output is equivalent to human experts.
Learning Management Systems Evaluation
Phase 2 : LMSs Quality Assessment

27. 27 Fig 9. LMSs System Quality of Neutrosophic Expert System 2/22/2018
Learning Management Systems Evaluation
Phase 2 : LMSs Quality Assessment
System Quality
Usability
Efficiency
Error Tolerance
Learnability
Memorability
User Satisfaction
Accessibility
Navigability
Robustness
Understandable
Reliability
Fault Tolerance
Maturity
Recoverability

28. Fig 11. Efficiency Indeterminacy Input Membership
Fig 10. Efficiency True Input Membership
28
Fig 12. Efficiency False Input Membership
2/22/2018
Learning Management Systems Evaluation
Phase 2 : LMSs Quality Assessment

29. Fig 14. Indeterminacy System Quality Knowledge Base
Fig 13. True System Quality Knowledge Base
29
Fig 15. False System Quality Knowledge Base
2/22/2018
Learning Management Systems Evaluation
Phase 2 : LMSs Quality Assessment

30. Fig 17. System Quality Indeterminacy Membership
Fig 16. System Quality System Quality True Membership
30
Fig 18. System Quality False Membership
Learning Management Systems Evaluation
Phase 2 : LMSs Quality Assessment

31. 31
Fig 19. Factors Affecting LMSs
2/22/2018
Learning Management Systems Evaluation
Phase 3 : LMSs success measurement
LMSs Success
Personal Factors
Learner
Instructor
System Factors
System Quality
Information Quality
Service Quality
Organizational
Factors
Management Support
Training Support
Supportive
Factors
Ethical and Legal Issues
Cost

32. SystemDesign
32
Organizational
Factors
- Management
Support
- Training
Supportive
Factors
- Ethical and
Legal issues
- Cost
System Usage Perceived Usefulness Perceived Ease of Use
System Outcome User Satisfaction Intention to Use
LMSs Success
Fig 20. The Proposed LMSs Success Measurement Model 2/22/2018
Learning Management Systems Evaluation
Phase 3 : LMSs success measurement
System Factors
- System Quality
- Information
Quality
- Service Quality
Personal Factors
- Instructor
- Learner

33. Agenda
33
Introduction : Problem Statement , Contribution
Related Work
Research Methodology
Learning Management Systems Evaluation
Results
Conclusion and Future Work
2/22/2018

34. 34
Fig 21. Criteria Weight in Neutrosophic Numbers
2/22/2018
Results
Phase 1 : LMSs Software Selection
Cost
Evaluative
Tools
Compatibility Support Sustainability
TRUE 0.4292 0.6382 0.5632 0.5011 0.4779
INDETERMINANACY 0.5902 0.3298 0.4087 0.5027 0.5404
FALSE 0.5708 0.3618 0.4368 0.4989 0.5221
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7

35. 35
Fig 22. Criteria Weight in Deneutrosophied Numbers
0.4195
0.6542
0.57725
0.4992
0.46875
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Cost Evaluative Tools Compatibility Support Sustainability
2/22/2018
Results
Phase 1 : LMSs Software Selection

36. 36 2/22/2018
Results
Phase 1 : LMSs Software Selection
Moodle Atutor Dokeos Sakai ILIAS
TRUE 0.8838 0.8709 0.8315 0.8147 0.802
INDETERMINANACY 0.0949 0.112 0.1655 0.1895 0.2096
FALSE 0.1162 0.1219 0.1685 0.1853 0.198
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Fig 23. Overall Score of LMSs Alternatives

37. 37 2/22/2018
Results
Phase 2 : LMSs Quality Assessment
1 2 3 4 5 6 7
Fuzzy 0.4722 0.5625 0.5952 0.6061 0.6458 0.6944 0.7333
Neutrosophic 0.4931 0.5193 0.5378 0.546 0.5446 0.5837 0.6336
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
SystemQuality
Fig 24. System Quality in Fuzzy and de-neutrosophied numbers

38. 38 2/22/2018
Results
Table 3. Factor Importance by Fuzzy Set and Neutrosophic Set
Phase 3 : LMSs success measurement
Factors Dimensions Factors Importance
Fuzzy
set
Neutrosophic set Deneutrosophied
set
Personal Factors F1: Learner Characteristics 64% (0.66, 0.35, 0.27) 67%
F2: Instructor Characteristics 74% (0.75, 0.36, 0.26) 70%
System Factors F3: System Quality 64% (0.66, 0.36, 0.31) 66%
F4: Information Quality 62% (0.64, 0.32, 0.29) 68%
F5: Service Quality 64% (0.66, 0.36, 0.30) 66%
Organizational
Factors
F6: Management Support 74% (0.75, 0.35, 0.28) 70%
F7: Training 60% (0.61, 0.34, 0.31) 65%
Supportive
Factors
F8: Ethical and Legal issues 65% (0.67, 0.32, 0.27) 68%
F9: Cost 61% 0.63, 0.36, 0.35) 64%

39. 39 2/22/2018
Results
Phase 3 : LMSs success measurement
Dimension Weight Importance
Fuzzy
set
Neutrosophic set Deneutrosophied
number
W1: Learner effect on perceived usefulness. 62% (0.64, 0.32, 0.20) 70%
W2: Instructor effect on perceived usefulness. 63% (0.53, 0.25, 0.23) 70%
W3: System quality affect perceived usefulness. 70% (0.61, 0.37, 0.20) 67%
W4: Information quality effect on perceived usefulness. 70% (0.72, 0.34, 0.29) 69%
W5: Service quality effect on perceived usefulness. 62% (0.64, 0.32, 0.34) 66%
W6: Management support impact on perceived usefulness. 58% (0.61, 0.37, 0.38) 62%
W7: Training effect on perceived usefulness. 62% (0.64, 0.32, 0.20) 69%
W8: Ethical and legal issues affect perceived usefulness. 74% (0.75, 0.30, 0.20) 73%
W9: Cost affect perceived usefulness of learners. 61% (0.63, 0.35, 0.37) 64%
Table 4. Weight Importance by Fuzzy Set and Neutrosophic Set

40. Agenda
40
Introduction : Problem Statement , Contribution
Related Work
Research Methodology
Learning Management Systems Evaluation
Results
Conclusion and Future Work
2/22/2018

41.  Applying neutrosophic logic as theoretical and practical
aspects of in LMSs evaluation.
 Presenting a comprehensive model to evaluate LMSs under
uncertainty.
 Using neutrosophic sets gives obvious intuition than the
fuzzy logic which is limited in representing paradoxes.
 Neutrosophic logic is needed for suitable description of an
object in uncertain environment such as expert system.
41
Conclusion

42.  Personal LMSs.
 Adaptive test generation
 Talent LMSs.
42
Future Work
2/22/2018

43. There is no way to express sincerely thank to the experts :
 Alberto Caballero [Associate professor, Computer Science, Universidad Católica San Antonio
de Murcia, Spain ]
 Alex Salas [Learning Management Systems Admin]
 David Cook [Lecturer in the School of Science and a member of the ECU Security Research
Institute]
 Krzysztof Nesterowicz [E‐Learning specialist, PhD candidate, National University of Public
Service, Budapest, Hungary]
 Mugenyi Justice Kintu [PhD candidate, Dept. of Educational Sciences, Vrije Universiteit
Brussel]
 Stylianos Sergis [M.Sc. in "Informatics in Education", PhD candidate, University of Piraeus,
Greece]
 Tarik A. Rashid [Associate Professor on Salahaddin University, Kardistan]
 Vjekoslav Hlede [Senior Learning Management System Specialist, PhD candidate, American
Association of Anesthesiologists]
2/22/201843
Thanks Experts

44. 2/22/201844
Thanks