2. SCOPE
● Emergence of Intelligent Tutoring System (ITS)
● Central framework of ITS.
● Different Modules
● New Generation ITS (NGITS)
● Need of emotional feedback
● Facial Emotion Analysis
● Conclusions
● Future Work
2
3. EXISTING TUTORING SYSTEMS (CAI)
● E-learning – Web Based Tutorials using Audio/Video.
● Advantages:
Low Cost.
Learn when you need.
Not constrained by geographical location.
● Disadvantages:
Based on Simple Computer Aided Instructions
No student teacher feedback.
Lack of understanding.
Appears to be boring.
3
4. INTELLIGENT TUTORIAL SYSTEM
Outgrowth of Computer Aided Instructions
with added intelligence.
System where teacher-student communication
strengthens learning process.
Tracks student’s performance.
Adaptive system where next state of instruction
is dependant upon student’s performance.
4
5. CENTRAL FRAMEWORK OF ITS
Traditionally ITS has the following components:
DOMAIN PEDAGOGICAL
MODULE MODULE
STUDENT TASK
MODULE ENVIRONMENT
Fig.1: ITS Framework 5
6. DOMAIN MODULE
Knowledge database.
Depends upon the domain in which the ITS is
intended to instruct.
Domain module is prepared with a view of
cognitive psychology of human skill
acquisition.
Knowledge DECLARATIVE or PROCEDURAL
6
7. Example
An ITS to tutor school mathematics.
Q. How to evaluate an expression with different arithmetic operands?
Declarative: Procedural:
Priority1. ‘( operation )’ Evaluation: (3-4) x 18 / 9 + 5 - 8
Step1. A= (3-4) = -1
Priority2. ‘/’
Step2. B= 18 / 9 = 2
Priority3. ‘x’
Step3. C= A x B = -1 x 2 = -2
Priority4. ‘+’
Step4. D= C+5 = -2+5 = 3
Priority5. ‘-’
Step5. Ans.= 3-8 = -5
Combination of both provides effective learning process 7
8. TASK ENVIRONMENT
Interface for student - teacher communication
GUI may serve the purpose
Tutor displays instructions. Problem Statement Tutor Instructions
Student gives input through a
text editor.
Output Window Students Worksheet
A simulation response to the
student forms a feedback to the
student.
Fig.2: Typical Task Environment
Feedback helps student in
reasoning. 8
9. PEDAGOGICAL / TUTORING MODEL
● Structuring the instructions.
At curriculum level it is sequence of information.
At problem solving level it can intervene to advise
students.
● Next instructions should be on the basis of present
state (can be modeled as a tree data structure).
● Present state includes (Logical decision):
Current stage of domain module.
Knowledge and Emotional state of student. 9
10. INSTRUCTION STRUCTURES
1. Primitive linear structure C
1
C
2
C
3
C
4
C
n
C C
3 4
2. Branched structure C
1
C
2
C
3
C
7
C
n
C C
6 8
3. Multilevel structure C
1
C
2
C
5
C
n
C C
3 4
C C C C1 C
6 7 8 0 n
Cn is quantum of domain C
9
knowledge / information in
the nth stage C1 C1 C1 C1 C 10
1 2 3 4 n
11. STUDENT MODULE
Record of students knowledge state.
Student module is dynamic: Knowledge state is changing
(Modeling is complex).
e.g: Identifying Operands
Identifying Operators
Doing operations
Evaluating Expressions
Solving Equations
Statistical methods used for estimating student’s knowledge state (by
psychophysicist Green and Swets, 1973)
Intelligence of Domain Expert module assess student’s
11
performance
12. ITS DESIGN FLOW
TUTORING SYSTEM
Modeler
Predicted and
Preferred behavior Relations and Update Model
Student Prototypes
Expert Simulator Knowledge Base Student Model
Explanation Data
Problem Solving Students Current state
Situation
Tutor
Problem
Information Advice &Explanation
Data Request
Problem Student 12
Problem Data
13. A T YPICAL ITS
Problem Statement Tutor Instructions
1.Evaluate the expression: Problempriority donest priority are
YouSolved!nd operations
Evaluate 2 1 ☺nd independent
1st have the operation.
st
Hint: Calculate 1operationoperation
Wrong! Now prioritypriority
an done.
2
Z= (3-4) x 18 / 9 + 5 – 8 dependant. the is alsostindependent
Write downCalculate 1st expression
Operation whichmodified priority
Operation.now on the1modified
operation Calculate operation
expressionoperation. Calculate 1st
operation
of the first
priority operation
Output Window Students Worksheet
1. A = -3 5 - 8 = -3
2. B=2 18/9 = 2
3. C= -1 3 - 4 = -1
4. Expression -1 x 2 – 3
5. -2
Wrong -1– 3 =1-2
2 x2=
6. -5 -2 – 3 = -5
13
14. PRACTICALLY IMPLEMENTED SYSTEMS
1. The PUMP Algebra Tutor (PAT)
(Anderson, Corbett, Koedinger and
Pelletier, 1995):
Used for tutoring introductory
algebra in Pittsburg Schools.
Fig.3.: The PAT GUI (Courtesy: Ref.4)
2. The SHERLOCK Project (Lesgold,
Laioie, Bunzo and Eggan, 1992;
Katz and Lesgold, 1993):
It is a practice environment for
electronics troubleshooting
Fig.4.: The SHERLOCK interface commissioned by Air-Force. 14
(Courtesy: Lawrence Elbaum Associates)
15. SHORTCOMING OF CURRENT SYSTEMS
Q. What is the value of force acting on a body of mass
3Kg and moving with a retardation of 4m/s2?
Ans. : -12N Confident
Ans. : -12N Not-Sure
Evaluation by Current systems (Based on knowledge state):
In either of the above cases student will be
assumed to have a knowledge of ‘FORCE’.
15
16. NEW GENERATION ITS
Aimed at developing more adaptive tutors
Expert Module evaluates both:
Knowledge state
Emotional / Mental state
Measuring mental state of learner by bio signals analysis:
• Facial expressions
• Signals from Brain
• Electro-dermal signals
• ECG signals, etc.
16
17. FACIAL EMOTION ANALYSIS
Challenges:
● Involves lot of real time image processing.
● Time is the constraint.
● Should be processed in parallel with knowledge
evaluation.
Hardware / Software co-design approach is adopted for
fast processing.
17
19. Object Segmentation and Labeling
● Labeling : Smoothing small variations in intensity.
● Segmentation : Finding edges or sharp transitions.
Smoothening with linear
resistive network blurs edges
of objects.
Resistive Fuse networks are
used to label and
segmenting the image. 19
20. Resistive Fuse Network
The resistive fuse acts as a linear resistor
for |Vdiff| < Voff
Acts as an open circuit for |Vdiff| > Voff
Fig.5.: Fuse Resistor
The change in voltage at each node
can be calculated from Kirchoff’s current law:
Vout2,2(t+1) – Vout2,2(t) =
v[ ∑ G(Vout i,j – Vout2,2) + σ(Vin2,2- Vout2,2) ]
i,j∈N
2,2
Fig.6.: Segmentation Circuit 20
21. The equation v[ ∑ G(Vout i,j – Vout2,2) + σ(Vin2,2- Vout2,2) ]
i,j∈N
2,2
can be realized in FPGA:
Fig.7.: Raster Scan of the image
(Courtesy: Ref.7)
e = sample pixel
a – i = neighbor pixel
Smem = pixel data Vin is stored
Dmem = Output data Vout is stored
LUT1 performs σ evaluation
LUT2 performs G (fuse value)
per channel (RGB)
21
Fig.8.: FPGA implementation (Courtesy: Ref.7)
22. Gabor Wavelet Transform (GWT)
Wavelet function is multiplication of a
harmonic function and gaussian function
determines frequency
and the direction
GWT is performed over
the segmented object for
feature extraction
22
Fig.9.: GWTs of the face (Courtesy: Ref.7)
23. Matching & Interpretation
Edges of features (eye / lips / nose): Sobel / Gaussian operator
Matching: Genetic Algorithm (GA)
•GA is an iterative process .
•In each step shapes are Fig.10: Matching eye with an ellipse (Courtesy: Ref.8)
matched with known curves
•Termination occurs when
error is minimized.
Fig.11.: Matching lips with an irregular ellipse
(Courtesy: Ref.8)
Final matched curve parameters (like major axis/ minor axis) are matched
with known values to predict emotions:
Happy
Sad 23
Frustration, etc.
24. CONCLUSIONS
ITS has been proved efficient and stronger than simple CAI
Involves student in sustained reasoning activity.
Problem solving tutor helps conceptual understanding as well as
solving real life problems related to a domain.
High level GUI attracts students for learning
The ability to read the mental state of the learner through facial
emotional analysis: Increases Adaptability
Repeated instructions can be delivered on the basis of mental
satisfaction
Helps student in sound understanding
However the accuracy of the emotional analysis can be improved
significantly adding voice information
The main drawback is it is an one-to-one process 24
25. FUTURE DIRECTIONS
Different bio-signals processing: Assess mental state of the
student more correctly.
Electro dermal
signal(GSC): Human
skin is a weak conductor of
electricity.
Fig: Galvanic Skin Response (Courtesy: Springer Images)
Brain Signals (EEG): Higher frequency beta waves (15-25Hz)
and low theta waves implies seriousness
ECG can be analyzed to detect stress, low confidence of the student.
Real time processing several bio-signals will make the design
complex.
But even if half the ability of real human tutor is realized the payoff
to the society will be substantial 25
26. REFERENCES
1. Abdolhossein Sarrafzadeh, Hamid Gholam Hosseini, Chao Fan, Scott P. Overmyer; Facial
Expression Analysis for Estimating Learner’s Emotional State in Intelligent Tutoring
Systems; Proceedings of the The 3rd IEEE International Conference on Advanced Learning
Technologies (ICALT’03); 2003
2. Morteza BIGLARI-ABHARI, Abbas BIGDELI; FPGA Implementation of Facial Expression
Analysis For Intelligent Tutoring Systems; Proceedings of the II International Conference on
Multimedia and Information & Communication Technologies in Education; ICTE ‘2003, Spain
3. Sunandan Chakraborty, Devshri Roy, Anupam Basu; Development of Knowledge Based
Intelligent Tutoring System; Indian Institute of Technology, Kharagpur, India; 2001
4. M.Helander, T. K. Landauer, P. Prabhu (Eds), Elsevier Science B. V.; Intelligent Tutoring
Systems, Handbook of Human-Computer Interaction, Second Edition; 1997
5. Arjen Hoekstra and Joris Janssen; Linking Bio-signals to Transfer of Knowledge Towards
Mind-reading ECAs; Faculty of Electrical Engineering, Mathematics and Computer Science
University of Twente, The Netherlands
6. Teppei NAKANO, Hiroshi ANDO, Hideaki ISHIZU, Takashi MORIE, Atsushi IWATA;
Coarse Image Region Segmentation Using Resistive-fuse Networks Implemented in FPGA;
Graduate School of Life Science and Systems Engineering,Kyushu Institute of Technology;
Graduate School of Advanced Sciences of Matter, Hiroshima University; Hiroshima Prefecture
Industrial Research Institute
7. T. Nakano, T. Morie and A. Iwata; A Face/Object Recognition System Using FPGA
Implementation of Coarse Region Segmentation; SICE Annual Conference in Fukui, Fukui
University, Japan; 2003
8. M. Karthigayan, M. Rizon, R. Nagarajan and Sazali Yaacob; Genetic Algorithm and Neural
26
Network for Face Emotion Recognition; School of Mechatronics Engineering, Universiti
Malaysia Perlis (UNIMAP); 2006
