Presentation of my master degree project in the International Conference on Computer Aided Blended Learning, November 2008.

1. Introduction Background The solution Conclusions
Programming Learning: a Hierarchical Model Based
Diagnosis Approach
Wellington R. Pinheiro, Leliane N. de Barros, Karina V. Delgado
Department of Computer Science
Institute of Mathematics and Statistics - University of S˜o Paulo
a
{wrp,leliane,kvd}@ime.usp.br
November 5th, 2008
ICBL- November 5th, 2008 Slide: 1/22

2. Introduction Background The solution Conclusions
Contents
1 Introduction
2 Background
3 The solution
4 Conclusions
ICBL- November 5th, 2008 Slide: 2/22

3. Introduction Background The solution Conclusions
General Concepts
Intelligent Tutoring System for Programming
Intelligent Tutoring System [Wenger, 1987] (ITS) is an e-learning
tool that uses Artificial Intelligence techniques.
An ITS for programming should be able to:
propose programming problems based on the student knowledge
level;
detect possible faults in the student program solution (i.e., to
diagnose the student’s program);
communicate the detected faults in such a way that the student
can learn with this experience;
construct the student model (representation of the student’s
knowledge) based on the results of the tutor diagnosis.
ICBL- November 5th, 2008 Slide: 3/22

4. Introduction Background The solution Conclusions
General Concepts
Intelligent Tutoring System for Programming
Intelligent Tutoring System [Wenger, 1987] (ITS) is an e-learning
tool that uses Artificial Intelligence techniques.
An ITS for programming should be able to:
propose programming problems based on the student knowledge
level;
detect possible faults in the student program solution (i.e., to
diagnose the student’s program);
communicate the detected faults in such a way that the student
can learn with this experience;
construct the student model (representation of the student’s
knowledge) based on the results of the tutor diagnosis.
ICBL- November 5th, 2008 Slide: 3/22

5. Introduction Background The solution Conclusions
General Concepts
Diagnosis of the student’s program
ProPAT [Delgado, 2005]
Features:
Able to detect logical faults in students’ programs.
Based on previous work that uses model based diagnosis to
perform program debugging (model based software debugging).
Only needs the program, the test cases and some information
about the programming language to detect the faulty
instructions.
Limitation:
The faults are communicated to the student in terms of program
lines, which gives little or no information to the student to find
out his mistakes or misconceptions.
ICBL- November 5th, 2008 Slide: 4/22

6. Introduction Background The solution Conclusions
General Concepts
Model Based Diagnosis
Model based diagnosis (MBD) [de Kleer and Williams, 1987][Reiter, 1987]
An Artificial Intelligence technique to find faulty components in
physical devices.
Reasons over models that describe the correct behavior of the system.
Involves three sub-tasks [Benjamins, 1993]: symptom detection,
hypothesis generation and hypothesis discrimination.
Model based software debugging (MBSD) [Mayer et al., 2002]
An application of MBD to program debugging.
Basic idea: step by step program simulation that points out the
instructions that justifies the observable discrepancies (i.e., the
differences between program outputs and expected outputs -
symptoms).
Can be used by an ITS to try to understand the difference between
the student’s intentions and his program.
ICBL- November 5th, 2008 Slide: 5/22

7. Introduction Background The solution Conclusions
General Concepts
MBD versus MBSD
Comparison between model based diagnosis and model based software
debugging
Model based diagnosis Model based software debugging
ICBL- November 5th, 2008 Slide: 6/22

8. Introduction Background The solution Conclusions
General Concepts
Model based software debugging
ICBL- November 5th, 2008 Slide: 7/22

9. Introduction Background The solution Conclusions
General Concepts
MBSD for programming learning: limitations
Pointing out the possible faulty lines of the program can not be
informative enough for a novice to correct his program.
Even for a regular size program, there can be many hypotheses
of failure to be communicated (and discriminated) to the
student, which can make him tired and confused.
A novice student does not have enough knowledge to make
predictions about the program behavior.
ICBL- November 5th, 2008 Slide: 8/22

10. Introduction Background The solution Conclusions
Goals
Proposal
To extend the MBSD technique with the use of Hierarchical Model
Based Diagnosis (hierarchical diagnosis), so that:
Faults in the student’s program can be found at different levels
of the program abstraction.
Functions, procedures and programming (elementary) patterns,
are viewed as abstract components.
The communication with the student can be made in terms of
these abstract components, i.e., through a high-level language
instead of program lines.
In the average case, the tutor would communicate a smaller set
of hypotheses of failure to the student.
ICBL- November 5th, 2008 Slide: 9/22

11. Introduction Background The solution Conclusions
HMBD
Hierarchical Model Based Diagnosis (HMBD)
A technique used to increase performance during the diagnosis.
The system is modeled at different levels of abstraction.
Abstractions are used to represent components (structural
abstraction) or the system behavior (behavioral abstraction).
Uses traditional MBD techniques to find the solution.
ICBL- November 5th, 2008 Slide: 10/22

12. Introduction Background The solution Conclusions
HMBD
Models and abstractions
Example:
ICBL- November 5th, 2008 Slide: 11/22

13. Introduction Background The solution Conclusions
HMBD
Abstraction tree
Subsystems are supposed to be independent
ICBL- November 5th, 2008 Slide: 12/22

14. Introduction Background The solution Conclusions
HMBD
Abstract component
Models of an abstract component
An abstract component can be described by:
internal structural model, external structural model and behavioral model.
Behavior of an abstract component
An abstract component The behavior of AC 1 is So, the behavior of C 1,
AC 1 and the aggre- assumed as correct C 2 and C 3 must be as-
gated components C 1, sumed as correct
C 2 and C 3
This behavior can be formally described by the following axiom:
ok(CA) → ok(C1 ) ∧ ok(C2 ) ∧ · · · ∧ ok(Cm ) (1)
ICBL- November 5th, 2008 Slide: 13/22

15. Introduction Background The solution Conclusions
HMBD
Top-down Hierarchical Diagnosis
An algorithm to perform hierarchical diagnosis can be described as
[Mozetiˇ, 1991]:
c
1 Verify the observations that can be used at each level of
abstraction.
2 For each level i, from l to 0, where l is the highest level that
does not hide all the connections with value, find the
hypothesis set with an MBD algorithm. For each abstract
component CA not involved in any hypothesis set at level i,
apply the Axiom 1 for the level i − 1.
3 Return the hypothesis set for level 0.
ICBL- November 5th, 2008 Slide: 14/22

16. Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22

17. Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22

18. Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22

19. Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22

20. Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22

21. Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22

22. Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22

23. Introduction Background The solution Conclusions
HMBD
HMBD example
ICBL- November 5th, 2008 Slide: 15/22

24. Introduction Background The solution Conclusions
MBSD+HMBD
Basic idea of MBSD + HMBD
Give the opportunity for the student to understand and correct
the faults in the program at different levels of abstraction.
After the hypothesis set for a specific level has been found, the
student can choose between:
change his program with an expectation to correct the faults;
discriminate the hypotheses at the current level of abstraction
(by supplying new observations about the expected behavior of
the program);
try to correct the problem at a more detailed level, by replacing
an abstract component by its aggregated components
(refinement).
ICBL- November 5th, 2008 Slide: 16/22

25. Introduction Background The solution Conclusions
MBSD+HMBD
Program example
Programming Problem: Given a set v of integer numbers, calculate the difference
between the sum of even numbers and odd numbers.
1 public int evenMinusOdd(int v[]) {
2 int n = v.length;
* 3 int counter = 1;
4 int evenSum = 0;
5 int oddSum = 0;
6
* 7 while (counter < n) {
* 8 if (v[counter] % 2 == 0) {
* 9 evenSum = evenSum + v[counter];
* 10 } else {
* 11 oddSum = oddSum + v[counter];
* 12 }
* 13 counter = counter + 1;
* 14 }
15 return evenSum - oddSum;
16 }
(*) Lines used to compose a Counting Loop elementary pattern.
ICBL- November 5th, 2008 Slide: 17/22

26. Introduction Background The solution Conclusions
MBSD+HMBD
Program model
Level 1
v.length n = v.length
n
Assignment counter
evenSum = 0
const1 evenSum Counting evenSun
evenSum - oddSum result oddSum = 0
0 Assignment Loop oddSum
Expr Assignment
oddSum = 0 oddSum
const2
0 Assignment
Level 0
const3 counter = 1
v.length n = v.length n 1 Assignment
Assignment counter
counter
LOOP
evenSun
const1 evenSum = 0 evenSum counter < n evenSum - oddSum oddSum = 0
result
Assignment oddSum
0 Expr Expr Assignment
LOOP
const2 oddSum = 0 oddSum BODY
0 Assignment
ICBL- November 5th, 2008 Slide: 18/22

27. Introduction Background The solution Conclusions
MBSD+HMBD
Characteristics of an algorithm for MBSD + HMBD
We consider the following items as important characteristics of our
algorithm to MBSD + HMBD for programming learning:
The student can discriminate the hypotheses of failure at
different levels of abstraction.
Abstract components can be refined according to the decision
of the student.
At any level of abstraction, if the student corrects the faults in
his program, the diagnosis process is finished (even if the
current level has abstract components).
All observations can be used during the diagnosis process (i.e.,
there is no connection with observed value hidden in an
abstract component, during the whole diagnosis process).
ICBL- November 5th, 2008 Slide: 19/22

28. Introduction Background The solution Conclusions
Conclusions and future work
In this presentation, we showed:
A technique of model based diagnosis to locate logical faults in
programs (MBSD) and its limitations when applied to programming
learning.
An extension of the hierarchical diagnosis to detect faults in programs
represented with different levels of abstraction, considering abstract
components, such as functions, procedures and elementary patterns.
A suitable approach to debug the students’ program and communicate
the faults in a high-level language (considering abstractions), that we
believe to be more appropriate to promote learning.
We have planned to:
Finish the implementation for MBSD + HMBD;
Do an experimental evaluation with students.
ICBL- November 5th, 2008 Slide: 20/22

29. Introduction Background The solution Conclusions
References
Benjamins, R. (1993).
Problem Solving Methods for Diagnosis.
PhD thesis, University of Amsterdam.
Chittaro, L. and Ranon, R. (2004).
Hierarchical model-based diagnosis based on structural abstraction.
Artif. Intell., 155(1-2):147–182.
de Kleer, J. and Williams, B. C. (1987).
Diagnosing multiple faults.
Artif. Intell., 32(1):97–130.
Delgado, K. V. (2005).
Diagn´stico baseado em modelos num sistema tutor inteligente para programa¸˜oo
o ca
com padr˜es pedag´gicos.
o o
Disserta¸˜o de mestrado, Instituto de Matem´tica e Estat´
ca a ıstica.
Johnson, W. L. and Soloway, E. (1984).
Proust: Knowledge-based program understanding.
In ICSE ’84: Proc. of the 7th international conference on Software engineering,
pages 369–380, Piscataway, NJ, USA. IEEE Press.
ICBL- November 5th, 2008 Slide: 21/22

30. Introduction Background The solution Conclusions
References
Mayer, W., Stumptner, M., Wieland, D., and Wotawa, F. (2002).
Observations and results gained from the jade project.
In Proc. of the 13th International Workshop on Principles of Diagnosis,
Semmering, Austria.
Mozetiˇ, I. (1991).
c
Hierarchical model-based diagnosis.
Int. J. Man-Mach. Stud., 35(3):329–362.
Reiter, R. (1987).
A theory of diagnosis from first principles.
Artif. Intell., 32(1):57–95.
Wenger, E. (1987).
Artificial intelligence and tutoring systems: Computational and cognitive
approaches to the communication of knowledge.
Morgan Kaufmann Press.
ICBL- November 5th, 2008 Slide: 22/22