Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
University of Pittsburgh Intelligent Systems Program
Outline
• Introduction
• Hypothesis testing (pre-study)
• Findings and challenges
• JavaParser
• Conclusion &Future works
...
Introduction
There are two possible ways for modeling
student’s knowledge:
• Coarse-grained knowledge modeling
• Fine-grai...
Motivation
Fine-grained indexing?
Do we really need it ? ...
4AIEDCS 2013
• An important aspect of task sequencing in Adaptive
Hypermedia is the granularity of the domain model
and the task indexi...
• Typical approach to present programming knowledge
which uses coarse-grained topics like “loops” and
“increment” allows r...
Outline
• Introduction
• Hypothesis testing (pre-study)
• Findings and challenges
• JavaParser
• Conclusion &Future works
...
Pre-study
Knowledge Maximizer:
• An exam preparation tool for Java programming
• based on a fine-grained concept model of ...
11AIEDCS 2013
KM Parameters:
• How prepared is the student to do the
activity?
12





r
r
M
i
i
M
i
ii
w
wk
K
K: user knowledge le...
• How prepared is the student to do the
activity?
• What is the impact of the activity?
13




 o
o
M
i
i
M
i
ii
w
w...
• How prepared is the student to do the
activity?
• What is the impact of the activity?
• What is the value of repeating t...
• Determining the sequence of top 10
activity with the highest rank using:
15
SIKR 
R : activity rank
K : knowledge lev...
Evaluation
• We conducted a classroom study for the Java Programming
undergraduate course.
• Study started on Dec. 4th 201...
Evaluation Measures
We grouped participants into two groups:
• KM: those who made at least ten attempt using
KM (n = 9)
• ...
Outline
• Introduction
• Hypothesis testing (pre-study)
• Findings and challenges
• JavaParser
• Conclusion &Future works
...
System Usage Summary
19
6.20%
43.50%
50.20%
34.60%
45.30%
20.10%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80...
Results
20
58% 64%
0%
50%
100%
KM QG,P+
Success rate
AIEDCS 2013
-12.80% -13.70%
11.53%
-15.00%
-10.00%
-5.00%
0.00%
5.00%...
• Fine-grained knowledge modeling will push students
toward appropriate complex questions which does not
lead to miserable...
JAVA Ontology
http://www.sis.pitt.edu/~paws/ont/java.owl
23AIEDCS 2013
~ 400 node in ontology
~ 160 concept (leaf nodes)
24AIEDCS 2013
 103 JAVA Parameterized
Question
 1-5 classes per question
 5-52 concepts per question
~ 40% of questions...
25AIEDCS 2013
52 concepts or more
in a question ?
26AIEDCS 2013
JavaParser: A tool for automatic
indexing of Java Problems
Outline
• Introduction
• Hypothesis testing (pre-study)
• Findings and challenges
• JavaParser
• Conclusion &Future works
...
Java Parser
• Developed using Eclipse AST Tree API
• The AST tree is semantically analyzed using
the information in each o...
Modifiers
Name
Parameters
Body
Structural properties of a method declaration
29
Exceptions
Return Type
Example
30
Public Method Declaration
AIEDCS 2013
Example
31
Return Type
Void
AIEDCS 2013
Example
32
FormalMethodParameter
Single Variable Declaration
AIEDCS 2013
Example
33
Exception
AIEDCS 2013
Example
34
Super Method Invocation
AIEDCS 2013
• Current version of JavaParser is able to extract
98.77% of the concepts in 103 manually
indexed questions.
• Average num...
Missed concept
• InheritanceBasedPolymorphism
• SuperclassSubclassConversion
• PolymorphicObjectCreationStatement
• Method...
Demo
http://adapt2.sis.pitt.edu/javaparser/ParseQuestion.jsp
AIEDCS 2013 37
Outline
• Introduction
• Hypothesis testing (pre-study)
• Findings and challenges
• JavaParser
• Conclusion &Future works
...
Future work
Use the results of fine-indexing for:
• Improving the user modeling service
• cross content sequencing and pro...
Thank you!
Contact: roh38@pitt.edu
45AIEDCS 2013
Upcoming SlideShare
Loading in …5
×

Java parser a fine grained indexing tool and its application

329 views

Published on

AIED CS 2013

Published in: Data & Analytics

Java parser a fine grained indexing tool and its application

  1. 1. University of Pittsburgh Intelligent Systems Program
  2. 2. Outline • Introduction • Hypothesis testing (pre-study) • Findings and challenges • JavaParser • Conclusion &Future works 2AIEDCS 2013
  3. 3. Introduction There are two possible ways for modeling student’s knowledge: • Coarse-grained knowledge modeling • Fine-grained knowledge modeling 3AIEDCS 2013
  4. 4. Motivation Fine-grained indexing? Do we really need it ? ... 4AIEDCS 2013
  5. 5. • An important aspect of task sequencing in Adaptive Hypermedia is the granularity of the domain model and the task indexing. • In general, the sequencing algorithm can better determine the appropriate task if the granularity of the domain model and the task indexing is finer. • However, fine-grained domain models that dissect a domain into dozens or hundreds of knowledge units are much harder to develop and to use for indexing. 5AIEDCS 2013
  6. 6. • Typical approach to present programming knowledge which uses coarse-grained topics like “loops” and “increment” allows reasonable sequencing during course. (Brusilovsky et al. 2009, Hsiao et al. 2010, Kavcic 2004, Vesin et al. 2012) • However, this approach fails in providing advance sequencing such as providing support for exam preparation or remediation. 6AIEDCS 2013
  7. 7. Outline • Introduction • Hypothesis testing (pre-study) • Findings and challenges • JavaParser • Conclusion &Future works 7AIEDCS 2013
  8. 8. Pre-study Knowledge Maximizer: • An exam preparation tool for Java programming • based on a fine-grained concept model of Java knowledge • assumes a student already completed a considerable amount of work • goal is to help her define gaps in knowledge and try to redress them as soon as possible. 8AIEDCS 2013
  9. 9. 11AIEDCS 2013
  10. 10. KM Parameters: • How prepared is the student to do the activity? 12      r r M i i M i ii w wk K K: user knowledge level Mr :prerequisite concepts ki :knowledge in Ci wi : weight of Ci for activity AIEDCS 2013
  11. 11. • How prepared is the student to do the activity? • What is the impact of the activity? 13      o o M i i M i ii w wk I )1( I : activity impact Mo : outcome concepts AIEDCS 2013
  12. 12. • How prepared is the student to do the activity? • What is the impact of the activity? • What is the value of repeating the activity again? 14 1 1   t s S : inverse success rate for the activityS S : number of success in the activity t : number of times the activity is done AIEDCS 2013
  13. 13. • Determining the sequence of top 10 activity with the highest rank using: 15 SIKR  R : activity rank K : knowledge level in prerequisites of activity I : activity Impact : Inverse of success rateS AIEDCS 2013
  14. 14. Evaluation • We conducted a classroom study for the Java Programming undergraduate course. • Study started on Dec. 4th 2012 about a week before the final exam. • The course also used QuizGuide (QG), and Progressor+ (P+) systems to access Java questions (available from the beginning of the semester). • All these systems used same 103 parameterized Java questions. 16AIEDCS 2013
  15. 15. Evaluation Measures We grouped participants into two groups: • KM: those who made at least ten attempt using KM (n = 9) • QG/P+: those who made no attempt using KM and at least 10 attempt with QG/P+ (n = 16) For each group we measured: • Number of questions (attempts) done using each system • Success Rate 17AIEDCS 2013
  16. 16. Outline • Introduction • Hypothesis testing (pre-study) • Findings and challenges • JavaParser • Conclusion &Future works 18AIEDCS 2013
  17. 17. System Usage Summary 19 6.20% 43.50% 50.20% 34.60% 45.30% 20.10% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00% Easy Moderate Complex Attempts per Question Compelixty KM QG,P+ AIEDCS 2013
  18. 18. Results 20 58% 64% 0% 50% 100% KM QG,P+ Success rate AIEDCS 2013 -12.80% -13.70% 11.53% -15.00% -10.00% -5.00% 0.00% 5.00% 10.00% 15.00% Class QG/P+ KM Average Relative Progress Percentage
  19. 19. • Fine-grained knowledge modeling will push students toward appropriate complex questions which does not lead to miserable fails in those questions . • During exam preparation, complex questions are more useful, since they target more concepts at once. • This helps students fill the gaps in their knowledge in a more efficient way. (Eg. : 6 easy question must be done to get the same outcome as only 1 complex question!) 22AIEDCS 2013
  20. 20. JAVA Ontology http://www.sis.pitt.edu/~paws/ont/java.owl 23AIEDCS 2013 ~ 400 node in ontology ~ 160 concept (leaf nodes)
  21. 21. 24AIEDCS 2013  103 JAVA Parameterized Question  1-5 classes per question  5-52 concepts per question ~ 40% of questions has more than 20 concepts
  22. 22. 25AIEDCS 2013 52 concepts or more in a question ?
  23. 23. 26AIEDCS 2013 JavaParser: A tool for automatic indexing of Java Problems
  24. 24. Outline • Introduction • Hypothesis testing (pre-study) • Findings and challenges • JavaParser • Conclusion &Future works 27AIEDCS 2013
  25. 25. Java Parser • Developed using Eclipse AST Tree API • The AST tree is semantically analyzed using the information in each of its nodes. 28 1 2 3 AIEDCS 2013
  26. 26. Modifiers Name Parameters Body Structural properties of a method declaration 29 Exceptions Return Type
  27. 27. Example 30 Public Method Declaration AIEDCS 2013
  28. 28. Example 31 Return Type Void AIEDCS 2013
  29. 29. Example 32 FormalMethodParameter Single Variable Declaration AIEDCS 2013
  30. 30. Example 33 Exception AIEDCS 2013
  31. 31. Example 34 Super Method Invocation AIEDCS 2013
  32. 32. • Current version of JavaParser is able to extract 98.77% of the concepts in 103 manually indexed questions. • Average number of 8 extra concepts for each question indexed by automatic parsing. 35AIEDCS 2013
  33. 33. Missed concept • InheritanceBasedPolymorphism • SuperclassSubclassConversion • PolymorphicObjectCreationStatement • MethodInheritance • …. AIEDCS 2013 36
  34. 34. Demo http://adapt2.sis.pitt.edu/javaparser/ParseQuestion.jsp AIEDCS 2013 37
  35. 35. Outline • Introduction • Hypothesis testing (pre-study) • Findings and challenges • JavaParser • Conclusion &Future works 43AIEDCS 2013
  36. 36. Future work Use the results of fine-indexing for: • Improving the user modeling service • cross content sequencing and providing remediation in case of failing a question. • Predicting parts of code that might lead to student failure and provide hints accordingly. • Expand the parser to extract more elaborated concepts and programming patterns . 44AIEDCS 2013
  37. 37. Thank you! Contact: roh38@pitt.edu 45AIEDCS 2013

×