Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Exploring Problem Solving Paths
in a Java Programming Course
Roya Hosseini1
Arto Vihavainen2
Peter Brusilovsky1,3
1 Intell...
Goal of this work
• Identifying most common programming paths
– Tracking and analyzing subsequent programming
steps within...
Related Work
• Factors contributing to success in programming:
– aptitude: psychological tests, ... (Evans and Simkin
1989...
Related Work
• Data-driven approaches for analyzing
submission and snapshot streams:
– compilation behaviour
– time usage
...
Related Work
• Novice problem-solving behaviour
(Perkins et al. 1986, ..)
stoppers movers tinkerers
pic pic pic
This study
● Introductory programming course, fall 2012
● Programming in NetBeans:
– Unit tests designed to scaffold stude...
Example: Subsequent snapshots
import java.util.Scanner;
public class BiggerNumber {
public static void main(String[] args)...
Example: Subsequent snapshots
import java.util.Scanner;
public class BiggerNumber {
public static void main(String[] args)...
Example: Concepts from snapshots
ActualMethodParameter, MethodDefinition,
ObjectCreationStatement,
ObjectMethodInvocation,...
Dataset
● 101 participants (65 male, 36 female)
● 101 exercises
● 60+k snapshots
● Preprocessing excludes:
● users that ha...
Approach
● (1) Extracting concepts from snapshots
● (2) Analyzing conceptual changes between
sequential snapshots
Assumpti...
Global Level
● When, for each exercise, snapshots from all
students are put together, what do we see?
Assumption
Increment...
Global Level: Easy Exercise
Global Level: Medium Exercise
Global Level: Hard Exercise
Global Level: Programming Speed
Individual Level
● Looking at each exercise for each student
individually, what sort of student-specific
patterns do we se...
Identifying Programming Patterns: 1
snapshots Classify Label
Concept
Correctness
Concepts
Correctness Same Increase Decrea...
Identifying Programming Patterns: 2
Pattern
Mining
(SPAM)
Snapshots
Student s1: Exercise e1
…
Student sn: Exercise en
Comm...
Individual Level: Builder
Individual Level: Massager
Individual Level: Reducer
Individual Level: Struggler
Common Programming Patterns
Pattern Support Pattern Support
BB 0.304 SSS 0.128
SB 0.281 BR 0.127
BS 0.215 BBB 0.123
SS 0.1...
Common Student Patterns
Overall Result
• Incremental program building hypothesis not
always true!
• Still, it is dominant behaviour among plenty o...
Discussion
• Building and reduction steps happen in:
– all stages of program development
– across levels of problem diffic...
Future Work
• Do the introduced programming patterns
generalize in other datasets?
• What about different data granulariti...
Future Work
• Automatic Student Modeling?
Yudelson et al.;
“Investigating Automatic Student Modeling in a Java, Educationa...
Thank you!
Q&A
Upcoming SlideShare
Loading in …5
×

Ppig2014 problem solvingpaths

266 views

Published on

PPIG 2014

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Ppig2014 problem solvingpaths

  1. 1. Exploring Problem Solving Paths in a Java Programming Course Roya Hosseini1 Arto Vihavainen2 Peter Brusilovsky1,3 1 Intelligent Systems Program, University of Pittsburgh 2 Department of Computer Science, University of Helsinki 3 School of Information Sciences, University of Pittsburgh Intelligent Systems Program University of Helsinki Psychology of Programming Interest Group 25th Workshop 25-27 June 2014
  2. 2. Goal of this work • Identifying most common programming paths – Tracking and analyzing subsequent programming steps within programming assignments – Looking at the concepts that students use
  3. 3. Related Work • Factors contributing to success in programming: – aptitude: psychological tests, ... (Evans and Simkin 1989, ..) – background: mathematics, ... (White and Sivitanides 2003, ..) – psychological aspects: motivation, comfort- level, ... (Bergin and Reilly 2005, ..) – learning styles, ... (Thomas et al. 2002, ..)
  4. 4. Related Work • Data-driven approaches for analyzing submission and snapshot streams: – compilation behaviour – time usage Jadud (2005) and Vee (2006), ..
  5. 5. Related Work • Novice problem-solving behaviour (Perkins et al. 1986, ..) stoppers movers tinkerers pic pic pic
  6. 6. This study ● Introductory programming course, fall 2012 ● Programming in NetBeans: – Unit tests designed to scaffold students – Test My Code plugin Vihavainen et al. 2013 ● Recording snapshots: – correctness [0-1] – difficulty [1-5] (post assignment) – set of programming concepts Hosseini & Brusilovsky 2013
  7. 7. Example: Subsequent snapshots import java.util.Scanner; public class BiggerNumber { public static void main(String[] args) { Scanner input = new Scanner(System.in); System.out.println("Type a number: "); int first = Integer.parseInt(input.nextLine()); System.out.println("Type another number: "); int second = Integer.parseInt(input.nextLine()); if (first > second) System.out.println("nBigger number: " + first); if (first < second) System.out.println("nBigger number: " + second); else System.out.println(“nNumbers were equal: "); } } import java.util.Scanner; public class BiggerNumber { public static void main(String[] args) { Scanner input = new Scanner(System.in); System.out.println("Type a number: "); int first = Integer.parseInt(input.nextLine()); System.out.println("Type another number: "); int second = Integer.parseInt(input.nextLine()); if (first > second) System.out.println("nBigger number: " + first); else if (first < second) System.out.println("nBigger number: " + second); else System.out.println(“nNumbers were equal: "); } } Snapshot A Snapshot B
  8. 8. Example: Subsequent snapshots import java.util.Scanner; public class BiggerNumber { public static void main(String[] args) { Scanner input = new Scanner(System.in); System.out.println("Type a number: "); int first = Integer.parseInt(input.nextLine()); System.out.println("Type another number: "); int second = Integer.parseInt(input.nextLine()); if (first > second) System.out.println("nBigger number: " + first); if (first < second) System.out.println("nBigger number: " + second); else System.out.println(“nNumbers were equal: "); } } import java.util.Scanner; public class BiggerNumber { public static void main(String[] args) { Scanner input = new Scanner(System.in); System.out.println("Type a number: "); int first = Integer.parseInt(input.nextLine()); System.out.println("Type another number: "); int second = Integer.parseInt(input.nextLine()); if (first > second) System.out.println("nBigger number: " + first); else if (first < second) System.out.println("nBigger number: " + second); else System.out.println(“nNumbers were equal: "); } } Snapshot A Snapshot B
  9. 9. Example: Concepts from snapshots ActualMethodParameter, MethodDefinition, ObjectCreationStatement, ObjectMethodInvocation, PublicClassSpecifier, PublicMethodSpecifier, StaticMethodSpecifier, StringAddition, StringDataType, StringLiteral, LessExpression, java.lang.System.out.println, java.lang.System.out.print, ClassDefinition, ConstructorCall, FormalMethodParameter, GreaterExpression, IfStatement, IfElseStatement, ImportStatement, IntDataType, java.lang.Integer.parseInt, VoidDataType ActualMethodParameter, MethodDefinition, ObjectCreationStatement, ObjectMethodInvocation, PublicClassSpecifier, PublicMethodSpecifier, StaticMethodSpecifier, StringAddition, StringDataType, StringLiteral, LessExpression, java.lang.System.out.println, java.lang.System.out.print, ClassDefinition, ConstructorCall, FormalMethodParameter, GreaterExpression, IfElseIfStatement, IfElseStatement, ImportStatement, IntDataType, java.lang.Integer.parseInt, VoidDataType Concepts in Snapshot A Concepts in Snapshot B
  10. 10. Dataset ● 101 participants (65 male, 36 female) ● 101 exercises ● 60+k snapshots ● Preprocessing excludes: ● users that have not given consent ● users not participating in first week ● subsequent identical snapshots
  11. 11. Approach ● (1) Extracting concepts from snapshots ● (2) Analyzing conceptual changes between sequential snapshots Assumption Incremental building of the program Programming Behaviour Global Level Programming Behaviour Individual Level
  12. 12. Global Level ● When, for each exercise, snapshots from all students are put together, what do we see? Assumption Incremental building of the program Programming Behaviour Global Level
  13. 13. Global Level: Easy Exercise
  14. 14. Global Level: Medium Exercise
  15. 15. Global Level: Hard Exercise
  16. 16. Global Level: Programming Speed
  17. 17. Individual Level ● Looking at each exercise for each student individually, what sort of student-specific patterns do we see? Assumption Incremental building of the program Programming Behaviour Individual Level
  18. 18. Identifying Programming Patterns: 1 snapshots Classify Label Concept Correctness Concepts Correctness Same Increase Decrease Zero Struggler Struggler Struggler Decrease Struggler Struggler Struggler Increase Builder Builder Reducer Same Massager Builder Reducer
  19. 19. Identifying Programming Patterns: 2 Pattern Mining (SPAM) Snapshots Student s1: Exercise e1 … Student sn: Exercise en Common Patterns ~8k Student-Exercise
  20. 20. Individual Level: Builder
  21. 21. Individual Level: Massager
  22. 22. Individual Level: Reducer
  23. 23. Individual Level: Struggler
  24. 24. Common Programming Patterns Pattern Support Pattern Support BB 0.304 SSS 0.128 SB 0.281 BR 0.127 BS 0.215 BBB 0.123 SS 0.199 SSB 0.123
  25. 25. Common Student Patterns
  26. 26. Overall Result • Incremental program building hypothesis not always true! • Still, it is dominant behaviour among plenty of students (77.63%)
  27. 27. Discussion • Building and reduction steps happen in: – all stages of program development – across levels of problem difficulty – visible in both slow and fast students • Most students build programs incrementally: – starting with a small program – incrementally adding concepts – increasing its correctness (passing more tests)
  28. 28. Future Work • Do the introduced programming patterns generalize in other datasets? • What about different data granularities? • What is the influence of individual traits on programming behaviours (age, gender,…)?
  29. 29. Future Work • Automatic Student Modeling? Yudelson et al.; “Investigating Automatic Student Modeling in a Java, Educational Data Mining, 2014
  30. 30. Thank you! Q&A

×