Learning with Games - MRU SoTL Nov. 2010 1
Do Game Assignments HelpDo Game Assignments Help
Programming Students Learn Mor...
Learning with Games - MRU SoTL Nov. 2010 2
What I’m Playing Now
Learning with Games - MRU SoTL Nov. 2010 3
What Are We Trying to Do?
Constructivist and
Behaviourist (Instructionist)
appr...
Learning with Games - MRU SoTL Nov. 2010 4
What Are We Trying to Do?
Can we suggest a method or
protocol for experimenting...
Learning with Games - MRU SoTL Nov. 2010 5
Can We Determine If One Is
Working Better?
Use each method on different but
equ...
Learning with Games - MRU SoTL Nov. 2010 6
An Opportunistic Experiment
• Different sections of the same course -
(Computer...
Learning with Games - MRU SoTL Nov. 2010 7
Experimental Protocol
Four sections of the class were offered in the
Winter ses...
Learning with Games - MRU SoTL Nov. 2010 8
Experimental Protocol
Students submit their assignment solutions
using an on-li...
Learning with Games - MRU SoTL Nov. 2010 9
Tomb Raider
Assuming that the assignment grades in
both classes are equivalent,...
Learning with Games - MRU SoTL Nov. 2010 10
Behaviourist vs Contructivist
•Specified in great
detail (methods,
parameters,...
Learning with Games - MRU SoTL Nov. 2010 11
Constructivist Assignments*
1. Be accessible to everyone at the start.
2. Invi...
Learning with Games - MRU SoTL Nov. 2010 12
Behaviourist Assignments
1. Student Grades (% to letter)
2. A Point class
3. C...
Learning with Games - MRU SoTL Nov. 2010 13
Constructivist Assignments
1.Transition - convert small program to Java
2. Fir...
Learning with Games - MRU SoTL Nov. 2010 14
Measuring the Programs
They are some of the
oldest measures of
program complex...
Learning with Games - MRU SoTL Nov. 2010 15
Measuring the Programs
Elder Scrolls IV: Oblivion
These metrics are based on t...
Learning with Games - MRU SoTL Nov. 2010 16
Measuring the Programs
Program Length N
A measure of program length in terms o...
Learning with Games - MRU SoTL Nov. 2010 17
Measuring the Programs
Vocabulary n
The number distinct symbols used by the
pr...
Learning with Games - MRU SoTL Nov. 2010 18
Measuring the Programs
Lines of Code (LOC)
The number distinct symbols used by...
Learning with Games - MRU SoTL Nov. 2010 19
Measuring the Programs
Effort:
the relationship between an estimate of
the vol...
Learning with Games - MRU SoTL Nov. 2010 20
Measuring the Programs
Time to code: This is an
estimate of how long it would
...
Learning with Games - MRU SoTL Nov. 2010 21
Averaged over all assignments
Constructivist Behaviourist
# of tokens N: 481.1...
Learning with Games - MRU SoTL Nov. 2010 22
Games Vs Other
Assignments
Median Effort
Assignment Constructivist Behaviouris...
Learning with Games - MRU SoTL Nov. 2010 23
Games Vs Other Assignments
Median Time to Code
Assignment Constructivist Behav...
Learning with Games - MRU SoTL Nov. 2010 24
Measuring the Programs
Cyclomatic Complexity:
A measure of how many paths ther...
Learning with Games - MRU SoTL Nov. 2010 25
Measuring the Programs
Cyclomatic Complexity
The results of multiple experimen...
Learning with Games - MRU SoTL Nov. 2010 26
Cyclomatic Complexity
(Computed by JavaNCSS)
Assignment Constructivist Behavio...
Learning with Games - MRU SoTL Nov. 2010 27
Successor Classes
The students in the constructivist group
achieved an average...
Learning with Games - MRU SoTL Nov. 2010 28
Successor Classes
This improvement deteriorates with
time.
Taking the successo...
Learning with Games - MRU SoTL Nov. 2010 29
Conclusions
The constructivist assignment set
appears to require much more
eff...
Learning with Games - MRU SoTL Nov. 2010 30
Next Steps
• Repeated
• Find analogous
measures in other
work.
• Longer follow...
Learning with Games - MRU SoTL Nov. 2010 31
Thanks!!
Title Slide: Sonny Chan
Game Images courtesy of:
Official Game Sites ...
Upcoming SlideShare
Loading in …5
×

Game Assignments in computer Science

1,321 views

Published on

Using Standard Software Metrics to Analyze Student Effort on Games

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,321
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Game Assignments in computer Science

  1. 1. Learning with Games - MRU SoTL Nov. 2010 1 Do Game Assignments HelpDo Game Assignments Help Programming Students Learn More?Programming Students Learn More? Katrin Becker J.R.Parker Nov 2010 This session focuses on the measurable effects of using games as assignments in a programming class.
  2. 2. Learning with Games - MRU SoTL Nov. 2010 2 What I’m Playing Now
  3. 3. Learning with Games - MRU SoTL Nov. 2010 3 What Are We Trying to Do? Constructivist and Behaviourist (Instructionist) approaches are both useful in some circumstances. Can we see an advantage to either one in the first CS courses?
  4. 4. Learning with Games - MRU SoTL Nov. 2010 4 What Are We Trying to Do? Can we suggest a method or protocol for experimenting with this kind of question? Eve online
  5. 5. Learning with Games - MRU SoTL Nov. 2010 5 Can We Determine If One Is Working Better? Use each method on different but equivalent CS102 classes. Record: o Amount of work done o Progress in successor courses Blood Magic
  6. 6. Learning with Games - MRU SoTL Nov. 2010 6 An Opportunistic Experiment • Different sections of the same course - (Computer Science 233) were allowed to proceed individually, with no formal coordination of assignments or exams. • This was not the norm, but it did allow us to conduct a simple comparison of the methods used in the two courses. • Specifically, the assignments given to the students could be assessed and compared.Heavy Rain
  7. 7. Learning with Games - MRU SoTL Nov. 2010 7 Experimental Protocol Four sections of the class were offered in the Winter session. One instructor chose to assign a behaviorist assignment set, the other a constructivist set, both using the Java language. The courses were taught to the same course description and explicit syllabus, and should have included the same basic content. Limbo
  8. 8. Learning with Games - MRU SoTL Nov. 2010 8 Experimental Protocol Students submit their assignment solutions using an on-line system, allowing us to collect all of the solutions as they were completed. We used standard software engineering complexity metrics to determine how complex the solutions were. Student grades in successor course were collected and averaged for comparison. Limbo
  9. 9. Learning with Games - MRU SoTL Nov. 2010 9 Tomb Raider Assuming that the assignment grades in both classes are equivalent, then the assignments that involved the largest amount of work or effort should be an indicator of how much was learned. Using Observed Effort
  10. 10. Learning with Games - MRU SoTL Nov. 2010 10 Behaviourist vs Contructivist •Specified in great detail (methods, parameters, even names) •Little room for variation •Designed to mimic a specific tiny application that is fully understood. •Stated Objective; details left to student •Considerable room for variation •Designed to reside at the edge of the student’s ZPD
  11. 11. Learning with Games - MRU SoTL Nov. 2010 11 Constructivist Assignments* 1. Be accessible to everyone at the start. 2. Invite students to make decisions. 3. Encourage "what if" questions. 4. Encourage students to use their own methods. 5. Promote discussion and communication. 6. Be replete with patterns. 7. Lead somewhere. 8. Have an element of surprise. 9. Be enjoyable. 10. Be extendable. (*G. H. Wheatley) School Tychoon
  12. 12. Learning with Games - MRU SoTL Nov. 2010 12 Behaviourist Assignments 1. Student Grades (% to letter) 2. A Point class 3. Class extension (shape, circle) 4. Mortgage Calculator (Swing GUI) 5. Greenhouse simulation (Sensors control devices) Secret of Monkey Island
  13. 13. Learning with Games - MRU SoTL Nov. 2010 13 Constructivist Assignments 1.Transition - convert small program to Java 2. First Class - implement a class that serves as an enhanced version of a data type that was used in the previous program. 3. Encapsulation; Simple Data Structures - Write an ACSII-graphics version of the Four Seasons Solitaire game. 4. Parsing -Design and write a recursive parser for expressions. 5. Inheritance -ASCII-graphics version of the Centipede arcade game. Mass Effect
  14. 14. Learning with Games - MRU SoTL Nov. 2010 14 Measuring the Programs They are some of the oldest measures of program complexity (1977). We chose the commonly encountered Halstead metrics because… they are commonly encountered. Grim Fandango
  15. 15. Learning with Games - MRU SoTL Nov. 2010 15 Measuring the Programs Elder Scrolls IV: Oblivion These metrics are based on the simple measurements: n1 = the number of distinct operators n2 = the number of distinct operands N1 = the total number of operators N2 = the total number of operands
  16. 16. Learning with Games - MRU SoTL Nov. 2010 16 Measuring the Programs Program Length N A measure of program length in terms of the number of tokens used by the program. It is calculated as N = N1 + N2 (total # operators & operands)
  17. 17. Learning with Games - MRU SoTL Nov. 2010 17 Measuring the Programs Vocabulary n The number distinct symbols used by the program. It is calculated as n = n1 + n2 (# distinct operators & operands)
  18. 18. Learning with Games - MRU SoTL Nov. 2010 18 Measuring the Programs Lines of Code (LOC) The number distinct symbols used by the program. It is calculated as LOC = N / 3.14
  19. 19. Learning with Games - MRU SoTL Nov. 2010 19 Measuring the Programs Effort: the relationship between an estimate of the volume of information required to specify a software program (V) and the program level, a measure of the relation between the volumes of the most compact representation and the actual program. E = V/PL PL = 1 / (( n1 / 2 ) * ( N2 / n2 )) V= N * (LOG2 n)
  20. 20. Learning with Games - MRU SoTL Nov. 2010 20 Measuring the Programs Time to code: This is an estimate of how long it would generally take to write the program. T = E/K For the Java language the constant 0.9 was used.
  21. 21. Learning with Games - MRU SoTL Nov. 2010 21 Averaged over all assignments Constructivist Behaviourist # of tokens N: 481.1 368.6 Vocabulary: 533.8 315.4 Length: 4846.4 2556.2 (total=29,078.5) (total=12,780.8) LOC: 153.2 117.4 (total=919.3) (total=586.9)
  22. 22. Learning with Games - MRU SoTL Nov. 2010 22 Games Vs Other Assignments Median Effort Assignment Constructivist Behaviourist 1 9178.415946 2 18316 10102 3 29045 7669.9 4 22018 11427 5 40438 22972   Unbiased mean 24,91013,623   Total Effort 149,461 68,117
  23. 23. Learning with Games - MRU SoTL Nov. 2010 23 Games Vs Other Assignments Median Time to Code Assignment Constructivist Behaviourist 1 2.8328 4.9215 2 5.6531 3.1178 3 8.9647 2.3672 4 6.7956 3.527 5 12.481 7.0902   Unbiased mean     7.69 4.20  Total Time 46.13 21.02
  24. 24. Learning with Games - MRU SoTL Nov. 2010 24 Measuring the Programs Cyclomatic Complexity: A measure of how many paths there are through a program. A graph can be created in which each node corresponds to a block of code, and each arc to a branch. The cyclomatic number is the maximum number of linearly independent circuits.
  25. 25. Learning with Games - MRU SoTL Nov. 2010 25 Measuring the Programs Cyclomatic Complexity The results of multiple experiments (G.A. Miller) suggest that modules approach zero defects when McCabe's Cyclomatic Complexity is within 7+/- 2. Cyclomatic Complexity between 10 and 15 minimized the number of module changes.
  26. 26. Learning with Games - MRU SoTL Nov. 2010 26 Cyclomatic Complexity (Computed by JavaNCSS) Assignment Constructivist Behaviourist 1 8 28 2 3.87  1.69 3 4.18    1.69 4 3.05    1.13 5 2.15    1.7
  27. 27. Learning with Games - MRU SoTL Nov. 2010 27 Successor Classes The students in the constructivist group achieved an average grade 0.7 higher than did the behaviourist group in the successor programming class; Grades are computed on a 4-point scale, so this would represent an 18% improvement Ace Combat 5
  28. 28. Learning with Games - MRU SoTL Nov. 2010 28 Successor Classes This improvement deteriorates with time. Taking the successor 4 months later reduces the improvement to 0.1 from 0.7, or 2.5%. Improvement is somewhat higher for the 'A' students. Neo Steam The Shattered Continent
  29. 29. Learning with Games - MRU SoTL Nov. 2010 29 Conclusions The constructivist assignment set appears to require much more effort than the behaviourist set. We have results of other measures, including cyclomatic complexity, methods per class, and object oriented metrics, that confirm this. Constructivist taught students do slightly better in the next course. Super Mario Galaxy 2
  30. 30. Learning with Games - MRU SoTL Nov. 2010 30 Next Steps • Repeated • Find analogous measures in other work. • Longer follow- up.*** Super Mario Galaxy 2
  31. 31. Learning with Games - MRU SoTL Nov. 2010 31 Thanks!! Title Slide: Sonny Chan Game Images courtesy of: Official Game Sites (images are identified by game) Fan Art Mobygames.com Gamespot.com IGN.com Gamewallpapers.com

×