This document discusses a study that compared the effects of using constructivist versus behaviorist assignments in a programming class. The study found that the constructivist assignments, which gave students more freedom and flexibility, required significantly more effort from students compared to the behaviorist assignments. Various metrics were used to measure the complexity of programs developed in each condition. Students who completed the constructivist assignments performed slightly better in subsequent programming courses on average. The findings provide preliminary evidence that constructivist assignments may help students learn more, but larger and longer-term studies are needed.

1. Learning with Games - MRU SoTL Nov. 2010 1
Do Game Assignments HelpDo Game Assignments Help
Programming Students Learn More?Programming Students Learn More?
Katrin Becker
J.R.Parker
Nov 2010
This session focuses on the measurable effects of using
games as assignments in a programming class.

2. Learning with Games - MRU SoTL Nov. 2010 2
What I’m Playing Now

3. Learning with Games - MRU SoTL Nov. 2010 3
What Are We Trying to Do?
Constructivist and
Behaviourist (Instructionist)
approaches are both useful in
some circumstances. Can we
see an advantage to either
one in the first CS courses?

4. Learning with Games - MRU SoTL Nov. 2010 4
What Are We Trying to Do?
Can we suggest a method or
protocol for experimenting
with this kind of question?
Eve online

5. Learning with Games - MRU SoTL Nov. 2010 5
Can We Determine If One Is
Working Better?
Use each method on different but
equivalent CS102 classes.
Record:
o Amount of work done
o Progress in successor courses Blood Magic

6. Learning with Games - MRU SoTL Nov. 2010 6
An Opportunistic Experiment
• Different sections of the same course -
(Computer Science 233) were allowed to
proceed individually, with no formal
coordination of assignments or exams.
• This was not the norm, but it did allow us to
conduct a simple comparison of the methods
used in the two courses.
• Specifically, the assignments given to the
students could be assessed and compared.Heavy Rain

7. Learning with Games - MRU SoTL Nov. 2010 7
Experimental Protocol
Four sections of the class were offered in the
Winter session.
One instructor chose to assign a behaviorist
assignment set, the other a constructivist
set, both using the Java language.
The courses were taught to the same course
description and explicit syllabus, and should
have included the same basic content.
Limbo

8. Learning with Games - MRU SoTL Nov. 2010 8
Experimental Protocol
Students submit their assignment solutions
using an on-line system, allowing us to
collect all of the solutions as they were
completed.
We used standard software engineering
complexity metrics to determine how
complex the solutions were.
Student grades in successor course were
collected and averaged for comparison.
Limbo

9. Learning with Games - MRU SoTL Nov. 2010 9
Tomb Raider
Assuming that the assignment grades in
both classes are equivalent, then the
assignments that involved the largest
amount of work or effort should be an
indicator of how much was learned.
Using Observed Effort

10. Learning with Games - MRU SoTL Nov. 2010 10
Behaviourist vs Contructivist
•Specified in great
detail (methods,
parameters, even
names)
•Little room for
variation
•Designed to mimic a
specific tiny
application that is
fully understood.
•Stated Objective;
details left to
student
•Considerable room
for variation
•Designed to reside at
the edge of the
student’s ZPD

11. Learning with Games - MRU SoTL Nov. 2010 11
Constructivist Assignments*
1. Be accessible to everyone at the start.
2. Invite students to make decisions.
3. Encourage "what if" questions.
4. Encourage students to use their own methods.
5. Promote discussion and communication.
6. Be replete with patterns.
7. Lead somewhere.
8. Have an element of surprise.
9. Be enjoyable.
10. Be extendable. (*G. H. Wheatley)
School Tychoon

12. Learning with Games - MRU SoTL Nov. 2010 12
Behaviourist Assignments
1. Student Grades (% to letter)
2. A Point class
3. Class extension (shape, circle)
4. Mortgage Calculator (Swing GUI)
5. Greenhouse simulation (Sensors
control devices)
Secret of Monkey Island

13. Learning with Games - MRU SoTL Nov. 2010 13
Constructivist Assignments
1.Transition - convert small program to Java
2. First Class - implement a class that serves as an
enhanced version of a data type that was used in
the previous program.
3. Encapsulation; Simple Data Structures - Write an
ACSII-graphics version of the Four Seasons Solitaire
game.
4. Parsing -Design and write a recursive parser for
expressions.
5. Inheritance -ASCII-graphics version of the Centipede
arcade game.
Mass Effect

14. Learning with Games - MRU SoTL Nov. 2010 14
Measuring the Programs
They are some of the
oldest measures of
program complexity
(1977).
We chose the commonly
encountered Halstead
metrics because…
they are commonly
encountered.
Grim Fandango

15. Learning with Games - MRU SoTL Nov. 2010 15
Measuring the Programs
Elder Scrolls IV: Oblivion
These metrics are based on the simple
measurements:
n1 = the number of distinct operators
n2 = the number of distinct operands
N1 = the total number of operators
N2 = the total number of operands

16. Learning with Games - MRU SoTL Nov. 2010 16
Measuring the Programs
Program Length N
A measure of program length in terms of the
number of tokens used by the program.
It is calculated as
N = N1 + N2
(total # operators & operands)

17. Learning with Games - MRU SoTL Nov. 2010 17
Measuring the Programs
Vocabulary n
The number distinct symbols used by the
program.
It is calculated as
n = n1 + n2
(# distinct operators & operands)

18. Learning with Games - MRU SoTL Nov. 2010 18
Measuring the Programs
Lines of Code (LOC)
The number distinct symbols used by the
program.
It is calculated as
LOC = N / 3.14

19. Learning with Games - MRU SoTL Nov. 2010 19
Measuring the Programs
Effort:
the relationship between an estimate of
the volume of information required to
specify a software program (V) and the
program level, a measure of the relation
between the volumes of the most compact
representation and the actual program.
E = V/PL
PL = 1 / (( n1 / 2 ) * ( N2 / n2 ))
V= N * (LOG2 n)

20. Learning with Games - MRU SoTL Nov. 2010 20
Measuring the Programs
Time to code: This is an
estimate of how long it would
generally take to write the
program.
T = E/K
For the Java language the
constant 0.9 was used.

21. Learning with Games - MRU SoTL Nov. 2010 21
Averaged over all assignments
Constructivist Behaviourist
# of tokens N: 481.1 368.6
Vocabulary: 533.8 315.4
Length: 4846.4 2556.2
(total=29,078.5)
(total=12,780.8)
LOC: 153.2 117.4
(total=919.3) (total=586.9)

22. Learning with Games - MRU SoTL Nov. 2010 22
Games Vs Other
Assignments
Median Effort
Assignment Constructivist Behaviourist
1 9178.415946
2 18316 10102
3 29045 7669.9
4 22018 11427
5 40438 22972
Unbiased mean 24,91013,623
Total Effort 149,461 68,117

23. Learning with Games - MRU SoTL Nov. 2010 23
Games Vs Other Assignments
Median Time to Code
Assignment Constructivist Behaviourist
1 2.8328 4.9215
2 5.6531 3.1178
3 8.9647 2.3672
4 6.7956 3.527
5 12.481 7.0902
Unbiased mean 7.69 4.20
Total Time 46.13 21.02

24. Learning with Games - MRU SoTL Nov. 2010 24
Measuring the Programs
Cyclomatic Complexity:
A measure of how many paths there are
through a program.
A graph can be created in which each
node corresponds to a block of code, and
each arc to a branch. The cyclomatic
number is the maximum number of
linearly independent circuits.

25. Learning with Games - MRU SoTL Nov. 2010 25
Measuring the Programs
Cyclomatic Complexity
The results of multiple experiments
(G.A. Miller) suggest that modules
approach zero defects when McCabe's
Cyclomatic Complexity is within 7+/- 2.
Cyclomatic Complexity between 10 and
15 minimized the number of module
changes.

26. Learning with Games - MRU SoTL Nov. 2010 26
Cyclomatic Complexity
(Computed by JavaNCSS)
Assignment Constructivist Behaviourist
1 8 28
2 3.87 1.69
3 4.18 1.69
4 3.05 1.13
5 2.15 1.7

27. Learning with Games - MRU SoTL Nov. 2010 27
Successor Classes
The students in the constructivist group
achieved an average grade 0.7 higher
than did the behaviourist group in the
successor programming class;
Grades are computed on a 4-point scale,
so this would represent an 18%
improvement
Ace Combat 5

28. Learning with Games - MRU SoTL Nov. 2010 28
Successor Classes
This improvement deteriorates with
time.
Taking the successor 4 months later
reduces the improvement to 0.1 from
0.7, or 2.5%.
Improvement is somewhat higher for the
'A' students.
Neo Steam The Shattered Continent

29. Learning with Games - MRU SoTL Nov. 2010 29
Conclusions
The constructivist assignment set
appears to require much more
effort than the behaviourist set.
We have results of other measures,
including cyclomatic complexity,
methods per class, and object
oriented metrics, that confirm this.
Constructivist taught students do
slightly better in the next course.
Super Mario Galaxy 2

30. Learning with Games - MRU SoTL Nov. 2010 30
Next Steps
• Repeated
• Find analogous
measures in other
work.
• Longer follow-
up.***
Super Mario Galaxy 2

31. Learning with Games - MRU SoTL Nov. 2010 31
Thanks!!
Title Slide: Sonny Chan
Game Images courtesy of:
Official Game Sites (images are identified by game)
Fan Art
Mobygames.com
Gamespot.com
IGN.com
Gamewallpapers.com