Educational Games for K-12 Computer Science (ACSE 2020)

Educational
Games for
K-12 Computer
Science
Michael Miljanovic • Jeremy S. Bradbury
Software Quality Research Lab
Ontario Tech University
Oshawa, ON, Canada
michael.miljanovic@ontariotechu.ca
jeremy.bradbury@ontariotechu.ca
http://www.sqrlab.ca
ACSE 2020

Acknowledgements
Michael Miljanovic, Luisa Rojas Garcia, Jana Kanagalingam,
Onyedikachi Kalu, Daniel Hope, Damon Barton,
Kashif Hussain, Ibrahim Mushtaq, Scott McLean
2© M.A. Miljanovic & J.S. Bradbury

What is a Serious Game?
• A game that is not purely for entertainment
• Has a serious purpose – usually learning
• Can still be fun!
• Serious game ≠ gamification

Serious Programming Games
Code Hunt, Code Combat
Lightbot 2.0, Gidget

Questions
What kinds of serious
programming games
are available?
What programming
concepts does each
serious game cover?
Do existing serious
games actually help
student learn
programming?

Michael A. Miljanovic, Jeremy S.
Bradbury. “A Review of Serious
Games for Programming,” Proc. of the
4th Joint Conference on Serious
Games (JCSG 2018), pages 204-216,
Nov. 7-8, 2018.
A Review of
Serious
Programming
Games

Identification
Serious Programming
Games
(49)
Games in the
Research Literature
(40)
Industry/Commercial
Games
(4)
Academic/Research
Games
(36)
Commercial Games
(9)

Audience
14%
8%
43%
35%
Children
High School
Adult Novices
Adult Adepts

Serious Programming Game Content
ACM/IEEE-CS Joint Task
Force on Computing
Curricula: Computer
Science Curricula 2013.
ACM Press & IEEE
Computer Society Press
(2013).
Available at:
https://www.acm.org/binar
ies/content/assets/educatio
n/cs2013_web_final.pdf

Algorithms &
Design
(e.g. Problem
Solving)
Fundamental
Programming
Concepts
(e.g.
Conditionals
& iteratives)
Development
Methods
(e.g.
Debugging
strategies)
Fundamental
Data
Structures
(e.g. Arrays)
Software
Design
(e.g. Object-
oriented
design)
Software Development Fundamentals
Software
Engineering
ACM/IEEE-CS Joint Task Force on Computing Curricula: Computer Science Curricula 2013. ACM Press & IEEE Computer
Society Press (2013).

Assessment of serious
programming game
content available at:
http://bit.ly/csgamescontent
List of games available at:
http://bit.ly/csgameslist

Assessment of serious
programming game
evaluations available at:
http://bit.ly/cagameseval
List of games available at:
http://bit.ly/csgameslist
Serious Programming Game Evaluations

Evaluation of Serious Programming
Games
• Learning efficacy of serious programming
games is not well understood
• Lack of standardized evaluation methodology
• Few studies include empirical data for
reproducibility

Serious Games in the SQR Lab

Bradbury. “Robot ON!: A Serious
Game for Improving Programming
Comprehension,” Proc. of
the 5th International Workshop on
Games and Software Engineering
(GAS 2016), pages 33-36, May
2016.
Robot ON!: A
Serious Game for
Improving
Programming
Comprehension

Motivation
• Many students struggle in introductory
computer programming courses
• Students especially struggle to understand
code they did not write themselves
• Most serious games about programming
involve players writing their own code
• BUT students lack programming
comprehension skills

Robot ON!
• Two dimensional
• The player can run, climb, or fall through lines of
code
• Mimics a real code environment by allowing
navigation similar to a word processor cursor
• Tools provided to the player allow them to
interact with different puzzle tasks
• Each line of code may have any number of puzzle
tasks

Robot ON! Puzzle Tasks
• Puzzle tasks focus on understanding key
programming concepts.
Variable Values
Data Type
Identification
Program
Statements
Control Flow

Robot ON! Demo
https://robogames.science.uoit.ca/RoboDemo/

Customizing Robot ON!
• Robot ON! was designed to be customizable
and extendible by course instructors
• Levels can be created for different programming
languages
• Puzzle tasks can be created to target specific
students and learning materials
• Creating puzzle tasks provides students a chance
to learn through failure!

Bradbury. “RoboBUG: A Serious
Game for Learning
Debugging,” Proc. of the
13th Annual ACM International
Computing Education Research
Conference (ICER 2017), pages 93-
100, Aug. 2017.
RoboBUG: A
Serious Game
for Learning
Debugging

Debugging Techniques for Novices
Code
Tracing
BreakpointsPrint
Statements
Divide &
Conquer

RoboBUG Demo
https://robogames.science.uoit.ca/RoboDemo/

Luisa Rojas Garcia, Michael A.
Miljanovic, Jeremy S. Bradbury.
“Threaded Paws: A Serious
Game for Learning the Pitfalls of
Concurrent Programming,” In
Technology Showcase at the
27th Annual International
Conference on Computer Science
and Software Engineering
(CASCON 2017), Nov. 2017.
Threaded Paws: A
Serious Game for
Learning the Pitfalls
of Concurrent
Programming

Concurrency Pitfalls
• Starvation
• Deadlock

Threaded Paws Demo
http://www.sqrlab.ca/threaded-paws/

Bradbury. “GidgetML: An Adaptive
Serious Game for Enhancing First
Year Programming Labs,” Proc. of
the 42nd International Conference
on Software Engineering (ICSE
2020), The Software Engineering
Education and Training (SEET)
track, May 2020. (to appear)
GidgetML
Bradbury. “Making Serious
Programming Games
Adaptive,” Proc. of the 4th Joint
Conference on Serious Games (JCSG
2018), pages 253-259, Nov. 7-8,
2018.

Problem
• Computer science
(CS) is a heavily
targeted field for
serious games
• Programming
games are rarely
customizable
– Large workloads for
experts

Our Solution
• Automatically adapt game play tasks based on
non-invasive player assessments
• Use Competence-based Knowledge Space
Theory (CbKST) to aid in selecting tasks that
improve learner competences

Automatic Adaptation
Identify Model
BuildEvaluate

Identify
• Technical Factors
• Prefer open-source P
• High quality and robust code P
• Games with a playability study P
• Learning Factors
• Determine topics and learning outcomes P
• Prefer games with a diverse audience P
• Games with existing evaluations PPP

Creating an Adaptive Version of
Gidget

Task Model
Goal/Objective
• Move Gidget
to the dog
Learning
Content
• Debugging
Game
Elements
• Starting code
Parameters
• Energy limit

Task Model
Goal/Objective
• Move Gidget
to the dog
Learning
Content
• Debugging
Game
Elements
• Starting code
Parameters
• Energy limit
Success Rate
• Number of
failed
submissions
Comprehension
• Errors in
levels with
conditionals
Engagement
• Completed
10/15 levels
before
quitting
Play Skill
• Completed
previous level
in 3 minutes
Assessment Model

GidgetML Adaptations
Low Competence High Competence

Research
Questions
© M.A. Miljanovic & J.S. Bradbury 36
Does Gidget benefit
from adaptation?
Is an adaptive version
of Gidget effective at
adapting to a learner’s
level of competency?

Results
• 100 first year undergraduate students enrolled in a CS
program
– Average age: 18.5 years
– 81 men, 12 women, 7 transgendered or not willing
to specify
• Administered in first day of laboratories, across 6
different sections
• 23% had never taken a CS course before
• 30% had never written a program before

Participants
32%
68%
Gidget GidgetML

Variance of Failure Rate

Variance of Energy Usage

Summary
• There is data to support implementing an
adaptive version of Gidget
– Learners demonstrated a high level of variance in
both their failures and the efficiency of their
solutions
• GidgetML significantly decreased variance
among those who played the game
– Did not require substantial alterations to the game’s
source code
– Variance continued to decrease as more data was
collected

Serious Games in the SQR Lab
• All games are open sourced and available on
GitHub
https://github.com/sqrlab

Educational Games for K-12 Computer Science (ACSE 2020)

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

Educational Games for K-12 Computer Science (ACSE 2020)