Talk slides presented at Software Engineering Educators Symposium (SEES) 2014, Hong Kong, November 17, 2014. http://cs.nju.edu.cn/changxu/sees/

1. Gamifying Teaching and
Learning of Software
Engineering and
Programming
Tao Xie
University of Illinois at Urbana-Champaign
In collaboration with Nikolai Tillmann, Jonathan de Halleux, and Judith Bishop
(Microsoft Research)
http://pex4fun.com
https://www.codehunt.com
Related Papers/Resources: https://sites.google.com/site/asergrp/projects/ese

2. Working for fun
Enjoyment adds to long term retention on a task
Discovery is a powerful driver, contrasting with
direct instructions
Gaming joins these two, and is hugely popular
Can we add these elements to coding?
Write a program to determine all the sets of
effectively identical rooms in a maze. (A page of
background, sample input and output given)

3. Testing Tool  Educational Gaming
Dynamic Symbolic Execution (Pex)
Pex for Fun:
Interactive Gaming for
Teaching and Learning
Support
https://www.codehunt.com
http://research.microsoft.com/pex/ http://pex4fun.com

4. Agenda
Background
Coding Duels
Educational Platform
Experiences
Conclusion

5. Agenda
Background
Coding Duels
Educational Platform
Experiences
Conclusion

6. Background
Dynamic Symbolic Execution (DSE)
aka. Concolic Testing
[Godefroid et al. 05][Sen et al. 05][Tillmann et al. 08]
Instrument code to explore feasible paths
http://research.microsoft.com/pex/

7. http://research.microsoft.com/pex/

8. Dynamic Symbolic Execution in Pex
void CoverMe(int[] a)
{
if (a == null) return;
if (a.Length > 0)
if (a[0] == 1234567890)
throw new Exception("bug");
}
F
a.Length>0
F T
a[0]==123…
T
F
a==null
T
Choose next path
Solve Execute&Monitor
Constraints to solve
a!=null
a!=null &&
a.Length>0
a!=null &&
a.Length>0 &&
a[0]==123456890
Input
null
{}
{0}
{123…}
Observed constraints
a==null
a!=null &&
!(a.Length>0)
a==null &&
a.Length>0 &&
a[0]!=1234567890
a==null &&
a.Length>0 &&
a[0]==1234567890
Done: There is no path left.
http://pex4fun.com/HowDoesPexWork

9. Pex is Part of Visual Studio 2015!
• As new feature of “Smart Unit Tests”
http://www.visualstudio.com/en-us/news/vs2015-preview-vs#Testing

10. Coding Duels
1,594,092 clicked 'Ask Pex!'

11. Coding Duels
Pex computes “semantic diff” in cloud
secret reference implementation vs.
code written in browser
You win when Pex finds no differences
For more info, see our ICSE 2013 SEE paper:
http://web.engr.illinois.edu/~taoxie/publications/icse13see-pex4fun.pdf

12. Behind the Scene of Pex for Fun
Secret Implementation
class Secret {
public static int Puzzle(int x) {
if (x <= 0) return 1;
return x * Puzzle(x-1);
}
}
Player Implementation
class Player {
public static int Puzzle(int x) {
return x;
}
}
class Test {
public static void Driver(int x) {
if (Secret.Puzzle(x) != Player.Puzzle(x))
throw new Exception(“Mismatch”);
}
}
behavior
Secret Impl == Player Impl
12
1,594,092

13. Code Hunt Programming Game

14. Code Hunt Programming Game

15. Code Hunt Programming Game

16. Code Hunt Programming Game

17. Code Hunt Programming Game

18. Code Hunt Programming Game

19. Code Hunt Programming Game

20. Code Hunt Programming Game

21. Code Hunt Programming Game

22. Code Hunt Programming Game

23. Code Hunt Programming Game

24. Code Hunt Programming Game

25. Code Hunt Programming Game

26. Leaderboard and Dashboard
Publically visible, updated during the contest
Visible only to the organizer

27. It’s a game!
iterative gameplay
adaptive
personalized
no cheating
clear winning criterion
code
test cases

28. Audiences
Students: proceed through a sequence on
puzzles to learn and practice.
Educators: exercise different parts of a
curriculum, and track students’ progress
Recruiters: use contests to inspire communities
and results to guide hiring
Researchers: mine extensive data in Azure to
evaluate how people code and learn

29. Social Experience
Community
High score lists, leaderboard
Live feed
http://pex4fun.com/Community.aspx
http://pex4fun.com/Livefeed.aspx

30. Agenda
Background
Coding Duels
Educational Platform
Experiences
Conclusion

31. Teaching and Learning

32. Skills Being Trained
Induction
Problem solving/debugging
Program understanding/programming
Testing
Specification writing

33. Coding Duels for Course Assignments
@Grad Software Engineering Course
Observed Benefits
• Automatic Grading
• Real-time Feedback (for Both Students and Teachers)
• Fun Learning Experiences
http://pexforfun.com/gradsofteng

34. Coding Duels for Training Testing
public static string Puzzle(int[] elems, int capacity, int elem) {
if ((maxsize <= 0) || (elems == null) || (elems.Length > (capacity + 1)))
return "Assumption Violation!";
Stack s= new Stack(capacity);
for (int i = 0; i < elems.Length; i++)
s.Push(elems[i]);
int origSize = s.GetNumOfElements();
//Please fill in below test scenario on the s stack
//The lines below include assertions to assert the program behavior
PexAssert.IsTrue(s.GetNumOfElements() == origSize + 1);
PexAssert.IsTrue(s.Top() == elem); PexAssert.IsTrue(!s.IsEmpty());
PexAssert.IsTrue(s.IsMember(elem));
return s.GetNumOfElements().ToString() + "; “ + s.Top().ToString() + "; “
+ s.IsMember(elem).ToString() + "; " + s.IsEmpty();
}
Set up a stack with some elements
Cache values used in assertions

35. Coding Duel Competition
@ICSE 2011
http://pexforfun.com/icse2011

36. Example User Feedback
Released since 2010
“I used to love the first person shooters and the
satisfaction of blowing away a whole team of
Noobies playing Rainbow Six, but this is far more
fun.”
X
“I’m afraid I’ll have to constrain myself to spend just an hour
or so a day on this really exciting stuff, as I’m really stuffed
with work.”
“It really got me *excited*. The part that got me most is
about spreading interest in teaching CS: I do think that it’s
REALLY great for teaching | learning!”

37. Recap: Code Hunt: A Game for Coding
For individuals (K12, introductory courses, geeks)
For competitions at any level, world-wide or in house
Based on long-term research on symbolic program
analysis (Pex, Z3)
Works with Java and C#
Runs in any modern browser
Now working on tablets and phones
www.codehunt.com

38. Summary: Testing Tool Educational Gaming
DSE/Pex
Pex for Fun:
Interactive Gaming for
Teaching and Learning
Support
https://www.codehunt.com
http://pex4fun.com

39. Testing Tool  Educational Gaming
http://research.microsoft.com/pex/ http://pex4fun.com
Support
Q & A
Thank you!
Contact: taoxie@illinois.edu http://www.cs.illinois.edu/homes/taoxie/
Related Papers/Resources: https://sites.google.com/site/asergrp/projects/ese

40. Summary: Testing Tool Educational Gaming
DSE/Pex
Pex for Fun:
Interactive Gaming for
Teaching and Learning
Support
https://www.codehunt.com
http://pex4fun.com