Master presentation at Graz University of Technology, January 2015

1. The Use of Modern Controller Devices
at Schools: Game-Based Learning with
the Leap Motion
Norbert Spot
Supervisor: Assoc.Prof. Dipl.-Ing. Dr.techn. Martin Ebner

2. Contents
• Introduction
• Motivation
• The Prototype
• Evaluation
• Conclusion
• Demo

3. Introduction
• Innovative Controller Devices
• MS Kinect
• MakeyMakey
• Bare Conductive’s Touch Board
• Leap Motion

4. MS Kinect
• RGB Camera (VGA @ 30Hz)
• depth sensor (IR laser with a
monochrome CMOS, VGA @
30Hz)
• multi array microphone
• full-body 3D motion capture,
facial recognition, voice
recognition
• Xbox 360, Xbox One,
Windows PC

5. MakeyMakey
• invention kit
• turns everyday objects into touch-
pads
• clip two objects to the MakeyMakey
(yourself and a banana)
• when you touch the banana the
device sends a keyboard message
to the computer
• works with any material that
conducts at least a tiny bit of
electricity

6. Bare Conductive Touch Board
• similar to MakeyMakey
• works great with Bare
Conductive's Electric Paint
• paint a light switch or a
custom interactive surface
• connect anything conductive

7. Leap Motion
• 3D IR Camera
• 3 IR light sources, 2 wide
angle (150 degrees)
monochrome IR cameras @
200Hz
• tracks all 10 fingers, both
hands and arms
• precision up to 0,01mm*
• works on Mac, Linux, Windows
(native, web)
*real world avarage accuracy 0,7mm (avarage accuracy of the human hand 0,4mm)

9. Natural User Interface
• according to D. Plemmons and P. Mandel
intuitive interfaces are
• learnable
• understandable
• habitual
• natural

10. – Steve Ballmer
“I believe we will look back on 2010 as the
year we expanded beyond the mouse and
keyboard and started incorporating more
natural forms of interaction such as touch,
speech, gestures, handwriting, and vision --
what computer scientists call the “NUI” or
natural user interface.”

11. Game-Based Learning
• playing games may support the development of
certain skills and strategies
• problem-solving
• decision-making
• understanding complex systems
• planning, data handling

12. Motivation
• Game-Based learning with computers can be
motivational
• the use of iPads at schools makes teaching/
learning more interesting
• possibilities provided by the Leap Motion

13. • Leap Motion
• assessing a CT scan with
the SW called Osirix
• virtual keyboard DexType
• using the Leap Motion with
Oculus Rift
• controlling a drone
• painting Heineken bottles
• games

14. The Goal
• make use of a modern controller device, which
provides a natural user interface
• create a prototype of an application with game-
based learning in mind
• see how it performs in the real world

15. The Prototype
• a game for exercising the small multiplication
chart
• built in Unity3D
• scripts written in C#

16. Unity3D
• full-featured 3D and 2D game
development environment
• works on Windows and
Mac OS X
• it was used to create
• Bad Piggies
• Monument Valley
• Threes

17. The Prototype

18. Challenges
• dealing with Unity3D and C#
• designing an interesting game world and an intuitive
interface
• navigating with a "virtual joystick”
• efficient random exercise generation
• Fisher and Yates method
• the game menu

19. The Game Menu

20. The Game Menu

21. Evaluation
• 3rd grade elementary school
students
• kids interviewed with the help
of cut-off technique
• a small group got 5
statements
• ranked the statements as a
group with the help of
smileys

22. Evaluation
• statements
• I would like to play the game again
• It was easy to pop the balloons
• It was easy to find the balloons
• I think the exercises were easy
• I think the game was easy to play

23. Evaluation
• majority gave the best mark for all the
statements
• the kids loved the game
• “The best game I have ever played”

24. Evaluation
• four groups at the end
• kids good in playing the game
• kids good in math, having problems with
gameplay
• kids good in gameplay, slower in math
• kids bad in math, having problems with
gameplay

25. Conclusion
• Game-Based learning with the Leap Motion is
fun and motivational
• might not be for everyone
• fun way to practice a school subject

26. Demo

27. Thank you for your attention!

28. • Colgan A. (2014) How Does the Leap Motion Controller Work? http://
blog.leapmotion.com/hardware-to-software-how-does-the-leap-motion-
controller-work/
• Griffiths, M. (2014) Playing video games is good for your brain – here’s how
http://theconversation.com/playing-video-games-is-good-for-your-brain-heres-
how-34034
• Kuntz Ch. (2013) Controlling Osirix with the Leap Motion While Scrubbed into
Surgery https://www.facebook.com/photo.php?v=10152121411384392
• Liebeskind D. (2014) The Beginning of a Drone Revolution http://
blog.leapmotion.com/the-beginning-of-a-drone-revolution/
• Plemmons D., Mandel P. Introduction to Motion Control https://
developer.leapmotion.com/articles/intro-to-motion-control
• Weichert F., Bachmann D., Rudak B., Fisseler D. (2013) Analysis of the
Accuracy and Robustness of the Leap Motion Controller, Department of
Computer Science VII, Technical University Dortmund, Dortmund, Germany