Presentation at IEEE TALE 2021

1. Is Augmented Reality Robot as Effective as
Physical Robot in Motivating Students
to Learn Programming?
Oka Kurniawan, Norman Lee, Nachamma Sockalingam
December 2021

2. A BETTER WORLD BY DESIGN.
Overview of Our Presentation
Motivation and Earlier Study
Literature Review
About our Physical and AR Robots
Results - Analysis- Discussion

3. A BETTER WORLD BY DESIGN.
How to Make Learning of Computing Exciting?
Systematic literature review: teaching novices
programming using robots, by Major L., et. al., (2011)
• 75 % report robots are an effective teaching tool and
can help novice programmers in their studies.
• 25 reports that robot is effective, 5 mixed, 1
ineffective
• 23 use physical robots: 16 effective, 4 mixed, 1
ineffective
• 7 use simulated robots: 6 effective, 1 unclassifiable
• 4 use physical + simulated robots: 3 effective, 1
mixed
Methods: course critique surveys, questionnaires,
interviews, focus groups, comparative analysis, student
grades analysis.
Further research is needed to assess the effectiveness
of robot simulator.
Photo by Andy Kelly on Unsplash

4. A BETTER WORLD BY DESIGN.
Challenges in Using Physical Robot
Hardware Failures
Debugging
Lack of Simulator
Maintenance
Fagin, Barry, and Laurence
Merkle. 2003. ‘Measuring the
Effectiveness of Robots in
Teaching Computer Science’.
ACM SIGCSE Bulletin 35 (1):
307.
https://doi.org/10.1145/79254
8.611994.

5. Previous study shows no significant difference in learning gain,
Physical robots has an edge for motivation and interest
Programming
Skills
Confidence
Interest
Kurniawan, O., Lee, N. T. S., Datta, S.,
Sockalingam, N., & Leong, P. K. (2018).
Effectiveness of Physical Robot Versus
Robot Simulator in Teaching Introductory
Programming. In 2018 IEEE International
Conference on Teaching, Assessment, and
Learning for Engineering (TALE) (pp.
486–493).
What if we used AR instead of
simulation?

6. Using physical markers to write code
AR gives immediate feedback on the running code
Boonbrahm, Salin, Poonpong Boonbrahm, Charlee Kaewrat, Prasert Pengkaew, and Prathomjit Khachorncharoenkul. 2019.
‘Teaching Fundamental Programming Using Augmented Reality’. International Journal of Interactive Mobile Technologies
13 (7): 31–43. https://doi.org/10.3991/ijim.v13i07.10738.

7. Similar physical markers for tangible computational thinking
Limited activity boards
Goyal, Sidhant, Rohan S. Vijay, Charu Monga, and Pratul Kalita. 2016. ‘Code Bits: An Inexpensive Tangible Computational
Thinking Toolkit for K-12 Curriculum’. TEI 2016 - Proceedings of the 10th Anniversary Conference on Tangible Embedded
and Embodied Interaction, 441–47. https://doi.org/10.1145/2839462.2856541.

8. Mixed Reality Device for programming
Costly and heavy
A BETTER WORLD BY DESIGN.
https://youtu.be/15fhK7Tyxek

9. Comparison of AR using different form factor
Participants enjoyed Mobile AR the most
Dass, Nathan, Joonyoung Kim, Sam Ford, Sudeep Agarwal, and Duen Horng Polo Chau. 2018. ‘Augmenting Coding: Augmented Reality for Learning
Programming’. ACM International Conference Proceeding Series, 156–59. https://doi.org/10.1145/3202667.3202695.
A BETTER WORLD BY DESIGN.

10. Allows configurable tile using physical markers
Limited commands for code
Kim, Joonyoung, Sudeep Agarwal, Kristina Marotta, Siwei Li, Jonathan Leo, and Duen Horng Chau. 2019.
‘Mixed Reality for Learning Programming’. Proceedings of the 18th ACM International Conference on
Interaction Design and Children, IDC 2019, 574–79. https://doi.org/10.1145/3311927.3325335.
A BETTER WORLD BY DESIGN.

11. SUTD AR Robot
A BETTER WORLD BY DESIGN.
https://youtu.be/Qc9h8ru71Uo

12. Comparing Physical Robot and AR Robot
A BETTER WORLD BY DESIGN.

13. Setup of Physical Robot
A BETTER WORLD BY DESIGN.

14. Setup of AR Robot
A BETTER WORLD BY DESIGN.

15. A BETTER WORLD BY DESIGN.
Bringing AR Technology into the Classroom
Detect
flat
surface
Detect
Wall
markers
Detect
Robot
Write
Code
Test
Coloured floor object
Wall boundary
Simulated Robot

16. No Significant Difference for
Intrinsic Motivation Inventory and Confidence Survey
A BETTER WORLD BY DESIGN.
Participants’ confidence level in computing after Task 1 (Survey 1). The
statements are categorized into either positive or negative statements. The
Likert scores for both positive and negative statements show consistency in
participants’ responses. There is no significant difference between the group
using augmented reality robot and the group using physical robot for confidence
level.
Participants’ intrinsic motivation inventory scores after Task 1. The statements
are categorized into either positive or negative statements. The Likert scores for
both positive and negative statements show consistency in participants’
responses. There is no significant difference between the group using
augmented reality robot and the group using physical robot for the intrinsic
motivation.

17. Students with AR Robot have fewer negative statements
On their learning experience
A BETTER WORLD BY DESIGN.
Statements
Category
Raw Filtered
Physical
Robot
AR
Robot
Physical
Robot
AR
Robot
Positive 15 14 12 10
Negative 8 6 7 0
“The least favourable experience is the 'environmental' factors such as the thick
cables hindering the movements of the robot which is completely understandable.”
“I have the fear of breaking the robot with my code.”
“Robotic hardware malfunction.”
“Robot behaves strangely sometimes despite code appearing to be sound.”
Filtered counts removes
statements that does not
involve the
characteristics of the
technology either
Physical Robot or AR
Robot.
An example of a
removed statement:
“The IDE can be
improved: regarding
indentations of the
codes.”

18. Positive statements highlight the characteristics
Of Physical Robot and AR Robot
A BETTER WORLD BY DESIGN.
PHYSICAL ROBOT
“The most favourable experience would
be to see physically the output of the
code I computed.”
“It is more interactive and engaging to
see an actual robot to code.”
“There’s something satisfying about
seeing a tangible object operate from
your code.”
AR ROBOT
“Seeing how the robot moves onscreen
even though it doesn’t exist in real life”
“I like that it is very logical and can be
understood in a flowchart (student helper
taught me). I also like that it is
accessible. There is no need for the
physical device.”
“The most favourable one would be how
the environmental factors with the
physical robot are eliminated with the
AR version.”

20. Thank you
A BETTER WORLD BY DESIGN.
Scan QR code or visit our website to try
our AR robot.
https://www.okakurniawan.net/robotar