Presentation for WiPSCE 2017 - Development of Computational Thinking Skills through Unplugged Activities in Primary School. Full-text available at: https://doi.org/10.1145/3137065.3137069

1. Development of Computational Thinking Skills
through Unplugged Activities in Primary School
Christian P. BRACKMANN1, Marcos ROMÁN-GONZÁLEZ2, Gregorio ROBLES2,
Jesús MORENO-LEÓN2, Ana CASALI3, Dante BARONE1
1(IFFAT, UFRGS - Brazil), 2(UNED, URJC - Spain), 3(UNR - Argentina)
{WiPSCE, Nijmegen, 09th November 2017}

2. INTRODUCTION
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Computational thinking (CT) is nowadays being
widely adopted and investigated in many countries
all over the world….

3. INTRODUCTION
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Two main approaches to develop
CT skills in school

4. INTRODUCTION
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Two main approaches to develop
CT skills in school
Computer
programming
activities

5. INTRODUCTION
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Two main approaches to develop
CT skills in school
Computer
programming
activities
Fundamental
way that enables
CT to come alive
A demonstration
of computational
competencies

6. INTRODUCTION
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Two main approaches to develop
CT skills in school
Computer
programming
activities
Fundamental
way that enables
CT to come alive
A demonstration
of computational
competencies
Unplugged activities

7. INTRODUCTION
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Two main approaches to develop
CT skills in school
Computer
programming
activities
Fundamental
way that enables
CT to come alive
A demonstration
of computational
competencies
Unplugged activities
For schools
without basic
technology
resources
For early ages
with limited
screen time

8. INTRODUCTION
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Two main approaches to develop
CT skills in school
Computer
programming
activities
Fundamental
way that enables
CT to come alive
A demonstration
of computational
competencies
Unplugged activities
For schools
without basic
technology
resources
For early ages
with limited
screen time

9. BACKGROUND
To what extent has the unplugged approach been investigated regarding CT
development?
• SLR (Kalelioglu et al., 2016)  34 of 125 articles (27,2%)
To what extent are in-service teachers using unplugged activities in their CS
lessons?
• Survey (Sentance et al., 2015)  47 of 357 teachers (13%)
To what extent do unplugged activities enhance confidence and interest in CS?
• Non conclusive results
To what extent does the unplugged approach works for CS teachers training?
• Survey (Curzon et al., 2013, 2014)  Positive results
To what extent are unplugged activities effective to improve CT skills?
• Pre-experimental research: pretest-posttest without control group (e.g.,
Rodríguez et al., 2017)  Positive results in middle-school

10. BACKGROUND
To what extent has the unplugged approach been investigated regarding CT
development?
• SLR (Kalelioglu et al., 2016)  34 of 125 articles (27,2%)
To what extent are in-service teachers using unplugged activities in their CS
lessons?
• Survey (Sentance et al., 2015)  47 of 357 teachers (13%)
To what extent do unplugged activities enhance confidence and interest in CS?
• Non conclusive results
To what extent does the unplugged approach works for CS teachers training?
• Survey (Curzon et al., 2013, 2014)  Positive results
To what extent are unplugged activities effective to improve CT skills?
• Pre-experimental research: pretest-posttest without control group (e.g.,
Rodríguez et al., 2017)  Positive results in middle-school

11. BACKGROUND
To what extent has the unplugged approach been investigated regarding CT
development?
• SLR (Kalelioglu et al., 2016)  34 of 125 articles (27,2%)
To what extent are in-service teachers using unplugged activities in their CS
lessons?
• Survey (Sentance et al., 2015)  47 of 357 teachers (13%)
To what extent do unplugged activities enhance confidence and interest in CS?
• Non conclusive results
To what extent does the unplugged approach works for CS teachers training?
• Survey (Curzon et al., 2013, 2014)  Positive results
To what extent are unplugged activities effective to improve CT skills?
• Pre-experimental research: pretest-posttest without control group (e.g.,
Rodríguez et al., 2017)  Positive results in middle-school

12. BACKGROUND
To what extent has the unplugged approach been investigated regarding CT
development?
• SLR (Kalelioglu et al., 2016)  34 of 125 articles (27,2%)
To what extent are in-service teachers using unplugged activities in their CS
lessons?
• Survey (Sentance et al., 2015)  47 of 357 teachers (13%)
To what extent do unplugged activities enhance confidence and interest in CS?
• Non conclusive results
To what extent does the unplugged approach works for CS teachers training?
• Survey (Curzon et al., 2013, 2014)  Positive results
To what extent are unplugged activities effective to improve CT skills?
• Pre-experimental research: pretest-posttest without control group (e.g.,
Rodríguez et al., 2017)  Positive results in middle-school

13. BACKGROUND
To what extent has the unplugged approach been investigated regarding CT
development?
• SLR (Kalelioglu et al., 2016)  34 of 125 articles (27,2%)
To what extent are in-service teachers using unplugged activities in their CS
lessons?
• Survey (Sentance et al., 2015)  47 of 357 teachers (13%)
To what extent do unplugged activities enhance confidence and interest in CS?
• Non conclusive results
To what extent does the unplugged approach works for CS teachers training?
• Survey (Curzon et al., 2013, 2014)  Positive results
To what extent are unplugged activities effective to improve CT skills?
• Pre-experimental research: pretest-posttest without control group (e.g.,
Rodríguez et al., 2017)  Positive results in middle-school

14. BACKGROUND
There is a lack of experimental
investigations that prove the effectiveness
of the unplugged activities in the
development of CT skills, particularly in
Primary School

15. RESEARCH QUESTION
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November

16. WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Is the unplugged approach effective for developing
CT skills, specifically in Primary School?
RESEARCH QUESTION

17. RESEARCH QUESTION
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Computational
Thinking
Problem-Solving
Is the unplugged approach effective for developing
CT skills, specifically in Primary School?

18. METHOD
{Research Design}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November

19. METHOD
{Research Design}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Week #1

20. METHOD
{Research Design}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Week #1 Week #2 – Week #6

21. METHOD
{Research Design}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Week #1 Week #7Week #2 – Week #6

22. METHOD
{Participants}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Grade Age Condition
Gender
Total
Boys Girls
School A 5th 10-11 y.o.
Control
Experimental
School B 6th 11-12 y.o.
Control
Experimental
Total 73

23. METHOD
{Participants}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Grade Age Condition
Gender
Total
Boys Girls
School A 5th 10-11 y.o.
Control 10 13
Experimental 10 10
School B 6th 11-12 y.o.
Control 6 8
Experimental 9 7
Total 73

24. METHOD
{Participants}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Grade Age Condition
Gender
Total
Boys Girls
School A 5th 10-11 y.o.
Control 10 13 23
Experimental 10 10 20
School B 6th 11-12 y.o.
Control 6 8
Experimental 9 7
Total 73

25. METHOD
{Participants}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Grade Age Condition
Gender
Total
Boys Girls
School A 5th 10-11 y.o.
Control 10 13 23
Experimental 10 10 20
School B 6th 11-12 y.o.
Control 6 8 14
Experimental 9 7 16
Total 73

26. METHOD
{Participants}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Grade Age Condition
Gender
Total
Boys Girls
School A 5th 10-11 y.o.
Control 10 13 23
Experimental 10 10 20
School B 6th 11-12 y.o.
Control 6 8 14
Experimental 9 7 16
Total 35 38 73

27. METHOD
{Teaching Materials}

28. METHOD
{Teaching Materials}
Example #1: “Decomposition” activity

29. METHOD
{Teaching Materials}
Example #1: “Decomposition” activity Example #2: “Monica’s Map” activity

30. METHOD
{Teaching Materials}
Example #1: “Decomposition” activity Example #2: “Monica’s Map” activity
Example #3: “Elephants” activity

31. METHOD
{Teaching Materials}
Example #4: “Tetris” activity

32. METHOD
{Teaching Materials}
Example #4: “Tetris” activity Example #5: “Repetition Drawing” activity

33. METHOD
{Teaching Materials}
Example #4: “Tetris” activity Example #5: “Repetition Drawing” activity
Example #6: “Monica’s Automata”

34. METHOD
{Assessment Instruments}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
https://goo.gl/5O06Oh

35. METHOD
{Assessment Instruments}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
https://goo.gl/5O06Oh
The CT-test has a precise (although necessarily
reductionist) operational definition of CT:

36. METHOD
{Assessment Instruments}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
https://goo.gl/5O06Oh
The CT-test has a precise (although necessarily
reductionist) operational definition of CT:
“CT is the ability to formulate and solve problems
by relying on fundamental concepts of
computation: sequences, loops, conditionals,
functions, and variables; and using the inherent
logic of computer programming”

37. METHOD
{Assessment Instruments}
The specifications of the CT-test are:
• Target population: 5th – 10th Grade students (10-16 years old)
• Type of instrument: multiple-choice test.
• Length: 28 items; administered online in a maximum time of 45 minutes.

38. METHOD
{Assessment Instruments}
Computational
Concept
• Basic directions
• Loops-Repeat times
• Loops-Repeat until
• If-Simple conditional
• If/else-Complex
conditional
• While conditional
• Simple functions
The specifications of the CT-test are:
• Target population: 5th – 10th Grade students (10-16 years old)
• Type of instrument: multiple-choice test.
• Length: 28 items; administered online in a maximum time of 45 minutes.

39. METHOD
{Assessment Instruments}
Computational
Concept
• Basic directions
• Loops-Repeat times
• Loops-Repeat until
• If-Simple conditional
• If/else-Complex
conditional
• While conditional
• Simple functions
Style of Answers
• Visual arrows
• Visual blocks
The specifications of the CT-test are:
• Target population: 5th – 10th Grade students (10-16 years old)
• Type of instrument: multiple-choice test.
• Length: 28 items; administered online in a maximum time of 45 minutes.

40. METHOD
{Assessment Instruments}
Computational
Concept
• Basic directions
• Loops-Repeat times
• Loops-Repeat until
• If-Simple conditional
• If/else-Complex
conditional
• While conditional
• Simple functions
Style of Answers
• Visual arrows
• Visual blocks
Required Cognitive
Task
• Sequencing
• Completing
• Debugging
The specifications of the CT-test are:
• Target population: 5th – 10th Grade students (10-16 years old)
• Type of instrument: multiple-choice test.
• Length: 28 items; administered online in a maximum time of 45 minutes.

41. METHOD
{Assessment Instruments}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Item #8 (‘maze’):
loops ‘repeat times’ (nested); ‘visual blocks’; ‘sequencing’.

42. METHOD
{Assessment Instruments}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Item #11 (‘maze’):
loops ‘repeat until + repeat times’ (nested); ‘visual arrows’; ‘debugging’.

43. METHOD
{Assessment Instruments}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Item #18 (‘maze’):
loops ‘repeat until’ + if/else conditional (nested); ‘visual blocks’; ‘sequencing’.

44. METHOD
{Assessment Instruments}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Item #26 (‘canvas’):
loops ‘repeat times’ + simple functions (nested); ‘visual blocks’; ‘completing’.

45. METHOD
{Assessment Instruments}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
https://goo.gl/5O06Oh

46. METHOD
{Assessment Instruments}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
https://goo.gl/5O06Oh
Reliability :
• Internal Consistency  r=.80
• Temporal Stability  r=.70
Validity:
• Content Validity
• Criterion Validity
• Cognitive variables
• Non-cognitive variables
• Convergent Validity
• Bebras Tasks
• Dr. Scratch
Román-Gonzálezetal.(2015,2017a,2017b)

47. RESULTS AND DISCUSSION
{Quantitative Results}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November

48. RESULTS AND DISCUSSION
{Quantitative Results}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
d=.17
d=.80

49. RESULTS AND DISCUSSION
{Quantitative Results}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
ANCOVA
F(1,72) = 11.69**
Global Effect Size
d=.59
{moderate-large effect}
d=.17
d=.80

50. RESULTS AND DISCUSSION
{Quantitative Results}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
ANCOVA
F(1,72) = 11.69**
Global Effect Size
d=.59
{moderate-large effect}
d=.17
d=.80
≈
Global Effect Size
d=.62
12 weeks quasi-experiment
Code.org Course
7th & 8th Grade
(Román-González, 2016)

51. RESULTS AND DISCUSSION
{Quantitative Results}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
5th Grade
Global Effect Size
d=.55
6th Grade
Global Effect Size
d=.63

52. RESULTS AND DISCUSSION
{Quantitative Results}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
5th Grade
Global Effect Size
d=.55
6th Grade
Global Effect Size
d=.63

53. RESULTS AND DISCUSSION
{Qualitative Results}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
“Decomposition” activity

54. RESULTS AND DISCUSSION
{Qualitative Results}
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
“Decomposition” activity

55. LIMITATIONS AND THREATS TO VALIDITY
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
• The CT-test is heavily focused on ‘computational concepts’,
only partially covers ‘computational practices’, and ignores
‘computational perspectives’.
• The CT-test has a (deliberately) reductionist conception of CT,
which puts over-emphasis on path-finding algorithms.

56. LIMITATIONS AND THREATS TO VALIDITY
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
• The CT-test is heavily focused on ‘computational concepts’,
only partially covers ‘computational practices’, and ignores
‘computational perspectives’.
• The CT-test has a (deliberately) reductionist conception of CT,
which puts over-emphasis on path-finding algorithms.
• The unplugged activities used along the research might be
excessively aligned with the items of the CT-test, which could
inflate the effect size of the quasi-experiment.
• The unplugged activities were conducted by one of the
researchers, not by the regular teacher.
• Small size of the sample in our quasi-experiment (N < 120).

57. CONCLUSIONS AND FURTHER RESEARCH
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
• Conclusion #1: The unplugged approach may be
effective to develop computational thinking in
Primary School.

58. CONCLUSIONS AND FURTHER RESEARCH
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
• Conclusion #1: The unplugged approach may be
effective to develop computational thinking in
Primary School.
• Conclusion #2: The effect size of the unplugged
approach to develop CT, seems to be similar to the
one of programming.

59. CONCLUSIONS AND FURTHER RESEARCH
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
• Conclusion #1: The unplugged approach may be
effective to develop computational thinking in
Primary School.
• Conclusion #2: The effect size of the unplugged
approach to develop CT, seems to be similar to the
one of programming.
• Conclusion #3: This results reinforce the conception
of CT as a problem-solving cognitive ability whose
development can be disconnected from computer
programming.

60. CONCLUSIONS AND FURTHER RESEARCH
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
How and when to combine and/or to merge
unplugged and plugged activities, in order to
optimize the CT development?

61. CONCLUSIONS AND FURTHER RESEARCH
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
How and when to combine and/or to merge
unplugged and plugged activities, in order to
optimize the CT development?
Klopfenstein et al., 2017

62. REFERENCES
Paul Curzon. 2013. cs4fn and computational thinking unplugged. In Proceedings of the 8thWorkshop in Primary and
Secondary Computing Education. ACM, 47–50.
Paul Curzon, Peter W McOwan, Nicola Plant, and Laura R Meagher. 2014. Introducing teachers to computational thinking
using unplugged storytelling. In Proceedings of the 9th Workshop in Primary and Secondary Computing Education. ACM,
89–92.
Filiz Kalelioglu, Yasemin Gülbahar, and Volkan Kukul. 2016. A Framework for Computational Thinking Based on a
Systematic Research Review. Baltic Journal of Modern Computing 4, 3 (2016), 583.
Lorenz Klopfenstein, Andiy Fedosyeyev, and Alessandro Bogliolo. 2017. Bringing an unplugged coding card game to
augmented reality. 9800–9805.
Brandon Rodriguez, Kennicutt Stephen, Cyndi Rader, and Tracy Camp. 2017. Assessing Computational Thinking in CS
Unplugged Activities. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education.
ACM, Seattle, Washington, USA, 501–506.
Marcos Román-González. 2015. Computational Thinking Test: Design Guidelines and Content Validation. In Proceedings
of the 7th Annual International Conference on Education and NewLearning Technologies (EDULEARN 2015). IATED,
Barcelona, Spain, 2436–2444.
Marcos Román-González. 2016. Codigoalfabetización y Pensamiento Computacional en Educación Primaria y
Secundaria: Validación de un Instrumento y Evaluación de Programas. Ph.D. Dissertation. Universidad Nacional de
Educación a Distancia, Madrid, Spain.
Marcos Román-González, Juan-Carlos Pérez-González, and Carmen Jiménez-Fernández. 2017. Which cognitive abilities
underlie computational thinking? Criterion validity of the Computational Thinking Test. Computers in Human Behavior
72 (July 2017), 678–691.
Marcos Román-González, Jesús Moreno-León, and Gregorio Robles. 2017. Complementary Tools for Computational
Thinking Assessment. In Proceedings of International Conference on Computational Thinking Education (CTE 2017), S. C
Kong, J Sheldon, and K. Y Li (Eds.). The Education University of Hong Kong, 154–159.
Sue Sentance and Andrew Csizmadia. 2015. Teachers’ perspectives on successful strategies for teaching Computing in
school. Paper presented at IFIP TCS 2015 (2015).

63. FOLLOW UP & CONTACT
WiPSCE 2017 - The 12th Workshop in Primary and Secondary Computing Education - Nijmegen, 9th November
Thanks for
your attention
mroman@edu.uned.es