This document investigates the use of scaffolded engineering notebooks to enhance students' collaboration and communication skills in the context of the First Lego League (FLL) robotics competition. It discusses the methodology, data collection, and emerging themes from research involving coaches and students, highlighting the importance of instructional strategies in developing these 21st-century skills. The study aims to contribute to the literature on project-based learning and provide resources for new FLL coaches.

1. Investigating the use of scaffolded engineering
notebooks in developing school students’
communication and collaboration skills.
ASERA 2022
Michael Graffin (PhD Student, Curtin University) @mgraffin
Supervisors: Dr Rachel Sheffield & Dr Rekha Koul
1

2. Overview
• Key Definitions
• Research Context
• Conceptual Framework
• Methodology
• Research Questions & Data
Instruments
• Themes (thus far)
2

3. Defining 21st Century Collaboration &
Communication Skills / Transversal Competencies
3
Collaboration
Communication
• A complex, dynamic process involving the exchange of knowledge,
opinions, and ideas in pursuit of a common goal or shared
understanding.
• Dependent on effective communication, cooperation, and active
engagement of all participants. (Hesse et al., 2015; Koul et al., 2020; Sale, 2020; Scoular et al., 2020a)
• Purposeful interactions with various audiences. (Hesse et al, 2015; Scoular et al., 2020a)
• Shaped by time, social and cultural context, and mode. (Koul et al., 2021)

4. 4
What is the FIRST LEGO League Challenge?
A hands-on, multidisciplinary project-based-learning (PjBL) educational robotics
competition incorporating a robot game, innovation (research) project, and an
emphasis on ‘core values’, including teamwork. Use of engineering notebooks
for documentation is strongly encouraged.
(Condliffe et al., 2017; Jumaat et al., 2017; Papert, 1991, 1996).
Team Building Task Competition Robot RFID Parcel security solution

5. 5
FLL in the Literature
• Competition participation positively impacts on students’
STEM attitudes, and 21st Century skill development, BUT this
is influenced by coaches’ knowledge, skills, and pedagogical
approach. (Burack et al., 2019; Chalmers, 2013; Chen, 2018; Eguchi, 2017; Meschede et al., 2022)
• This study aims to explore best-practice instructional
strategies for scaffolding students’ learning and development
of 21st Century collaboration and communication skills in the
context of the FLL robotics competition.

6. 6
Conceptual
Framework
(Jan 2022)

7. 7
Research Methodology
• Qualitative multiple case-study methodology based on
an interpretivist paradigm (Merriam & Tisdell, 2016; Cohen et al., 2017; Creswell &
Guetterman, 2019).
• Focus on exploring the lived experience and
perspectives of FLL coaches and students (Cohen et al, 2017; Merriam
& Tisdell, 2016).

8. 8
Methods
Data Collection
• Aim to collect rich qualitative data and to triangulate findings from semi-structured
observations, interviews, and content analysis of team engineering notebooks in the
2021 and 2022 FLL seasons (Cohen et al., 2017; Merriam & Tisdell, 2016).
Case Selection
Multiple case studies – purposeful sample of teams in their first three years of competition.
• Each team (case) involving 1 -2 coaches and 6 -10 students, aged between 9-16.
• Note: Coaches may coach multiple teams.
Sites
• Phase 1 (2021): Home-school, Catholic K-12, and Independent K-12
• Phase 2 1-2 Catholic/Independent Schools, and a Community team (e.g. library/home-
school).
• Australian competition season: August-December.

9. 9
Phase 1: Data Instruments & Analysis (2021-22)
RQ Instruments Analysis
1. How do coaches use
scaffolded engineering
notebooks to support
their students’
development of
collaboration and
communication skills in
FIRST LEGO League?
• Semi-structured interviews with
FLL coaches pre-and post-season
• Semi-structured observations of
coaches’ instructional practice
• Interviews about instances
(Treagust & Duit, 2008).
• Team Engineering Notebook
artefacts
• Content analysis of transcripts - initial
coding and category
development (NVivo)
• Content Analysis of Observation
protocol notes
• Member check (later in 2022)
• Content analysis of engineering
notebooks
2. How to students use
scaffolded engineering
notebooks to support
their collaboration and
communication in
FIRST LEGO League?
• Pre- and post-season focus group
interviews with students including a
team building (performance) task.
• Team Engineering Notebook
artefacts
• Content analysis of transcripts and
engineering notebooks
• Collaboration performance
assessment (Menekse et al., 2017)
• Content analysis - initial coding and
category development

10. 10
Phase 2: Data Instruments & Analysis (2022-23)
RQ Instruments Analysis
3. How does the professional
learning resource support
coaches' application of
the identified scaffolding
strategies to guide their
students’ reflection and the
articulation of their team
collaboration and
communication processes?
• Semi-structured interviews with
FLL coaches pre-and post-
season
• Semi-structured observations of
coaches’ instructional practice
• Reflective interviews about
instances (Treagust & Duit,
2008).
• Team Engineering Notebook
artefacts
• Content analysis of transcripts - initial
coding and category development
• Using Phase 1 observation protocol
• Member check
• Content analysis of engineering
notebooks

11. Phase 1 Case Studies (2021)
11
Site 1
School Independent K-12
Participating
Coaches
(Pseudonyms)
Daniel (Head of Science and
Technology)
Emma (Secondary Science
teacher)
Participating
Students
11 boys (2 Teams)
Year 5-10 (age 11-15)
Prior Robotics /
Other Coaching
Experience
Both coaches and students had
no prior FLL competition
experience.
Daniel – 15 years' experience
coaching Science competitions.
Site 2
Registered Home-School
Claire (Former scientist and
business manager, now
educator)
Eleanor (Home-school educator)
3 boys, 4 girls. (1 team)
Age 9-15
(Equivalent Years 4-10)
Coaches and students
participating in their second year
of FLL competition.
Site 3
Catholic K-12
Kelly (Head of Learning
Technologies)
12 boys (2 teams)
Year 7 (n=11), Yr 8 (n=1)
Age 12-14
Coach – Four years prior FLL
coaching experience.
Students were participating in
their first year of FLL.

12. Four Emerging Themes
1. Coaches’ prior experience coaching or leading collaborative teams in educational
and professional contexts explicitly informs their instructional practice.
2. Coaches valued engineering notebooks as a tool for documenting FLL team
processes, learning, and decision making to support student accountability,
formative assessment, and tournament judging.
12

13. Emerging Themes (cont.)
3. Coaches support students’ learning
and skill development through their
use of guided questioning/
prompting, modelling, explicit
teaching, and engineering notebook
documentation routines.
4. Coaches sought to scaffold students’
verbal and written reflection on their
team processes and learning
activities using engineering notebook
prompt questions, graphic organisers,
team-building activities, and the
official FLL judging rubrics.
13
Excerpt from Site 1 Engineering Notebooks

14. 14
Site 1 coaches routinely left
feedback and prompt questions
in their teams’ EN’s.
One team was significantly more
reflective & better at
documentation than the other.
Site 1 Team 1
Team Building Task
21/10/2021

15. 15
Home-school Mission Strategy
Spreadsheet
Example Robot Game Table Set Up
Site 1 Secondary Science Classroom

16. 16
“So, I am a big proponent of graphic
organisers and teaching the students
how to use charts in order to structure
their thinking. And then they're able to
take what they put in those charts, and
kind of jazz it up a bit. And then they can
pull, when they go to actually create
their logbook [for judging], they have
something to go back on and refer to …”
Kelly, Site 3 Coach

17. 17
Where to from here?
• Contribute to the literature regarding the development of collaboration
and communication skills in integrated STEM and other project-based
learning contexts.
• Development and publication of an adaptable professional development
resource for new FLL coaches and teams.
• Opportunities for future research
• Use of digital tools for scaffolding and documentation
• Possible adaptation of notebooks for classroom STEM projects

18. 18
Questions and Feedback
My home-school “research assistant”!