1. UNDERSTANDING ACADEMIC ENGAGEMENT AS A
MULTIDIMENSIONAL AND DYNAMIC CONCEPT
Data analysis: methods and procedures
Aleksandra Lazareva
LET Master's Program
University of Oulu

2. Student engagement
● Defining student engagement: linked to interest,
emotions, motivation, and the theory of self-
regulated learning in general
● Multiple factors affecting student engagement
● Educational context: individual needs, school-level
factors, classroom context (autonomy support, task
characteristics, classroom structure, teacher
support, and peers)

3. Original direction of the study
How does use of technology in classroom affects
students' engagement?

4. Data collection
● Spring 2013 (second semester in the LET studies)
● 5 researchers
● Teacher training school in Northern Finland
● English language classroom
● Eleven 4th graders (in three groups) and their English
teacher
● Intervention: 1 month, 6 lessons (45 minutes)
● Video observations, learning diaries, students'
products

5. Methods of data analysis: on-task/off-
task and phase-shift analysis
● Context-sensitive and process-oriented methods to
study students' learning (Boekaerts & Corno, 2005;
Järvelä et al., 2001; Perry & VandeKamp, 2000).
● Dynamic assessment: the main point is to capture the
changes of an individual’s motivation in connection to
the contextual changes (Järvelä et al., 2001).
● On-task/off-task analysis method is the most basic one
to describe motivational characteristics of a learning
activity (Järvelä et al., 2001; Järvelä et al., 2008).

6. On-task/off-task analysis
● “On-task episode” is the phenomenon that implies
student approaching a task by “attending to the task or
presenting task-focused nonverbal signs” (Järvelä et al.,
2008, p. 306).
● “Off-task episode” means avoidance of a learning
activity, such as “turning away, approaching other
students for telling jokes or other substitute activities
instead of learning” (Järvelä et al., 2008, p. 306).
● My coding was based on the previous research and
own specific guidelines

7. On-task/off-task analysis
A student was considered to be on-task if he/she:
● was asking task-related questions from the teacher, indicating
involvement in the activity and readiness to begin;
● was regulating group’s behavior according to task instructions;
● was indicating involvement and willingness to work on the task by non-
verbal means (e.g., leaning towards the iPad and revising instructions
from the blackboard oor textbook);
● was providing task-related explanation or technological help to a peer.
A student student was considered to be off-task if he/she:
● was indicating loss of focus on the task by non-verbal behaviors (e.g.,
staring into the air and looking bored);
● was demonstrating disruptive behavior.

8. On-task/off-task and phase-shift
analysis
● I chose specific learning situations to be analyzed
(same for the three groups): activities with iPads (fun,
choice, meaningfulness)
● I translated and transcribed them
● I watched the selected video episodes and marked
each student's activity as on-task or off-task (10-
seconds interval)
● I counted the time spent on-task and off-task
● I counted the amount of shifts (coherence of the
activity)

9. On-task/off-task and phase-shift
analysis: Example

10. Summing up the results

11. Research questions: Inductive approach
● The video data were good for describing engagement
● At the same time while I was carrying out the analysis I was
reading more theoretical and research literature
● I realized that use of technology has to be taken into
account with multiple classroom factors
● While carrying out the on-task/off-task analysis I kept my
mind open about other aspects that I could explore
● I changed the focus from technological aspects only to
classroom factors in general. Technology became only one
sub-point

12. Research questions: inductive approach
The main aim of the study is to better understand the dynamics of
individual students' engagement in a real classroom context.
1. Did students perceive iPads as increasing the attractiveness of the
task, and how did it influence engagement?
2. How did students perceive the degree to which they were able to
make choice within the task, and how did it influence engagement?
3. How did meaningfulness of the task for an individual student
influence engagement?
4. How did different types of interactions with the teacher influence
engagement?
5. What strategies did students use to regulate each other's task
engagement, and how successful were they?

13. Classroom environment factors:
Descriptive analysis

14. Question 1: iPads and fun part of the
task
● Students thought iPad made tasks more fun
● iPad initiated engagement by causing positive
emotions and situational interest
– Antti: I will take this, I will take this...
– Aleksi: Hey, hey, hey, look who is drawing first, yeah, I
will draw first!
– Jussi: Then me, then me, Aleksi, give it to me after that.
● iPads were distracting for some of the students

15. Question 2: Autonomy support
● Tasks differed in the level of autonomy support
● Was difficult to perceive autonomy support for
students
● From video observations it was seen that students
were eager to make choices within the task
– Terhi (asking the teacher): With what program?
– Teacher: You can choose yourself with what program.
– Terhi (asking the teacher): Can I write to the fun part, for example,
sports?
– Teacher: Yes.

16. Question 3: Meaningfulness
● Tasks differed in the level of meaningfulness
– Anna: Stockholm is my favorite place in Sweden.

17. Question 4: Teacher support
● Three types of interactions with the teacher:
– provision of additional instructions (e.g., answering
students' questions about the task)
– regulation of students' behavior in the group (e.g., telling
individual students off)
– intensive assistance of an individual student or a pair (i.e.,
fulfilling a task with students step by step) (see Rogat &
Linnenbrink-Garcia, 2011)
● Different effect on students' engagement

18. Question 5: Peers
● Most part of students' attempts to regulate each other's
behavior were related to structuring the activity, mainly by
coordinating turn-taking and using support materials in a
group/pair:
– Petri: Close your books! Elina, close your book!
– Laura: You should let Elina draw. Next – to Elina...
● In case when own efforts to structure the activity in the
group were not sufficient, a student would draw the
teacher's attention in order for her to help:
– Ville: Aleksi is playing a fool...
– Anna: Antti doesn't begin to do...

19. Discussion
● Findings of this study go in line with previous
research findings
● The study was carried out in an authentic classroom
context
● Source of engagement and disengagement were
established
● Practical implications

20. Practical implications

21. References
● Boekaerts, M. & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment
and intervention. Applied Psychology: An International Review, 54(2), 199–231.
● Järvelä, S., Salonen, P., & Lepola, J. (2001). Dynamic assessment as a key to understanding student
motivation in a classroom context. In P. R. Pintrich & M. L. Maehr (Eds.), New directions in
measures and methods: Advances in motivation and achievement, Volume 12 (pp. 207–240).
Bingley: Emerald Group Publishing Limited.
● Järvelä, S., Veermans, M., & Leinonen, P. (2008). Investigating student engagement in computer-
supported inquiry: a process-oriented analysis. Social Psychology of Education, 11, 299–322.
● Perry, N. E. & VandeKamp, K. O. (2000). Creating classroom contexts that support young children's
development of self-regulated learning. International Journal of Educational Research, 7(33), 821–
843.
● Rogat, T. K. & Linnenbrink-Garcia, L. (2011). Socially shared regulation in collaborative groups: An
analysis of the interplay between quality of social regulation and group processes. Cognition and
Instruction, 29, 375–415.