This document summarizes a presentation on developing a unified framework for analyzing online and offline learning interactions. The presentation covers the researcher's background and motivation for studying this topic. It then outlines the presentation structure, which includes reviewing the state of the art in learning analytics and interactions, defining relevant concepts and research questions, and discussing the methodology and future directions of developing a framework for analyzing different units of learning interactions across online and offline contexts. The focus is on establishing an appropriate unit of analysis to gain insights into learning patterns from both online and offline data sources.

1. UNIFIED FRAMEWORK FOR
ONLINE AND OFFLINE
LEARNING ANALYTICS
MAKA ERADZE
PHD STUDENT, SCHOOL OF DIGITAL TECHNOLOGIES,
RESEARCH FELLOW AT UNIVERSITA NAPOLI
FEDERICO II
RESEARCH SEMINAR, 11.05.2016

2. PRESENTER
• Educational background – humanities (teacher of
languages and literatures) - from modern and ancient
Greek to interdisciplinary TEL research
• Experience – 2005 to 2012 – software localizer—>teacher
network coordinator (Deer Leap Foundation, MoES of
Georgia), eLearning coordinator –(Tbilisi State University,
Georgia)
• Current job – Università di Napoli Federico II, EMMA
project, research fellow
• Research - Unified framework for online and offline
learning analytics

3. MOTIVATION FOR THE
RESEARCH
Personal
• Personal experience through participation into online fora
(yes, Latin plural for forum - fora J )
• Personal ethnographic observations on my informal learning
patterns led me to Learning Analytics research
• How? Ask Mart!
Professional:
• Interactions and social dimension of learning leading to big
questions like:
• what exactly are we analysing
• What is my research question?
• How are we going to break the concepts down to analyzable
units?

4. STRUCTURE OF THE
PRESENTATION
• Background and statement of the problem
• State of the art
• General methodological framework of the research
• Literature review
• Work already done
• Towards a framework - Mind map and diagram
• Future research - validation phase
• Applicability
• Discussion part

5. BACKGROUND AND
STATEMENT OF THE
PROBLEM
• The need for this kind of research emerges from the
idea that
• When dealing with data (especially big), we have to
think about our questions first and search for patterns
in learning data with specific questions and
frameworks in mind. For this, we need to shape our
philosophical and methodological standing.
• We need to think also about the real world
interactions and think about the ways to facilitate and
connect learning interaction analytics.
• This research aims at the development of a unified
model for offline and online analytics with a purpose
of gaining insight into learning interactions.

6. BACKGROUND AND
STATEMENT OF THE
PROBLEM
• It is focused on the development of a methodological
framework for the analysis of offline and online interaction
patterns and open learning environments.
• Centered on the idea of the unit of analysis and its
appropriateness for learning analytics research and practice,
it can be applicable to any kind of interaction analysis
(learning analytics).
• How is the issue of the units of analysis presented in different
disciplines (and learning analytics?)
• What are the philosophical concepts and considerations
behind the issue of the unit of analysis?
• How can we make use of xAPI statements and frame this
discourse in a philosophically and methodologically
appropriate way?

7. RESEARCH
QUESTIONS
• How can we analyze learning interactions using different
frameworks, theories, philosophies (and pedagogic
scenarios)?
• What is the appropriate unit of analysis for learning
interaction analytics in online and offline settings?

8. GENERAL THEORETICAL AND
METHODOLOGICAL FRAMEWORK
OF THE RESEARCH
Theoretically
• Constructivist approach
• This research does not use particular theoretical framework. It
rather reviews interdisciplinary approaches and creates a
meta language to redescribe the methodological framework
for learning analytics research
Methodologically
• Design based research
• Mainly based on the extensive literature review and inquiry
stage
• Based on the literature review artifacts will be created that
redescribe and place the issue of the unit of analysis within
the LA research
• This will serve to create a detailed ontology and validate it

9. STATE OF THE ART -
INTERACTIONS
• In order to actually define how can we make use of interaction
traces and data and locate its analytical framework, it is
necessary to define the concept of interaction as it is to
educational sciences - TEL.
• Interaction functions in a different way in virtual learning
environments where its must convey, mimic or recreate the
traditional communication patterns or build a new type of
learning ambience.
• Underlining the term - learning interaction. I share Wagner’
definition of interaction as follows “reciprocal events that
require at least two objects and two actions. Interactions occur
when these objects and events mutually influence each
other” (Wagner, 1994).

10. STATE OF THE ART -
INTERACTIONS
• According to Anderson (Anderson 2003), interaction online
(or offline) can happen human and inanimate actors.
• Moore’s theory of Three Types of Interaction
• To Moor learner-instructor interaction is a four-stage support
of the learner:
1. Designing the content, maintaining motivation, self-direction
2. Making presentations (student/teacher)
3. Practicing and applying acquired competences.
4. Counseling and encouragement of learners according their
levels of progress. The third type of interaction – learner-
learner for Moor is a new dimension in distance education

11. STATE OF THE ART-
DEFINITIONS
• Learning analytics is the measurement, collection,
analysis and reporting of data about learners and
their contexts, for purposes of understanding and
optimizing learning and the environments in which it
occurs.
• LA as a technology-enhanced learning (TEL)
research area that focuses on the development of
methods for analyzing and detecting patterns within
data collected from educational settings, and
leverages those methods to support the learning
experience. (Chatti et al 2015)

12. STATE OF THE ART- LA, ITS
CHALLENGES AND FUTURE
DIRECTIONS
Learning analytics is inherently
interdisciplinary
• Operates on the verge of many different
fields
Still in its infancy
• It still has to establish itself as field with its
own traditions and theoretical frameworks

13. • Two systematic literature reviews found:
• Nistor et al(2015): The main issue with LA research is
the frequent lack of an explicit theoretical framework
from educational perspective. The authors call on the
educational and psychological theories for significant
progress of upcoming LA research
• Gasevic et al(2015): researchers from educational
sciences tend to publish more mature research and
show more rigor
• Fortunately, the field is evolving quickly and one measure
of this growth is the increasing breadth of methodologies
being applied to learning‐related data sources (Pardo et al,
2014)

14. • Big learning analytics
t h a t c o m e f r o m
heterogeneous sources
• o p e n l e a r n i n g
environments (MOOCs),
with data coming from
outside of platform.
• Mobile learning analytics
• Context modeling
• Privacy-aware analytics.
• Personalized learning analytics.
• Life-long learner modeling
• Learning analytics for open
assessment.
• Embedded Learning analytics
• Learning analytics design
patterns.
• Learning analytics evaluation.
Chatti, M. A., Lukarov, V., Thüs, H., Muslim, A., Yousef, A. M. F., Wahid, U., ... &
Schroeder, U. (2014). Learning Analytics: Challenges and Future Research Directions.

15. FACTORS
Different factors are involved in LA
Greller and Draschler

16. From Chatti

17. LIT.REVIEW -UNIT OF
ANALYSIS
• Interdisciplinary approach:
• When choosing what to analyze, how to analysis, what to take
into account, how to operationalize questions, what
methodology to use, what discourse to take in in any field, unit
of analysis is at the same time a problematic but also an
important decision to make.
• Learning Interaction Analysis needs its own unit of analysis
too.
• influenced by the work of Matusov who has given an
extensive account on this issue from different perspectives.
• Building on his understanding on the unit of analysis we will
also bring arguments from:
• Socio-cultural research, education, human-computer
interaction, computer science, sociology, technology-
enhanced learning, psychology and other fields.

18. LIT.REVIEW -UNIT OF
ANALYSIS
• This is not a methodological decision only, but also a
political, and of course, philosophical.
• Mostly, two types of approaches are used for the
discussion on the unit of analysis: holistic and
reductionist.
• Holistic approach believes that the unit has to express a
whole phenomenon, while reductionist approach breaks the
unit of analysis down to its most fine grain unit.
• Matusov thinks, that the unit of analysis changes with the
aim of the research.
• To Vigotsky it is a “self-contained” “unified smallest
whole”.

19. UNIT OF ANALYSIS
• Matusov takes the socio-cultural approach to the issue
and bases his discussion mainly on psychologist
tradition.
• According to Matusov one must be careful when
searching for a general, universal unit of analysis,
because it always is shaped by the purpose of the
researcher and the subject of study.
• The construct “unit of analysis” is mainly used for the
methodological critique of others’ research

20. UNIT OF ANALYSIS
The discussion about the unit of analysis is also a
paradigmatic discussion – exact sciences vs social sciences
with
• “The exact sciences constitute a monologic form of
knowledge: the intellect contemplates a thing and expounds
upon it. There is only one subject here-cognizing
(contemplating) and speaking (expounding). In opposition to
the subject there is only a voiceless thing. Any object of
knowledge (including man) can be perceived and cognized as
a thing. But a subject as such cannot be perceived and
studied as a thing, for as a subject it cannot, while remaining
a subject, become voiceless, and, consequently, cognition of it
can only be dialogic.” (Bakhtin in Matusov)

22. UNIT OF ANALYSIS
• According to Lefstein, while replacing the reductive units of
analysis with more holistic ones, the issue of manageability
arises.
• Sometimes it can be hard to draw the boundaries, what is the
exact phenomenon to be expressed holistically.
He argues, that “interactional structures of three or more turns
is a methodologically expedient and theoretically sound unit for
the analysis of classroom discourse”.
• To him, unit of analysis is a “discourse move” and dialogical.
Though, Lefstein also suggests that aside from the appropriate
phenomenon, the unit of analysis has to be expanded to
sequence.

23. UNIT OF ANALYSIS
Stahl from his side discusses the issue of unit of analysis in cognition –
• concepts (Plato),
• mental and material objects (Descartes) (and relationship between them)
• observable physical objects (empiricism),
• mind’s structuring categorization efforts (Kant).
All of the approaches dealt with the inner functions of an individual mind.
• Hegel entered the discussion with a larger unit of analysis – historically, socially and
culturally determined.
• Marx views the main unit of analysis as social movements, class conflicts and
transformations of economic systems.
• Hegel’s philosophy shaped three mainstream schools of thought – Marx (critical
social theory), Heidegger (existential phenomenology) and Wittgenstein (linguistic
analysis).
• To Stahl, these three main directions influence how the CSCL units of analysis are
formed: For Heidegger the unit of analysis was the man with unified experience of
being-in-the-world.
• Wingenstein’s shifted unit of analysis from mental meanings to interpersonal
communications in the context of getting something done together. (Stahl)

24. UNIT OF ANALYSIS -
LATOUR
• Latour thinks the issue brings us to
• what phenomenon is worth studying and ‘worth’ for what.
According to a sociocultural framework, this question cannot
be addressed within purely methodological considerations as
it both involves conceptual (paradigmatic) debates within the
academic field and is shaped by practical needs and financial
(and political and institutional) pressure outside of the field
and science in general (Latour, 1987 in Matusov).
• For Latour and his Action-Network Theory, the main unit of
analysis is embedded in the relationship between the
actions.

25. UNIT OF ANALYSIS
• Stahl believes that Engestrom is the one taking the unit of analysis to the whole
activity system.
• Engeström in his “Learning By Expanding” writes that the insertion of cultural
artifacts into human actions was revolutionary in that the basic unit of analysis now
overcame the split between the Cartesian individual and the untouchable societal
structure.” After this the understanding of an individual could no longer happen
without his or her cultural means; and the understanding of a society was bound to
individuals who use and produce artifacts. He believes that the limitation of the first
generation was that the unit of analysis remained individually focused. The object-
oriented and artifact-mediated collective activity system is the prime unit of analysis
in cultural-historical studies of human conduct;
• But to Stahl’s understanding Engestrom’s theory is not interested in group
knowledge building but rather with organizational management of the group,
Influenced by Marx, theory tries to see societal issues in the making.

26. UNIT OF ANALYSIS
• Stahl believes that even in distributed cognition, which
deals with group-cognitive phenomena, mostly socio-
technical systems and highly developed artifacts are
analyzed.
• Suthers (2010) with his uptake framework proposes that
the event is the core if we want analyze data and
understand which interactions lead to learning. The
uptake framework suggests that interaction is relational
by its nature, so the most important unit of analysis is not
isolated acts, but rather relationships (based, again, on
Latour 2005) between acts.

27. UNIT OF ANALYSIS -
KAPTELININ
In activity-theory
• activity, in a broad sense, is an interaction of the actor (e.g., a
human being) with the world. The interaction, according to
activity theory terminology, is described as a process relating
the subject (S) and the object (O). A common way to
represent activity is “S ⇔ O.
• According to Kaptelinin, activity is different from other types of
interaction by two key aspects: (a) subjects of activities have
needs, which should be met through an interaction with the
world, and (b) activities and their subjects mutually determine
one another; or, more generally, activities are generative
forces that transform both subjects and objects. Subjects
have needs.

28. KAPTELININ –
ACTIVITY THEORY
• Kaptelinin presents the Activity as
• a “unit of life” of a material subject existing in the
objective world.
• Subjects have their own needs and to survive, have to
engage in activities, that is, interact with the worldy
objects fulfill their needs
• assumptions of the social nature of human mind and
inseparability of human mind and activity. (Kaptelinin, Activity
theory).
• Activity is of fundamental importance to psychology
because of its special function.
• It places the Subject into the objective reality and transforms
the reality into a form of subjectivity (Leontiev, 1978)

29. PAAVOLA AND
HAIKAINEN
• According to Paavola and Haikaninen, in trialogues, the
interaction through “shared objects” that are in the
process of being developed is emphasized.
• The objects of inquiry can be respresented by the
knowledge artefacts, practices, ideas, models,
representations, etc. but perceived as something to be
collaboratively developed.
• These objects are important in the interaction and it is an
interaction between subjects, other subjects and
“objects” but not between the subjects.
• This means that the things are being developed
collaboratively and they do not repeat the existing
knowledge

30. XAPI
Event-driven analytics
<Actor (learner)> <verb> <object>, with <result>, in <context>

31. REVIEW: XAPI
● The Experience API is a service that allows for statements of
experience (typically learning experiences, but could be any
experience) to be delivered to and stored securely in a Learning
Record Store
● The Experience API is dependent on Learning Activity Providers
to create and track learning.
● Learning Activity Provider is a software object that
communicates with the LRS to record information about the
learning experience.
● Learning activity is a unit of instruction, experience or
performance that has to be tracked.
● A Statement consists of <Actor (learner)> <verb> <object>, with
<result>, in <context> to track an aspect of a learning
experience.

32. REVIEW: XAPI
● It repeats the structure of the sentence in practically any
language and is human readable
• Actor data is a unique information that describes a
specific subject, such as a student or group of students.
• Verb data classifies the type of activity the actor
participated in and often links to a human readable
description of the event.
• Object data will link to an artifact that is typically a
byproduct of or related to the activity

33. XAPI
● It is much more, than just a simple sentence-like structure. The
context is very important and it makes possible to relate one
activity to another*
• Quoting Verbert, Suthers and Rosen, Kevan and Ryan believe that
the issue of variety of educational data taxonomies and also the
distributed learning events collection challenges can be solved by
xAPI*
● Some researchers believe that it is aligned with constructivist
approach and activity theory
● xAPI has a lot of potential for The xAPI offers a renewed
opportunity to research, develop, and explore theories involving
learning beyond academia’s digital environments.
Kevan, J. M., & Ryan, P. R. Experience API: Flexible, Decentralized and Activity-Centric Data Collection.Technology,
Knowledge and Learning, 1-7.

34. WORK ALREADY
DONE/PUBLICATIONS
• Eradze, M. Kai, P., Laanpere, M.: (2013) Analysing Learning Flows in
Digital Learning Ecosystems, KMEL 2013, ICWL2013, Taiwan,
Springer (LNCS), October 2013
• The first paper that opens the discussion on the unit of analysis, with
detailed background on the interactions, literature review
• Eradze M., Laanpere M.: (2014) Interrelation Between Pedagogical
Design and Learning Interaction Patterns in Different Virtual
Learning Environments, HCII 2014, Greece, Springer (LNCS) June
2014
• Three different platforms studied with CoI framework and coding template
(Two MOOC platforms, Moodle and Edufeedr)
• Eradze, M., Valjataga, T., Laanpere M.: (2014) Observing the use of e-
textbooks in the classroom: towards “Offline” Learning Analytics,
FeT workshop, ICWL 2014, Tallinn, Estonia, Springer (LNCS)
• Observations in the classroom, tried to document offline interactions with
an mobile app and xAPI type of statements.

35. PUBLICATION IN
PROGRESS
Unit of Analysis in Learning Analytics, Maka Eradze, Mart
Laanpere
• The article overviews the state of the art of unit of analysis
issue in educational research, learning analytics and
offers a conceptual framework based on the idea of an
appropriate unit of analysis for learning interaction
analysis.
• Aiming for - Journal of Computing in Higher Education -
Special Issue on Advancing Research on Open
Education.

36. MIND MAP –
REDESCRIBING THE
UNIT OF ANALYSIS/XAPI

37. DIAGRAM -
REDESCRIBING THE
UNIT OF ANALYSIS/XAPI

38. FUTURE RESEARCH -
VALIDATION PHASE
• Based on the inquiry stage and the artefacts, a detailed
ontology will be developed
• A single semantic structure that can map teacher
intentions – what they are planning to do and what
actually is taking place
• The ontology will be mapped to specific scenarios
• Together with teachers validate the ontology with
participatory approach

39. APPLICABILITY
• This ontology will be applicable to any context and to any
data
• It will be flexible and can be applied to different
pedagogic scenarios, philosophies, methodologies
• With this ontology we will be able to grasp the whole
structures of interactions and analyse it holistically

40. I know this is not yet a defense and am not yet on Ithaca and I am no
Odysseus (rather Penelope), but this is a journey I am enjoying so:

41. EXPRESSION OF
GRATITUDE
• To all in the School of Digital Technologies for
understanding my absent presence or rather present
absence
• To Mart for being a challenging supervisor and….for
scaffolding/challenging/believing in me J
• To Kai for giving valuable, critical and helpful feedback

42. DISCUSSION
Need feedback J
• Desperately!
• For thoughts and feedback coming up later: maka@tlu.ee