Slides from our workshp at National Institute of Education in Singapore from May 10, 2018

1. Introduction to
Epistemic Network
Analysis
Vitomir Kovanovic,
University of South Australia
#vkovanovic
Vitomir.Kovanovic@unisa.edu.au
1

2. What is Epistemic Network
Analysis (ENA)
• Epistemic Network Analysis is a network-based method for
analysing codified data.
• Developed by Professor David Shaffer from the University of
Wisconsin Madison (UWM) and his team.
• There is a web interface and R package
• http://epistemicnetwork.org
2

3. The original problem of ENA
• Understand how people become professionals
• Involves understanding of the ways important concepts –codes–
interact together
• The applications of ENA expanded far beyond epistemology domain
• New term: Quantitative ethnography.
• Can be used to understand how different codes co-occur.
3

4. What is Epistemology?
Epistemology studies the nature of
knowledge, justification, and the
rationality of belief
New term: quantitative ethnography
4

5. Video
https://www.youtube.com/watch?v=wrTiXNIeHZA
5

6. ENA in Education
• Often used for understanding of student conversations and
discussion messages.
• Also used for analysis of interview data.
6

7. Key concepts in ENA
• Codes: a set of concepts whose interactions we want to understand
• Unit of analysis: objects for which we want to understand
interactions between the codes
• Stanza(Conversation): Units in which we measure code co-
occurrence
7

8. How ENA works: Example
dataset
8
• Units of analysis: Individual Students
• Stanzas: Individual messages

9. How ENA works: Example
dataset
9
• Codes:
• Data
• Technical Constraints
• Performance Parameters
• Client and Consultant Requests
• Design Reasoning
• Collaboration

10. How ENA works: Code co-
occurrence matrix
10
• Code co-occurrence in stanzas is used to produce code co-occurrence
matrix for each unit of analysis (i.e., person)
Data Technical
Constraints
Performance
Parameters
Client and
Consultant
Requests
Design
Reasoning
Collaboration
Data / 120 80 323 52 32
Technical
Constraints
/ 23 120 112 32
Performance
Parameters
/ 17 28 152
Client and
Consultant
Requests
/ 21 68
Design
Reasoning
/ 12
Collaboration /

11. How ENA works: Code co-
occurrence matrix
11
• Code co-occurrence in stanzas is used to produce code co-occurrence
matrix for each unit of analysis (i.e., person)
C1 C2 C3 C4 C5 C6
C1 / 120 80 323 52 32
C2 / 23 120 112 32
C3 / 17 28 152
C4 / 21 68
C5 / 12
C6 /

12. How ENA works: Code co-
occurrence matrix
12
• Code co-occurrence in stanzas is used to produce code co-occurrence
matrix for each unit of analysis (i.e., person)
C1 C2 C3 C4 C5 C6
C1 / 120 80 323 52 32
C2 / 23 120 112 32
C3 / 17 28 152
C4 / 21 68
C5 / 12
C6 /

13. How ENA works: Matrix to
vector
13
• Co-occurrence matrices are “flattened out” into vectors of N(N-1)/2 elements
6*5/2=15 columns (dimensions)
C1-
C2
C1-
C3
C1-
C4
C1-
C5
C1-
C6
C2-
C3
C2-
C4
C2-
C5
C2-
C6
C3-
C4
C3-
C5
C3-
C6
C4-
C5
C4-
C6
C5-
C6
U1 120 80 323 52 32 23 120 112 32 17 28 152 21 68 12
C1 C2 C3 C4 C5 C6
C1 / 120 80 323 52 32
C2 / 23 120 112 32
C3 / 17 28 152
C4 / 21 68
C5 / 12
C6 /

14. How ENA works: Matrix to
vector
14
• Co-occurrence matrices are “flattened out” into vectors of N(N-1)/2 elements
6*5/2=15 columns (dimensions)
C1-
C2
C1-
C3
C1-
C4
C1-
C5
C1-
C6
C2-
C3
C2-
C4
C2-
C5
C2-
C6
C3-
C4
C3-
C5
C3-
C6
C4-
C5
C4-
C6
C5-
C6
U1 120 80 323 52 32 23 120 112 32 17 28 152 21 68 12
C1 C2 C3 C4 C5 C6
C1 / 120 80 323 52 32
C2 / 23 120 112 32
C3 / 17 28 152
C4 / 21 68
C5 / 12
C6 /

15. How ENA works: Matrix to
vector
15
• Co-occurrence matrices are “flattened out” into vectors of N(N-1)/2 elements
6*5/2=15 columns (dimensions)
C1-
C2
C1-
C3
C1-
C4
C1-
C5
C1-
C6
C2-
C3
C2-
C4
C2-
C5
C2-
C6
C3-
C4
C3-
C5
C3-
C6
C4-
C5
C4-
C6
C5-
C6
U1 120 80 323 52 32 23 120 112 32 17 28 152 21 68 12
C1 C2 C3 C4 C5 C6
C1 / 120 80 323 52 32
C2 / 23 120 112 32
C3 / 17 28 152
C4 / 21 68
C5 / 12
C6 /

16. How ENA works: Matrix to
vector
16
• Co-occurrence matrices are “flattened out” into vectors of N(N-1)/2 elements
6*5/2=15 columns (dimensions)
C1-
C2
C1-
C3
C1-
C4
C1-
C5
C1-
C6
C2-
C3
C2-
C4
C2-
C5
C2-
C6
C3-
C4
C3-
C5
C3-
C6
C4-
C5
C4-
C6
C5-
C6
U1 120 80 323 52 32 23 120 112 32 17 28 152 21 68 12
C1 C2 C3 C4 C5 C6
C1 / 120 80 323 52 32
C2 / 23 120 112 32
C3 / 17 28 152
C4 / 21 68
C5 / 12
C6 /

17. How ENA works: Matrix to
vector
17
• Co-occurrence matrices are “flattened out” into vectors of N(N-1)/2 elements
6*5/2=15 columns (dimensions)
C1-
C2
C1-
C3
C1-
C4
C1-
C5
C1-
C6
C2-
C3
C2-
C4
C2-
C5
C2-
C6
C3-
C4
C3-
C5
C3-
C6
C4-
C5
C4-
C6
C5-
C6
U1 120 80 323 52 32 23 120 112 32 17 28 152 21 68 12
C1 C2 C3 C4 C5 C6
C1 / 120 80 323 52 32
C2 / 23 120 112 32
C3 / 17 28 152
C4 / 21 68
C5 / 12
C6 /

18. How ENA works: Matrix to
vector
18
• Co-occurrence matrices are “flattened out” into vectors of N(N-1)/2 elements
6*5/2=15 columns (dimensions)
C1-
C2
C1-
C3
C1-
C4
C1-
C5
C1-
C6
C2-
C3
C2-
C4
C2-
C5
C2-
C6
C3-
C4
C3-
C5
C3-
C6
C4-
C5
C4-
C6
C5-
C6
U1 120 80 323 52 32 23 120 112 32 17 28 152 21 68 12
C1 C2 C3 C4 C5 C6
C1 / 120 80 323 52 32
C2 / 23 120 112 32
C3 / 17 28 152
C4 / 21 68
C5 / 12
C6 /

19. How ENA works: Matrix to
vector
19
• Co-occurrence matrices are converted to vectors and joined together
to form Analytic space of N*(N-1)/2 elements
C1-
C2
C1-
C3
C1-
C4
C1-
C5
C1-
C6
C2-
C3
C2-
C4
C2-
C5
C2-
C6
C3-
C4
C3-
C5
C3-
C6
C4-
C5
C4-
C6
C5-
C6
U1 120 80 323 52 32 23 120 112 32 17 28 152 21 68 12
U2
U3
…
…
U20 32 125 112 52 32 17 128 211 54 36 63 85 109 276 42
• NOTE: Each vector is a point in a 15-dimensional space
• EACH GRAPH IS A POINT

20. How ENA works: Matrix to
vector
20
• Co-occurrence matrices are converted to vectors and joined together
to form Analytic space of N*(N-1)/2 elements
1-2 1-3 1-4 1-5 1-6 2-3 2-4 2-5 2-6 3-4 3-5 3-6 4-5 4-6 5-6
U1 120 80 323 52 32 23 120 112 32 17 28 152 21 68 12
U2
U3
…
…
U20 32 125 112 52 32 17 128 211 54 36 63 85 109 276 42
• NOTE: Each vector is a point in a 15-dimensional space
• EACH GRAPH IS A POINT

21. How ENA works: Singular Value
Decomposition of Analytic Space
21
• Approximate N columns with a smaller number R of “composite columns”
• The whole point is to be able to plot N dimensions on a 2D plot

22. How ENA works: Singular Value
Decomposition of Analytic Space
22
• Approximate N columns with a smaller number R of “composite columns”
• The whole point is to be able to plot N dimensions on a 2D plot
m=1,000 students
n=100 edges
A=100,000
U = 1,000 x 1,000 = 1,000,000
VT= 100 x 100 = 10,000

23. How ENA works: Singular Value
Decomposition of Analytic Space
23
• Approximate N columns with a smaller number R of “composite columns”
• The whole point is to be able to plot N dimensions on a 2D plot
m=1,000 students
n=100 edges
A=100,000

24. How ENA works: Singular Value
Decomposition of Analytic Space
24
• Approximate N columns with a smaller number R of “composite columns”
• The whole point is to be able to plot N dimensions on a 2D plot
m=1,000 students
n=100 edges
A=100,000
r=2 (keep top two singular values)
U=1,000 x 2 = 2,000
VT=2 x 100 = 200
Total=2,200

25. How ENA works: Singular Value
Decomposition of Analytic Space
25
• Approximate N columns with a smaller number R of “composite columns”
• The whole point is to be able to plot N dimensions on a 2D plot
m=1,000 students
n=100 edges
A=100,000
r=2 (singular values)
U=1,000 x 2 = 2,000
VT=2 x 100 = 200
Total=2,200
Latent factor scores (student 2D coordinates)
Latent factor coefficients
(code pair 2D coordinates)

26. Visualising SVD’ed Analytic
Space: Projection space
26

27. Visualising individual graphs:
ENA network models
27

28. Visualising individual graphs:
ENA network models
28

29. Visualising individual graphs:
ENA network models
29

30. Remarks on coding
• Code values can be
 Boolean: 0 if code does not occurs, 1 if it does
 Integer: 0 if code does not occur, N if it does N times
 Fractional number: Value indicating “strength” or “association” of the code to the text
• In case of binary values, co-occurrence is 1 if both codes occur
• In case or integer or fractional numbers, co-occurrence score is the product if the
individual scores.
• Fractions useful for:
 LDA topic modelling:
 Each topic is a code, code value are topic associations to individual texts
• Integers useful for:
 Word count analysis:
 Each word (category) is a code, co-occurrence value is the product of code scores.
30

31. Moving stanza
• Stanza can be moving,
specially useful for
conversations where
individual messages
are too short
31

32. ENA Example 1: CoI + LDA
E Ferreira, R., Kovanovic, V., Gasevic, D., & Rolim, V. (2018). Towards
Combined Network and Text Analytics of Student Discourse in Online
Discussions. In The 19th International Conference on Artificial
Intelligence in Education. London, UK.
• Understand the development of cognitive presence with respect to
different course topics
 CoI process model, does not pay attention to course content
• Examine the role of instructional intervention of role assignment
32

33. ENA Example 1: CoI + LDA
• 1,747 messages from 6 course offers
• Each message coded for the level of cognitive presence:
 Triggering Event
 Exploration
 Integration
 Resolution
 Other
• Applied topic modelling to pick course topics
 Extracted topics were corresponding to course topics
 + one topic regarding logistics
33

34. Results: Projection graph all
students
34

35. Results: Projection graph
(intervention + control groups)
35
Control
Intervention

36. Results: ENA network model for
all students
36All students

37. Results: ENA network model for
two student groups
37Control Intervention

38. Social-Epistemic Network
Signature

39. ENA clusters

40. Topics per cluster
Cluster 1 Cluster 2
Cluster 3
Cluster 4

41. Social centrality per ENA cluster

42. Recap
• ENA works on codified data
• We need to define
 Codes
 Units
 Stanzas
• Unit’s co-occurrence matrices are converted to vectors
• All unit’s merged to form Analytic space matrix
• Analytic space is reduced to 2D with SVD
• Plot units on the 2D plot
• Plot codes on the 2D plot
42

43. Practical example
• Data: download from http://bit.ly/enanie
• Go to http://epistemicnetwork.org and create account
43