Information behavior or learning behavior? Empirical experience with web analytics - presentation from Information interactions 2019. Information interaction 2019

1. Information behavior
or learning behavior?
Empirical experience
with web analytics
Michal Černý, KISK FF MU
Information interaction 2019

2. Educational environment
• The kind of information environment in which one learns.
• Can be online, offline, onlife.
• Often the educational environment is hybrid.
• We commonly study topics such as climate or classroom atmosphere
in education. But we don't study the whole environment. (Lakoff,
Johnson)
• Learning is a specific process of structuring and using information.
Learning is the process of changing the cognitive landscape of our
mind. Learning can be a form of information interaction.

3. Research sample
• Two courses:
• Digital competence (blended learning, mandatory course) – VIKBA14
• Digital Competence Development Course (Online, optional course) – KPI33
• Number of students (in the beginning):
• KPI33: 94
• VIKBA14: 52
• At the end: 108 students completed the courses.

4. Course description
• It is one common environment for
an online course.
• Students read the study materials
and complete the test and self-
evaluation at the end of the
semester.
• The course has continuous tasks
with a 14-day periodicity.

5. Research tools
• Google Analytics
• SmartLook
• Knowledge test
• Self-assessment tools

6. Google Analytics
• Avg. Time on page or visit
• Pageviews data
• User interest data
• Connection point data
• …

7. Smartlook
• Heatmaps
• Record user behavior on the site
• Analysis of page clicks
• Analysis of more complex
interactions (we have a small
sample for this)

8. Knowledge test
• Two test questions per competency (21
competencies in five domains) => 42
questions per test
• Some students had few points => and
repetition the test.

9. Self-assessment
• Through a range of 0..8 (I do not have competence… I am a field
changing expert).
• The self-assessment questionnaire copied DigComp 2.1.
• Students received a questionnaire in which they sorted competency
domains by importance.

10. Results

11. Educational Environment: read time per clicks
per visit
0:00
1:12
2:24
3:36
4:48
6:00
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
Timereading[min]
The number of clicks on the page visited

12. Educational Environment: The number of
clicks per page view
0
200
400
600
800
1000
1200
1400
0 100 200 300 400 500 600 700 800
Thenumberofpageviews
Clicks

13. The relationship between the number of clicks per
minute of reading and the number of clicks per
visit
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
0.50
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
Thenumberofclicksperminuteofreadingandvisit
The number of clicks per visit

14. Data from Google Analytics illustrate
nonlinear learning

15. Smartlook Heatmap shows clicks on one
selected page

16. How do students evaluate themselves as part
of their digital competencies?
TABLE 1 OF COMPETENCE STUDENTS KPI33 BY 8 (AND 0) LEVELS AND AVERAGE ON EACH COMPETENCE
COMPETENCE / LEVEL 0 1 2 3 4 5 6 7 8 AVERG.
BROWSING, SEARCHING AND FILTERING DATA, INFORMATION AND
DIGITAL CONTENT
0 1 6 9 17 8 19 4 0 4,5
EVALUATING DATA, INFORMATION AND DIGITAL CONTENT 0 1 3 6 21 10 17 5 1 4,8
MANAGING DATA, INFORMATION AND DIGITAL CONTENT 0 1 8 1 21 12 17 3 1 4,6
INTERACTING THROUGH DIGITAL TECHNOLOGIES 0 1 5 8 17 6 21 4 2 4,7
SHARING THROUGH DIGITAL TECHNOLOGIES 0 1 5 11 12 7 23 5 0 4,7
ENGAGING IN CITIZENSHIP THROUGH DIGITAL TECHNOLOGIES 3 2 12 25 6 2 11 3 0 3,5
COLLABORATING THROUGH DIGITAL TECHNOLOGIES 0 1 2 9 13 7 23 7 2 5,0
NETIQUETTE 1 0 7 16 15 8 13 4 0 4,2
MANAGING DIGITAL IDENTITY 1 1 8 11 26 3 10 4 0 4,0
DEVELOPING DIGITAL CONTENT 1 3 5 10 12 7 19 6 1 4,5
INTEGRATING AND RE-ELABORATING DIGITAL CONTENT 3 4 10 7 21 3 14 1 1 3,8
COPYRIGHT AND LICENCES 0 5 13 15 20 2 6 3 0 3,5
PROGRAMMING 14 19 9 4 6 3 4 3 2 2,3
PROTECTING DEVICES 0 7 10 17 15 7 7 1 0 3,5
PROTECTING PERSONAL DATA AND PRIVACY 1 8 7 18 12 2 12 3 1 3,7
PROTECTING HEALTH AND WELL-BEING 1 2 16 10 19 7 7 2 0 3,6
PROTECTING THE ENVIRONMENT 3 4 7 14 19 5 10 2 0 3,7
SOLVING TECHNICAL PROBLEMS 2 9 8 18 9 5 9 4 0 3,5
IDENTIFYING NEEDS AND TECHNOLOGICAL RESPONSES 1 4 16 14 14 3 9 3 0 3,5
CREATIVELY USING DIGITAL TECHNOLOGIES 0 6 18 16 7 7 6 4 0 3,4
IDENTIFYING DIGITAL COMPETENCE GAPS 1 9 11 16 17 5 5 0 0 3,2

17. How do students evaluate themselves as part
of their digital competencies?
TABLE 2 OF COMPETENCE STUDENTS VIKBA14 BY 8 (AND 0) LEVELS AND AVERAGE ON EACH COMPETENCE
COMPETENCE / LEVEL 0 1 2 3 4 5 6 7 8 AVERG
BROWSING, SEARCHING AND FILTERING DATA, INFORMATION
AND DIGITAL CONTENT 0 1 1 8 16 5 10 0 0 4.3
EVALUATING DATA, INFORMATION AND DIGITAL CONTENT
0 1 2 8 13 8 6 3 0 4.3
MANAGING DATA, INFORMATION AND DIGITAL CONTENT
0 0 4 8 13 4 9 3 0 4.4
INTERACTING THROUGH DIGITAL TECHNOLOGIES
0 1 3 8 12 8 7 2 0 4.3
SHARING THROUGH DIGITAL TECHNOLOGIES 1 0 6 11 6 10 3 4 0 4.0
ENGAGING IN CITIZENSHIP THROUGH DIGITAL TECHNOLOGIES
1 3 9 9 11 3 3 1 1 3.4
COLLABORATING THROUGH DIGITAL TECHNOLOGIES
0 1 4 4 16 6 7 2 1 4.4
NETIQUETTE 0 0 4 9 13 8 4 3 0 4.2
MANAGING DIGITAL IDENTITY 0 3 3 8 13 6 5 3 0 4.0
DEVELOPING DIGITAL CONTENT 0 1 4 11 10 5 5 5 0 4.2
INTEGRATING AND RE-ELABORATING DIGITAL CONTENT 0 0 12 10 11 3 3 2 0 3.5
COPYRIGHT AND LICENCES 0 2 7 9 16 2 4 1 0 3.6
PROGRAMMING 4 12 10 5 1 0 5 3 1 2.7
PROTECTING DEVICES 0 3 10 8 8 3 7 2 0 3.7
PROTECTING PERSONAL DATA AND PRIVACY 1 1 7 12 7 3 7 3 0 3.8
PROTECTING HEALTH AND WELL-BEING 2 3 6 8 15 3 5 0 0 3.5
PROTECTING THE ENVIRONMENT 0 1 9 7 16 6 2 0 0 3.6
SOLVING TECHNICAL PROBLEMS 0 7 11 9 5 3 2 3 1 3.2
IDENTIFYING NEEDS AND TECHNOLOGICAL RESPONSES
0 4 7 13 5 2 9 1 0 3.6
CREATIVELY USING DIGITAL TECHNOLOGIES 1 2 9 10 9 4 3 3 0 3.5
IDENTIFYING DIGITAL COMPETENCE GAPS 0 4 9 5 11 5 5 2 0 3.7

18. How does this assessment relate to the test
results?
TABLE 3. SELF-ASSESSMENT DATA
COMPETENCE AREAS CORRECTNESS [%] SELF-EVALUATION
VIKBA14
SELF-EVALUATION
KPI33
SELF-EVALUATION
AVRG.
INFORMATION AND DATA
LITERACY
65.9 4.3 4.6 4.5
COMMUNICATION AND
COLLABORATION
64.5 4.1 4.4 4.2
DIGITAL CONTENT CREATION 72.6 3.5 3.5 3.5
SAFETY 72.8 3.6 3.6 3.6
PROBLEM SOLVING 66.6 3.5 3.4 3.4

19. Is there a relationship between how they are
evaluated and what is their movement on the
course pages
• There is a -0.6 correlation between
the correct answers in the test and
the self-assessment, which can be
interpreted as not being able to
evaluate students adequately.
• In terms of self-learning behaviour
and web analytics, the -0.7
correlation is between self-
assessment and total time spent on
the page.
• The positive correlation between
time spent on the page and the
correct answer is didactically valuable
and is relatively strong (0.6).
TABLE 4. WEB ANALYTICS, SELF-ASSESSMENT AND STUDY RESULTS
COMPETENCE AREAS CORRECTNESS
[%]
SELF-EVALUATION
AVRG.
TIME
SPENT
WITH
DOMAIN
[S]
DOMAIN
INTERACTION
TIME
SPENT ON
THE PAGE
[S]
INTERACTION
VISIT
INFORMATION AND
DATA LITERACY
65.9 4.5 931 2 747 186.2 3.1
COMMUNICATION
AND COLLABORATION
64.5 4.2 1 466 1 782 183.3 3.5
DIGITAL CONTENT
CREATION
72.6 3.5 1 171 1 975 195.2 4.0
SAFETY 72.8 3.6 1 280 1 313 213.3 2.8
PROBLEM SOLVING 66.6 3.4 1 248 638.0 208 1.4

20. What topics are interested in on the site? Are they
reflected in other measurable parameters?
TABLE 5. SORTING DOMAINS BY IMPORTANCE. STUDENTS VIKBA14
VIKBA14 MAX. 2 3 4 MIN. SCORE
INFORMATION AND DATA
LITERACY
19 9 9 0 1
159
COMMUNICATION AND
COLLABORATION
3 2 8 19 7
92
DIGITAL CONTENT
CREATION
0 6 2 8 23
69
SAFETY 16 15 5 3 0 161
PROBLEM SOLVING 1 7 15 9 7 103
TABLE 6. SORTING DOMAINS BY IMPORTANCE. STUDENTS KPI33
VIKBA14 MAX. 2 3 4 MIN. SCORE
INFORMATION AND DATA
LITERACY
22 12 5 1 0
175
COMMUNICATION AND
COLLABORATION
3 4 7 17 9
95
DIGITAL CONTENT
CREATION
0 3 4 11 22
68
SAFETY 14 12 8 4 2 152
PROBLEM SOLVING 1 9 16 7 7 110

21. What topics are interested in on the site? Are they
reflected in other measurable parameters?
TABLE 7. SORTING DOMAINS OF IMPORTANCE. STANDARDIZED SCORE.
COMPLEX VIKBA14 KPI33
IMPORTANT
SCORE
INFORMATION AND DATA LITERACY
4.0 4.3 4.1
COMMUNICATION AND
COLLABORATION
2.3 2.3 2.3
DIGITAL CONTENT CREATION
1.7 1.7 1.7
SAFETY
4.0 3.7 3.9
PROBLEM SOLVING
2.6 2.7 2.6
• Values ​​of 0.43 reached the
relationship between self-assessment
and perceived domain importance,
while a negative correlation of mean
significance can be seen between
perceived usefulness and time spent
studying the domain (-0.48).
• There is only a weak correlation of
0,25 between site interaction and
perceived importance.
• Interestingly, there is no correlation
between the correctness of the
response and the perceived
usefulness (-0.1).

22. Conclusion
• In terms of learning behaviour and web analytics, the -0.7 correlation is
exciting, which is between self-assessment and total time spent on the page.
• In other words - the stronger the students in the area, the less time they spend
on the page, their content can be expected to be easier for them to adopt.
Conversely, the positive correlation between time spent on the page and the
correct answer is didactically valuable and is relatively strong (0.6).
• It may be interpreted to mean that the time spent studying a given dimension has
a positive effect on the test result, which mostly corresponds to its construction.
• Values ​​of 0.43 achieve a relationship between self-evaluation and perceived
domain importance, while a negative correlation of mean significance can be
seen between perceived importence and time spent studying the domain (-0.48).
• There is only a weak correlation of 0.25 between site interaction and perceived
importance. It can, therefore, be expected that there is a specific link between
perceived usefulness and interactions, albeit not strong.

23. Further research
• I plan to systematically focus on nonlinear learning behavior and
description of its models from data obtained from Google Analytics
and Smartlook (analysis using tools from video analysis).
• I will plan to measure differences between different courses (Is the
behavior the same everywhere? My two courses are in the same
environment and usually have very similar characteristics…)
• Action research support - improving individual courses based on data.

24. Thank you for your attention