The document summarizes a digital learning project on risk-based food import inspections. It describes: 1) The project goals of developing online training on risk-based food import inspections for food inspectors. 2) The design process including storyboarding, expert reviews, usability testing, and evaluating the learning experience. 3) Key results showing the design was effective at improving knowledge but needed optimization for efficiency and advanced learners. Further iterations were planned to enhance the learning experience design.

1. Digital Learning
Professor Dr. Isa Jahnke
from the Lens of Learning Experience Design
Institute of Educational Sciences

2. Road Map
Stage 1
Introduction
Stage 2
Theoretical Lens
Stage 4
Conclusion & Next steps
Stage 3
Projects,
Examples
Prof. Dr. Isa Jahnke 2

3. About me
Professor
• Dortmund / Germany (2008-2011)
• Umeå / Sweden (2011-2015)
• Missouri / U.S.A. (2015-2021)
• Münster / Germany (2021-active)
I taught
my first blended learning seminar: 2001
my first online course: 2016
My research interests:
• teaching and learning with digital technologies
• sociotechnical design

4. … they change, how we
communicate, express, share, network
receive information, connect
collaborate, … and learn.
Digital technologies (or ICT) change
the ways of human interaction …
e.g., Jahnke & Koch, 200
9
“First we shape the tools, and then the tools shape us”
McLuhan, 1967
By now, we know

5. Classroom
/ Course
Classroom /
Course
Digital-enhanced classroom:
Physical and online spaces are
merging
We go to college/university
because of
getting access to learning processes
Twitter, FB,
GroupApps, …
Interactive/Live
Broadcasting, …
Websites,
Blogs, …
and
more
Traditional classroom:
Separation
We went to college/univ.
because of
getting access to information
CrossActionSpaces
Jahnke, 2015
Prof. Dr. Isa Jahnke 5

6. Physical, online spaces, and digital technologies are merging
into new spaces, where human action materializes:
co-expanded, co-located communication spaces
CrossActionSpaces
Published in
Jahnke, 2015

7. Human interaction  crossaction
Cross-action
(crossing interaction)
• humans connect to each others’ resources
• no clear difference between offline and online (Floridi)
• a person is at different places at the same time
• mix of bots/technologies and humans
• fusion of formal and informal spaces
• Tension of technology openness and (social) limitations,
constrains or design trade-offs
Under these conditions of CrossActionSpaces,
how can we design for
(student) learning experiences?
Artist: Ralf Jahnke-Wachholz

8. Stage 2
Theoretical Lens
Prof. Dr. Isa Jahnke 8

9. Active learning is an ”umbrella” term that refers to a group of instructional
strategies that facilitate learners to actively engage with course
material, other students, learning process, e.g., discussions, problem-
solving, project-based learning, …
Activity-based model of instruction
Premise: Students don’t learn because the instructor does some activity,
but students learn through their own activity
What is it not?
contrast to "traditional" methods, in which the learners are passive
e.g., lecture
Context
Active learning has many different facets and can be applied in all disciplines
and study programs
Central Component:
Active Learning
Chi, 2009
Hodges, 2018
Prof. Dr. Isa Jahnke 9

10. Why active learning?
• it increases positive learning outcomes,
• student performance (better grades, deeper knowledge),
• students learn more than in traditional lectures
Challenges:
a) Resistance from instructors and learners
b) Misperception: actual learning vs. feeling of learning
Freeman et al.. 2014
Vetter et al., 2020
Theobald et al., 2020
Deslauriers et al., 2019
Prof. Dr. Isa Jahnke 10

11. a) Passiv c) Active
Student-centered
b) Active
Teacher-centered
Recorded lecture,
Zoom-lecture
• Students create artifacts with
technologies (mindtools) to
showcase learning
• Students have control over
the learning process,
• “Producers“ (digital making)
Examples
Bloom‘s lower
order thinking
skills (e.g.,
understanding)
ZOOM-Lecture with
Mentimeter, Miro, chat for
questions/answers
Bloom‘s higher order
thinking skills
(e.g., applying, critical analyzing,
to produce sth. novel)
Bloom‘s lower
order thinking
skills (e.g.,
to get an overview)
• Listening
• Students have no
control over the
learning process
• “Consumers”
• Interactive
• Students are actively
encouraged to think
• Students have no
control over the
learning process
Students produce videos, digital
games, Padlets, mindmaps,
“digital making”, etc.
Characteris
tics
Compe-
tences
developed
What course design supports active
student-centered learning with
digital technologies?
Prof. Dr. Isa Jahnke 11

12. Learning from vs. with technology
Jonassen, 1996
Learning from
technologies
Learning about
technologies
Learning with
technologies
Drill and practice, tutorials,
memorizing (passive
learning)
Computer literacy Active learning, higher order
learning skills
• Learner has no input
into the process,
• students are controlled
by the technology
• Learning about how to
use the technology,
• to understand how the
computer works
• Intellectual partnership,
• computer enhances
learner thinking /learning
Computer program is
programmed to teach the
student, to direct activities
toward the acquisition of
prespecified knowledge or
skills
Memorizing parts of facts about
technologies is relatively
meaningless; better would be to
understanding results from using
not memorizing
Technology use to extend
cognitive functioning during
learning and engage learners in
cognitive operations while
constructing knowledge that they
would not otherwise been capable
of.
Video-recorded
lecture Students use technolo
gy
as mindtools
Prof. Dr. Isa Jahnke

13.  Educational Design
Research Methodology
„Complex mess between technology &
instructional methods“
Jahnke et al., 2020
Edtechbooks.org/UX
Kozma, 2000
McKenney &
Reeves, 2018
Elements are intertwined with each other.
Prof. Dr. Isa Jahnke 13

14. Mental Model
(Learning design)
Action Digitalized
Material
Action
materialized
in data
e.g., Learning goals,
game-based
assignments,
assessments, videos
etc.
Teacher
Norman 1988, Jonassen 1995
Pham & Jahnke (under review)
cf. Herrmann & Kienle // N. Luhmann
How the
teacher does it
(the practice)
(Digital) Learning
Experience Design
Prof. Dr. Isa Jahnke 14

15. Mental Model
(Learning design)
Action Digitalized
Material
Action
materialized
in data
e.g., Learning goals,
game-based
assignments,
assessments, videos
etc.
Teacher
Norman 1988, Jonassen 1995
Pham & Jahnke (under review)
cf. Herrmann & Kienle // N. Luhmann
How the
teacher does it
(the practice)
Mental Model
(Learning design)
Digitalized
Material
Action
Students
The person is experiencing the
system through interacting with it
(Learner Experience)
Prof. Dr. Isa Jahnke 15
(Digital) Learning
Experience Design

16. Mental Model
(Learning design)
Action Digitalized
Material
Action
materialized
in data
e.g., Learning goals,
game-based
assignments,
assessments, videos
etc.
Teacher
Norman 1988, Jonassen 1995
Pham & Jahnke (under review)
cf. Herrmann & Kienle // N. Luhmann
How the
teacher does it
(the practice)
Mental Model
(Learning design)
Digitalized
Material
Action
Students
The person is experiencing the
system through interacting with it
(Learner Experience)
„Translation
Issues“
Prof. Dr. Isa Jahnke 16
(Digital) Learning
Experience Design

17. Start here:
Design
(Draft)
Learner/User
Experience
(LX) (e.g.,
positive
learning
experience)
Re-
Design 1
Learner/User
Experience (LX)
(e.g., effective
learning)
Re-
Design 2
Honebein & Reigeluth, 2021
McKenney & Reeves, 2018
Prof. Dr. Isa Jahnke 17
Research to Improve
(not Research to Prove)

18. Stage 3
Projects, Examples
Prof. Dr. Isa Jahnke 18

19. Context:
• Food imports must be quick (otherwise the food will rot),
but it must also be ensured that checks are carried out in order to identify bad goods
• Different countries have different approaches
e.g., China: every single food item is controlled, USA: risk-based inspection
• Project took 4 months: September 2020 – December 2022
Goal
• Developing Online Training of „Risk-based assessment of food when it is imported”
• We specified the learning objectives
• Policies, roles, and methods to guide risk-based food import inspections
• Target learners: Food inspectors, inspection officers, competent authorities from differ
ent countries, learners in the field
Jahnke, Li, Singh, Yu, & Riedel
(under review)
Prof. Dr. Isa Jahnke
CAFNR Project
Example
19

20. Phases
Storyboards:
• Outcomes:
Goals 1 (..)
• Content (Text,
Videos)
• Tasks/Assignmen
t
• Assessment
(e.g., Quizzes)
Moderator,
Content-Experts
(USDA, FDA),
Design-Team
Paper
Plan/Prototype
(Learning
design)
Experts/
Heuristics
Review
Study 1
Collect and
analyze dat
a
Nielsen’s heuristic
s & sociotechnical-
pedagogical
heuristics
Prof. Dr. Isa Jahnke
Re-
design
Design
team
20

21. Phases
Storyboards:
• Outcomes:
Goals 1 (..)
• Content (Text,
Videos)
• Tasks/Assignmen
t
• Assessment
(e.g., Quizzes)
Moderator,
Content-Experts
(USDA, FDA),
Design-Team
Paper
Plan/Prototype
(Learning
design)
Experts/
Heuristics
Review
Re-
design
User
Experience/
UX
w/ 10 p
Study 1
Collect and
analyze dat
a
Study 2
Collect and
analyze dat
a
Design
team
Nielsen’s heuristic
s & sociotechnical-
pedagogical
heuristics
• Task-based think alouds,
• SUS survey
• Task/System level satisfaction
• Follow-up interview
Prof. Dr. Isa Jahnke
Re-
design
Design
team
21

22. Phases
Storyboards:
• Outcomes:
Goals 1 (..)
• Content (Text,
Videos)
• Tasks/Assignmen
t
• Assessment
(e.g., Quizzes)
Moderator,
Content-Experts
(USDA, FDA),
Design-Team
Paper
Plan/Prototype
(Learning
design)
Experts/
Heuristics
Review
Re-
design
Re-
design Launch
(…)
User
Experience/
UX
w/ 10 p
Learner
Experience
/LX
w/ 46 p
Study 1
Collect and
analyze dat
a
Study 3
Collect and
analyze dat
a
Study 2
Collect and
analyze dat
a
Design
team
Design
team
Design
team
Nielsen’s heuristic
s & sociotechnical-
pedagogical
heuristics
• Task-based think alouds,
• SUS survey
• Task/System level satisfaction
• Follow-up interview
• Pre/Posttests
• T-test analysis
Prof. Dr. Isa Jahnke 22

23. Phases
Storyboards:
• Outcomes:
Goals 1 (..)
• Content (Text,
Videos)
• Tasks/Assignmen
t
• Assessment
(e.g., Quizzes)
Moderator,
Content-Experts
(USDA, FDA),
Design-Team
Paper
Plan/Prototype
(Learning
design)
Experts/
Heuristics
Review
Re-
design
Re-
design Launch
(…)
User
Experience/
UX
w/ 10 p
Learner
Experience
/LX
w/ 46 p
Study 1
Collect and
analyze dat
a
Study 3
Collect and
analyze dat
a
Study 2
Collect and
analyze dat
a
Design
team
Design
team
Design
team
Nielsen’s heuristic
s & sociotechnical-
pedagogical
heuristics
• Task-based think alouds,
• SUS survey
• Task/System level satisfaction
• Follow-up interview
• Pre/Posttests
• T-test analysis
Prof. Dr. Isa Jahnke 23

24. Instructional flow – What is your pedagogical foundation and how is it
embedded into your innovative course design?
(Gagne et al. 1992; Lee et al. 2021)
1. An aha-moment: Learner understands relevance of the topic
(Gagne’s #1: Gain attention).
2. Learner engages actively with the content
(Gagne’s #4: Present the content).
3. Learner applies what s/he has learned; CAFNR: gamified exercises
(Gagne’s #6: Elicit performance / students do practice).
4. Learner receives immediate feedback to their performance: automated
feedback (Gagne’s #7: Provide feedback and #8: Assess performance).
Prof. Dr. Isa Jahnke 24

25. Prof. Dr. Isa Jahnke 25

26. Prof. Dr. Isa Jahnke 26

27. Paper Prototype w/ Expert Heuristics Evaluation:
Study 1 - Results
Positive results
• Clearly understandable learning goals
• Contents are coordinated with learning objectives and suitable for this target group
• Sequential units that build on what has been previously learned
• Variety of strategies to 'involve’ and encourage learners to engage with the material
Need for improvement
• Challenge: Balance active and passive learning
• Design learning activities that correspond to the learning culture of the host
country (socio-cultural dimension)
• Videos under six minutes! Not longer, people drop out
• Activities with 'real world' tasks (authentic for the learners!, not for
designers/teachers)
Prof. Dr. Isa Jahnke 27

28. Usability / User Experience (UX)
Study 2 - Results
More or less positiv
• SUS: Result B (scale A-D, F), that is a “good” design (Sauro, 2011)
• SUS: Majority of users found the learning design to be "acceptable"
Need for improvement (Appeal)
1. Layout
• Inadequate design: a lot of empty spaces; buttons were placed in imperceptible places
• Use of low contrast colors in some videos
• Font size in the images too small
2. Functionality
o Lack of important functions, including the help function and the "Back" button
o Lack of immediate feedback regarding the course enrollment status
o External links do not open in separate windows
3. Content
o Instructional videos were not fully interactive and had no subtitles
o There was a lack of adequate instructions for the activity
Prof. Dr. Isa Jahnke 28

29. Learning Experience (LX)
Study 3 - Results
Efficiency?
• 31 of 46 study participants did the online training in 2 hours (expected)
• Time period: at least 40 minutes, at most 24 hours
• Average time: 4 hours - too long!
Effectiveness - effective Learning?
• 43 of 46 participants improved their posttest scores - statistically significant before vs. after
• Guess vs. knowledge: higher points in the after-test (significant difference)
• BUT: only 16 out of 46 achieved the target number of points (see table) - not good!
Level Pre-test
Percentage
Pre-test
Score
# Learners
Pre-test
score
Targete
d
Posttest
%
Targeted
Posttest
Score
# Learners
Meeting
Targeted
Score
Level One
(Beginning)
0% - 40% 0-18 19 60% 27 12 of 19
Level Two
(Far but
Likely)
41% - 55% 19-24 25 70% 31 4 of 25
Level Three
(Close to
Proficient)
56% - 70% 25-31 2 80% 36 0 of 2
Level Four
(Proficient)
71% - 85% 32-38 0 90% 41 0
Level Five
(Expert)
86% - 100% 39-45 0 95% 43 0
Total 46 16
29

30. What is next in CAFNR?
Efficiency
• Back to the content experts to optimize “self-paced” training
 Focus group interviews with the 9 specialist experts
• Terminology and feedback should be improved
Effectiveness
• It works! It is effective – for beginners: “Expertise Reversal Effect” (Kalyuga, 2003)
• Add advanced active learning strategies: From active behavior to active thinking ?
• Next step: Module 2
Lessons Learned
• Iterative design and research combination is useful for improving learning designs
• The approach helps questioning one's own theories/assumptions about learning
design:
Research to improve (not just research to prove).
• This new approach is called "Learning Experience Design" (Schmidt et al. 2020) with
three dimensions of
o Technological dimension: User Experience and usability methods for HCI aspects
o Pedagogical dimension: Instructional design
o Social or socio-cultural dimension: learning experience design for range of learners
Wang & Wegerif,
2019
Prof. Dr. Isa Jahnke 30

31. Mobile Microlearning in Journalism Education –
gamified; active, but: not student-centered?
Context:
• Learning goal of the micro course (5 units): apply the 5C model of journalistic writing
• Didactic flow: aha moment, interactive content, games-bas. exercises, automated feedback
• No longer than 5 minutes per session; on the smart phone
RQ: To what extent does the micro-course design promote student learning? (Learner Experience?)
Method: Educational Design Research with UX/LX: task-based think alouds, pre/post-tests online (et
c.);
Studying effectiveness, efficiency and appeal of the micro-course (Honebein & Honebein, 2015)
Results:
Mikro-Kurs is efficient, it speaks to learners (appeal), 26 of 35 people have achieved relative learning
growth
(approx. 70%) => relatively effective!
What needs to be improved:
a) Automated instant feedback: tailor feedback more to learners (use AI für personalized
feedback)
b) Interactive content: adapt real world examples to the context of the learner
c) Challenge: adapting the content to the screen size of a mobile phone
d) From individual learning to: learning with others?
Lee, Jahnke, &
Austin, 2021
Jahnke, Lee, Minh,
Hao & Austin (2019)
Prof. Dr. Isa Jahnke 31

32. Active Learning with smart glasses (AR) and
iPads in dentistry education
Mårell-Olsson & Jahnke, 2019
Context / Method
• 1 instructor, 18 students,
• 3 iterations in 1 semester
• Interviews and observation
Problem was
• Complex ways of communication between
teacher and students during internship (Praktikum)
• Feedback process in inconvenient/awkward
RQ: How can teacher-student communication be improved through smart glasses
?
What potentials or challenges are there that inform future design?
Results
• Breaking routines: group sees new opportunities for learning, especially new communi
cation channels that improved the feedback process and quality
• More efficient communication and improved coordination
• Send photos quickly instead of texts: improved real-time learning
Prof. Dr. Isa Jahnke 32

33. Student developed games for learning
with AR-technologies
Context: Blended Learning Course, 4 groups (3-5 students) developed gamified AR-applications
RQ: How do non-tech students work together when developing ICT products? (Active L. Roles?)
Method: Model by Chi & Wylie (2014) used for analysis, 1) active, 2) constructive, 3) interactive (highest
level)
Results:
• 2 groups between 2.-3. level
• 1 group had problems with active learning, “struggled when authoritative information was absent”
• 1 group, new form: co-design level (4th new level), “filled missing resources”
Implication:
• Students think it is not allowed to break out of or deviate from the seminar/course design,
to think independently or to surprise the teacher with their own learning initiative
• Digital learning design requires supporting tools for learners and supporting structures for learners
to reach the co-designer level
Jahnke, Kroll, Todd, &
Nolte, 2020
Libway Ethical Dilemma (Ethik-Theories in Practice):
AI has taken over the university. Your decision aff
ects everyone in the university
WordScramble / History
33

34. 34
RQ: To what extent is it possible to develop a set of STP heuristics; what quality is it?
Method: Iterative development (Quiñones et al., 2018), literature review with 195 items, grou
ping, 14 new heuristics tested against Nielsen and Nokelainen heuristics
Results:
• Socio-technical-pedagogical / STP heuristics (with 14 items) for the design and evaluatio
n
of online courses
• Design recommendations for improving online courses
Jahnke, Riedel, Singh,
& Moore, accepted
OPEN ACCESS
Context: Online Courses
(a) Master Gardener (b) Fire Service Instructor
Literature: Separation of technological usability-
heuristics (Nielsen, 1994) and pedagog. heuristics
(Nokelainen, 2006) - neglecting social heuristics
(Jahnke et al., 2020)
Prof. Dr. Isa Jahnke
Active Learning in Online Courses

35. Stage 4
Conclusion & Next steps

36. Active Learning Design with Digital
Technologies - Conclusion
=Intertwinement of pedagogy, technology, social dimension;
co-evolutionary growth (not just one or the other)
Under what conditions does Active Learning make sense?
• Not applicable in each situation (see micro course)
• Good for in-depth learning
Which digital technologies support Active Learning?
Thesis: Every digital technology can be (re) used for LXD,
it is a question of “how” you use it as a mind tool in the learning design.
When is it effective?
• Agile mindset of teachers: Teacher as Designer
• Shape the context: e.g., consider group dynamics of learners
• Learning processes (creation of learning artifacts) & iterative feedback processes (guided practice)
• Not without instructional strategies (direct instruction)
What needs still to be done in “digital active learning forms”?, e.g.
Promote learners as co-designers: break out of given structures/roles, think beyond their own horizo
n
& support them
Prof. Dr. Isa Jahnke 36

37. Design for
social relations
Learning
activities
Learning
goals/ILO
Use digital
Technologies as
mindtools
Process-based,
formative Assessment
1
2
3
4
5
From traditional teaching (inner layer = 1)
to
active/meaningful practices (outer layer = 5)
DDD as framework to design
digital active learning
37
Jahnke et al., 2017
Jahnke, 2015
Prof. Dr. Isa Jahnke 37

38. Prof. Dr. Isa Jahnke 38

39. Checklists
for planning and practice
Online :
https://www.isa-jahnke.com/teaching
Prof. Dr. Isa Jahnke 39

40. Schmidt et al., (2020): Learner and User Experience Research
https://edtechbooks.org/ux (open access book)
Learning Experience Design (LXD)
- connecting didaktik (pedagogical strategies), technology use, social dimension
Prof. Dr. Isa Jahnke 40

41. People’s Choice Award
at AECT2021
(10 mins. video shared via Twitter @isaja)
Prof. Dr. Isa Jahnke 41
Open Access book:
Edtechbooks.org/ux

42. Prof. Dr. Isa Jahnke
Email
isa.jahnke@uni-muenster.de
Website
http://www.isa-jahnke.com
Twitter
@isaja
Thanks!
Happy to discuss!