Adriana Berlanga, Gaston Burek, Fridolin Wild

1. Language Technologies for
Lifelong Learning

2. Lifelong Learning &
Language Technologies
Adriana Berlanga
Open University, NL
Gaston Burek
University of Tubingen, DE
Fridolin Wild
Open University, UK
European Summer School on Technology Enhanced Learning
Terchova, Slovakia. June, 2009

3. Outline
• Lifelong learning
• Critical supporting activities
• Language Technologies
• LTfLL Project: Language Technologies for
Lifelong Learning
• LTfLL: Positioning of the learner in a domain
• Questions
• Discussion
3

4. Lifelong Learning
4

5. Critical support activities
• Assessment
– Formative feedback
• Answering questions
– Routing questions
– Formulating personalised answer
• Monitoring progress
– Drop out prevention
• Supporting groups and communities
– Selecting and creating groups
– Providing overviews 5
Van Rosmalen et al. (2008)

6. Critical support activities
6

7. Language Technologies
Question mark photo by Leo Reynolds. Licensed under Creative Commons.
7

8. Natural Language Processing
(NLP)
Natural Language = Human speech
Analyzes spoken or written language
Discover terms, generate text, translation,
synthesis…
– Computational Linguistics
– Artificial Intelligence
 Language Technologies
8

9. Working Principle
9
Landauer (2007)

10. Latent Semantic Analysis (LSA)
Input (e.g., documents) term = feature
vocabulary = ordered set of features
Only the red terms appear in more
than one document, so strip the rest.
TEXTMATRIX
{M}=
10
Deerwester, Dumais, Furnas, Landauer, and Harshman (1990)

11. How to support these activities
in a (semi-) automatic way?
11

12. Language Technologies for
Lifelong Learning

13. LTfLL – Objective
To create a set of next-generation support and
advice services that will enhance individual
and collaborative building of
competences and knowledge creation in
educational as well as organizational settings.
The project makes extensive use of language
technologies and cognitive models in the
services.
13

14. LTfLL - Themes
Theme 2
support
feedback
services
Theme 1 Theme 3
position of social and
the learner informal
in a domain learning
LTfLL
14

15. Positioning
• To determine in a (semi-)automatic way learner’s prior
knowledge –by analyzing her Portfolio and the domain of
study– to recommend learning materials or courses to follow.
• To provide formative feedback with regard to the learner’s
profile in the domain of study and recommend remedial actions
to overcome conceptual gaps.
Learner Support & Feedback services
• To offer recommendations based on an analysis of interactions
in collaborative learning using chats and discussion forums.
• To offer recommendations based on the analysis of textual
outputs by the learner.
Supporting social & informal learning services
• To provide recommendations on the basis of the learner’s
profile, interests, preferences, network and learning task. This
requires implementing a Common Semantic Framework (i.e. an
ontology).
• To provide a list of search results prioritized and categorized
according to the conditions specified by the learner, and the 15
opinions of the learner’s trusted network of contacts.

16. LTfLL - Themes
Theme 2
support
feedback
services
Theme 1 Theme 3
position of social and
the learner informal
in a domain learning
LTfLL
16

17. Positioning
Question mark photo by Leo Reynolds. Licensed under Creative Commons.
17

18. Positioning
• Automatically determine learner’s
knowledge in a domain, given the
chosen learning goal
18

19. Positioning
To determine in a (semi-)
To provide formative feedback
automatic way learner’s prior
with regard to the learner’s profile
knowledge –by analyzing her
in the domain of study and
Portfolio and the domain of study–
recommend remedial actions to
to recommend learning
overcome conceptual gaps
materials or courses to follow
Provide formative
Locate best suitable
feedback and
learning materials or
recommend
courses to follow
remedial actions
19

20. Positioning - Adriana Berlanga & Fridolin Wild
FORMATIVE FEEDBACK
20

21. Formative feedback
• Services will offer semi-automatic
measurement of conceptual development
• Diagnosing conceptual development
– Person’s knowledge of a domain by looking
on how s/he organizes the concepts of such
domain
– Novice vs. expert approach
21

22. The approach:
Novice vs. Expert
Novices and experts differ in
• How they express the concepts underlying a
domain
• How they discriminate relevant from non-
relevant information
• And how they use and relate the concepts to
one another
22

23. Exploring the approach
SHOWCASE
24

24. Procedure
1. Ask learners what they know
2. Find out what learners should know
“Expert-reference model”
3. Compare
4. Provide feedback and recommendations 25

25. 1. Ask learners what they know
A think-aloud protocol was designed. Learners
must explain the relevant concepts in a domain
and how they are related
Examples
• Medical students (U Manchester) had a think-
aloud session to explain concepts they just
studied
• Researchers (OUNL) had a think-aloud session
to explain concepts they develop in their work
 The think-aloud audio is transcribed: text
26

26. 2. Find out what learners should know
“Expert-reference model”
1. Theoretical expert model, documents of a
particular course (e.g., course material, tutor
notes, presentations)
2. Archetypical expert model, state-of the art
information (e.g., scientific literature)
3. Emerging expert model, concepts and the
relations a group of people (co-workers,
peers...) use to describe a domain
3. Compare
27

27. 2. Find out what learners should know
“Expert-reference model”= Theoretical expert model
Learner Expert
Generation of expert and student concept
maps
Leximancer 28

28. 2. Find out what learners should know
“Expert-reference model”= Archetypical expert model
Expert map generated from LSA
29
scientific literature

29. 2. Find out what learners should know
“Expert-reference model”= Emergent expert model
Leximancer 30

30. 4. Provide feedback and recommendations
 These are the concepts you mentioned the most
 From your peers these are the most mentioned
concepts
 The differences are:
 This means that you might find useful to
• Read this material
• Do this activity
• Contact this person
• …
 Observations

31. Positioning
To determine in a (semi-)
To provide formative feedback
automatic way learner’s prior
with regard to the learner’s profile
knowledge –by analyzing her
in the domain of study and
Portfolio and the domain of study–
recommend remedial actions to
to recommend learning
overcome conceptual gaps
materials or courses to follow
Provide formative
Locate best suitable
feedback and
learning materials or
recommend
courses to follow
remedial actions
32

32. Positioning – Gaston Burek
LOCATE BEST SUITABLE
LEARNING MATERIALS
33

33. Positioning Principles
Learner Objectives Evidence Materials Automatic Assessors
indicated indicated indicated by recommendations decision
by learner by learner assessors
John S. Topics 1 Self Assessment Course No automatic Accept
documents yes/no
Topics 2 Certificates Course No automatic Accept
documents yes/no
Topics 3 Portfolio Materials LSA or Accept
Documents or docs knowledge rich yes/no
accepted/ approach
rejected
Topics 4 Formal Assessment LSA or Accept
assessment samples knowledge rich yes/no
accepted/ approach
rejected
34

34. Objectives
 A plan for extending language technologies for
positioning
• Implementing a showcase and scenarios to
investigate and validate the approaches and give
input to the design of the first release of the
positioning (Wild, Hoisl and Burek, 2009)
• To implement a positioning service that
assess learner competences and recommend
a sequence of learning material according to
learning goals.
35

35. Scope of the analysis (text)
• Learning history captured in the learner portfolio can
be documented in different formats including text
• Texts (learning materials, learner produced docs,
etc.) may use long linguistic expressions to describe
high level conceptualisations and terms for more
precise descriptions
• Curricula contain high level descriptions of learning
activity materials and goals
• Reduced number of samples generate sparse
linguistic data
• Learning materials and e-portfolios can be in different
36
languages (future work)

36. Ongoing Analysis
• Finding prototypes
• Students discussions on safe prescribing:
Classified according expected learning outcomes
related subtopics topics (A, B, C, D, E, F)
Graded by experts (poor, fair, good, excelent)
Methodologies
• LSA
• KNN
• Permutation tests
37

37. Scenario
Learning goal: specified by job descriptions
Stakeholders objectives:
• Unemployed individual: to acquire qualifications needed to find a
job according to available vacancies
• Individual currently employed: to acquire the competences to be
useful for his company and reduce the risk of loosing his/her job
• Educational provider: delivering high quality personalised
technical education
• Employment Governmental agency: to reduce unemployment
• Enterprises: to develop employees competences for them to be
capable to undertake duties required within the organisation
according to predefined expectations
38

38. uestions?
Question mark photo by bureau120. Licensed under Creative Commons.

39. DISCUSSION
40

40. Guiding questions
Theme 2
support
feedback
services
Theme 1 Theme 3
position of social and
the learner informal
in a domain learning
1. How can you validate services for these areas?
2. Big challenges? LTfLL
3. Other experiences? 41
4. Who is already working with NLP technologies for TEL?

41. Workshop A
NLP-based
experiments in TEL
Today @ 16.15
42

42. Adriana Berlanga
adriana.berlanga@ou.nl
Gaston Burek
gaston.burek@googlemail.com
Fridolin Wild
f.wild@open.ac.uk
www.ltfll-project.org
DSpace
dspace.ou.nl/simple-search?query=LtfLL
43

43. References
• Berlanga, A. J., Kalz, M., Stoyanov, S., Van Rosmalen, P., Smithies, A., &
Braidman, I. (2009). Using Language Technologies to Diagnose Learner´s
Conceptual Development. In Proceedings of the 9th IEEE Int. Conference on
Advanced Learning Technologies. IEEE, in press.
• Deerwester, S., Dumais, S.T., Landauer, T., Furnas, G.W., & Harshman, R.A.
(1990): Indexing by Latent Semantic Analysis. In: Journal of the Society for
Information Science, 41(6):391-407
• Landauer, T. (2007). LSA as a Theory of Meaning. Handbook of Latent
Semantic Analysis, Mahaw: Lawrence Erlbaum Associates, 3-35.
• Van Rosmalen, P. Sloep, P.B., Kester, L., Brouns, F., De Croock, Pannekeet,
K., & Koper, R. (2008). A learner support model based on peer tutor selection.
Journal of Computer Assisted Learning, 24(1), 74-86.
• Wild, F., Hoisl, B., & Burek, G. (2009): Positioning for Conceptual Development
using Latent Semantic Analysis. In: Basili, Pennacchiotti (Eds.): Proceedings of
the EACL 2009 Workshop on GEMS: GEometical Models of Natural Language
Semantics, Association for Computational Linguistics, 41–48.
44