Presentation for the 1st International Workshop on Multimedia Technologies and Distant Learning at ACM Multimedia 2009 Ralf Klamma, Marc Spaniol, Matthias Jarke Beijing, China, October 23, 2009

1. CUELC
Knowledge Multimedia Processes
in Technology Enhanced Learning
Ralf Klamma1, Marc Spaniol2, Matthias Jarke1
1RWTH Aachen University, Germany &
2Max Planck Institute for Computer Science
ACM Multimedia WS
Multimedia Technologies for Distant Learning MTDL 2009,
Lehrstuhl Informatik 5
(Informationssysteme) Beijing, China, October 23, 2009
Prof. Dr. M. Jarke
I5-RK-0909-1

2. Agenda
 Theoretical background
CUELC
– Cross-media theory of transcription
– SECI model by Nonaka
 Knowledge Multimedia Processes
– Transcriptions
– Practiced/Formalized Localizations
– Addressing
 Evaluations
 Conclusions & Outlook
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-2

3. Cross-Media Theory
of Transcription
Addressee-based
Transcription
Transcript
Transcript
CUELC
Pre-Texts
Selection of
Pre-Texts
Integration into the
cultural Archive
 Strategies of transcriptivity
 Collection of learning materials are re-structured by new media
 Design is specific for media and learning communities by default
 Strategies of addressing
 Social Software promotes the globalization of address spaces
 Personalization is mission critical for learning communities
 Strategies of localisation
 Re-organization of local practices is stimulated by new media
Lehrstuhl Informatik 5
(Informationssysteme)
 Need to model learning practice explicitly
Prof. Dr. M. Jarke Jäger, Stanitzek: Transkribieren - Medien/Lektüre 2002
I5-RK-0909-3

4. Multimedia Knowledge Management
for Technology Enhanced Learning
 Synthesis:Transcriptivity / SECI Model / Communities of Practice
CUELC  Goal: Media theoretic focus on technology enhanced learning
Externalization:
Pre-Texts Transcription Transcript
Socialization: Combination:
practiced Localization formalized Localization
Transcript
&
Context
Lehrstuhl Informatik 5 Internalization:
(Informationssysteme)
Prof. Dr. M. Jarke Addressing
I5-RK-0909-4 EC-TEL 2007 [Spaniol et al.]

5. Transcription:
an IS view
Pre-Texts Inter- & intra-media transcription
Transcript
CUELC
 Examples and Requirements on IS :
– Creation and maintenance of multimedia metadata
– Representation of multimedia relationships (hypermedia)
– Extension of archiv for multimedia content
⇒ Implications for the management of multimedia:
1. Deskriptors for multimedia
2. Multimedia graph representation
Lehrstuhl Informatik 5
(Informationssysteme) 3. Extensibility of schemata and descriptors
Prof. Dr. M. Jarke
I5-RK-0909-5

6. Transkription
Media: M ∈{M „Text“ , M „Image“ , M „Audio“ , M „Video“ }n ∪ {P}, n ∈ N 0
CUELC
Media in IS: M P
Transcripts:
t p := (τ (p), µ (p), ι (p))
Transcription:
τ :P → M P
Meta-transcription
(manual): µ : P → M„Text“ ∪ ∅
Meta-transcription
(in IS ): ι : P → M„Text“
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-6

7. Formalized Localization:
an IS view
Pre-Texts Inter- & intra-media transcription
Transcript
CUELC
 Examples and Requirements on IS:
– Individual categorization of media
– Creation and Maintenance of reference collections
– Fine granular management of access rights
⇒ Implications for the management of metadata:
1. Schemata for classification and vocabularies
2. Variation sets of multimedia
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
3. Digital rights management
I5-RK-0909-7

8. Formalized Localization:
Keyword Index/ Category System
Keyword index: λ c,1
m1
S λ 1,2 M
CUELC S := {s} λ 3,2
s1 m2
Access relation: λ 1,d
s3 m3
λ i : S → P (M) s2
sc md
Category index:
K := {k} k := (id, s) root λ c,1
m1
λ 1,2
κ root,2 κ root,1
OK M
(Tree)-Order: λ 3,2
k1 m2
o := (k, k' ,κ (k id , k'id )) λ 1,d
κ 1,3 k3 m3
Access relation : k2
κ 2,c
Lehrstuhl Informatik 5
λ k : ID → P (M)
(Informationssysteme)
Prof. Dr. M. Jarke kc md
I5-RK-0909-8 ID

9. Formalized Localization :
Local Access Relation
Global access relation: root root
κ root,2 κ root,1
κ root,c κ root,1
OK insert kη OK
CUELC
lglobal := {K global , O global , λglobal }
k1 k1
Local access relation:
κ 1,3
κ 1,η
delete kη
k3 kη
l local := {K local , O lodal , λlodal }
κ η,3
k2 k3
rename k2
κ 2,c
kc
Opeations in
κ c,2
kc and kc
category schemata : k2
- Insert root λ c,1
- Delete m1
λ 1,2
κ root,2 κ root,1
- Rename
OK M
λ 3,2
k1 m2
λ 1,d
κ 1,3
Operations in k3 λ 2,3
m3
media relations: k2
- Insert
κ 2,c
insert λ 2,3 md
Lehrstuhl Informatik 5 kc
(Informationssysteme)
- Delete
Prof. Dr. M. Jarke
I5-RK-0909-9
delete λ 2,3

10. Adressing:
an IS View
represent & discuss in context Transcript
CUELC in
context
 Examples and Requirements on IS :
– Representations of multimedia content in context
– Options for legitimate peripheral participation
– (asynchronous) cooperation support
⇒ Implications for the management of metadata:
1. Metadata driven adaption of devices
2. Profiles
Lehrstuhl Informatik 5
(Informationssysteme) 3. User preferences and usage histories
Prof. Dr. M. Jarke
I5-RK-0909-10

11. Adressing
Discussions: dtp := (t p ,δ (t p ))
CUELC
δ t : T → M„Text“
Hypermedia documents: h t,l,d := (t u , lv , dtw , µ (h))
p p
u, v, w ∈ N 0
Discussions on dht,l,d := (ht,l,d ,δ h (ht,l,d ))
hypermedia documents:
δ h : H → M„Text“
Context-aware adressing α l :T →I
Lehrstuhl Informatik 5
of multimedia content:
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-11

12. Validation of the Concept
CUELC
Self-observation tools
for Communities
Socio-technical Measure,
information system Analyses,
Development Simulate
Collaborative adaptive
learning platforms
CESE: Hypertext Environment for Talmud tractates NMV & MEDINA: Dublin Core & MPEG-7 based Media Tagging
MARS: Transcription of electro-acoustic music ACIS: GIS-Multimedia Management for Cultural Heritage
SOCRATES: Chat for Communities of Aphasics MIST: Non-linear Story-Telling
MECCA: Collaborative Screening of Movies PROLEARN Academy, GRAECULUS, Multimedia Metadata,
Lehrstuhl Informatik 5 VEL 2.0: Virtual Entrepreneurship Lab CUELC, Bamyian Valley: Community Portals
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-12 Klamma, Spaniol, Cao: MPEG-7 Compliant Community Hosting, JUKM, Springer 2006

13. MIST – Multimedia Management
• Creation and management:
CUELC - Media collections
- Metadata
- Media variations
• Collaborative indexing based on:
- Free text annotations (à la Flickr)
- Semantic MPEG-7 basetypes
 Agent
 Event
 Concept
 Object
 Place
Lehrstuhl Informatik 5  Time
(Informationssysteme)
Prof. Dr. M. Jarke  State
I5-RK-0909-13

14. MIST – Story Creation
CUELC
Structured of
Structure access
“Mapping“of
to contents of the
(sub)-problems to
Begin-
multimedia archive
Middle-
artifacts
End-Sequences
 Illustration of episodic knowledge
– Modeling of non-linear stories based on Movement Oriented Design (MOD)
[Sharda 2005]; Cooperation with Victoria University Melbourne, Australia
Lehrstuhl Informatik 5 – Decomposition of stories according to a problem hierarchy
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-14
– Recall of semantic high-quality metadata on multimedia artifacts

15. MIST – Story Consumption
Optional
CUELC successor-
sequences
Currently
selected
artifact
MPEG-7 based
annotations
Free text
annotations
 Non-linear access sequences in multimedia stories,
i.e. “success vs. failure“ depending on access sequence
 Context information via multimedia annotations
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-15

16. M. Spaniol, Y. Cao, R. Klamma, P. Moreno-Ger, B. Fernández Manjón, J. Luis Sierra, G. Toubekis :
From Story-Telling to Educational Gaming: The Bamiyan Valley Case ,
in F. Li, J. Zhao, T. K. Shih R. Lau, Q. Li, D. McLeod (Eds.) Advances in Web Based Learning - ICWL 2008,
<e-Adventure> Editor
 Authoring environment
CUELC for point and click
adventure games
 Refinement of the raw
educational game
derived from the
non-linear story
 Specificationof items
and character
references
 Definition of dialogs
⇒ Fully featured adventure game for vocational training
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-16

17. Conclusions & Outlook
 Cross mediality for collaborative learning platforms supports
CUELC – Evolutinary development of multimedia learning services
– Configuration and maintenance of multimedia learning repositories
 Semantic self organization of learning communities
supported by
– Flexible semantic enrichment of complex processes based on
multimedia artifacts
– Re-contextualization of multimedia knowledge in stories & games
 Ongoing and future work
– Mobile Storytelling based on a simplified mobile user interface
– Use of templates for supporting inexperienced storytellers
– Mashing up different stories to create and share innovative
knowledge
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-17

18. Join us!
 http://www.role-project.eu/
CUELC
 LinkedIn
http://www.linkedin.com/groupInvitation?gid=1590487
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-18