Making sense of collaboratively annotated multimedia metadata for (mobile) digital story-telling and educational gaming First International Workshop on Story-Telling and Educational Games (STEG 2008) Maastricht, The Netherlands September 16th 2008 Pablo Moreno Ger , Marc Spaniol, Enrique López Mañas Niels Drobek, Baltasar Fernández-Manjón Universidad Complutense RWTH - Aachen Max Planck Institute

Motivation Educational gaming could use some… Attractive narratives Good stories interweaving content and interest Well-structured and correctly paced narratives for learning situations Story-telling could use some… Visual support (eye-candy) Interactivity Actually, these are not new ideas Games tell stories Interactive stories are games

Educational gaming could use some…

Attractive narratives

Good stories interweaving content and interest

Well-structured and correctly paced narratives for learning situations

Story-telling could use some…

Visual support (eye-candy)

Interactivity

Actually, these are not new ideas

Games tell stories

Interactive stories are games

Education 2.0 New media offers the consumers the opportunity of contributing their own content The Internet is going 2.0 Education is going 2.0 Even videogames are going 2.0 (MMOs, Spore) Web 2.0 brought forth the prosumer Producer + consumer Learning 2.0 is introducing the teaner Teacher + learner

New media offers the consumers the opportunity of contributing their own content

The Internet is going 2.0

Education is going 2.0

Even videogames are going 2.0 (MMOs, Spore)

Web 2.0 brought forth the prosumer

Producer + consumer

Learning 2.0 is introducing the teaner

Teacher + learner

But… Can teaners really produce high-quality educational content? Instructor guidance Peer-correction and improvement Is 2.0 educational gaming even possible?

Can teaners really produce high-quality educational content?

Instructor guidance

Peer-correction and improvement

Is 2.0 educational gaming even possible?

The objectives: Use together digital story-telling and educational gaming platforms Provide a collaborative authoring process for story-driven educational gaming Facilitate the creation and maintenance of metadata annotations

Use together digital story-telling and educational gaming platforms

Provide a collaborative authoring process for story-driven educational gaming

Facilitate the creation and maintenance of metadata annotations

MIST: Media Integrated Story-Telling Support of an international community of researchers in order to preserve the Afghan cultural heritage by: - Exchange of multimedia artifacts - MPEG-7 based semantic multimedia annotations - Re-contextualization multimedia contents via non-linear stories

Support of an international community of researchers in order to preserve the Afghan cultural heritage by:

MIST – Multimedia Management Creation and management: Media collections Metadata Media variations Collaborative indexing based on: Free text annotations (à la Flickr) Semantic MPEG-7 basetypes Agent Event Concept Object Place Time State

Creation and management:

Media collections

Metadata

Media variations

Collaborative indexing based on:

Free text annotations (à la Flickr)

Semantic MPEG-7 basetypes

Agent

Event

Concept

Object

Place

Time

State

MIST – Story Creation Illustration of episodic knowledge Modeling of non-linear stories based on Movement Oriented Design (MOD) Decomposition of stories according to a problem hierarchy Recall of semantic high-quality metadata on multimedia artifacts

Illustration of episodic knowledge

Modeling of non-linear stories based on Movement Oriented Design (MOD)

Decomposition of stories according to a problem hierarchy

Recall of semantic high-quality metadata on multimedia artifacts

MIST – Story Consumption Non-linear access sequences in multimedia stories, i.e. “success vs. failure“ depending on access sequence Context information via multimedia annotations

Non-linear access sequences in multimedia stories, i.e. “success vs. failure“ depending on access sequence

Context information via multimedia annotations

<e-Adventure> Instructor-oriented authoring tool for the creation of low-cost educational adventure games for online education http://e-adventure.e-ucm.es

Instructor-oriented authoring tool for the creation

of low-cost educational adventure games for online education

<e-Adventure> Instructor-oriented authoring tool for the creation of low-cost educational adventure games for online education Why adventure games? Biased for content instead of action Narrative and puzzles drive the game They promote significant learning skills Problem solving Action planning Exploration Situated reasoning Discovey-based learning

Instructor-oriented authoring tool for the creation

of low-cost educational adventure games for online education

Why adventure games?

Biased for content instead of action

Narrative and puzzles drive the game

They promote significant learning skills

Problem solving

Action planning

Exploration

Situated reasoning

Discovey-based learning

<e-Adventure> Instructor-oriented authoring tool for the creation of low-cost educational adventure games for online education

Instructor-oriented authoring tool for the creation

of low-cost educational adventure games for online education

<e-Adventure> Instructor-oriented authoring tool for the creation of low-cost educational adventure games for online education Games as Learning Objects <e-Adventure> exports the games as standards-compliant Learning Objects IMS Content Packaging + IEEE LOM Metadata

Instructor-oriented authoring tool for the creation

of low-cost educational adventure games for online education

Games as Learning Objects

<e-Adventure> exports the games as standards-compliant Learning Objects

IMS Content Packaging + IEEE LOM Metadata

The best of both worlds Game document <e-Adventure> game MIST XML-binding MIST story Edit Refine Adaptation process

The advantages MIST interactive stories can be enriched and executed in a visually appealing environment <e-Adventure> games can get better story-lines written in MIST 2.0 (1.5?) creation of educational games

MIST interactive stories can be enriched and executed in a visually appealing environment

<e-Adventure> games can get better story-lines written in MIST

2.0 (1.5?) creation of educational games

Case-study: Cultural Heritage Tasks in the Bamiyan Valley Documentation of consolidation measures at Bamiyan cliffsite Help local workers understand the task and the procedures involved Gather annotated media and transmit it through story-driven educational games

Documentation of consolidation measures at Bamiyan cliffsite

Help local workers understand the task and the procedures involved

Gather annotated media and transmit it through story-driven educational games

The Bamiyan e-Adventure

But… Are we really using everything that MIST and <eAdventure> have to offer? ?

Are we really using everything that

MIST and <eAdventure> have to offer?

In MIST… MIST offers a very powerful metadata management system Huge media repository MPEG-7 metadata annotated collaboratively Metadata for each individual media resource Different categories (location, coordinates, current state, historical interest…) Secured database with access control

MIST offers a very powerful metadata management system

Huge media repository

MPEG-7 metadata annotated collaboratively

Metadata for each individual media resource

Different categories (location, coordinates, current state, historical interest…)

Secured database with access control

In <e-Adventure>… Games can be exported as standards-compliant Learning Objects Packaged according to the IMS Content Packaging specification Annotated with IEEE Learning Object Metadata

Games can be exported as standards-compliant Learning Objects

Packaged according to the IMS Content Packaging specification

Annotated with IEEE Learning Object Metadata

Open questions… Which metadata standards should we use? MPEG7 IEEE LOM Dublin-Core On which level should we maintain metadata? Game/story Individual resource How can we maintain the metadata? We need good tools to allow users to maintain the metadata collaboratively

Which metadata standards should we use?

MPEG7

IEEE LOM

Dublin-Core

On which level should we maintain metadata?

Game/story

Individual resource

How can we maintain the metadata?

We need good tools to allow users to maintain the metadata collaboratively

Resource-level metadata MPEG 7 seems to be the most powerful Our solution: Use MIST metadata for the resource level Avoid loosing the benefits of MIST when converting the story into a game Game document <e-Adventure> game MD Unit MD Unit MD Unit Collaboratively annotated Flexible approach Exported from MIST as an XML file Runtime link to MIST storage Open specification: MPEG7, DublinCore, …

MPEG 7 seems to be the most powerful

Our solution:

Use MIST metadata for the resource level

Avoid loosing the benefits of MIST when converting the story into a game

Collaboratively annotated

Flexible approach

Exported from MIST as an XML file

Runtime link to MIST storage

Open specification: MPEG7, DublinCore, …

Game/story-level metadata Leverage <e-Adventure> LMS-orientation MPEG7 is more powerful, but IEEE LOM is more extended in educational contexts Our solution: Use IEEE LOM for game-level metadata Learning Object (IMS Content Packaging) LO Metadata (IEEE LOM) Game document <e-Adventure> game MD Unit MD Unit MD Unit

Leverage <e-Adventure> LMS-orientation

MPEG7 is more powerful, but IEEE LOM is more extended in educational contexts

Our solution:

Use IEEE LOM for game-level metadata

Going mobile <m-Adventure> to be released early 2009

<m-Adventure> to be released early 2009

Going mobile MIST for mobile environments currently under development Mobile story-player Mobile edition of media metadata MD-annotation in media-acquisition time GPS cameras PDAs in field-work Fast consultation of experts

MIST for mobile environments currently under development

Mobile story-player

Mobile edition of media metadata

MD-annotation in media-acquisition time

GPS cameras

PDAs in field-work

Fast consultation of experts

Metadata for mobility New metadata dimension Device capabilities and profiles Available standards/specifications MPEG-7: Good for media, poor for hardware MPEG-21: Hardware descriptions W3C’s Composite Capabilities/Preferences Profiles (CC/PP) UAProf Wireless Universal Resource File (WURFL) …

New metadata dimension

Device capabilities and profiles

Available standards/specifications

MPEG-7: Good for media, poor for hardware

MPEG-21: Hardware descriptions

W3C’s Composite Capabilities/Preferences Profiles (CC/PP)

UAProf

Wireless Universal Resource File (WURFL)

…

Conclusions MIST + <e-Adventure> is: Web 2.0 creation of educational adventure games Enriched game-like interactive story-telling Story-driven educational gaming Metadata gathering and maintenance is a significant but necessary challenge Profesional annotation of metadata does not work Collaborative tagging does work (at least in the Web 2.0) Metadata for multimedia and education: complicated Different specifications for different purposes Metadata for mobility: complete mess Too many competing formats

MIST + <e-Adventure> is:

Web 2.0 creation of educational adventure games

Enriched game-like interactive story-telling

Story-driven educational gaming

Metadata gathering and maintenance is a significant but necessary challenge

Profesional annotation of metadata does not work

Collaborative tagging does work (at least in the Web 2.0)

Metadata for multimedia and education: complicated

Different specifications for different purposes

Metadata for mobility: complete mess

Too many competing formats

Future work Completing the integration between MIST and <e-Adventure> Two-way metadata interoperability Collaborative development of story-driven educational games Tap into RWTH Aachen’s vast repository of annotated media Mobile versions of both MIST and <e-Adventure> Assure the interoperability from the beginning Unification of mobile metadata approaches

Completing the integration between MIST and <e-Adventure>

Two-way metadata interoperability

Collaborative development of story-driven educational games

Tap into RWTH Aachen’s vast repository of annotated media

Mobile versions of both MIST and <e-Adventure>

Assure the interoperability from the beginning

Unification of mobile metadata approaches

Your turn…

Making sense of collaboratively annotated multimedia metadata for (mobile) digital story-telling and educational gaming First International Workshop on Story-Telling and Educational Games (STEG 2008) Maastricht, The Netherlands September 16th 2008 Pablo Moreno Ger , Marc Spaniol, Enrique López Mañas Niels Drobek, Baltasar Fernández-Manjón Universidad Complutense RWTH - Aachen Max Planck Institute