This presentation by Elena Not (FBK) and Daniela Petrelli (SHU) has been shown at the 6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013) , which was co-located with the 21st conference on User Modeling, Adaptation, and Personalization (UMAP 2013). The research presented here is part of the meSch project. The project (2013-2016) receives funding from the European Community’s Seventh Framework Programme ‘ICT for access to cultural resources’ (ICT Call 9: FP7-ICT- 2011-9) under the Grant Agreement 600851. See: http://mesch-project.eu/

1. 6th International Workshop on Personalized Access to Cultural Heritage
PATCH2013 @ UMAP2013 – Roma, 14th June 2013
Curators in the Loop: a Quality Control Process
for Personalization for Tangible Interaction
in Cultural Heritage
Not Elena
Fondazione
Bruno Kessler
Daniela Petrelli
Sheffield Hallam
University

2. Summary
• Tangible interaction as a key to unlock the
synergy between digital and physical
• The meSch scenario
• Personalization challenges
• A pivotal role for authors
• The meSch research agenda
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

3. Challenges for personalization deployed in cultural sites
• Portable devices and displays are an
opportunity, but divert attention
• Personal devices interfere with the natural
social flow
• Difficult trade-off between deep
personalization and effort of content
preparation
• Investment on hardware and software
architectures
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

4. Objectives of the meSch project
<< To bridge digital content and the materiality
of cultural heritage through tangible and
embedded interaction that create personally
meaningful, sensorily rich, and socially
expanded visitor experiences. >>
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

5. The physical back at the center
<< A cultural space with smart objects,
each with their own digital content
embedded therein,
which will be revealed if and when
conditions are right,
for example, when visitors have reached
the proper time in the storyline,
or a group of them is acting in a certain
way, or another smart object is close by. >>
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

6. Creating personalized experiences for a cultural space
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

7. Shift the focus from visitors to curators
• Address the overall cycle of personalization:
from creation, to delivery onsite and online, to
feedback to creation
– Exploit tangible interaction to bridge the gap
between digital, material and social
– Put curators in control of authoring and adaptivity,
both in content and in context
– Reuse experience of other visitors and authors to
improve the system
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

8. Interaction through smart objects
• Direct manipulation of augmented exhibits
– Miniaturized multisensory integration
platforms
[Petrelli et al., CHI 2010]
– Consumer electronics
[Petrelli et al., DIS 2012]
– Simple tags
Replicas supporting
interaction
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

9. Interaction through smart objects (2)
• Facilitating objects
– To activate
functionalities for small,
big, fragile exhibits
– To provide interaction
context
– To guide
Replicas supporting
interaction
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

10. Interaction through smart objects (3)
• Ecology of smart objects
– Distributed stories
– Context-aware network
• Opportunities for social
interaction
– Sharing of presentations
– Co-action
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

11. Adaptivity of content and interaction
• The complexity of creating an adaptive TI
experience:
– Conception of the experience
– Selection of (alternative) narrative threads and
content
– Selection of unlocking (inter-)actions
– Micro-decisions during contextual onsite delivery
– Adaptation of follow-up online exploration
• Tackling complexity: The curator and the system
to share the task
– The benefits of human judgment
– The strengths of automatic mechanisms
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

12. Where personalization technology helps
Recommendation of
relevant material,
exploiting onsite
interaction
What other visitors/
authors have selected
is used to refine the
system’s behaviour
Adaptive lenses for
finding the
multimedia material
for the authoring
task (content
filtering and
recommendation)
Contextual
delivery of
adaptive content
and interaction
Templates and instantiation
rules to compose adaptive
content + interaction
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

13. A necessary decoupling
Personalization of content Personalization in context
which content is more
which interaction
interesting for which
mechanisms are more
people
engaging and for whom
• To manage complexity
• To distribute authoring tasks according to
expertise
• To foster flexibility and reusability
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

14. A necessary decoupling
Treasure hunt in a historic museum
Treasure hunt in a science museum
– multiple stories for the same interaction
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

15. A necessary decoupling
Simply unfolding a map
Looking for companions holding the map of
the opposing army and joining the pieces
– same story for multiple interactions
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

16. Recommended
content for online
interaction
LAYER 1
Digital Content search/
recommendation
Retrieved content
for authoring task
Digital archives
Data Integration and
Access Component
• Content- and collaborative-based recommendation
 similarity/novelty with respect to what already selected
for presentation
 query relaxations to suggest alternative content
 similarity with respect to other filled-in templates
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)
 similarity with respect to media/format/size
ADAPTIVITY OF CONTENT
Supporting personalization in content

17. Recommended
content for online
interaction
LAYER 1
Digital Content search/
recommendation
Retrieved content
for authoring task
Digital archives
LAYER 2
Support for Narration design
Adaptive narrative
structure
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)
Data Integration and
Access Component
ADAPTIVITY OF CONTENT
Supporting personalization in content (2)

18. Layer 2 – support for narration design
• Rules and templates to guide content
composition
– choose the style/genre of the story (e.g.
emotional, anecdotal, factual, …)
– Micro-activation networks to:
• orchestrate the presence of alternative narrative
threads in the same object
• how a story can split over different objects
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

19. Layer 2 – support for narration design (2)
• Co-design with curators pre-packaged
schemas, e.g.:
– alternative object interpretations (e.g., historical
vs. artistic description; functional vs. fabrication
info)
– skeletons for narratives based on a temporal
sequence (e.g., the life stages of a historical
character)
– reflecting a certain topic organization (e.g.,
comparison of different making techniques)
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

20. Recommended
content for online
interaction
LAYER 1
Digital Content search/
recommendation
Retrieved content
for authoring task
Digital archives
LAYER 2
Support for Narration design
Adaptive narrative
structure
LAYER 3
Support for Experience design
Adaptive experience
structure
LAYER 4
Context-aware adaptive instantiation
Personalized experience delivery
through smart exhibits
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)
Data Integration and
Access Component
ADAPTIVITY IN CONTEXT ADAPTIVITY OF CONTENT
Supporting personalization in context

21. Layer 3 – support for experience design
• Rules and templates to guide interaction
composition
– Dictionary of available (inter)actions to release
content
•
•
•
•
Interaction by object manipulation
Interaction by movement
Interaction by co-presence
Interaction by co-activities
– Micro-activation networks to:
• orchestrate the presence of alternative interactive behaviour
in the same object
• How different objects contribute to the same interaction
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

22. Layer 3 – support for experience design (2)
• Co-design with exhibition designers/artists
pre-packaged schemas:
– the visitor goes to the object vs. the object goes
with the visitor
– extensive use of collaborative multi-user actions
(such as people marching in line or joining pieces)
– object manipulation (e.g., with content disclosed
by varied or prolonged manipulations)
– objects search (e.g. in a treasure hunt or to find
your enemy to unlock the full story of the battle)
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

23. Layer 4 – context-aware adaptive instantiation
• Low-level adaptivity decisions to be taken
autonomously by the augmented objects
according to the specific interaction context
• Mediation strategies
– Conflicting content decisions (e.g., several visitors
following different narrative threads are close to
the object at the same time)
– Different valid (inter)actions (e.g., pick up/put in
the basket; one person moving / several persons
moving)
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

24. meSch research plan
 concept generation and sketching-in-hardware
what a personalized visit can be
 analyzing the technical requirements of the
multilayer personalization architecture
• unpack the process curators and artists go
through when creating a new exhibition
• define and test various classes of content and
interaction rules and pre-packaged schemas
• co-design the authoring tools with heritage
professionals
• implement the multipurpose personalization
services
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

25. Conclusion
• Tangible interaction favors the synergy between digital
and material
– Preserves the centrality of the visit experience
– Encourages social interaction
– Supports forms of personalization in content and
interaction
• Pivotal role of curators
– Technology helps coping with complexity
– Human-supervision assures quality
• Flexible personalization component
– serve different personalization tasks
– portable to different content and (inter)actions
vocabularies
– reusable in different physical sites.
6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013)

26. Not Elena
Fondazione Bruno Kessler
not@fbk.eu
The project (2013-2016) receives funding from the European Community’s Seventh
Framework Programme ‘ICT for access to cultural resources’ (ICT Call 9: FP7-ICT2011-9) under the Grant Agreement 600851.
Daniela Petrelli
Sheffield Hallam University
D.Petrelli@shu.ac.uk