The document discusses the RSL hypermedia metamodel and its applications in creating cross-media solutions to improve information management and navigation across digital and physical environments. It highlights issues with current document formats and proposes solutions like the mindxpres presentation platform for enhanced cross-media interaction and content reuse. Various references to academic proceedings and frameworks are cited to support the development of these technologies.

1. 2 December 2005
The RSL Hypermedia Metamodel and Its
Application in Cross-Media Solutions
Prof. Beat Signer
beatsigner.com
Web & Information Systems Engineering Lab
Department of Computer Science
Vrije Universiteit Brussel

2. 01001000100
0101001100010010100010010
0100100010010001000100001001000
100010001100001001000110010101010110
1100010010100010010010010100010101011001
101001100001100 101010100
0010100 100 110101010101
00100 01010101000
1001 1111111111
111 0000000010
0 11111111
0101001011
010101010
111010101
001010010110
Cross-Media Information Spaces

3. Fragmentation of information
across different applications,
devices and digital & physical
information environments

4. Information
Systems &
Management
Information
Visualisation
& Navigation
Information
Visualisation
nd Navigation
Human-Machine &
Human-Information
Interaction
CISA
RSL
MindXpres
OC2
associative file system
data-driven storytelling
XIMA
cross-device interaction
iServer/iPaper
ViDaX
data physicalisation
OpenHPS
non-linear storytelling
PaperProof
iGesture
PaperSketch
TangHo
Context Modelling Toolkit
Midas
SpeeG2
PaperPoint
Print-n-Link
digital libraries
Mudra mixed reality
ArtVis
open cross-media linking
PimVis
DocTr
EdFest
source code visualisation
INFEX
technology-enhanced learning
Cross-Media Technologies WEB & INFORMATION
SYSTEMS ENGINEERING

5. Data-centric approach for
cross-media information
solutions based on a
hypermedia metamodel

6. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

7. RSL Hypermedia Metamodel & Link Server
B. Signer and M.C. Norrie, As We May Link: A General Meta-
model for Hypermedia Systems, Proceedings of ER 2007x

8. RSL Hypermedia Metamodel
B. Signer and M.C. Norrie, As We May Link: A General Meta-
model for Hypermedia Systems, Proceedings of ER 2007x

9. Interactive Paper (EdFest)
B. Signer, M. Grossniklaus and M.C. Norrie, Interactive Paper as a Mobile
Client for a Multi-Channel Web Information System WWWJ 10(4), 2007 x

10. PimVis Personal Information Management
S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

11. PimVis Setup
S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

12. PimVis Document View
S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

13. PimVis Focus View
S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

14. Object-Concept-Context (OC2) Model
object layer
structural link
concept layer
context layer
navigational link
associative link
weighted link
Object 1 Object n
Object 2
Context 1 Context n
Concept 1
Concept 2 Concept n
extent link
OC2 conceptual model
RSL-based metamodel
S. Trullemans and B. Signer, Towards a Conceptual Framework and Metamodel for Context-
aware Personal Cross-Media Information Management Systems, Proceedings of ER 2014

15. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

16. What is Wrong with Digital Documents?
Most people don't understand the logic of the concept:
"What You See Is What You Get" is based on printing
the document out ("get" means "get WHEN YOU
PRINT IT OUT"). And that means a metaphysical shift:
a document can only consist of what can be printed!
[...] No overlays [...] – PAPER UNDER GLASS.
Ted Nelson, Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009
Existing document formats often based on
the simulation of paper affordances on
desktop computers
B. Signer, What is Wrong with Digital Documents? A Conceptual Model for Struc-
tural Cross-Media Content Composition and Reuse, Proceedings of ER 2010

17. What is Wrong with Slideware?
▪ Simulation of physical slides
▪ limited space due to slide concept
▪ linear navigation
▪ monolithic documents
▪ difficult to reuse content and embed rich media types

18. MindXpres Presentation Platform
▪ Flexible representation
of presentations
▪ use of structural RSL links
▪ separation of content and
structure
▪ Extensible platform
▪ content-based approach
▪ cross-media content reuse
via transclusion
▪ non-linear navigation and
zoomable user interface
▪ associative linking
▪ rich media types
R. Roels and B. Signer, MindXpres: An Extensible Content-driven
Cross-Media Presentation Platform, Proceedings of WISE 2014

19. MindXpres Document Representation
R. Roels and B. Signer, A Conceptual Framework and Content Model for
Next Generation Presentation Solutions, PACMHCI 3(Issue EICS), 2019

20. MindXpres (Video)

21. Different context-dependent
adaptive overlay structures on top
of (parts of) content
Clear separation between content
and structure enables cross-media
content reuse (transclusion)

22. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

23. PimVis Personal Information Management
S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

24. eSPACE Framework for Hybrid UIs
A. Sanctorum and B. Signer, A Unifying Reference Framework and Model
for Adaptive Distributed Hybrid User Interfaces, Proceedings of RCIS 2019
x

25. XR-based Interaction
▪ Controlling individual IoT
devices
▪ Define relationships and
rules between IoT devices
▪ Linking any physical
artefact to other physical
artefacts or digital
information and services
▪ use of natural (pointing)
gestures and interactions
▪ “drag and drop” between
IoT devices
Elena Zambon
Tangible Holograms (TangHo)

26. The Lost Cosmonaut
▪ Writing as a collaborative
act of memory and story-
telling
▪ Reading, writing and
narrative as an act of
making sense
▪ Handwriting as a tool
for human-computer
interaction
In collaboration with Axel Vogelsang,
Artists in Labs Programme
A. Vogelsang and B. Signer, The Lost Cosmonaut: An Interactive Narrative En-
vironment on Basis of Digitally Enhanced Paper, Proceedings of ICVS 2005

27. Open Cross-Media Linking
local visual plug-ins
external visual plug-ins
communicate
Cross-Media Link Browser & Editor

28. Unified Cross-Media Information Spaces

29. Common underlying
representation of data, cross-
media links and structures via
RSL-based hypermedia engine
supporting legacy applications

30. RSL-based Hypermedia Engine
RSL-based Hypermedia Engine
Legacy Applications
File System Driver
Hypermedia-enabled Applications
HT2021.docx
HT2021.docx
…
B. Signer, R. Roels, R. van Barlingen and B. Willems, Back to the Future: Bringing Original Hyper-
media and Cross-Media Concepts to Modern Desktop Environments, Proceedings of Hypertext 2021

31. RSL-based Hypermedia Engine …
signer
My Pictures
Vancouver ER 2010
Granville
Island
Vancouver
Harbour
Public
Market
Keynote
J. Thorp
Olga and
Lamia
Dinner
ER 2010
multiple classification
structural link
navigational link
My Videos
Vancouver
Aquarium
ER 2010
Keynote
cross-media association
My Presentations
ER 2010 WISE 2009
cross-media
transclusion
single slide
transclusion
(via OOXML
RSL selector)

32. Next Generation File Manager
B. Signer, R. Roels, R. van Barlingen and B. Willems, Back to the Future: Bringing Original Hyper-
media and Cross-Media Concepts to Modern Desktop Environments, Proceedings of Hypertext 2021

33. System-wide hypermedia engine
at operating system level, enabling
cross-media information manage-
ment across application boundaries

34. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 34
June 6, 2023
References
▪ B. Signer, Fundamental Concepts for
Interactive Paper and Cross-Media Information
Spaces, Second Edition, ISBN 978-3-8370-2713-6,
August 2017
▪ B. Signer and M.C. Norrie, As We May Link: A General
Metamodel for Hypermedia Systems, Proceedings of
ER 2007, 26th International Conference on Conceptual
Modeling, Auckland, New Zealand, November 2007
▪ https://beatsigner.com/publications/signer_ER2007.pdf

35. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 35
June 6, 2023
References ...
▪ B. Signer, Towards Cross-Media Information
Spaces and Architectures, Proceedings of RCIS 2019,
13th International Conference on Research Challenges
in Information Science, Brussels, Belgium, May 2019
▪ https://beatsigner.com/publications/signer_RCIS2019.pdf
▪ B. Signer, M. Grossniklaus and M.C. Norrie,
Interactive Paper as a Mobile Client for a Multi-Channel
Web Information System, World Wide Web Journal
(WWW), Vol. 10, No. 4, Springer, December 2007
▪ https://beatsigner.com/publications/signer_WWWJ2007.pdf

36. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 36
June 6, 2023
References ...
▪ S. Trullemans, A. Sanctorum and B. Signer,
PimVis: Exploring and Re-finding Documents in Cross-
Media Information Spaces, Proceedings of AVI 2016,
International Working Conference on Advanced Visual
Interfaces, Bari, Italy, June 2016
▪ https://beatsigner.com/publications/trullemans_AVI2016.pdf
▪ S. Trullemans and B. Signer, Towards a Conceptual
Framework and Metamodel for Context-aware Personal
Cross-Media Information Management Systems, Pro-
ceedings of ER 2014, 33rd International Conference on
Conceptual Modelling, Atlanta, USA, October, 2014
▪ https://beatsigner.com/publications/trullemans_ER2014.pdf

37. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 37
June 6, 2023
References ...
▪ B. Signer, What is Wrong with Digital Documents?
A Conceptual Model for Structural Cross-Media Content
Composition and Reuse, Proceedings of ER 2010,
29th International Conference on Conceptual Modeling,
Vancouver, Canada, November 2010
▪ https://beatsigner.com/publications/signer_ER2010.pdf
▪ R. Roels and B. Signer, MindXpres: An
Extensible Content-driven Cross-Media Presentation
Platform, Proceedings of WISE 2014, 15th International
Conference on Web Information System Engineering,
Thessaloniki, Greece, October 2014
▪ https://beatsigner.com/publications/roels_WISE2014.pdf

38. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 38
June 6, 2023
References ...
▪ R. Roels and B. Signer, A Conceptual Frame-
work and Content Model for Next Generation
Presentation Solutions, Proceedings of the ACM on
Human-Computer Interaction (PACMHCI), 2019
▪ https://beatsigner.com/publications/roels_PACMHCI2019.pdf
▪ A. Sanctorum and B. Signer, A Unifying Reference
Framework and Model for Adaptive Distributed Hybrid
User Interfaces, Proceedings of RCIS 2019,
13th International Conference on Research Challenges
in Information Science, Brussels, Belgium, May 2019
▪ https://beatsigner.com/publications/sanctorum_RCIS2019.pdf

39. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 39
June 6, 2023
References ...
▪ A. Vogelsang and B. Signer, The Lost Cosmonaut:
An Interactive Narrative Environment on Basis of
Digitally Enhanced Paper, Proceedings of the Inter-
national Conference on Virtual Storytelling 2005,
Strasbourg, France, December 2005
▪ https://beatsigner.com/publications/vogelsang_ICVS2005.pdf
▪ A.A.O. Tayeh, Payam Ebrahimi and Beat Signer, Cross-
Media Document Linking and Navigation, Proceedings
of DocEng 2018, 18th ACM Symposium on Document
Engineering, Halifax, Canada, August 2018
▪ https://beatsigner.com/publications/tayeh_DocEng2018.pdf

40. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 40
June 6, 2023
References ...
▪ B. Signer, R. Roels, R. van Barlingen and
B. Willems, Back to the Future: Bringing Original
Hypermedia and Cross-Media Concepts to Modern
Desktop Environments, Proceedings of Hypertext 2021,
Virtual Event, August 2021
▪ https://beatsigner.com/publications/signer_Hypertext2021.pdf