The document discusses cross-media information spaces and architectures (CISA). It describes Prof. Beat Signer's research group which focuses on managing and interacting with information across media boundaries using a data-centric approach based on a hypermedia metamodel. The group develops technologies for cross-media linking, flexible document representation, and cross-media user interfaces. Several projects are mentioned, including systems for personal information management, content-driven presentations, interactive paper, and dynamic data physicalization.

1. 2 December 2005
Bridging the Gap: Managing and Interacting
with Information Across Media Boundaries
Prof. Beat Signer
beatsigner.com
Web & Information Systems Engineering Lab
Department of Computer Science
Vrije Universiteit Brussel

2. 01001000100
0101001100010010100010010
0100100010010001000100001001000
100010001100001001000110010101010110
1100010010100010010010010100010101011001
101001100001100 101010100
0010100 100 110101010101
00100 01010101000
1001 1111111111
111 0000000010
0 11111111
0101001011
010101010
111010101
001010010110
Cross-Media Information Spaces

3. Fragmentation of information
across different applications,
devices and digital & physical
information environments

4. Information
Systems &
Management
Information
Visualisation
& Navigation
Information
Visualisation
nd Navigation
Human-Machine &
Human-Information
Interaction
CISA
RSL
MindXpres
OC2
associative file system
data-driven storytelling
XIMA
cross-device interaction
iServer/iPaper
ViDaX
data physicalisation
OpenHPS
non-linear storytelling
PaperProof
iGesture
PaperSketch
TangHo
Context Modelling Toolkit
Midas
SpeeG2
PaperPoint
Print-n-Link
digital libraries
Mudra mixed reality
ArtVis
open cross-media linking
PimVis
DocTr
EdFest
source code visualisation
INFEX
technology-enhanced learning
Cross-Media Technologies WEB & INFORMATION
SYSTEMS ENGINEERING

5. Data-centric approach for
cross-media information
solutions based on a
hypermedia metamodel

6. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

7. RSL Hypermedia Metamodel & Link Server
B. Signer and M.C. Norrie, As We May Link: A General Meta-
model for Hypermedia Systems, Proceedings of ER 2007x

8. RSL Hypermedia Metamodel
B. Signer and M.C. Norrie, As We May Link: A General Meta-
model for Hypermedia Systems, Proceedings of ER 2007x

9. Interactive Paper (EdFest)
B. Signer, M. Grossniklaus and M.C. Norrie, Interactive Paper as a Mobile
Client for a Multi-Channel Web Information System WWWJ 10(4), 2007 x

10. PimVis: Personal Information Management
S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

11. PimVis Setup
S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

12. PimVis Document View
S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

13. PimVis Focus View
S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

14. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

15. What is Wrong with Digital Documents?
Most people don't understand the logic of the concept:
"What You See Is What You Get" is based on printing
the document out ("get" means "get WHEN YOU
PRINT IT OUT"). And that means a metaphysical shift:
a document can only consist of what can be printed!
[...] No overlays [...] – PAPER UNDER GLASS.
Ted Nelson, Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009
Existing document formats often based on
the simulation of paper affordances on
desktop computers

16. What is Wrong with Slideware?
▪ Simulation of physical slides
▪ limited space due to slide concept
▪ linear navigation
▪ monolithic documents
▪ difficult to reuse content and embed rich media types

17. MindXpres Presentation Platform
▪ Flexible representation
of presentations
▪ use of structural RSL links
▪ separation of content and
structure
▪ Extensible platform
▪ content-based approach
▪ cross-media content reuse
via transclusion
▪ non-linear navigation and
zoomable user interface
▪ associative linking
▪ rich media types
R. Roels and B. Signer, MindXpres: An Extensible Content-driven
Cross-Media Presentation Platform, Proceedings of WISE 2014

18. MindXpres Document Representation
R. Roels and B. Signer, A Conceptual Framework and Content Model for
Next Generation Presentation Solutions, PACMHCI 3(Issue EICS), 2019

19. MindXpres (Video)

20. Different context-dependent
adaptive overlay structures on top
of (parts of) content
Clear separation between content
and structure enables cross-media
content reuse (transclusion)

21. Cross-Media Linking
Flexible Document Representation
Cross-Media User Interfaces

22. PimVis: Paper-Digital User Interface
S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

23. ArtVis Tangible User Interface
B. Dumas, B. Moerman, S. Trullemans and B. Signer, ArtVis: Combining Advanced Visualisation and Tangible
Interaction for the Exploration, Analysis and Browsing of Digital Artwork Collections, Proceedings of AVI 2014

24. Dynamic Data Physicalisation
B. Signer, P. Ebrahimi, T.J. Curtin and A.K.A. Abdullah, Towards a Framework for Dynamic Data
Physicalisation, International Workshop Toward a Design Language for Data Physicalization, 2018

25. eSPACE Framework for Hybrid UIs
A. Sanctorum and B. Signer, A Unifying Reference Framework and Model
for Adaptive Distributed Hybrid User Interfaces, Proceedings of RCIS 2019
x

26. Unified Cross-Media Information Spaces

27. Common underlying
representation of data, cross-
media links and structures via
RSL-based hypermedia engine
supporting legacy applications

28. RSL-based Hypermedia Engine
RSL-based Hypermedia Engine
Legacy Applications
File System Driver
Hypermedia-enabled Applications
HT2021.docx
HT2021.docx
…
B. Signer, R. Roels, R. van Barlingen and B. Willems, Back to the Future: Bringing Original Hyper-
media and Cross-Media Concepts to Modern Desktop Environments, Proceedings of Hypertext 2021

29. RSL-based Hypermedia Engine …
signer
My Pictures
Vancouver ER 2010
Granville
Island
Vancouver
Harbour
Public
Market
Keynote
J. Thorp
Olga and
Lamia
Dinner
ER 2010
multiple classification
structural link
navigational link
My Videos
Vancouver
Aquarium
ER 2010
Keynote
cross-media association
My Presentations
ER 2010 WISE 2009
cross-media
transclusion
single slide
transclusion
(via OOXML
RSL selector)

30. Next Generation File Manager
B. Signer, R. Roels, R. van Barlingen and B. Willems, Back to the Future: Bringing Original Hyper-
media and Cross-Media Concepts to Modern Desktop Environments, Proceedings of Hypertext 2021

31. System-wide hypermedia engine
at operating system level, enabling
cross-media information manage-
ment across application boundaries

32. References
▪ B. Signer, Fundamental Concepts for
Interactive Paper and Cross-Media Information
Spaces, Second Edition, ISBN 978-3-8370-2713-6,
August 2017
▪ B. Signer and M.C. Norrie, As We May Link: A General
Metamodel for Hypermedia Systems, Proceedings of
ER 2007, 26th International Conference on Conceptual
Modeling, Auckland, New Zealand, November 2007
▪ https://beatsigner.com/publications/signer_ER2007.pdf

33. References ...
▪ B. Signer, M. Grossniklaus and M.C. Norrie,
Interactive Paper as a Mobile Client for a Multi-Channel
Web Information System, World Wide Web Journal
(WWW), Vol. 10, No. 4, Springer, December 2007
▪ https://beatsigner.com/publications/signer_WWWJ2007.pdf
▪ S. Trullemans, A. Sanctorum and B. Signer,
PimVis: Exploring and Re-finding Documents in Cross-
Media Information Spaces, Proceedings of AVI 2016,
International Working Conference on Advanced Visual
Interfaces, Bari, Italy, June 2016
▪ https://beatsigner.com/publications/trullemans_AVI2016.pdf

34. References ...
▪ R. Roels and B. Signer, MindXpres: An
Extensible Content-driven Cross-Media Presentation
Platform, Proceedings of WISE 2014, 15th International
Conference on Web Information System Engineering,
Thessaloniki, Greece, October 2014
▪ https://beatsigner.com/publications/roels_WISE2014.pdf
▪ R. Roels and B. Signer, A Conceptual Framework and
Content Model for Next Generation Presentation
Solutions, Proceedings of the ACM on Human-Computer
Interaction (PACMHCI), 2019
▪ https://beatsigner.com/publications/roels_PACMHCI2019.pdf

35. References ...
▪ B. Signer, What is Wrong with Digital
Documents? A Conceptual Model for Structural Cross-
Media Content Composition and Reuse, Proceedings of
ER 2010, 29th International Conference on Conceptual
Modeling, Vancouver, Canada, November 2010
▪ https://beatsigner.com/publications/signer_ER2010.pdf
▪ B. Dumas, B. Moerman, S. Trullemans and B. Signer,
ArtVis: Combining Advanced Visualisation and Tangible
Interaction for the Exploration, Analysis and Browsing of
Digital Artwork Collections, Proceedings of AVI 2014,
Como, Italy, May 2014
▪ https://beatsigner.com/publications/dumas_AVI2014.pdf

36. References ...
▪ B. Signer, P. Ebrahimi, T.J. Curtin and Ahmed
K.A. Abdullah, Towards a Framework for Dynamic Data
Physicalisation, International Workshop Toward a Design
Language for Data Physicalisation, Berlin, Germany,
October 2018
▪ https://beatsigner.com/publications/signer_DataPhys2018.pdf
▪ A. Sanctorum and B. Signer, A Unifying Reference
Framework and Model for Adaptive Distributed Hybrid
User Interfaces, Proceedings of RCIS 2019, 13th
International Conference on Research Challenges in
Information Science, Brussels, Belgium, May 2019
▪ https://beatsigner.com/publications/sanctorum_RCIS2019.pdf

37. References ...
▪ B. Signer, R. Roels, R. van Barlingen and
B. Willems, Back to the Future: Bringing Original
Hypermedia and Cross-Media Concepts to Modern
Desktop Environments, Proceedings of Hypertext 2021,
Virtual Event, August 2021
▪ https://beatsigner.com/publications/signer_Hypertext2021.pdf

38. Prof. Dr. Beat Signer
Cross-MediaTechnology, Interac-
tive Paper, Data Physicalisation
Dr. Audrey Sanctorum
User-defined XDI and IoT Inter-
action, Human-AI Interaction
CISA
Human-Machine &
Human-Information
Interaction
Information
Systems &
Management
Information
Visualisation
& Navigation
WEB & INFORMATION
SYSTEMS ENGINEERING
CROSS-MEDIA INFORMATION SPACES
AND ARCHITECTURES (CISA)
Maxim Van de Wynckel
Hybrid Positioning, Implicit
Human-Computer Interaction
Yoshi Malaise
Technology-enhanced Learning,
Content-driven Presentations

39. WEB & INFORMATION
SYSTEMS ENGINEERING
CISA
Human-Machine &
Human-Information
Interaction
Information
Systems &
Management
Information
Visualisation
& Navigation
CROSS-MEDIA INFORMATION SPACES
AND ARCHITECTURES (CISA)
Ekene Attoh
IoT Middleware, Context-aware
Computing, Implicit HCI
Isaac Valadez
Knowledge Physicalisation and
Augmentation, Tangible UIs
Xuyao Zhang
Extensible Platform for Dynamic
Data Physicalisation
Ingela Rossing
Dynamic Data Physicalisation
Framework and Guidelines

40. WEB & INFORMATION
SYSTEMS ENGINEERING
CISA
Human-Machine &
Human-Information
Interaction
Information
Systems &
Management
Information
Visualisation
& Navigation
CROSS-MEDIA INFORMATION SPACES
AND ARCHITECTURES (CISA)
Arun Sojan
Physicalisation for Digital Twins
Piet Van Der Paelt
Julia-based Framework for
Simulation and Optimisation
Evan Cole
Technology-enhanced Learning,
Study Lenses
Migdeily Cantera
End-User Development, Mixed
Reality IoT UIs, Intelligibility