Webinar given at icity Lab Talks - The Digital Value Chain In this talk, I will first provide an overview of the lab’s research on a general data-driven approach for cross-media information system and architectures based on the resource-selector-link (RSL) hypermedia metamodel. We will then have a look at several cross-media applications for personal information management and next-generation presentation solutions (MindXpres). Finally, I will outline the lab’s most recent research on tangible interaction and dynamic data physicalisation.

1. 2 December 2005
Cross-Media Technologies and Applications
Future Directions for Personal Information Management
Prof.Dr.Beat Signer
beatsigner.com
Web & Information Systems Engineering Lab
Department of Computer Science
Vrije Universiteit Brussel
WEB & INFORMATION
SYSTEMS ENGINEERING
speakerdeck.com/signer/icity

2. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022

3. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
01001000100
0101001100010010100010010
0100100010010001000100001001000
100010001100001001000110010101010110
1100010010100010010010010100010101011001
101001100001100 101010100
0010100 100 110101010101
00100 01010101000
1001 1111111111
111 0000000010
0 11111111
0101001011
010101010
111010101
001010010110
Cross-Media Information Spaces and Architectures

4. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
Information
Systems &
Management
Information
Visualisation
& Navigation
Information
Visualisation
nd Navigation
Human-Machine &
Human-Information
Interaction
CISA
RSL [4]
MindXpres [8]
OC2 [5]
associative file system [2]
data-driven storytelling [11]
XIMA [13]
cross-device interaction [9]
iServer/iPaper [3]
ViDaX [16]
data physicalisation [32]
OpenHPS [18]
non-linear storytelling [27]
PaperProof [29]
iGesture [23]
PaperSketch [30]
TangHo [31]
Context Modelling Toolkit [10]
Midas [21]
SpeeG2 [24]
PaperPoint [26]
Print-n-Link [17]
digital libraries [12]
Mudra [22] mixed reality [28]
ArtVis [15]
open cross-media linking [1]
PimVis [14]
DocTr [6]
EdFest [25]
source code visualisation [19]
INFEX [7]
technology-enhanced learning [20]
Cross-Media Information Spaces and Architectures

5. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
RSL Hypermedia Metamodel
Global Information Systems Group, ETH Zurich

6. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
Personal Information Management (PIM)
▪ Keeping, organising and
re-finding information
▪ digital and physical
▪ Study of human-
information interaction
▪ files, piles, mixtures, …
▪ Cross-Media PIM system
▪ explicit as well as implicit
associations between entities

7. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
PimVis Setup

8. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
Mixed Reality in IoT Environments
▪ eSPACE IoT platform
▪ end-user authoring
▪ mashup tool for digital and
physical (IoT) components
▪ Context Modelling Toolkit
(CMT)
▪ OpenHPS hybrid
positioning system
▪ stream-based processing
▪ combination of existing and
new positioning techniques
▪ decentralised sensor fusion
Elena Zambon

9. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
Interactive Paper and Tangible Interfaces
▪ Bridging the paper-digital
divide
▪ seamless transition
▪ multimodal interaction
▪ new interaction and design
patterns
▪ Advanced visualisation
techniques in combination
with tangible user
interfaces (TUI)
▪ e.g. ArtVis
Bram Moerman
Global Information Systems Group, ETH Zurich

10. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
Tangible Hologram (TangHo) Prototype

11. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
Collaborative TangHo Interaction

12. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
Dynamic Data Physicalisation
▪ Physical objects used for
input as well as output
▪ Dynamic data physicali-
sation
▪ use physical variables such
as temperature or texture
▪ exploration of big data sets
▪ dynamic data physicalisation
framework
- data physicalisation grammar

13. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
What is Wrong with Slideware?
▪ Simulation of physical slides
▪ limited space due to the slide concept
▪ linear navigation from slide to slide
▪ difficult to reuse content and embed rich media types
▪ MindXpres addresses these issues
▪ unlimited canvas with zoomable user interface
▪ non-linear navigation and associative linking
▪ content-based approach with automatic visualisation

14. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
MindXpres Presentation Platform
▪ Extensible prototyping
platform for novel presen-
tation concepts
▪ content-based approach
- separation of content and presen-
tation (automatic visualisation)
▪ cross-media content reuse
▪ non-linear navigation via
zoomable user interface
▪ connectivity and interactivity
▪ Rich-media plug-ins
▪ e.g. source code or
interactive data visualisation

15. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
Interactive Source Code Plug-in
Paul Mestereaga

16. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
Personalised Learning Environments
▪ Suggestion of learning
content
▪ knowledge graphs
▪ user knowledge graph
▪ learning paths
▪ Diagnostic assessments
to detect knowledge gaps
▪ Automatic content adap-
tation
▪ based on user preferences
and constraints
Enhanced Curriculum
Assessments
Tagged Learning Content
Knowledge Graphs
Adaptive Learning
Content
Learning Paths
Knowledge Representation Framework
Assessment Module
Assessment Analyser Knowledge Gap Analyser
Adaptive Learning Environment
Suggestion Module
User Knowledge
Graph
Personalised
Presentation Module
Learner Constraints
Groups Module

17. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
RSL-based Hypermedia Engine
RSL-based Hypermedia Engine
Legacy Applications
File System Driver
Hypermedia-enabled Applications
HT2021.docx
HT2021.docx
…

18. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
Models, Frameworks and Platforms
dynamic data physicalisation framework
RSL hypermedia metamodel eSPACE IoT platform iGesture framework
OpenHPS hybrid positioning system MindXpres presentation platform

20. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
References
[4] B. Signer and M.C. Norrie, As We May Link: A General
Metamodel for Hypermedia Systems, Proceedings of
ER 2007, 26th International Conference on Conceptual
Modeling, Auckland, New Zealand, November 2007
▪ https://beatsigner.com/publications/signer_ER2007.pdf
[8] R. Roels and B. Signer, MindXpres: An Exten-
sible Content-driven Cross-Media Presentation
Platform, Proceedings of WISE 2014, 15th Interna-
tional Conference on Web Information System
Engineering, Thessaloniki, Greece, October, 2014
▪ https://beatsigner.com/publications/roels_WISE2014.pdf

21. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
References ...
[10] S. Trullemans and B. Signer, A Multi-layered
Context Modelling Approach for End Users, Expert
Users and Programmers, Proceedings of SERVE
2016, International Workshop on Smart Ecosystems
cReation by Visual dEsign, Bari, Italy, June 2016
▪ https://beatsigner.com/publications/trullemans_SERVE2016.pdf
[14] S. Trullemans, A. Sanctorum and B. Signer,
PimVis: Exploring and Re-finding Documents in Cross-
Media Information Spaces, Proceedings of AVI 2016,
International Working Conference on Advanced Visual
Interfaces, Bari, Italy, June 2016
▪ https://beatsigner.com/publications/trullemans_AVI2016.pdf

22. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
References ...
[15] B. Dumas, B. Moerman, S. Trullemans and
B. Signer, ArtVis: Combining Advanced Visualisation
and Tangible Interaction for the Exploration, Analysis
and Browsing of Digital Artwork Collections, Proceed-
ings of AVI 2014, Como, Italy, May 2014
▪ https://beatsigner.com/publications/dumas_AVI2014.pdf
[18] M. Van de Wynckel and B. Signer, OpenHPS: An
Open Source Hybrid Positioning System, Technical
Report WISE Lab, WISE-2020-01, December 2020
▪ https://beatsigner.com/publications/vanDeWynckel_CoRR2020.pdf

23. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
References ...
[19] R. Roels, P. Mestereaga and B. Signer, An
Interactive Source Code Visualisation Plug-in for the
MindXpres Presentation Platform, Communications in
Computer and Information Science (CCIS), 583, 2016
▪ https://beatsigner.com/publications/roels_CCIS2016.pdf
[20] Y. Malaise and B. Signer, Personalised Learning
Environments Based on Knowledge Graphs and the
Zone of Proximal Development, Proceedings of
CSEDU 2022, International Conference on Computer
Supported Education, Online Event, April 2022
▪ https://beatsigner.com/publications/malaise_CSEDU2022.pdf

24. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
References ...
[15] B. Dumas, B. Moerman, S. Trullemans and
B. Signer, ArtVis: Combining Advanced Visualisation
and Tangible Interaction for the Exploration, Analysis
and Browsing of Digital Artwork Collections, Proceed-
ings of AVI 2014, Como, Italy, May 2014
▪ https://beatsigner.com/publications/dumas_AVI2014.pdf
[18] M. Van de Wynckel and B. Signer, OpenHPS: An
Open Source Hybrid Positioning System, Technical
Report WISE Lab, WISE-2020-01, December 2020
▪ https://beatsigner.com/publications/vanDeWynckel_CoRR2020.pdf

25. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
References ...
[25] B. Signer, M. Grossniklaus and M.C. Norrie,
Interactive Paper as a Mobile Client for a Multi-
Channel Web Information System, World Wide Web
Journal (WWW), Vol. 10, No. 4, Springer,
December 2007
▪ https://beatsigner.com/publications/signer_WWWJ2007.pdf
[28] L. Hoste and B. Signer, Expressive Control of Indirect
Augmented Reality During Live Music Performances,
Proceedings of NIME 2013, 13th International Confe-
rence on New Interfaces for Musical Expression,
Daejeon, Korea Republic, May 2013
▪ https://beatsigner.com/publications/hoste_NIME2013.pdf

26. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
References ...
[31] B. Signer and T.J. Curtin, Tangible Holograms:
Towards Mobile Physical Augmentation of Virtual
Objects, Technical Report WISE Lab, WISE-2017-01,
March 2017
▪ https://beatsigner.com/publications/signer_arXiv2017.pdf
[32] B. Signer, P. Ebrahimi, T.J. Curtin and Ahmed
K.A. Abdullah, Towards a Framework for Dynamic
Data Physicalisation, International Workshop Toward
a Design Language for Data Physicalisation, Berlin,
Germany, October 2018
▪ https://beatsigner.com/publications/signer_DataPhys2018.pdf

27. Beat Signer - Department of Computer Science - bsigner@vub.be
June 9, 2022
References ...
▪ B. Signer, Towards Cross-Media Information
Spaces and Architectures, Proceedings of RCIS 2019,
Brussels, Belgium, May 2019
▪ https://beatsigner.com/publications/signer_RCIS2019.pdf
▪ MindXpres Project
▪ https://mindxpres.com
▪ OpenHPS
▪ https://openhps.org
▪ iGesture
▪ http://igesture.org