Presentation given at the Vrije Universiteit Brussel, Brussels, Belgium, April 2009 ABSTRACT: While there have been dramatic increases in the use of digital technologies for information storage, processing and delivery over the last few decades, the affordances of paper have ensured its retention as a key information medium. Despite predictions of the paperless office, paper is ever more present in our daily work. However, there is a gap between the paper and digital worlds: information present in paper documents cannot be transferred seamlessly to digital media and digital services are not easily accessible from the paper world. In this seminar talk I will present an information-centric approach for integrating paper with digital as well as physical media based on a general cross-media information platform (iServer). Some details about the architecture and implementation of the iServer platform as well as the underlying RSL metamodel for cross-media linking will be highlighted. A selection of interactive paper applications that have been developed based on this platform over the past few years will be presented, including the EdFest interactive paper guide for the Edinburgh festivals and the PaperPoint tool for paper-driven PowerPoint presentations. Challenges and solutions for novel forms of interactive paper and cross-media publishing are discussed based on the presented applications. The second part of my talk will address future directions for research on interactive paper and cross-media information management. This includes specific extensions of the iServer platform and RSL model as well as the application of our solution in new domains such as digital libraries, cross-media annotation and retrieval or personal cross-media information management that goes beyond the hierarchical information management imposed by the desktop metaphor.

1. The Remediation of Paper
Creating Affinities between Paper and
Cross-Media Information Spaces
Beat Signer
Department of Computer Science
Vrije Universiteit Brussel
Cross-Media Information Spaces, April 22, 2009

2. The Paperless Office
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

3. The Myth of the Paperless Office
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

4. Bridging the Paper-Digital Divide
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

5. EdFest Project
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

6. Digital Pen and Paper
 Anoto offers
 camera technology
 pattern license
(virtual paper space
of 60 million km2)
 Pen manufacturers
 Nokia
 Maxell
 Adapx
 Livescribe
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

7. EdFest Demo
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

8. PaperPoint Presentation Tool
 Mobile presentation tool
 Non-linear presentations
 Real-time annotations
 Digital whiteboard
 Multi-pen support
 brainstorming
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

9. Applications
 Enhanced Reading  Enhanced Writing
 interactive paper maps  photo album
 interactive festival brochures  researcher's notebook
 educational materials  mammography
 auction brochures  query by sketching
 scientific publications  …
 …
 Artists' Installations
 Paper-based Interfaces  Lost Cosmonaut
 PaperPoint  Generosa Enterprise
 document proof-editing
 digital libraries  Interactive Tabletops
 video analysis tool  photo browser
 educational games  collaborative sketch tool
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

10. Research
Link Metamodel (RSL) Tools
Cross-Media Server / Publishing Applications
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

11. Cross-Media Server (iServer)
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

12. Active Page Areas and Layers
image
video clip
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

13. RSL Metamodel and iServer
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

14. Structure
Links
Users
Layers
Plug-ins Active
Content
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

15. iServer Resource Plug-ins
Medium Resource Selector
paper document page shape
web page XHTML document XPointer
movie mpeg file, avi file etc. time span
movie mpeg file, avi file etc. shape
sound mp3 file, wav file etc. time span
image gif file, jpeg file etc. shape
physical object RFID space RFID tag
... ... ...
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

16. Active Components
 Execute code (services) on the client and server side
AC Manager
Active Component
ACS Manager A
Active Component Active Component
Stub B B
Request Sender
Client Device
Active Component
AC Names HTTP
C
iServer AC Names
Client Server
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

17. Active Components …
Button Image
Slider Sound
Slider2D Movie
DateSelector SQLClient
Browser XCMRequest
MapSelector ClientCtrlRequest
Rating PaperPointControl
Capture PaperPointAnnot
CaptureAndSpeak ...
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

18. Cross-Media Authoring and Publishing
 Manual link authoring
 Content-driven publishing
 e.g. EdFest brochure
 Automatic link authoring
 e.g. Print-n-Link
 Collaborative authoring
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

19. Manual Authoring Tool
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

20. Cross-Media Publishing Framework
iServer
cross-media link server
XML link
XCM iPublish definition
content publishing publishing framework
data
publish • paper link data
Structure Style • active areas on paper
• layers
• digital resources
import • user model
PDF
XML data XSL CSS
of festival
venues
and events Information-centric approach
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

21. EdFest Architecture Overview
Context XCM
Engine content publishing
platform
Context DB Metadata DB Appln DB
Text-to-Speech
Engine ICR Software
Client handwriting
Controller recognition
Active
iPaper Client iServer/iPaper
cross-media
Components
link server
Link DB
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

22. Print-n-Link
 Citations build the basis
for a scientific web of
publications
 Use digital services to
search / retrieve cited
publications
 Automatic authoring
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

23. Collaborative Authoring
web pages
consumers /
publishers
publisher
videos
books
presentation slides
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

24. Paper-Digital Proof-Editing (PaperProof)
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

25. iGesture Framework
 Gesture recognition tool
 create/test gesture sets
 create/test algorithms
 Device independent
 digital pen, tablet PC,
mouse, Wii remote, …
 Open Source
 initial release in Feb 2007
http://www.igesture.org
 >4500 downloads
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

26. Rough Guide
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

27. Interactive Table (iTable)
Pen-Based Google Earth Browser Pen-Based Drawing Tool
Photo Browser Photo Browser (Slider)
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

28. Interactive Table (iTable) …
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

29. Pulse Smartpen
 Record and replay
synchronised ink & audio
 1 GB memory
 16'000 pages
 200 hours voice recording
 96x18 OLED Display
 Open application platform
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

30. Ongoing and Future Work
 Cross-Media Information Spaces and
Architectures (CISA) research team
 cross-media lab space ("meeting room of the future")
 General and extensible representations for
cross-media information spaces
 interlinked structural components rather than
document-centric representation of information
 data no longer "owned" by applications
 replacement of traditional hierarchical file systems
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

31. CISA Research
 New forms of cross-media publishing
methodologies and tools
 global publishing infrastructures and architectures
 personal cross-media information management
 information sharing and collaborative authoring
 cross-media annotations
 Innovative multimodal user interfaces (digital
and physical) to cross-media information spaces
 includes further research on interactive paper
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

32. Conclusions
 Interactive paper framework (iPaper)
 information-centric approach supports different
technologies, functionalities and applications
 rapid prototyping (reusable active components)
 separation of authoring and programming
 Extensible platform for cross-media information
management (iServer) based on RSL metamodel
 Cross-media authoring and publishing tools
 manual, content-driven, automatic and collaborative
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

33. Acknowledgements
 GlobIS group @ ETH Zurich
 in particular Prof. Moira C. Norrie, Nadir Weibel and
Adriana Ispas
 Paper++ and PaperWorks members
 ... and many other collaborators on individual
projects
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

34. Further Information
 http://vub.academia.edu/BeatSigner
 Fundamental Concepts for Interactive Paper
and Cross-Media Information Spaces, Beat
Signer, ISBN 978-3-8370-2713-6, May 2008
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be