Integrating Paper and Digital Information Spaces
The iServer / iPaper Solution
Beat Signer
Global Information Systems Group
ETH Zurich

http://vub.academia.edu/BeatSigner




                                             31 March 2008
Paper in Everyday Settings




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
Input Devices




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
Digital Pen and Paper
 Developed by Anoto
 Different pen manufacturers
               Logitech, Maxel, Nokia,
                Adapx, Livescribe ...

 Pattern space
               60 million km2




31 March 2008                      Beat Signer <signer@inf.ethz.ch> / ETH Zurich             4
                                                                                   DFKI Seminar
Interactive Paper
                                                 Linking active areas to resources

                                                                   image




                                                                video clip




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich                DFKI Seminar
iPaper @ ETH Zurich
                                                                             Tools
                   Cross-Media
                Link Infrastructure




                                                                             Applications
                         Publishing




31 March 2008                Beat Signer <signer@inf.ethz.ch> / ETH Zurich                  DFKI Seminar
Interactive Paper Applications
 Enhanced Reading                                           Enhanced Writing
               interactive paper maps                                   proof-editing system
               interactive festival brochures                           photo album
               educational materials                                    researcher's notebook
               auction brochures                                        mammography
               scientific publications                                  search system
               …                                                        …
 Paper-Based Interfaces                                     Artists' Installations
         PaperPoint                                                  Lost Cosmonaut
         image retrieval / digital libraries                         Generosa Enterprise
         video analysis tool
         educational games                                  Interactive TableTops
                     http://www.globis.ethz.ch/research/paper/applications/
31 March 2008                         Beat Signer <signer@inf.ethz.ch> / ETH Zurich               DFKI Seminar
PaperPoint Presentation Tool
                                       Mobile presentation tool
                                       Non-linear presentations
                                       Paper-based real-time
                                             annotations

                                       Digital whiteboard
                                       Multi-pen support
                                                   brainstorming



31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich       DFKI Seminar
EdFest Project




31 March 2008    Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
EdFest Demo




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
EdFest Architecture Overview
                                                     Context                      XCM
                                                                             content publishing
                                                     Engine                       platform




                                                    Context DB            Metadata DB Appln DB
                 Text-to-Speech
                 Engine                              Client
                                                                                       ICR Software
                                                    Controller                         handwriting
                                                                                       recognition

                                                iServer/iPaper                 Active
                iPaper Client
                                                    cross-media              Components
                                                     link server




                                                      Link DB


31 March 2008             Beat Signer <signer@inf.ethz.ch> / ETH Zurich                     DFKI Seminar
Content Publishing Framework
                                                                                  iServer / iPaper
                                                                                 cross-media link server


         XCM                                                        XML link
                            iPublish
                                                                    definition
content publishing   publishing framework
                                                                    data
                                                                                      •   paper link data
                                                        publish                       •   active areas
                     Structure       Style                                            •   layers
                                                                                      •   digital resources
                                                                                      •   user model
   import                                                               PDF




     XML data of      XSL             CSS
   festival venues
     and events

31 March 2008                    Beat Signer <signer@inf.ethz.ch> / ETH Zurich                        DFKI Seminar
Design of Interactive Paper Documents




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
Rough Guide




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
PaperProof




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
Paper-Digital Structural Mapping




 Dual document representation in digital and
     physical document spaces

31 March 2008      Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
Print-n-Link
                                       Citations build the basis
                                             for a scientific web of
                                             publications

                                       Use digital services to
                                             search / retrieve cited
                                             publications

                                       Automatic authoring


31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich          DFKI Seminar
Other iPaper Applications …




                   PaperWeb                                               Natural History Museum




         Lost Cosmonaut (Axel Vogelsang)                                     iTable (Google Earth)

31 March 2008                     Beat Signer <signer@inf.ethz.ch> / ETH Zurich                      DFKI Seminar
Different Types of Link Authoring
 Manual authoring of links
 Content publishing framework for Interactive Paper
               EdFest brochure – PDF and links generated from
                publishing database
 Automatic authoring of links based on document
     analysis (PDF)
               Print-n-Link system for scientific publications
 Collaborative authoring based on P2P link sharing

31 March 2008                   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
Digital Link Authoring Tool




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
iPaper Architecture




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
RSL Metamodel and Resource Plug-ins
                                                                       link

                                             Entities



                                Selectors               Resources




                 Shape          Page                              Query          DB Object
                Time Span /    Movie Clip                         XPointer           XHTML
                                                                                 Data/Metadata
                Spatial Info                                                       Document



         iPaper                                         Application Database
           Movie Clips                                   HTML Documents


31 March 2008                    Beat Signer <signer@inf.ethz.ch> / ETH Zurich                   DFKI Seminar
Structure


         Links

                                                                                 Users




         Layers
                            Plug-ins                                          Active
                                                                              Content




31 March 2008                 Beat Signer <signer@inf.ethz.ch> / ETH Zurich        DFKI Seminar
RSL Links




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
RSL Structural Links




31 March 2008   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
RSL Structural Links …
 Structure over data
               e.g. regular document (chapters, sections, ...)
               reuse of resources  transclusion (Ted Nelson)
               context-dependant structures
 Structure over structures
               structural composition of data and substructures can
                be combined




31 March 2008                   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
RSL Structural Links …
 Structure over links
               put different navigational links in relation to each other
               e.g. single structural link to define linear order over
                set of links  trails and tours




31 March 2008                    Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
Active Content
 Execute code (services) on the client and server side

                                                                                                AC Manager
                                                                                              Active Component
                            ACS Manager                                                               A

                           Active Component                                                   Active Component
                                 Stub B                                                               B
                                                       Request Sender
           Client Device




                                                                                              Active Component
                               AC Names                                 HTTP
                                                                                                      C


                                                                                   iServer         AC Names


                                                   Client                                                    Server


31 March 2008                                 Beat Signer <signer@inf.ethz.ch> / ETH Zurich                       DFKI Seminar
Active Components
                    Button                                                  Image

                    Slider                                                  Sound

                   Slider2D                                                 Movie

                 DateSelector                                             SQLClient

                   Browser                                                XCMRequest

                 MapSelector                                   ClientCtrlRequest

                    Rating                                     PaperPointControl

                   Capture                                        PaperPointAnnot

                CaptureAndSpeak                                              ...

31 March 2008             Beat Signer <signer@inf.ethz.ch> / ETH Zurich                DFKI Seminar
iServer Interfaces
 Java API
 XML import and export
 Web Service
 Interface based on XML and XSLT
               XHTML
               WAP, i-mode
               VoiceXML



31 March 2008                 Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
Conclusions
 General framework for interactive paper
 Cross-media information server
               extensibility (resource plug-ins)
               link service acessible through various APIs
               active content
 Novel forms of interactive paper publishing and
     paper-based user interfaces



31 March 2008                    Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar
References
 iPaper webpage
               http://www.ipaper.inf.ethz.ch
 YouTube videos
               http://www.youtube.com/user/ethglobis
 Fundamental Concepts for Interactive Paper and
     Cross-Media Information Spaces, B. Signer,
     Dissertation ETH Zurich Nr. 16218, 2006


31 March 2008                   Beat Signer <signer@inf.ethz.ch> / ETH Zurich   DFKI Seminar

Integrating Paper and Digital Information Spaces - The iServer / iPaper Solution

  • 1.
    Integrating Paper andDigital Information Spaces The iServer / iPaper Solution Beat Signer Global Information Systems Group ETH Zurich http://vub.academia.edu/BeatSigner 31 March 2008
  • 2.
    Paper in EverydaySettings 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 3.
    Input Devices 31 March2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 4.
    Digital Pen andPaper  Developed by Anoto  Different pen manufacturers  Logitech, Maxel, Nokia, Adapx, Livescribe ...  Pattern space  60 million km2 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich 4 DFKI Seminar
  • 5.
    Interactive Paper Linking active areas to resources image video clip 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 6.
    iPaper @ ETHZurich Tools Cross-Media Link Infrastructure Applications Publishing 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 7.
    Interactive Paper Applications Enhanced Reading  Enhanced Writing  interactive paper maps  proof-editing system  interactive festival brochures  photo album  educational materials  researcher's notebook  auction brochures  mammography  scientific publications  search system  …  …  Paper-Based Interfaces  Artists' Installations  PaperPoint  Lost Cosmonaut  image retrieval / digital libraries  Generosa Enterprise  video analysis tool  educational games  Interactive TableTops http://www.globis.ethz.ch/research/paper/applications/ 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 8.
    PaperPoint Presentation Tool  Mobile presentation tool  Non-linear presentations  Paper-based real-time annotations  Digital whiteboard  Multi-pen support  brainstorming 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 9.
    EdFest Project 31 March2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 10.
    EdFest Demo 31 March2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 11.
    EdFest Architecture Overview Context XCM content publishing Engine platform Context DB Metadata DB Appln DB Text-to-Speech Engine Client ICR Software Controller handwriting recognition iServer/iPaper Active iPaper Client cross-media Components link server Link DB 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 12.
    Content Publishing Framework iServer / iPaper cross-media link server XCM XML link iPublish definition content publishing publishing framework data • paper link data publish • active areas Structure Style • layers • digital resources • user model import PDF XML data of XSL CSS festival venues and events 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 13.
    Design of InteractivePaper Documents 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 14.
    Rough Guide 31 March2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 15.
    PaperProof 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 16.
    Paper-Digital Structural Mapping Dual document representation in digital and physical document spaces 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 17.
    Print-n-Link  Citations build the basis for a scientific web of publications  Use digital services to search / retrieve cited publications  Automatic authoring 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 18.
    Other iPaper Applications… PaperWeb Natural History Museum Lost Cosmonaut (Axel Vogelsang) iTable (Google Earth) 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 19.
    Different Types ofLink Authoring  Manual authoring of links  Content publishing framework for Interactive Paper  EdFest brochure – PDF and links generated from publishing database  Automatic authoring of links based on document analysis (PDF)  Print-n-Link system for scientific publications  Collaborative authoring based on P2P link sharing 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 20.
    Digital Link AuthoringTool 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 21.
    iPaper Architecture 31 March2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 22.
    RSL Metamodel andResource Plug-ins link Entities Selectors Resources Shape Page Query DB Object Time Span / Movie Clip XPointer XHTML Data/Metadata Spatial Info Document iPaper Application Database Movie Clips HTML Documents 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 23.
    Structure Links Users Layers Plug-ins Active Content 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 24.
    RSL Links 31 March2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 25.
    RSL Structural Links 31March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 26.
    RSL Structural Links…  Structure over data  e.g. regular document (chapters, sections, ...)  reuse of resources  transclusion (Ted Nelson)  context-dependant structures  Structure over structures  structural composition of data and substructures can be combined 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 27.
    RSL Structural Links…  Structure over links  put different navigational links in relation to each other  e.g. single structural link to define linear order over set of links  trails and tours 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 28.
    Active Content  Executecode (services) on the client and server side AC Manager Active Component ACS Manager A Active Component Active Component Stub B B Request Sender Client Device Active Component AC Names HTTP C iServer AC Names Client Server 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 29.
    Active Components Button Image Slider Sound Slider2D Movie DateSelector SQLClient Browser XCMRequest MapSelector ClientCtrlRequest Rating PaperPointControl Capture PaperPointAnnot CaptureAndSpeak ... 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 30.
    iServer Interfaces  JavaAPI  XML import and export  Web Service  Interface based on XML and XSLT  XHTML  WAP, i-mode  VoiceXML 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 31.
    Conclusions  General frameworkfor interactive paper  Cross-media information server  extensibility (resource plug-ins)  link service acessible through various APIs  active content  Novel forms of interactive paper publishing and paper-based user interfaces 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar
  • 32.
    References  iPaper webpage  http://www.ipaper.inf.ethz.ch  YouTube videos  http://www.youtube.com/user/ethglobis  Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, B. Signer, Dissertation ETH Zurich Nr. 16218, 2006 31 March 2008 Beat Signer <signer@inf.ethz.ch> / ETH Zurich DFKI Seminar