Bridging the Paper-Digital Divide: The iPaper Interactive Paper Framework


Published on

Presentation given at USI research seminar, Faculty of Informatics, University of Lugano, May 2007

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Bridging the Paper-Digital Divide: The iPaper Interactive Paper Framework

  1. 1. Bridging the Paper-Digital Divide: The iPaper Interactive Paper Framework Beat Signer Institute for Information Systems, ETH Zurich 8092 Zurich, Switzerland USI Seminar, May 30, 2007
  2. 2. The Paperless Office (1975) USI Seminar, May 30, 2007 Beat Signer, 2
  3. 3. The "Paperless Office" (2003) USI Seminar, May 30, 2007 Beat Signer, 3
  4. 4. The Myth of the Paperless Office  For decades, people have predicted the office of the future as a paperless office  documents generated, published and distributed electronically  documents read electronically  What has happened to this imminent revolution? USI Seminar, May 30, 2007 Beat Signer, 4
  5. 5. Affordances of Paper  The physical properties of an object determine how people use that object  Properties of paper  light, flexible, robust, porous, opaque, transparent, …  Human actions  grasping, folding, tearing, carrying, writing on, …  Paper supports forms of collaboration and interaction difficult to mimic in the digital world USI Seminar, May 30, 2007 Beat Signer, 5
  6. 6. E-Paper versus Digitally Augmented Paper Electronic Paper (E-Paper) Digitally Augmented Paper USI Seminar, May 30, 2007 Beat Signer,
  7. 7. Outline  Motivation  Interactive paper framework (iPaper)  Cross-media link server (iServer)  Interactive paper applications  functionality  authoring  Future work USI Seminar, May 30, 2007 Beat Signer,
  8. 8. Augmented Paper Projects Digital Desk, Wellner, 1991 Listen Reader, Back et al., 2001 Audio Notebook, Books with Voices Stifelman et al., 2001 Klemmer et al., 2003 ButterflyNet, Yeh et al., 2005 USI Seminar, May 30, 2007 Beat Signer,
  9. 9. Digital Pen and Paper  Developed by Anoto  Different pen manufacturers  Logitech, Maxel, Nokia, ...  Pattern space  60 million km2 USI Seminar, May 30, 2007 Beat Signer, 9
  10. 10. Problems of Existing Applications  Many projects focus on the input device, paper, printing and other hardware technologies rather than on the data integration and information management aspects  isolated solutions  The linking of paper tends to be based on physical rather than information-centric concepts  difficult to integrate new input / output devices USI Seminar, May 30, 2007 Beat Signer, 10
  11. 11. Our Approach  General interactive framework  different forms of paper-based interactions (enhanced reading, enhanced writing, …)  rapid prototyping as well as large scale publishing  integration of new types of resources and services  integration of new input devices  The key to a highly integrated interactive paper solution lies in the introduction of a platform for general cross-media information management USI Seminar, May 30, 2007 Beat Signer, 11
  12. 12. Active Areas and Virtual Page Layers image video clip USI Seminar, May 30, 2007 Beat Signer,
  13. 13. iPaper Architecture USI Seminar, May 30, 2007 Beat Signer,
  14. 14. iServer Metamodel and Resource Plug-ins link Entities Selectors Resources Shape Page Query DB Object Time Span / Movie Clip XPointer XHTML Data/Metadata Spatial Info Document iPaper Application Database Movie Clips HTML Documents USI Seminar, May 30, 2007 Beat Signer,
  15. 15. iSERVER iServer Links link parameter (1,*) (1,*) Has (0,*) HasSource Links HasTarget Preferences Preferences (0,*) (0,*) (0,*) parameter entity (0,*) AccessibleTo (0,*) user (0,*) (0,*) (0,*) Properties HasProperties Entities (0,*) (0,*) Users HasMembers (1,1) InaccessibleTo Layers partition partition (0,*) layer individual group (0,*) (0,*) |Layers| OnLayer CreatedBy Individuals Groups Users layer (1,1) Plug-ins selector resource Active (1,1) (0,*) (0,*) Selectors RefersTo Resources Contains Layers (0,*) Resources iPAPER page resource activeComponent medium (0,*) Active Com- OnPage Pages Containers Media (1,1) ponents shape (0,*) Shapes HasShapes (1,1) medium document Contains (0,*) Movies partition (2,*) Documents Pages rectangle polygon ellipse circle complexShape medium medium Complex Rectangles Polygons Ellipses Circles Images Webpages Shapes USI Seminar, May 30, 2007 Beat Signer,
  16. 16. Active Content  Execute code (services) on the client and server side AC Manager Active Component ACS Manager A Active Component Active Component Stub B B Request Sender Client Device Active Component AC Names HTTP C iServer AC Names Client Server USI Seminar, May 30, 2007 Beat Signer,
  17. 17. Active Components Button Image Slider Sound Slider2D Movie DateSelector SQLClient Browser XCMRequest MapSelector ClientCtrlRequest Rating PaperPointControl Capture PaperPointAnnot CaptureAndSpeak ... USI Seminar, May 30, 2007 Beat Signer,
  18. 18. Active Components … <?xml version="1.0" encoding="UTF-8" ?> PAPERPOINT_CONTROL <iserver> <activeComponent id="s1a" creator="beat"> Show slide 1 <name>Show slide 1</name> <properties> command showSlide <parameter> <key></key> slide 1 <value>showSlide</value> </parameter> <parameter> <key></key> <value>1</value> </parameter> </properties> <identifier>PAPERPOINT_CONTROL</identifier> </activeComponent> ... </iserver> USI Seminar, May 30, 2007 Beat Signer,
  19. 19. PaperPoint Presentation Tool USI Seminar, May 30, 2007 Beat Signer,
  20. 20. XML-Based Authoring ... ... <document id="handout" creator="beat"> <activeComponent id="fsa" creator="beat"> <name>Slide Handout</name> <name>First slide action</name> <id>h1</id> <properties> <size> <parameter> <width>210</width> <key></key> <height>297</height> <value>firstSlide</value> </size> </parameter> <content>handout.pdf</content> </properties> </document> <identifier>PAPERPOINT_CONTROL</identifier> <page id="p1" creator="beat" document="h1"> </activeComponent> <name>Page 1, Slide Controller</name> ... <number>1</number> <link id="fsl" creator="beat" sources="fs" </page> targets="fsa"> ... <name>Show the first slide</name> <rectangle id="fs" creator="beat" </link> layer="default" resource="p1"> ... <name>First Slide</name> <upperLeft> <point><x>24</x><y>12</y></point> </upperLeft> <size> <width>25</width><height>8</height> </size> </rectangle> ... USI Seminar, May 30, 2007 Beat Signer,
  21. 21. Digital Link Authoring Tool USI Seminar, May 30, 2007 Beat Signer,
  22. 22. PaperPoint Architecture USI Seminar, May 30, 2007 Beat Signer,
  23. 23. Edinburgh Fringe Festival  World's largest international arts festival  4 weeks  ~1700 events  ~27000 performances USI Seminar, May 30, 2007 Beat Signer, 23
  24. 24. EdFest Components USI Seminar, May 30, 2007 Beat Signer, 24
  25. 25. EdFest Booklet USI Seminar, May 30, 2007 Beat Signer, 25
  26. 26. Content Publishing Framework iServer cross-media link server XML link XCM iPublish definition content publishing publishing framework data publish • paper link data Structure Style • active areas on paper • layers • digital resources import • user model PDF XML data XSL CSS of festival venues and events USI Seminar, May 30, 2007 Beat Signer,
  27. 27. Print-n-Link  Citations build the basis for a scientific web of publications  Use digital services to search / retrieve cited publications  Automatic authoring USI Seminar, May 30, 2007 Beat Signer,
  28. 28. Print-n-Link Architecture Interactive Paper Architecture Desktop (x,y) Coordinates iSERVER iPAPER Active Content Text-to- Speech iPaper iPaper Plug-in Client Link DB Publish Positions and Active Content Definition Document Retrieval Print-n-Link Engine Web Search Citation Document Services Retrieval Lookup Print-n-Link: Weaving the Paper Web Semantic Mapper Printing Moira C. Norrie Beat Signer Nadir Weibel Dept. of Computer Science Dept. of Computer Science Dept. of Computer Science ETH Zurich ETH Zurich ETH Zurich 8092 Zurich, Switzerland 8092 Zurich, Switzerland 8092 Zurich, Switzerland Publish Display ABSTRACT Citation DB Citations engines and web browsers. The ease with which we can Citations form the basis for a web of scientific publications. not only find documents on the web, but also follow links Search engines, embedded hyperlinks and digital libraries to related documents is considered to be one of the main all simplify the task of finding publications of interest on advantages of working with digital documents as opposed the web and navigating to cited publications or web sites. to paper equivalents. Y many studies have shown that et However the actual reading of publications often takes place paper is still the preferred med ium when it comes to reading on paper and frequently on the move. We present a system documents for reasons of comfort, mobility and also ease of Print-n-Link that uses technologies for interactive paper to annotation. enhance the reading process by enabling users to access dig- In this paper, we show how emerging technologies for in- ital information and/or searches for cited documents from a teractive paper enable a bridge to be built between paper printed version of a publication using a digital pen for inter- and digital document spaces that allows users to travel back and forth between the two using the medium most appropri- iDOC action. A special virtual printer driver automatically gener- ates links from paper to digital services during the printing ate to the current task and sit uation. Users can print doc- process based on an analysis of PDF documents. Depend- uments for reading and, by activating hyperlinks on paper ing on the user setting and interaction gesture, the system with a digital pen, they can not only access digital informa- may retrieve metadata about the citation and inform the tion about related documents, but also initiate web searches user through an audio channel or directly display the cited to retrieve copies of those documents. The retrieved docu- document on the user’s screen. ments may in turn be printed for reading with automatically generated embedded hyperlinks on the paper. Mobility is supported by providing disconnected users on the move with Categories and Subject Descriptors audio information about related documents and the option D.2.11 [ Software Engineering ]: Software Architectures; of adding them to a request list to be processed when recon- H.4.m [ Information Systems Applications ]: Miscella- nected. neous; H.5.m [ Information Interfaces and Presenta- As an application, we take the everyday academic task of tion ]: Miscellaneous reading scientific publications. Whether the reading activ- Semantic Mapper ity is for the purpose of knowledge discovery or reviewing, readers regularly look up citations to other publications by General Terms flicking to the reference list at the back of the paper. In some Design, Algorithms, Experimentation cases, a search for the cited publication may follow and, if found, often it is printed and added to a pile of documents to Keywords be read. Citations are the established means of providing the links that form the basis of the scientific web of publications Publish Interactive paper, document integration, digital library, ci- that spans the digital and paper worlds. Just as citations tation management within digital documents can be used to automatically gen- 1. INTRODUCTION erate hyperlinks to cited documents, we can automatically generate links within paper documents through an analysis Citation Printer Nowadays we are all familiar with the notion of hyperlinks as a means of creating interconnected document spaces that of the PDF file when the document is printed. Section 2 presents a more detailed look at the activity of reading scientific publications and a discussion of related Document Print Driver can be easily searched and navigated with the help of search work. Section 3 describes the digital pen and paper tech- nologies and how these can be used to support interaction Analyser Pattern between paper and digital services. We then present the Permission to make digital or hard copies of all or part of this work for general functionality of our enhanced reading system in Sec- personal or classroom use is granted without fee provided that copies are tion 4 and the overall architecture in Section 5. Details of not made or distributed for profit or commercial advantage and that copies the virtual printer component used to analyse PDF docu- bear this notice and the full citation on the first page. To copy otherwise, to ments for automatic link generation as well as printing are republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. given in Section 6. Concluding remarks follow in Section 7. DocEng’06, October 10–13, 2006, Amsterdam, The Netherlands. Copyright 2006 ACM 1-59593-515-0/06/0010 ...$5.00. Document DB USI Seminar, May 30, 2007 Beat Signer,
  29. 29. Interactive Paper Applications  Enhanced Reading  Enhanced Writing  interactive paper maps  photo album  interactive festival brochures  researcher's notebook  educational materials  mammography  auction brochures  search system  scientific publications  …  …  Paper-Based Interfaces  Artists' Installations  PaperPoint  Lost Cosmonaut  digital libraries  Generosa Enterprise  video analysis tool  educational games  Interactive TableTops  … USI Seminar, May 30, 2007 Beat Signer,
  30. 30. Current Projects  PaperWorks (FP6-516895)  Interweaving Paper and Digital Documents  King's College London, Acreo AB, Anoto, ETH Zurich, Arjowiggins SAS, Brunel University, Malmo University, Pearson Education  DELOS  Network of Excellence on Digital Libraries  Paper-based user interfaces for image retrieval (collaboration with Heiko Schuldt, University of Basel) USI Seminar, May 30, 2007 Beat Signer,
  31. 31. Future Work  Design guidelines for interactive paper interfaces  functionalty (active components) and design  Cross-media authoring and publishing  analysis of existing publishing processes (Pearson)  general document model for mapping between printed and digital document instances  Extensions of iServer  collaborative cross-media information sharing  new resource plug-ins USI Seminar, May 30, 2007 Beat Signer,
  32. 32. Mapping between Printed and Digital Instances static physical rendering dynamic logical document USI Seminar, May 30, 2007 Beat Signer,
  33. 33. Conclusions  Interactive paper framework (iPaper)  information-centric approach supports different technologies, functionalities and applications  rapid prototyping (reusable active components)  Extensible platform for cross-media information management (iServer)  Cross-media authoring / publishing tools  manual, content-driven and automatic link authoring USI Seminar, May 30, 2007 Beat Signer,