Interactive Paper: Past, Present and Future

Beat Signer and Moira C. Norrie
Department of Computer Science
Vrije Universiteit Brussel


http://www.beatsigner.com




                                  PaperComp 2010, September 26, 2010
Paper Computing in the Past




                                                                                                   Punch Cards, since ca. 1725




                                     Reading Wheel, Ramelli, 1588                                            Electro Junior Toy

PaperComp 2010, September 26, 2010                                  Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Paper Computing in the Past ...




                                     DigitalDesk, Wellner, 1991                                Listen Reader, Back et al., 2000




                                      Ariel, Mackay et al., 1995                              Audio Notebook, Stifelman, 1993

PaperComp 2010, September 26, 2010                                 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Current Approaches
                    Electronic Paper                        Augmented Paper




                                       Amazon Kindle                      Anoto Digital Pen and Paper Technology

PaperComp 2010, September 26, 2010                     Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Interactive Paper Frameworks
 Tools offered by Anoto
           SDK for PC applications and network applications
           Forms Design Kit (FDK)
 PaperToolkit
           integration of interactive and batched input handling
 Letras
           distributed event processing pipeline



PaperComp 2010, September 26, 2010          Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Interactive Paper Frameworks
 iPaper
           information-centric approach supports different
            technologies, functionalities and applications
           separation of authoring and programming
           rapid prototyping (active components)
 Livescribe's Smartpen SDK
           application sharing and the user as a developer
 ...

PaperComp 2010, September 26, 2010         Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Future Challenges and Directions
 Future challenges for augmented and interactive
     paper solutions include
        software framework interoperability and scalability
        automatic application deployment
        hardware and material sciences
        visual and interaction design
        cross-media information management
        deeper understanding of existing work practices
        …


PaperComp 2010, September 26, 2010       Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Input Device Independence




                                     iPaper, Paper++ Project, 2001                                 iPaper, Paper++ Project, 2003




                                      iPaper, EdFest Project, 2005                     iPaper, The Lost Cosmonaut Project, 2004

PaperComp 2010, September 26, 2010                                   Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Input Device Independence ...




                                            Bähr et al., PaperComp 2010                               Bonnard et al., PaperComp 2010




                                     Portocarrero et al., PaperComp 2010                               Suzuki et al., PaperComp 2010

PaperComp 2010, September 26, 2010                                         Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Digital Ink Abstraction
 Most frameworks and toolkits define their own
     proprietary digital ink representation
           existing standards (e.g. InkML) are often not used
           is there a need for new common data formats?
 Common data formats can help to
           exchange data between applications and across
            different interactive paper frameworks
           realise mixed-media environments


PaperComp 2010, September 26, 2010         Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Scalability and Application Deployment
 Problem of existing augmented paper solutions
           isolated applications
           missing cross-application interaction
 Interactive Paper 2.0
           global and open architecture
           unique resolving of interactive paper services based
            on a global paper lookup service (PLS)
             - automatic application deployment
           service oriented architecture to enable cross-
            application interaction and composition
PaperComp 2010, September 26, 2010          Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Visual and Interaction Design
 Visualisation of interactive
     areas and functionality
      • design patterns
 Interaction design
                                                                            EdFest Project, 2004




   • online versus offline
            input processing
      • lack of modal dialogues
                                                                            EdFest Project, 2005

      • multimodal interaction
 Findings might be applicable to traditional GUIs
    e.g. cross-application widget interaction
PaperComp 2010, September 26, 2010   Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Innovative Hardware and Materials
 Mobile solutions for feed-
     back on paper documents
      • spatially aware pro-
            jection of information
      • digital pen with display
      • voice feedback
                                                                  MouseLight, Song et al., 2010




      • ...
 Fusion of electronic paper
     and interactive paper
      • printed electronics
                                                                      PaperWorks Project, 2007

PaperComp 2010, September 26, 2010   Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Paper-Digital Authoring and Publishing
 Manual authoring
 Content-driven publishing
  • e.g. EdFest brochure
  • integration with third-
            party applications
                                                                   iPaper Manual Authoring Tool


      • what about versioning?
 Automatic link authoring
  • e.g. Print-n-Link
 Collaborative authoring
                                                                              Print-n-Link, 2006

PaperComp 2010, September 26, 2010   Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Conclusions
 Various approaches for interactive paper and
     paper computing
           are we missing a common abstraction layer in existing
            architectures and do we need Interactive Paper 2.0?
 It might be the right time to think about the
     integration of existing solutions to share efforts
        cross-disciplinary workshops
          - e.g. CoPADD and PaperComp
        Network of Excellence (NoE)?


PaperComp 2010, September 26, 2010        Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
Further Information
      Fundamental Concepts for Interactive Paper
       and Cross-Media Information Spaces, Beat
       Signer, ISBN 978-3-8370-2713-6, May 2008
 Interactive Paper: Past, Present and Future
           http://vub.academia.edu/BeatSigner/Papers/216727/Intera
            ctive_Paper_Past_Present_and_Future
 Research papers about iPaper are available on
           http://vub.academia.edu/BeatSigner
 Presentation available on
           http://www.slideshare.net/signer/

PaperComp 2010, September 26, 2010              Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be

Interactive Paper: Past, Present and Future

  • 1.
    Interactive Paper: Past,Present and Future Beat Signer and Moira C. Norrie Department of Computer Science Vrije Universiteit Brussel http://www.beatsigner.com PaperComp 2010, September 26, 2010
  • 2.
    Paper Computing inthe Past Punch Cards, since ca. 1725 Reading Wheel, Ramelli, 1588 Electro Junior Toy PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 3.
    Paper Computing inthe Past ... DigitalDesk, Wellner, 1991 Listen Reader, Back et al., 2000 Ariel, Mackay et al., 1995 Audio Notebook, Stifelman, 1993 PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 4.
    Current Approaches Electronic Paper Augmented Paper Amazon Kindle Anoto Digital Pen and Paper Technology PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 5.
    Interactive Paper Frameworks Tools offered by Anoto  SDK for PC applications and network applications  Forms Design Kit (FDK)  PaperToolkit  integration of interactive and batched input handling  Letras  distributed event processing pipeline PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 6.
    Interactive Paper Frameworks iPaper  information-centric approach supports different technologies, functionalities and applications  separation of authoring and programming  rapid prototyping (active components)  Livescribe's Smartpen SDK  application sharing and the user as a developer  ... PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 7.
    Future Challenges andDirections  Future challenges for augmented and interactive paper solutions include  software framework interoperability and scalability  automatic application deployment  hardware and material sciences  visual and interaction design  cross-media information management  deeper understanding of existing work practices  … PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 8.
    Input Device Independence iPaper, Paper++ Project, 2001 iPaper, Paper++ Project, 2003 iPaper, EdFest Project, 2005 iPaper, The Lost Cosmonaut Project, 2004 PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 9.
    Input Device Independence... Bähr et al., PaperComp 2010 Bonnard et al., PaperComp 2010 Portocarrero et al., PaperComp 2010 Suzuki et al., PaperComp 2010 PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 10.
    Digital Ink Abstraction Most frameworks and toolkits define their own proprietary digital ink representation  existing standards (e.g. InkML) are often not used  is there a need for new common data formats?  Common data formats can help to  exchange data between applications and across different interactive paper frameworks  realise mixed-media environments PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 11.
    Scalability and ApplicationDeployment  Problem of existing augmented paper solutions  isolated applications  missing cross-application interaction  Interactive Paper 2.0  global and open architecture  unique resolving of interactive paper services based on a global paper lookup service (PLS) - automatic application deployment  service oriented architecture to enable cross- application interaction and composition PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 12.
    Visual and InteractionDesign  Visualisation of interactive areas and functionality • design patterns  Interaction design EdFest Project, 2004 • online versus offline input processing • lack of modal dialogues EdFest Project, 2005 • multimodal interaction  Findings might be applicable to traditional GUIs  e.g. cross-application widget interaction PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 13.
    Innovative Hardware andMaterials  Mobile solutions for feed- back on paper documents • spatially aware pro- jection of information • digital pen with display • voice feedback MouseLight, Song et al., 2010 • ...  Fusion of electronic paper and interactive paper • printed electronics PaperWorks Project, 2007 PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 14.
    Paper-Digital Authoring andPublishing  Manual authoring  Content-driven publishing • e.g. EdFest brochure • integration with third- party applications iPaper Manual Authoring Tool • what about versioning?  Automatic link authoring • e.g. Print-n-Link  Collaborative authoring Print-n-Link, 2006 PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 15.
    Conclusions  Various approachesfor interactive paper and paper computing  are we missing a common abstraction layer in existing architectures and do we need Interactive Paper 2.0?  It might be the right time to think about the integration of existing solutions to share efforts  cross-disciplinary workshops - e.g. CoPADD and PaperComp  Network of Excellence (NoE)? PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
  • 16.
    Further Information  Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, Beat Signer, ISBN 978-3-8370-2713-6, May 2008  Interactive Paper: Past, Present and Future  http://vub.academia.edu/BeatSigner/Papers/216727/Intera ctive_Paper_Past_Present_and_Future  Research papers about iPaper are available on  http://vub.academia.edu/BeatSigner  Presentation available on  http://www.slideshare.net/signer/ PaperComp 2010, September 26, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be