i3 – Recent iServer, iPaper and iGesture Developments

Beat Signer
http://www.beatsigner.com

Institute for Information Systems
ETH Zurich
CH-8092 Zurich, Switzerland




                                                5 December 2007
iServer Extensions and Modifications
 RSL (resource-selector-link) model is the new
     name for the underlying iServer model
 Context Resolver
 Layers
 Structural Links




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   2
RSL Context Resolver




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   3
RSL Layers




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   4
RSL Structural Links




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   5
RSL Structural Links …
 Structure over data
            structural relationship of different resources
            e.g. regular document (chapters, sections, ...)
            possible to reuse the same resource in different
             structures  transclusion (Ted Nelson)
            context-dependant structures based on context
             resolver concept




5 December 2007           Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   6
RSL Structural Links …
 Structure over structures
            reuse of complete substructures
            structural composition of data and substructures can
             be combined
 Structure over links
            put different navigational links in relation to each other
            e.g. single structural link to define linear order over
             set of links  trails and tours



5 December 2007           Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   7
Structure


         Links

                                                                                                    Users




         Layers
                              Plug-ins                                                           Active
                                                                                                 Content




5 December 2007           Beat Signer / Institute for Information Systems / signer@inf.ethz.ch              8
RSL Metamodel
 Beat Signer and Moira C. Norrie, As We May Link:
 A General Metamodel for Hypermedia Systems,
 In Proceedings of ER 2007, 26th International Conference on
 Conceptual Modeling, Auckland, New Zealand, November 2007




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
Digital Link Authoring Tool




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   10
Digital Link Authoring Tool …
 Extension of iServer authoring tool, Adriana
 Integration of earlier student projects (e.g. active
     component authoring tool)
 Focus on the iPaper authoring plug-in but later
     other plug-ins may be integrated




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   11
iPaper Templates
 Links that occur on multiple pages (e.g. controls,
     headers, footers) should not have to be repeated
 Templates can be defined on document or
     individual page level
 Multiple templates can be combined (page and
     document level)
 PaperPoint currently uses a single template to
     define the functionality available on all slides
5 December 2007    Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   12
iServer / iPaper Infrastructure
 Resource plug-in management
 Application management
 Pattern space management, Monica Frisoni
 I/O pen connector (Bluetooth), Claudio Hatz
 ...




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   13
PaperPoint v0.5
                                                     New design / functionality
                                                                  colours, pointer types, …

                                                     Single step printing
                                                           process

                                                     User studies, Elke
                                                                  interviews
                                                                  lab sessions
                                                                  long term usage


5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch         14
PaperPoint v0.5
 Beat Signer and Moira C. Norrie, PaperPoint: A Paper-Based
 Presentation and Interactive Paper Prototyping Tool, In
 Proceedings of TEI 2007, First International Conference on
 Tangible and Embedded Interaction, Baton Rouge, USA,
 February 2007




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
Interactive Table (iTable)




         Pen-Based Google Earth Browser                                          Pen-Based Drawing Tool




                  Photo Browser                                                   Photo Browser (Slider)

5 December 2007             Beat Signer / Institute for Information Systems / signer@inf.ethz.ch           16
Interactive Table (iTable) …




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
Interactive Table (iTable) …
 Extension of iTable interface, Matthias Geel
            calibration of beamer and input area
            general desktop control active components
 Applications
            Pen-based Google Earth Explorer
            Pen-based Drawing Tool
            Photo Browser
            …


5 December 2007          Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   18
Rough Guide




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   19
Query by Sketching (QbS)
                                                                 Pen-based sketching
                                                                       interface
                                                                              University of Basel
                                                                              GlobIS group

 Start date: October 2007 (2 year project), Adriana

 Michael Springmann, Adriana Ispas, Heiko Schuldt, Moira
 C. Norrie and Beat Signer, Towards Query by Sketch,
 Second DELOS Conference on Digital Libraries, Pisa, Italy,
 December 2007
5 December 2007    Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
PaperWeb
                                                     Automatic authoring of
                                                           active areas
                                                                  all hyperlinks become
                                                                   interactive on the
                                                                   printout version




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch     21
Remote Sketching on Paper
                                                     Integration of Skype and
                                                           paper-based remote
                                                           sketching, Hermann
                                                           Hofstetter
                                                                  HCI group University of
                                                                   Konstanz, H. Reiterer
                                                                  GlobIS group




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
Powerwall, Konstanz
                                                              Facts
                                                                           5.20 m x 2.5 m
                                                                           8 beamers
                                                                           4640 x 1920 pixels
                                                                           soft-edge-blending
 Visualisation of large data sets
 Input devices
            3D tracker, laser "pen", mouse

 iPaper as a new input device?
5 December 2007            Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
Wellness Game Project, Shanghai
                                                     Science, Technology &
                                                           Culture Festival, May 2008

                                                     Projects
                                                                  Thai Chi, Native Systems Group,
                                                                   Dennis Majoe
                                                                  China Gates, Native Systems
                                                                   Group, Art Clay
                                                                  Real Time Panorama, Native
                                                                   Systems Group, Jürg Gutknecht
                                                                  iPaper Project, GlobIS Group,
                                                                   Moira C. Norrie



5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch               24
Multi Digital, LZ Medien




5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   25
PaperProof
                                                      Planned user studies,
                                                            Nadir & Elke

                                                      Definition of new gestures
                                                            for edit commands?

 Nadir Weibel, Adriana Ispas, Beat Signer and Moira C. Norrie,
 PaperProof: A Paper-Digital Proof-Editing System,
 Proceedings of CHI 2008, 26th International Conference on
 Human Factors in Computing Systems (Interactivity Track),
 Florence, Italy, April 2008
5 December 2007    Beat Signer / Institute for Information Systems / signer@inf.ethz.ch   26
iGesture Framework
                                                     http://www.igesture.org
                                                                  Open Source (Apache 2.0)
                                                                  initial release February 2007
                                                                  1682 downloads

                                                     Presented at ICDAR 2007
                                                                  FKI Group, University of Berne

                                                                  DFKI, Germany

                                                                  Computer Vision Center,
                                                                   Barcelona


5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch          27
iGesture Framework …
 New input handler for graphics tablets and
     Tablet PCs, Michele Croci
 Beat Signer, Ueli Kurmann and Moira C. Norrie, iGesture:
 A General Gesture Recognition Framework, Proceedings of
 ICDAR 2007, 9th International Conference on Document
 Analysis and Recognition, Curitiba, Brazil, September 2007
 Beat Signer, Moira C. Norrie and Ueli Kurmann, iGesture:
 A Java Framework for the Development and Deployment of
 Stroke-Based Online Gesture Recognition Algorithms,
 Technical Report ETH Zurich, TR561, September 2007
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
Geco (Gesture Control)
                                                     Mapping of gestures
                                                           (iGesture gesture sets),
                                                           Michele Croci
                                                                  hotkeys
                                                                  command line calls
                                                                  …

                                                     Initial release (freeware)
                                                           planned within next month


5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
Department Events
 Maturandentage 2008
            January 8/9
            PaperProof and PaperPoint
 Jahr der Informatik
            visit different locations
            PaperPoint as presentation tool?




5 December 2007            Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
iSERVER
iServer
    Links                                                                                                   link                                                                                                      parameter
                                                                                   (1,*)                             (1,*)                                                Has                  (0,*)
                                          HasSource                                            Links                                   HasTarget                                                         Preferences
                                                                                                                                                                      Preferences
                                                                                   (0,*)                       (0,*)
                                                                                                                                                                        (0,*)

                   parameter                                                                              entity (0,*)               AccessibleTo          (0,*)                        user
                                (0,*)                                      (0,*)                                                                                                               (0,*)
          Properties                     HasProperties                                       Entities                (0,*)
                                                                                                                                                           (0,*)
                                                                                                                                                                        Users                            HasMembers
                                                                                                                   (1,1)
                                                                                                                                     InaccessibleTo
      Layers                                                                                  partition
                                                                                                                                                                                  partition
                                                                                                                                                                                                              (0,*)

                       layer                                                                                                                                                      individual                             group
                                (0,*)                                                                                                                      (0,*)
            |Layers|                       OnLayer                                                                                     CreatedBy                      Individuals                           Groups
                                                                                                                                                                                                                                   Users
                       layer
                                            (1,1)                  Plug-ins
                                                        selector                                                                                resource

             Active                                                (1,1)                                                     (0,*)                         (0,*)
                                           Selectors                                        RefersTo                                  Resources                        Contains
             Layers

                                                                                                                                                                                (0,*)
                                                                                                                                                                                                                      Resources
iPAPER                                                                                                                                             page                            resource            activeComponent                medium
                                                                                                                             (0,*)                                                             Active Com-
                                                                                             OnPage                                      Pages                        Containers                                                  Media
                                                                    (1,1)                                                                                                                       ponents
                                                         shape
                                                                     (0,*)
                                            Shapes                                                  HasShapes                           (1,1)
                                                                                                                                                                                                               medium
                                                                                                                                                                                  document

                                                                                                                                       Contains               (0,*)                                Movies
                                            partition                                                         (2,*)                                                   Documents
                                                                                                                                        Pages
       rectangle               polygon                   ellipse                           circle         complexShape                                                                                         medium                 medium

                                                                                                      Complex
Rectangles         Polygons                Ellipses                        Circles                                                                                                                 Images                   Webpages
                                                                                                      Shapes




 5 December 2007                                               Beat Signer / Institute for Information Systems / signer@inf.ethz.ch                                                                                                       36
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
5 December 2007   Beat Signer / Institute for Information Systems / signer@inf.ethz.ch

i3 - Recent iServer, iPaper and iGesture Developments

  • 1.
    i3 – RecentiServer, iPaper and iGesture Developments Beat Signer http://www.beatsigner.com Institute for Information Systems ETH Zurich CH-8092 Zurich, Switzerland 5 December 2007
  • 2.
    iServer Extensions andModifications  RSL (resource-selector-link) model is the new name for the underlying iServer model  Context Resolver  Layers  Structural Links 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 2
  • 3.
    RSL Context Resolver 5December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 3
  • 4.
    RSL Layers 5 December2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 4
  • 5.
    RSL Structural Links 5December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 5
  • 6.
    RSL Structural Links…  Structure over data  structural relationship of different resources  e.g. regular document (chapters, sections, ...)  possible to reuse the same resource in different structures  transclusion (Ted Nelson)  context-dependant structures based on context resolver concept 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 6
  • 7.
    RSL Structural Links…  Structure over structures  reuse of complete substructures  structural composition of data and substructures can be combined  Structure over links  put different navigational links in relation to each other  e.g. single structural link to define linear order over set of links  trails and tours 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 7
  • 8.
    Structure Links Users Layers Plug-ins Active Content 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 8
  • 9.
    RSL Metamodel BeatSigner and Moira C. Norrie, As We May Link: A General Metamodel for Hypermedia Systems, In Proceedings of ER 2007, 26th International Conference on Conceptual Modeling, Auckland, New Zealand, November 2007 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 10.
    Digital Link AuthoringTool 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 10
  • 11.
    Digital Link AuthoringTool …  Extension of iServer authoring tool, Adriana  Integration of earlier student projects (e.g. active component authoring tool)  Focus on the iPaper authoring plug-in but later other plug-ins may be integrated 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 11
  • 12.
    iPaper Templates  Linksthat occur on multiple pages (e.g. controls, headers, footers) should not have to be repeated  Templates can be defined on document or individual page level  Multiple templates can be combined (page and document level)  PaperPoint currently uses a single template to define the functionality available on all slides 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 12
  • 13.
    iServer / iPaperInfrastructure  Resource plug-in management  Application management  Pattern space management, Monica Frisoni  I/O pen connector (Bluetooth), Claudio Hatz  ... 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 13
  • 14.
    PaperPoint v0.5  New design / functionality  colours, pointer types, …  Single step printing process  User studies, Elke  interviews  lab sessions  long term usage 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 14
  • 15.
    PaperPoint v0.5 BeatSigner and Moira C. Norrie, PaperPoint: A Paper-Based Presentation and Interactive Paper Prototyping Tool, In Proceedings of TEI 2007, First International Conference on Tangible and Embedded Interaction, Baton Rouge, USA, February 2007 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 16.
    Interactive Table (iTable) Pen-Based Google Earth Browser Pen-Based Drawing Tool Photo Browser Photo Browser (Slider) 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 16
  • 17.
    Interactive Table (iTable)… 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 18.
    Interactive Table (iTable)…  Extension of iTable interface, Matthias Geel  calibration of beamer and input area  general desktop control active components  Applications  Pen-based Google Earth Explorer  Pen-based Drawing Tool  Photo Browser  … 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 18
  • 19.
    Rough Guide 5 December2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 19
  • 20.
    Query by Sketching(QbS)  Pen-based sketching interface  University of Basel  GlobIS group  Start date: October 2007 (2 year project), Adriana Michael Springmann, Adriana Ispas, Heiko Schuldt, Moira C. Norrie and Beat Signer, Towards Query by Sketch, Second DELOS Conference on Digital Libraries, Pisa, Italy, December 2007 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 21.
    PaperWeb  Automatic authoring of active areas  all hyperlinks become interactive on the printout version 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 21
  • 22.
    Remote Sketching onPaper  Integration of Skype and paper-based remote sketching, Hermann Hofstetter  HCI group University of Konstanz, H. Reiterer  GlobIS group 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 23.
    Powerwall, Konstanz  Facts  5.20 m x 2.5 m  8 beamers  4640 x 1920 pixels  soft-edge-blending  Visualisation of large data sets  Input devices  3D tracker, laser "pen", mouse  iPaper as a new input device? 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 24.
    Wellness Game Project,Shanghai  Science, Technology & Culture Festival, May 2008  Projects  Thai Chi, Native Systems Group, Dennis Majoe  China Gates, Native Systems Group, Art Clay  Real Time Panorama, Native Systems Group, Jürg Gutknecht  iPaper Project, GlobIS Group, Moira C. Norrie 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 24
  • 25.
    Multi Digital, LZMedien 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 25
  • 26.
    PaperProof  Planned user studies, Nadir & Elke  Definition of new gestures for edit commands? Nadir Weibel, Adriana Ispas, Beat Signer and Moira C. Norrie, PaperProof: A Paper-Digital Proof-Editing System, Proceedings of CHI 2008, 26th International Conference on Human Factors in Computing Systems (Interactivity Track), Florence, Italy, April 2008 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 26
  • 27.
    iGesture Framework  http://www.igesture.org  Open Source (Apache 2.0)  initial release February 2007  1682 downloads  Presented at ICDAR 2007  FKI Group, University of Berne  DFKI, Germany  Computer Vision Center, Barcelona 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 27
  • 28.
    iGesture Framework … New input handler for graphics tablets and Tablet PCs, Michele Croci Beat Signer, Ueli Kurmann and Moira C. Norrie, iGesture: A General Gesture Recognition Framework, Proceedings of ICDAR 2007, 9th International Conference on Document Analysis and Recognition, Curitiba, Brazil, September 2007 Beat Signer, Moira C. Norrie and Ueli Kurmann, iGesture: A Java Framework for the Development and Deployment of Stroke-Based Online Gesture Recognition Algorithms, Technical Report ETH Zurich, TR561, September 2007 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 29.
    Geco (Gesture Control)  Mapping of gestures (iGesture gesture sets), Michele Croci  hotkeys  command line calls  …  Initial release (freeware) planned within next month 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 30.
    Department Events  Maturandentage2008  January 8/9  PaperProof and PaperPoint  Jahr der Informatik  visit different locations  PaperPoint as presentation tool? 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 31.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 32.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 33.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 34.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 35.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 36.
    iSERVER iServer Links link parameter (1,*) (1,*) Has (0,*) HasSource Links HasTarget Preferences Preferences (0,*) (0,*) (0,*) parameter entity (0,*) AccessibleTo (0,*) user (0,*) (0,*) (0,*) Properties HasProperties Entities (0,*) (0,*) Users HasMembers (1,1) InaccessibleTo Layers partition partition (0,*) layer individual group (0,*) (0,*) |Layers| OnLayer CreatedBy Individuals Groups Users layer (1,1) Plug-ins selector resource Active (1,1) (0,*) (0,*) Selectors RefersTo Resources Contains Layers (0,*) Resources iPAPER page resource activeComponent medium (0,*) Active Com- OnPage Pages Containers Media (1,1) ponents shape (0,*) Shapes HasShapes (1,1) medium document Contains (0,*) Movies partition (2,*) Documents Pages rectangle polygon ellipse circle complexShape medium medium Complex Rectangles Polygons Ellipses Circles Images Webpages Shapes 5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch 36
  • 37.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 38.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 39.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 40.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 41.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch
  • 42.
    5 December 2007 Beat Signer / Institute for Information Systems / signer@inf.ethz.ch