• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Cross-Media Information Systems - Quo Vadis?
 

Cross-Media Information Systems - Quo Vadis?

on

  • 5,029 views

Seminar presentation about cross-media information spaces given at EPFL.

Seminar presentation about cross-media information spaces given at EPFL.

Statistics

Views

Total Views
5,029
Views on SlideShare
969
Embed Views
4,060

Actions

Likes
1
Downloads
17
Comments
0

8 Embeds 4,060

http://www.beatsigner.com 2157
http://wise.vub.ac.be 1873
https://wise.vub.ac.be 16
http://translate.googleusercontent.com 9
http://www.linkedin.com 2
http://localhost 1
http://webcache.googleusercontent.com 1
http://wise 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Cross-Media Information Systems - Quo Vadis? Cross-Media Information Systems - Quo Vadis? Presentation Transcript

    • Cross-Media Information Systems - Quo Vadis?Prof. Beat Signerhttp://www.beatsigner.comWeb & Information Systems Engineering Lab (WISE)Department of Computer ScienceVrije Universiteit Brussel 2 December 2005
    • Fluid Cross-Media Information Spaces 01001000100 0101001100010010100010010 0100100010010001000100001001000 100010001100001001000110010101010110 1100010010100010010010010100010101011001 101001100001100 101010100 00101000100 110101010101 00100 01010101000 1001 1111111111 111 0000000010 0 11111111 0101001011 010101010 111010101 001010010110
    • WEB & INFORMATION SYSTEMS ENGINEERING CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA)Prof. Dr. Beat Signer Dr. Bruno DumasInteractive Paper, Cross-Media Multimodal Interaction, Cross- Information InformationInformation Architectures Media Information Architectures Systems & Visualisation Management & Navigation CISA Human-Machine & Human-Information InteractionLode Hoste Brecht De RoomsMultimodal Interaction, Midas Implicit Context-aware Human-and Mudra Framework Computer Interaction, HCII
    • WEB & INFORMATION SYSTEMS ENGINEERING CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA)Reinout Roels Ahmed A.O. TayehMindXpres Next Generation Open Cross-Media Authoring, Information InformationPresentation Tool Fluid Document Formats Systems & Visualisation Management & Navigation CISA Human-Machine & Human-Information InteractionSandra TrullemansPersonal Cross-MediaInformation Management
    • PARC User Interface (Mid 70s)  Desktop metaphor  WIMP interaction  "filing cabinets" with hierarchical folders  Paper paradigm  "What You See Is What You Get" (WYSIWYG)  what about richer document formats that existed at that time? Xerox Star 8010 [http://www.digibarn.com/collections/systems/xerox-8010/]November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Digital Documents as a Paper Simulator? Most people dont understand the logic of the concept: "What You See Is What You Get" is based on printing the document out ("get" means "get WHEN YOU PRINT IT OUT"). And that means a metaphysical shift: a document can only consist of what can be printed! [...] No overlays [...] – PAPER UNDER GLASS. Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009 Ted Nelson When data of any sort are placed in storage, they are filed alphabetically or numerically, and information is found (when it is) by tracing it down from subclass to subclass. It can be in only one place, unless duplicates are used [...] The human mind does not work that way. It operates by association. As We May Think, Atlantic Monthly, July 1945 Vannevar BushNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • What is Wrong with Digital Documents?  Existing document formats are based on the simulation of paper affordances on desktop computers  How to manage mixed-media "documents" in open and fluid cross-media information spaces?  on the data level - context-sensitive adaptation - cross-media transclusion  on the visualisation and navigation level - zoomable user interfaces  on the cross-media interaction level - fluid multimodal cross-media interfaces  Remediation of the "paper simulation" approach  WYSIWYG is only one out of many options!November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • RSL Metamodel and iServerNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Structure Links Users Layers Plug-ins Active ContentNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Bridging the Paper-Digital DivideNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Interactive Paper Applications  Enhanced Reading  Enhanced Writing  interactive paper maps  photo album  interactive festival brochures  researchers notebook  educational materials  mammography annotation  auction brochures  query by sketching  scientific publications  …  …  Art Installations  Paper-based Interfaces  Lost Cosmonaut  PaperPoint  Generosa Enterprise  document proof-editing  digital libraries  Interactive Tabletops  video analysis tool  photo browser  educational games  collaborative sketch toolNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Interactive Festival Guide (EdFest)November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • The Lost Cosmonaut  Writing as a collaborative act of memory and story- telling  Reading, writing and narrative as an act of making sense  Handwriting as a tool for human-computer interaction In collaboration with Axel Vogelsang, Artists in Labs ProgrammeNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • ArtVisNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • ArtVis ...November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • ArtVis ...  Advanced visualisation techniques in combination with a TUI  explore Web Gallery of Art  faceted browsing  Phidgets-based TUI  RFID-tagged physical objects  Three main components to explore, analyse and browse the informationNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • RSL-based Associative File System structural link navigational link signer My Pictures My Videos My Presentations Vancouver ER 2010 Vancouver ER 2010 ER 2010 WISE 2009 Aquarium Keynote single slide cross-media transclusion Granville Vancouver Public Keynote Olga and Dinner transclusion (via OOXML Island Harbour Market J. Thorp Lamia ER 2010 RSL selector) multiple classification cross-media associationNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • RSL-based Associative File System ...November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • What is Wrong with Slideware?  Simulation of physical slides  limited space due to the slide concept  linear navigation from slide to slide  difficult to reuse content and embed rich media types  MindXpres addresses these issues  unlimited canvas with zoomable user interface  non-linear navigation and associative linking  content-based approach with automatic visualisationNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • MindXpres Presentation Tool  HTML5-based extensible core with three types of JavaScript plug-ins  components - source code, graph, video, ...  containers - slide, image gallery, ...  structures - grid, graph, clustered, ...  Presentation as RSL- based overlay structure  enables navigation across different presentationsNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • MindXpres Presentation Tool ...November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • MindXpres Presentation Tool ...  Functional MindXpres prototype  Ongoing work  visual authoring tool - alternative to XML-based authoring  rich media plug-ins - videos with subtitles, annotations etc. - statistical data with alternative visualisations - live sensor data - ...  audience-driven presentations  collaborative annotations and information sharing  multimodal interaction (e.g. integration of PaperPoint concepts)  import of existing slideware formatsNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Fluid Multimodal Cross-Media Interfaces  Investigation of mobile and distributed document formats  Mobile Cross-Media Applications (MobiCraNT)  Passive or active information units (crosslets)  processing and storage of sensor data  binding of specific multimodal interactions to resources  Implicit context-aware human-computer interaction (HCII)  Personal Cross-Media Information ManagementNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Multimodal and Multi-Touch Interaction  Midas/Mudra Framework  declarative definition of multimodal and multi-touch interactions  rule-based language approach  rapid protoyping and application development  iGesture Workbench  create and test gesture sets and algorithms  different modalities: digital pen, Wii remote, …  open source (www.igesture.org)  SpeeG  multimodal speech- and body gesture-based text input  real-time correction of recognised voice inputNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Open Cross-Media Linking # target types open cross-media  Four existing classes of linking cross-media link systems  A: single source and single target type ... B2  B: single source and D2 multiple target types 5  C: multiple source and single target type 4 B1  D: multiple source and D1 multiple target types 3  Our proposed open cross- 2 media link approach covers A1... An C1 C2 C3 the whole space 1 # source types 1 2 3 4 5 ...November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Open Cross-Media Authoring Client Application Resource Plug-in Repository Visual Plug-ins Visual Plug-ins Data Plug-ins Cross-Media Cross-Media Browser & Editor Link Service Visual Plug-ins Data Plug-insNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Open Cross-Media Authoring ...  Integration of a new resource type includes  development of data and visual plug-ins  registration with the resource plug-in repository  Cross-media browser loads plug-ins on demand  internal or external (third party application) visualisation  Third party applications delegate the visualisation of a link target to the cross-media browser  similar to "integration" of third-party applications in web browsers  Non-monolithic link model and authoring tool  resource-specific plug-insNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Open Cross-Media Browser and Editor showEntity() showEntity(), getSelectedEntity() Menu Source Window Target Window C R R S U L D Cross-media browser & editor Client applicationNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Visual Plug-ins  Cross-media browser deals only with RSL-specific data but not with any media-specific details  create/delete links (CRUD), user management, …  Definition of link source and target anchors (resources or selectors) handled by visual plug-ins  separation of concerns  Two basic purposes  rendering of specific resource types (resources and selectors)  creation and deletion of resources and selectors  Well-defined visual plug-in APINovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • Conclusions  Time to question existing document formats which are based on the simulation of paper on desktop computers  RSL-based representation of open and fluid cross-media information spaces  interactive paper solutions  associative file system  MindXpres presentation tool  crosslet-based fluid multimodal interfaces  open cross-media information browser and editor  Interlinked structural components rather than a document-centric representation of information  data no longer "owned" by applicationsNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • References  Beat Signer, Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, ISBN 978-3-8370-2713-6, May 2008  Vannevar Bush, As We May Think, Atlanic Monthly, July 1945  Beat Signer and Moira C. Norrie, As We May Link: A General Metamodel for Hypermedia Systems, Proceedings of ER 2007, Auckland, New Zealand, November 2007November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • References ...  Beat Signer, What is Wrong with Digital Documents? A Conceptual Model for Structural Cross- Media Content Composition and Reuse, Proceedings of ER 2010, Vancouver, Canada, November 2010  Moira C. Norrie, Beat Signer, Michael Grossniklaus, Rudi Belotti, Corsin Decurtins and Nadir Weibel, Context- Aware Platform for Mobile Data Management, Journal on Wireless Networks (WINET), Vol. 13, No. 6, Springer, December 2007  ArtVis  http://wise.vub.ac.be/content/artvis-exploring- information-through-advanced-visualisation-techniquesNovember 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • References ...  Axel Vogelsang and Beat Signer, The Lost Cosmonaut: An Interactive Narrative Environment on Basis of Digitally Enhanced Paper, Proceedings of International Conference on Virtual Storytelling, Strasbourg, France, December 2005  Christophe Scholiers, Lode Hoste, Beat Signer and Wolfgang De Meuter, Midas: A Declarative Multi-Touch Interaction Framework, Proceedings of TEI 2011, Funchal, Portugal, January 2011November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • References ...  Lode Hoste, Bruno Dumas and Beat Signer, Mudra: A Unified Multimodal Interaction Framework, Proceedings of ICMI 2011, Alicante, Spain, November 2011  Beat Signer, Ueli Kurmann and Moira C. Norrie, iGesture: A General Gesture Recognition Framework, Proceedings of ICDAR 2007, Curitiba, Brazil, September 2007  Lode Hoste, Bruno Dumas and Beat Signer, SpeeG: A Multimodal Speech- and Gesture-based Text Input Solution, Proceedings of AVI 2012, Capri Island, Italy, May 2012November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be
    • References ...  Beat Signer and Moira C. Norrie, A Model and Architecture for Open Cross-Media Annotation and Link Services, Information Systems 36(3), Elsevier, May 2011November 2, 2012 Beat Signer - Department of Computer Science - bsigner@vub.ac.be