Your SlideShare is downloading. ×
Cross-Media Information Spaces and Architectures (CISA)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Cross-Media Information Spaces and Architectures (CISA)

2,878
views

Published on

Published in: Science, Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,878
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
36
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. 2 December 2005Cross-Media Information Spaces and ArchitecturesProf. Beat Signerhttp://www.beatsigner.comWeb & Information Systems Engineering Lab (WISE)Department of Computer ScienceVrije Universiteit BrusselWEB & INFORMATIONSYSTEMS ENGINEERING
  • 2. 01001000100010100110001001010001001001001000100100010001000010010001000100011000010010001100101010101101100010010100010010010010100010101011001101001100001100 10101010000101000100 11010101010100100 010101010001001 1111111111111 00000000100 111111110101001011010101010111010101001010010110Fluid Cross-Media Information Spaces
  • 3. Prof. Dr. Beat SignerInteractive Paper, Cross-MediaInformation ArchitecturesDr. Bruno DumasMultimodal Interaction, Cross-Media Information ArchitecturesLode HosteMultimodal Interaction, Midasand Mudra FrameworkBrecht De RoomsImplicit Context-aware Human-Computer Interaction, HCIICISAHuman-Machine &Human-InformationInteractionInformationSystems &ManagementInformationVisualisation& NavigationWEB & INFORMATIONSYSTEMS ENGINEERINGCROSS-MEDIA INFORMATION SPACESAND ARCHITECTURES (CISA)
  • 4. Sandra TrullemansPersonal Cross-MediaInformation ManagementReinout RoelsMindXpres: Extensible Content-driven Presentation ToolAhmed A.O. TayehOpen Cross-Media Authoring,Fluid Document FormatsWEB & INFORMATIONSYSTEMS ENGINEERINGCISAHuman-Machine &Human-InformationInteractionInformationSystems &ManagementInformationVisualisation& NavigationCROSS-MEDIA INFORMATION SPACESAND ARCHITECTURES (CISA)
  • 5. Methodology Conceptual ModellingGeneral FrameworksApplicationsUser StudiesRapidPrototyping
  • 6. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013PARC User Interface (Mid 70s) Desktop metaphor WIMP interaction "filing cabinets" withhierarchical folders Paper paradigm "What You See Is WhatYou Get" (WYSIWYG) what about richerdocument formats thatexisted at that time?Xerox Star 8010 [http://www.digibarn.com/collections/systems/xerox-8010/]
  • 7. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Digital Documents as a Paper Simulator?Vannevar BushTed NelsonMost people dont understand the logic of theconcept: "What You See Is What You Get" isbased on printing the document out ("get"means "get WHEN YOU PRINT IT OUT"). And thatmeans a metaphysical shift: a document canonly consist of what can be printed! [...] Nooverlays [...] – PAPER UNDER GLASS.When data of any sort are placed in storage,they are filed alphabetically or numerically,and information is found (when it is) bytracing it down from subclass to subclass. Itcan be in only one place, unless duplicatesare used [...] The human mind does not workthat way. It operates by association.As We May Think, Atlantic Monthly, July 1945Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009
  • 8. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013What is Wrong with Digital Documents? Existing document formats are based on the simulationof paper affordances on desktop computers How to manage mixed-media "documents" in open andfluid cross-media information spaces? on the data level- context-sensitive adapatation- cross-media transclusion on the visualisation and navigation level- zoomable user interfaces on the cross-media interaction level- fluid multimodal cross-media interfaces Remediation of the "paper simulation" approach WYSIWYG is only one out of many options!CISAHuman-Machine &Human-InformationInteractionInformationSystems &ManagementInformationVisualisation& Navigation
  • 9. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013RSL Metamodel and iServerGlobal Information Systems Group, ETH Zurich
  • 10. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013LinksUsersLayersPlug-insStructureActiveContentGlobal Information Systems Group, ETH Zurich
  • 11. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Bridging the Paper-Digital Divide
  • 12. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Paper-Digital IntegrationAmazon Kindle Anoto Digital Pen and Paper TechnologyAugmented / Interactive PaperElectronic Paper
  • 13. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Digital Pen and Paper Technology Anoto offers camera technology pattern license(virtual paper spaceof 60 million km2) Pen manufacturers Nokia Maxell Adapx Livescribe
  • 14. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Digital PensNokia SU-1B Magicomm G303 Livescribe EchoSmartpenFLY FusionPentop Computer
  • 15. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Cross-Media Link Server (iServer)Global Information Systems GroupETH Zurich
  • 16. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Interactive Paper Applications Enhanced Reading paper-digital maps interactive festival brochures educational materials auction brochures scientific publications … Enhanced Writing photo album researchers notebook mammography annotation query by sketching … Paper-based Interfaces PaperPoint document proof-editing digital libraries video analysis tool educational games Art Installations Lost Cosmonaut Generosa Enterprise Interactive Tabletops photo browser collaborative sketch tool
  • 17. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Interactive Festival Guide (EdFest)Global Information Systems GroupETH Zurich
  • 18. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013EdFestGlobal Information Systems Group, ETH Zurich
  • 19. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Visual and Interaction Design Visualisation of interactiveareas and functionality design patterns Interaction design online versus offline inputprocessing lack of modal dialogues multimodal interaction Findings might be appli-cable to traditional GUIs e.g. cross-application widgetinteraction Global Information Systems GroupETH Zurich
  • 20. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013PaperPoint Presentation Tool Mobile PowerPointpresentation tool non-linear presentations real-time annotations digital whiteboard ... Multi-pen support brainstormingGlobal Information Systems GroupETH Zurich
  • 21. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Paper-Digital MapsSzabolcs Becze
  • 22. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013RSL-based Associative File SystemsignerMy PicturesVancouver ER 2010GranvilleIslandVancouverHarbourPublicMarketKeynoteJ. ThorpOlga andLamiaDinnerER 2010multiple classificationstructural linknavigational linkMy VideosVancouverAquariumER 2010Keynotecross-media associationMy PresentationsER 2010 WISE 2009cross-mediatransclusionsingle slidetransclusion(via OOXMLRSL selector)Gregory Cardone
  • 23. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013RSL-based Associative File System ...Gregory Cardone
  • 24. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Personal Information Management Keeping, organising andre-finding information digital and physical Investigating user-information interaction Conceptual PIM model(based on RSL) Cross-media PIM system document tracking context model ...Sandra Trullemans
  • 25. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Interactive TablesPen-Based Google Earth Browser Pen-Based Drawing ToolPhoto Browser Photo Browser (Slider)Global Information Systems Group, ETH Zurich
  • 26. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013What is Wrong with Slideware? Simulation of physical slides limited space due to the slide concept linear navigation from slide to slide difficult to reuse content and embed rich media types MindXpres addresses these issues unlimited canvas with zoomable user interface non-linear navigation and associative linking content-based approach with automatic visualisation
  • 27. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013MindXpres Presentation Tool HTML5-based extensiblecore with three types ofJavaScript plug-ins components- source code, graph, video, ... containers- slide, image gallery, ... structures- grid, graph, clustered, ... Presentation as RSL-based overlay structure enables navigation acrossmultiple presentations Reinout Roels
  • 28. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013MindXpres Presentation Tool ...Reinout Roels
  • 29. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013MindXpres Presentation Tool ... Functional MindXpres prototype Ongoing work visual authoring tool- alternative to XML-based authoring rich media plug-ins- videos with subtitles, annotations etc.- statistical data with alternative visualisations- live sensor data- ... audience-driven presentations collaborative annotations and information sharing multimodal interaction (e.g. integration of PaperPoint concepts) import of existing slideware formats
  • 30. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013ArtVis Advanced visualisationtechniques in combinationwith a TUI explore Web Gallery of Art faceted browsing Phidgets-based TUI RFID-tagged physicalobjects Three main componentsto explore, analyse andbrowse the informationBram Moerman
  • 31. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013ArtVis ...Bram Moerman
  • 32. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013ArtVis ...Bram Moerman
  • 33. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Video: ArtVisBram Moerman
  • 34. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Multimodal and Multi-Touch InteractionEPOC, emotiv Echo Smartpen, LivescribeKinect,Microsoft
  • 35. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013 Midas/Mudra Framework declarative definition of multimodaland multi-touch interactions rule-based language approach rapid protoyping and application development iGesture Workbench create and test gesture sets and algorithms different modalities: digital pen, Wii remote, … open source (www.igesture.org) HephaisTK Toolkit domain-specific human-computer dialoguelanguage (SMUIML) for multimodal I/O speech recognisers, RFID readers, …Multimodal and Multi-Touch Interaction
  • 36. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Mudra Declarative rule-based language Fusion across different levels of abstraction fact base Rapid prototyping simple integration of new input devices integration of external gesture recognisers
  • 37. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Home Automation / Office of the Future
  • 38. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013SpeeG2UserSpeech recognition(Microsoft SAPI 5.4)Skeletal tracking(Microsoft Kinect)5423SpeeG2 GUI61Sven De Kock
  • 39. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Video: SpeeG2 Performance (21 WPM)Sven De Kock
  • 40. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Video: Kinect Presenter
  • 41. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Video: Augmented Reality (Mental Nomad)Toon Duwee
  • 42. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Fluid Multimodal Cross-Media Interfaces Investigation of mobile and distributed document formats Mobile Cross-Media Applications Passive or active information units (crosslets) processing and storage of sensor data binding of specific multimodal interactions to resources Implicit context-aware human-computer interaction (HCII)
  • 43. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Student Projects Interactive paper and augmented reality various apprenticeships and thesis proposals in thearea of interactive paper and augmented reality Multimodal and Multi-touch Interaction next generation text input devices (e.g. combine voice and Kinect) multimodal mobile interaction Cross-Media Information Spaces and Architectures fluid cross-media document formats MindXpres presentation tool, office of the future, PIM, ... If you should be interested in one of these topics(or if you have your own ideas) please contact us! http://wise.vub.ac.be/thesis_proposals
  • 44. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013WISE Courses Advanced Topics in Information Systems seminar about recent developments in information systems information systems and information management information visualisation and navigation human-machine and human-information interaction [http://wise.vub.ac.be/content/advanced-topics-information-systems] Next Generation User Interfaces general HCI introduction multimodal interaction (theoretical concepts, fusion and fission, ...) adaptation pen-based interaction tangible interaction and augmented reality [http://wise.vub.ac.be/content/next-generation-user-interfaces]WEB & INFORMATIONSYSTEMS ENGINEERING
  • 45. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013WISE Courses ... User Interface Design general HCI introduction- theory- modelling- evolution new forms of human-machine interaction [http://wise.vub.ac.be/node/15] Web Information Systems hypermedia and the World Wide Web Internet architecture and protocols Web 2.0, Rich Internet Applications and Web 3.0 web search [http://wise.vub.ac.be/content/web-information-systems]WEB & INFORMATIONSYSTEMS ENGINEERING
  • 46. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013Conclusions Innovative cross-media document formats which gobeyond simulating paper on desktop computers RSL-based representation of open and fluid cross-mediainformation spaces interactive paper solutions personal information management and associative file systems MindXpres presentation tool Multimodal and multi-touch interaction Midas/Mudra, HephaisTK and iGesture SpeeG2, Kinect Presenter Augmented reality ArtVis and Mental Nomad
  • 47. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013References B. Signer and M.C. Norrie, As We May Link: A GeneralMetamodel for Hypermedia Systems, Proceedings of the26th International Conference on Conceptual Modeling(ER 2007), Auckland, New Zealand, November 2007 http://wise.vub.ac.be/sites/default/files/publications/ER2007.pdf Bachelor and Master Thesis Proposals http://wise.vub.ac.be/thesis_proposals B. Signer, Fundamental Concepts forInteractive Paper and Cross-Media InformationSpaces, ISBN 978-3-8370-2713-6, May 2008
  • 48. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013References ... B. Signer and M.C. Norrie, Interactive Paper: Past,Present and Future, Proceedings of the 1st InternationalWorkshop on Paper Computing (PaperComp 2010),Copenhagen Denmark, September 2010 http://wise.vub.ac.be/sites/default/files/publications/PaperComp2010.pdf B. Signer and M.C. Norrie, PaperPoint: A Paper-based Presentation and Interactive Paper PrototypingTool, Proceedings of the First International Conferenceon Tangible and Embedded Interaction (TEI 2007),Baton Rouge, USA, February 2007 http://wise.vub.ac.be/sites/default/files/publications/Signer2007.pdf
  • 49. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013References ... B. Signer, M. Grossniklaus and M.C.Norrie,Interactive Paper as a Mobile Client for a Multi-ChannelWeb Information System, World Wide Web Journal(WWW), Vol. 10, No. 4, Springer, December 2007 http://wise.vub.ac.be/sites/default/files/publications/WWWJ2007.pdf L. Hoste, B. Dumas and B. Signer, Mudra: A UnifiedMultimodal Interaction Framework, Proceedings of the13th International Conference on Multimodal Interaction(ICMI 2011), Alicante, Spain, November 2011 http://wise.vub.ac.be/sites/default/files/publications/ICMI2011.pdf
  • 50. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013References ... B. Signer, U. Kurmann and M.C. Norrie. M.C.,iGesture: A General Gesture Recognition Framework,Proceedings of the 9th International Conference onDocument Analysis and Recognition (ICDAR 2007),Curitiba, Brazil, September 2007 http://wise.vub.ac.be/sites/default/files/publications/ICDAR2007.pdf L. Hoste, B. Dumas and B. Signer, SpeeG: A MultimodalSpeech- and Gesture-based Text Input Solution,Proceedings of the International Working Conference onAdvanced Visual Interfaces (AVI 2012), Capri Island,Italy, May 2012 http://wise.vub.ac.be/sites/default/files/publications/AVI2012.pdf
  • 51. Beat Signer - Department of Computer Science - bsigner@vub.ac.beApril 24, 2013References ... L. Hoste and B. Signer, Expressive Control ofIndirect Augmented Reality During Live MusicPerformances, Proceedings of the 13th InternationalConference on New Interfaces for Musical Expression,Daejeon (NIME 2013), Korea Republic, May 2013 ArtVis Project http://wise.vub.ac.be/content/artvis-exploring-information-through-advanced-visualisation-techniques