Analytical Profile of Coleus Forskohlii | Forskolin .pptx
Introducing Tangible Holograms for Data Physicalisation and Big Data Exploration
1. 2 December 2005
Introducing Tangible Holograms for
Data Physicalisation and Big Data Exploration
Prof. Beat Signer
http://www.beatsigner.com
Web & Information Systems Engineering Lab
Department of Computer Science
Vrije Universiteit Brussel
WEB & INFORMATION
SYSTEMS ENGINEERING
3. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 3September 23, 2017
Prof. Dr. Beat Signer
Interactive Paper, Cross-Media
Information Architectures
Audrey Sanctorum
User-defined Cross-Device and
Cross-Media Interaction
Reinout Roels
MindXpres: Extensible Content-
driven Presentation Tool
CISA
Human-Machine &
Human-Information
Interaction
Information
Systems &
Management
Information
Visualisation
& Navigation
WEB & INFORMATION
SYSTEMS ENGINEERING
CROSS-MEDIA INFORMATION SPACES
AND ARCHITECTURES (CISA)
Payam Ebrahimi
Data Physicalisation
4. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 4September 23, 2017
Dr. Ahmed A.O. Tayeh
Open Cross-Media Authoring,
Fluid Document Formats
WEB & INFORMATION
SYSTEMS ENGINEERING
CISA
Human-Machine &
Human-Information
Interaction
Information
Systems &
Management
Information
Visualisation
& Navigation
CROSS-MEDIA INFORMATION SPACES
AND ARCHITECTURES (CISA)
Kushal Soni
Smart Environments, Implicit
Human-Computer Interaction
Jan Maushagen
Learning Analytics, Adaptive
Persuasive ICT Tools
Sandra Trullemans
Personal Cross-Media
Information Management
5. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 5September 23, 2017
WEB & INFORMATION
SYSTEMS ENGINEERING
CISA
Human-Machine &
Human-Information
Interaction
Information
Systems &
Management
Information
Visualisation
& Navigation
CROSS-MEDIA INFORMATION SPACES
AND ARCHITECTURES (CISA)
Lars Van Holsbeeke
Smart Environments, Implicit
Human-Computer Interaction
6. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 6September 23, 2017
Cross-Media Information Spaces and Architectures
Information
Systems &
Management
Information
Visualisation
& Navigation
Information
Visualisation
nd NavigationHuman-Machine &
Human-Information
Interaction
CISA
RSL [4]
MindXpres [7]
OC2 [5]
associate file system [2]
data-driven storytelling [10]
XIMA [12]
cross-device interaction [8]
iServer/iPaper [3]
ViDaX [15]
data physicalisation [30]
map-based interaction [17]
non-linear storytelling [25]
PaperProof [27]iGesture [21]
Query by Sketch [28]
PaperSketch [29]
Context Modelling Toolkit [9]
Midas [19]
SpeeG2 [22]
PaperPoint [24]
Print-n-Link [16]
digital libraries [11]
Mudra [20] mixed reality [26]
ArtVis [14]
open cross-media linking [1]
PimVis [13]
DocTr [6]
EdFest [23]
source code visualisation [18]
7. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 7September 23, 2017
ArtVis: TUI-based Information Exploration
Advanced visualisation
techniques in combination
with a TUI
explore Web Gallery of Art
faceted browsing
phidgets-based TUI
RFID-tagged physical
objects
Three main components
to explore, analyse and
browse the information
Bram Moerman
8. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 8September 23, 2017
ArtVis Setup
Bram Moerman
9. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 9September 23, 2017
ArtVis Visualisation
Bram Moerman
10. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 10September 23, 2017
Tangible Bits (Ishii & Ullmer, 1997)
Beyond GUIs ("painted bits")
Tangible User Interfaces (TUIs)
augment the physical space by
coupling digital information to
everyday objects and environments
physical instantiation of GUI
elements in TUI
Key concepts
interactive surfaces
unidirectional coupling of atoms and bits
ambient media (inspired by Life Wire)
Tangible bits mainly act as input controls for digital
information and services (output on computer screens)
Hiroshi Ishii Brygg Ullmer
11. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 11September 23, 2017
Radical Atoms (Ishii et al., 2012)
Vision taking a leap beyond Tangible Bits
assuming a hypothetical generation of materials that can
dynamically change their form and appearance
vision-driven design research
Radical Atoms is
a computationally transformable and reconfigurable material that
is bidirectionally coupled with an underlying digital model (bits)
- changes to digital model are reflected in physical representation and vice versa
a future material that can transform its shape, conform to
constraints and inform the users of its affordances
a vision for the future of human-material interaction and Material
User Interfaces (MUIs) in which all digital information has a
physical manifestation so that we can directly interact with it
12. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 12September 23, 2017
Radical Atoms …
Transform
interface can transform its
shape to modify the model
and reflect changes in the
computational model
Conform
interface has to conform to
some physical laws and user
constraints (e.g. for safety)
Inform
user has to be informed
about changing interface
affordances
13. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 13September 23, 2017
Interactions with Radical Atoms
Direct touch and gestural interaction
gestures coupled with direct touch create an interaction
appropriate for Radical Atoms since users are able to rapidly
reform dynamic materials at all scales
Context-aware transformation
context-aware transformations of the tool/interface
- e.g. screwdriver adapting to the type of screw it is operating on
Shape-memory clay: Perfect Red
Perfect Red is a fictional material that can be sculpted like and
responds according to rules inspired by CAD operations
- e.g. if we split a piece in two even halves, then the operations performed on
one part are mirrored on the other part
14. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 14September 23, 2017
TRANSFORM
TRANSFORM, MIT
Dynamic shape display
that can physically render
3D content
tangible interaction with
digital content
- geospatial data
- 3D modelling
- …
TRANSFORM display can also
interact with the physical world around it
remote users can be displayed physically
Step towards MIT’s vision of Radical Atoms
15. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 15September 23, 2017
Video: TRANSFORM
16. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 16September 23, 2017
Data Physicalisation (Jansen et al., 2015)
Physical objects used for output
often static encoding of digital
information
physical visualisations can outperform
their virtual counterparts
How can we achieve dynamic data
physicalisation with dynamic affordances
e.g. 3D visualisation with additional dimensions represented
by physical variables such as temperature, texture, weight or
plasticity
TRANSFORM is one possible approach; tangible holograms are
another experimental mobile solution
- can be used for multidimensional collaborative exploration and analysis of
big data sets
17. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 17September 23, 2017
Mixed Reality
"The perceived congruence concerns both visual as well
as tactile perception. A perfect hologram will appear
embodied but trying to touch it will destroy the illusion."
Microsoft HoloLens
Jansen, 2014 [34]
18. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 18September 23, 2017
Tangible Hologram (TangHo) Setup
Timothy J. Curtin
19. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 19September 23, 2017
Tangible Graph Hologram
20. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 20September 23, 2017
Collaborative TangHo Interaction
21. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 21September 23, 2017
Robotic Arm Prototype
Lego Mindstorms-based
6DOF arm prototype
motors can be locked or set to
float mode
- useful for bidirectional I/O
replaceable sphere for different
types of non-visual feedback
Challenges
inverse kinematics (final system
will be body mounted)
- use robotics toolbox (MATLAB)
limited HoloLens hand tracking
3D printing of final arm
Timothy J. Curtin
22. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 22September 23, 2017
TangHo Prototype
Timothy J. Curtin
23. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 23September 23, 2017
TangHo Interactions
Timothy J. Curtin
24. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 24September 23, 2017
Data Physicalisation Framework
Inspired by research agenda of Jansen et al. [34]
Understanding the design space
how do users map data to physicalisations
Understanding the perceptual effectiveness
e.g. what are the just-noticeable differences that a physical
variable can convey?
Framework for dynamic data physicalisation
data processing with device-independent output
- TextDIO: text for static or dynamic reports
- PhysDIO: data points, data variables and physical variables
text and visualisation drivers for representation on specific
devices
- e.g. tangible hologram prototype
25. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 25September 23, 2017
Conclusions
Tangible holograms for data physicalisation and
simulation of radical atoms
physical augmentation of highly configurable virtual objects rather
than programming of physical material (programmable matter)
mobile solution that is no longer bound to a specific place such as
a the TRANSFORM table
modular solution that can be configured for specific domains
(e.g. different spheres for various physical variables)
Data physicalisation framework for investigating new
opportunities in dynamic data physicalisation
new data physicalisation design guidelines (as in InfoVis)
new evaluation techniques
…
26. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 26September 23, 2017
References
B. Signer, Fundamental Concepts for
Interactive Paper and Cross-Media Information
Spaces, ISBN 978-3-8370-2713-6, May 2008
ArtVis Project
http://www.beatsigner.com/flyers/ArtVis.pdf
Tangible Holograms
http://beatsigner.com/tangibleHolograms.html
TRANSFORM
https://www.youtube.com/watch?v=lCARHatJQJA
27. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 27September 23, 2017
References ...
[1] B. Signer and M.C. Norrie, A Model and
Architecture for Open Cross-Media Annotation and
Link Services, Information Systems 36(3), Elsevier,
May 2011
http://beatsigner.com/publications/signer_IS2011.pdf
[2] B. Signer, What is Wrong with Digital Documents?
A Conceptual Model for Structural Cross-Media
Content Composition and Reuse, Proceedings of the
29th International Conference on Conceptual Modeling
(ER 2010), Vancouver, Canada, November 2010
http://beatsigner.com/publications/signer_ER2010.pdf
28. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 28September 23, 2017
References ...
[3] B. Signer, Fundamental Concepts for Interactive
Paper and Cross-Media Information Spaces,
Dissertation ETH No. 16218, Zurich, Switzerland,
2005
http://beatsigner.com/publications/signer2006.pdf
[4] B. Signer and M.C. Norrie, As We May Link: A General
Metamodel for Hypermedia Systems, Proceedings of
the 26th International Conference on Conceptual
Modeling (ER 2007), Auckland, New Zealand,
November 2007
http://beatsigner.com/publications/signer_ER2010.pdf
29. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 29September 23, 2017
References ...
[5] S. Trullemans and B. Signer, Towards a Con-
ceptual Framework and Metamodel for Context-Aware
Personal Cross-Media Information Management
Systems, Proceedings of ER 2014, 33rd International
Conference on Conceptual Modelling, Atlanta, USA,
October, 2014
http://beatsigner.com/publications/trullemans_ER2014.pdf
[6] S. Trullemans, A. Vercruysse and B. Signer, DocTr:
A Unifying Framework for Tracking Physical Docu-
ments and Organisational Structures, Proceedings of
EICS 2016, Brussels, Belgium, June 2016
http://beatsigner.com/publications/trullemans_EICS2016.pdf
30. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 30September 23, 2017
References ...
[7] R. Roels and B. Signer, MindXpres: An Exten-
sible Content-driven Cross-Media Presentation
Platform, Proceedings of WISE 2014, 15th Interna-
tional Conference on Web Information System
Engineering, Thessaloniki, Greece, October, 2014
http://beatsigner.com/publications/roels_WISE2014.pdf
[8] A. Sanctorum and B. Signer, Towards User-defined
Cross-Device Interaction, Proceedings of DUI 2016,
5th Workshop on Distributed User Interfaces, Lugano,
Switzerland, June 2016
http://beatsigner.com/publications/sanctorum_DUI2016.pdf
31. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 31September 23, 2017
References ...
[9] S. Trullemans and B. Signer, A Multi-layered
Context Modelling Approach for End Users, Expert
Users and Programmers, Proceedings of the Interna-
tional Workshop on Smart Ecosystems cReation by
Visual dEsign (SERVE 2016), Bari, Italy, June 2016
http://beatsigner.com/publications/trullemans_SERVE2016.pdf
[10] R. Roels, Y. Baeten and B. Signer, An Interactive
Data Visualisation Approach for Next Generation
Presentation Tools: Towards Rich Presentation-based
Data Exploration and Storytelling, Proceedings of
CSEDU 2016, Rome, Italy, April, 2016
http://beatsigner.com/publications/roels_CSEDU2016.pdf
32. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 32September 23, 2017
References ...
[11] M.C. Norrie, B. Signer and N. Weibel, Interactive
Paper as a Reading Medium in Digital Libraries, Pro-
ceedings of ECDL 2008, 12th European Conference
on Research and Advanced Technology for Digital
Libraries, Aarhus, Denmark, September 2008
http://beatsigner.com/publications/norrie_ECDL2008.pdf
[12] B. Signer, M.C. Norrie, P. Geissbuehler and
D. Heiniger, Telephone Interface for Avalanche War-
nings based on Information Server for Adaptable Con-
tent Delivery, Proceedings of Pervasive 2002, Zurich,
Switzerland, August 2002
http://beatsigner.com/publications/signer_Pervasive2002.pdf
33. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 33September 23, 2017
References ...
[13] S. Trullemans, A. Sanctorum and B. Signer,
PimVis: Exploring and Re-finding Documents in Cross-
Media Information Spaces, Proceedings of the Inter-
national Working Conference on Advanced Visual
Interfaces (AVI 2016), Bari, Italy, June 2016
http://beatsigner.com/publications/trullemans_AVI2016.pdf
[14] B. Dumas, B. Moerman, S. Trullemans and B. Signer,
ArtVis: Combining Advanced Visualisation and Tan-
gible Interaction for the Exploration, Analysis and
Browsing of Digital Artwork Collections, Proceedings of
AVI 2014, Como, Italy, May 2014
http://beatsigner.com/publications/dumas_AVI2014.pdf
34. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 34September 23, 2017
References ...
[15] B. Dumas, T. Broché, L. Hoste and B. Signer,
ViDaX: An Interactive Semantic Data Visualisation and
Exploration Tool, Proceedings of AVI 2012, Internatio-
nal Working Conference on Advanced Visual Inter-
faces, Capri Island, Italy, May 2012
http://beatsigner.com/publications/dumas_AVI2012.pdf
[16] Moira C. Norrie, Beat Signer and Nadir Weibel,
Print-n-Link: Weaving the Paper Web, Proceedings of
DocEng 2006, ACM Symposium on Document
Engineering, Amsterdam, The Netherlands,
October 2006
http://beatsigner.com/publications/norrie_DocEng2006.pdf
35. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 35September 23, 2017
References ...
[17] M. Grossniklaus, M.C. Norrie, B. Signer and
N. Weibel ,Putting Location-Based Services on the
Map, Proceedings of W2GIS 2006, Hong Kong, China,
December 2006
http://beatsigner.com/publications/grossniklaus_W2GIS2006.pdf
[18] R. Roels, P. Mestereaga and B. Signer, An Interactive
Source Code Visualisation Plug-in for the MindXpres
Presentation Platform, Communications in Computer
and Information Science (CCIS), 583, 2016
http://beatsigner.com/publications/roels_CCIS2016.pdf
36. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 36September 23, 2017
References ...
[19] C. Scholiers, L. Hoste, B. Signer and W. De
Meuter, Midas: A Declarative Multi-Touch Interaction
Framework, Proceedings of TEI 2011, 5th Interna-
tional Conference on Tangible, Embedded and Em-
bodied Interaction, Funchal, Portugal, January 2006
http://beatsigner.com/publications/scholliers_TEI2011.pdf
[20] L. Hoste, B. Dumas and B. Signer, Mudra: A Unified
Multimodal Interaction Framework, Proceedings of
ICMI 2011, 13th International Conference on Multi-
modal Interaction (ICMI 2011), Alicante, Spain,
November 2011
http://beatsigner.com/publications/scholliers_TEI2011.pdf
37. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 37September 23, 2017
References ...
[21] B. Signer, U. Kurmann and M.C. Norrie,
iGesture: A General Gesture Recognition Framework,
Proceedings of the 9th International Conference on
Document Analysis and Recognition (ICDAR 2007),
Curitiba, Brazil, September 2007
http://beatsigner.com/publications/signer_ICDAR2007.pdf
[22] L. Hoste and B. Signer, SpeeG2: A Speech- and
Gesture-based Interface for Efficient Controller-free
Text Entry, Proceedings of ICMI 2013, 15th Inter-
national Conference on Multimodal Interaction,
Sydney, Australia, December 2013
http://beatsigner.com/publications/hoste_ICMI2013.pdf
38. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 38September 23, 2017
References ...
[23] B. Signer, M. Grossniklaus and M.C. Norrie,
Interactive Paper as a Mobile Client for a Multi-
Channel Web Information System, World Wide Web
Journal (WWW), Vol. 10, No. 4, Springer,
December 2007
http://beatsigner.com/publications/signer_WWWJ2007.pdf
[24] B. Signer and M.C. Norrie, PaperPoint: A Paper-
based Presentation and Interactive Paper Prototyping
Tool, Proceedings of TEI 2007, First International
Conference on Tangible and Embedded Interaction,
Baton Rouge, USA, February 2007
http://beatsigner.com/publications/signer_TEI2007.pdf
39. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 39September 23, 2017
References ...
[25] A. Vogelsang and B. Signer, The Lost Cosmo-
naut: An Interactive Narrative Environment on Basis of
Digitally Enhanced Paper, Proceedings of the Inter-
national Conference on Virtual Storytelling 2005,
Strasbourg, France, December 2005
http://beatsigner.com/publications/vogelsang_ICVS2005.pdf
[26] L. Hoste and B. Signer, Expressive Control of Indirect
Augmented Reality During Live Music Performances,
Proceedings of NIME 2013, 13th International Confe-
rence on New Interfaces for Musical Expression,
Daejeon, Korea Republic, May 2013
http://beatsigner.com/publications/hoste_NIME2013.pdf
40. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 40September 23, 2017
References ...
[27] N. Weibel, A. Ispas, B. Signer and M.C. Norrie,
PaperProof: A Paper-Digital Proof-Editing System,
Proceedings of CHI 2008, 26th International Confe-
rence on Human Factors in Computing Systems
(Interactivity Track), Florence, Italy, April 2008
http://beatsigner.com/publications/weibel_CHI2008.pdf
[28] A. Ispas, H. Schuldt, M.C. Norrie and B. Signer,
Towards Query by Sketch, Michael Springmann,
Second DELOS Conference on Digital Libraries, Pisa,
Italy, December 2007
http://beatsigner.com/publications/springmann_DELOS2007.pdf
41. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 41September 23, 2017
References ...
[29] N. Weibel, B. Signer, M.C. Norrie, H. Hofstetter,
H.-C. Jetter and H. Reiterer, PaperSketch: A Paper-
Digital Collaborative Remote Sketching Tool, Proceed-
ings of IUI 2011, International Conference on Intelli-
gent User Interfaces, Palo Alto, USA, February 2011
http://beatsigner.com/publications/weibel_IUI2011.pdf
[30] B. Signer and T.J. Curtin, Tangible Holograms:
Towards Mobile Physical Augmentation of Virtual
Objects, Technical Report WISE Lab, WISE-2017-01,
March 2017
http://beatsigner.com/publications/signer_arXiv2017.pdf
42. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 42September 23, 2017
References ...
[31] H. Ishii and B. Ullmer, Tangible Bits: Towards
Seamless Interfaces between People, Bits and Atoms,
Proceedings of CHI 1997, ACM Conference on Human
Factors in Computing Systems, Atlanta, USA,
March 1997
http://dx.doi.org/10.1145/258549.258715
[32] H. Ishii, D. Lakatos, L. Bonanni and J.-B. Labrune,
Radical Atoms: Beyond Tangible Bits, Toward
Transformable Materials, interactions, 19(1),
January 2012
http://dx.doi.org/10.1145/2065327.2065337
43. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 43September 23, 2017
References ...
[33] Y. Jansen et al., Opportunities and Challenges
for Data Physicalizations, Proceedings of CHI 2015,
ACM Conference on Human Factors in Computing
Systems, Seoul, Korea, April 2015
https://doi.org/10.1145/2702123.2702180
[34] Y. Jansen, Physical and Tangible Information
Visualization, 2014, Ph.D. Dissertation, Université
Paris-Sud
https://tel.archives-ouvertes.fr/tel-00981521