3D User Interfaces for Information Systems Based on UsiXML
•Download as PPT, PDF•
1 like•470 views
Since many years, 3D interactive systems have demonstrated some benefits in reproducing adequately the reality, in improving it, and even in augmenting it by providing the user with unprecedented actions. 3D User Interfaces are becoming the primary subject of interest of a growing community of researchers and developers adopting different approaches for specifying and creating 3DUIs. Providing development methods and software support for 3DUIs is a complex problem. In this paper, we argue that developing 3DUIs for Information Systems is an activity that would benefit from the application of a model-driven development methodology composed of: a set of models defined according to an ontology, a language that expresses these models, and a structured method manipulating these models.
Report
Share
Report
Share
![A Model-Driven Approach for Developing 3D User Interfaces of Information Systems in a Principle–Based Way Juan Gonzalez-Calleros, Jean Vanderdonckt UsiXML Workshop EICS, Berlin, 20 June 2010 ,[object Object]](https://image.slidesharecdn.com/gonzalez-usixml2010-100909123737-phpapp02/85/3D-User-Interfaces-for-Information-Systems-Based-on-UsiXML-1-320.jpg)
Recommended
BIM Companies in USA
Mechanical engineering solution is core to any construction project and TechnoStruct LLP as an industry expert ensures that it delivers sustainable construction designs & techniques and complete support throughout the construction project that includes and is not limited to- feasibility studies, system analysis, schematic design, contract documents, design development, HVAC Controls and Plumbing System.
Unified Geometry Modelling for Pervasive Design and Fabrication
Conventional Computer Aided Design (CAD), Computer Aided Engineering (CAE), and Computer Aided Manufacturing (CAM) employed discrete geometry models that could not be easily exchanged and effectively employed for biomech design and fabrication.
SIMTech has patented a common data model for representation of geometry, attributes, physical states and other characteristics of novel manufacturing processes. The method could be extended to integrate these discrete point unit processes into a unified geometrical model for pervasive design and verification.
Showing User Interface Adaptivity by Animated Transitions
In order to reduce the inevitable end user disruption and cognitive perturbation induced by adapting a graphical user interface, the results of the adaptation could be conveyed to the end user by animating a transition scenario showing the evolution from the user interface before adaptation to the user interface after adaptation. A transition scenario consists of a sequence of adaptation operations (e.g., set/change a property of a widget, replace a widget by another, resize a widget) belonging to a catalogue of operations defined as an Extended Backus-Naur Form grammar. Each transition operation has a range from a single widget (e.g., this “Ok” button) to a selection of widgets based on a selector mechanism (e.g., all validation widgets of this family of interfaces). A transition scenario is built either automatically by any adaptation algorithm or interactively by a specific editor for designers. An animator then executes the animation scenario by parsing each adaptation operation one by one or in a grouped mode and by rendering them by an animated transition on a user interface model. The type (e.g., wipe, box in, box out) and parameters (e.g., animation speed, pace, direction) of each animated transition have been selected based on usability guidelines for animation. A user study suggests that a transition scenario reinforces understandability and trust, while still suffering from lag.
Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces (DUIs) have become one vivid area of research and development in
Human-Computer Interaction (HCI) where many dramatic changes occur in the way we can interact
with interactive systems. DUIs attempt to surpass user interfaces that are manipulated only by a single
end user, on the same computing platform, and in the same environment, with little or no variations
among these axes. In contrast to such currently existing user interfaces, DUIs enable end
users to distribute any user interface element, ranging from the largest one to the smallest one,
across one or many of these dimensions at design and/or run-time: across different users, across different computing platforms, and across different physical environments. In this way, end users
could be engaged in distributed tasks that are regulated by distribution rules, many of them being
currently used in the real world. This paper provides a conceptual framework that invites us to rethink
traditional user interfaces in a distributed way based on the locus of distribution control: in the hands of the end user, under control of the system, or in mixed-initiative way. Any user interface submitted to distribution may also be subject to adaptation with respect to the user, the platform, and the environment.
Recommended
BIM Companies in USA
Mechanical engineering solution is core to any construction project and TechnoStruct LLP as an industry expert ensures that it delivers sustainable construction designs & techniques and complete support throughout the construction project that includes and is not limited to- feasibility studies, system analysis, schematic design, contract documents, design development, HVAC Controls and Plumbing System.
Unified Geometry Modelling for Pervasive Design and Fabrication
Conventional Computer Aided Design (CAD), Computer Aided Engineering (CAE), and Computer Aided Manufacturing (CAM) employed discrete geometry models that could not be easily exchanged and effectively employed for biomech design and fabrication.
SIMTech has patented a common data model for representation of geometry, attributes, physical states and other characteristics of novel manufacturing processes. The method could be extended to integrate these discrete point unit processes into a unified geometrical model for pervasive design and verification.
Showing User Interface Adaptivity by Animated Transitions
In order to reduce the inevitable end user disruption and cognitive perturbation induced by adapting a graphical user interface, the results of the adaptation could be conveyed to the end user by animating a transition scenario showing the evolution from the user interface before adaptation to the user interface after adaptation. A transition scenario consists of a sequence of adaptation operations (e.g., set/change a property of a widget, replace a widget by another, resize a widget) belonging to a catalogue of operations defined as an Extended Backus-Naur Form grammar. Each transition operation has a range from a single widget (e.g., this “Ok” button) to a selection of widgets based on a selector mechanism (e.g., all validation widgets of this family of interfaces). A transition scenario is built either automatically by any adaptation algorithm or interactively by a specific editor for designers. An animator then executes the animation scenario by parsing each adaptation operation one by one or in a grouped mode and by rendering them by an animated transition on a user interface model. The type (e.g., wipe, box in, box out) and parameters (e.g., animation speed, pace, direction) of each animated transition have been selected based on usability guidelines for animation. A user study suggests that a transition scenario reinforces understandability and trust, while still suffering from lag.
Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces (DUIs) have become one vivid area of research and development in
Human-Computer Interaction (HCI) where many dramatic changes occur in the way we can interact
with interactive systems. DUIs attempt to surpass user interfaces that are manipulated only by a single
end user, on the same computing platform, and in the same environment, with little or no variations
among these axes. In contrast to such currently existing user interfaces, DUIs enable end
users to distribute any user interface element, ranging from the largest one to the smallest one,
across one or many of these dimensions at design and/or run-time: across different users, across different computing platforms, and across different physical environments. In this way, end users
could be engaged in distributed tasks that are regulated by distribution rules, many of them being
currently used in the real world. This paper provides a conceptual framework that invites us to rethink
traditional user interfaces in a distributed way based on the locus of distribution control: in the hands of the end user, under control of the system, or in mixed-initiative way. Any user interface submitted to distribution may also be subject to adaptation with respect to the user, the platform, and the environment.
Flippable User Interfaces for Internationalization
The language reading direction is probably one of the most determinant factors influencing the successful internationalization of graphical user interfaces, beyond their mere translation. Western languages are read from left to right and top to bottom, while Arabic languages and Hebrew are read from right to left and top to bottom, and Oriental languages are read from top to bottom. In order to address this challenge, we introduce flippable user interfaces that enable the end user to change the reading direction of a graphical user interface by flipping it into the desired reading direction by direct manipulation. This operation automatically and dynamically changes the user interface layout based on a generalized concept of reading direction and translates it according to the end user’s preferences.
Faure vanderdonckt co-summit2013-final
User Interface eXtensible Markup Language (UsiXML) is a formal Domain-Specific Language (DSL) used in Human-Computer Interaction (HCI) and Software Engineering (SE) in order to describe any user interface of any interactive application independently of any implementation technology. A user interface may involve variations depending on: the context of use (in which the user is carrying out her interactive task), the device or the computing platform (on which the user is working), the language (used by the user), the organization (to which the user belongs), the user profile, the interaction modalities (e.g., graphical, vocal, tactile, haptics)
Generative Pattern-Based Design of User Interfaces
This paper suggests a method for developing graphical user interfaces based on generative patterns. A generative pattern contains portions of previously designed user interfaces are expressed through models that are either partially or totally instantiated. These portions could be identified and re-applied to a new design case study by generating code by instantiating the specifications contained in the models. The method involves typical models found in user interface development life cycle such as task, domain, abstract user interface, concrete user interface, final user interface, context model, and mappings between them. Any model could virtually be the source of a pattern and could be described, searched, matched, retrieved, and assembled together so as to create a new graphical user interface. For this purpose, a software has been developed that manages generative patterns by combining an existing user interface description language (UsiXML – user interface extensible markup language) with concepts addressing problems raised by pattern description and matching in a pattern-based language (PLML – Pattern Language Markup Language, a language was introduced to uniformly represent user interface patterns). Once instantiated from the generative patterns, the models give rise to a model-driven engineering based on model-to-model transformation and model-to-code compilation.
Towards Canonical Task Types for User Interface Design
Task models are the cornerstone of user-centred design methodologies for user interface design. Therefore, they deserve attention in order to produce them effectively and efficiently, while guaranteeing the reproducibility of a task model: different persons should in principle obtain the same task model, or a similar one, for the same problem. In order to provide user interface designers with some guidance for task modelling, a list of canonical task types is proposed that offers a unified definition of frequently used tasks types in a consistent way. Each task type consists of a a task action coupled with a task object, each of them being written according to design guidelines. This list provides the following benefits: tasks are modelled in a more consistent way, their definition is more communicable and shared, task models can be efficiently used for model-driven engineering of user interfaces.
User Interface Derivation from Business Processes: A Model-Driven Approach fo...
This presentation defines a model-driven approach for organizational engineering in which user interfaces of information systems are derived from business processes. This approach consists of four steps: business process modeling in the context of organizational engineering, task model derivation from the business process model, task refinement, and user interface model derivation from the task model. Each step contributes to specify and refine map-pings between the source and the target model. In this way, each model modification could be adequately propagated in the rest of the chain. By applying this model-driven approach, the user inter-faces of the information systems are directly meeting the require-ments of the business processes and are no longer decoupled from them. This approach has been validated on a case study in a large bank-insurance company
IRJET- A Quality Watch Android Based Application for Monitoring Robotic A...
https://www.irjet.net/archives/V5/i3/IRJET-V5I3331.pdf
Development Prototype Design of Virtual Assembly Application-Based Leap Motion
Innovation in design engineering practice is very important in the world of manufacturing in the increasingly competitive global market. Prototyping and evaluation measures are inseparable from the design process in the manufacture of a product. And made one of many physical prototypes require very expensive and time consuming, so the technology of Virtual Reality (VR) is needed, so the industry can quickly and precisely in the decision. VR technology combines a human being with a computer environment visually, touch and hearing, so that the user as if into the virtual world. The goal is that users with hand movements can interact with what is displayed on the computer screen or the user can interact with the environment is unreal to be added into the real world. VR is required for simulations that require a lot of interaction such as prototype assembly methods, or better known as the Virtual Assembly. Virtual Assembly concept which was developed as the ability to assemble a real representation of the physical model, the 3D models in CAD software by simulating the natural movement of the human hand. Leap Motion (accuracy of 0.01mm) was used to replace Microsoft's Kinect (accuracy of 1.5cm) and Motion Glove with flex sensors (accuracy of 1°) in several previous research. Leap mot ion controller is a device that captures every movement of the hand to then be processed and integrated with 3D models in CAD software. And simulation of assembly process virtually in CA D software with hand gestures detected by the leap mot ion, assembly parts can be driven either in translation or rotation, zooming and adding the assembly constraint. It also can perform mouse functions (such as left-click, middle-click, right-click and move the mouse cursor position) to a virtual assembly process simulation on CAD software.
Customizing Model of Mobile Service Computing on Cloud of Things
https://irjet.net/archives/V4/i3/IRJET-V4I3359.pdf
IRJET - Hand Gestures Recognition using Deep Learning
https://www.irjet.net/archives/V7/i3/IRJET-V7I31032.pdf
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-driven engineering (MDE) of user interfaces consists in describing a user interface and aspects involved in it (e.g., task, domain, context of use) in models from which a final interface is produced. With one big win in mind: when the user’s requirements or the context of use change, the models change accordingly and so does the supporting user interface. Models and a method for developing user interfaces based on MDE are presented in this tutorial supporting forward engineering (a new interface is produced), reverse engineering (an existing interface is improved), and lateral engineering (an existing interface is adapted to a new context of use). Software supporting this method will be used based on UsiXML (User Interface eXten-sible Markup Language), a XML-compliant user interface description language.
Advancing Electrical Design in High-Rise Buildings with Revit 3D Modeling
https://www.irjet.net/archives/V10/i11/IRJET-V10I11128.pdf
IRJET- Application of Revit as Building Information Modeling (BIM) for Integr...
https://www.irjet.net/archives/V5/i10/IRJET-V5I1003.pdf
More Related Content
Viewers also liked
Flippable User Interfaces for Internationalization
The language reading direction is probably one of the most determinant factors influencing the successful internationalization of graphical user interfaces, beyond their mere translation. Western languages are read from left to right and top to bottom, while Arabic languages and Hebrew are read from right to left and top to bottom, and Oriental languages are read from top to bottom. In order to address this challenge, we introduce flippable user interfaces that enable the end user to change the reading direction of a graphical user interface by flipping it into the desired reading direction by direct manipulation. This operation automatically and dynamically changes the user interface layout based on a generalized concept of reading direction and translates it according to the end user’s preferences.
Faure vanderdonckt co-summit2013-final
User Interface eXtensible Markup Language (UsiXML) is a formal Domain-Specific Language (DSL) used in Human-Computer Interaction (HCI) and Software Engineering (SE) in order to describe any user interface of any interactive application independently of any implementation technology. A user interface may involve variations depending on: the context of use (in which the user is carrying out her interactive task), the device or the computing platform (on which the user is working), the language (used by the user), the organization (to which the user belongs), the user profile, the interaction modalities (e.g., graphical, vocal, tactile, haptics)
Generative Pattern-Based Design of User Interfaces
This paper suggests a method for developing graphical user interfaces based on generative patterns. A generative pattern contains portions of previously designed user interfaces are expressed through models that are either partially or totally instantiated. These portions could be identified and re-applied to a new design case study by generating code by instantiating the specifications contained in the models. The method involves typical models found in user interface development life cycle such as task, domain, abstract user interface, concrete user interface, final user interface, context model, and mappings between them. Any model could virtually be the source of a pattern and could be described, searched, matched, retrieved, and assembled together so as to create a new graphical user interface. For this purpose, a software has been developed that manages generative patterns by combining an existing user interface description language (UsiXML – user interface extensible markup language) with concepts addressing problems raised by pattern description and matching in a pattern-based language (PLML – Pattern Language Markup Language, a language was introduced to uniformly represent user interface patterns). Once instantiated from the generative patterns, the models give rise to a model-driven engineering based on model-to-model transformation and model-to-code compilation.
Towards Canonical Task Types for User Interface Design
Task models are the cornerstone of user-centred design methodologies for user interface design. Therefore, they deserve attention in order to produce them effectively and efficiently, while guaranteeing the reproducibility of a task model: different persons should in principle obtain the same task model, or a similar one, for the same problem. In order to provide user interface designers with some guidance for task modelling, a list of canonical task types is proposed that offers a unified definition of frequently used tasks types in a consistent way. Each task type consists of a a task action coupled with a task object, each of them being written according to design guidelines. This list provides the following benefits: tasks are modelled in a more consistent way, their definition is more communicable and shared, task models can be efficiently used for model-driven engineering of user interfaces.
Viewers also liked (6)
Flippable User Interfaces for Internationalization
Flippable User Interfaces for Internationalization
Generative Pattern-Based Design of User Interfaces
Generative Pattern-Based Design of User Interfaces
Towards Canonical Task Types for User Interface Design
Towards Canonical Task Types for User Interface Design
Similar to 3D User Interfaces for Information Systems Based on UsiXML
User Interface Derivation from Business Processes: A Model-Driven Approach fo...
This presentation defines a model-driven approach for organizational engineering in which user interfaces of information systems are derived from business processes. This approach consists of four steps: business process modeling in the context of organizational engineering, task model derivation from the business process model, task refinement, and user interface model derivation from the task model. Each step contributes to specify and refine map-pings between the source and the target model. In this way, each model modification could be adequately propagated in the rest of the chain. By applying this model-driven approach, the user inter-faces of the information systems are directly meeting the require-ments of the business processes and are no longer decoupled from them. This approach has been validated on a case study in a large bank-insurance company
IRJET- A Quality Watch Android Based Application for Monitoring Robotic A...
https://www.irjet.net/archives/V5/i3/IRJET-V5I3331.pdf
Development Prototype Design of Virtual Assembly Application-Based Leap Motion
Innovation in design engineering practice is very important in the world of manufacturing in the increasingly competitive global market. Prototyping and evaluation measures are inseparable from the design process in the manufacture of a product. And made one of many physical prototypes require very expensive and time consuming, so the technology of Virtual Reality (VR) is needed, so the industry can quickly and precisely in the decision. VR technology combines a human being with a computer environment visually, touch and hearing, so that the user as if into the virtual world. The goal is that users with hand movements can interact with what is displayed on the computer screen or the user can interact with the environment is unreal to be added into the real world. VR is required for simulations that require a lot of interaction such as prototype assembly methods, or better known as the Virtual Assembly. Virtual Assembly concept which was developed as the ability to assemble a real representation of the physical model, the 3D models in CAD software by simulating the natural movement of the human hand. Leap Motion (accuracy of 0.01mm) was used to replace Microsoft's Kinect (accuracy of 1.5cm) and Motion Glove with flex sensors (accuracy of 1°) in several previous research. Leap mot ion controller is a device that captures every movement of the hand to then be processed and integrated with 3D models in CAD software. And simulation of assembly process virtually in CA D software with hand gestures detected by the leap mot ion, assembly parts can be driven either in translation or rotation, zooming and adding the assembly constraint. It also can perform mouse functions (such as left-click, middle-click, right-click and move the mouse cursor position) to a virtual assembly process simulation on CAD software.
Customizing Model of Mobile Service Computing on Cloud of Things
https://irjet.net/archives/V4/i3/IRJET-V4I3359.pdf
IRJET - Hand Gestures Recognition using Deep Learning
https://www.irjet.net/archives/V7/i3/IRJET-V7I31032.pdf
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-driven engineering (MDE) of user interfaces consists in describing a user interface and aspects involved in it (e.g., task, domain, context of use) in models from which a final interface is produced. With one big win in mind: when the user’s requirements or the context of use change, the models change accordingly and so does the supporting user interface. Models and a method for developing user interfaces based on MDE are presented in this tutorial supporting forward engineering (a new interface is produced), reverse engineering (an existing interface is improved), and lateral engineering (an existing interface is adapted to a new context of use). Software supporting this method will be used based on UsiXML (User Interface eXten-sible Markup Language), a XML-compliant user interface description language.
Advancing Electrical Design in High-Rise Buildings with Revit 3D Modeling
https://www.irjet.net/archives/V10/i11/IRJET-V10I11128.pdf
IRJET- Application of Revit as Building Information Modeling (BIM) for Integr...
https://www.irjet.net/archives/V5/i10/IRJET-V5I1003.pdf
Ijmet 10 02_009
An important factor in the construction of an information model of a construction
object is that various participants in the process of modeling construction objects work
with different, sometimes contradictory information obtained at different stages. Based
on the concept of life cycle management (life cycle) of construction objects, all
information obtained at the previous stages from different participants should describe
it as a single information object, around which various processes take place: design,
construction, operation, repair and diagnostics of the technical condition, each of which
uses and adds to the information model or that information about the object. The article
deals with the problem of generalization of construction information on construction
objects
Computer-Aided Design of Raft Foundation using Excel VBA and FORTRAN
https://www.irjet.net/archives/V10/i4/IRJET-V10I483.pdf
Similar to 3D User Interfaces for Information Systems Based on UsiXML (20)
User Interface Derivation from Business Processes: A Model-Driven Approach fo...
User Interface Derivation from Business Processes: A Model-Driven Approach fo...
IRJET- A Quality Watch Android Based Application for Monitoring Robotic A...
IRJET- A Quality Watch Android Based Application for Monitoring Robotic A...
Softwareapplicationassignment 141204035809-conversion-gate02
Softwareapplicationassignment 141204035809-conversion-gate02
Development Prototype Design of Virtual Assembly Application-Based Leap Motion
Development Prototype Design of Virtual Assembly Application-Based Leap Motion
Customizing Model of Mobile Service Computing on Cloud of Things
Customizing Model of Mobile Service Computing on Cloud of Things
IRJET - Hand Gestures Recognition using Deep Learning
IRJET - Hand Gestures Recognition using Deep Learning
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Advancing Electrical Design in High-Rise Buildings with Revit 3D Modeling
Advancing Electrical Design in High-Rise Buildings with Revit 3D Modeling
IRJET- Application of Revit as Building Information Modeling (BIM) for Integr...
IRJET- Application of Revit as Building Information Modeling (BIM) for Integr...
Computer-Aided Design of Raft Foundation using Excel VBA and FORTRAN
Computer-Aided Design of Raft Foundation using Excel VBA and FORTRAN
Towards Method Engineering of Model-Driven User Interface Development
Towards Method Engineering of Model-Driven User Interface Development
More from Jean Vanderdonckt
To the end of our possibilities with Adaptive User Interfaces
Slides of the keynote presented at the 1st International Workshop on Human-in-the-Loop Applied Machine Learning (HITLAML '23)
September 04 - 06, 2023 - Belval, Luxembourg.
This presentation summarizes the evolution of techniques used to adapt the user interfaces to the context of use, which is composed of the user, the platform, and the environment.
Engineering the Transition of Interactive Collaborative Software from Cloud C...
Paper presented at EICS '22: https://dl.acm.org/doi/10.1145/3532210
The "Software as a Service" (SaaS) model of cloud computing popularized online multiuser collaborative software. Two famous examples of this class of software are Office 365 from Microsoft and Google Workspace. Cloud technology removes the need to install and update the software on end users' computers and provides the necessary underlying infrastructure for online collaboration. However, to provide a good end-user experience, cloud services require an infrastructure able to scale up to the task and allow low-latency interactions with a variety of users worldwide. This is a limiting factor for actors that do not possess such infrastructure. Unlike cloud computing which forgets the computational and interactional capabilities of end users' devices, the edge computing paradigm promises to exploit them as much as possible. To investigate the potential of edge computing over cloud computing, this paper presents a method for engineering interactive collaborative software supported by edge devices for the replacement of cloud computing resources. Our method is able to handle user interface aspects such as connection, execution, migration, and disconnection differently depending on the available technology. We exemplify our approach by developing a distributed Pictionary game deployed in two scenarios: a nonshared scenario where each participant interacts only with their own device and a shared scenario where participants also share a common device, including a TV. After a theoretical comparative study of edge vs. cloud computing, an experiment compares the two implementations to determine their effect on the end user's perceived experience and latency vs. real latency
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
Presentation of ACM EICS '22 paper: https://dl.acm.org/doi/10.1145/3532207
Eye movement analysis is a popular method to evaluate whether a user interface meets the users' requirements and abilities. However, with current tools, setting up a usability evaluation with an eye-tracker is resource-consuming, since the areas of interest are defined manually, exhaustively and redefined each time the user interface changes. This process is also error-prone, since eye movement data must be finely synchronised with user interface changes. These issues become more serious when the user interface layout changes dynamically in response to user actions. In addition, current tools do not allow easy integration into interactive applications, and opportunistic code must be written to link these tools to user interfaces. To address these shortcomings and to leverage the capabilities of eye-tracking, we present UsyBus, a communication framework for autonomous, tight coupling among reusable agents. These agents are responsible for collecting data from eye-trackers, analyzing eye movements, and managing communication with other modules of an interactive application. UsyBus allows multiple heterogeneous eye-trackers as input, provides multiple configurable outputs depending on the data to be exploited. Modules exchange data based on the UsyBus communication framework, thus creating a customizable multi-agent architecture. UsyBus application domains range from usability evaluation to gaze interaction applications design. Two case studies, composed of reusable modules from our portfolio, exemplify the implementation of the UsyBus framework.
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
Paper presented at ACM EICS '22
Finger-based gesture input becomes a major interaction modality for surface computing. Due to the low precision of the finger and the variation in gesture production, multistroke gestures are still challenging to recognize in various setups. In this paper, we present µV, a multistroke gesture recognizer that addresses the properties of articulation, rotation, scaling, and translation invariance by combining $P+'s cloud-matching for articulation invariance with !FTL's local shape distance for RST-invariance. We evaluate µV against five competitive recognizers on MMG, an existing gesture set, and on two new versions for smartphones and tablets, MMG+ and RMMG+, a randomly rotated version on both platforms. µV is significantly more accurate than its predecessors when rotation invariance is required and not significantly inferior when it is not. µV is also significantly faster than others with many samples and not significantly slower with few samples
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
The body of knowledge accumulated by gesture elicitation studies (GES), although useful, large, and extensive, is also heterogeneous, scattered in the scientific literature across different venues and fields of research, and difficult to generalize to other contexts of use represented by different gesture types, sensing devices, applications, and user categories. To address such aspects, we introduce RepliGES, a conceptual space that supports (1) replications of gesture elicitation studies to confirm, extend, and complete previous findings, (2) reuse of previously elicited gesture sets to enable new discoveries, and (3) extension and generalization of previous findings with new methods of analysis and for new user populations towards consolidated knowledge of user-defined gestures. Based on RepliGES, we introduce GEStory, an interactive design space and visual tool, to structure, visualize and identify user-defined gestures from a number of 216 published gesture elicitation studies
Gesture-based information systems: from DesignOps to DevOps
Keynote address for the 29th International Conference on Information Systems Development ISD'2021 (Valencia, Spain, September 8-10, 2021). See https://isd2021.webs.upv.es/program.php#keynotes
This talk promotes the Seven I':
Implementation continuity
Inclusion of end-users
Interaction first
Integration among stakeholders
Iteration short
Incremental progress
Innovation openness
Engineering Slidable User Interfaces with Slime
Intra-platform plasticity regularly assumes that the display of a computing platform remains fixed and rigid during interactions with the platform in contrast to reconfigurable displays, which can change form depending on the context of use. In this paper, we present a model-based approach for designing and deploying graphical user interfaces that support intra-platform plasticity for reconfigurable displays. We instantiate the model for E3Screen, a new device that expands a conventional laptop with two slidable, rotatable, and foldable lateral displays, enabling slidable user interfaces. Based on a UML class diagram as a domain model and a SCRUD list as a task model, we define an abstract user interface as interaction units with a corresponding master-detail design pattern. We then map the abstract user interface to a concrete user interface by applying rules for the reconfiguration, concrete interaction, unit allocation, and widget selection and implement it in JavaScript. In a first experiment, we determine display configurations most preferred by users, which we organize in the form of a state-transition diagram. In a second experiment, we address reconfiguration rules and widget selection rules. A third experiment provides insights into the impact of the lateral displays on a visual search task.
Evaluating Gestural Interaction: Models, Methods, and Measures
Lecture 4: Evaluating Gestural Interaction: Models, Methods, and Measures
Francqui Chair in Computer Science 2020 VUB, Jean Vanderdonckt, 18 May 2021
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
Lecture 3: Conducting a Gesture Elicitation Study: How to Get the Best Gestures From People?
Francqui Chair in Computer Science 2020 VUB, Jean Vanderdonckt, 27 April 2021
Designing Gestural Interaction: Challenges and Pitfalls
Lecture 2: Designing Gestural Interaction:Challenges and Pitfalls
Francqui Chair in Computer Science 2020 VUB, Jean Vanderdonckt, 20 April 2021
Fundamentals of Gestural Interaction
Lecture 1: Fundamentals of Gestural Interaction
Francqui Chair in Computer Science 2020
VUB, Jean Vanderdonckt, 5 March 2020
Gestural Interaction, Is it Really Natural?
Francqui Chair in Computer Science 2020
Inaugural lesson, VUB
Jean Vanderdonckt, 20 February 2020
User-centred Development of a Clinical Decision-support System for Breast Can...
See the paper at https://www.scitepress.org/Link.aspx?doi=10.5220/0010258900600071
We conducted a user-centered design of a clinical decision-support system for breast cancer screening, diagnosis, and reporting based on stroke gestures. We combined knowledge elicitation interviews, scenario-focused questionnaires, and paper mock-ups to understand user needs. Multi-fidelity (low and high) prototypes were designed and compared first in vitro in a usability laboratory, then in vivo in the real world. The resulting user interface provides radiologists with a platform that integrates domain-oriented tools for the visualization of mammograms, the manual, and the semi-automatic annotation of breast cancer findings based on stroke gestures. The contribution of this work lies in that, to the best of our knowledge, stroke gestures have not yet been applied to the annotation of mammograms. On the one hand, although there is a substantial amount of research done in stroke-based interaction, none focuses especially on the domain of breast cancer annotation. On the other hand, typical gestures in breast cancer annotation tools are those with a keyboard and a mouse
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Ensuring responsive design of web applications requires their user interfaces to be able to adapt according to different contexts of use, which subsume the end users, the devices and platforms used to carry out the interactive tasks, and also the environment in which they occur. To address the challenges posed by responsive design, aiming to simplify their development by factoring out the common parts from the specific ones, this paper presents Quill, a web-based development environment that enables various stakeholders of a web application to collaboratively adopt a model-based design of the user interface for cross-platform deployment. The paper establishes a series of requirements for collaborative model-based design of cross-platform web user interfaces motivated by the literature, observational and situational design. It then elaborates on potential solutions that satisfy these requirements and explains the solution selected for Quill. A user survey has been conducted to determine how stakeholders appreciate model-based design user interface and how they estimate the importance of the requirements that lead to Quill
Attach Me, Detach Me, Assemble Me like you Work
Detachable user interfaces consist of graphical user interfaces whose parts or whole can be detached at run-time from their host, migrated onto an- other computing platform while carrying out the task, possibly adapted to the new platform and attached to the target platform in a peer-to-peer fashion. De- taching is the property of splitting a part of a UI for transferring it onto another platform. AttAaching is the reciprocal property: a part of an existing interface can be attached to the currently being used interface so as to recompose another one on-demand, according to user's needs, task requirements. Assembling inter- face parts by detaching and attaching allows dynamically composing, decom- posing and re-composing new interfaces on demand. To support this interaction paradigm, a development infrastructure has been developed based on a series of primitives such as display, undisplay, copy, expose, return, transfer, delegate, and switch. We exemplify it with QTkDraw, a painting application with attach- ing and detaching based on the development infrastructure.
The Impact of Comfortable Viewing Positions on Smart TV Gestures
Whereas gesture elicitation studies for TV interaction
assume that participants adopt an upright, frontal viewing
position, we asked 21 participants to hold a natural, comfortable
viewing position, the posture they adopt when watching TV
at home. By involving a broad selection of users regarding
age, profession, our study targets a higher ecological validity
than in existing studies. Agreements rates were lower than existing studies using an upright, frontal viewing position. Participants experienced problems due to (1) having to use their slave hand instead of their dominant hand, (2) being in a certain orientation with their head making it more difficult to perform some physical movements, and (3) being hindered in their movement by the sofa there lay on. Since each person may have a different
position inducing different gestures due to the aforementioned
problems, the effect of a comfortable viewing position is analyzed
by comparison to gestures for a frontal position.
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
This paper presents empirical results about user-de ned gestures
for head and shoulders by analyzing 308 gestures elicited from 22 participants for 14 referents materializing 14 different types of tasks in IoT context of use. We report an overall medium consensus but with medium variance (mean: .263, min: .138, max: .390 on the unit scale) between participants gesture proposals, while their thinking time were less similar (min: 2.45 sec, max: 22.50 sec), which suggests that head and shoulders gestures are not all equally easy to imagine and to produce. We point to the challenges of deciding which head and shoulders gestures
will become the consensus set based on four criteria: the agreement rate, their individual frequency, their associative frequency, and their unicity.
Paper accessible at https://dial.uclouvain.be/pr/boreal/en/object/boreal%3A213794
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
Instead of relying on graphical or vocal modalities for searching
an item by keyword (called K-Menu), this paper presents the G-Menu exploiting gesture interaction and gesture recognition: when a user sketches a keyword by gesturing the first letters of its label, a menu with items related to the recognized letters is constructed dynamically and presented to the user for selection and auto-completion. The selection can be completed either gesturally by an appropriate gesture (called the G-Menu) or by touch only (called the T-Menu). This paper compares the three types of menu, i.e., by keyword, by gesture, and by touching, in a user study with twenty participants on their item selection time (for measuring task efficiency), their error rate (for measuring task effectiveness),
and their subjective satisfaction (for measuring user satisfaction).
Paper accessible at https://dial.uclouvain.be/pr/boreal/en/object/boreal%3A213790
Vector-based, Structure Preserving Stroke Gesture Recognition
Unistroke and multistroke gesture recognizers have always striven to reach some robustness with respect to
all variations encountered when people issue gestures by hand
on touch surfaces or with sensing devices. For this purpose,
successful stroke recognizers rely on a gesture recognition
algorithm that satisfies a series of invariance properties such
as: stroke-order invariance, stroke-number invariance, stroke direction invariance, position, scale, and rotation invariance.
Before initiating any recognition activity, these algorithms
ensure these properties by performing several pre-processing
operations. These operations induce an additional computational
cost to the recognition process, as well as a potential error
bias. To cope with this problem, we introduce an algorithm that
ensures all these properties analytically instead of statistically
based on a vector algebra. Instead of points, the recognition
algorithm works on vectors between vectors. We demonstrate
that this approach not eliminates the need for these preprocessing
operations but also satisfies an entire structure preserving
transformation.
Paper available at https://dial.uclouvain.be/pr/boreal/en/object/boreal%3A217006
An ontology for reasoning on body-based gestures
Body-based gestures, such as acquired by Kinect sensor, today benefit from efficient tools for their recognition and development, but less for automated reasoning. To facilitate this activity, an ontology for structuring body-based gestures, based on user, body and body parts, gestures, and environment, is designed and encoded in Ontology Web Language according to modelling triples (subject, predicate, object). As a proof-of-concept and to feed this ontology, a gesture elicitation study collected 24 participants X 19 referents for IoT tasks = 456 elicited body-based gestures, which were classified and expressed according to the ontology.
See paper at https://dl.acm.org/citation.cfm?id=3328238
More from Jean Vanderdonckt (20)
To the end of our possibilities with Adaptive User Interfaces
To the end of our possibilities with Adaptive User Interfaces
Engineering the Transition of Interactive Collaborative Software from Cloud C...
Engineering the Transition of Interactive Collaborative Software from Cloud C...
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
Gesture-based information systems: from DesignOps to DevOps
Gesture-based information systems: from DesignOps to DevOps
Evaluating Gestural Interaction: Models, Methods, and Measures
Evaluating Gestural Interaction: Models, Methods, and Measures
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
Designing Gestural Interaction: Challenges and Pitfalls
Designing Gestural Interaction: Challenges and Pitfalls
User-centred Development of a Clinical Decision-support System for Breast Can...
User-centred Development of a Clinical Decision-support System for Breast Can...
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
The Impact of Comfortable Viewing Positions on Smart TV Gestures
The Impact of Comfortable Viewing Positions on Smart TV Gestures
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
Vector-based, Structure Preserving Stroke Gesture Recognition
Vector-based, Structure Preserving Stroke Gesture Recognition
Recently uploaded
Communications Mining Series - Zero to Hero - Session 1
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
A tale of scale & speed: How the US Navy is enabling software delivery from l...
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
By Design, not by Accident - Agile Venture Bolzano 2024
As presented at the Agile Venture Bolzano, 4.06.2024
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
The Art of the Pitch: WordPress Relationships and Sales
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Recently uploaded (20)
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
3D User Interfaces for Information Systems Based on UsiXML
- 2. ITEA UsiXML project #08026, 2009-2012
- 3. Locate your sit in a stadium ITEA UsiXML project #08026, 2009-2012
- 8. 3D Rendering of a 2D GUI ITEA UsiXML project #08026, 2009-2012 VUITOOLKIT [Moli08]
- 10. ITEA UsiXML project #08026, 2009-2012 Support for different representations Support for basic Haptic Interaction 3DUIs
- 11. Method Outline ITEA UsiXML project #08026, 2009-2012 Task and Domain Model Model to Model Abstract UI Model Model to Model Concrete UI Model Code Generation Final UI Control Task and Domain Model Physical Control Software Control Physical interaction object 2D 3D
- 14. Method Outline ITEA UsiXML project #08026, 2009-2012
- 16. ITEA UsiXML project #08026, 2009-2012
- 17. ITEA UsiXML project #08026, 2009-2012
- 20. ITEA UsiXML project #08026, 2009-2012