An Efficient Approach of Segmentation and Blind Deconvolution in Image Restor...iosrjce
This paper introduces the concept of Blind Deconvolution for restoration of a digital image and
small segments of a single image that has been degraded due to some noise. Concept of Image Restoration is
used in various areas like in Robotics to take decision, Biomedical research for analysis of tissues, cells and
cellular constituents etc. Segmentation is used to divide an image into multiple meaningful regions. Concept of
segmentation is helpful for restoration of only selected portion of the image hence reduces the complexity of the
system by focusing only on those parts of the image that need to be restored. There exist so many techniques for
the restoration of a degraded image like Wiener filter, Regularized filter, Lucy Richardson algorithm etc. All
these techniques use prior knowledge of blur kernel for restoration process. In Blind Deconvolution technique
Blur kernel initially remains unknown. This paper uses Gaussian low pass filter to convolve an image. Gaussian
low pass filter minimize the problem of ringing effect. Ringing effect occurs in image when transition between
one point to another is not clearly defined. After removing these ringing effects from the restored image,
resultant image will be clear in visibility. The aim of this paper is to provide better algorithm that can be helpful
in removing unwanted features from the image and the quality of the image can be measured in terms of
PSNR(Peak Signal-to-Noise Ratio) and MSE(Mean Square error). Proposed Technique also works well with
Motion Blur.
Effective segmentation of sclera, iris and pupil in noisy eye imagesTELKOMNIKA JOURNAL
In today’s sensitive environment, for personal authentication, iris recognition is the most attentive
technique among the various biometric technologies. One of the key steps in the iris recognition system is
the accurate iris segmentation from its surrounding noises including pupil and sclera of a captured
eye-image. In our proposed method, initially input image is preprocessed by using bilateral filtering.
After the preprocessing of images contour based features such as, brightness, color and texture features
are extracted. Then entropy is measured based on the extracted contour based features to effectively
distinguishing the data in the images. Finally, the convolution neural network (CNN) is used for
the effective sclera, iris and pupil parts segmentations based on the entropy measure. The proposed
results are analyzed to demonstrate the better performance of the proposed segmentation method than
the existing methods.
Bieg Eye And Pointer Coordination In Search And Selection TasksKalle
Selecting a graphical item by pointing with a computer mouse is a ubiquitous task in many graphical user interfaces. Several techniques have been suggested to facilitate this task, for instance, by reducing the required movement distance. Here we
measure the natural coordination of eye and mouse pointer
control across several search and selection tasks. We find that users automatically minimize the distance to likely targets in an intelligent, task dependent way. When target location is highly predictable, top-down knowledge can enable users to initiate pointer movements prior to target fixation. These findings question the utility of existing assistive pointing techniques and suggest that alternative approaches might be more effective.
An Efficient Approach of Segmentation and Blind Deconvolution in Image Restor...iosrjce
This paper introduces the concept of Blind Deconvolution for restoration of a digital image and
small segments of a single image that has been degraded due to some noise. Concept of Image Restoration is
used in various areas like in Robotics to take decision, Biomedical research for analysis of tissues, cells and
cellular constituents etc. Segmentation is used to divide an image into multiple meaningful regions. Concept of
segmentation is helpful for restoration of only selected portion of the image hence reduces the complexity of the
system by focusing only on those parts of the image that need to be restored. There exist so many techniques for
the restoration of a degraded image like Wiener filter, Regularized filter, Lucy Richardson algorithm etc. All
these techniques use prior knowledge of blur kernel for restoration process. In Blind Deconvolution technique
Blur kernel initially remains unknown. This paper uses Gaussian low pass filter to convolve an image. Gaussian
low pass filter minimize the problem of ringing effect. Ringing effect occurs in image when transition between
one point to another is not clearly defined. After removing these ringing effects from the restored image,
resultant image will be clear in visibility. The aim of this paper is to provide better algorithm that can be helpful
in removing unwanted features from the image and the quality of the image can be measured in terms of
PSNR(Peak Signal-to-Noise Ratio) and MSE(Mean Square error). Proposed Technique also works well with
Motion Blur.
Effective segmentation of sclera, iris and pupil in noisy eye imagesTELKOMNIKA JOURNAL
In today’s sensitive environment, for personal authentication, iris recognition is the most attentive
technique among the various biometric technologies. One of the key steps in the iris recognition system is
the accurate iris segmentation from its surrounding noises including pupil and sclera of a captured
eye-image. In our proposed method, initially input image is preprocessed by using bilateral filtering.
After the preprocessing of images contour based features such as, brightness, color and texture features
are extracted. Then entropy is measured based on the extracted contour based features to effectively
distinguishing the data in the images. Finally, the convolution neural network (CNN) is used for
the effective sclera, iris and pupil parts segmentations based on the entropy measure. The proposed
results are analyzed to demonstrate the better performance of the proposed segmentation method than
the existing methods.
Bieg Eye And Pointer Coordination In Search And Selection TasksKalle
Selecting a graphical item by pointing with a computer mouse is a ubiquitous task in many graphical user interfaces. Several techniques have been suggested to facilitate this task, for instance, by reducing the required movement distance. Here we
measure the natural coordination of eye and mouse pointer
control across several search and selection tasks. We find that users automatically minimize the distance to likely targets in an intelligent, task dependent way. When target location is highly predictable, top-down knowledge can enable users to initiate pointer movements prior to target fixation. These findings question the utility of existing assistive pointing techniques and suggest that alternative approaches might be more effective.
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...CSCJournals
In this paper, an approach is developed for segmenting an image into major surfaces and potential objects using RGBD images and 3D point cloud data retrieved from a Kinect sensor. In the proposed segmentation algorithm, depth and RGB data are mapped together. Color, texture, XYZ world coordinates, and normal-, surface-, and graph-based segmentation index features are then generated for each pixel point. These attributes are used to cluster similar points together and segment the image. The inclusion of new depth-related features provided improved segmentation performance over RGB-only algorithms by resolving illumination and occlusion problems that cannot be handled using graph-based segmentation algorithms, as well as accurately identifying pixels associated with the main structure components of rooms (walls, ceilings, floors). Since each segment is a potential object or structure, the output of this algorithm is intended to be used for object recognition. The algorithm has been tested on commercial building images and results show the usability of the algorithm in real time applications.
Image fusion is an image enhancement approach for increasing the visual perception from two or more image into a single image. Each image is obtained from different object in focus. This process is now broadly used in various application of image processing such as medical imaging such as MRI, CT [18] and PET, remote sensing, satellite imaging, in design of intelligent robot etc. In this paper we have gone through the literature work done by the various researchers to obtain high quality improved image by combining important and desirable features from two or more images into a single image. Different image fusion rules, like maximum selection scheme, weighted average scheme and window based verification scheme are discussed. We also discuss about the image fusion techniques using DWT. Distinct blurred images are fused using DWT, SWT and using local correlation and their results are compared. The role of fusion in image enhancement is also taken into consideration. The results of different fusion technique are also compared, which are furnished in pictures and tables.
Near Reversible Data Hiding Scheme for images using DCTIJERA Editor
Certain level of modifications to the original content can be acceptable. These schemes are called nearreversible.
It has emerging application in remote sensing. In remote sensing application the image is captured
while monitoring the damaged regions in the natural disasters such as tsunami, volcanic eruption, etc. Instead of
more alterations to the pixels or coefficients here we go for less alterations , low hiding capacity and
complexity that exhibit while reversible data hiding. There exist a few near reversible hiding schemes which
address the limitations to the triangular trade-off that exists between capacity, robustness and visual quality
characteristics. However, for achieving better vision of image, there is a need for near reversible data hiding
schemes. Therefore, the traditional metric
PSNR are not sufficient to assess the distorted image when the data is embedded in image for assessing the
better visual quality of the image with hidden data we present a HVS based metrics like PSNR-HVS, MSSIM.
Using DCT evaluates the overall image quality
Removal of Unwanted Objects using Image Inpainting - a Technical ReviewIJERA Editor
Image In painting, the technique to change image in undetectable structure, it itself is an ancient art. There are
various goals and applications of image in painting which includes restoration of damaged painting and also to
replace/remove the selected objects. This paper, describes various techniques that can help in removing
unwanted objects from image. Even the in painting fundamentals are directly further, most inpainting techniques
available in the literature are difficult to understand and implement.
ADVANCED SINGLE IMAGE RESOLUTION UPSURGING USING A GENERATIVE ADVERSARIAL NET...sipij
The resolution of an image is a very important criterion for evaluating the quality of the image. Higher resolution of image is always preferable as images of lower resolution are unsuitable due to fuzzy quality. Higher resolution of image is important for various fields such as medical imaging; astronomy works and so on as images of lower resolution becomes unclear and indistinct when their sizes are enlarged. In recent times, various research works are performed to generate higher resolution of an image from its lower resolution. In this paper, we have proposed a technique of generating higher resolution images form lower resolution using Residual in Residual Dense Block network architecture with a deep network. We have also compared our method with other methods to prove that our method provides better visual quality images.
Hardoon Image Ranking With Implicit Feedback From Eye MovementsKalle
In order to help users navigate an image search system, one could
provide explicit information on a small set of images as to which
of them are relevant or not to their task. These rankings are learned
in order to present a user with a new set of images that are relevant
to their task. Requiring such explicit information may not
be feasible in a number of cases, we consider the setting where
the user provides implicit feedback, eye movements, to assist when
performing such a task. This paper explores the idea of implicitly
incorporating eye movement features in an image ranking task
where only images are available during testing. Previous work had
demonstrated that combining eye movement and image features improved
on the retrieval accuracy when compared to using each of
the sources independently. Despite these encouraging results the
proposed approach is unrealistic as no eye movements will be presented
a-priori for new images (i.e. only after the ranked images are
presented would one be able to measure a user’s eye movements
on them). We propose a novel search methodology which combines
image features together with implicit feedback from users’
eye movements in a tensor ranking Support Vector Machine and
show that it is possible to extract the individual source-specific
weight vectors. Furthermore, we demonstrate that the decomposed
image weight vector is able to construct a new image-based semantic
space that outperforms the retrieval accuracy than when solely
using the image-features.
PROXIMITY-BASED ADAPTATION OF CONTENT TO GROUPS OF VIEWERS OF PUBLIC DISPLAYSijujournal
Responsive design adapts web content to different viewing contexts to deliver an optimal
viewing and interaction experience. Recent work proposed a model and framework for
proximity-based adaptation of web content as a new dimension for responsive web design.
While it was shown that the model improves the perception and user engagement for single
viewers, until now, the effect had not been investigated for multiple simultaneous viewers
who may be at different distances from the display. In this paper, we report on an initial
study that evaluated and compared the effects of using the average distance of viewers as
the basis for handling adaptation of content to multiple viewers with a classic one that
adapts content based only on display characteristics. Our results show that the adaptive
model provides a better view of the content and improves user engagement, but can be
confusing when serving multiple viewers.
Proximity Based Adaptation of Content to Groups of Viewers of Public Displays ijujournal
Responsive design adapts web content to different viewing contexts to deliver an optimal
viewing and interaction experience. Recent work proposed a model and framework for
proximity-based adaptation of web content as a new dimension for responsive web design.
While it was shown that the model improves the perception and user engagement for single
viewers, until now, the effect had not been investigated for multiple simultaneous viewers
who may be at different distances from the display. In this paper, we report on an initial
study that evaluated and compared the effects of using the average distance of viewers as
the basis for handling adaptation of content to multiple viewers with a classic one that
adapts content based only on display characteristics. Our results show that the adaptive
model provides a better view of the content and improves user engagement, but can be
confusing when serving multiple viewers.
PROXIMITY-BASED ADAPTATION OF CONTENT TO GROUPS OF VIEWERS OF PUBLIC DISPLAYSijujournal
Responsive design adapts web content to different viewing contexts to deliver an optimal
viewing and interaction experience. Recent work proposed a model and framework for
proximity-based adaptation of web content as a new dimension for responsive web design.
While it was shown that the model improves the perception and user engagement for single
viewers, until now, the effect had not been investigated for multiple simultaneous viewers
who may be at different distances from the display. In this paper, we report on an initial
study that evaluated and compared the effects of using the average distance of viewers as
the basis for handling adaptation of content to multiple viewers with a classic one that
adapts content based only on display characteristics. Our results show that the adaptive
model provides a better view of the content and improves user engagement, but can be
confusing when serving multiple viewers.
Porta Ce Cursor A Contextual Eye Cursor For General Pointing In Windows Envir...Kalle
Eye gaze interaction for disabled people is often dealt with by designing ad-hoc interfaces, in which the big size of their elements compensates for both the inaccuracy of eye trackers and the instability of the human eye. Unless solutions for reliable eye cursor control are employed, gaze pointing in ordinary graphical operating environments is a very difficult task. In this paper we present an eye-driven cursor for MS Windows which behaves differently according to the “context”. When the user’s gaze is perceived within the desktop or a folder, the cursor can be discretely shifted from one icon to another. Within an application window or where there are no icons, on the contrary, the cursor can be continuously and precisely moved. Shifts in the four directions (up, down, left, right) occur through dedicated buttons. To increase user awareness of the currently pointed spot on the screen while continuously moving the cursor, a replica of the spot is provided within the active direction button, resulting in improved pointing performance.
Visualisation for ontology sense-making: A tree-map based algorithmic approachCSITiaesprime
Ontology sense-making or visual comprehension of the ontological schemata and structure are vital for cross-validation purposes of the ontology increment during the process of applied ontology construction. Also, it is important to query the ontology in order to verify the accuracy of the stored knowledge embeddings. This will boost the interactions between domain specialists and ontologists in applied ontology construction processes. Hence existing mechanisms have numerous of deficiencies (discussed in the paper), a new algorithm is proposed in this research to boost the efficiency of usage of tree-maps for effective ontology sense making. Proposed algorithm and prototype are quantitatively and qualitatively assessed for their accuracy and efficacy.
CHAINED DISPLAYS: CONFIGURATION OF MULTIPLE CO-LOCATED PUBLIC DISPLAYSIJCNCJournal
Networks of pervasive display systems involving public and semi-public displays have
allowed experiences to be created that span across multiple displays to achieve a stronger
effect on the viewers. However, little research has been done so far on the configuration
of content for multiple displays, especially when encountered in sequence in what is
commonly referred to as chained displays. As a first step towards determining appropriate
configuration strategies for chained displays, we have identified and investigated different
approaches for configuring content. We report on a user study on the effect of the different
configuration models in terms of usability and user engagement.
Stellmach Advanced Gaze Visualizations For Three Dimensional Virtual Environm...Kalle
Gaze visualizations represent an effective way for gaining fast insights into eye tracking data. Current approaches do not adequately support eye tracking studies for three-dimensional (3D) virtual environments. Hence, we propose a set of advanced gaze visualization techniques for supporting gaze behavior analysis in such environments. Similar to commonly used gaze visualizations for twodimensional
stimuli (e.g., images and websites), we contribute advanced 3D scan paths and 3D attentional maps. In addition, we introduce a models of interest timeline depicting viewed models, which can be used for displaying scan paths in a selected time segment. A prototype toolkit is also discussed which combines an implementation of our proposed techniques. Their potential for facilitating eye tracking studies in virtual environments was supported by a user study among eye tracking and visualization experts.
Computers help us handle and process tons of information data. Most of the time all this data is so dense, it’s almost impossible to understand from just looking at a bunch of numbers. Some of the data could be analyzed by computers, but most of the time there must be somebody, a real thinking person, who shall interpret the data and take conclusions from it to make decisions, analyze. Scientific Visualization is about converting numbers into a representation of reality, something more graphic so that a human being can understand and/or communicate.
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...CSCJournals
In this paper, an approach is developed for segmenting an image into major surfaces and potential objects using RGBD images and 3D point cloud data retrieved from a Kinect sensor. In the proposed segmentation algorithm, depth and RGB data are mapped together. Color, texture, XYZ world coordinates, and normal-, surface-, and graph-based segmentation index features are then generated for each pixel point. These attributes are used to cluster similar points together and segment the image. The inclusion of new depth-related features provided improved segmentation performance over RGB-only algorithms by resolving illumination and occlusion problems that cannot be handled using graph-based segmentation algorithms, as well as accurately identifying pixels associated with the main structure components of rooms (walls, ceilings, floors). Since each segment is a potential object or structure, the output of this algorithm is intended to be used for object recognition. The algorithm has been tested on commercial building images and results show the usability of the algorithm in real time applications.
Image fusion is an image enhancement approach for increasing the visual perception from two or more image into a single image. Each image is obtained from different object in focus. This process is now broadly used in various application of image processing such as medical imaging such as MRI, CT [18] and PET, remote sensing, satellite imaging, in design of intelligent robot etc. In this paper we have gone through the literature work done by the various researchers to obtain high quality improved image by combining important and desirable features from two or more images into a single image. Different image fusion rules, like maximum selection scheme, weighted average scheme and window based verification scheme are discussed. We also discuss about the image fusion techniques using DWT. Distinct blurred images are fused using DWT, SWT and using local correlation and their results are compared. The role of fusion in image enhancement is also taken into consideration. The results of different fusion technique are also compared, which are furnished in pictures and tables.
Near Reversible Data Hiding Scheme for images using DCTIJERA Editor
Certain level of modifications to the original content can be acceptable. These schemes are called nearreversible.
It has emerging application in remote sensing. In remote sensing application the image is captured
while monitoring the damaged regions in the natural disasters such as tsunami, volcanic eruption, etc. Instead of
more alterations to the pixels or coefficients here we go for less alterations , low hiding capacity and
complexity that exhibit while reversible data hiding. There exist a few near reversible hiding schemes which
address the limitations to the triangular trade-off that exists between capacity, robustness and visual quality
characteristics. However, for achieving better vision of image, there is a need for near reversible data hiding
schemes. Therefore, the traditional metric
PSNR are not sufficient to assess the distorted image when the data is embedded in image for assessing the
better visual quality of the image with hidden data we present a HVS based metrics like PSNR-HVS, MSSIM.
Using DCT evaluates the overall image quality
Removal of Unwanted Objects using Image Inpainting - a Technical ReviewIJERA Editor
Image In painting, the technique to change image in undetectable structure, it itself is an ancient art. There are
various goals and applications of image in painting which includes restoration of damaged painting and also to
replace/remove the selected objects. This paper, describes various techniques that can help in removing
unwanted objects from image. Even the in painting fundamentals are directly further, most inpainting techniques
available in the literature are difficult to understand and implement.
ADVANCED SINGLE IMAGE RESOLUTION UPSURGING USING A GENERATIVE ADVERSARIAL NET...sipij
The resolution of an image is a very important criterion for evaluating the quality of the image. Higher resolution of image is always preferable as images of lower resolution are unsuitable due to fuzzy quality. Higher resolution of image is important for various fields such as medical imaging; astronomy works and so on as images of lower resolution becomes unclear and indistinct when their sizes are enlarged. In recent times, various research works are performed to generate higher resolution of an image from its lower resolution. In this paper, we have proposed a technique of generating higher resolution images form lower resolution using Residual in Residual Dense Block network architecture with a deep network. We have also compared our method with other methods to prove that our method provides better visual quality images.
Hardoon Image Ranking With Implicit Feedback From Eye MovementsKalle
In order to help users navigate an image search system, one could
provide explicit information on a small set of images as to which
of them are relevant or not to their task. These rankings are learned
in order to present a user with a new set of images that are relevant
to their task. Requiring such explicit information may not
be feasible in a number of cases, we consider the setting where
the user provides implicit feedback, eye movements, to assist when
performing such a task. This paper explores the idea of implicitly
incorporating eye movement features in an image ranking task
where only images are available during testing. Previous work had
demonstrated that combining eye movement and image features improved
on the retrieval accuracy when compared to using each of
the sources independently. Despite these encouraging results the
proposed approach is unrealistic as no eye movements will be presented
a-priori for new images (i.e. only after the ranked images are
presented would one be able to measure a user’s eye movements
on them). We propose a novel search methodology which combines
image features together with implicit feedback from users’
eye movements in a tensor ranking Support Vector Machine and
show that it is possible to extract the individual source-specific
weight vectors. Furthermore, we demonstrate that the decomposed
image weight vector is able to construct a new image-based semantic
space that outperforms the retrieval accuracy than when solely
using the image-features.
PROXIMITY-BASED ADAPTATION OF CONTENT TO GROUPS OF VIEWERS OF PUBLIC DISPLAYSijujournal
Responsive design adapts web content to different viewing contexts to deliver an optimal
viewing and interaction experience. Recent work proposed a model and framework for
proximity-based adaptation of web content as a new dimension for responsive web design.
While it was shown that the model improves the perception and user engagement for single
viewers, until now, the effect had not been investigated for multiple simultaneous viewers
who may be at different distances from the display. In this paper, we report on an initial
study that evaluated and compared the effects of using the average distance of viewers as
the basis for handling adaptation of content to multiple viewers with a classic one that
adapts content based only on display characteristics. Our results show that the adaptive
model provides a better view of the content and improves user engagement, but can be
confusing when serving multiple viewers.
Proximity Based Adaptation of Content to Groups of Viewers of Public Displays ijujournal
Responsive design adapts web content to different viewing contexts to deliver an optimal
viewing and interaction experience. Recent work proposed a model and framework for
proximity-based adaptation of web content as a new dimension for responsive web design.
While it was shown that the model improves the perception and user engagement for single
viewers, until now, the effect had not been investigated for multiple simultaneous viewers
who may be at different distances from the display. In this paper, we report on an initial
study that evaluated and compared the effects of using the average distance of viewers as
the basis for handling adaptation of content to multiple viewers with a classic one that
adapts content based only on display characteristics. Our results show that the adaptive
model provides a better view of the content and improves user engagement, but can be
confusing when serving multiple viewers.
PROXIMITY-BASED ADAPTATION OF CONTENT TO GROUPS OF VIEWERS OF PUBLIC DISPLAYSijujournal
Responsive design adapts web content to different viewing contexts to deliver an optimal
viewing and interaction experience. Recent work proposed a model and framework for
proximity-based adaptation of web content as a new dimension for responsive web design.
While it was shown that the model improves the perception and user engagement for single
viewers, until now, the effect had not been investigated for multiple simultaneous viewers
who may be at different distances from the display. In this paper, we report on an initial
study that evaluated and compared the effects of using the average distance of viewers as
the basis for handling adaptation of content to multiple viewers with a classic one that
adapts content based only on display characteristics. Our results show that the adaptive
model provides a better view of the content and improves user engagement, but can be
confusing when serving multiple viewers.
Porta Ce Cursor A Contextual Eye Cursor For General Pointing In Windows Envir...Kalle
Eye gaze interaction for disabled people is often dealt with by designing ad-hoc interfaces, in which the big size of their elements compensates for both the inaccuracy of eye trackers and the instability of the human eye. Unless solutions for reliable eye cursor control are employed, gaze pointing in ordinary graphical operating environments is a very difficult task. In this paper we present an eye-driven cursor for MS Windows which behaves differently according to the “context”. When the user’s gaze is perceived within the desktop or a folder, the cursor can be discretely shifted from one icon to another. Within an application window or where there are no icons, on the contrary, the cursor can be continuously and precisely moved. Shifts in the four directions (up, down, left, right) occur through dedicated buttons. To increase user awareness of the currently pointed spot on the screen while continuously moving the cursor, a replica of the spot is provided within the active direction button, resulting in improved pointing performance.
Visualisation for ontology sense-making: A tree-map based algorithmic approachCSITiaesprime
Ontology sense-making or visual comprehension of the ontological schemata and structure are vital for cross-validation purposes of the ontology increment during the process of applied ontology construction. Also, it is important to query the ontology in order to verify the accuracy of the stored knowledge embeddings. This will boost the interactions between domain specialists and ontologists in applied ontology construction processes. Hence existing mechanisms have numerous of deficiencies (discussed in the paper), a new algorithm is proposed in this research to boost the efficiency of usage of tree-maps for effective ontology sense making. Proposed algorithm and prototype are quantitatively and qualitatively assessed for their accuracy and efficacy.
CHAINED DISPLAYS: CONFIGURATION OF MULTIPLE CO-LOCATED PUBLIC DISPLAYSIJCNCJournal
Networks of pervasive display systems involving public and semi-public displays have
allowed experiences to be created that span across multiple displays to achieve a stronger
effect on the viewers. However, little research has been done so far on the configuration
of content for multiple displays, especially when encountered in sequence in what is
commonly referred to as chained displays. As a first step towards determining appropriate
configuration strategies for chained displays, we have identified and investigated different
approaches for configuring content. We report on a user study on the effect of the different
configuration models in terms of usability and user engagement.
Stellmach Advanced Gaze Visualizations For Three Dimensional Virtual Environm...Kalle
Gaze visualizations represent an effective way for gaining fast insights into eye tracking data. Current approaches do not adequately support eye tracking studies for three-dimensional (3D) virtual environments. Hence, we propose a set of advanced gaze visualization techniques for supporting gaze behavior analysis in such environments. Similar to commonly used gaze visualizations for twodimensional
stimuli (e.g., images and websites), we contribute advanced 3D scan paths and 3D attentional maps. In addition, we introduce a models of interest timeline depicting viewed models, which can be used for displaying scan paths in a selected time segment. A prototype toolkit is also discussed which combines an implementation of our proposed techniques. Their potential for facilitating eye tracking studies in virtual environments was supported by a user study among eye tracking and visualization experts.
Computers help us handle and process tons of information data. Most of the time all this data is so dense, it’s almost impossible to understand from just looking at a bunch of numbers. Some of the data could be analyzed by computers, but most of the time there must be somebody, a real thinking person, who shall interpret the data and take conclusions from it to make decisions, analyze. Scientific Visualization is about converting numbers into a representation of reality, something more graphic so that a human being can understand and/or communicate.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Stellmach.2011.designing gaze supported multimodal interactions for the exploration of large image collections
1. Designing Gaze-supported Multimodal Interactions for the
Exploration of Large Image Collections
Sophie Stellmach∗, Sebastian Stober†, Andreas Nürnberger†, Raimund Dachselt∗
Faculty of Computer Science
University of Magdeburg, Germany
{stellmach, dachselt}@acm.org; {stober, andreas.nuernberger}@ovgu.de
ABSTRACT
While eye tracking is becoming more and more relevant as
a promising input channel, diverse applications using gaze
control in a more natural way are still rather limited. Though
several researchers have indicated the particular high poten-
tial of gaze-based interaction for pointing tasks, often gaze-
only approaches are investigated. However, time-consuming
dwell-time activations limit this potential. To overcome this,
we present a gaze-supported fisheye lens in combination with
(1) a keyboard and (2) and a tilt-sensitive mobile multi-
touch device. In a user-centered design approach, we elicited
how users would use the aforementioned input combinations.
Based on the received feedback we designed a prototype sys-
tem for the interaction with a remote display using gaze and
a touch-and-tilt device. This eliminates gaze dwell-time acti-
vations and the well-known Midas Touch problem (uninten- Figure 1: The envisioned setting for gaze-supported
tionally issuing an action via gaze). A formative user study interactions using a large remote display.
testing our prototype provided further insights into how well
the elaborated gaze-supported interaction techniques were
experienced by users.
video, image, and music collections demand new approaches
to keep this content accessible to a user. In contrast to
Categories and Subject Descriptors queries posed as text or as an example (e.g., a similar ob-
H.5.2 [User Interfaces]: [User-centered design, Input de- ject), exploratory systems address scenarios where users can-
vices and strategies, Interaction styles] not formulate such a query – e.g., because the retrieval goal
cannot be clearly specified in advance. A common approach
General Terms to this problem is to provide an overview visualization of the
content together with some means for navigation.
Design Representing large image collections on a screen may lead
to the challenge how to provide sufficient details while still
Keywords maintaining an adequate context due to the limited screen
Eye tracking, multimodal interaction, gaze control space. While available display sizes are increasing, suitable
focus and context techniques remain crucial for the explo-
ration of large data representations. A comprehensive re-
1. INTRODUCTION view about focus and context techniques is presented by
One of the big challenges of computer science in the 21st Cockburn et al. [5]. This issue plays an important role in
century is the digital media explosion. Steadily growing various application areas including the exploration of infor-
∗
User Interface & Software Engineering Group mation visualizations, geographic information systems, vir-
†
Data & Knowledge Engineering Group tual 3D scenes, but also the interaction with 2D graphical
user interfaces (GUIs).
A fisheye lens is one solution to locally emphasize items
of interest while still maintaining context information (local
Permission to make digital or hard copies of all or part of this work for zooming). In this regard, information about a user’s visual
personal or classroom use is granted without fee provided that copies are attention can support the accentuation of important content
not made or distributed for profit or commercial advantage and that copies (i.e., data that are currently looked at). This is also bene-
bear this notice and the full citation on the first page. To copy otherwise, to ficial for gaze-based selections (see e.g., [15, 2, 11]) as small
republish, to post on servers or to redistribute to lists, requires prior specific target items are difficult to hit via gaze. Several researchers
permission and/or a fee.
NGCA ’11, May 26-27 2011, Karlskrona, Sweden have indicated the high potential of gaze-based interaction
Copyright 2011 ACM 978-1-4503-0680-5/11/05 ...$10.00. for efficient pointing tasks (e.g., [13, 23, 2]) as gaze often
2. precedes a manual action. However, the Midas Touch prob- While the SpringLens is also a context-preserving magnify-
lem (unintentionally issuing an action via gaze) poses one of ing glass (similar to the fisheye lens used by Ashmore et al.
the major challenges for gaze-based interaction [13, 12]. One [2]) it has the additional advantage to be able to magnify
approach to overcome this problem is a suitable combination multiple areas of custom shape and to apply data-driven
with additional input modalities (see e.g., [4]). distortions in real-time. After all, while the SpringLens rep-
This motivated us to design gaze-supported remote inter- resents an interesting distortion technique, it has not yet
actions for exploring extensive amounts of images displayed, been combined with gaze input. In this regard, Shoemaker
for example, on large screens as illustrated in Figure 1. and Gutwin [18] also describe fisheye lenses for multi-point
For this purpose smartphones provide interesting features interactions, however, they only use single-mouse input. In-
for interacting with remote multimedia content, such as ac- terestingly, they apply a dwell-time (via mouse) to trigger
celerometers for tilt and throw gestures and touch-sensitive the fisheye lens instead of using an additional button. There-
screens [7]. In the scope of this paper, we term these type of fore, this approach could be of interest for the adaptation to
devices as touch-and-tilt devices. In addition, we look at a a gaze-only interaction.
combination of gaze and a keyboard. Thereby, the keyboard Facilitating a touch-and-tilt device for the gaze-supported
can be seen as a representative modality for other input de- exploration of large image collections on a remote display
vices that have distinct physical buttons, such as gaming has not been investigated so far (at least to the best of our
controller or remote controls for television sets. knowledge). However, several works present gaze-supported
Our work contributes to a deeper investigation of gaze- interaction with large displays for target selections [3] or
supported techniques for the exploration of large image col- for example in combination with freehand pointing [21] and
lections using an adaptive non-linear fisheye lens (among hand gestures [22].
others). The remaining paper is structured as follows: First, In a nutshell, gaze-based fisheye lenses have not been com-
we discuss how gaze has been applied for the interaction with bined with additional input devices yet. Also, they have
fisheye lenses in Section 2. In Section 3 we introduce a mul- not been used for the exploration of large image collections.
timedia retrieval tool that we enhanced for gaze-supported However, previous work provides a good foundation and
interaction with (1) a keyboard and (2) a mobile touch-and- leads on what to consider for gaze-supported fisheye lenses,
tilt device. Following a user-centered design approach, po- such as hiding the lens if not explicitly required [2, 18].
tential users have been interviewed on how they would op-
erate such input combinations for browsing in a large set of
images. The resulting user-elicited gaze-supported interac- 3. GAZE GALAXY
tion techniques are presented in Section 4. Based on these This section describes the general design of our exploratory
insights, we propose a conflict-free set of multimodal interac- multimedia retrieval system called GazeGalaxy with its im-
tion techniques which we implemented in a prototype system plemented SpringLens technique. The underlying applica-
(see Section 5). This system was evaluated in a formative tion is outlined in Section 3.1. As our aim is to advance the
user study which is reported in Section 6. The received qual- system for a novel gaze-supported interaction, the interac-
itative results and the concluding discussion in Section 7 tion tasks and functionality mappings that are available in
provide guidance on how to further improve the design of GazeGalaxy are specifically addressed in Section 3.2. The
gaze-supported interaction techniques. novel gaze-supported interaction techniques, however, will
be described in Section 5 based on initial interviews with
2. RELATED WORK potential future users that are reported in Section 4.
Eye pointing speed and accuracy can be improved by tar-
get expansions [15, 2]. For this purpose, Ashmore et al. [2] 3.1 System Description
describe gaze-based fisheye lenses to locally magnify the dis- GazeGalaxy is based on a multimedia retrieval system for
play at the point-of-regard (POG) which allows preserving multi-facet exploration of music [19] and image collections
the peripheral resolution. Another approach is to decrease [20] as presented by Stober et al. The application uses a
the size of peripheral windows (minification) to preserve the galaxy-metaphor to visualize a collection which can com-
focal window at the original resolution [9]. prise several thousand objects. As illustrated in Figure 2,
Ashmore et al. [2] point out that hiding the fisheye lens an overview of the entire collection is presented by displaying
during visual search helps the user to get a better overview some of the items as spatially well distributed thumbnails for
before making a selection. In addition, they also claim that a orientation. The remaining content is represented as indi-
localized target expansion has the advantage of maintaining vidual points (“stars”). The initial distribution of the objects
detail and context. Miniotas et al. [15], however, express is computed using multi-dimensional scaling [14]. This di-
that the benefits of dynamic target expansions are arguable mensionality reduction technique produces a 2D projection
due to inaccurate and jittering eye movements. As they of the high-dimensional dataset while trying to correctly pre-
point out themselves, this can be compensated by specialized serve the distances between the objects.1
algorithms to stabilize the eye cursor [24, 15]. This results in neighborhoods of similar objects (i.e. with
Cockburn et al. [5] provide a comprehensive review about small pairwise distances) based on features the user can
focus-and-context techniques including fisheye views. As an weight, such as a particular color distribution [20]. Users
example, Ashmore et al. [2] use an underlying elastic mesh can enlarge interesting neighborhoods with a SpringLens-
for the fisheye deformations with a flat lens top (with a con- based fisheye lens causing more thumbnails to be displayed
stant zooming level). They use a single lens with a gaze in a specific region and at a larger size (see Figure 2).
dwell-based activation. Another and particularly promising
fisheye lens technique is the Spring–Lens from Germer et al. 1
Object distances are computed from content-based fea-
[10] for distorting images based on a mass-spring model. tures that are automatically extracted.
3. Task Default interaction technique
Change... Keyboard & Mouse
Lens position Right-click + Drag mouse pointer
Lens magnification Right-click + Mouse-wheel
Pan Cursor keys or
Left-click + Drag
Zoom Mouse-wheel or Press +/- keys
Thumbnail size Press PageUp/PageDown keys
Table 1: The main interaction tasks that are avail-
able in GazeGalaxy with the current task mappings.
Figure 2: Screenshot of GazeGalaxy. (The partial
grid is only displayed to show the underlying mesh cussed in the order as listed in Table 1 for (1) and (2). For
structure. Inverted color scheme for printing.) this, we asked each participant how he/she could imagine to
perform a certain action with each input combination. Par-
ticipants could describe multiple approaches for interaction.
3.2 Interaction Tasks and Mappings 4.1 User-elicited Task Mappings
GazeGalaxy supports common interaction tasks for the
In the following, we discuss the qualitative feedback re-
exploration of information spaces [17], such as overview,
ceived from the eight interviewees. As assumed, the inter-
zoom + pan, and details on demand. Additional actions,
viewees were concerned that gaze should not be overloaded
for example, to toggle filter modes, are also implemented
with too many tasks as an undisturbed visual inspection of
but will not be emphasized further in this paper as we focus
the screen is essential for exploration tasks. Hence, all in-
mainly on the lens interaction and on panning and zoom-
terviewees expressed the opinion that the gaze input should
ing in the workspace as these are crucial tasks for various
be explicitly activated by a mode switch (more about possi-
application contexts. The interaction tasks that we want to
ble ways to toggle modes will be explained in the following
investigate further in the scope of this paper are listed in
paragraphs).
Table 1 with the currently implemented functionality map-
pings using keyboard and mouse. With respect to these task
mappings, we want to additionally point out that some tasks
4.1.1 Keyboard and Gaze
are currently mapped repeatedly to different input variants. For the combination of a keyboard and an eye tracker, all
Panning can, for example, be performed by either dragging interviewees agreed that the gaze may substitute the mouse
the workspace with a mouse or by using the cursor buttons well as a pointing device, for example, for setting the lens
on the keyboard. This gives users a clear and non-conflicting position. However, all interviewees asked for the possibility
variety to choose the technique that fits best to their indi- to switch modes so that the gaze input would not be active
vidual interaction style. at all times to decrease the mental workload (and the Midas
Touch problem). Thus, participants proposed an approach
similar to switching between upper and lower case charac-
4. PRE-USER SURVEY ters on a typewriter: Either continuously hold a certain key
Aiming for a user-centered development, we conducted in- (e.g., Shift) to activate a mode or discretely toggle modes
terviews with several volunteers to incorporate users already by pressing a key (e.g., the Caps Lock key). While six inter-
at an early stage of the design process. This helped us to viewees preferred the former approach, two also suggested a
find out how they would spontaneously use eye gaze with combination of both.
either a keyboard or a touch-and-tilt device to interact with Panning the workspace could be performed using the cur-
the GazeGalaxy tool. This is a similar procedure as pre- sor keys as indicated by six interviewees. Two participants
sented by Nielsen et al. [16] for the development of natural could also imagine to use their gaze at the screen borders
interaction interfaces. for panning (similar to the approach described by Adams
Participants. Eight un-paid volunteers (six male, two et al. [1]). Thus, if looking at an item close to the upper
female) participated in our interview. All were students or screen border, this item will slowly move towards the center
employees at a local university. Six participants had prior of the screen. In this respect, one interviewee also proposed
experience with eye tracking technology. All were familiar to look at an item and hit a certain key (e.g., C) to center
with mobile touch-and-tilt devices. the currently fixated item.
Procedure. First, the GazeGalaxy tool was introduced For adapting the other interaction tasks (lens magnifica-
to each participant. The main interaction tasks (see Ta- tion, zoom level, and thumbnail size), all interviewees in-
ble 1) were demonstrated using a mouse and a keyboard. dicated that they would use keyboard buttons. One user
Afterwards, each participant was asked to imagine to work expressed the idea that holding a certain button could initi-
with GazeGalaxy on a large remote display using a combi- ate displaying a graphical scale for selecting, for example, a
nation of (1) keyboard and gaze and (2) touch-and-tilt and thumbnail size by gaze. This approach is also applicable to
gaze. They were encouraged to actually stand up and hold other aspects such as the zoom level and lens size. Except
a smartphone (an iPhone) to better put themselves in the for this idea the integration of additional GUI elements was
envisioned situation. Then the interaction tasks were dis- not proposed.
4. 4.1.2 Touch-and-tilt and Gaze 5. DESIGN OF GAZE-SUPPORTED INTER-
Most controversy was on how to map panning and zoom- ACTIONS
ing tasks to a combination of gaze and a touch-and-tilt de- We structured the received user feedback and elaborated
vice - not resulting in any prevailing overall preferences. individual interaction sets that are free of ambiguous map-
First of all, seven participants preferred holding the touch- pings (i.e., conflict-free): one set for condition (1) keyboard
and-tilt device in one hand with a vertical layout (“as you and gaze and one for condition (2) touch-and-tilt and gaze.
would normally hold a cell phone”). Although one partic- In general, we decided on the interaction technique that was
ipant also proposed to use the horizontal layout holding most frequently mentioned for a certain task (given that this
the device in both hands, his main preference remained for technique has not already been assigned to another task).
the first approach. This has important implications for the This approach worked well except for condition (2) as no
touchscreen design as most interface elements should then clear overall user preferences could be identified for panning
be easily reachable by the thumb (as illustrated in Figure 1). and zooming. An overview of the elaborated interaction sets
Lens position. Six interviewees would follow an analog is listed in Table 2. In the following, we explain the individ-
approach for positioning the lens as for the keyboard and ual sets in more detail.
gaze combination: while touching the mobile screen the lens
is positioned at the point-of-regard. Also a discrete toggle 5.1 Keyboard & Gaze
using a virtual button on the touch device has been men- The gaze is used to indicate where to position the fish-
tioned by two people. Two interviewees would not use gaze eye lens and where to zoom in. These actions will, however,
for adapting the lens position, but instead would use the only be carried out, if an additional key is pressed on the
touchscreen as a corresponding representation of the large keyboard (e.g., pressing the PageUp key to zoom in or hold-
display. Thus, if the user tabs into the left upper corner on ing the Ctrl key for lens positioning). The other tasks are
the touchscreen, the lens will be positioned at the left upper mapped to different buttons on the keyboard as listed in
corner of the large remote display. Table 2.
Lens size. While six respondents would use a virtual
slider with a one-touch slide gesture for imitating the mouse 5.2 Touch-and-tilt & Gaze
scroll wheel, also a pinch gesture was proposed. Before deciding on particular task mappings, several basic
Panning. While three people suggested to look at the conditions had to be specified that came up during the in-
screen borders to pan (as previously mentioned for the key- terviews. As several respondents expressed that they would
board condition), another three would rather use a panning prefer holding the touch-and-tilt device in one hand and ide-
gesture on the touch device. Two interviewees could imagine ally only use the thumb to interact, we decided to refrain
to use the tilt functionality. Another idea was to stick (glue) from using multitouch input such as a pinch gesture and in-
the workspace to the current gaze position while touching a stead focus on single-touch gestures (at least for the scope of
designated area on the mobile display. this paper). Furthermore, two people were concerned that
Zooming. While all interviewees agreed on setting the using a tilt could unintentionally be performed while talk-
zooming pivot by gaze (i.e., a zoom is performed at the ing or moving around. Thus, just as well as for gaze-based
position where the user currently looks at), answers on how input (the Midas Touch problem), we decided that an ad-
to zoom in and out differed significantly. Three interviewees ditional explicit action (namely touching a certain area on
voted for tilting the mobile device forward and backward the touchscreen) is required to issue a gaze- or tilt-based ac-
to zoom in and out. Two respondents mentioned a pinch tion. Finally, the need to shift the visual attention between
gesture on the touchscreen; two others a slide gesture. One mobile and remote display should be kept to a minimum.
person would use a pull gesture (quickly moving the device Thus, the interface and interaction with the mobile device
towards the user) to zoom in and a throw gesture (quickly should be designed to allow blind interaction. This means
moving the device away from the user) to zoom out (see e.g., that regions on the touchscreen should be large enough and
[7]). Also, the possibility to display additional GUI elements arranged in a way to allow users to interact with it without
on the large display was mentioned, however, not preferred. looking at the mobile screen.
Thumbnail size. Participants suggested to use a pinch Based on these basic assumptions we implemented a first
gesture (as mentioned by three respondents), a touch slider prototype for exploring data in the GazeGalaxy tool via gaze
(two respondents), a virtual button on the touchscreen (one and touch-and-tilt input. Preliminary graphical user inter-
respondent), to tilt the device while continuously touching it face designs for the touch-and-tilt device are shown in Fig-
for a mode switch (one respondent), and to turn the mobile ure 3. The design of this prototype and the elaborated in-
device like a compass (one respondent). teraction techniques are discussed in detail in the following.
The gathered user feedback provided valuable first in- We distinguish three different modes: (a) pan+zoom, (b)
sights for deciding on suitable techniques for a more natural fisheye lens, and (c) thumbnail size. While we combined
gaze-supported interaction. However, further analysis for mode (a) and (b) on one screen of the multitouch device, we
the specification of unambiguous interactions is required as decided to use an additional screen for altering the thumb-
several techniques were mentioned repeatedly for different nail size. This was also motivated by the idea to integrate
tasks, such as tilting the device to pan, zoom, and adapt new tasks (e.g., for filtering the displayed content) in the
the thumbnail size. Especially for the combination of touch, future that would hardly fit on one single screen while still
tilt, and gaze input we discuss design considerations in more providing the possibility for a blind interaction. The tabs
detail in the next section. can be switched at the top of the touchscreen to leave the
centered and easier to reach regions for more frequent tasks.
The first screen is divided into two active areas: a large
region for the pan+zoom mode to the left and a smaller
5. Task Different input combinations
Change... Keyboard & Eye Tracker Touch-and-tilt device & Eye Tracker
Lens position Look + Hold key Look + Touch
Lens magnification Press keys (e.g. 8 and 2 on the num pad) Touch slide gesture
Pan Cursor keys or Relative panning on touchscreen
Look at screen borders Look at screen borders
Zoom Look + Press +/- keys Look + Touch + Tilt
Thumbnail size Press PageUp/PageDown keys Touch slide gesture (with mode switch)
Table 2: The main interaction tasks that are available in the GazeGalaxy tool and possible functionality
mappings to different multimodal input combinations.
used as a starting position. Tilting the device with respect
to this starting position by at least a certain threshold will
result in a zoom action. Thus, we use the relative position-
ing data instead of defining absolute positions for how to
hold the device. Dachselt and Buchholz [6] use absolute po-
sitions which may cause problems due to differing physical
constraints (i.e., some people cannot bend their wrists as
much as others).
Thumbnail size mode. The second screen on the mobile
device can be reached by touching the second tab at the top.
Here the thumbnail size can be altered by performing a slide
gesture. Moving the finger up results in larger and down in
smaller thumbnail sizes.
Figure 3: A first user interface prototype for the After all, the elaborated interaction sets do not require
touch-and-tilt device. any gaze dwell-time activations and thus should allow for a
more fluent and quick gaze-based interactions. We made a
particular point of eliminating the Midas Touch problem by
area for the lens mode to the right (see Figure 3). As soon accompanying gaze-based and tilt interactions with an addi-
as one of the two areas is touched, the corresponding mode is tional explicit action such as pressing a button or touching
activated. Thus, no actions (whether by gaze or tilt) will be the mobile screen.
performed if there is no touch event. As mentioned before
this aims for preventing the Midas Touch problem. 5.3 System Setup
Fisheye lens mode. The fisheye lens is positioned by We enhanced GazeGalaxy to support various input modal-
looking at a location on the remote display while putting a ities. For gathering gaze data we use a Tobii T60 table-
finger on the lens area at the right of the first screen. If the mounted eye tracker. As touch-and-tilt device we use an
finger slides up from the initial touch position, the lens size iPhone/iPod Touch, but the system could easily be adapted
will be increased (and vice versa). to other multitouch smartphone devices as well. Finally,
Pan+Zoom mode. If the user touches the pan area, a computer is needed for executing GazeGalaxy. For our
the pan+zoom mode will be activated. Once this mode is system setup, we run GazeGalaxy on the same system that
active, panning can either be performed by looking at the hosts the Tobii T60 eye tracker. The devices can communi-
screen borders (from the remote screen) or by panning on cate among each other using the local area network. Thus, to
the touchscreen (as also described in [6]). For our prototype communicate with the iPhone/iPod the computer requires
we use rectangular active regions at the screen borders for a wireless network adapter. The communication is handled
gaze panning. This eventually is the same approach as used by a Virtual Reality Peripheral Network (VRPN) interface
by Adams et al. [1] for their screen panning regions. If the that we have extended to support various input devices. An
user looks at the left screen border, the workspace will shift overview of the system setup is presented in Figure 4. In
to the right. This allows to move items that are currently the near future, the system will be further extended by ad-
close to the left border closer to the center of the screen. ditional input modalities, such as a mobile binocular eye
Panning via touch is based on relative touch positions in tracker. This will allow to test the implemented interaction
the pan area on the multitouch screen. This means that techniques with larger remote displays as originally envi-
no matter where the user touches the pan area initially, an sioned in Figure 1.
upwards movement of the finger will lead to shifting the
workspace up (analog for the other directions). This further
supports a blind interaction.
As soon as touching the pan area, the user can also zoom 6. FORMATIVE USER STUDY
by tilting the device forward and backward as also proposed To obtain first indications on the usability of our elabo-
by Dachselt and Buchholz [6]. For this, the orientation of rated prototype for the combination of gaze and touch-and-
the mobile device when activating the pan+zoom mode is tilt input, we conducted a formative qualitative user study
6. Figure 5: A participant standing in front of the Tobii
T60 eye tracker to interact via gaze and an iPod
Touch with the GazeGalaxy tool.
Figure 4: Schematic overview of the system setup
for the gaze-supported multimodal interaction with
GazeGalaxy.
order to explore a large image collection (with 800 pictures).
At the end, the experimenter concluded by asking the par-
ticipants about particular advantages and disadvantages of
that gathered participants’ impressions for using these tech- each technique (again in the order as presented in Table 2).
niques with GazeGalaxy.
Participants. Six staff members of a local university
participated in the study (all male, with an average age of 6.1 Results
29). While three people had already participated in the pre- Overall the combination of gaze and a remote touch-and-
user survey, for the other three people the topic and the tilt interaction was perceived very positively. Three partici-
presented techniques were completely new. All participants pants described it as intuitive and easy to understand. Nev-
were right-handed. ertheless, interacting via gaze felt unfamiliar (mentioned by
Apparatus. We used the system setup described in Sec- two participants). Some participants were actually surprised
tion 5.3. The Tobii T60 allows determining screen gaze po- that the interaction worked that well as they had never used
sitions at a frame rate of 60 Hz based on corneal-reflections gaze input before. Some minor remarks were mentioned
that are identified in streamed video data. For stabiliz- how to further improve the interactions especially concern-
ing the gaze cursor, we used a speed reduction technique ing better visual feedback about the current gaze position
of Zhang et al. [24]. This means that raw gaze data were and active mode. In principle, all participants found it good
partially integrated with the previous gaze position. In ad- that an additional explicit action (i.e. a touch-event in a des-
dition, a minimal threshold distance (30 pixels) had to be ignated area) had to be performed to enable gaze and tilt
traveled via gaze to assign the current value as new gaze cur- input. However, one participant suggested that he did not
sor position. For the gaze panning, rectangular pan regions find this intuitive as it could be confusing which area must
extending to 100 pixels at each screen boarder were defined be touched to issue a certain command. Two participants
(at a 1280 x 1024 screen resolution). found it particularly interesting that the rough gaze position
As touch-and-tilt device we used an iPod Touch (2nd gen- is sufficient for interaction. Although the interaction with
eration). This device allows multitouch interaction on a mo- only one hand was considered convenient, the possibility to
bile screen and provides a three-axis accelerometer for tilt- turn the iPod and use a two-handed interaction on a hori-
ing. The graphical user interface from the iPod was designed zontal screen layout was proposed by one participant again
according to the screen prototype illustrated in Figure 3. (see Section 4).
The eye tracker was positioned on an elevated rack (see Lens position. Setting the fisheye lens position via gaze
Figure 5) so that the participants could comfortably stand was described as natural and useful. Only moving the lens
in front of it. This should give them a better feeling for the if touching the device was found very helpful, as the fear
remote interaction with a distant display and should show us existed to easily loose orientation (as mentioned by five par-
how users would hold the mobile device in such a situation. ticipants). Three participants would have liked to use the
Procedure. Participants were welcomed and briefly in- gaze lens while also being able to pan and zoom. This is
troduced to the purpose of the survey (to get first impres- currently not supported for the assumed thumb-only inter-
sions on some novel interaction techniques). First, it was action. One participant proposed a persistent gaze mode as
checked that the eyes of a user could be correctly detected by mentioned in the pre-user survey for the Caps Lock-mode.
the eye tracker. Otherwise the distance and angle to the eye Lens size. Changing the magnification of the fisheye lens
tracker screen had to be adjusted. Secondly, an eye tracker via a slider on the right side of the screen was found good
calibration was performed. Then the GazeGalaxy tool was (four participants). However, the layout may need to be
started and the interaction tasks with their particular map- adjusted for left-handers. The slider could be improved by
pings were explained in the order as listed in Table 2. The a flick gesture to set a motion in action. One participant
participants could directly test the techniques and were en- wished for a larger slider area and for arranging the slider
couraged to verbally express their thoughts about the inter- to the opposite side as he argued that he could reach the
action. After all techniques had been explained, the partic- opposite side better with his thumb. Additional ideas for
ipants could test the different interaction techniques in any changing the lens size included a dwell-time based increase
7. in size or incorporating the distance from a user’s head to • The Midas Touch problem is avoided as users can com-
the screen. municate via an additional input channel whether an
Panning via touch. All participants found it good to action is really intended.
be able to move the workspace by sliding the finger on the
touchscreen. As mentioned before, an additional flick ges- • For the same reason, there is no need for dwell-time
ture to start a certain motion that slowly subsides would activations which otherwise would slow down the in-
be nice. One person proposed that instead of using a touch teraction.
event, a “flight-mode” could be activated to move through • The different input modalities complement well for
the scene by tilting the iPod. supporting multiple tasks simultaneously (such as pan-
Panning via gaze. As described in Section 5, it is also ning and zooming), which is difficult for gaze-only in-
possible to move the view by looking at the screen borders. teraction.
Three participants found this particularly useful for mov-
ing items that are cut off by a screen border towards the The importance of well-designed feedback was confirmed
screen center. In this respect, two participants mentioned as users want to be assured that the system understood their
that they find this technique ideal for fine adjustments of the intentions correctly and that the intended mode has been ac-
view, while they would rather use the touch panning to cross tivated. Visual feedback about the current mode could, for
larger distances. In addition, larger margins were desired (as example, be indicated by adapting the cursor’s shape and
indicated by two participants) as it was sometimes difficult color as done by Istance et al. [12] for different gaze interac-
to hit the screen corners via gaze. It was also suggested to tion conditions. Feedback also plays an important role for
increase the panning speed towards the screen borders. Fur- identifying tracking problems (whether of the gaze or touch
thermore, two participants missed some form of feedback for and tilt data) – e.g., if the communication to the devices is
the active margin regions such as a highlight-on-hover effect. temporarily lost. At the current stage, the GazeGalaxy ap-
Zooming via tilt and gaze pivoting. It was found plication’s only direct feedback is the focus visualization by
very intuitive by all participants that the view zooms in at the SpringLens. Further possibilities also incorporating hap-
the location that is currently looked at. While most par- tic feedback (e.g., vibrations) and auditory feedback (e.g.,
ticipants liked to tilt forward/backward for zooming, one beep sounds) for not overloading the visual input channel
participant intuitively double-tabbed on the panning region need to be investigated.
to zoom, and another one tilted left and right. Both partic- Recalling the original application scenario, exploratory
ipants expressed that they in general do not like to use the search, the combination of a fisheye-based visualization with
forward and backward tilt as they fear to need to twist their gaze input for focus control indeed seems to be very promis-
wrists uncomfortably. After explaining that our technique ing according to received user feedback (which confirms find-
uses the device’s relative motions after touching the panning ings from Ashmore et al. [2]). The GazeGalaxy application
area (see Figure 5.2), they dismissed their concerns. could clearly be further elaborated – especially in terms of
Thumbnail size. Using a touch slide gesture for chang- additional gaze-contingent visualizations [8] to emphasize
ing the thumbnail size was found good. However, four par- objects of (visual) interest. Furthermore, the possibilities
ticipants disliked the need to switch between tabs. On the for blind interaction with the system should be extended:
one hand, three participants explained that they did not like Here, using the orientation of the touch-and-tilt device (i.e.
to look down from the primary screen (the Tobii monitor) to vertical or horizontal layout) as mode switch is an interesting
the mobile device to switch modes. In this regard, it was pro- option as an alternative to the tabs interface.
posed to rearrange the tab icons to switch between tabs even For the near future, porting the system to a large dis-
without looking at the mobile screen (blind interaction), for play as shown in Figure 1 is pursued using a mobile eye
example, by locating each tab icon at different screen cor- tracker. Finally, the implemented novel interaction tech-
ners. On the other hand, the second mode only contained niques in GazeGalaxy need to be compared against other
the feature to change the thumbnail size and although this input combinations in terms of retrieval performance and in-
is not a frequent task, it may disrupt the workflow. Thus, teraction efficiency for the originally intended setting (some
different solutions were offered by the participants, includ- exploratory task).
ing to integrate this task into the first tab-screen or to use
a multitouch gesture after all (e.g., a pinch). 8. CONCLUSION
This paper presented a detailed description of a user-
centered design process for gaze-supported interaction tech-
niques for the exploration of large image collections. For
this purpose, gaze input was combined with additional in-
7. DISCUSSION put modalities: (1) a keyboard and (2) a mobile tilt-enabled
All in all, the combination of gaze, touch, and tilt input multitouch screen. The integration of user feedback at such
for interacting with a remote display was perceived as very an early stage of the design process allowed for the develop-
promising in the formative user study. At first sight, such a ment of novel and more natural gaze-supported interaction
combination appears to be a step backwards with respect to techniques. While gaze acted as a pointing modality, the
intuitive interaction design. Although using gaze for point- touch and tilt actions complemented the interaction for a
ing is considered very intuitive, having to use another input multifaceted interaction. Based on user-elicited interaction
channel simultaneously increases the effort and complexity techniques we developed an extended multimedia retrieval
of the interaction. However, as participants from both stud- system, GazeGalaxy, that can be controlled via gaze and
ies reported, this is accepted, because this allows for a more touch-and-tilt input to explore large image collections. First
relaxed gaze-based interaction for the following reasons: user impressions on the implemented interaction techniques
8. were gathered and discussed. Results indicate that gaze in- [11] D. W. Hansen, H. H. T. Skovsgaard, J. P. Hansen,
put may serve as a natural input channel as long as certain and E. Møllenbach. Noise tolerant selection by gaze-
design considerations are taken into account. First, gaze controlled pan and zoom in 3d. In Proceedings of
data is inherently inaccurate and thus interaction should ETRA’08, pages 205–212. ACM, 2008.
not rely on precise positions. Using the gaze positions for
setting a fisheye lens and zooming in at the point-of-regard [12] H. Istance, R. Bates, A. Hyrskykari, and S. Vickers.
were described as intuitive. Secondly, users should be able Snap clutch, a moded approach to solving the midas
to confirm actions with additional explicit commands to pre- touch problem. In Proceedings of ETRA’08, pages 221–
vent unintentional actions. 228. ACM, 2008.
[13] R. J. K. Jacob. What you look at is what you get: eye
9. ACKNOWLEDGMENTS movement-based interaction techniques. In Proceedings
We want to thank all volunteers for their insightful com- of CHI’90, pages 11–18, 1990.
ments in our evaluations. This research was supported by [14] J. Kruskal and M. Wish. Multidimensional scaling.
the German National Merit Foundation, the German Re- Sage, 1986.
search Foundation (DFG) project AUCOMA, and the Ger-
man Ministry of Education and Science (BMBF) project ˇ
[15] D. Miniotas, O. Spakov, and I. S. MacKenzie. Eye gaze
ViERforES-II (01IM10002B). interaction with expanding targets. In Proceedings of
CHI’04 - Extended abstracts, pages 1255–1258, 2004.
References [16] M. Nielsen, M. St¨rring, T. Moeslund, and E. Granum.
o
[1] N. Adams, M. Witkowski, and R. Spence. The inspec- A procedure for developing intuitive and ergonomic ges-
tion of very large images by eye-gaze control. In Pro- ture interfaces for HCI. In Gesture-Based Communi-
ceedings of the working conference on Advanced visual cation in Human-Computer Interaction, volume 2915
interfaces, AVI ’08, pages 111–118, 2008. of Lecture Notes in Computer Science, pages 105–106.
Springer Berlin / Heidelberg, 2004.
[2] M. Ashmore, A. T. Duchowski, and G. Shoemaker. Ef-
ficient eye pointing with a fisheye lens. In Proceedings of [17] B. Shneiderman. The eyes have it: A task by data type
Graphics Interface 2005, GI ’05, pages 203–210, 2005. taxonomy for information visualizations. In Proceedings
of the IEEE Symposium on Visual Languages, 1996.
[3] H.-J. Bieg. Gaze-augmented manual interaction. In
[18] G. Shoemaker and C. Gutwin. Supporting multi-point
Proceedings of CHI’09 - Extended abstracts, pages
interaction in visual workspaces. In Proceedings of
3121–3124, 2009.
CHI’07, pages 999–1008, 2007.
[4] E. Castellina and F. Corno. Multimodal gaze inter- [19] S. Stober and A. N¨rnberger. A multi-focus zoomable
u
action in 3d virtual environments. In COGAIN ’08: interface for multi-facet exploration of music collec-
Proceedings of the 4th Conference on Communication tions. In 7th International Symopsium on Computer
by Gaze Interaction, pages 33–37, 2008. Music Modeling and Retrieval, pages 339–354, 2010.
[5] A. Cockburn, A. Karlson, and B. B. Bederson. A re- [20] S. Stober, C. Hentschel, and A. N¨rnberger. Multi-
u
view of overview+detail, zooming, and focus+context facet exploration of image collections with an adaptive
interfaces. ACM Comput. Surv., Vol. 41:1–31, 2009. multi-focus zoomable interface. In WCCI ’10: Proceed-
ings of 2010 IEEE World Congress on Computational
[6] R. Dachselt and R. Buchholz. Throw and tilt – seamless Intelligence, pages 2780–2787, 2010.
interaction across devices using mobile phone gestures.
In Lecture Notes in Informatics, Vol. P-133, MEIS ’08, [21] D. Vogel and R. Balakrishnan. Distant freehand point-
pages 272–278, 2008. ing and clicking on very large, high resolution displays.
In UIST ’05: Proceedings of the 18th annual ACM
[7] R. Dachselt and R. Buchholz. Natural throw and tilt symposium on User interface software and technology,
interaction between mobile phones and distant displays. pages 33–42, 2005.
In Proceedings of CHI’09 - Extended abstracts, pages
3253–3258, 2009. [22] B. Yoo, J.-J. Han, C. Choi, K. Yi, S. Suh, D. Park, and
C. Kim. 3d user interface combining gaze and hand ges-
[8] A. T. Duchowski, N. Cournia, and H. Murphy. Gaze- tures for large-scale display. In Proceedings of CHI’10
contingent displays: A review. CyberPsychology & Be- - Extended abstracts, pages 3709–3714, 2010.
havior, 7(6):621–634, 2004.
[23] S. Zhai, C. Morimoto, and S. Ihde. Manual and gaze
[9] D. Fono and R. Vertegaal. Eyewindows: evaluation of input cascaded (magic) pointing. In Proceedings of
eye-controlled zooming windows for focus selection. In CHI’99, pages 246–253, 1999.
Proceedings of CHI’05, pages 151–160, 2005. [24] X. Zhang, X. Ren, and H. Zha. Improving eye cur-
sor’s stability for eye pointing tasks. In Proceedings of
[10] T. Germer, T. G¨tzelmann, M. Spindler, and
o
CHI’08, pages 525–534, 2008.
T. Strothotte. SpringLens – Distributed Nonlinear
Magnifications. In Eurographics - Short Papers, pages
123–126. EGPub, 2006.