Master thesis at Dublin City University, September 2013.
Co-advised by Xavier Giró-i-Nieto, Kevin McGuinness and Noel O'Connor.
More details:
https://imatge.upc.edu/web/publications/investigating-eeg-saliency-and-segmentation-applications-image-processing
BSc thesis by Marcel Tella-Amo at the Telecommunications Engineering studies in the School of Engineering of Terrassa (EET), Universitat Politècnica de Cataluna (UPC).
Co-advised by Xavier Giró-i-Nieto and Albert Gil.
This diploma thesis aims to provide a framework for developing web applications for ImagePlus, the software develpment platform in C++ of the Image Processing Group of the Technical University of Catalonia (UPC). These web applications are to demonstrate the functionality of the image processing algorithms to any visitor to the group website. Developers are also benefited from this graphical user interface because they can easily create Graphical User Interfaces (GUIs) for the processing algorithms.
More information:
QoMEX2014 - Analysing the Quality of Experience of Multisensory Media from Me...Jacob Donley
This presentation was given at QoMEX 2014, the 6th International Workshop on Quality of Multimedia Experience.
Abstract:
This paper investigates the Quality of Experience (QoE) of multisensory media by analysing biosignals collected by electroencephalography (EEG) and eye gaze sensors and comparing with subjective ratings. Also investigated is the impact on QoE of various levels of synchronicity between the sensory effect and target video scene. Results confirm findings from previous research that show sensory effects added to videos increases the QoE rating. While there was no statistical difference observed for the QoE ratings for different levels of sensory effect synchronicity, an analysis of raw EEG data showed 25% more activity in the temporal lobe during asynchronous effects and 20-25% more activity in the occipital lobe during synchronous effects. The eye gaze data showed more deviation for a video with synchronous effects and the EEG showed correlating occipital lobe activity for this instance. These differences in physiological responses indicate sensory effect synchronicity may affect QoE despite subjective ratings appearing similar.
Depth-Based Real Time Head Motion Tracking Using 3D Template Matching愚 屠
In this work, we propose a system to estimate head poses only using depth information in real-time. An optimization method based on 3D model fitting is developed. We iteratively minimize the distance between source and target point clouds of a user’s head. The method give fully real-time responses (30fps) without the GPU speedup. We adopt a commodity depth sensor named Microsoft Kinect as well as Asus Xtion, and use the depth image as the only input so that our system will not be affected by illumination variations. However, the simplicity of this acquisition device comes at the cost of frequent noises in the acquired data. We demonstrate that 6 degrees of freedom real-time head motion tracking in 3D space can be achieved with such noisy depth data.
BSc thesis by Marcel Tella-Amo at the Telecommunications Engineering studies in the School of Engineering of Terrassa (EET), Universitat Politècnica de Cataluna (UPC).
Co-advised by Xavier Giró-i-Nieto and Albert Gil.
This diploma thesis aims to provide a framework for developing web applications for ImagePlus, the software develpment platform in C++ of the Image Processing Group of the Technical University of Catalonia (UPC). These web applications are to demonstrate the functionality of the image processing algorithms to any visitor to the group website. Developers are also benefited from this graphical user interface because they can easily create Graphical User Interfaces (GUIs) for the processing algorithms.
More information:
QoMEX2014 - Analysing the Quality of Experience of Multisensory Media from Me...Jacob Donley
This presentation was given at QoMEX 2014, the 6th International Workshop on Quality of Multimedia Experience.
Abstract:
This paper investigates the Quality of Experience (QoE) of multisensory media by analysing biosignals collected by electroencephalography (EEG) and eye gaze sensors and comparing with subjective ratings. Also investigated is the impact on QoE of various levels of synchronicity between the sensory effect and target video scene. Results confirm findings from previous research that show sensory effects added to videos increases the QoE rating. While there was no statistical difference observed for the QoE ratings for different levels of sensory effect synchronicity, an analysis of raw EEG data showed 25% more activity in the temporal lobe during asynchronous effects and 20-25% more activity in the occipital lobe during synchronous effects. The eye gaze data showed more deviation for a video with synchronous effects and the EEG showed correlating occipital lobe activity for this instance. These differences in physiological responses indicate sensory effect synchronicity may affect QoE despite subjective ratings appearing similar.
Depth-Based Real Time Head Motion Tracking Using 3D Template Matching愚 屠
In this work, we propose a system to estimate head poses only using depth information in real-time. An optimization method based on 3D model fitting is developed. We iteratively minimize the distance between source and target point clouds of a user’s head. The method give fully real-time responses (30fps) without the GPU speedup. We adopt a commodity depth sensor named Microsoft Kinect as well as Asus Xtion, and use the depth image as the only input so that our system will not be affected by illumination variations. However, the simplicity of this acquisition device comes at the cost of frequent noises in the acquired data. We demonstrate that 6 degrees of freedom real-time head motion tracking in 3D space can be achieved with such noisy depth data.
Inspired by Wheatstone’s original stereoscope and augmenting it with modern factored light field synthesis, we present a new near-eye display technology that supports focus cues. These cues are critical for mitigating visual discomfort experienced in commercially-available head mounted displays and providing comfortable, long-term immersive experiences.
Time-series forecasting of indoor temperature using pre-trained Deep Neural N...Francisco Zamora-Martinez
Artificial neural networks have proved to be good at time-series forecasting
problems, being widely studied at literature. Traditionally, shallow
architectures were used due to convergence problems when dealing with deep
models. Recent research findings enable deep architectures training, opening a
new interesting research area called deep learning. This paper presents a study
of deep learning techniques applied to time-series forecasting in a real indoor
temperature forecasting task, studying performance due to different
hyper-parameter configurations. When using deep models, better generalization
performance at test set and an over-fitting reduction has been observed.
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Multi illuminant estimation with c...IEEEBEBTECHSTUDENTPROJECTS
To Get any Project for CSE, IT ECE, EEE Contact Me @ 09666155510, 09849539085 or mail us - ieeefinalsemprojects@gmail.com-Visit Our Website: www.finalyearprojects.org
This document provides an overview of deep generative learning and summarizes several key generative models including GANs, VAEs, diffusion models, and autoregressive models. It discusses the motivation for generative models and their applications such as image generation, text-to-image synthesis, and enhancing other media like video and speech. Example state-of-the-art models are provided for each application. The document also covers important concepts like the difference between discriminative and generative modeling, sampling techniques, and the training procedures for GANs and VAEs.
Inspired by Wheatstone’s original stereoscope and augmenting it with modern factored light field synthesis, we present a new near-eye display technology that supports focus cues. These cues are critical for mitigating visual discomfort experienced in commercially-available head mounted displays and providing comfortable, long-term immersive experiences.
Time-series forecasting of indoor temperature using pre-trained Deep Neural N...Francisco Zamora-Martinez
Artificial neural networks have proved to be good at time-series forecasting
problems, being widely studied at literature. Traditionally, shallow
architectures were used due to convergence problems when dealing with deep
models. Recent research findings enable deep architectures training, opening a
new interesting research area called deep learning. This paper presents a study
of deep learning techniques applied to time-series forecasting in a real indoor
temperature forecasting task, studying performance due to different
hyper-parameter configurations. When using deep models, better generalization
performance at test set and an over-fitting reduction has been observed.
IEEE 2014 MATLAB IMAGE PROCESSING PROJECTS Multi illuminant estimation with c...IEEEBEBTECHSTUDENTPROJECTS
To Get any Project for CSE, IT ECE, EEE Contact Me @ 09666155510, 09849539085 or mail us - ieeefinalsemprojects@gmail.com-Visit Our Website: www.finalyearprojects.org
This document provides an overview of deep generative learning and summarizes several key generative models including GANs, VAEs, diffusion models, and autoregressive models. It discusses the motivation for generative models and their applications such as image generation, text-to-image synthesis, and enhancing other media like video and speech. Example state-of-the-art models are provided for each application. The document also covers important concepts like the difference between discriminative and generative modeling, sampling techniques, and the training procedures for GANs and VAEs.
Machine translation and computer vision have greatly benefited from the advances in deep learning. A large and diverse amount of textual and visual data have been used to train neural networks whether in a supervised or self-supervised manner. Nevertheless, the convergence of the two fields in sign language translation and production still poses multiple open challenges, like the low video resources, limitations in hand pose estimation, or 3D spatial grounding from poses.
The transformer is the neural architecture that has received most attention in the early 2020's. It removed the recurrency in RNNs, replacing it with and attention mechanism across the input and output tokens of a sequence (cross-attenntion) and between the tokens composing the input (and output) sequences, named self-attention.
These slides review the research of our lab since 2016 on applied deep learning, starting from our participation in the TRECVID Instance Search 2014, moving into video analysis with CNN+RNN architectures, and our current efforts in sign language translation and production.
Machine translation and computer vision have greatly benefited of the advances in deep learning. The large and diverse amount of textual and visual data have been used to train neural networks whether in a supervised or self-supervised manner. Nevertheless, the convergence of the two field in sign language translation and production is still poses multiple open challenges, like the low video resources, limitations in hand pose estimation, or 3D spatial grounding from poses. This talk will present these challenges and the How2✌️Sign dataset (https://how2sign.github.io) recorded at CMU in collaboration with UPC, BSC, Gallaudet University and Facebook.
https://imatge.upc.edu/web/publications/sign-language-translation-and-production-multimedia-and-multimodal-challenges-all
https://imatge-upc.github.io/synthref/
Integrating computer vision with natural language processing has achieved significant progress
over the last years owing to the continuous evolution of deep learning. A novel vision and language
task, which is tackled in the present Master thesis is referring video object segmentation, in which a
language query defines which instance to segment from a video sequence. One of the biggest chal-
lenges for this task is the lack of relatively large annotated datasets since a tremendous amount of
time and human effort is required for annotation. Moreover, existing datasets suffer from poor qual-
ity annotations in the sense that approximately one out of ten language expressions fails to uniquely
describe the target object.
The purpose of the present Master thesis is to address these challenges by proposing a novel
method for generating synthetic referring expressions for an image (video frame). This method pro-
duces synthetic referring expressions by using only the ground-truth annotations of the objects as well
as their attributes, which are detected by a state-of-the-art object detection deep neural network. One
of the advantages of the proposed method is that its formulation allows its application to any object
detection or segmentation dataset.
By using the proposed method, the first large-scale dataset with synthetic referring expressions for
video object segmentation is created, based on an existing large benchmark dataset for video instance
segmentation. A statistical analysis and comparison of the created synthetic dataset with existing ones
is also provided in the present Master thesis.
The conducted experiments on three different datasets used for referring video object segmen-
tation prove the efficiency of the generated synthetic data. More specifically, the obtained results
demonstrate that by pre-training a deep neural network with the proposed synthetic dataset one can
improve the ability of the network to generalize across different datasets, without any additional annotation cost. This outcome is even more important taking into account that no additional annotation cost is involved.
Master MATT thesis defense by Juan José Nieto
Advised by Víctor Campos and Xavier Giro-i-Nieto.
27th May 2021.
Pre-training Reinforcement Learning (RL) agents in a task-agnostic manner has shown promising results. However, previous works still struggle to learn and discover meaningful skills in high-dimensional state-spaces. We approach the problem by leveraging unsupervised skill discovery and self-supervised learning of state representations. In our work, we learn a compact latent representation by making use of variational or contrastive techniques. We demonstrate that both allow learning a set of basic navigation skills by maximizing an information theoretic objective. We assess our method in Minecraft 3D maps with different complexities. Our results show that representations and conditioned policies learned from pixels are enough for toy examples, but do not scale to realistic and complex maps. We also explore alternative rewards and input observations to overcome these limitations.
https://imatge.upc.edu/web/publications/discovery-and-learning-navigation-goals-pixels-minecraft
Peter Muschick MSc thesis
Universitat Pollitecnica de Catalunya, 2020
Sign language recognition and translation has been an active research field in the recent years with most approaches using deep neural networks to extract information from sign language data. This work investigates the mostly disregarded approach of using human keypoint estimation from image and video data with OpenPose in combination with transformer network architecture. Firstly, it was shown that it is possible to recognize individual signs (4.5% word error rate (WER)). Continuous sign language recognition though was more error prone (77.3% WER) and sign language translation was not possible using the proposed methods, which might be due to low accuracy scores of human keypoint estimation by OpenPose and accompanying loss of information or insufficient capacities of the used transformer model. Results may improve with the use of datasets containing higher repetition rates of individual signs or focusing more precisely on keypoint extraction of hands.
https://github.com/telecombcn-dl/lectures-all/
These slides review techniques for interpreting the behavior of deep neural networks. The talk reviews basic techniques such as the display of filters and tensors, as well as more advanced ones that try to interpret which part of the input data is responsible for the predictions, or generate data that maximizes the activation of certain neurons.
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
https://telecombcn-dl.github.io/dlai-2020/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
https://telecombcn-dl.github.io/drl-2020/
This course presents the principles of reinforcement learning as an artificial intelligence tool based on the interaction of the machine with its environment, with applications to control tasks (eg. robotics, autonomous driving) o decision making (eg. resource optimization in wireless communication networks). It also advances in the development of deep neural networks trained with little or no supervision, both for discriminative and generative tasks, with special attention on multimedia applications (vision, language and speech).
Giro-i-Nieto, X. One Perceptron to Rule Them All: Language, Vision, Audio and Speech. In Proceedings of the 2020 International Conference on Multimedia Retrieval (pp. 7-8).
Tutorial page:
https://imatge.upc.edu/web/publications/one-perceptron-rule-them-all-language-vision-audio-and-speech-tutorial
Deep neural networks have boosted the convergence of multimedia data analytics in a unified framework shared by practitioners in natural language, vision and speech. Image captioning, lip reading or video sonorization are some of the first applications of a new and exciting field of research exploiting the generalization properties of deep neural representation. This tutorial will firstly review the basic neural architectures to encode and decode vision, text and audio, to later review the those models that have successfully translated information across modalities.
Image segmentation is a classic computer vision task that aims at labeling pixels with semantic classes. These slides provide an overview of the basic approaches applied from the deep learning field to tackle this challenge and presents the basic subtasks (semantic, instance and panoptic segmentation) and related datasets.
Presented at the International Summer School on Deep Learning (ISSonDL) 2020 held online and organized by the University of Gdansk (Poland) between the 30th August and 2nd September.
http://2020.dl-lab.eu/virtual-summer-school-on-deep-learning/
https://imatge-upc.github.io/rvos-mots/
Video object segmentation can be understood as a sequence-to-sequence task that can benefit from the curriculum learning strategies for better and faster training of deep neural networks. This work explores different schedule sampling and frame skipping variations to significantly improve the performance of a recurrent architecture. Our results on the car class of the KITTI-MOTS challenge indicate that, surprisingly, an inverse schedule sampling is a better option than a classic forward one. Also, that a progressive skipping of frames during training is beneficial, but only when training with the ground truth masks instead of the predicted ones.
Deep neural networks have achieved outstanding results in various applications such as vision, language, audio, speech, or reinforcement learning. These powerful function approximators typically require large amounts of data to be trained, which poses a challenge in the usual case where little labeled data is available. During the last year, multiple solutions have been proposed to leverage this problem, based on the concept of self-supervised learning, which can be understood as a specific case of unsupervised learning. This talk will cover its basic principles and provide examples in the field of multimedia.
More from Universitat Politècnica de Catalunya (20)
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
National Security Agency - NSA mobile device best practices
Eva Mohedano, "Investigating EEG for Saliency and Segmentation Applications in Image Processing"
1. Investigating EEG for Saliency and
Segmentation Applications in Image
Processing
Eva Mohedano
1
2. CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions
2
3. CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions
3
4. 1- PROBLEM STATEMENT
Design a system based on a Brain Computer Interface (BCI) wich measure
Electroencephalography (EEG) signals to answer the following questions:
1 - Are the EEG signals to compute Saliency Maps?
2 - Are the EEG signals useful to images segmentation?
EEG Signals
Visual stimulus
BCI
Data Processing
4
5. 1- PROBLEM STATEMENT
Design a system based on a Brain Computer Interface (BCI) wich measure
Electroencephalography (EEG) signals to answer the following questions:
1 - Are the EEG signals to compute Saliency Maps?
2 - Are the EEG signals useful to images segmentation?
Visual stimulus
BCI
Data Processing
5
6. 1- PROBLEM STATEMENT
Design a system based on a Brain Computer Interface (BCI) wich measure
Electroencephalography (EEG) signals to answer the following questions:
1 - Are the EEG signals to compute Saliency Maps?
2 - Are the EEG signals useful to images segmentation?
Visual stimulus
BCI
Data Processing
6
7. 1- PROBLEM STATEMENT
1 - Are the EEG signals to compute Saliency Maps?
Motivation:
- New way to compute maps of the atention of the image
based directly in the reaction of the brain and not in the
features of the images (Niebur and Koch (1996) algorithm).
Visual stimulus
BCI
Data Processing
7
8. 1- PROBLEM STATEMENT
2 - Are the EEG signals useful to images segmentation?
Motivation:
- Reduce the user interaction to the minimun expression.
- Measure the brain reaction at local scale of the image.
Visual stimulus
BCI
Data Processing
8
9. CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions
9
10. 2- RELATED WORK
2.1 – BCI in image processing applications
The oddball paradigm
10
11. 2- RELATED WORK
2.1 – BCI in image processing applications
The oddball paradigm
P300
11
12. 2- RELATED WORK
2.1 – BCI in image processing applications
The oddball paradigm
P300
• Speed rate around 10Hz
• Usually experiements centered to
find target images not target regions
12
13. 2- RELATED WORK
Event-Related Potential
•8 electrodes placed mainly in the
posterior points on the scalp.
• Which is consistent with the
discriminating
activity
typically
produced by a P300 ERP.
[Optimising the Number of Channels in
EEG-Augmented Image Search. Graham
Healy]
13
14. 2- RELATED WORK
Event-Related Potential
•8 electrodes placed mainly in
How to present the scalp. to the
image
posterior points on the
generate and is consistent signal?
• Which detect this with the
discriminating
activity
produced by a P300 ERP.
typically
[Optimising the Number of Channels in
EEG-Augmented Image Search. Graham
Healy]
14
15. CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions
15
16. 3- LOCAL EXPLORATION OF THE IMAGE
First Design: Sliding Window
http://www.youtube.com/watch?v=bKTGKVx58Ps
16
17. 3- LOCAL EXPLORATION OF THE IMAGE
CHALLENGE 1
• Eyes movement affect to the EEG signals – Introduce Artifacts to the signal
Opened eyes / Closed eyes. Image from the slides Dr. Ranjith Polusani
17
18. 3- LOCAL EXPLORATION OF THE IMAGE
CHALLENGE 2
• Progressive inspection may not generate a useful reaction in the EEG waves.
Suggestions meeting Thomas Ward and Nima Bidgely Shamlo :
- Follow Oddball Paradigm and perform RSVP od the windows
P300
SNAP - Simulation and Neuroscience
Application Platform
18
20. 3- LOCAL EXPLORATION OF THE IMAGE
CHALLENGE 4
•Size of the object / window
Grabcut Dataset – Objects of different size
Suggestions meeting Thomas Ward and Nima Bidgely Shamlo :
To use images with an homogeneus background with a salient object.
The number of distractors (windows with background) must be higher
than the number of targets (windows with object).
20
23. 3- LOCAL EXPLORATION OF THE IMAGE
CHALLENGES
SOLUTIONS
1 - Eyes movement
Display a fixed window on the screen
2 - Progressive inspection
SNAP to perform a random RSVP
3 - Syncronitzation Problem
First test with flashes to find ERPS
4 - Size of the object / window
Generate my own dataset
5 - What am I seeing?
• Is it just noise?
• Am I able to detect something?
Real time visualitzation of the signal
23
24. CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions
24
26. 4- EXPERIMENTAL SET-UP
a) Device Calibration
?
Is the device well
connected?
Is the syncronitzation
method correct ?
Am I able to detect
something?
26
27. 4- EXPERIMENTAL SET-UP
a) Device Calibration
SIGNAL EXPECTED - Closed eyes – Alpha waves (8-12 Hz)
Closed-eye EEG alpha waves (10-20 channels Pz-Top, Fz-Bottom) extracted from http://blog.grahamhealy.com/
27
29. 4- EXPERIMENTAL SET-UP
a) Device Calibration
Finding ERPS response after a white flash
SIGNAL EXPECTED: After the flash P100 and a negative peak between 150-200ms
Presenting a serie of white flashes (2 seconds between the flashes)
29
30. 4- EXPERIMENTAL SET-UP
a) Device Calibration
Finding ERPS response after a white flash
SIGNAL OBTAINED: 60 Flashes to get the response.
Averaged ERP waveform per channel
Channel
P100 (ms)
1
130
320
2
90
210
3
90
210
4
10
220
5
110
220
6
90
210
7
10
220
8
100
22
Mean
10
N1 (ms)
80
23
Amplitude (uV)
5
0
-5
-10
-15
0
100
200
300
400
500
600
Time (ms)
700
800
900
1000
30
31. CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions
31
32. 5- SIGNAL PROCESSING OF EEG SIGNALS
Data adquisition
100 windows per Image
1 Target
99 Distractors
32
33. 5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Single trial
One Image
1
2
3
4
5
6
7
8
33
34. 5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Single trial - PROBLEM
- Signal very noisy
- Single Targets and Single
Distractors very similar
34
35. 5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Feature
96 Distractors
Mean Absolute Amplitude
looks different
96 Targets
Energy from 0 to
600ms
Feature for the window
presented
35
36. 5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Averaged trials
1 averaged target
99 averaged
distractors
Energy from
0-600 ms
36
37. 5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Averaged trials
SINGLE
AVERAGED
1 x 32 single repeats
1 averaged target
99 x 32 single
repeats
99 averaged
distractors
1 x 32 window repeats
1 averaged target
99 x 32 single
repeats
Average
by 32
99 averaged
distractors
1 x 32 window repeats
1 averaged target
99 x 32 single
repeats
99 averaged
distractors
37
38. 5- SIGNAL PROCESSING OF EEG SIGNALS
Preprocessing: Averaged trials
1 averaged target
Problem:
Too few (target) samples
for training
99 averaged
distractors
1 averaged target
99 averaged
distractors
1 averaged target
99 averaged
distractors
38
39. 5- SIGNAL PROCESSING OF EEG SIGNALS
Bootstrapping
SINGLE
AVERAGED
1 x 32 single repeats
1 x 32 averaged target
99 x 32 single
repeats
99 x 1
averaged distractors
1 x 32 window repeats
99 x 32 single
repeats
1 x 32 window repeats
99 x 32 single
repeats
1 x 32 averaged target
Bootstrapping
99 x 1
averaged distractors
1 x 32 averaged target
99 x 1
averaged distractors39
40. 5- SIGNAL PROCESSING OF EEG SIGNALS
Classification
Problem:
Unbalanced dataset for
binary classification
AVERAGED
1 x 32 averaged target
99 x 1
averaged distractors
1 x 32 averaged target
99 x 1
averaged distractors
1 x 32 averaged target
99 x 1
averaged distractors40
41. 5- SIGNAL PROCESSING OF EEG SIGNALS
Classification
AVERAGED
AVERAGED
1 x 32 averaged targets
99 x 1
averaged distractors
1 x 32 averaged targets
Subsample
1 x 32 averaged targets
99 x 1
averaged distractors
1 x 32 averaged targets
Subsample
1 x 32 averaged targets
99 x 1
averaged distractors
32 x 1 avgd distractors
32 x 1 avgd distractors
1 x 32 averaged targets
Subsample
32 x 1 avgd distractors
41
42. 5- SIGNAL PROCESSING OF EEG SIGNALS
Classification
AVERAGED
HISTOGRAM
1 x 32 averaged targets
32 x 1 avgd distractors
1 x 32 averaged targets
32 x 1 avgd distractors
1 x 32 averaged targets
32 x 1 avgd distractors
42
43. 5- SIGNAL PROCESSING OF EEG SIGNALS
Classification
AVERAGED
1 x 32 averaged targets
32 x 1 avgd distractors
1 x 32 averaged targets
32 x 1 avgd distractors
SVMTRAIN
(linear
kernel)
Classifier
1 x 32 averaged targets
32 x 1 avgd distractors
43
44. 5- SIGNAL PROCESSING OF EEG SIGNALS
Classification
100 x 32 single repeats
Average
by 32
100 avgd windows
44
45. 5- SIGNAL PROCESSING OF EEG SIGNALS
Classification
100 x 32 single repeats
Average
by 32
100 avgd samples
Classifier
SVM
PREDICT
45
46. 5- SIGNAL PROCESSING OF EEG SIGNALS
8 samples feature vectors
Cross validation approach 3 train + 1 test
47. CONTENT
1 - Problem statement
2 – Related Work
3 – Local exploration of the image
4 – Experimental set-up
5 – Signal Processing of EEG Signals
6 – Conclusions
47
48. 6- CONCLUSIONS
- Results from sythetic images provide and evidence that BCI devices could be
used to located an object into an image.
- Simplicity of the system: Energy value from 8 channels to train SVM with
lineal kernel.
Future work
-Study the impact of the number of repetitions.
- Extract better features.
-Analize data from real images.
-Tool to evaluate and compare the EEG mask (ROC, Jaccard index)
48