MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop

•

0 likes•147 views

This document discusses different approaches to evaluating information retrieval (IR) systems, including user studies, in-situ evaluation, A/B testing, interleaving, and collection-based evaluation using test collections like Cranfield. It describes the TREC Session Track, which aimed to improve search over an entire user session, and the TREC Tasks Track, which focuses on understanding and assisting users with their underlying tasks. The CLEF Dynamic Search for Complex Tasks task also examines simulating users to dynamically build test collections and understand good document rankings and sessions for complex tasks.

Science

IR
evaluation:
Putting
the

user
back
in
the
loop
Evangelos Kanoulas
e.kanoulas@uva.nl

Change
the
search
algorithm.
How
can
we
know
whether
we
made

the
users
happier?

Different
approaches

to
evaluation
• User-‐studies
• In-‐situ
evaluation
• A/B
Testing
• Interleaving
• Collection-‐based

evaluation

A/B
Testing
Baseline
(control) Experimental
(treatment)

Machine
Learning
• Feature
vectors
• Labels
Cranfield Collections
Information
Retrieval
• Documents
• Queries
• Labels
– relevance

judgments
Query
1 Query
2 Query
N

Cranfield Paradigm
• Simple
user
model
• Controlled
experiments
• Reusable
but
static
test

collections
Online
Evaluation
• Full
user
participation
• Many
degrees
of
freedom
• Unrepeatable
experiments
System
Focus User
Focus
Evaluation
Landscape
TREC
Tasks TREC
Session

TREC
Total
Recall

TREC
Open
Search

TREC
Total
Recall
results
human assessor
search algorithm
query
document
collection

TREC
Session
Track
[2010-‐2014]
1. improve
search
by
using
session
information
2. improve
search
over
an
entire
user’s
session

instead
of
a
single
query

Test
Collection
Û the set of clicked URLs/snippets.
Test Collection Statistics
2011 2012 2013 2014
collection ClueWeb09 ClueWeb09 ClueWeb12 ClueWeb12
topic properties
topic set size 62 48 61 60
topic cat. dist. known-item 10 exploratory,
6 interpretive,
20 known-item,
12 known-subj
10 exploratory,
9 interpretive,
32 known-item,
10 known-subj
15 exploratory,
15 interpretive,
15 known-item,
15 known-subj
session properties
user population U. She eld U. She eld U. She eld + IR
researchers
MTurk
search engine BOSS+CW09
ﬁlter
BOSS+CW09
ﬁlter
indri indri
total sessions 76 98 133 1,257
sessions per topic 1.2 2.0 2.2 21.0
mean length (in queries) 3.7 3.0 3.7 3.7
median time between queries 68.5s 66.7s 72.2s 25.6s
relevance judgments
topics judged 62 48 49 51
total judgments 19,413 17,861 13,132 16,949

TREC
Tasks
Track
[2015–now]
1. understand
underlying
user’s
task
2. assist
user
in
completing
the
task

Make Improvements At Home
TASK
UNDERSTANDING

Make Improvements At Home
TASK
COMPLETION

CLEF
Complex
Tasks
[now]
1. Produce
methodology
and
algorithms
that
will

lead
to
a
dynamic
test
collection by
simulating

users
2. Understand
and
quantify
what
constitutes
a
good

ranking
of
documents
at
different
stages of
a

session,
and
a
good
overall session

MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop

Presenter: Samuel G. Fadel UNIFESP at MediaEval 2016: Predicting Media Interestingness Task In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Jurandy Almeida Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_28.pdf Video: https://youtu.be/YLthKNczlcA Abstract: This paper describes the approach proposed by UNIFESP for the MediaEval 2016 Predicting Media Interestingness Task and for its video subtask only. The proposed approach is based on combining learning-to-rank algorithms for predicting the interestingness of videos by their visual content.

MediaEval 2016 - HUCVL Predicting Interesting Key Frames with Deep Models

multimediaeval

Presenter: Göksu Erdoğan HUCVL at MediaEval 2016: Predicting Interesting Key Frames with Deep Models In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Goksu Erdogan, Aykut Erdem, Erkut Erdem Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_18.pdf Video: https://youtu.be/A--6O2v81Cw Abstract: In MediaEval 2016, we focus on the image interestingness subtask which involves predicting interesting key frames of a video in the form of a movie trailer. We specifically propose three different deep models for this subtask. The first two models are based on fine-tuning two pretrained models, namely AlexNet and MemNet, where we cast the interestingness prediction as a regression problem. Our third deep model, on the other hand, depends on a triplet network which is comprised of three instances of the same feedforward network with shared weights, and trained according to a triplet ranking loss. Our experiments demonstrate that all these models provide relatively similar and promising results on the image interestingness subtask.

MediaEval 2016 - MLPBOON Predicting Media Interestingness System

multimediaeval

Presenter: Jayneel Parekh The MLPBOON Predicting Media Interestingness System for MediaEval 2016 In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Jayneel Parekh, Sanjeel Parekh Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_25.pdf Video: https://youtu.be/nAnrdYiy7nc Abstract: This paper describes the system developed by team MLPBOON for MediaEval 2016 Predicting Media Interestingness Image Subtask. After experimenting with various features and classifiers on the development dataset, our final system involves use of CNN features (fc7 layer of AlexNet) for the input representation and logistic regression as the classifier. For the proposed method, the MAP for the best run reaches a value of 0.229.

MediaEval 2016 - ININ Submission to Zero Cost ASR Task

multimediaeval

Presenter: Tejas Godambe ININ Submission to Zero Cost ASR Task at MediaEval 2016 In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Tejas Godambe, Naresh Kumar, Pavan Kumar, Veera Raghavendra, Aravind Ganapathiraju Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_31.pdf Video: https://youtu.be/e70xRjsUUts Abstract: This paper details the experiments conducted to train an as good performing Vietnamese speech recognition system as possible using public domain data only, as a part of the Zero Cost task at MediEval 2016. We explored techniques related to audio preprocessing, use of speaker’s pitch information, data perturbation, for building subspace Gaussian mixture acoustic model which is known for estimating robust parameters when the amount of data is less, and also unsupervised adaptation, RNN language model based lattice rescoring and system combination using ROVER technique.

Comparison of Fine-tuning and Extension Strategies for Deep Convolutional Neu...

InVID Project

MediaEval 2016 - BUT Zero-Cost Speech Recognition

multimediaeval

Presenter: Miroslav Skácel BUT Zero-Cost Speech Recognition 2016 System Description In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Miroslav Skácel, Martin Karafiát, Lucas Ondel, Albert Uchytil, Igor Szöke Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_48.pdf Video: https://youtu.be/0pNiLLVTa28 Abstract: This paper describes our work on developing speech recognizers for Vietnamese. It focuses on procedures to prepare provided data precisely. We aim on analysis of the textual transcriptions in particular. Methods to filter out defective data to improve performance of final system are proposed and described in detail. We also propose cleaning of other textual data used for language modeling. Several architectures are investigated to reach both sub-tasks goals. The achieved results are discussed.

Long-term Face Tracking in the Wild using Deep Learning

Elaheh Rashedi

This paper investigates long-term face tracking of a specific person given his/her face image in a single frame as a query in a video stream. Through taking advantage of pre-trained deep learning models on big data, a novel system is developed for accurate video face tracking in the unconstrained environments depicting various people and objects moving in and out of the frame. In the proposed system, we present a detection-verification-tracking method (dubbed as 'DVT') which accomplishes the long-term face tracking task through the collaboration of face detection, face verification, and (short-term) face tracking. An offline trained detector based on cascaded convolutional neural networks localizes all faces appeared in the frames, and an offline trained face verifier based on deep convolutional neural networks and similarity metric learning decides if any face or which face corresponds to the queried person. An online trained tracker follows the face from frame to frame. When validated on a sitcom episode and a TV show, the DVT method outperforms tracking-learning-detection (TLD) and face-TLD in terms of recall and precision. The proposed system is also tested on many other types of videos and shows very promising results.

This paper discusses challenges in contextual task analysis and the need of tools that support analysts to collect such information in context. Specifically we argue that the analysis of collaborative and distributed tasks can be supported by ambulatory assessment tools. We illustrate how contextual task analysis can be supported by TEMPEST, a platform originally created for experience sampling and more generally, longitudinal ambulatory assessment studies. We present a case study that illustrates the extent to which this tool meets the needs of real-world task analysis, describing the gains in efficiency it can provide but also directions for the development of tool support for task analysis.

Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test ...

Sebastiano Panichella

Introduction to Model-Based Machine Learning

Daniel Emaasit

The field of machine learning has seen the development of thousands of learning algorithms. Typically, scientists choose from these algorithms to solve specific problems. Their choices often being limited by their familiarity with these algorithms. In this classical/traditional framework of machine learning, scientists are constrained to making some assumptions so as to use an existing algorithm. This is in contrast to the model-based machine learning approach which seeks to create a bespoke solution tailored to each new problem.

Backbone can not be trained at once rolling back to pre trained network for p...

NAVER Engineering

Step zhedong

哲东郑

Developing Computational Skills in the Sciences with Matlab Webinar 2017SERC at Carleton College

Comparison of papers NN-filter

saman shaheen

Difference Between filter based method and feature selection: Dataset selection can lead to better performance for cross project defect prediction(CPDP). On the other hand, feature selection and data quality are issues to consider in CPDP. With the availability of thehuge amount of data that can be obtained from mining software historical repositories, it becomes possible to have some features (metrics) that are not correlated with the faults, which consequently mislead the learning algorithm and thus decrease its performance. We aim at utilizing the Nearest Neighbor (NN)-Filter, embedded in genetic algorithm to produce validation sets for generating evolving training datasets to tackle CPDP while accounting for potential noise in defect labels. We also investigate the impact of using different feature sets. A novel FS approach is proposed to enhance the performance of a layered recurrent neural network (L-RNN), which is used as a classification technique for the SFP problem.

MediaEval 2015 - RFA at MediaEval 2015 Affective Impact of Movies Task: A Mul...

multimediaeval

The MediaEval 2015 Affective Impact of Movies Task challenged participants to automatically find violent scenes in a set of videos and, also, to predict the affective impact that video content will have on viewers. We propose the use of several multimodal descriptors, such as visual, motion and auditory features, then we fuse their predictions to detect the violent or affective content. Our best-performing run with regard to the offcial metric received a MAP of 0.1419 in the violence detection task, and an accuracy of 45.038% for the arousal estimation and 36.123% for the valence estimation. http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

Multimedia Answer Generation for Community Question Answering

SWAMI06

Community question answering (cQA) services have gained popularity over the past years. It not only allows community members to post and answer questions but also enables general users to seek information froma comprehensive set of well-answered questions. However, existing cQA forums usually provide only textual answers, which are not informative enough for many questions. In this paper, we propose a scheme that is able to enrich textual answers in cQA with appropriate media data. Our scheme consists of three components: answer medium selection, query generation for multimedia search, and multimedia data selection and presentation. This approach automatically determines which type of media information should be added for a textual answer. It then automatically collects data from the web to enrich the answer. By processing a large set of QA pairs and adding them to a pool, our approach can enable a novel multimedia question answering (MMQA) approach as users can find multimedia answers by matching their questions with those in the pool. Different from a lot ofMMQAresearch efforts that attempt to directly answer questions with image and video data, our approach is built based on community-contributed textual answers and thus it is able to deal with more complex questions.We have conducted extensive experiments on a multi-source QA dataset. The results demonstrate the effectiveness of our approach.

Object Detection and Recognition

Intel Nervana

Base Calling Error Toleration in Reference Base Assembly

Hadi Gharibi

The neural tangent link between CNN denoisers and non-local filters

Julián Tachella

MediaEval 2016 - Emotion in Music Task: Lessons Learned

multimediaeval

Video Retrieval for Multimedia Verification of Breaking News on Social Networks

InVID Project

This slideset presents an approach to automatically detecting breaking news events from social media streams, using event detection to collecting near real time relevant video documents from social networks regarding that breaking news. A visual analytics dashboard provides access to the results of the content processing pipeline, providing a rich interactive interface to explore emerging stories and select video material around those stories for verification.

MediaEval 2016: A Multimodal System for the Verifying Multimedia Use Task

multimediaeval

Presenter: Maigrot Cédric MediaEval 2016: A Multimodal System for the Verifying Multimedia Use Task In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Cédric Maigrot, Vincent Claveau, Ewa Kijak, Ronan Sicre Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_45.pdf Video: https://youtu.be/ay1zWydnijY Abstract: This paper presents a multi-modal hoax detection system composed of text, source, and image analysis. As hoax can be very diverse, we want to analyze several modalities to better detect them. This system is applied in the context of the Verifying Multimedia Use task of MediaEval 2016. Experiments show the performance of each separated modality as well as their combination.

MediaEval 2016 - LAPI @ 2016 Retrieving Diverse Social Images Task: A Pseudo-...

multimediaeval

Presenter: Bogdan Boteanu, LAPI @ 2016 Retrieving Diverse Social Images Task: A Pseudo-Relevance Feedback Diversification Perspective In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Bogdan Boteanu, Mihai G. Constantin, Bogdan Ionescu Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_20.pdf Video: https://youtu.be/mDI8Z31p7TY Abstract: In this paper we present the results achieved during the 2016 MediaEval Retrieving Diverse Social Images Task, using an approach based on pseudo-relevance feedback, in which human feedback is replaced by an automatic selection of images. The proposed approach is designed to have in priority the diversification of the results, in contrast to most of the existing techniques that address only the relevance. Diversification is achieved by exploiting a hierarchical clustering scheme followed by a diversification strategy. Methods are tested on the benchmarking data and results are analyzed. Insights for future work conclude the paper.

MediaEval 2015 - Verifying Multimedia Use at MediaEval 2015

multimediaeval

This paper provides an overview of the Verifying Multimedia Use task that takes places as part of the 2015 MediaEval Benchmark. The task deals with the automatic detection of manipulation and misuse of Web multimedia content. Its aim is to lay the basis for a future generation of tools that could assist media professionals in the process of verification. Examples of manipulation include maliciously tampering with images and videos, e.g., splicing, removal/addition of elements, while other kinds of misuse include the reposting of previously captured multimedia content in a different context (e.g., a new event) claiming that it was captured there. For the 2015 edition of the task, we have generated and made available a large corpus of real-world cases of images that were distributed through tweets, along with manually assigned labels regarding their use, i.e. misleading (fake) versus appropriate (real). http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

MediaEval 2016 - TUD-MMC Predicting media Interestingness Task

multimediaeval

Presenter: Cynthia Liem TUD-MMC at MediaEval 2016: Predicting Media Interestingness Task In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Cynthia Liem Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_30.pdf Video: https://youtu.be/NQan10E_-kE Abstract: This working notes paper describes the TUD-MMC entry to the MediaEval 2016 Predicting Media Interestingness Task. Noting that the nature of movie trailer shots is different from that of preceding tasks on image and video interestingness, we propose two baseline heuristic approaches based on the clear occurrence of people. MAP scores obtained on the development set and test set suggest that our approaches cover a limited but non-marginal subset of the interestingness spectrum. Most strikingly, our obtained scores on the Image and Video Subtasks are comparable or better than those obtained when evaluating the ground truth annotations of the Image Subtask against the Video Subtask and vice versa

MediaEval 2016 - Placing Images with Refined Language Models and Similarity S...

multimediaeval

Presenter: Giorgos Kordopatis-Zilos Placing Images with Refined Language Models and Similarity Search with PCA-reduced VGG Features In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Giorgos Kordopatis-Zilos, Adrian Popescu, Symeon Papadopoulos, Yiannis Kompatsiaris Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_13.pdf Video: https://youtu.be/WR4I3CWjcR4 Abstract: We describe the participation of the CERTH/CEA-LIST team in the MediaEval 2016 Placing Task. We submitted five runs to the estimation-based sub-task: one based only on text by employing a Language Model-based approach with several refinements, one based on visual content, using geospatial clustering over the most visually similar images, and three based on a hybrid scheme exploiting both visual and textual cues from the multimedia items, trained on datasets of different size and origin. The best results were obtained by a hybrid approach trained with external training data and using two publicly available gazetteers.

What's hot

ELLA LC algorithm presentation in ICIP 2016

InVID Project

Audio augmentation

Tomoya Koike

Infusing Digital Technologies for an Engineering LaboratoryAlex See

Towards Task Analysis Tool Support

Suzanne Kieffer

Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test ...

Sebastiano Panichella

Introduction to Model-Based Machine Learning

Daniel Emaasit

Backbone can not be trained at once rolling back to pre trained network for p...

NAVER Engineering

Step zhedong

哲东郑

Developing Computational Skills in the Sciences with Matlab Webinar 2017SERC at Carleton College

Comparison of papers NN-filter

saman shaheen

MediaEval 2015 - RFA at MediaEval 2015 Affective Impact of Movies Task: A Mul...

multimediaeval

Multimedia Answer Generation for Community Question Answering

SWAMI06

Object Detection and Recognition

Intel Nervana

Base Calling Error Toleration in Reference Base Assembly

Hadi Gharibi

The neural tangent link between CNN denoisers and non-local filters

Julián Tachella

What's hot (15)

ELLA LC algorithm presentation in ICIP 2016

Audio augmentation

Infusing Digital Technologies for an Engineering Laboratory

Towards Task Analysis Tool Support

Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test ...

Introduction to Model-Based Machine Learning

Backbone can not be trained at once rolling back to pre trained network for p...

Step zhedong

Developing Computational Skills in the Sciences with Matlab Webinar 2017

Comparison of papers NN-filter

MediaEval 2015 - RFA at MediaEval 2015 Affective Impact of Movies Task: A Mul...

Multimedia Answer Generation for Community Question Answering

Object Detection and Recognition

Base Calling Error Toleration in Reference Base Assembly

The neural tangent link between CNN denoisers and non-local filters

Viewers also liked

MediaEval 2016 - Emotion in Music Task: Lessons Learned

multimediaeval

Video Retrieval for Multimedia Verification of Breaking News on Social Networks

InVID Project

MediaEval 2016: A Multimodal System for the Verifying Multimedia Use Task

multimediaeval

MediaEval 2016 - LAPI @ 2016 Retrieving Diverse Social Images Task: A Pseudo-...

multimediaeval

MediaEval 2015 - Verifying Multimedia Use at MediaEval 2015

multimediaeval

MediaEval 2016 - TUD-MMC Predicting media Interestingness Task

multimediaeval

MediaEval 2016 - Placing Images with Refined Language Models and Similarity S...

multimediaeval

MediaEval 2015 - GTM-UVigo Systems for Person Discovery Task at MediaEval 2015

multimediaeval

In this paper, we present the systems developed by GTMUVigo team for the Multimedia Person Discovery in Broadcast TV task at MediaEval 2015. The systems propose two different strategies for person discovery in audio through speaker diarization (one based on an online clustering strategy with error correction using OCR information and the other based on agglomerative hierarchical clustering) as well as intrashot and intershot trategies for face clustering. http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

MediaEval 2015 - JRS at Synchronization of Multi-user Event Media Task

multimediaeval

The event synchronisation task addresses the problem of aligning media (i.e., photo and video) streams (“galleries”) from different users temporally and identifying coherent events in the streams. Our approach uses the visual similarity of image/key frame pairs based on full matching of SIFT descriptors with geometric verification. Based on the visual similarity and the given time information, a probabilistic algorithm is employed, where in each run a hypothesis is calculated for the set of time offsets with respect to the reference gallery. From the gathered hypotheses, the final set of time offsets is calculated as the medoid of all hypotheses. http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

MediaEval 2015 - CERTH at MediaEval 2015 Synchronization of Multi-User Event ...

multimediaeval

This paper describes the results of our participation to the Synchronization of Multi-User Event Media Task at the MediaEval 2015 challenge. Using multiple similarity measures, we identify pairs of similar media from different galleries. We use a graph-based approach to temporally synchronize user galleries; subsequently we use time information, geolocation information and visual concept detection results to cluster all photos into different sub-events. Our method achieves good accuracy on considerably diverse datasets. http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

Media REVEALr: A social multimedia monitoring and intelligence system for Web...

Symeon Papadopoulos

MediaEval 2015 - Synchronization of Multi-User Event Media at MediaEval 2015:...

multimediaeval

The objective of this paper is to provide an overview of the Synchronization of Multi-User Event Media (SEM) Task, which is part of the MediaEval Benchmark for Multimedia Evaluation. The SEM task was initially presented at MediaEval in 2014, with the goal of proposing a challenge in aligning multiple users’ photo galleries related to the same event but with unreliable timestamps. Besides aligning the pictures on a common timeline, participants were also required to detect the sub-events and cluster the pictures accordingly. For 2015 we have decided to extend the task also to other types of media, thus including audio and video information for a more complete and diversified representation of the analyzed event. http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

MediaEval 2016 - Simula Team @ Context of Experience Task

multimediaeval

Presenter: Konstantin Pogorelov Simula @ MediaEval 2016 Context of Experience Task In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Konstantin Pogorelov, Michael Riegler, Pål Halvorsen, Carsten Griwodz Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_53.pdf Video: https://youtu.be/FTIeGpHhURU Abstract: This paper presents our approach for the Context of Multimedia Experience Task of the MediaEval 2016 Benchmark. We present different analyses of the given data using different subsets of data sources and combinations of it. Our approach gives a baseline evaluation indicating that metadata approaches work well but that also visual features can provide useful information for the given problem to solve.

MediaEval 2016: LAPI at Predicting Media Interestingness Task

multimediaeval

Presenter: Mihai Gabriel Constantin LAPI at MediaEval 2016 Predicting Media Interestingness Task In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Mihai G. Constantin, Bogdan Boteanu, Bogdan Ionescu Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_23.pdf Video: https://youtu.be/4VKeMMeroG0 Abstract: This paper will present our results for the MediaEval 2016 Predicting Media Interestingness task. We proposed an approach based on video descriptors and studied several machine learning models, in order to detect the optimal configuration and combination for the descriptors and algorithms that compose our system.

The InVID Plug-in: Web Video Verification on the Browser

InVID Project

MediaEval 2016 - Verifying Multimedia Use Task Overview

multimediaeval

Presenters: Stuart E. Middleton and Christina Boididou Verifying Multimedia Use at MediaEval 2016 In Working Notes Proceedings of the MediaEval 2016 Workshop, Hilversum, Netherlands, October 20-21, CEUR-WS.org (2016) by Christina Boididou, Symeon Papadopoulos, Duc-Tien Dang-Nguyen, Giulia Boato, Michael Riegler, Stuart E. Middleton, Andreas Petlund, and Yiannis Kompatsiaris Paper: http://ceur-ws.org/Vol-1739/MediaEval_2016_paper_3.pdf Video: https://youtu.be/2Jx6OliFR-0 Abstract: This paper provides an overview of the Verifying Multimedia Use task that takes places as part of the 2016 MediaEval Benchmark. The task motivates the development of automated techniques for detecting manipulated and misleading use of web multimedia content. Splicing, tampering and reposting videos and images are examples of manipulation that are part of the task definition. For the 2016 edition of the task, a corpus of images/videos and their associated posts is made available, together with labels indicating the appearance of misuse (fake) or not (real) in each case as well as some useful post metadata.

Viewers also liked (16)

MediaEval 2016 - Emotion in Music Task: Lessons Learned

Video Retrieval for Multimedia Verification of Breaking News on Social Networks

MediaEval 2016: A Multimodal System for the Verifying Multimedia Use Task

MediaEval 2016 - LAPI @ 2016 Retrieving Diverse Social Images Task: A Pseudo-...

MediaEval 2015 - Verifying Multimedia Use at MediaEval 2015

MediaEval 2016 - TUD-MMC Predicting media Interestingness Task

MediaEval 2016 - Placing Images with Refined Language Models and Similarity S...

MediaEval 2015 - GTM-UVigo Systems for Person Discovery Task at MediaEval 2015

MediaEval 2015 - JRS at Synchronization of Multi-user Event Media Task

MediaEval 2015 - CERTH at MediaEval 2015 Synchronization of Multi-User Event ...

Media REVEALr: A social multimedia monitoring and intelligence system for Web...

MediaEval 2015 - Synchronization of Multi-User Event Media at MediaEval 2015:...

MediaEval 2016 - Simula Team @ Context of Experience Task

MediaEval 2016: LAPI at Predicting Media Interestingness Task

The InVID Plug-in: Web Video Verification on the Browser

MediaEval 2016 - Verifying Multimedia Use Task Overview

Similar to MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop

From Exploration to Construction  - How to Support the Complex Dynamics of In...

TimelessFuture

Search engines on the Web provide a world of information at our fingertips, and the answers to many of our common questions are just one click away. However, for the complex and multifaceted tasks involving a process of knowledge construction, various information seeking models describe an intricate set of cognitive stages (Kuhlthau, 2004; Vakkari, 2001). These stages influence the interplay of users’ feelings, thoughts and actions. Despite the evidence of the models, common search engines, nowadays the prime intermediaries between information and user, still feature a streamlined set of 'ten blue links'. While efficient for lookup tasks, this approach may not be beneficial for supporting sustained information-intensive tasks and knowledge construction. Would there be other approaches to support the complex dynamics of these ventures? Based on previous experiments, this talk discusses how the utility of search functionality during different stages of complex tasks is essentially dynamic. This provides opportunities for designing 'stage-aware' search systems, which may evolve along with a user's information journey.

Modelling Time-aware Search Tasks for Search Personalisation

Thanh Vu

Parts 1 & 2: WWW 2018 Tutorial: Understanding User Needs & Tasks

Rishabh Mehrotra

Recommending Sequences RecTour 2017

Gunjan Kumar

UX and Usability Workshop Southampton Solent University

Dr.Mohammed Alhusban

Usability testing through the decades

UX Firm, LLC

Agile2012 presentation miki_konno (aug2012)drewz lin

WSDM 2011 - Nicolaas Matthijs and Filip Radlinski

Nicolaas Matthijs

Usability Testing for Qualitative Researchers - QRCA NYC Chapter event

Kay Aubrey

The goal of this presentation is to give attendees a deeper understanding of usability testing so they can leverage it in their own work. The material will shed light on what is important to the research buyer and will help the research provider to better understand how to plan, moderate, and report on a usability study. It will also provide information on where they can go to learn more about this very practical qualitative method. Kay will cover what a usability test is and when to use it, the key planning steps, the language around it, and the unique insights this method produces. She will also discuss the various approaches a market researcher can take when running a usability study at different points in a product’s development (e.g., concept, early prototype, released product).

UXprobe workshop at Dare Festival 2016

UXprobe

Methodology and Campaign Design for the Evaluation of Semantic Search Tools

Stuart Wrigley

The main problem with the state of the art in the semantic search domain is the lack of comprehensive evaluations. There exist only a few efforts to evaluate semantic search tools and to compare the results with other evaluations of their kind. In this paper, we present a systematic approach for testing and benchmarking semantic search tools that was developed within the SEALS project. Unlike other semantic web evaluations our methodology tests search tools both automatically and interactively with a human user in the loop. This allows us to test not only functional performance measures, such as precision and recall, but also usability issues, such as ease of use and comprehensibility of the query language. The paper describes the evaluation goals and assumptions; the criteria and metrics; the type of experiments we will conduct as well as the datasets required to conduct the evaluation in the context of the SEALS initiative. To our knowledge it is the first effort to present a comprehensive evaluation methodology for Semantic Web search tools.

Gunjan insight student conference v2

Gunjan Kumar

Temporal based Recommendation System

Nurfadhlina Mohd Sharef

Assessment

Jody DeRidder

Developing a digital library doesn’t end when content goes online. You need to know whether what you are doing is effective; whether you’re reaching your users, whether you’re providing them with what they need in the form they need it, and whether you are doing this in the most cost-effective way that you can. This presentation examines the challenges inherent in assessing three different aspects of digital libraries: costs, user needs, and benefits.

Usability Testing Methodsdillarja

Conducting Remote Unmoderated Usability Testing: Part 2

UserZoom

After learning the basics of remote unmoderated usability testing in Part 1, view this webinar on-demand with Ann Rochanayon, Director of UX/CX Research at UserZoom, to learn how usability studies are set up in UserZoom. Ann shows you why UserZoom leads the pack by taking you through the step-by-step study design as well as the research findings. View this 30-min webinar on-demand to learn: -How to set up a usability study in UserZoom -How to build questions -How to build tasks -How to validate tasks

Conducting Remote Unmoderated Usability Testing: Part 1 - RemoteUX Training W...

UserZoom

Remote unmoderated usability testing has become popular and for good reason: it empowers UX Researchers and Designers to conduct more studies with less resources, in less time, with the benefit of having participants in their natural environment. Are you missing out on this opportunity? Join Ann Rochanayon, Director of UX/CX Research at UserZoom, in this webinar on-demand to learn the basics of remote unmoderated usability testing and how to get started. This 30-min webinar on-demand covers: -An Introduction to unmoderated remote usability testing -Defining goals / hypothesis -Determining the tasks -Determining study length -Determining the panel source -General guidelines, types of questions to include, data collection -Sample intro questions, tasks and wrap-up questions

User Centered Design in short

silvana churruca

7. evalution of interactive system

Kh Ravy

Jan Moons at WUD16

UX Antwerp Meetup

World Usability Day 2016 in Antwerp (Belgium), Thursday, November 10th - Jan Moons, UX expert and co-founder at UXprobe "Hands on with Lean and Agile User Testing" Jan Moons shows how to use the latest tools to easily integrate user testing into a lean process. Discover how user testing can be the answer for problems of conversion, usability, and UX quality. In the workshop you will explore all sides of user testing (be the user, be the moderator, be the client) and you will see how lean and agile user testing can be. Jan is the co-founder of UXprobe, company that is focused on a mission of helping companies build great digital products that deliver a fantastic user experience. Jan has almost 20 years of experience as a software engineer and is a certified usability designer.

Similar to MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop (20)

From Exploration to Construction  - How to Support the Complex Dynamics of In...

Modelling Time-aware Search Tasks for Search Personalisation

Parts 1 & 2: WWW 2018 Tutorial: Understanding User Needs & Tasks

Recommending Sequences RecTour 2017

UX and Usability Workshop Southampton Solent University

Usability testing through the decades

Agile2012 presentation miki_konno (aug2012)

WSDM 2011 - Nicolaas Matthijs and Filip Radlinski

Usability Testing for Qualitative Researchers - QRCA NYC Chapter event

UXprobe workshop at Dare Festival 2016

Methodology and Campaign Design for the Evaluation of Semantic Search Tools

Gunjan insight student conference v2

Temporal based Recommendation System

Assessment

Usability Testing Methods

Conducting Remote Unmoderated Usability Testing: Part 2

Conducting Remote Unmoderated Usability Testing: Part 1 - RemoteUX Training W...

User Centered Design in short

7. evalution of interactive system

Jan Moons at WUD16

More from multimediaeval

Classification of Strokes in Table Tennis with a Three Stream Spatio-Temporal...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper62.pdf YouTube: https://youtu.be/gV-rvV3iFDA Pierre-Etienne Martin, Jenny Benois-Pineau, Boris Mansencal, Renaud Péteri and Julien Morlier : Classification of Strokes in Table Tennis with a Three Stream Spatio-Temporal CNN for MediaEval 2020. Proc. of MediaEval 2020, 14-15 December 2020, Online. This work presents a method for classifying table tennis strokes using spatio-temporal convolutional neural networks. The fine-grained classification is performed on trimmed video segments recorded at 120 fps with different players performing in natural conditions. From those segments, the frames are extracted, their optical flow is computed and the pose of the player is estimated. From the optical flow amplitude, a region of interest is inferred. A three stream spatio-temporal convolutional neural network using combination of those modalities and 3D attention mechanisms is presented in order to perform classification. Presented by: Pierre-Etienne Martin

HCMUS at MediaEval 2020: Ensembles of Temporal Deep Neural Networks for Table...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper50.pdf Hai Nguyen-Truong, San Cao, N. A. Khoa Nguyen, Bang-Dang Pham, Hieu Dao, Minh-Quan Le, Hoang-Phuc Nguyen-Dinh, Hai-Dang Nguyen and Minh-Triet Tran : HCMUS at MediaEval 2020: Ensembles of Temporal Deep Neural Networks for Table Tennis Strokes Classification Task. Proc. of MediaEval 2020, 14-15 December 2020, Online. The Sports Video Classification Tasks in the Multimedia Evaluation 2020 Challenge focuses on classifying different types of table tennis strokes in video segments. In this task, we - the HCMUS Team - perform multiple experiments, which includes a combination of models such as SlowFast, Optical Flow, DensePose, R2+1, Channel-Separated Convolutional Networks, to classify 21 types of table tennis strokes from video segments. In total, we submit eight runs corresponding to five different models with different sets of hyper-parameters in each of our models. In addition, we apply some pre-processing techniques on the dataset in order for our model to learn and classify more accurately. According to the evaluation results, one of our team's methods out-performs the other team's. In particular, our best run achieves 31.35\% global accuracy, and all of our methods show potential results in terms of local and global accuracy for action recognition tasks.

Sports Video Classification: Classification of Strokes in Table Tennis for Me...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper2.pdf YouTube: https://youtu.be/-bRL868b8ys Pierre-Etienne Martin, Jenny Benois-Pineau, Boris Mansencal, Renaud Péteri, Laurent Mascarilla, Jordan Calandre and Julien Morlier : Sports Video Classification: Classification of Strokes in Table Tennis for MediaEval 2020. Proc. of MediaEval 2020, 14-15 December 2020, Online. Fine-grained action classification has raised new challenges compared to classical action classification problems. Sport video analysis is a very popular research topic, due to the variety of application areas, ranging from multimedia intelligent devices with user-tailored digests, up to analysis of athletes' performances. Running since 2019 as a part of MediaEval, we offer a task which consists in classifying table tennis strokes from videos recorded in natural conditions at the University of Bordeaux. The aim is to build tools for teachers, coaches and players to analyse table tennis games. Such tools could lead to an automatic profiling of the player and adaptation of his training for improving his/her sport skills more efficiently. Presented by: Pierre-Etienne Martin

Predicting Media Memorability from a Multimodal Late Fusion of Self-Attention...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper61.pdf YouTube: https://youtu.be/brmI4g3jLS4 Ricardo Kleinlein, Cristina Luna-Jiménez, Fernando Fernández-Martínez and Zoraida Callejas : Predicting Media Memorability from a Multimodal Late Fusion of Self-Attention and LSTM Models. Proc. of MediaEval 2020, 14-15 December 2020, Online. This paper reports on the GTH-UPM team experience in the Predicting Media Memorability task at MediaEval 2020. Teams were requested to predict memorability scores at both short-term and long-term, understanding such score as a measure of whether a video was perdurable in a viewer's memory or not. Our proposed system relies on a late fusion of the scores predicted by three sequential models, each trained over a different modality: video captions, aural embeddings and visual optical flow-based vectors. Whereas single-modality models show a low or zero Spearman correlation coefficient value, their combination considerably boosts performance over development data up to 0.2 in the short-term memorability prediction subtask and 0.19 in the long-term subtask. However, performance over test data drops to 0.016 and -0.041, respectively. Presented by: Ricardo Kleinlein

Essex-NLIP at MediaEval Predicting Media Memorability 2020 Task

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper52.pdf Janadhip Jacutprakart, Rukiye Savran Kiziltepe, John Q. Gan, Giorgos Papanastasiou and Alba G. Seco de Herrera : Essex-NLIP at MediaEval Predicting Media Memorability 2020 Task. Proc. of MediaEval 2020, 14-15 December 2020, Online. In this paper, we present the methods of approach and the main results from the Essex NLIP Team’s participation in the MediEval 2020 Predicting Media Memorability task. The task requires participants to build systems that can predict short-term and long-term memorability scores on real-world video samples provided. The focus of our approach is on the use of colour-based visual features as well as the use of the video annotation meta-data. In addition, hyper-parameter tuning was explored. Besides the simplicity of the methodology, our approach achieves competitive results. We investigated the use of different visual features. We assessed the performance of memorability scores through various regression models where Random Forest regression is our final model, to predict the memorability of videos.

Overview of MediaEval 2020 Predicting Media Memorability task: What Makes a V...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper6.pdf YouTube: https://youtu.be/ySGGu_4vaxs Alba García Seco De Herrera, Rukiye Savran Kiziltepe, Jon Chamberlain, Mihai Gabriel Constantin, Claire-Hélène Demarty, Faiyaz Doctor, Bogdan Ionescu and Alan F. Smeaton : Overview of MediaEval 2020 Predicting Media Memorability task: What Makes a Video Memorable? Proc. of MediaEval 2020, 14-15 December 2020, Online. This paper describes the MediaEval 2020 Predicting Media Memorability task. After first being proposed at MediaEval 2018, the Predicting Media Memorability task is in its 3rd edition this year, as the prediction of short-term and long-term video memorability (VM) remains a challenging task. In 2020, the format remained the same as in previous editions. This year the videos are a subset of the TRECVid 2019 Video to Text dataset, containing more action rich video content as compare with the 2019 task. In this paper a description of some aspects of this task is provided, including its main characteristics, a description of the collection, the ground truth dataset, evaluation metrics and the requirements for the run submission. Presented by: Rukiye Savran Kiziltepe

Fooling an Automatic Image Quality Estimator

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper45.pdf Benoit Bonnet, Teddy Furon and Patrick Bas : Fooling an Automatic Image Quality Estimator. Proc. of MediaEval 2020, 14-15 December 2020, Online. In this paper we present our work on the 2020 MediaEval task: Pixel "Privacy: Quality Camouflage for Social Images". Blind Image Quality Assessment (BIQA) is a classifier that for any given image will return a quality score. Our task is to modify an image to decrease its BIQA score while maintaining a good perceived quality. Since BIQA is a deep neural network, we worked on an adversarial attack approach of the problem.

Fooling Blind Image Quality Assessment by Optimizing a Human-Understandable C...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper16.pdf YouTube: https://youtu.be/ix_b9K7j72w Zhengyu Zhao : Fooling Blind Image Quality Assessment by Optimizing a Human-Understandable Color Filter. Proc. of MediaEval 2020, 14-15 December 2020, Online. This paper presents the submission of our RU-DS team to the Pixel Privacy Task 2020. We propose to fool the blind image quality assessment model by transforming images based on optimizing a human-understandable color filter. In contrast to the common work that relies on small, $L_p$-bounded additive pixel perturbations, our approach yields large yet smooth perturbations. Experimental results demonstrate that in the specific context of this task, our approach is able to achieve strong adversarial effects, but has to sacrifice the image appeal. Presented by: Zhengyu Zhao

Pixel Privacy: Quality Camouflage for Social Images

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper77.pdf YouTube: https://youtu.be/8Rr4KknGSac Zhuoran Liu, Zhengyu Zhao, Martha Larson and Laurent Amsaleg : Pixel Privacy: Quality Camouflage for Social Images. Proc. of MediaEval 2020, 14-15 December 2020, Online. High-quality social images shared online can be misappropriated for unauthorized goals, where the quality filtering step is commonly carried out by automatic Blind Image Quality Assessment (BIQA) algorithms. Pixel Privacy benchmarks privacy-protective approaches that protect privacy-sensitive images against unethical computer vision algorithms. In the 2020 task, participants are encouraged to develop camouflage methods that can effectively decrease the BIQA quality score of high-quality images and maintain image appeal. The camouflaged images need to be either imperceptible to the human eye, or it can be a visible enhancement. Presented by: Zhuoran Liu

HCMUS at MediaEval 2020:Image-Text Fusion for Automatic News-Images Re-Matching

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper73.pdf YouTube: https://youtu.be/TadJ6y7xZeA Thuc Nguyen-Quang, Tuan-Duy Nguyen, Thang-Long Nguyen-Ho, Anh-Kiet Duong, Xuan-Nhat Hoang, Vinh-Thuyen Nguyen-Truong, Hai-Dang Nguyen and Minh-Triet Tran : HCMUS at MediaEval 2020:Image-Text Fusion for Automatic News-Images Re-Matching. Proc. of MediaEval 2020, 14-15 December 2020, Online. Matching text and images based on their semantics has an important role in cross-media retrieval. However, text and images in articles have a complex connection. In the context of MediaEval 2020 Challenge, we propose three multi-modal methods for mapping text and images of news articles to the shared space in order to perform efficient cross-retrieval. Our methods show systemic improvement and validate our hypotheses, while the best-performed method reaches a recall@100 score of 0.2064. Presented by: Thuc Nguyen-Quang

Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attentio...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper72.pdf Sabarinathan D and Suganya Ramamoorthy : Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attention Unit. Proc. of MediaEval 2020, 14-15 December 2020, Online. Colorectal cancer is the third most common cause of cancer worldwide. In the era of medical Industry, identifying colorectal cancer in its early stages has been a challenging problem. Inspired by these issues, the main objective of this paper is to develop a Multi supervision net algorithm for segmenting polys on a comprehensive dataset. The risk of colorectal cancer could be reduced by early diagnosis of poly during a colonoscopy. The disease and their symptoms are highly varying and always a need for a continuous update of knowledge for the doctors and medical analyst. The diseases fall into different categories and a small variation of symptoms may lead to higher rate of risk. We have taken Medico polyp challenge dataset, which consists of 1000 segmented polyp images from gastrointestinal track. We proposed an efficient Net B4 as a pre-trained architecture in multi-supervision net. The model is trained with multiple output layers. We present quantitative results on colorectal dataset to evaluate the performance and achieved good results in all the performance metrics. The experimental results proved that the proposed model is robust and provides a good level of accuracy in segmenting polyps on a comprehensive dataset for different metrics such as Dice coefficient, Recall, Precision and F2.

HCMUS at Medico Automatic Polyp Segmentation Task 2020: PraNet and ResUnet++ ...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper47.pdf YouTube: https://youtu.be/vMsM4zg2-JY Tien-Phat Nguyen, Tan-Cong Nguyen, Gia-Han Diep, Minh-Quan Le, Hoang-Phuc Nguyen-Dinh, Hai-Dang Nguyen and Minh-Triet Tran : HCMUS at Medico Automatic Polyp Segmentation Task 2020: PraNet and ResUnet++ for Polyps Segmentation. Proc. of MediaEval 2020, 14-15 December 2020, Online. The Medico task, MediaEval 2020, explores the challenge of building accurate and high-performance algorithms to detect all types of polyps in endoscopic images. We proposed different approaches leveraging the advantages of either ResUnet++ or PraNet model to efficiently segment polyps in colonoscopy images, with modifications on the network structure, parameters, and training strategies to tackle various observed characteristics of the given dataset. Our methods outperform the other teams' methods, for both accuracy and efficiency. After the evaluation, we are at top 2 for task 1 (with Jaccard index of 0.777, best Precision and Accuracy scores) and top 1 for task 2 (with 67.52 FPS and Jaccard index of 0.658).

Depth-wise Separable Atrous Convolution for Polyps Segmentation in Gastro-Int...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper31.pdf Syed Muhammad Faraz Ali, Muhammad Taha Khan, Syed Unaiz Haider, Talha Ahmed, Zeshan Khan and Muhammad Atif Tahir : Depth-wise Separable Atrous Convolution for Polyps Segmentation in Gastro-Intestinal Tract. Proc. of MediaEval 2020, 14-15 December 2020, Online. Identification of polyps in endoscopic images is critical for the diagnosis of colon cancer. Finding the exact shape and size of polyps requires the segmentation of endoscopic images. This research explores the advantage of using depth-wise separable convolution in the atrous convolution of the ResUNet++ architecture. Deep atrous spatial pyramid pooling was also implemented on the ResUNet++ architecture. The results show that architecture with separable convolution has a smaller size and fewer GFLOPs without degrading the performance too much.

Deep Conditional Adversarial learning for polyp Segmentation

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper22.pdf Debapriya Banik and Debotosh Bhattacharjee : Deep Conditional Adversarial learning for polyp Segmentation. Proc. of MediaEval 2020, 14-15 December 2020, Online. This approach has addressed the Medico automatic polyp segmentation challenge which is a part of Mediaeval 2020. We have proposed a deep conditional adversarial learning based network for the automatic polyp segmentation task. The network comprises of two interdependent models namely a generator and a discriminator. The generator network is a FCN employed for the prediction of the polyp mask while the discriminator enforces the segmentation to be as similar as the real segmented mask (ground truth). Our proposed model achieved a comparative result on the test dataset provided by the organizers of the challenge.

A Temporal-Spatial Attention Model for Medical Image Detection

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper21.pdf Hwang Maxwell, Wu Cai, Hwang Kao-Shing, Xu Yong Si and Wu Chien-Hsing : A Temporal-Spatial Attention Model for Medical Image Detection. Proc. of MediaEval 2020, 14-15 December 2020, Online. A local region model with attentive temporal-spatial pathways is proposed for automatically learning various target structures. The attentive spatial pathway highlights the salient region to generate bounding boxes and ignores irrelevant regions in an input image. The proposed attention mechanism allows efficient object localization and the overall predictive performance is increased because there are fewer false positives for the object detection task for medical images with manual annotations. The experimental results show that proposed models consistently increase the base architectures' predictive performance for different datasets and training sizes without undue computational efficiency.

HCMUS-Juniors 2020 at Medico Task in MediaEval 2020: Refined Deep Neural Netw...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper20.pdf YouTube: https://youtu.be/CVelQl5Luf0 Quoc-Huy Trinh, Minh-Van Nguyen, Thiet-Gia Huynh and Minh-Triet Tran : HCMUS-Juniors 2020 at Medico Task in MediaEval 2020: Refined Deep Neural Network and UNet for Polyps Segmentation. Proc. of MediaEval 2020, 14-15 December 2020, Online. The Medico: Multimedia Task focuses on developing an efficient and accurate framework to computer-aided diagnosis systems for automatic polyp segmentation to detect all types of polyps in endoscopic images of the gastrointestinal (GI) tract. We are HCMUS-team approach a solution, which includes combination Residual module, Inception module, Adaptive Convolutional neural network with Unet model and PraNet to semantic segmentation all types of polyps in endoscopic images. We submit multiple runs with different architecture and parameters in our model. Our methods show potential results in accuracy and efficiency through multiple experiments.

Fine-tuning for Polyp Segmentation with Attention

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper15.pdf Rabindra Khadka : Transfer of Knowledge: Fine-tuning for Polyp Segmentation with Attention. Proc. of MediaEval 2020, 14-15 December 2020, Online. This paper describes how the transfer of prior knowledge can effectively take on segmentation tasks with the help of attention mechanisms. The UNet model pretrained on brain MRI dataset was fine-tuned with the polyp dataset. Attention mechanism was integrated to focus on relevant regions in the input images. The implemented architecture is evaluated on 200 validation images based on intersection over union and dice score between groundtruth and predicted region. The model demonstrates a promising result with computational efciency.

Bigger Networks are not Always Better: Deep Convolutional Neural Networks for...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper12.pdf Adrian Krenzer and Frank Puppe : Bigger Networks are not Always Better: Deep Convolutional Neural Networks for Automated Polyp Segmentation. Proc. of MediaEval 2020, 14-15 December 2020, Online. This paper presents our team's (AI-JMU) approach to the Medico automated polyp segmentation challenge. We consider deep convolutional neural networks to be well suited for this task. To determine the best architecture we test and compare state of the art backbones and two different heads. Finally we achieve a Jaccard index of 73.74\% on the challenge test set. We further demonstrate that bigger networks do not always perform better. However the growing network size always increases the computational complexity.

Insights for wellbeing: Predicting Personal Air Quality Index using Regressio...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper51.pdf Amel Ksibi, Amina Salhi, Ala Alluhaidan and Sahar A. El-Rahman : Insights for wellbeing: Predicting Personal Air Quality Index using Regression Approach. Proc. of MediaEval 2020, 14-15 December 2020, Online. Providing air pollution information to individuals enables them to understand the air quality of their living environments. Thus, the association between people’s wellbeing and the properties of the surrounding environment is an essential area of investigation. This paper proposes Air Quality Prediction through harvesting public/open data and leveraging them to get the Personal Air Quality index. These are usually incomplete. To cope with the problem of missing data, we applied the KNN imputation method. To predict Personal Air Quality Index, we apply a voting regression approach based on three base regressors which are Gradient Boosting regressor, Random Forest regressor, and linear regressor. Evaluating the experimental results using the RMSE metric, we got an average score of 35.39 for Walker and 51.16 for Car.

Use Visual Features From Surrounding Scenes to Improve Personal Air Quality ...

multimediaeval

Paper: http://ceur-ws.org/Vol-2882/paper40.pdf YouTube: https://youtu.be/SL5Hvu1mARY Trung-Quan Nguyen, Dang-Hieu Nguyen and Loc Tai Tan Nguyen : Use Visual Features From Surrounding Scenes to Improve Personal Air Quality Data Prediction Performance. Proc. of MediaEval 2020, 14-15 December 2020, Online. In this paper, we propose a method to predict the personal air quality index in an area by using the combination of the levels of the following pollutants: PM2.5, NO2, and O3, measured from the nearby weather stations of that area, and the photos of surrounding scenes taken at that area. Our approach uses the Inverse Distance Weighted (IDW) technique to estimate the missing air pollutant levels and then use regression to integrate visual features from taken photos to optimize the predicted values. After that, we can use those values to calculate the Air Quality Index (AQI). The results show that the proposed method may not improve the performance of the prediction in some cases.

More from multimediaeval (20)

Classification of Strokes in Table Tennis with a Three Stream Spatio-Temporal...

HCMUS at MediaEval 2020: Ensembles of Temporal Deep Neural Networks for Table...

Sports Video Classification: Classification of Strokes in Table Tennis for Me...

Predicting Media Memorability from a Multimodal Late Fusion of Self-Attention...

Essex-NLIP at MediaEval Predicting Media Memorability 2020 Task

Overview of MediaEval 2020 Predicting Media Memorability task: What Makes a V...

Fooling an Automatic Image Quality Estimator

Fooling Blind Image Quality Assessment by Optimizing a Human-Understandable C...

Pixel Privacy: Quality Camouflage for Social Images

HCMUS at MediaEval 2020:Image-Text Fusion for Automatic News-Images Re-Matching

Efficient Supervision Net: Polyp Segmentation using EfficientNet and Attentio...

HCMUS at Medico Automatic Polyp Segmentation Task 2020: PraNet and ResUnet++ ...

Depth-wise Separable Atrous Convolution for Polyps Segmentation in Gastro-Int...

Deep Conditional Adversarial learning for polyp Segmentation

A Temporal-Spatial Attention Model for Medical Image Detection

HCMUS-Juniors 2020 at Medico Task in MediaEval 2020: Refined Deep Neural Netw...

Fine-tuning for Polyp Segmentation with Attention

Bigger Networks are not Always Better: Deep Convolutional Neural Networks for...

Insights for wellbeing: Predicting Personal Air Quality Index using Regressio...

Use Visual Features From Surrounding Scenes to Improve Personal Air Quality ...

Recently uploaded

Leaf Initiation, Growth and Differentiation.pdf

RenuJangid3

What is greenhouse gasses and how many gasses are there to affect the Earth.

moosaasad1975

Astronomy Update- Curiosity’s exploration of Mars _ Local Briefs _ leadertele...

NathanBaughman3

Structures and textures of metamorphic rocks

kumarmathi863

GBSN - Microbiology (Lab 4) Culture Media

Areesha Ahmad

Mammalian Pineal Body Structure and Also Functions

YOGESH DOGRA

Comparative structure of adrenal gland in vertebrates

sachin783648

role of pramana in research.pptx in science

sonaliswain16

RNA INTERFERENCE: UNRAVELING GENETIC SILENCING

AADYARAJPANDEY1

Introduction: RNA interference (RNAi) or Post-Transcriptional Gene Silencing (PTGS) is an important biological process for modulating eukaryotic gene expression. It is highly conserved process of posttranscriptional gene silencing by which double stranded RNA (dsRNA) causes sequence-specific degradation of mRNA sequences. dsRNA-induced gene silencing (RNAi) is reported in a wide range of eukaryotes ranging from worms, insects, mammals and plants. This process mediates resistance to both endogenous parasitic and exogenous pathogenic nucleic acids, and regulates the expression of protein-coding genes. What are small ncRNAs? micro RNA (miRNA) short interfering RNA (siRNA) Properties of small non-coding RNA: Involved in silencing mRNA transcripts. Called “small” because they are usually only about 21-24 nucleotides long. Synthesized by first cutting up longer precursor sequences (like the 61nt one that Lee discovered). Silence an mRNA by base pairing with some sequence on the mRNA. Discovery of siRNA? The first small RNA: In 1993 Rosalind Lee (Victor Ambros lab) was studying a non- coding gene in C. elegans, lin-4, that was involved in silencing of another gene, lin-14, at the appropriate time in the development of the worm C. elegans. Two small transcripts of lin-4 (22nt and 61nt) were found to be complementary to a sequence in the 3' UTR of lin-14. Because lin-4 encoded no protein, she deduced that it must be these transcripts that are causing the silencing by RNA-RNA interactions. Types of RNAi ( non coding RNA) MiRNA Length (23-25 nt) Trans acting Binds with target MRNA in mismatch Translation inhibition Si RNA Length 21 nt. Cis acting Bind with target Mrna in perfect complementary sequence Piwi-RNA Length ; 25 to 36 nt. Expressed in Germ Cells Regulates trnasposomes activity MECHANISM OF RNAI: First the double-stranded RNA teams up with a protein complex named Dicer, which cuts the long RNA into short pieces. Then another protein complex called RISC (RNA-induced silencing complex) discards one of the two RNA strands. The RISC-docked, single-stranded RNA then pairs with the homologous mRNA and destroys it. THE RISC COMPLEX: RISC is large(>500kD) RNA multi- protein Binding complex which triggers MRNA degradation in response to MRNA Unwinding of double stranded Si RNA by ATP independent Helicase Active component of RISC is Ago proteins( ENDONUCLEASE) which cleave target MRNA. DICER: endonuclease (RNase Family III) Argonaute: Central Component of the RNA-Induced Silencing Complex (RISC) One strand of the dsRNA produced by Dicer is retained in the RISC complex in association with Argonaute ARGONAUTE PROTEIN : 1.PAZ(PIWI/Argonaute/ Zwille)- Recognition of target MRNA 2.PIWI (p-element induced wimpy Testis)- breaks Phosphodiester bond of mRNA.)RNAse H activity. MiRNA: The Double-stranded RNAs are naturally produced in eukaryotic cells during development, and they have a key role in regulating gene expression .

Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...

Sérgio Sacani

We characterize the earliest galaxy population in the JADES Origins Field (JOF), the deepest imaging field observed with JWST. We make use of the ancillary Hubble optical images (5 filters spanning 0.4−0.9µm) and novel JWST images with 14 filters spanning 0.8−5µm, including 7 mediumband filters, and reaching total exposure times of up to 46 hours per filter. We combine all our data at > 2.3µm to construct an ultradeep image, reaching as deep as ≈ 31.4 AB mag in the stack and 30.3-31.0 AB mag (5σ, r = 0.1” circular aperture) in individual filters. We measure photometric redshifts and use robust selection criteria to identify a sample of eight galaxy candidates at redshifts z = 11.5 − 15. These objects show compact half-light radii of R1/2 ∼ 50 − 200pc, stellar masses of M⋆ ∼ 107−108M⊙, and star-formation rates of SFR ∼ 0.1−1 M⊙ yr−1 . Our search finds no candidates at 15 < z < 20, placing upper limits at these redshifts. We develop a forward modeling approach to infer the properties of the evolving luminosity function without binning in redshift or luminosity that marginalizes over the photometric redshift uncertainty of our candidate galaxies and incorporates the impact of non-detections. We find a z = 12 luminosity function in good agreement with prior results, and that the luminosity function normalization and UV luminosity density decline by a factor of ∼ 2.5 from z = 12 to z = 14. We discuss the possible implications of our results in the context of theoretical models for evolution of the dark matter halo mass function.

Cancer cell metabolism: special Reference to Lactate Pathway

AADYARAJPANDEY1

Normal Cell Metabolism: Cellular respiration describes the series of steps that cells use to break down sugar and other chemicals to get the energy we need to function. Energy is stored in the bonds of glucose and when glucose is broken down, much of that energy is released. Cell utilize energy in the form of ATP. The first step of respiration is called glycolysis. In a series of steps, glycolysis breaks glucose into two smaller molecules - a chemical called pyruvate. A small amount of ATP is formed during this process. Most healthy cells continue the breakdown in a second process, called the Kreb's cycle. The Kreb's cycle allows cells to “burn” the pyruvates made in glycolysis to get more ATP. The last step in the breakdown of glucose is called oxidative phosphorylation (Ox-Phos). It takes place in specialized cell structures called mitochondria. This process produces a large amount of ATP. Importantly, cells need oxygen to complete oxidative phosphorylation. If a cell completes only glycolysis, only 2 molecules of ATP are made per glucose. However, if the cell completes the entire respiration process (glycolysis - Kreb's - oxidative phosphorylation), about 36 molecules of ATP are created, giving it much more energy to use. IN CANCER CELL: Unlike healthy cells that "burn" the entire molecule of sugar to capture a large amount of energy as ATP, cancer cells are wasteful. Cancer cells only partially break down sugar molecules. They overuse the first step of respiration, glycolysis. They frequently do not complete the second step, oxidative phosphorylation. This results in only 2 molecules of ATP per each glucose molecule instead of the 36 or so ATPs healthy cells gain. As a result, cancer cells need to use a lot more sugar molecules to get enough energy to survive. Unlike healthy cells that "burn" the entire molecule of sugar to capture a large amount of energy as ATP, cancer cells are wasteful. Cancer cells only partially break down sugar molecules. They overuse the first step of respiration, glycolysis. They frequently do not complete the second step, oxidative phosphorylation. This results in only 2 molecules of ATP per each glucose molecule instead of the 36 or so ATPs healthy cells gain. As a result, cancer cells need to use a lot more sugar molecules to get enough energy to survive. introduction to WARBERG PHENOMENA: WARBURG EFFECT Usually, cancer cells are highly glycolytic (glucose addiction) and take up more glucose than do normal cells from outside. Otto Heinrich Warburg (; 8 October 1883 – 1 August 1970) In 1931 was awarded the Nobel Prize in Physiology for his "discovery of the nature and mode of action of the respiratory enzyme. WARNBURG EFFECT : cancer cells under aerobic (well-oxygenated) conditions to metabolize glucose to lactate (aerobic glycolysis) is known as the Warburg effect. Warburg made the observation that tumor slices consume glucose and secrete lactate at a higher rate than normal tissues.

Orion Air Quality Monitoring Systems - CWS

Columbia Weather Systems

Lab report on liquid viscosity of glycerin

ossaicprecious19

Lateral Ventricles.pdf very easy good diagrams comprehensive

silvermistyshot

In silico drugs analogue design: novobiocin analogues.pptx

AlaminAfendy1

THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.

Sérgio Sacani

The return of a sample of near-surface atmosphere from Mars would facilitate answers to several first-order science questions surrounding the formation and evolution of the planet. One of the important aspects of terrestrial planet formation in general is the role that primary atmospheres played in influencing the chemistry and structure of the planets and their antecedents. Studies of the martian atmosphere can be used to investigate the role of a primary atmosphere in its history. Atmosphere samples would also inform our understanding of the near-surface chemistry of the planet, and ultimately the prospects for life. High-precision isotopic analyses of constituent gases are needed to address these questions, requiring that the analyses are made on returned samples rather than in situ.

PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION

ChetanK57

SCHIZOPHRENIA Disorder/ Brain Disorder.pdf

SELF-EXPLANATORY

Nucleic Acid-its structural and functional complexity.

Nistarini College, Purulia (W.B) India

The ASGCT Annual Meeting was packed with exciting progress in the field advan...

Health Advances

Recently uploaded (20)

Leaf Initiation, Growth and Differentiation.pdf

What is greenhouse gasses and how many gasses are there to affect the Earth.

Astronomy Update- Curiosity’s exploration of Mars _ Local Briefs _ leadertele...

Structures and textures of metamorphic rocks

GBSN - Microbiology (Lab 4) Culture Media

Mammalian Pineal Body Structure and Also Functions

Comparative structure of adrenal gland in vertebrates

role of pramana in research.pptx in science

RNA INTERFERENCE: UNRAVELING GENETIC SILENCING

Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...

Cancer cell metabolism: special Reference to Lactate Pathway

Orion Air Quality Monitoring Systems - CWS

Lab report on liquid viscosity of glycerin

Lateral Ventricles.pdf very easy good diagrams comprehensive

In silico drugs analogue design: novobiocin analogues.pptx

THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.

PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION

SCHIZOPHRENIA Disorder/ Brain Disorder.pdf

Nucleic Acid-its structural and functional complexity.

The ASGCT Annual Meeting was packed with exciting progress in the field advan...

MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop

1. IR evaluation: Putting the user back in the loop Evangelos Kanoulas e.kanoulas@uva.nl

2. Change the search algorithm. How can we know whether we made the users happier?

3. Different approaches to evaluation • User-‐studies • In-‐situ evaluation • A/B Testing • Interleaving • Collection-‐based evaluation

4. in-‐situ evaluation

5. A/B Testing Baseline (control) Experimental (treatment)

8. collection-‐based evaluation

9. Machine Learning • Feature vectors • Labels Cranfield Collections Information Retrieval • Documents • Queries • Labels – relevance judgments Query 1 Query 2 Query N

10.

11. Cranfield Paradigm • Simple user model • Controlled experiments • Reusable but static test collections Online Evaluation • Full user participation • Many degrees of freedom • Unrepeatable experiments System Focus User Focus Evaluation Landscape TREC Tasks TREC Session TREC Total Recall TREC Open Search

12. TREC Total Recall results human assessor search algorithm query document collection

13. TREC Session Track

14. TREC Session Track [2010-‐2014] 1. improve search by using session information 2. improve search over an entire user’s session instead of a single query

15. Paris Luxurious Hotels Paris Hilton

16. Test Collection Û the set of clicked URLs/snippets. Test Collection Statistics 2011 2012 2013 2014 collection ClueWeb09 ClueWeb09 ClueWeb12 ClueWeb12 topic properties topic set size 62 48 61 60 topic cat. dist. known-item 10 exploratory, 6 interpretive, 20 known-item, 12 known-subj 10 exploratory, 9 interpretive, 32 known-item, 10 known-subj 15 exploratory, 15 interpretive, 15 known-item, 15 known-subj session properties user population U. She eld U. She eld U. She eld + IR researchers MTurk search engine BOSS+CW09 ﬁlter BOSS+CW09 ﬁlter indri indri total sessions 76 98 133 1,257 sessions per topic 1.2 2.0 2.2 21.0 mean length (in queries) 3.7 3.0 3.7 3.7 median time between queries 68.5s 66.7s 72.2s 25.6s relevance judgments topics judged 62 48 49 51 total judgments 19,413 17,861 13,132 16,949

17. TREC Session Track [2010-‐2014] 1. improve search by using session information 2. improve search over an entire user’s session instead of a single query

18.

19. TREC Tasks Track

20. TREC Tasks Track [2015–now] 1. understand underlying user’s task 2. assist user in completing the task

21. Make Improvements At Home TASK UNDERSTANDING

22. Make Improvements At Home TASK COMPLETION

23. TREC Session Track [2010-‐2014] 1. improve search by using session information 2. improve search over an entire user’s session instead of a single query

24. CLEF Dynamic Search for Complex Tasks

25. CLEF Complex Tasks [now] 1. Produce methodology and algorithms that will lead to a dynamic test collection by simulating users 2. Understand and quantify what constitutes a good ranking of documents at different stages of a session, and a good overall session

26. TREC Open Search

MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop

Recommended

Recommended

More Related Content

What's hot

What's hot (15)

Viewers also liked

Viewers also liked (16)

Similar to MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop

Similar to MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop (20)

More from multimediaeval

More from multimediaeval (20)

Recently uploaded

Recently uploaded (20)

MediaEval 2016 - IR Evaluation: Putting the User Back in the Loop