Talk by Vasileios Mezaris, titled "Misinformation on the internet: Video and AI", delivered at the "Age of misinformation: an interdisciplinary outlook on fake news" webinar, on 17 December 2020.
Video Activism panel for 09 Women Who Tech Telesummit May 12, 2009 Moderator: Shirley Sexton, Panelists: Matisse Bustos Hawkes, WITNESS, Erica Priggen, Free range Studios, Ramya Raghavan, YouTube
Supporting Activists to Preserve Video Documentation dri_ireland
This presentation was delivered by Yvonne Ng, Archives Manager at WITNESS, as part of ‘Engaging Communities with Archives: Video as a tool for activism, advocacy, and archival work’, a collaborative online event hosted by the Digital Repository of Ireland (DRI) and the Public Record Office of Northern Ireland (PRONI) on 7 Sept 2021. The webinar focused on archival initiatives and participatory projects that aim to train or support community groups in using video to tell personal stories, bring about social change, or archive and preserve activism and advocacy work.
The presentation focuses on WITNESS’s work and how they support people to use video as a tool for activism and advocacy.
Video Activism panel for 09 Women Who Tech Telesummit May 12, 2009 Moderator: Shirley Sexton, Panelists: Matisse Bustos Hawkes, WITNESS, Erica Priggen, Free range Studios, Ramya Raghavan, YouTube
Supporting Activists to Preserve Video Documentation dri_ireland
This presentation was delivered by Yvonne Ng, Archives Manager at WITNESS, as part of ‘Engaging Communities with Archives: Video as a tool for activism, advocacy, and archival work’, a collaborative online event hosted by the Digital Repository of Ireland (DRI) and the Public Record Office of Northern Ireland (PRONI) on 7 Sept 2021. The webinar focused on archival initiatives and participatory projects that aim to train or support community groups in using video to tell personal stories, bring about social change, or archive and preserve activism and advocacy work.
The presentation focuses on WITNESS’s work and how they support people to use video as a tool for activism and advocacy.
[Webinar Slides] 4 Ways to Protect Your Captured Data from Theft and Complian...AIIM International
Learn about the risks and challenges of typical imaging and 4 ways you can make sure your captured information is protected.
For more information on Scanning & Capture, visit our resource center: http://www.aiim.org/Resource-Centers/Scanning-and-Capture.
Too often do professionals focus on individual social or digital media platforms/products - how to effectively use Twitter, or set up a Linkedin profile, etc. This presentation is a step back, attempting to give a simple, big picture overview of the online ecosystem and the major trends defining it.
20160317 ARMA Wyoming Social Media Security ThreatsJesse Wilkins
This session from the ARMA Wyoming 2016 Spring Seminar outlined the different security threats social media provides to individuals as well as organizations.
In April 2004, a bold experiment by the Infosecurity Tradeshow in London proved what everyone suspected, over 70% of people passing through Liverpool Street Station would reveal their password in exchange for candy (http://news.bbc.co.uk/2/hi/technology/3639679.stm). Some commentators applauded this validation of a previously unproven assumption about Londoner’s attitudes towards password secrecy. Other commentators had serious ethical concerns with the experiment.
This candy-for-password experiment got me thinking about health privacy/security experiments. Many suspect that the healthcare system has serious human and technical privacy vulnerabilities, but how can we validate this suspicion? Would a patient hand over their provincial health number for a chocolate bar? Would a medical professional hand over a patient’s information for a chai latte? The more I thought about it, the more extreme – and both frightening and funny – the research projects became.
Presentation of the InVID tool for social media verificationInVID Project
Presentation of the InVID tool for social media verification through contextual analysis, at the Media Informatics Lab meeting on detection and verification of socially shared videos.
Secure, social, cloud - mutually exclusive or perfect partners?Stuart Barr
Is cloud-based social software inherently risky for enterprises looking to take advantage of the new wave of technology innovation or is it possible to have your cake and eat it?
Mobile security is a growing issue, with the increased uptake of smartphones and tablets. Learn what risks exist and how you can protect your devices from malware and data loss.
The tragic terrorist attacks in Paris have fueled, even more than before, a strong campaign against widespread adoption of encryption tools, even if it soon became clear that the terrorists hadn't used any. While in USA plans for the adoption of compulsory backdoors, in order to circumvent encryption, have been apparently abandoned, many European States seem to perceive encryption software and devices as one of the worst evils. Which is the role of FOSS and FOSS developers in this scenario? How can we enhance communication privacy, without breaking any regulations?
Social media mining for sensing and responding to real-world trends and eventsYiannis Kompatsiaris
Social media have transformed the Web into an interactive sharing platform where users upload data and media, comment on, and share this content within their social circles. The large-scale availability of user-generated content in social media platforms has opened up new possibilities for studying and understanding real-world phenomena, trends and events. The objective of this talk is to provide an overview of social media mining, which offers a unique opportunity to to discover, collect, and extract relevant information in order to provide useful insights. It will include key challenges and issues, such as fighting misinformation, data collection, analysis and visualization components, applications, results and demos from multiple areas ranging from news to environmental and security ones.
Unmasking deepfakes: A systematic review of deepfake detection and generation...Araz Taeihagh
Due to the fast spread of data through digital media, individuals and societies must assess the reliability of information. Deepfakes are not a novel idea but they are now a widespread phenomenon. The impact of deepfakes and disinformation can range from infuriating individuals to affecting and misleading entire societies and even nations. There are several ways to detect and generate deepfakes online. By conducting a systematic literature analysis, in this study we explore automatic key detection and generation methods, frameworks, algorithms, and tools for identifying deepfakes (audio, images, and videos), and how these approaches can be employed within different situations to counter the spread of deepfakes and the generation of disinformation. Moreover, we explore state-of-the-art frameworks related to deepfakes to understand how emerging machine learning and deep learning approaches affect online disinformation. We also highlight practical challenges and trends in implementing policies to counter deepfakes. Finally, we provide policy recommendations based on analyzing how emerging artificial intelligence (AI) techniques can be employed to detect and generate deepfakes online. This study benefits the community and readers by providing a better understanding of recent developments in deepfake detection and generation frameworks. The study also sheds a light on the potential of AI in relation to deepfakes.
Video & AI: capabilities and limitations of AI in detecting video manipulationsVasileiosMezaris
Invited presentation given by Dr. Vasileios Mezaris during the Greek Media Literacy Week 2019; specifically, presented in the international conference on "Disinformation in Cyberspace: Media literacy meets Artificial Intelligence" that was organized as part of the Media Literacy Week 2019 in Athens, Greece, on November 15, 2019.
Identity REvolution multi disciplinary perspectivesKarlos Svoboda
The identity [r]evolution is happening. Who are
you, who am I in the information society ?
In recent years, the convergence of several factors – technological, political, economic –
has accelerated a fundamental change in our networked world. On a technological level, information
becomes easier to gather, to store, to exchange
and to process. The belief that more information
brings more security has been a strong political
driver to promote information gathering since September 11. Profiling intends to transform information into knowledge in order to anticipate one’s behaviour, or needs, or preferences. It can lead to
categorizations according to some specific risk criteria, for example, or to direct and personalized
marketing. As a consequence, new forms of identities appear. They are not necessarily related to our
names anymore. They are based on information,
on traces that we leave when we act or interact,
when we go somewhere or just stay in one place,
or even sometimes when we make a choice. They
are related to the SIM cards of our mobile phones,
to our credit card numbers, to the pseudonyms
that we use on the Internet, to our email addresses,
to the IP addresses of our computers, to our profiles… Like traditional identities, these new forms of
identities can allow us to distinguish an individual
within a group of people, or describe this person as
belonging to a community or a category.
AI: The New Player in Cybersecurity (Nov. 08, 2023)Takeshi Takahashi
These slides outline how AI is influencing cybersecurity.
Note that they were used in the keynote speech at the event "Defense and Security 2023" held in Thailand on November 8, 2023.
[Webinar Slides] 4 Ways to Protect Your Captured Data from Theft and Complian...AIIM International
Learn about the risks and challenges of typical imaging and 4 ways you can make sure your captured information is protected.
For more information on Scanning & Capture, visit our resource center: http://www.aiim.org/Resource-Centers/Scanning-and-Capture.
Too often do professionals focus on individual social or digital media platforms/products - how to effectively use Twitter, or set up a Linkedin profile, etc. This presentation is a step back, attempting to give a simple, big picture overview of the online ecosystem and the major trends defining it.
20160317 ARMA Wyoming Social Media Security ThreatsJesse Wilkins
This session from the ARMA Wyoming 2016 Spring Seminar outlined the different security threats social media provides to individuals as well as organizations.
In April 2004, a bold experiment by the Infosecurity Tradeshow in London proved what everyone suspected, over 70% of people passing through Liverpool Street Station would reveal their password in exchange for candy (http://news.bbc.co.uk/2/hi/technology/3639679.stm). Some commentators applauded this validation of a previously unproven assumption about Londoner’s attitudes towards password secrecy. Other commentators had serious ethical concerns with the experiment.
This candy-for-password experiment got me thinking about health privacy/security experiments. Many suspect that the healthcare system has serious human and technical privacy vulnerabilities, but how can we validate this suspicion? Would a patient hand over their provincial health number for a chocolate bar? Would a medical professional hand over a patient’s information for a chai latte? The more I thought about it, the more extreme – and both frightening and funny – the research projects became.
Presentation of the InVID tool for social media verificationInVID Project
Presentation of the InVID tool for social media verification through contextual analysis, at the Media Informatics Lab meeting on detection and verification of socially shared videos.
Secure, social, cloud - mutually exclusive or perfect partners?Stuart Barr
Is cloud-based social software inherently risky for enterprises looking to take advantage of the new wave of technology innovation or is it possible to have your cake and eat it?
Mobile security is a growing issue, with the increased uptake of smartphones and tablets. Learn what risks exist and how you can protect your devices from malware and data loss.
The tragic terrorist attacks in Paris have fueled, even more than before, a strong campaign against widespread adoption of encryption tools, even if it soon became clear that the terrorists hadn't used any. While in USA plans for the adoption of compulsory backdoors, in order to circumvent encryption, have been apparently abandoned, many European States seem to perceive encryption software and devices as one of the worst evils. Which is the role of FOSS and FOSS developers in this scenario? How can we enhance communication privacy, without breaking any regulations?
Social media mining for sensing and responding to real-world trends and eventsYiannis Kompatsiaris
Social media have transformed the Web into an interactive sharing platform where users upload data and media, comment on, and share this content within their social circles. The large-scale availability of user-generated content in social media platforms has opened up new possibilities for studying and understanding real-world phenomena, trends and events. The objective of this talk is to provide an overview of social media mining, which offers a unique opportunity to to discover, collect, and extract relevant information in order to provide useful insights. It will include key challenges and issues, such as fighting misinformation, data collection, analysis and visualization components, applications, results and demos from multiple areas ranging from news to environmental and security ones.
Unmasking deepfakes: A systematic review of deepfake detection and generation...Araz Taeihagh
Due to the fast spread of data through digital media, individuals and societies must assess the reliability of information. Deepfakes are not a novel idea but they are now a widespread phenomenon. The impact of deepfakes and disinformation can range from infuriating individuals to affecting and misleading entire societies and even nations. There are several ways to detect and generate deepfakes online. By conducting a systematic literature analysis, in this study we explore automatic key detection and generation methods, frameworks, algorithms, and tools for identifying deepfakes (audio, images, and videos), and how these approaches can be employed within different situations to counter the spread of deepfakes and the generation of disinformation. Moreover, we explore state-of-the-art frameworks related to deepfakes to understand how emerging machine learning and deep learning approaches affect online disinformation. We also highlight practical challenges and trends in implementing policies to counter deepfakes. Finally, we provide policy recommendations based on analyzing how emerging artificial intelligence (AI) techniques can be employed to detect and generate deepfakes online. This study benefits the community and readers by providing a better understanding of recent developments in deepfake detection and generation frameworks. The study also sheds a light on the potential of AI in relation to deepfakes.
Video & AI: capabilities and limitations of AI in detecting video manipulationsVasileiosMezaris
Invited presentation given by Dr. Vasileios Mezaris during the Greek Media Literacy Week 2019; specifically, presented in the international conference on "Disinformation in Cyberspace: Media literacy meets Artificial Intelligence" that was organized as part of the Media Literacy Week 2019 in Athens, Greece, on November 15, 2019.
Identity REvolution multi disciplinary perspectivesKarlos Svoboda
The identity [r]evolution is happening. Who are
you, who am I in the information society ?
In recent years, the convergence of several factors – technological, political, economic –
has accelerated a fundamental change in our networked world. On a technological level, information
becomes easier to gather, to store, to exchange
and to process. The belief that more information
brings more security has been a strong political
driver to promote information gathering since September 11. Profiling intends to transform information into knowledge in order to anticipate one’s behaviour, or needs, or preferences. It can lead to
categorizations according to some specific risk criteria, for example, or to direct and personalized
marketing. As a consequence, new forms of identities appear. They are not necessarily related to our
names anymore. They are based on information,
on traces that we leave when we act or interact,
when we go somewhere or just stay in one place,
or even sometimes when we make a choice. They
are related to the SIM cards of our mobile phones,
to our credit card numbers, to the pseudonyms
that we use on the Internet, to our email addresses,
to the IP addresses of our computers, to our profiles… Like traditional identities, these new forms of
identities can allow us to distinguish an individual
within a group of people, or describe this person as
belonging to a community or a category.
AI: The New Player in Cybersecurity (Nov. 08, 2023)Takeshi Takahashi
These slides outline how AI is influencing cybersecurity.
Note that they were used in the keynote speech at the event "Defense and Security 2023" held in Thailand on November 8, 2023.
Aggregating and Analyzing the Context of Social Media ContentSymeon Papadopoulos
Introduction to the Context Analysis and Aggregation service of InVID. Given at the Workshop on Content Verification Tools hosted by the journalists' association in Thessaloniki, Greece on June 6, 2018.
The “deepfake” phenomenon — using machine learning to generate synthetic video, audio and text content — is an ominous example of how quickly new technologies can be diverted from their original purposes. Month by month, it is becoming easier and cheaper to create fakes that are increasingly difficult to distinguish from genuine artefacts.
Mastering Digital Media Literacy: Navigating Information in the Digital AgeSelcen Ozturkcan
- Critically evaluate digital media: Identify trustworthy sources and detect bias and (mis/dis)misinformation.
- Consume digital media ethically: Engage responsibly online and understand media's societal impact.
- Utilize digital media literacy tools: Use fact-checking websites and frameworks to verify information.
What could kill NSTIC? A friendly threat assessment in 3 parts.Phil Wolff
At two events 18-months apart, teams of suits, geeks, and wonks (industry experts, technologists, public policy analysts) brainstormed and scored what could lead to failure of NSTIC, an international effort to create an identity ecosystem. The whitepaper at http://pde.cc/nsticrisks recaps the long list of potential threats, a shorter list of preventive strategies, compares the 2011 and 2012 events, and names the two greatest threats: poor user experience (harming trust, adoption, use) and imbalance among the forces tying the identity ecosystem together.
The Mass diffusion of digital media and the explosive growth of telecommunication are reshaping the lifestyles of ordinary people, research and industry. Over the last decades, the rise of digital telecommunication technologies has fundamentally altered how people work, think, communicate, and socialize. Despite the obvious progress of multimedia communications, these developments carry with them a number of risks such as copyright violation, prohibited usage and distribution of digital media, secret communications, and network security. Therefore, security, scalability and manageability amongst other become issues of serious concern, as current solutions do not satisfy anymore the growing demands of multimedia communications. Security has been in a center stage of international attention since we first-handedly witnesses the pervasiveness of terrorism activities. Lately, more and more government and industry resources are located to the researches of security systems. How can we ensure the trustworthiness of multimedia data? How can analysts extract intelligence from enormous video streams? How can visual or audio biometric features help to identity suspects? This research work examines multimedia security, its impact on our society and the place of our law on it, adopting literature review methods. This paper examines the meaning of multimedia security, its usefulness to our society and the place of our law on it and some possible interventions.
HOW TECHNOLOGY HAS IMPACTED THE LIVES OF CONSUMERS AND BUSINESS - slideshare.pdfLorenzoBielli2
Speak about internet users distribution in the world, the growth of the internet and the speed connections.
I Answer different questions such as:
what are the most used devices to access the internet?
how people use internet?
what people purchase online?
And much more like the key role of online videos and how technology affects consumers and businesses in «digital era».
Summary of problems and research results on the problem of verifying multimedia content on the Internet. Includes results from the REVEAL and InVID research projects. Presented at the Technology Forum, Thessaloniki, May 16, 2018.
Multi-Modal Fusion for Image Manipulation Detection and LocalizationVasileiosMezaris
Presentation of paper "Exploring Multi-Modal Fusion for Image Manipulation Detection and Localization", by K. Triaridis, V. Mezaris, delivered at 30th Int. Conf. on MultiMedia Modeling (MMM 2024), Amsterdam, NL, Jan.-Feb. 2024.
Presentation of our top-scoring solution to the MediaEval 2023 NewsImages Task, "Cross-modal Networks, Fine-Tuning, Data Augmentation and Dual Softmax Operation for MediaEval NewsImages 2023", by A. Leventakis, D. Galanopoulos, V. Mezaris, delivered at the 2023 Multimedia Evaluation Workshop (MediaEval'23), Amsterdam, NL, Feb. 2024.
Spatio-Temporal Summarization of 360-degrees VideosVasileiosMezaris
Presentation of paper "An Integrated System for Spatio-Temporal Summarization of 360-degrees Videos", by I. Kontostathis, E. Apostolidis, V. Mezaris, delivered at 30th Int. Conf. on MultiMedia Modeling (MMM 2024), Amsterdam, NL, Jan.-Feb. 2024.
Masked Feature Modelling for the unsupervised pre-training of a Graph Attenti...VasileiosMezaris
Presentation of paper "Masked Feature Modelling for the unsupervised pre-training of a Graph Attention Network block for bottom-up video event recognition", by D. Daskalakis, N. Gkalelis, V. Mezaris, delivered at IEEE ISM 2023, Dec. 2022, Laguna Hills, CA, USA.
Cross-modal Networks and Dual Softmax Operation for MediaEval NewsImages 2022VasileiosMezaris
Matching images to articles is challenging and can be considered a special version of the cross-media retrieval problem. This working note paper presents our solution for the MediaEval NewsImages
benchmarking task. We investigated the performance of two cross-modal networks, a pre-trained network and a trainable one, the latter originally developed for text-video retrieval tasks and adapted to the NewsImages task. Moreover, we utilize a method for revising the similarities produced by either one of the cross-modal networks, i.e., a dual softmax operation, to improve our solutions’ performance. We report the official results for our submitted runs and additional experiments we conducted to evaluate our runs internally.
TAME: Trainable Attention Mechanism for ExplanationsVasileiosMezaris
Presentation of paper "TAME: Attention Mechanism Based Feature Fusion for Generating Explanation Maps of Convolutional Neural Networks", by M. Ntrougkas, N. Gkalelis, V. Mezaris, delivered at IEEE ISM 2022, Dec. 2022, Naples, Italy.
The apparent “black box” nature of neural networks is a barrier to adoption in applications where explainability is essential. This paper presents TAME (Trainable Attention Mechanism for Explanations), a method for generating explanation maps with a multi-branch hierarchical attention mechanism. TAME combines a target model’s feature maps from multiple layers using an attention mechanism, transforming them into an explanation map. TAME can easily be applied to any convolutional neural network (CNN) by streamlining the optimization of the attention mechanism’s training method and the selection
of target model’s feature maps. After training, explanation maps can be computed in a single forward pass. We apply TAME to two widely used models, i.e. VGG-16 and ResNet-50, trained on ImageNet and show improvements over previous top-performing methods. We also provide a comprehensive ablation study comparing the performance of different variations of TAME’s architecture.
Presentation of paper "Gated-ViGAT: Efficient Bottom-Up Event
Recognition and Explanation Using a New Frame
Selection Policy and Gating Mechanism", by N. Gkalelis, D. Daskalakis, V. Mezaris, delivered at IEEE ISM 2022, Dec. 2022, Naples, Italy.
In this paper, Gated-ViGAT, an efficient approach for video event recognition, utilizing bottom-up (object) information, a new frame sampling policy and a gating mechanism is proposed. Specifically, the frame sampling policy uses weighted in-degrees (WiDs), derived from the adjacency matrices of graph attention networks (GATs), and a dissimilarity measure to select
the most salient and at the same time diverse frames representing
the event in the video. Additionally, the proposed gating mechanism fetches the selected frames sequentially, and commits early exiting when an adequately confident decision is achieved. In this way, only a few frames are processed by the computationally
expensive branch of our network that is responsible for the bottom-up information extraction. The experimental evaluation on two large, publicly available video datasets (MiniKinetics, ActivityNet) demonstrates that Gated-ViGAT provides a large computational complexity reduction in comparison to our previous approach (ViGAT), while maintaining the excellent event
recognition and explainability performance.
Explaining video summarization based on the focus of attentionVasileiosMezaris
Presentation of paper "Explaining video summarization based on
the focus of attention", by E. Apostolidis, G. Balaouras, V. Mezaris, I. Patras, delivered at IEEE ISM 2022, Dec. 2022, Naples, Italy.
In this paper we propose a method for explaining
video summarization. We start by formulating the problem as
the creation of an explanation mask which indicates the parts
of the video that influenced the most the estimates of a video
summarization network, about the frames’ importance. Then, we
explain how the typical analysis pipeline of attention-based networks for video summarization can be used to define explanation
signals, and we examine various attention-based signals that have
been studied as explanations in the NLP domain. We evaluate
the performance of these signals by investigating the video
summarization network’s input-output relationship according
to different replacement functions, and utilizing measures that quantify the capability of explanations to spot the most and
least influential parts of a video. We run experiments using an
attention-based network (CA-SUM) and two datasets (SumMe
and TVSum) for video summarization. Our evaluations indicate the advanced performance of explanations formed using the inherent attention weights, and demonstrate the ability of our
method to explain the video summarization results using clues
about the focus of the attention mechanism.
Combining textual and visual features for Ad-hoc Video SearchVasileiosMezaris
In this presentation, our work in the context of the Ad-hoc Video Search (AVS) Task of TRECVID 2022 is presented. Our participation in the AVS task is based on a cross-modal deep network architecture, T x V ("T times V"), which utilizes several textual and visual features. As part of the retrieval stage, a dual-softmax approach is also utilized to revise the calculated text-video similarities.
Explaining the decisions of image/video classifiersVasileiosMezaris
Presentation delivered by Vasileios Mezaris at the 1st Nice Workshop on Interpretability, November 2022, Nice, France.
This presentation starts by discussing the motivation of explainability approaches for image and video classifiers. Then, we focus on three distinct problems: learning how to derive explanations for the decisions of a legacy (trained) image classifier; designing a classifier for video event recognition that can also deliver explanations for its decisions; and, taking a first look at possible explanation signals of a video summarizer. Technical details of our proposed solutions to these three problems are presented. Besides quantitative results concerning the goodness of the derived explanations, qualitative examples are also discussed in order to provide insight on the reasons behind classification errors, including possible dataset biases affecting the trained classifiers.
Learning visual explanations for DCNN-based image classifiers using an attent...VasileiosMezaris
I. Gkartzonika, N. Gkalelis, V. Mezaris, "Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism", Proc. ECCV 2022 Workshop on Vision with Biased or Scarce Data (VBSD), Oct. 2022.
In this paper two new learning-based eXplainable AI (XAI) methods for deep convolutional neural network (DCNN) image classifiers, called L-CAM-Fm and L-CAM-Img, are proposed. Both methods use an attention mechanism that is inserted in the original (frozen) DCNN and is trained to derive class activation maps (CAMs) from the last convolutional layer’s feature maps. During training, CAMs are applied to the feature maps (L-CAM-Fm) or the input image (L-CAM-Img) forcing the attention mechanism to learn the image regions explaining the DCNN’s outcome. Experimental evaluation on ImageNet shows that the proposed methods achieve competitive results while requiring a single forward pass at the inference stage. Moreover, based on the derived explanations a comprehensive qualitative analysis is performed providing valuable insight for understanding the reasons behind classification errors, including possible dataset biases affecting the trained classifier.
Are all combinations equal? Combining textual and visual features with multi...VasileiosMezaris
D. Galanopoulos, V. Mezaris, "Are All Combinations Equal? Combining Textual and Visual Features with Multiple Space Learning for Text-Based Video Retrieval", Proc. ECCV 2022 Workshop on AI for Creative Video Editing and Understanding (CVEU), Oct. 2022.
In this paper we tackle the cross-modal video retrieval problem and, more specifically, we focus on text-to-video retrieval. We investigate how to optimally combine multiple diverse textual and visual features into feature pairs that lead to generating multiple joint feature spaces, which encode text-video pairs into comparable representations. To learn these representations our proposed network architecture is trained by following a multiple space learning procedure. Moreover, at the retrieval stage, we introduce additional softmax operations for revising the inferred query-video similarities. Extensive experiments in several setups based on three large-scale datasets (IACC.3, V3C1, and MSR-VTT) lead to conclusions on how to best combine text-visual features and document the performance of the proposed network.
Presentation of the paper titled "Summarizing videos using concentrated attention and considering the uniqueness and diversity of the video frames", by E. Apostolidis, G. Balaouras, V. Mezaris, I. Patras, delivered at the ACM Int. Conf. on Multimedia Retrieval (ICMR’22), Newark, NJ, USA, June 2022. The corresponding software is available at https://github.com/e-apostolidis/CA-SUM.
Presentation of the paper titled "A Web Service for Video Smart-Cropping", by K. Apostolidis, V. Mezaris, delivered at the IEEE Int. Symposium on Multimedia (ISM), Dec. 2021. The corresponding software and dataset are available at https://github.com/bmezaris/RetargetVid.
Presentation of the paper titled "Combining Global and Local Attention with Positional Encoding for Video Summarization", by E. Apostolidis, G. Balaouras, V. Mezaris, I. Patras, delivered at the IEEE Int. Symposium on Multimedia (ISM), Dec. 2021. The corresponding software is available at https://github.com/e-apostolidis/PGL-SUM.
Presentation slides for our paper "Combining Adversarial and Reinforcement Learning for Video Thumbnail Selection", ACM ICMR 2021. https://doi.org/10.1145/3460426.3463630.
We developed a new method for unsupervised video thumbnail selection. The developed network architecture selects video thumbnails based on two criteria: the representativeness and the aesthetic quality of their visual content. Training relies on a combination of adversarial and reinforcement learning. The former is used to train a discriminator, whose goal is to distinguish the original from a reconstructed version of the video based on a small set of candidate thumbnails. The discriminator’s feedback is a measure of the representativeness of the selected thumbnails. This measure is combined with estimates about the aesthetic quality of the thumbnails (made using a SoA Fully Convolutional Network) to form a reward and train the thumbnail selector via reinforcement learning. Experiments on two datasets (OVP and Youtube) show the competitiveness of the proposed method against other SoA approaches. An ablation study with respect to the adopted thumbnail selection criteria documents the importance of considering the aesthetics, and the contribution of this information when used in combination with measures about the representativeness of the visual content.
Hard-Negatives Selection Strategy for Cross-Modal RetrievalVasileiosMezaris
Cross-modal learning has gained a lot of interest recently, and many applications of it, such as image-text retrieval, cross-modal video search, or video captioning have been proposed. In this work, we deal with the cross-modal video retrieval problem. The state-of-the-art approaches are based on deep network architectures, and rely on mining hard-negative samples during training to optimize the selection of the network’s parameters. Starting from a state-of-the-art cross-modal architecture that uses the improved marginal ranking loss function, we propose a simple strategy for hard-negative mining to identify which training samples are hard-negatives and which, although presently treated as hard-negatives, are likely not negative samples at all and shouldn’t be treated as such. Additionally, to take full advantage of network models trained using different design choices for hard-negative mining, we examine model combination strategies, and we design a hybrid one effectively combining large numbers of trained models.
Slides for the paper titled "Structured pruning of LSTMs via Eigenanalysis and Geometric Median for Mobile Multimedia and Deep Learning Applications", by N. Gkalelis and V. Mezaris, presented at the 22nd IEEE Int. Symposium on Multimedia (ISM), Dec. 2020.
Slides for the paper "Performance over Random: A robust evaluation protocol for video summarization methods", authored by E. Apostolidis, E. Adamantidou, A. Metsai, V. Mezaris, and I. Patras, published in the Proceedings of ACM Multimedia 2020 (ACM MM), Seattle, WA, USA, Oct. 2020.
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...Ana Luísa Pinho
Functional Magnetic Resonance Imaging (fMRI) provides means to characterize brain activations in response to behavior. However, cognitive neuroscience has been limited to group-level effects referring to the performance of specific tasks. To obtain the functional profile of elementary cognitive mechanisms, the combination of brain responses to many tasks is required. Yet, to date, both structural atlases and parcellation-based activations do not fully account for cognitive function and still present several limitations. Further, they do not adapt overall to individual characteristics. In this talk, I will give an account of deep-behavioral phenotyping strategies, namely data-driven methods in large task-fMRI datasets, to optimize functional brain-data collection and improve inference of effects-of-interest related to mental processes. Key to this approach is the employment of fast multi-functional paradigms rich on features that can be well parametrized and, consequently, facilitate the creation of psycho-physiological constructs to be modelled with imaging data. Particular emphasis will be given to music stimuli when studying high-order cognitive mechanisms, due to their ecological nature and quality to enable complex behavior compounded by discrete entities. I will also discuss how deep-behavioral phenotyping and individualized models applied to neuroimaging data can better account for the subject-specific organization of domain-general cognitive systems in the human brain. Finally, the accumulation of functional brain signatures brings the possibility to clarify relationships among tasks and create a univocal link between brain systems and mental functions through: (1) the development of ontologies proposing an organization of cognitive processes; and (2) brain-network taxonomies describing functional specialization. To this end, tools to improve commensurability in cognitive science are necessary, such as public repositories, ontology-based platforms and automated meta-analysis tools. I will thus discuss some brain-atlasing resources currently under development, and their applicability in cognitive as well as clinical neuroscience.
Slide 1: Title Slide
Extrachromosomal Inheritance
Slide 2: Introduction to Extrachromosomal Inheritance
Definition: Extrachromosomal inheritance refers to the transmission of genetic material that is not found within the nucleus.
Key Components: Involves genes located in mitochondria, chloroplasts, and plasmids.
Slide 3: Mitochondrial Inheritance
Mitochondria: Organelles responsible for energy production.
Mitochondrial DNA (mtDNA): Circular DNA molecule found in mitochondria.
Inheritance Pattern: Maternally inherited, meaning it is passed from mothers to all their offspring.
Diseases: Examples include Leber’s hereditary optic neuropathy (LHON) and mitochondrial myopathy.
Slide 4: Chloroplast Inheritance
Chloroplasts: Organelles responsible for photosynthesis in plants.
Chloroplast DNA (cpDNA): Circular DNA molecule found in chloroplasts.
Inheritance Pattern: Often maternally inherited in most plants, but can vary in some species.
Examples: Variegation in plants, where leaf color patterns are determined by chloroplast DNA.
Slide 5: Plasmid Inheritance
Plasmids: Small, circular DNA molecules found in bacteria and some eukaryotes.
Features: Can carry antibiotic resistance genes and can be transferred between cells through processes like conjugation.
Significance: Important in biotechnology for gene cloning and genetic engineering.
Slide 6: Mechanisms of Extrachromosomal Inheritance
Non-Mendelian Patterns: Do not follow Mendel’s laws of inheritance.
Cytoplasmic Segregation: During cell division, organelles like mitochondria and chloroplasts are randomly distributed to daughter cells.
Heteroplasmy: Presence of more than one type of organellar genome within a cell, leading to variation in expression.
Slide 7: Examples of Extrachromosomal Inheritance
Four O’clock Plant (Mirabilis jalapa): Shows variegated leaves due to different cpDNA in leaf cells.
Petite Mutants in Yeast: Result from mutations in mitochondrial DNA affecting respiration.
Slide 8: Importance of Extrachromosomal Inheritance
Evolution: Provides insight into the evolution of eukaryotic cells.
Medicine: Understanding mitochondrial inheritance helps in diagnosing and treating mitochondrial diseases.
Agriculture: Chloroplast inheritance can be used in plant breeding and genetic modification.
Slide 9: Recent Research and Advances
Gene Editing: Techniques like CRISPR-Cas9 are being used to edit mitochondrial and chloroplast DNA.
Therapies: Development of mitochondrial replacement therapy (MRT) for preventing mitochondrial diseases.
Slide 10: Conclusion
Summary: Extrachromosomal inheritance involves the transmission of genetic material outside the nucleus and plays a crucial role in genetics, medicine, and biotechnology.
Future Directions: Continued research and technological advancements hold promise for new treatments and applications.
Slide 11: Questions and Discussion
Invite Audience: Open the floor for any questions or further discussion on the topic.
This presentation explores a brief idea about the structural and functional attributes of nucleotides, the structure and function of genetic materials along with the impact of UV rays and pH upon them.
What is greenhouse gasses and how many gasses are there to affect the Earth.moosaasad1975
What are greenhouse gasses how they affect the earth and its environment what is the future of the environment and earth how the weather and the climate effects.
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...Sérgio Sacani
Since volcanic activity was first discovered on Io from Voyager images in 1979, changes
on Io’s surface have been monitored from both spacecraft and ground-based telescopes.
Here, we present the highest spatial resolution images of Io ever obtained from a groundbased telescope. These images, acquired by the SHARK-VIS instrument on the Large
Binocular Telescope, show evidence of a major resurfacing event on Io’s trailing hemisphere. When compared to the most recent spacecraft images, the SHARK-VIS images
show that a plume deposit from a powerful eruption at Pillan Patera has covered part
of the long-lived Pele plume deposit. Although this type of resurfacing event may be common on Io, few have been detected due to the rarity of spacecraft visits and the previously low spatial resolution available from Earth-based telescopes. The SHARK-VIS instrument ushers in a new era of high resolution imaging of Io’s surface using adaptive
optics at visible wavelengths.
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...Sérgio Sacani
We characterize the earliest galaxy population in the JADES Origins Field (JOF), the deepest
imaging field observed with JWST. We make use of the ancillary Hubble optical images (5 filters
spanning 0.4−0.9µm) and novel JWST images with 14 filters spanning 0.8−5µm, including 7 mediumband filters, and reaching total exposure times of up to 46 hours per filter. We combine all our data
at > 2.3µm to construct an ultradeep image, reaching as deep as ≈ 31.4 AB mag in the stack and
30.3-31.0 AB mag (5σ, r = 0.1” circular aperture) in individual filters. We measure photometric
redshifts and use robust selection criteria to identify a sample of eight galaxy candidates at redshifts
z = 11.5 − 15. These objects show compact half-light radii of R1/2 ∼ 50 − 200pc, stellar masses of
M⋆ ∼ 107−108M⊙, and star-formation rates of SFR ∼ 0.1−1 M⊙ yr−1
. Our search finds no candidates
at 15 < z < 20, placing upper limits at these redshifts. We develop a forward modeling approach to
infer the properties of the evolving luminosity function without binning in redshift or luminosity that
marginalizes over the photometric redshift uncertainty of our candidate galaxies and incorporates the
impact of non-detections. We find a z = 12 luminosity function in good agreement with prior results,
and that the luminosity function normalization and UV luminosity density decline by a factor of ∼ 2.5
from z = 12 to z = 14. We discuss the possible implications of our results in the context of theoretical
models for evolution of the dark matter halo mass function.
Richard's entangled aventures in wonderlandRichard Gill
Since the loophole-free Bell experiments of 2020 and the Nobel prizes in physics of 2022, critics of Bell's work have retreated to the fortress of super-determinism. Now, super-determinism is a derogatory word - it just means "determinism". Palmer, Hance and Hossenfelder argue that quantum mechanics and determinism are not incompatible, using a sophisticated mathematical construction based on a subtle thinning of allowed states and measurements in quantum mechanics, such that what is left appears to make Bell's argument fail, without altering the empirical predictions of quantum mechanics. I think however that it is a smoke screen, and the slogan "lost in math" comes to my mind. I will discuss some other recent disproofs of Bell's theorem using the language of causality based on causal graphs. Causal thinking is also central to law and justice. I will mention surprising connections to my work on serial killer nurse cases, in particular the Dutch case of Lucia de Berk and the current UK case of Lucy Letby.
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...University of Maribor
Slides from:
11th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN), Niš, 3-6 June 2024
Track: Artificial Intelligence
https://www.etran.rs/2024/en/home-english/
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...
Misinformation on the internet: Video and AI
1. Misinformation on the internet:
Video and AI
Vasileios Mezaris
CERTH-ITI
Presentation at the "Age of misinformation: an interdisciplinary outlook on fake news" webinar
17th December 2020
2. Vasileios Mezaris, December 2020
Why focus on video?
• Powerful, sensational, persuasive medium
• “Believe nothing you hear, and only one half that you see.”
(Edgar Allan Poe, Nov. 1845)
3. Vasileios Mezaris, December 2020
Can I believe what I see?
• Is this video exactly as captured by the camera? (detection)
• What has been altered, and how do you know? (explanation)
• Why has it been altered? (understanding intent)
→ Intent is important; are the below edited videos carriers of
misinformation? Should they be banned from internet platforms?
Video credits: Media Forensics Challenge (MFC) dataset; GRIP, University Federico II, Naples
4. Vasileios Mezaris, December 2020
What is the role of AI?
• A powerful enabler of detecting and understanding video
manipulations
• But also a tool for promoting misinformation! (deepfakes; can
facilitate Distributed Denial of Truth (DDoT) attacks)
Prediction: Tampered Tampering localization / explanation (kind of…)
5. Vasileios Mezaris
Vasileios Mezaris, December 2020
InVID-WeVerify Verification Plugin: a “Swiss army knife” for
video verification
A browser plugin to debunk fake news
and to verify videos and images
>30.000
users
• Check prior video use: reverse video
search on the Web
• Check contextual information: Social-
media-based contextual analysis
• Keyframe/image inspection by
magnifying glass
• Check image (keyframe) forensics
Free! Get it from:
https://www.invid-project.eu/verify
Verifying videos: a browser plugin
H2020 IA, 2016-2018
Coordinator: CERTH
6. Vasileios Mezaris
Vasileios Mezaris, December 2020
A CERTH-ITI / InVID webpage for video fragmentation and reverse
web search; helps to quicky catch the “easy fakes” (try it!):
http://multimedia3.iti.gr/video_
fragmentation/service/start.html
Verifying videos: a web service
H2020 IA, 2016-2018
Coordinator: CERTH
7. Vasileios Mezaris
Vasileios Mezaris, December 2020
https://www.youtube.com/watch?v=OVAxQA3gMEo
“Video, the crowd in panic flees
from Notre Dame in Paris after
the attack of an armed man”
A video verification example
8. Vasileios Mezaris
Vasileios Mezaris, December 2020
AI4Media: A European Excellence Centre for Media, Society and
Democracy
• AI against disinformation is one of our use cases
• We are advancing the SoA in AI, in various directions (e.g. explainability, robustness)
More AI for verifying videos
H2020 RIA, 2020-2024
Coordinator: CERTH
9. Vasileios Mezaris
Vasileios Mezaris, December 2020
• Whether a video is a carrier of misinformation is not an easy yes/no question; may require
detecting & interpreting manipulations, and understanding intent
• Even relatively simple approaches, such as video fragmentation for reserse Web search, can
have great impact [1]
• AI is the enabler of more elaborate manipulation detection [2], but can also be a risk
[1] D. Teyssou, J.-M. Leung, E. Apostolidis, K. Apostolidis, S. Papadopoulos, M. Zampoglou, O. Papadopoulou, V. Mezaris, "The InVID Plug-in:
Web Video Verification on the Browser", Proc. Int. Workshop on Multimedia Verification (MuVer 2017) at ACM Multimedia 2017, Mountain
View, CA, USA, October 2017.
[2] G. Mercier, F. Markatopoulou, R. Cozien, M. Zampoglou, E. Apostolidis, A. Metsai, S. Papadopoulos, V. Mezaris, I. Patras, I. Kompatsiaris,
"Detecting Manipulations in Video", in book "Video Verification in the Fake News Era", V. Mezaris, L. Nixon, S. Papadopoulos, D. Teyssou
(Eds.), pp. 161-189, Springer, 2019.
[3] "Video verification in the fake news era", V. Mezaris, L. Nixon, S. Papadopoulos, D. Teyssou (Editors), Springer, 2019.
Concluding remarks
• Complete automation is not easy; often, AI+human symbiosis is the key [3]
• New challenges & advances in AI (e.g. Explainable AI) will create new
opportunities for even greater use of AI in verification (of video and more)
10. Vasileios Mezaris, December 2020
Questions?
Contact: Dr. Vasileios Mezaris
Research Director, Head of Intelligent Digital Transformation Laboratory
Information Technologies Institute (ITI) /
Centre for Research and Technology Hellas (CERTH)
Thermi-Thessaloniki, Greece
Email: bmezaris@iti.gr, web: http://www.iti.gr/~bmezaris/
This work was supported in part by the EU’s Horizon 2020 programme
grant agreements H2020-687786 InVID and H2020-951911 AI4Media