SlideShare a Scribd company logo

Video & AI: capabilities and limitations of AI in detecting video manipulations

This document discusses capabilities and limitations of AI in detecting video manipulations such as deepfakes. It describes how AI can help identify manipulated videos through techniques like reverse video search, near duplicate detection, and video forensics. While AI shows promise in combating disinformation, current methods still have limitations and complete automation remains a challenge. A portfolio of tools is needed since no single technique can fully solve the problem.

1 of 18
Download to read offline
Video & AI: capabilities and limitations of AI
in detecting video manipulations
Athens, Nov. 2019
Research Director, CERTH-ITI
Dr. Vasileios Mezaris
2
• Deep fakes are new…
• …but fakes are not a new phenomenon!
AI as facilitator of disinformation
Haiti earthquake video hoax (Jan. 2010): Video from a previous
earthquake in California published by major media, slightly modified
(cropped) and re-circulated in social media as a Haiti video; picked-
up and aired by mainstream media for hours
Presidential fake picture (Jan. 2013): Video frame of a surgery
operation in Mexico in 2008, was sold to and was mistakenly
published by a top newspaper, supposedly depicting the
Venezuelan president Hugo Chavez under surgery in a Cuban
hospital.
https://www.youtube.com/watch?v=JbzVhzNaTdI
3
• What makes deep fakes different?…
• Easier to create high-quality(?) fakes -> more people can do it
• Easier to create multiple fakes -> more media items can be created in
support of a disinformation campaign; can give rise to Distributed Denial of
Truth (DDoT) attacks
• …and what remains the same?
• A deep fake is usually an altered version of an original video; only the
means of introducing the alteration differs
• The techniques typically applied for detecting video-borne disinformation
are still useful (but there is also room for developing techniques specifically
tailored to detecting deep fakes)
AI as facilitator of disinformation
www.invid-project.eu
4
• How can AI help?
• Understand the structure of the video; select characteristic frames for
visual inspection or reverse search on the web
• Find near-duplicate videos, or semantically-related videos (of the same or
similar incidents)
• Video forensics
AI for combating disinformation
5
• Tool for verification of selected newsworthy videos
InVID Verification Plugin
• Free! Get it from: https://www.invid-project.eu/verify
Continues to be developed; now with new look in v0.71 (released June 7, 2019)
•
• Check prior video
use: reverse video
search on the Web
• Check contextual
information: Social-
media-based
contextual analysis
• View location, time
and other video
metadata
A browser plugin to debunk fake news
and to verify videos and images
>15.000
users
• Keyframe/image inspection by magnifying glass
• Check image (keyframe) forensics
6
• Tool for advanced video verification
• Check prior video use at the video-segment level:
InVID Verification Application
•
•
• Reverse video search also in the InVID repository; inspection by parallel playback
of query and duplicate video
• Check video origin and rights:
• Video logo detection
• Video rights management
• Check video forensics:
• Video forensic filters
• Frame-level video inspection
• Check additional contextual
information
• Store the automatic analysis results and manually add evaluation
comments and notes
• Free demo version accessible at: http://invid.condat.de

Recommended

Verifying Multimedia Content on the Internet
Verifying Multimedia Content on the InternetVerifying Multimedia Content on the Internet
Verifying Multimedia Content on the InternetSymeon Papadopoulos
 
Presentation of the InVID tools for image forensics analysis
Presentation of the InVID tools for image forensics analysisPresentation of the InVID tools for image forensics analysis
Presentation of the InVID tools for image forensics analysisInVID Project
 
The Inmates Are Running the Asylum: Why Some Multi-Factor Authentication Tech...
The Inmates Are Running the Asylum: Why Some Multi-Factor Authentication Tech...The Inmates Are Running the Asylum: Why Some Multi-Factor Authentication Tech...
The Inmates Are Running the Asylum: Why Some Multi-Factor Authentication Tech...Clare Nelson, CISSP, CIPP-E
 
DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...
DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...
DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...Symeon Papadopoulos
 

More Related Content

What's hot

Technologies and Policies for a Defensible Cyberspace
Technologies and Policies for a Defensible CyberspaceTechnologies and Policies for a Defensible Cyberspace
Technologies and Policies for a Defensible Cyberspacemark-smith
 
Kaseya Connect 2012 – A Kaspersky Researcher Perspective
Kaseya Connect 2012 – A Kaspersky Researcher PerspectiveKaseya Connect 2012 – A Kaspersky Researcher Perspective
Kaseya Connect 2012 – A Kaspersky Researcher PerspectiveKaseya
 
CSFI Stuxnet Report
CSFI Stuxnet ReportCSFI Stuxnet Report
CSFI Stuxnet ReportAmr Ali
 
Cees Snoek (UvA) @ CMC Video Formats
Cees Snoek (UvA) @ CMC Video FormatsCees Snoek (UvA) @ CMC Video Formats
Cees Snoek (UvA) @ CMC Video FormatsMedia Perspectives
 
Vulnerability Assessment, Physical Security, and Nuclear Safeguards
Vulnerability Assessment, Physical Security, and Nuclear SafeguardsVulnerability Assessment, Physical Security, and Nuclear Safeguards
Vulnerability Assessment, Physical Security, and Nuclear SafeguardsRoger Johnston
 
Symantec_2-4-5 nov 2010
Symantec_2-4-5 nov 2010Symantec_2-4-5 nov 2010
Symantec_2-4-5 nov 2010Agora Group
 
Zero-Knowledge Proofs: Identity Proofing and Authentication
Zero-Knowledge Proofs: Identity Proofing and AuthenticationZero-Knowledge Proofs: Identity Proofing and Authentication
Zero-Knowledge Proofs: Identity Proofing and AuthenticationClare Nelson, CISSP, CIPP-E
 

What's hot (8)

Butler
ButlerButler
Butler
 
Technologies and Policies for a Defensible Cyberspace
Technologies and Policies for a Defensible CyberspaceTechnologies and Policies for a Defensible Cyberspace
Technologies and Policies for a Defensible Cyberspace
 
Kaseya Connect 2012 – A Kaspersky Researcher Perspective
Kaseya Connect 2012 – A Kaspersky Researcher PerspectiveKaseya Connect 2012 – A Kaspersky Researcher Perspective
Kaseya Connect 2012 – A Kaspersky Researcher Perspective
 
CSFI Stuxnet Report
CSFI Stuxnet ReportCSFI Stuxnet Report
CSFI Stuxnet Report
 
Cees Snoek (UvA) @ CMC Video Formats
Cees Snoek (UvA) @ CMC Video FormatsCees Snoek (UvA) @ CMC Video Formats
Cees Snoek (UvA) @ CMC Video Formats
 
Vulnerability Assessment, Physical Security, and Nuclear Safeguards
Vulnerability Assessment, Physical Security, and Nuclear SafeguardsVulnerability Assessment, Physical Security, and Nuclear Safeguards
Vulnerability Assessment, Physical Security, and Nuclear Safeguards
 
Symantec_2-4-5 nov 2010
Symantec_2-4-5 nov 2010Symantec_2-4-5 nov 2010
Symantec_2-4-5 nov 2010
 
Zero-Knowledge Proofs: Identity Proofing and Authentication
Zero-Knowledge Proofs: Identity Proofing and AuthenticationZero-Knowledge Proofs: Identity Proofing and Authentication
Zero-Knowledge Proofs: Identity Proofing and Authentication
 

Similar to Video & AI: capabilities and limitations of AI in detecting video manipulations

Presentation of the InVID verification technologies at IPTC 2018
Presentation of the InVID verification technologies at IPTC 2018Presentation of the InVID verification technologies at IPTC 2018
Presentation of the InVID verification technologies at IPTC 2018InVID Project
 
The InVID Plug-in: Web Video Verification on the Browser
The InVID Plug-in: Web Video Verification on the BrowserThe InVID Plug-in: Web Video Verification on the Browser
The InVID Plug-in: Web Video Verification on the BrowserInVID Project
 
Aggregating and Analyzing the Context of Social Media Content
Aggregating and Analyzing the Context of Social Media ContentAggregating and Analyzing the Context of Social Media Content
Aggregating and Analyzing the Context of Social Media ContentSymeon Papadopoulos
 
VOGIN-IP-lezing-Zeno_ geradts
VOGIN-IP-lezing-Zeno_ geradtsVOGIN-IP-lezing-Zeno_ geradts
VOGIN-IP-lezing-Zeno_ geradtsvoginip
 
IRJET - Deepfake Video Detection using Image Processing and Hashing Tools
IRJET - Deepfake Video Detection using Image Processing and Hashing ToolsIRJET - Deepfake Video Detection using Image Processing and Hashing Tools
IRJET - Deepfake Video Detection using Image Processing and Hashing ToolsIRJET Journal
 
Overview of the InVID project
Overview of the InVID projectOverview of the InVID project
Overview of the InVID projectInVID Project
 
MINI PROJECT 2023 deepfake detection.pptx
MINI PROJECT 2023 deepfake detection.pptxMINI PROJECT 2023 deepfake detection.pptx
MINI PROJECT 2023 deepfake detection.pptxswathiravishankar3
 
Context Aggregation and Analysis: A tool for User-Generated Video Verification
Context Aggregation and Analysis: A tool for User-Generated Video VerificationContext Aggregation and Analysis: A tool for User-Generated Video Verification
Context Aggregation and Analysis: A tool for User-Generated Video VerificationOlga Papadopoulou
 
Context Aggregation and Analysis: A Tool for User- Generated Video Verificati...
Context Aggregation and Analysis: A Tool for User- Generated Video Verificati...Context Aggregation and Analysis: A Tool for User- Generated Video Verificati...
Context Aggregation and Analysis: A Tool for User- Generated Video Verificati...Weverify
 
A state of the art in journalism about fake image and video detection
A state of the art in journalism about fake image and video detectionA state of the art in journalism about fake image and video detection
A state of the art in journalism about fake image and video detectionInVID Project
 
A Neural Network Approach to Deep-Fake Video Detection
A Neural Network Approach to Deep-Fake Video DetectionA Neural Network Approach to Deep-Fake Video Detection
A Neural Network Approach to Deep-Fake Video DetectionIRJET Journal
 
Presentation of the InVID project and verification technologies
Presentation of the InVID project and verification technologiesPresentation of the InVID project and verification technologies
Presentation of the InVID project and verification technologiesInVID Project
 
A short introduction to multimedia forensics the science discovering the hist...
A short introduction to multimedia forensics the science discovering the hist...A short introduction to multimedia forensics the science discovering the hist...
A short introduction to multimedia forensics the science discovering the hist...Sebastiano Battiato
 
Deep fake video detection using machine learning.docx
Deep fake video detection using machine learning.docxDeep fake video detection using machine learning.docx
Deep fake video detection using machine learning.docxShakas Technologies
 
Implementation of-a-motion-detection-system
Implementation of-a-motion-detection-systemImplementation of-a-motion-detection-system
Implementation of-a-motion-detection-systemCemal Ardil
 
Rajarshi Gupta at AI Frontiers : Security is AI’s biggest challenge, AI is Se...
Rajarshi Gupta at AI Frontiers : Security is AI’s biggest challenge, AI is Se...Rajarshi Gupta at AI Frontiers : Security is AI’s biggest challenge, AI is Se...
Rajarshi Gupta at AI Frontiers : Security is AI’s biggest challenge, AI is Se...AI Frontiers
 
Kinect-like Devices: Principles, Brands and Cross-Applications
Kinect-like Devices: Principles, Brands and Cross-ApplicationsKinect-like Devices: Principles, Brands and Cross-Applications
Kinect-like Devices: Principles, Brands and Cross-ApplicationsVito Gentile
 
Context Aggregation and Analysis: A tool for User-Generated Video Verification
Context Aggregation and Analysis: A tool for User-Generated Video VerificationContext Aggregation and Analysis: A tool for User-Generated Video Verification
Context Aggregation and Analysis: A tool for User-Generated Video VerificationOlga Papadopoulou
 
Detecting Fake Videos: How Computational Archival Science Contribute to a Nov...
Detecting Fake Videos: How Computational Archival Science Contribute to a Nov...Detecting Fake Videos: How Computational Archival Science Contribute to a Nov...
Detecting Fake Videos: How Computational Archival Science Contribute to a Nov...Hoda Hamouda
 

Similar to Video & AI: capabilities and limitations of AI in detecting video manipulations (20)

Presentation of the InVID verification technologies at IPTC 2018
Presentation of the InVID verification technologies at IPTC 2018Presentation of the InVID verification technologies at IPTC 2018
Presentation of the InVID verification technologies at IPTC 2018
 
The InVID Plug-in: Web Video Verification on the Browser
The InVID Plug-in: Web Video Verification on the BrowserThe InVID Plug-in: Web Video Verification on the Browser
The InVID Plug-in: Web Video Verification on the Browser
 
Aggregating and Analyzing the Context of Social Media Content
Aggregating and Analyzing the Context of Social Media ContentAggregating and Analyzing the Context of Social Media Content
Aggregating and Analyzing the Context of Social Media Content
 
VOGIN-IP-lezing-Zeno_ geradts
VOGIN-IP-lezing-Zeno_ geradtsVOGIN-IP-lezing-Zeno_ geradts
VOGIN-IP-lezing-Zeno_ geradts
 
IRJET - Deepfake Video Detection using Image Processing and Hashing Tools
IRJET - Deepfake Video Detection using Image Processing and Hashing ToolsIRJET - Deepfake Video Detection using Image Processing and Hashing Tools
IRJET - Deepfake Video Detection using Image Processing and Hashing Tools
 
Overview of the InVID project
Overview of the InVID projectOverview of the InVID project
Overview of the InVID project
 
MINI PROJECT 2023 deepfake detection.pptx
MINI PROJECT 2023 deepfake detection.pptxMINI PROJECT 2023 deepfake detection.pptx
MINI PROJECT 2023 deepfake detection.pptx
 
Context Aggregation and Analysis: A tool for User-Generated Video Verification
Context Aggregation and Analysis: A tool for User-Generated Video VerificationContext Aggregation and Analysis: A tool for User-Generated Video Verification
Context Aggregation and Analysis: A tool for User-Generated Video Verification
 
Context Aggregation and Analysis: A Tool for User- Generated Video Verificati...
Context Aggregation and Analysis: A Tool for User- Generated Video Verificati...Context Aggregation and Analysis: A Tool for User- Generated Video Verificati...
Context Aggregation and Analysis: A Tool for User- Generated Video Verificati...
 
A state of the art in journalism about fake image and video detection
A state of the art in journalism about fake image and video detectionA state of the art in journalism about fake image and video detection
A state of the art in journalism about fake image and video detection
 
A Neural Network Approach to Deep-Fake Video Detection
A Neural Network Approach to Deep-Fake Video DetectionA Neural Network Approach to Deep-Fake Video Detection
A Neural Network Approach to Deep-Fake Video Detection
 
Malware goes to the movies
Malware goes to the moviesMalware goes to the movies
Malware goes to the movies
 
Presentation of the InVID project and verification technologies
Presentation of the InVID project and verification technologiesPresentation of the InVID project and verification technologies
Presentation of the InVID project and verification technologies
 
A short introduction to multimedia forensics the science discovering the hist...
A short introduction to multimedia forensics the science discovering the hist...A short introduction to multimedia forensics the science discovering the hist...
A short introduction to multimedia forensics the science discovering the hist...
 
Deep fake video detection using machine learning.docx
Deep fake video detection using machine learning.docxDeep fake video detection using machine learning.docx
Deep fake video detection using machine learning.docx
 
Implementation of-a-motion-detection-system
Implementation of-a-motion-detection-systemImplementation of-a-motion-detection-system
Implementation of-a-motion-detection-system
 
Rajarshi Gupta at AI Frontiers : Security is AI’s biggest challenge, AI is Se...
Rajarshi Gupta at AI Frontiers : Security is AI’s biggest challenge, AI is Se...Rajarshi Gupta at AI Frontiers : Security is AI’s biggest challenge, AI is Se...
Rajarshi Gupta at AI Frontiers : Security is AI’s biggest challenge, AI is Se...
 
Kinect-like Devices: Principles, Brands and Cross-Applications
Kinect-like Devices: Principles, Brands and Cross-ApplicationsKinect-like Devices: Principles, Brands and Cross-Applications
Kinect-like Devices: Principles, Brands and Cross-Applications
 
Context Aggregation and Analysis: A tool for User-Generated Video Verification
Context Aggregation and Analysis: A tool for User-Generated Video VerificationContext Aggregation and Analysis: A tool for User-Generated Video Verification
Context Aggregation and Analysis: A tool for User-Generated Video Verification
 
Detecting Fake Videos: How Computational Archival Science Contribute to a Nov...
Detecting Fake Videos: How Computational Archival Science Contribute to a Nov...Detecting Fake Videos: How Computational Archival Science Contribute to a Nov...
Detecting Fake Videos: How Computational Archival Science Contribute to a Nov...
 

More from VasileiosMezaris

Multi-Modal Fusion for Image Manipulation Detection and Localization
Multi-Modal Fusion for Image Manipulation Detection and LocalizationMulti-Modal Fusion for Image Manipulation Detection and Localization
Multi-Modal Fusion for Image Manipulation Detection and LocalizationVasileiosMezaris
 
CERTH-ITI at MediaEval 2023 NewsImages Task
CERTH-ITI at MediaEval 2023 NewsImages TaskCERTH-ITI at MediaEval 2023 NewsImages Task
CERTH-ITI at MediaEval 2023 NewsImages TaskVasileiosMezaris
 
Spatio-Temporal Summarization of 360-degrees Videos
Spatio-Temporal Summarization of 360-degrees VideosSpatio-Temporal Summarization of 360-degrees Videos
Spatio-Temporal Summarization of 360-degrees VideosVasileiosMezaris
 
TAME: Trainable Attention Mechanism for Explanations
TAME: Trainable Attention Mechanism for ExplanationsTAME: Trainable Attention Mechanism for Explanations
TAME: Trainable Attention Mechanism for ExplanationsVasileiosMezaris
 
Explaining video summarization based on the focus of attention
Explaining video summarization based on the focus of attentionExplaining video summarization based on the focus of attention
Explaining video summarization based on the focus of attentionVasileiosMezaris
 
Combining textual and visual features for Ad-hoc Video Search
Combining textual and visual features for Ad-hoc Video SearchCombining textual and visual features for Ad-hoc Video Search
Combining textual and visual features for Ad-hoc Video SearchVasileiosMezaris
 
Explaining the decisions of image/video classifiers
Explaining the decisions of image/video classifiersExplaining the decisions of image/video classifiers
Explaining the decisions of image/video classifiersVasileiosMezaris
 
Learning visual explanations for DCNN-based image classifiers using an attent...
Learning visual explanations for DCNN-based image classifiers using an attent...Learning visual explanations for DCNN-based image classifiers using an attent...
Learning visual explanations for DCNN-based image classifiers using an attent...VasileiosMezaris
 
Are all combinations equal? Combining textual and visual features with multi...
Are all combinations equal?  Combining textual and visual features with multi...Are all combinations equal?  Combining textual and visual features with multi...
Are all combinations equal? Combining textual and visual features with multi...VasileiosMezaris
 
CA-SUM Video Summarization
CA-SUM Video SummarizationCA-SUM Video Summarization
CA-SUM Video SummarizationVasileiosMezaris
 
Video smart cropping web application
Video smart cropping web applicationVideo smart cropping web application
Video smart cropping web applicationVasileiosMezaris
 
PGL SUM Video Summarization
PGL SUM Video SummarizationPGL SUM Video Summarization
PGL SUM Video SummarizationVasileiosMezaris
 
Hard-Negatives Selection Strategy for Cross-Modal Retrieval
Hard-Negatives Selection Strategy for Cross-Modal RetrievalHard-Negatives Selection Strategy for Cross-Modal Retrieval
Hard-Negatives Selection Strategy for Cross-Modal RetrievalVasileiosMezaris
 
Misinformation on the internet: Video and AI
Misinformation on the internet: Video and AIMisinformation on the internet: Video and AI
Misinformation on the internet: Video and AIVasileiosMezaris
 
PoR_evaluation_measure_acm_mm_2020
PoR_evaluation_measure_acm_mm_2020PoR_evaluation_measure_acm_mm_2020
PoR_evaluation_measure_acm_mm_2020VasileiosMezaris
 
GAN-based video summarization
GAN-based video summarizationGAN-based video summarization
GAN-based video summarizationVasileiosMezaris
 
Migration-related video retrieval
Migration-related video retrievalMigration-related video retrieval
Migration-related video retrievalVasileiosMezaris
 

More from VasileiosMezaris (20)

Multi-Modal Fusion for Image Manipulation Detection and Localization
Multi-Modal Fusion for Image Manipulation Detection and LocalizationMulti-Modal Fusion for Image Manipulation Detection and Localization
Multi-Modal Fusion for Image Manipulation Detection and Localization
 
CERTH-ITI at MediaEval 2023 NewsImages Task
CERTH-ITI at MediaEval 2023 NewsImages TaskCERTH-ITI at MediaEval 2023 NewsImages Task
CERTH-ITI at MediaEval 2023 NewsImages Task
 
Spatio-Temporal Summarization of 360-degrees Videos
Spatio-Temporal Summarization of 360-degrees VideosSpatio-Temporal Summarization of 360-degrees Videos
Spatio-Temporal Summarization of 360-degrees Videos
 
TAME: Trainable Attention Mechanism for Explanations
TAME: Trainable Attention Mechanism for ExplanationsTAME: Trainable Attention Mechanism for Explanations
TAME: Trainable Attention Mechanism for Explanations
 
Gated-ViGAT
Gated-ViGATGated-ViGAT
Gated-ViGAT
 
Explaining video summarization based on the focus of attention
Explaining video summarization based on the focus of attentionExplaining video summarization based on the focus of attention
Explaining video summarization based on the focus of attention
 
Combining textual and visual features for Ad-hoc Video Search
Combining textual and visual features for Ad-hoc Video SearchCombining textual and visual features for Ad-hoc Video Search
Combining textual and visual features for Ad-hoc Video Search
 
Explaining the decisions of image/video classifiers
Explaining the decisions of image/video classifiersExplaining the decisions of image/video classifiers
Explaining the decisions of image/video classifiers
 
Learning visual explanations for DCNN-based image classifiers using an attent...
Learning visual explanations for DCNN-based image classifiers using an attent...Learning visual explanations for DCNN-based image classifiers using an attent...
Learning visual explanations for DCNN-based image classifiers using an attent...
 
Are all combinations equal? Combining textual and visual features with multi...
Are all combinations equal?  Combining textual and visual features with multi...Are all combinations equal?  Combining textual and visual features with multi...
Are all combinations equal? Combining textual and visual features with multi...
 
CA-SUM Video Summarization
CA-SUM Video SummarizationCA-SUM Video Summarization
CA-SUM Video Summarization
 
Video smart cropping web application
Video smart cropping web applicationVideo smart cropping web application
Video smart cropping web application
 
PGL SUM Video Summarization
PGL SUM Video SummarizationPGL SUM Video Summarization
PGL SUM Video Summarization
 
Video Thumbnail Selector
Video Thumbnail SelectorVideo Thumbnail Selector
Video Thumbnail Selector
 
Hard-Negatives Selection Strategy for Cross-Modal Retrieval
Hard-Negatives Selection Strategy for Cross-Modal RetrievalHard-Negatives Selection Strategy for Cross-Modal Retrieval
Hard-Negatives Selection Strategy for Cross-Modal Retrieval
 
Misinformation on the internet: Video and AI
Misinformation on the internet: Video and AIMisinformation on the internet: Video and AI
Misinformation on the internet: Video and AI
 
LSTM Structured Pruning
LSTM Structured PruningLSTM Structured Pruning
LSTM Structured Pruning
 
PoR_evaluation_measure_acm_mm_2020
PoR_evaluation_measure_acm_mm_2020PoR_evaluation_measure_acm_mm_2020
PoR_evaluation_measure_acm_mm_2020
 
GAN-based video summarization
GAN-based video summarizationGAN-based video summarization
GAN-based video summarization
 
Migration-related video retrieval
Migration-related video retrievalMigration-related video retrieval
Migration-related video retrieval
 

Recently uploaded

dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...dkNET
 
Chemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptxChemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptxperiyar arts college
 
Anti-Obesity Activity of Anthocyanins and Corresponding Introduction in Dieta...
Anti-Obesity Activity of Anthocyanins and Corresponding Introduction in Dieta...Anti-Obesity Activity of Anthocyanins and Corresponding Introduction in Dieta...
Anti-Obesity Activity of Anthocyanins and Corresponding Introduction in Dieta...AmalDhivaharS
 
Weak-lensing detection of intracluster filaments in the Coma cluster
Weak-lensing detection of intracluster filaments in the Coma clusterWeak-lensing detection of intracluster filaments in the Coma cluster
Weak-lensing detection of intracluster filaments in the Coma clusterSérgio Sacani
 
Presentation for first doctoral seminar on Advances in poultry nutrition.pptx
Presentation for first doctoral seminar on Advances in poultry nutrition.pptxPresentation for first doctoral seminar on Advances in poultry nutrition.pptx
Presentation for first doctoral seminar on Advances in poultry nutrition.pptxPallaviMali14
 
electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.Silpa Selvaraj
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisLinaMarcelaCharrisRa
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisLinaMarcelaCharrisRa
 
American Eclipse A Nation’s Epic Race to Catch the_240225_095603
American Eclipse A Nation’s Epic Race to Catch the_240225_095603American Eclipse A Nation’s Epic Race to Catch the_240225_095603
American Eclipse A Nation’s Epic Race to Catch the_240225_095603SOCIEDAD JULIO GARAVITO
 
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptx
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptxCHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptx
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptxASWIN ANANDH
 
Open Access Publishing in Astrophysics and the Open Journal of Astrophysics
Open Access Publishing in Astrophysics and the Open Journal of AstrophysicsOpen Access Publishing in Astrophysics and the Open Journal of Astrophysics
Open Access Publishing in Astrophysics and the Open Journal of AstrophysicsPeter Coles
 
Differences between syrups and elixirs .pptx
Differences between  syrups and elixirs .pptxDifferences between  syrups and elixirs .pptx
Differences between syrups and elixirs .pptxushakiranmai4
 
Genetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdfGenetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdfmughalgumar440
 
Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)Ahmed Metwaly
 
A tale of two Lucys - Delft lecture - March 4, 2024
A tale of two Lucys - Delft lecture - March 4, 2024A tale of two Lucys - Delft lecture - March 4, 2024
A tale of two Lucys - Delft lecture - March 4, 2024Richard Gill
 
the menstrual cycle in female reproductive system
the menstrual cycle in female reproductive systemthe menstrual cycle in female reproductive system
the menstrual cycle in female reproductive systemGilmeTripole1
 
Presentacion Mariana Arango- biología molecular
Presentacion Mariana Arango- biología molecularPresentacion Mariana Arango- biología molecular
Presentacion Mariana Arango- biología molecularmarianaarangop
 
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...Uzay Emir
 
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdf
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdfUKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdf
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdfThane Heins
 
Ento-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture useeEnto-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture useeDrAnita Sharma
 

Recently uploaded (20)

dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
dkNET Webinar: The Collaborative Microbial Metabolite Center – Democratizing ...
 
Chemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptxChemical Bonding and it's Types 001.pptx
Chemical Bonding and it's Types 001.pptx
 
Anti-Obesity Activity of Anthocyanins and Corresponding Introduction in Dieta...
Anti-Obesity Activity of Anthocyanins and Corresponding Introduction in Dieta...Anti-Obesity Activity of Anthocyanins and Corresponding Introduction in Dieta...
Anti-Obesity Activity of Anthocyanins and Corresponding Introduction in Dieta...
 
Weak-lensing detection of intracluster filaments in the Coma cluster
Weak-lensing detection of intracluster filaments in the Coma clusterWeak-lensing detection of intracluster filaments in the Coma cluster
Weak-lensing detection of intracluster filaments in the Coma cluster
 
Presentation for first doctoral seminar on Advances in poultry nutrition.pptx
Presentation for first doctoral seminar on Advances in poultry nutrition.pptxPresentation for first doctoral seminar on Advances in poultry nutrition.pptx
Presentation for first doctoral seminar on Advances in poultry nutrition.pptx
 
electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.electrophoresis: types, advantages, disadvantages and applications.
electrophoresis: types, advantages, disadvantages and applications.
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina Charris
 
Seminario biología molecular Lina Charris
Seminario biología molecular Lina CharrisSeminario biología molecular Lina Charris
Seminario biología molecular Lina Charris
 
American Eclipse A Nation’s Epic Race to Catch the_240225_095603
American Eclipse A Nation’s Epic Race to Catch the_240225_095603American Eclipse A Nation’s Epic Race to Catch the_240225_095603
American Eclipse A Nation’s Epic Race to Catch the_240225_095603
 
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptx
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptxCHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptx
CHEMICAL TESTS FOR GLYCOSIDES AND ALKALOIDS.pptx
 
Open Access Publishing in Astrophysics and the Open Journal of Astrophysics
Open Access Publishing in Astrophysics and the Open Journal of AstrophysicsOpen Access Publishing in Astrophysics and the Open Journal of Astrophysics
Open Access Publishing in Astrophysics and the Open Journal of Astrophysics
 
Differences between syrups and elixirs .pptx
Differences between  syrups and elixirs .pptxDifferences between  syrups and elixirs .pptx
Differences between syrups and elixirs .pptx
 
Genetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdfGenetic Code. A comprehensive overview..pdf
Genetic Code. A comprehensive overview..pdf
 
Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)Introduction to Chromatography (Column chromatography)
Introduction to Chromatography (Column chromatography)
 
A tale of two Lucys - Delft lecture - March 4, 2024
A tale of two Lucys - Delft lecture - March 4, 2024A tale of two Lucys - Delft lecture - March 4, 2024
A tale of two Lucys - Delft lecture - March 4, 2024
 
the menstrual cycle in female reproductive system
the menstrual cycle in female reproductive systemthe menstrual cycle in female reproductive system
the menstrual cycle in female reproductive system
 
Presentacion Mariana Arango- biología molecular
Presentacion Mariana Arango- biología molecularPresentacion Mariana Arango- biología molecular
Presentacion Mariana Arango- biología molecular
 
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
Lung imaging Using 3D Dual-Echo FID Ultra-short Echo Time MRI with novel Rose...
 
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdf
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdfUKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdf
UKRAINIAN Kyiv Institute of Thermophysics Memorandum_English.pdf
 
Ento-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture useeEnto-322, Agrochemicals for agriculture usee
Ento-322, Agrochemicals for agriculture usee
 

Video & AI: capabilities and limitations of AI in detecting video manipulations

  • 1. Video & AI: capabilities and limitations of AI in detecting video manipulations Athens, Nov. 2019 Research Director, CERTH-ITI Dr. Vasileios Mezaris
  • 2. 2 • Deep fakes are new… • …but fakes are not a new phenomenon! AI as facilitator of disinformation Haiti earthquake video hoax (Jan. 2010): Video from a previous earthquake in California published by major media, slightly modified (cropped) and re-circulated in social media as a Haiti video; picked- up and aired by mainstream media for hours Presidential fake picture (Jan. 2013): Video frame of a surgery operation in Mexico in 2008, was sold to and was mistakenly published by a top newspaper, supposedly depicting the Venezuelan president Hugo Chavez under surgery in a Cuban hospital. https://www.youtube.com/watch?v=JbzVhzNaTdI
  • 3. 3 • What makes deep fakes different?… • Easier to create high-quality(?) fakes -> more people can do it • Easier to create multiple fakes -> more media items can be created in support of a disinformation campaign; can give rise to Distributed Denial of Truth (DDoT) attacks • …and what remains the same? • A deep fake is usually an altered version of an original video; only the means of introducing the alteration differs • The techniques typically applied for detecting video-borne disinformation are still useful (but there is also room for developing techniques specifically tailored to detecting deep fakes) AI as facilitator of disinformation www.invid-project.eu
  • 4. 4 • How can AI help? • Understand the structure of the video; select characteristic frames for visual inspection or reverse search on the web • Find near-duplicate videos, or semantically-related videos (of the same or similar incidents) • Video forensics AI for combating disinformation
  • 5. 5 • Tool for verification of selected newsworthy videos InVID Verification Plugin • Free! Get it from: https://www.invid-project.eu/verify Continues to be developed; now with new look in v0.71 (released June 7, 2019) • • Check prior video use: reverse video search on the Web • Check contextual information: Social- media-based contextual analysis • View location, time and other video metadata A browser plugin to debunk fake news and to verify videos and images >15.000 users • Keyframe/image inspection by magnifying glass • Check image (keyframe) forensics
  • 6. 6 • Tool for advanced video verification • Check prior video use at the video-segment level: InVID Verification Application • • • Reverse video search also in the InVID repository; inspection by parallel playback of query and duplicate video • Check video origin and rights: • Video logo detection • Video rights management • Check video forensics: • Video forensic filters • Frame-level video inspection • Check additional contextual information • Store the automatic analysis results and manually add evaluation comments and notes • Free demo version accessible at: http://invid.condat.de
  • 7. 7 InVID Verification Plugin – Reverse search https://www.youtube.com/watch?v=OVAxQA3gMEo “Video, the crowd in panic flees from Notre Dame in Paris after the attack of an armed man”
  • 8. 8 InVID Verification Plugin – Reverse search https://www.youtube.com/watch?v=OVAxQA3gMEo “Video, the crowd in panic flees from Notre Dame in Paris after the attack of an armed man”
  • 9. 9 InVID Verification Plugin – Reverse search https://www.youtube.com/watch?v=OVAxQA3gMEo “Video, the crowd in panic flees from Notre Dame in Paris after the attack of an armed man”
  • 10. 10 InVID Verification Plugin – Reverse search https://www.youtube.com/watch?v=OVAxQA3gMEo “Video, the crowd in panic flees from Notre Dame in Paris after the attack of an armed man”
  • 11. 11 InVID Verification Plugin – Reverse search https://www.youtube.com/watch?v=OVAxQA3gMEo “Video, the crowd in panic flees from Notre Dame in Paris after the attack of an armed man”
  • 12. 12 • Video forensics – a quick AI-based approach • Filters generating heat maps (e.g. Cobalt: compares the original video with a modified version of it, re-quantized using MPEG-4 at a different quality level, and generates the error video) • Use deep learning architectures (e.g. GoogLeNet, ResNet) to learn from both the video itself, and the outputs of these filters • No knowledge of the generator’s characteristics is assumed • Results are promising, but not a complete, deployable solution yet [1][2] AI-based video forensics [1] M. Zampoglou, F. Markatopoulou, G. Mercier, D. Touska, E. Apostolidis, S. Papadopoulos, R. Cozien, I. Patras, V. Mezaris, I. Kompatsiaris, "Detecting tampered videos with multimedia forensics and deep learning", Proc. 25th Int. Conf. on Multimedia Modeling (MMM2019), Thessaloniki, Greece, Springer LNCS vol. 11295, pp. 374-386, Jan. 2019. [2] "Video verification in the fake news era", V. Mezaris, L. Nixon, S. Papadopoulos, D. Teyssou (Editors), Springer, 2019.
  • 13. 13 AI-based video forensics Detected Fakes Fakes not detected (lower resolution) Videos from the InVID Fake Video Corpus 1280x720p480x360p 320x240p 320x240p
  • 14. 14 • Video forensics – a steganalysis- & AI-based approach • Steganalysis features are extracted from each frame [3] • Sliding windows of 128x128 pixels are considered • High pass filtering of pixel values, then calculate and quantize residuals; co- occurrence matrices are calculated • A recurrent autoencoder is trained on pristine frames of the video • The trained autoencoder assesses the remaining frames of the video • Forged spatial regions in manipulated frames are singled out as anomalous • Good results plus localization of the forgery; but sensitive to encoding AI-based video forensics [3] D. D’Avino, D. Cozzolino, G. Poggi, L. Verdoliva, “Autoencoder with recurrent neural networks for video forgery detection”. In: Electronic Imaging 2017.7 (2017), pp. 92–99.
  • 15. 15 • Video forensics – a steganalysis- & AI-based approach AI-based video forensics Video from the UNINA dataset http://www.grip.unina.it/
  • 16. 16 • Looking at the greater picture – directions & thoughts on deep fakes and AI-based video forensics • Manipulated videos and deep fakes still usually come from an original: if you find the original out there, you have documented the faking • Reverse video search is still a powerful tool • AI has great potential; is already used in different techniques for combatting disinformation, and will be used even more for video forensics and detecting deep fakes • But, no single tech seems to be the solution; we need a portfolio of tools [4] • Complete automation is a challenge! AI-based video forensics [4] "Video verification in the fake news era", V. Mezaris, L. Nixon, S. Papadopoulos, D. Teyssou (Editors), Springer, 2019.
  • 17. 17 Dr. Vasileios Mezaris Information Technologies Institute Centre for Research and Technology Hellas Thermi-Thessaloniki, Greece Tel: +30 2311 257770 Email: bmezaris@iti.gr, web: http://www.iti.gr/~bmezaris/ For more information on the InVID project, visit www.invid-project.eu Follow InVID on Twitter @InVID_EU Contact
  • 18. 18 This work was supported in part by the EU’s Horizon 2020 research and innovation programme under grant agreement H2020-687786 InVID. Acknowledgement