Presentation of the developed web application that allows the user to fragment a video into visually coherent parts, extract a number of representative keyframes, and apply a keyframe-based reverse search at the video-fragment level.

1. www.invid-project.eu
In Video Veritas – Verification of Social Media
Video Content for the News Industry
Evlampios Apostolidis, CERTH-ITI
Fake news based on video reuse and how to deal with
it: video fragmentation and reverse image search
Thessaloniki, June 2018

2. • How does this fake work?
• A previously existing video is reused under a different and irrelevant
context, aiming to deliberately mislead the viewers about a fact/event
Fake news based on video reuse
www.invid-project.eu

3. • How does this fake work?
• A previously existing video is reused under a different and irrelevant
context, aiming to deliberately mislead the viewers about a fact/event
Fake news based on video reuse
19/3/2017; Claim: Migrant seeking free
healthcare in a public hospital in France
www.invid-project.eu

4. • How does this fake work?
• A previously existing video is reused under a different and irrelevant
context, aiming to deliberately mislead the viewers about a fact/event
Fake news based on video reuse
19/3/2017; Claim: Migrant seeking free
healthcare in a public hospital in France
26/2/2017; Claim: Drunk patient
in a hospital in Novgorod, Russia
www.invid-project.eu

5. • How journalists deal with this kind of fake news?
• Try to find previous occurences of the video on the Web
Dealing with this type of fakes
www.invid-project.eu

6. • How journalists deal with this kind of fake news?
• Try to find previous occurences of the video on the Web
• What processes/tools do they follow/use for performing reverse
video search?
Dealing with this type of fakes
www.invid-project.eu

7. • How journalists deal with this kind of fake news?
• Try to find previous occurences of the video on the Web
• What processes/tools do they follow/use for performing reverse
video search?
• Take screenshots of the video and perform reverse search of these
screenshots with the help of the Google Images engine
Dealing with this type of fakes
www.invid-project.eu

8. • How journalists deal with this kind of fake news?
• Try to find previous occurences of the video on the Web
• What processes/tools do they follow/use for performing reverse
video search?
• Use of search-engine-based web applications or plug-ins, such as TinEye1
and RevEye2 that facilitate the reverse search of images related to the video
Dealing with this type of fakes
www.invid-project.eu
1. https://tineye.com/ and https://goo.gl/G5Rpu9
2. https://goo.gl/ZRHTDH

9. • How journalists deal with this kind of fake news?
• Try to find previous occurences of the video on the Web
• What processes/tools do they follow/use for performing reverse
video search?
• Use of web applications that allow reverse search of videos based on a
(limited) set of video thumbnails, such as the YouTube DataViewer1 and the
Custom Reverse Image Search2
Dealing with this type of fakes
www.invid-project.eu
1. https://citizenevidence.amnestyusa.org/
2. https://inteltechniques.com/osint/reverse.video.html

10. • How journalists deal with this kind of fake news?
• Try to find previous occurences of the video on the Web
• What processes/tools do they follow/use for performing reverse
video search?
• Time-consuming and cumbersome processes that:
• either involve manual generation and uploading of video screenshots
• or rely on the use of a limited set of video thumbnails
Dealing with this type of fakes
www.invid-project.eu

11. • Interactive tool for reverse video search on the Web
• Time-efficient process that requires minimum manual
intervention
• Fine-grained search at the video-fragment-level, through:
• segmentation of the video into visually coherent fragments
• extraction of representative keyframes for each video fragment
• reverse search of these keyframes via the Google search engine
The InVID solution
www.invid-project.eu

12. • User-generated videos (UGVs) are captured without interruption
using a single camera, thus, being single-shot videos
• Algorithms for shot boundary detection fail to reveal information
about the structure of these videos
• A more fine-grained segmentation into sub-shots, is needed!
Video fragmentation & keyframe selection
www.invid-project.eu

13. • The InVID approach
• The visual content of each frame is represented with the help of a 2D
Discrete Cosine Transform
• Video fragmentation into sub-shots is performed by assessing the visual
resemblance of neighboring frames using the cosine similarity
• The algorithm indicates both sub-shots with minor or no activity, and sub-
shots with gradually, but consistently, changing visual content
Video fragmentation & keyframe selection
• As representative keyframe:
• for the former type of sub-shots the
middle frame is selected
• for the latter type of sub-shots the
frame with the most pronounced
change of visual content is selected
• The analysis takes approx. 3% of the
video’s duration (being more than 30
faster than real-time processing)
www.invid-project.eu

14. The reverse video search tool
• Available at: http://multimedia3.iti.gr/video_fragmentation/
service/start.html
• Allows the analysis of both
online and locally stored videos
• Supported platforms: YouTube,
Facebook, Twitter, Instagram,
Vimeo, LiveLeak, DailyMotion and
Dropbox
• Supported video formats: mp4,
webm, avi, mov, wmv, ogv, mpg,
flv, mkv
www.invid-project.eu

15. The reverse video search tool
• The user can monitor the progress of the analysis, or close the
browser and be notified (by e-mail) when the results are ready
www.invid-project.eu

16. The reverse video search tool
• After the analysis ends, the user gets the set of extracted
keyframes and can perform reverse search by left or right
clicking on any of them
www.invid-project.eu

17. Thank you for your attention!
Any questions?
www.invid-project.eu
In Video Veritas!