Presentation at the NYC Media Lab (NYCML2018). There is a growing demand for news videos online, with more consumers preferring to watch the news than read or listen to it. On the publisher side, there is a growing effort to use video summarization technology in order to create easy-to-consume previews (trailers) for different types of broadcast programs. How can we measure the quality of video summaries and their potential to misinform? This workshop will inform participants about automatic video summarization algorithms and how to produce more “representative” video summaries. The research presented is from the FAIRview project and is supported by the Digital News Innovation Fund (DNI Fund), which is part of the Google News Initiative.

1. FAIRview:
Responsible Video
Summarization
/ Lora Aroyo
/ Tagasauris Inc
/ http://lora-aroyo.org
/ @laroyo

2. Agenda for today
3:00 - 3:20: Introduction of Video Summarization Context
3:20 - 3:50: Work in groups to answer the following questions
(discussion document: http://bit.ly/fairview_discussion)
Q1: How to increase the user awareness (e.g. through explanations, visualizations,
interaction, etc) on the following two points:
○ the video summary “representativeness” compared to the original video
○ the (possible) video summary “misinformation potential” compared to the original video
Q2: What are adequate success metrics for video summaries?
○ How to measure the ‘representativeness’?
○ How to measure ‘misinformation potential’?
○ How to evaluate both points?
Answer these questions in the following interaction scenarios:
● while watching the video summary
● when browsing video search results
● when comparing two or more video summaries
● when creating the video summary
● other interaction scenarios
3:50 - 4:00: Summary and conclusions
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

3. video is 64% of Internet traffic
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

4. more Americans prefer to watch their news (46%)
than to read it (35%) or listen to it (17%)
http://www.journalism.org/2016/07/07/pathways-to-news/
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

5. 300h of video uploaded each min on YouTube alone
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

6. in 2020 it would take a person more than 5 million
years to watch the videos uploaded in a month
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

7. at some point it all looks the same
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

8. … tons of videos but difficult to choose what to watch
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

9. how to make videos consumable
in the age of information overload & declining attention span?

10. Slide credit: @jess3 @slideshare
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

11. … video snacks = the new attention economy?
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

12. … opportunities of snackable content
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

13. … e.g. personalized thumbnails & previews
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

14. … e.g. 4-thumbnail summary in video search results
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

15. http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
…e.g. bumper ads & previews in video search results

16. … e.g. micro-moments in video
for on-demand discovery search
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

17. … e.g. contextualized hyperlinks in video
for direct engagement
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

18. … creating bite size info nuggets (video snacks)
that can quickly be consumed, understood & shared
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

19. Let’s look at an example
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

20. Scenes from HBO Series: Big Little Lies
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

21. Scenes from 1 Episode
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

22. Selected: 1 Scene
Selected: 1 Scene
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

23. Frames for the selected sceneSelected: 1 Scene
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

24. All Concepts describing the FrameSelected: 1 Frame
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

25. all this results in a lot of
video, image and label data
… that could be organized in lots of different storylines
i.e. video snacks
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

26. cars
kids
nature
guns hugs
TOPICS
The (infinite) stories
you can tell with data ...
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

27. cars
kids
nature
guns hugs
FORMATS
4-Frames Preview
6-Sec Trailer
Adaptive
Starting Frames
Skimming
Static
Dynamic
The (infinite) stories
you can tell with data ...
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

28. cars
kids
nature
guns hugs
INTERACTIONS
Hyperlinks
E-commerce links
Looping
Autoplay
Recommendation
Canvas
locate
buy
learnname
The (infinite) stories
you can tell with data ...
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

29. http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

30. … but how are these video stories created?
who selects what to include / exclude?
who chooses the summarization approaches?
what is the impact of different approaches?
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

31. … all these choices can amplify / diminish
a specific aspect or perspective in the original video,
and in this way introduce a bias
that can potentially lead to misinformation
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

32. FAIRView
● how to bring more awareness of all the perspectives,
topics and elements present in the original video
● what are indicators & evaluation criteria on how
these are represented in a video summary
● how to adapt existing summarization algorithms to
produce representative & explainable video
summaries
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

33. FAIRView
● study this problem in the context of news videos
● empower users with tools to evaluate
representativeness of videos
● gain a granular understanding of video content in
terms of perspectives, opinions, stories, etc.
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

34. Work in Groups
3:20 - 3:50: Work in groups to answer the following questions
(discussion document: http://bit.ly/fairview_discussion)
Q1: How to increase the user awareness (e.g. through explanations, visualizations,
interaction, etc) on the following two points:
○ the video summary “representativeness” compared to the original video
○ the (possible) video summary “misinformation potential” compared to the original video
Q2: What are adequate success metrics for video summaries?
○ How to measure the ‘representativeness’?
○ How to measure ‘misinformation potential’?
○ How to evaluate both points?
Answer these questions in the following interaction scenarios:
● while watching the video summary
● when browsing video search results
● when comparing two or more video summaries
● when creating the video summary
● other interaction scenarios
3:50 - 4:00: Summary and conclusions
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo

35. FAIRview:
Responsible Video
Summarization
/ Lora Aroyo
/ Tagasauris Inc
/ http://lora-aroyo.org
/ @laroyo