These are the slides for a talk I gave about my video analytics project at the CATAPULT Center at Indiana University (Bloomington) on October 16, 2014. The abstract is below: Cultural analytics, a newer branch of the digital humanities, is an approach that deploys computer technologies to analyze the formal features of art and culture, making them available to interpretive methods. Moving image media is particularly ripe for computational analysis given its increasing ubiquity in contemporary culture. Indeed, digital video—whether recorded digitally or digitized from film—is a rapidly expanding form of contemporary cultural production, one made possible by the proliferation of personal recording technologies and hosting platforms like YouTube, Vimeo and the Internet Archive. In short, video is one of the most compelling “big data,” issues of the current cultural moment; its formats are diverse, rapidly transmitted, and boundlessly large in number. Yet despite its scale and importance, video remains a daunting object for sustained research, for obstacles that are technological, institutional and conceptual in nature. In this talk, Virginia Kuhn will describe her large-scale video analytics project which is supported by the NSF’s XSEDE program (extreme science and engineering discovery environment) and her project team’s efforts at establishing a software workbench for video analysis, annotation, and visualization, using both current and experimental discovery methods.

1. The VAT
Video Analysis at Scale
VIRGINIA KUHN
VK
Catapult Center, IU
10/16/2014
vkuhn@cinema.usc.edu

2.  115 years of Cinema
 YouTube | Vimeo
 TED talks
 Incidental Video
 Academic Video
 Netflix
 Amazon
 Archives
VAT
10/16/2014
vkuhn@cinema.usc.edu
Video = Big Data
Scale = Micro + Macro

3. VAT
10/16/2014
vkuhn@cinema.usc.edu

4. The VAT
VAT
10/16/2014
vkuhn@cinema.usc.edu

5. VAT
10/16/2014
vkuhn@cinema.usc.edu
Why analyze?
 Technological evolution: e.g. When did cell
phones begin appearing in movies? How about
laptop computers?
 Can one identify Hitchcock films by their shot
length and camera angles? What about
Spielberg? Orson Welles? Ridley Scott?
 Are police officers depicted the same way in
fiction and nonfiction films such that their
authority is reinforced unquestioningly?
 Do film and television give accurate information
about medical issues? Legal issues?

6. VAT
10/16/2014
vkuhn@cinema.usc.edu
Challenges
Current tagging apps (limited)
 Computer vision (form + content)
 Hybrid (human-machine)
 Crowd-sourced tagging

7. extractors
 LSVA Extractor: Extracts the image
descriptors from the identified shots and
indexes them
 Video Extractor: Converts video file into
MP4 for viewing online and preview of
smaller size
 Cinemetrics Extractor: Identifies shots in
the uploaded video and marks them as
sections of the video (python)
 Movie Slice Extractor: Creates Movie Slice
visualization image from raw video file
VAT
10/16/2014
vkuhn@cinema.usc.edu

8. descriptors
17 descriptors
 Gabor energies
 RGB
Gray histogram
Color layout
Edge detection
VAT
10/16/2014
vkuhn@cinema.usc.edu

9. VAT Screen shot showing one video, with the
novel visualization of the footage at top:
10/16/2014
vkuhn@cinema.usc.edu

10. VAT
10/16/2014
vkuhn@cinema.usc.edu
Another screen shot showing one video, with
the novel visualization and player window.

11. VAT
10/16/2014
vkuhn@cinema.usc.edu
 http://ion-21-15.sdsc.edu:9000/

12.  TEAM
VAT
Virginia Kuhn, Alan Craig, Michael Simeone, Luigi Marini,
Sandeep Puthanveetil, Dave Bock, Mona Wong, Lianna
Diesendruck, Ritu Arora
NEXT STEPS:
1. Gateway community of practice: http://vat.xsede.org
2. Expand novel viz in order to treat film as more than a
set of stills
10/16/2014
vkuhn@cinema.usc.edu

13.  THANK YOU!
vkuhn@cinema.usc.edu
VAT
10/16/2014
vkuhn@cinema.usc.edu