Artificial Intelligence for the Film Industry

Artificial Intelligence
for the Film Industry
Georg Rehm
DFKI, Germany
Propellor FilmTech Meetup #1 – 25 July 2017 – Berlin, Germany

• Sites in Saarbrücken, Kaiserslautern,
Bremen, Berlin, Osnabrück, St. Wendel
• Intelligent software systems: robotics, agents,
image processing, language understanding,
augemented reality, 3D, knowledge management,
HMI, security, Industrie 4.0.
• 900+ staff – ca. 300 running projects
• CEO: Prof. Dr. Wolfgang Wahlster
Propellor FilmTech Meetup #1 – 25 July 2017
Deutschland-GmbH
2
German Research Centre for
Artificial Intelligence GmbH (founded in 1988)

Propellor FilmTech Meetup #1 – 25 July 2017 3

Artificial Intelligence
• Strong AI: hypothetical machine with a consciousness
and behaviour at least as flexible as that of a human.
• Weak AI: software without consciousness, tailored to
one specific purpose and task.
• Machine Learning: give “computers the ability
to learn without being explicitly programmed”
(Arthur Samuel, 1959)
• Examples: pattern recognition (e.g., hand
writing), predictions (stock exchange),
recommendations (films!) etc.

Data Intelligence
Current breakthroughs with machine learning methods
(Deep Learning). Also still in use: symbolic, rule-based methods

Language Technology
• Language Technology makes use of theoretical results
in linguistics in marketable solutions and applications.
• Uses research results from:
– Artificial Intelligence
– Computer Science
– Computational Linguistics
• Natural Language Processing
• Natural Language Understanding
– Psychology, Psycholinguistics
– Cognitive Science
• Language: Next big thing for AI!
Example Applications
• Spellchecker
• Dictation systems
• Translation systems
• Search engines
• Report generation
• Expert systems
• Dialogue systems
• Text summarisation

AI and the Film Industry
• AI and Language Technology:
Many breakthroughs in multiple
different application areas
• Focus: Film industry
• Massive potential!
Film
Industry
Language
Technology
AI and
Deep
Learning
Big Data
Fast
machines
and
networks
Internet of
Things
! Editing Trailers
! Writing Scripts
! Recommenders

• Simple Machine Learning
• Training data: 100 trailers
• Create model and apply it
(i.e., to the film “Morgan”)
• Watson selected scenes
• “A human editor was still
needed to patch the
scenes together to tell
a coherent story.”

• Good example of using
tech in a curation setting
• With the machine you’re
faster but you arrive at the
same result as the human
• The “AI” part is attributed
to the technology by the
(astonished) human who’s
also been influenced by
clever marketing
• Note: an “AI” is only good
at one very specific task!

• No, it didn’t.
• This is fake news
(category: clickbait).

• Simple ML again
• Training data: scripts
of sci-fi movies
• Neural network learns
patterns and is able to
generate a new script
• Deep Learning for
Natural Language
Generation (NLG)
• Can also be applied
to Shakespeare

• Simple ML again
• Training data: scripts
of sci-fi movies
• Neural network learns
patterns in scripts and
is able to generate
new script
• Deep Learning for
Natural Language
Generation (NLG)
• Can also be applied
to Shakespeare
To me, fair, so you never be,
Each trifle, way, when bore your beauty when,
Such hence your can still,
O thou how much were your self the wrong chide.
Thy youth’s time and face his form shall cover?
Now all fresh beauty, my love there,
Will ever time to greet, forget each, like ever decease,
But in a best at worship his glory die.
Stanley Xie, Ruchir Rastogi, Max Chang: “Deep Poetry: Word-Level and Character-
Level Language Models for Shakespearean Sonnet Generation” (Stanford)

• The automatically
generated script
doesn’t make any
sense whatsoever.
• “Sunspring” is an
interesting exercise
but, essentially,
unwatchable.

AI – Taking Stock
What AI is good at
• Identifying patterns
• Extracting structure
• Data analysis
• Mimicking regularities
• Important: training data
(ideally structured)
• Emulating smart
behaviour
What AI is really bad at
• Creativity
• Eloquence
• Curiosity
• Freshness
• Originality
• Poetry
• Out-of-the-box’ness
• Understanding of the
world that surrounds us

The Outer Limits
AI would even fail at this
seemingly simply task …

Even “automatic mockbuster generation”
required a level of creativity that is way beyond
anything Artificial Intelligence can achieve today.

https://medium.com/@bootstrappingme/the-german-artificial-intelligence-landscape-b3708b325124

Film AI Startups
• VaultML, ScriptBook, Pilot Movies: Project ticket sales
and box office performance (script or trailer analysis)
• Iris.tv: Better recommendations
• Qloo: Cultural AI, predicts the tastes for any target
audience and maps relationships (music, books, films)
• Valossa: Detects people, context, topics etc. in video
and audio streams (assist video content discovery)
• Cinuru: Customer Relationship Management
• Much more can be done …
http://www.nanalyze.com/2017/07/6-startups-ai-movies-entertainment/

Data for Film AI
• Current AI methods can do a lot with interesting data.
• What is “interesting data” in the film industry?
• Could be anything from every part of the life cycle:
– Scripts – Preferences – List of scenes
– Reviews – Films watched – Credits
– Emails – Categories – Rankings
– Production notes – Genres – Relations
– Demographics – Lexicons – Databases
– Statistics – Focus groups – Marketing
– Box office results – Target audience – etc.

Example Use Case
• Let’s have a look at a concrete use case and challenge
• Deep, context-aware recommendations that fit the
viewer’s mood, time constraints, interests, focus areas
• Example: you have ca. 60 minutes, you’re interested in
current politics in the US, have an upcoming trip to
Vancouver, like running, AI, languages and technology
• Recommender could suggest films or documentaries
that exactly fit this bill (using a deep user model)
• How? By pulling different sources of data together
• Calendar (upcoming trips and meetings), browsing and
search history, to do list, social media, IMDB profile etc.

Example: Details
• Data sources:
– Calendar: upcoming trip to Vancouver
– To do list: prepare the trip (e.g., “find running route”)
– Email archive: hotel booking in Vancouver
• The smart recommender algorithm could examine these
data points and help the user get a few things done
• Upcoming trip + likes running + location of hotel = videos
of running routes or running clubs in Vancouver
• Upcoming trip + likes running = films about, or including,
running that are set in or that were shot in Vancouver

Lifelogging and IoT
• Lifelogging = record your whole life
• Mobile phones and activity trackers
are getting closer (quantified self)
• Measuring heart-rate 24/7/365
• Advanced measurements like
VO2 max through several sensors
is consumer-grade technology!
• What about film-related data points?
• Measuring excitement, boredom, attention, repetition,
amazement, imitation, cringe-worthiness, disgust,
tenseness, eye-tracking etc.
https://en.wikipedia.org/wiki/Lifelog

Film and Quantified Self
• Vision: create deep user models by pulling together a
user’s heterogeneous information and data streams
(calendar, contacts, to do lists, profiles, youtube etc.)
• Add lifelogging-related data by tapping into activity
trackers, smart watches, mobile phone sensors
• Endless possibilities would emerge … – and will!
• Measure the reactions of one viewer or a whole theater
by measuring their vital stats when watching a film
• Revolutionise film development and test screenings
• Adapt films dynamically (insert explosion when bored)

• Propellor | Forum #1 created intriguing results
• Any Film, Anywhere – user model, watchlist, loc, reco
• Bubble Buster – user model, reco (safe & surprising)
• Super AI Brain – user model, reco
• Data of the Movie – user model, reco, biofeedback
• AI-based Storytelling – user model, audience
clustering, Big Data-based storytelling
http://www.propellorfilmtech.com/forum

Challenges
• Integration of heterogeneous data sources (from silos!)
into a unified and homogeneous model as well as
making this model available to recommender algorithms.
• Getting the data is hard, so is mapping the data.
• How do we get – on a very large scale – the data out of
connected devices (smart phones, smart watches,
activity trackers, tv sets etc.) into our own applications?
• The typical, very hard, AI challenges: How can we really
model creativity, originality etc.?

Thank you!
DKT kick-off meeting – 25 September 2015
Digital Curation Technologies
• Support and optimise digital curation through language and
knowledge technologies
• Develop innovative prototypes together with the SME partners
• Further develop DFKI technologies and transfer them into
industry through platform for digital curation technologies
Georg Rehm und Felix Sasaki. “Digital Curation Technologies.” In Proceedings of the 19th Annual
Conference of the European Association for Machine Translation (EAMT 2016), Riga, Lettland, Mai 2016
Georg Rehm und Felix Sasaki. “Digitale Kuratierungstechnologien – Verfahren für die effiziente
Verarbeitung, Erstellung und Verteilung qualitativ hochwertiger Medieninhalte.” In Proceedings der
Frühjahrstagung der Gesellschaft für Sprachtechnologie und Computerlinguistik (GSCL 2015), S. 138-139,
Duisburg, 2015
Sprach- und Wissenstechnologien
Kuratierungstechnologien
Branchentechnologien
Plattformtechnologie
Branchenlösungen
http://digitale-kuratierung.de

Artificial Intelligence for the Film Industry

More Related Content

What's hot

Similar to Artificial Intelligence for the Film Industry

More from Georg Rehm

Recently uploaded

Artificial Intelligence for the Film Industry