Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Artificial Intelligence for the Film Industry

2,605 views

Published on

Georg Rehm. Artificial Intelligence for the Film Industry. FilmTech Meetup Berlin. Berlin, Germany, July 2017. July 25, 2017.

Published in: Technology
  • Be the first to comment

Artificial Intelligence for the Film Industry

  1. 1. Artificial Intelligence for the Film Industry Georg Rehm DFKI, Germany Propellor FilmTech Meetup #1 – 25 July 2017 – Berlin, Germany
  2. 2. • Sites in Saarbrücken, Kaiserslautern, Bremen, Berlin, Osnabrück, St. Wendel • Intelligent software systems: robotics, agents, image processing, language understanding, augemented reality, 3D, knowledge management, HMI, security, Industrie 4.0. • 900+ staff – ca. 300 running projects • CEO: Prof. Dr. Wolfgang Wahlster Propellor FilmTech Meetup #1 – 25 July 2017 Deutschland-GmbH 2 German Research Centre for Artificial Intelligence GmbH (founded in 1988)
  3. 3. Propellor FilmTech Meetup #1 – 25 July 2017 3
  4. 4. Propellor FilmTech Meetup #1 – 25 July 2017 4
  5. 5. Propellor FilmTech Meetup #1 – 25 July 2017 5
  6. 6. Propellor FilmTech Meetup #1 – 25 July 2017 6
  7. 7. Artificial Intelligence • Strong AI: hypothetical machine with a consciousness and behaviour at least as flexible as that of a human. • Weak AI: software without consciousness, tailored to one specific purpose and task. • Machine Learning: give “computers the ability to learn without being explicitly programmed” (Arthur Samuel, 1959) • Examples: pattern recognition (e.g., hand writing), predictions (stock exchange), recommendations (films!) etc. Propellor FilmTech Meetup #1 – 25 July 2017 7
  8. 8. Propellor FilmTech Meetup #1 – 25 July 2017 8 Data Intelligence Current breakthroughs with machine learning methods (Deep Learning). Also still in use: symbolic, rule-based methods
  9. 9. Language Technology • Language Technology makes use of theoretical results in linguistics in marketable solutions and applications. • Uses research results from: – Artificial Intelligence – Computer Science – Computational Linguistics • Natural Language Processing • Natural Language Understanding – Psychology, Psycholinguistics – Cognitive Science • Language: Next big thing for AI! Propellor FilmTech Meetup #1 – 25 July 2017 9 Example Applications • Spellchecker • Dictation systems • Translation systems • Search engines • Report generation • Expert systems • Dialogue systems • Text summarisation
  10. 10. AI and the Film Industry • AI and Language Technology: Many breakthroughs in multiple different application areas • Focus: Film industry • Massive potential! Propellor FilmTech Meetup #1 – 25 July 2017 10 Film Industry Language Technology AI and Deep Learning Big Data Fast machines and networks Internet of Things ! Editing Trailers ! Writing Scripts ! Recommenders
  11. 11. Propellor FilmTech Meetup #1 – 25 July 2017 11
  12. 12. Propellor FilmTech Meetup #1 – 25 July 2017 12
  13. 13. Propellor FilmTech Meetup #1 – 25 July 2017 13 • Simple Machine Learning • Training data: 100 trailers • Create model and apply it (i.e., to the film “Morgan”) • Watson selected scenes • “A human editor was still needed to patch the scenes together to tell a coherent story.”
  14. 14. Propellor FilmTech Meetup #1 – 25 July 2017 14 • Good example of using tech in a curation setting • With the machine you’re faster but you arrive at the same result as the human • The “AI” part is attributed to the technology by the (astonished) human who’s also been influenced by clever marketing • Note: an “AI” is only good at one very specific task!
  15. 15. Propellor FilmTech Meetup #1 – 25 July 2017 15
  16. 16. Propellor FilmTech Meetup #1 – 25 July 2017 16 • No, it didn’t. • This is fake news (category: clickbait).
  17. 17. Propellor FilmTech Meetup #1 – 25 July 2017 17
  18. 18. Propellor FilmTech Meetup #1 – 25 July 2017 18
  19. 19. Propellor FilmTech Meetup #1 – 25 July 2017 19 • Simple ML again • Training data: scripts of sci-fi movies • Neural network learns patterns and is able to generate a new script • Deep Learning for Natural Language Generation (NLG) • Can also be applied to Shakespeare
  20. 20. Propellor FilmTech Meetup #1 – 25 July 2017 20 • Simple ML again • Training data: scripts of sci-fi movies • Neural network learns patterns in scripts and is able to generate new script • Deep Learning for Natural Language Generation (NLG) • Can also be applied to Shakespeare To me, fair, so you never be, Each trifle, way, when bore your beauty when, Such hence your can still, O thou how much were your self the wrong chide. Thy youth’s time and face his form shall cover? Now all fresh beauty, my love there, Will ever time to greet, forget each, like ever decease, But in a best at worship his glory die. Stanley Xie, Ruchir Rastogi, Max Chang: “Deep Poetry: Word-Level and Character- Level Language Models for Shakespearean Sonnet Generation” (Stanford)
  21. 21. Propellor FilmTech Meetup #1 – 25 July 2017 21 • The automatically generated script doesn’t make any sense whatsoever. • “Sunspring” is an interesting exercise but, essentially, unwatchable.
  22. 22. AI – Taking Stock What AI is good at • Identifying patterns • Extracting structure • Data analysis • Mimicking regularities • Important: training data (ideally structured) • Emulating smart behaviour What AI is really bad at • Creativity • Eloquence • Curiosity • Freshness • Originality • Poetry • Out-of-the-box’ness • Understanding of the world that surrounds us Propellor FilmTech Meetup #1 – 25 July 2017 22
  23. 23. The Outer Limits AI would even fail at this seemingly simply task …
  24. 24. Propellor FilmTech Meetup #1 – 25 July 2017 24
  25. 25. Propellor FilmTech Meetup #1 – 25 July 2017 25
  26. 26. Propellor FilmTech Meetup #1 – 25 July 2017 26 Even “automatic mockbuster generation” required a level of creativity that is way beyond anything Artificial Intelligence can achieve today.
  27. 27. Propellor FilmTech Meetup #1 – 25 July 2017 27 https://medium.com/@bootstrappingme/the-german-artificial-intelligence-landscape-b3708b325124
  28. 28. Film AI Startups • VaultML, ScriptBook, Pilot Movies: Project ticket sales and box office performance (script or trailer analysis) • Iris.tv: Better recommendations • Qloo: Cultural AI, predicts the tastes for any target audience and maps relationships (music, books, films) • Valossa: Detects people, context, topics etc. in video and audio streams (assist video content discovery) • Cinuru: Customer Relationship Management • Much more can be done … Propellor FilmTech Meetup #1 – 25 July 2017 28 http://www.nanalyze.com/2017/07/6-startups-ai-movies-entertainment/
  29. 29. Data for Film AI • Current AI methods can do a lot with interesting data. • What is “interesting data” in the film industry? • Could be anything from every part of the life cycle: – Scripts – Preferences – List of scenes – Reviews – Films watched – Credits – Emails – Categories – Rankings – Production notes – Genres – Relations – Demographics – Lexicons – Databases – Statistics – Focus groups – Marketing – Box office results – Target audience – etc. Propellor FilmTech Meetup #1 – 25 July 2017 29
  30. 30. Example Use Case • Let’s have a look at a concrete use case and challenge • Deep, context-aware recommendations that fit the viewer’s mood, time constraints, interests, focus areas • Example: you have ca. 60 minutes, you’re interested in current politics in the US, have an upcoming trip to Vancouver, like running, AI, languages and technology • Recommender could suggest films or documentaries that exactly fit this bill (using a deep user model) • How? By pulling different sources of data together • Calendar (upcoming trips and meetings), browsing and search history, to do list, social media, IMDB profile etc. Propellor FilmTech Meetup #1 – 25 July 2017 30
  31. 31. Example: Details • Data sources: – Calendar: upcoming trip to Vancouver – To do list: prepare the trip (e.g., “find running route”) – Email archive: hotel booking in Vancouver • The smart recommender algorithm could examine these data points and help the user get a few things done • Upcoming trip + likes running + location of hotel = videos of running routes or running clubs in Vancouver • Upcoming trip + likes running = films about, or including, running that are set in or that were shot in Vancouver Propellor FilmTech Meetup #1 – 25 July 2017 31
  32. 32. Lifelogging and IoT • Lifelogging = record your whole life • Mobile phones and activity trackers are getting closer (quantified self) • Measuring heart-rate 24/7/365 • Advanced measurements like VO2 max through several sensors is consumer-grade technology! • What about film-related data points? • Measuring excitement, boredom, attention, repetition, amazement, imitation, cringe-worthiness, disgust, tenseness, eye-tracking etc. Propellor FilmTech Meetup #1 – 25 July 2017 32 https://en.wikipedia.org/wiki/Lifelog
  33. 33. Film and Quantified Self • Vision: create deep user models by pulling together a user’s heterogeneous information and data streams (calendar, contacts, to do lists, profiles, youtube etc.) • Add lifelogging-related data by tapping into activity trackers, smart watches, mobile phone sensors • Endless possibilities would emerge … – and will! • Measure the reactions of one viewer or a whole theater by measuring their vital stats when watching a film • Revolutionise film development and test screenings • Adapt films dynamically (insert explosion when bored) Propellor FilmTech Meetup #1 – 25 July 2017 33
  34. 34. • Propellor | Forum #1 created intriguing results • Any Film, Anywhere – user model, watchlist, loc, reco • Bubble Buster – user model, reco (safe & surprising) • Super AI Brain – user model, reco • Data of the Movie – user model, reco, biofeedback • AI-based Storytelling – user model, audience clustering, Big Data-based storytelling Propellor FilmTech Meetup #1 – 25 July 2017 34 http://www.propellorfilmtech.com/forum
  35. 35. Challenges • Integration of heterogeneous data sources (from silos!) into a unified and homogeneous model as well as making this model available to recommender algorithms. • Getting the data is hard, so is mapping the data. • How do we get – on a very large scale – the data out of connected devices (smart phones, smart watches, activity trackers, tv sets etc.) into our own applications? • The typical, very hard, AI challenges: How can we really model creativity, originality etc.? Propellor FilmTech Meetup #1 – 25 July 2017 35
  36. 36. Thank you! Propellor FilmTech Meetup #1 – 25 July 2017 36 DKT kick-off meeting – 25 September 2015 Digital Curation Technologies • Support and optimise digital curation through language and knowledge technologies • Develop innovative prototypes together with the SME partners • Further develop DFKI technologies and transfer them into industry through platform for digital curation technologies Georg Rehm und Felix Sasaki. “Digital Curation Technologies.” In Proceedings of the 19th Annual Conference of the European Association for Machine Translation (EAMT 2016), Riga, Lettland, Mai 2016 Georg Rehm und Felix Sasaki. “Digitale Kuratierungstechnologien – Verfahren für die effiziente Verarbeitung, Erstellung und Verteilung qualitativ hochwertiger Medieninhalte.” In Proceedings der Frühjahrstagung der Gesellschaft für Sprachtechnologie und Computerlinguistik (GSCL 2015), S. 138-139, Duisburg, 2015 Sprach- und Wissenstechnologien Kuratierungstechnologien Branchentechnologien Plattformtechnologie Branchenlösungen http://digitale-kuratierung.de

×