Roelof Pieters, profile picture
Uploaded byRoelof Pieters
PDF, PPTX3,051 views

Creative AI & multimodality: looking ahead

This document discusses creative AI and multimodality. It begins by looking at current possibilities for creative AI, including appropriating standard neural networks for creative use, reinforcement learning approaches that frame creativity as a game, recurrent neural networks, sequence-to-sequence models that treat creativity as translation, autoencoders, attention-based models, and generative adversarial networks. It also discusses needs for creative AI, including developing a system that marries a creative process with creative outputs using minimal human input data but with its own style and the ability for human-level supervision to enable rapid experimentation. The document frames creative AI as a "brush" that can be used for painting.

Embed presentation

Download as PDF, PPTX
Creative AI & multimodality: looking ahead Roelof Pieters @graphific Imperial College London, 
 1 Dec 2015 roelof@graph-technologies.comhttp://artificialexperience.com/http://www.csc.kth.se/~roelof/
AICreative
AI I kinda expect the audience to know AI & Machine Learning
 Let’s move on shall we ?
AI All references to: - Arxiv or - GitXiv if the “code” or “dataset” is available Collaborative Open Computer Science more info (Medium)
AI > today’s focus
AI > today’s focus
“Deep learning is a set of algorithms in machine learning that attempt to learn in multiple levels, corresponding to different levels of abstraction.”
AI > today’s focus use of several modes (media) to create a single artifact. Multimodality “Mode” Socially and culturally shaped resource for making meaning. — Gunther Kress
Creativity
Creativity • Many definitions: philosophical, sociological, historical, practical
Creativity 1. Making unfamiliar combinations of familiar ideas. 2. Explore a structured conceptual space 3. (Radically) transforming ones structured conceptual space “Exploration” “Remix” “The Creative Mind”
 — Margaret Boden “Transformation”
• Skill • Appreciation • Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality. Creativity > “Traits” software has to exhibit in order to avoid easy criticism of being “non-creative”. (Simon Colton)
• Skill • Appreciation • Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
• Skill • Appreciation • Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
• Skill • Appreciation • Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
• Skill • Appreciation • Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
• Skill • Appreciation • Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
• Skill • Appreciation • Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
• Skill • Appreciation • Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
• Skill • Appreciation • Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
AICreative
Creative AI > Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
Creative AI > Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
Creative AI > Current possibilities > Appropriating “standard” nets for creative use Deep Dream see also: www.csc.kth.se/~roelof/deepdream/
Creative AI > Current possibilities > Appropriating “standard” nets for creative use Deep Dream see also: www.csc.kth.se/~roelof/deepdream/ codeyoutubeRoelof Pieters 2015
Creative AI > Current possibilities > Appropriating “standard” nets for creative use Deep Dream see also: www.csc.kth.se/~roelof/deepdream/ C.M.Kosemen & 
 Roelof Pieters (2015) Gizmodo
Creative AI > Current possibilities > Appropriating “standard” nets for creative use Leon A. Gatys, Alexander S. Ecker, Matthias Bethge , 2015. 
 A Neural Algorithm of Artistic Style (GitXiv) Style Net
Gene Kogan, 2015. Why is a Raven Like a Writing Desk? (vimeo)
Creative AI > Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
Creative AI > Current possibilities > Reinforcement Learning • AMN: Emilio Parisotto, Jimmy Lei Ba, Ruslan Salakhutdinov 2015, Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning (arxiv) • DQN: Mnih, Volodymyr, Kavukcuoglu, Koray, Silver, David, Rusu, Andrei A., Veness, Joel, Bellemare, Marc G., Graves, Alex, Riedmiller, Martin, Fidjeland, Andreas K., Ostrovski, Georg, Petersen, Stig, Beattie, Charles, Sadik, Amir, Antonoglou, Ioannis, King, Helen, Kumaran, Dharshan, Wierstra, Daan, Legg, Shane, and Hassabis, Demis. Human-level control through deep reinforcement learning. Nature, 518(7540): 529–533, 2015.
Creative AI > Current possibilities > Reinforcement Learning Ardi Tampuu, Tambet Matiisen, Dorian Kodelja, Ilya Kuzovkin, Kristjan Korjus, Juhan Aru, Jaan Aru, Raul Vicente, 2015 
 Multiagent Cooperation and Competition with Deep Reinforcement Learning (GitXiv) (YouTube)
Reinforcement Learning Ning Xie, Hirotaka Hachiya, Masashi Sugiyama, 2013 , 
 Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting (Paper, Lecture, YouTube)
(YouTube)
Ning Xie, Hirotaka Hachiya, Masashi Sugiyama, 2013
 Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting (Paper, Lecture, YouTube)
Creative AI > Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
Creative AI > Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
Creative AI > Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-encoders • Attention-based Models • Generative Adversarial Nets
Creative AI > Current possibilities • Standard (“denoising”) Autoencoders • Variational Autoencoder (VAE) / Stochastic Gradient VB • Deep Convolutional Inverse Graphics Network • Variational RNN (VRNN) Vincent et al, 2010. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion (paper) (code)
Creative AI > Current possibilities • Standard “denoising” Autoencoders • Variational Autoencoder (VAE) / Stochastic Gradient VB • Deep Convolutional Inverse Graphics Network • Variational RNN (VRNN) • Diederik P Kingma, Max Welling, 2013. 
 Auto-Encoding Variational Bayes (GitXiv)
Creative AI > Current possibilities • Standard “denoising” Autoencoders • Variational Autoencoder (VAE) • Deep Convolutional Inverse Graphics Network (modified VAE) • Variational RNN (VRNN) Tejas D. Kulkarni, Will Whitney, Pushmeet Kohli, Joshua B. Tenenbaum, 2015 Deep Convolutional Inverse Graphics Network (GitXiv)
Creative AI > Current possibilities • Standard “denoising” Autoencoders • Variational Autoencoder (VAE) • Deep Convolutional Inverse Graphics Network • Variational RNN (VRNN) (VAE at every time step) Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron Courville, Yoshua Bengio, 2015
 A Recurrent Latent Variable Model for Sequential Data (GitXiv) VAEVAEVAE
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron Courville, Yoshua Bengio , 2015. 
 A Recurrent Latent Variable Model for Sequential Data (GitXiv) (Audio Samples)
Creative AI > Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, Daan Wierstra, 2015
 DRAW: A Recurrent Neural Network For Image Generation (GitXiv) Variational Auto-Encoder Deep Recurrent Attentive Writer (DRAW) Network
(YouTube)
Creative AI > Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adverserial Nets
Emily Denton, Soumith Chintala, Arthur Szlam, Rob Fergus, 2015. 
 Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks (GitXiv)
Alec Radford, Luke Metz, Soumith Chintala , 2015. 
 Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (GitXiv)
Alec Radford, Luke Metz, Soumith Chintala , 2015. 
 Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (GitXiv)
”turn” vector created from four averaged samples of faces looking left vs looking right. Alec Radford, Luke Metz, Soumith Chintala , 2015. 
 Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (GitXiv)
walking through the manifold
top: unmodified samples bottom: same samples dropping out ”window” filters
Autonomy Supervision Creativity? - unsupervised training - generator/discrimator - latent/z space - auto encoders - multimodality - query - target/class
Creativity? Process Result
Creative AI > Needs as I see it Creative AI as a “tool”
 or “brush” to paint with
A system which marries the need for a creative process with the need for a creative output • with as less human input as possible (data) • with its own style • with the possibility for human level supervision for rapid experimentation Creative AI > a “brush”
A system which marries the need for a creative process with the need for a creative output • with as less human input as possible ( ) • with its own style • with the possibility for human level supervision for rapid experimentation Creative AI > a “brush” data
Creative AI > a “brush” > data • reuse nets as much as possible • combining unsupervised & supervised • multiple modalities • plug in external knowledge bases
Creative AI > a “brush” > data input • unlabeled & labeled data • external knowledge bases (dbpedia, wikipedia) • one-shot learning • zero-shot learning Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Christopher D. Manning, Andrew Y. Ng, 2013
 Zero-Shot Learning Through Cross-Modal Transfer a zero-shot model that can predict both seen and unseen classes
Creative AI > a “brush” > data input Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Christopher D. Manning, Andrew Y. Ng, 2013
 Zero-Shot Learning Through Cross-Modal Transfer (slides)
Creative AI > a “brush” > data input Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Christopher D. Manning, Andrew Y. Ng, 2013
 Zero-Shot Learning Through Cross-Modal Transfer (slides)
Creative AI > a “brush” > data input Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Christopher D. Manning, Andrew Y. Ng, 2013
 Zero-Shot Learning Through Cross-Modal Transfer (slides)
A system which marries the need for a creative process with the need for a creative output • with as less human input as possible (data) • with its own style • with the possibility for human level 
 for rapid experimentation Creative AI > a “brush” supervision
Creative AI > a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
Creative AI > a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
Creative AI > a “brush” > data Deep Dream Alexander Mordvintsev, Christopher Olah, Mike Tyka, 2015. 
 Inceptionism: Going Deeper into Neural Networks Google Research Blog
Creative AI > a “brush” > data Deep Dream Roelof Pieters, 2015 DeepDream - Class visualization Experiment (link)
Roelof Pieters, 2015 DeepDream - Class visualization Experiment (link)
Creative AI > a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
Creative AI > a “brush” > data Deep Dream Roelof Pieters, 2015 DeepDream - Overview of standard bvlc googlenet (inception) layers (link) Constrain Layers
Creative AI > a “brush” > data Deep Dream Roelof Pieters, 2015 Single Unit Activations (early layer) (Flickr Album) Constrain Units
Creative AI > a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
Creative AI > a “brush” > data Deep Dream Roelof Pieters, 2015 DeepDream Video (GitHub)
Creative AI > a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
Creative AI > a “brush” > data Style Net Roelof Pieters (graphific) (tweet) Roelof Pieters (graphific) (tweet)
Creative AI > a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
Image -> Text “A person riding a motorcycle on a dirt road.”???
Image -> Text “Two hockey players are fighting over the puck.”???
Image -> Text Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua Bengio, Show, Attend and Tell: Neural Image Caption Generation with Visual Attention (arxiv) (info) (code) Andrej Karpathy Li Fei-Fei , 2015. 
 Deep Visual-Semantic Alignments for Generating Image Descriptions (pdf) (info) (code) Oriol Vinyals, Alexander Toshev, Samy Bengio, Dumitru Erhan , 2015. Show and Tell: A Neural Image Caption Generator (arxiv)
Text -> Image “A stop sign is flying in blue skies.” “A herd of elephants flying in the blue skies.” Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, Ruslan Salakhutdinov, 2015. Generating Images from Captions with Attention (arxiv) (examples)
Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, Ruslan Salakhutdinov, 2015. Generating Images from Captions with Attention (arxiv) (examples) Text -> Image
Subhashini Venugopalan, Marcus Rohrbach, Jeff Donahue, Raymond Mooney, Trevor Darrell, Kate Saenko , 2015. Sequence to Sequence -- Video to Text (GitXiv) Video -> Text
A system which marries the need for a creative process with the need for a creative output • with as less human input as possible (data) • with its own style • with the possibility for human level supervision for 
 Creative AI > a “brush” rapid experimentation
Creative AI > a “brush” > rapid experimentation
Widening Deepening Tianqi Chen, Ian Goodfellow, Jonathon Shlens, 2015. Net2Net: Accelerating Learning via Knowledge Transfer (arxiv) / code (torch) Reusing Nets: Bigger Net
Teacher and Student net Hint training Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, Yoshua Bengio, 2014. FitNets: Hints for Thin Deep Nets (arxiv) Knowledge distillation SVHN Error MNIST Error Reusing Nets: Smaller Net
Hashed Net Wenlin Chen, James T. Wilson, Stephen Tyree, Kilian Q. Weinberger, Yixin Chen, 2015. Compressing Neural Networks with the Hashing Trick (arxiv) Shrinking Nets: Hashing
Song Han, Huizi Mao, William J. Dally, 2015. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding (arxiv) Shrinking Nets: Pruning, Quantization & Huffman coding
Creative AI > a “brush” > rapid experimentation • experiments need “tooling”, specialised design software to • try things • explore latent spaces (z-space) • push the AI in the right direction • be surprised by AI
Creative AI > a “brush” > rapid experimentation human-machine collaboration
Creative AI > a “brush” > rapid experimentation (YouTube, Paper)
Creative AI > a “brush” > rapid experimentation (YouTube, Paper)
Creative AI > a “brush” > rapid experimentation (Vimeo, Paper)
Creative AI > a “brush” > rapid experimentation • Advertising and marketing • Architecture • Crafts • Design: product, graphic and fashion design • Film, TV, video, radio and photography • IT, software and computer services • Publishing • Museums, galleries and libraries • Music, performing and visual arts
Questions? love letters? existential dilemma’s? academic questions? gifts? find me at:
 www.csc.kth.se/~roelof/ roelof@kth.se

More Related Content

PDF
Multi-modal embeddings: from discriminative to generative models and creative ai
byRoelof Pieters
 
PDF
Deep Neural Networks 
that talk (Back)… with style
byRoelof Pieters
 
PDF
Zero shot learning through cross-modal transfer
byRoelof Pieters
 
PDF
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
byMatthew Lease
 
PPTX
Deep Learning for Artificial Intelligence (AI)
byEr. Shiva K. Shrestha
 
PPTX
Deep Learning - Convolutional Neural Networks - Architectural Zoo
byChristian Perone
 
PDF
Deep Learning for NLP: An Introduction to Neural Word Embeddings
byRoelof Pieters
 
PDF
Intro To Convolutional Neural Networks
byMark Scully
 
Multi-modal embeddings: from discriminative to generative models and creative ai
byRoelof Pieters
 
Deep Neural Networks 
that talk (Back)… with style
byRoelof Pieters
 
Zero shot learning through cross-modal transfer
byRoelof Pieters
 
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
byMatthew Lease
 
Deep Learning for Artificial Intelligence (AI)
byEr. Shiva K. Shrestha
 
Deep Learning - Convolutional Neural Networks - Architectural Zoo
byChristian Perone
 
Deep Learning for NLP: An Introduction to Neural Word Embeddings
byRoelof Pieters
 
Intro To Convolutional Neural Networks
byMark Scully
 

What's hot

PDF
Advanced Deep Architectures (D2L6 Deep Learning for Speech and Language UPC 2...
byUniversitat Politècnica de Catalunya
 
PDF
Deep Learning and Text Mining
byWill Stanton
 
PDF
Tutorial on Deep Learning
byinside-BigData.com
 
PDF
Deep Learning and Reinforcement Learning
byRenārs Liepiņš
 
PDF
Practical Deep Learning for NLP
byTextkernel
 
PDF
The Unreasonable Benefits of Deep Learning
byindico data
 
PDF
Deep Learning Class #0 - You Can Do It
byHolberton School
 
PPTX
Deep Learning - A Literature survey
byAkshay Hegde
 
PDF
Donner - Deep Learning - Overview and practical aspects
byVienna Data Science Group
 
PDF
Deep Learning: a birds eye view
byRoelof Pieters
 
PDF
Deep Learning and the state of AI / 2016
byGrigory Sapunov
 
PPTX
Introduction to Deep Learning
byMustafa Aldemir
 
PPTX
What Deep Learning Means for Artificial Intelligence
byJonathan Mugan
 
PPTX
Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...
byAndrew Gardner
 
PDF
Deep learning - Conceptual understanding and applications
byBuhwan Jeong
 
PDF
[DSC x TAAI 2016] 林守德 / 人工智慧與機器學習在推薦系統上的應用
by台灣資料科學年會
 
PDF
An introduction to Deep Learning
byJulien SIMON
 
ODP
Start a deep learning startup - tutorial
byMostapha Benhenda
 
PDF
Deep Learning And Business Models (VNITC 2015-09-13)
byHa Phuong
 
PDF
Deep learning - A Visual Introduction
byLukas Masuch
 
Advanced Deep Architectures (D2L6 Deep Learning for Speech and Language UPC 2...
byUniversitat Politècnica de Catalunya
 
Deep Learning and Text Mining
byWill Stanton
 
Tutorial on Deep Learning
byinside-BigData.com
 
Deep Learning and Reinforcement Learning
byRenārs Liepiņš
 
Practical Deep Learning for NLP
byTextkernel
 
The Unreasonable Benefits of Deep Learning
byindico data
 
Deep Learning Class #0 - You Can Do It
byHolberton School
 
Deep Learning - A Literature survey
byAkshay Hegde
 
Donner - Deep Learning - Overview and practical aspects
byVienna Data Science Group
 
Deep Learning: a birds eye view
byRoelof Pieters
 
Deep Learning and the state of AI / 2016
byGrigory Sapunov
 
Introduction to Deep Learning
byMustafa Aldemir
 
What Deep Learning Means for Artificial Intelligence
byJonathan Mugan
 
Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...
byAndrew Gardner
 
Deep learning - Conceptual understanding and applications
byBuhwan Jeong
 
[DSC x TAAI 2016] 林守德 / 人工智慧與機器學習在推薦系統上的應用
by台灣資料科學年會
 
An introduction to Deep Learning
byJulien SIMON
 
Start a deep learning startup - tutorial
byMostapha Benhenda
 
Deep Learning And Business Models (VNITC 2015-09-13)
byHa Phuong
 
Deep learning - A Visual Introduction
byLukas Masuch
 

Viewers also liked

PDF
Deep Learning for Information Retrieval
byRoelof Pieters
 
PDF
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
byRoelof Pieters
 
PDF
Graph, Data-science, and Deep Learning
byRoelof Pieters
 
PDF
Deep Learning as a Cat/Dog Detector
byRoelof Pieters
 
PDF
Explore Data: Data Science + Visualization
byRoelof Pieters
 
PDF
Multi modal retrieval and generation with deep distributed models
byRoelof Pieters
 
PDF
Deep learning for natural language embeddings
byRoelof Pieters
 
PDF
Deep Learning for Natural Language Processing: Word Embeddings
byRoelof Pieters
 
PDF
Visual-Semantic Embeddings: some thoughts on Language
byRoelof Pieters
 
PDF
Learning to understand phrases by embedding the dictionary
byRoelof Pieters
 
PDF
Building a Deep Learning (Dream) Machine
byRoelof Pieters
 
PDF
Deep Learning, an interactive introduction for NLP-ers
byRoelof Pieters
 
PDF
Deep Learning & NLP: Graphs to the Rescue!
byRoelof Pieters
 
PPTX
Deep neural networks
bySi Haem
 
PDF
Deep Learning - Convolutional Neural Networks
byChristian Perone
 
PDF
Applying Machine Learning to Social Media Data - Dung Phan, Data Scientist, C...
byWithTheBest
 
PDF
Deep Learning London Meetup - 28 May 2014
byPersontyle
 
PPTX
The Future of Curated Content - Contextual Machine Learning w/ Thoughtly
byChase Perkins
 
PPTX
ECML-2015 Presentation
byAnirban Santara
 
PPTX
Eddl5131 assignment 1 march2013
bygmorong
 
Deep Learning for Information Retrieval
byRoelof Pieters
 
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
byRoelof Pieters
 
Graph, Data-science, and Deep Learning
byRoelof Pieters
 
Deep Learning as a Cat/Dog Detector
byRoelof Pieters
 
Explore Data: Data Science + Visualization
byRoelof Pieters
 
Multi modal retrieval and generation with deep distributed models
byRoelof Pieters
 
Deep learning for natural language embeddings
byRoelof Pieters
 
Deep Learning for Natural Language Processing: Word Embeddings
byRoelof Pieters
 
Visual-Semantic Embeddings: some thoughts on Language
byRoelof Pieters
 
Learning to understand phrases by embedding the dictionary
byRoelof Pieters
 
Building a Deep Learning (Dream) Machine
byRoelof Pieters
 
Deep Learning, an interactive introduction for NLP-ers
byRoelof Pieters
 
Deep Learning & NLP: Graphs to the Rescue!
byRoelof Pieters
 
Deep neural networks
bySi Haem
 
Deep Learning - Convolutional Neural Networks
byChristian Perone
 
Applying Machine Learning to Social Media Data - Dung Phan, Data Scientist, C...
byWithTheBest
 
Deep Learning London Meetup - 28 May 2014
byPersontyle
 
The Future of Curated Content - Contextual Machine Learning w/ Thoughtly
byChase Perkins
 
ECML-2015 Presentation
byAnirban Santara
 
Eddl5131 assignment 1 march2013
bygmorong
 

Similar to Creative AI & multimodality: looking ahead

PDF
7-200404101602.pdf
byssuser07e9f2
 
PPTX
Generative models
byBirger Moell
 
PDF
What is Generative AI and How does it works?
byE42 (Light Information Systems Pvt Ltd)
 
PPTX
Generative_AI_Presentation_Detailed.pptx
bybadavathsrikanth150
 
PPTX
BIMA Breakfast Briefing | Creative Ai
byBIMA (British Interactive Media Association)
 
PPTX
SPI a presentation for seminar and project initiation course
bylk0685781
 
PDF
Unleashing Creative Potential: Diving Deep into the World of Generative Artif...
bybanarasone9
 
PDF
Discovering Generative AI's Creative Power: A Deep Dive Into Neural Networks
byArnav Malhotra
 
PDF
Slides, thesis dissertation defense, deep generative neural networks for nove...
bymehdi Cherti
 
PPTX
Introduction to Generative AI_Engineering.pptx
bypratyushnayak777
 
PPTX
Generative AI Descriptive MOdel ai .pptx
byKanchanaRSVVV
 
PDF
Creativity through deep learning
byAkin Osman Kazakci
 
PPTX
Generative_AI_Detailed_Presentation.pptx
byಚಂದ್ರು ಗುಡದಾ
 
PPTX
Fostering creativity in autonomous systems
bySarah Shuchi
 
PDF
Deep learning applications - for fun and profit
byRoberto Pereira Silveira
 
PPTX
Introduction to Generative AI refers to a subset of artificial intelligence
byKongu Engineering College, Perundurai, Erode
 
PDF
introduction to the world of generative AI
byssudhar40
 
PPTX
2 day Deep Learning Workshop at Karunya - Session 1
byRajagopal A
 
PDF
Generative AI Development Services | GenAI Consulting Company | Eoxys IT
byeoxysitsolutionllp
 
PDF
Some Preliminary Thoughts on Artificial Intelligence - April 20, 2023.pdf
byKent Bye
 
7-200404101602.pdf
byssuser07e9f2
 
Generative models
byBirger Moell
 
What is Generative AI and How does it works?
byE42 (Light Information Systems Pvt Ltd)
 
Generative_AI_Presentation_Detailed.pptx
bybadavathsrikanth150
 
BIMA Breakfast Briefing | Creative Ai
byBIMA (British Interactive Media Association)
 
SPI a presentation for seminar and project initiation course
bylk0685781
 
Unleashing Creative Potential: Diving Deep into the World of Generative Artif...
bybanarasone9
 
Discovering Generative AI's Creative Power: A Deep Dive Into Neural Networks
byArnav Malhotra
 
Slides, thesis dissertation defense, deep generative neural networks for nove...
bymehdi Cherti
 
Introduction to Generative AI_Engineering.pptx
bypratyushnayak777
 
Generative AI Descriptive MOdel ai .pptx
byKanchanaRSVVV
 
Creativity through deep learning
byAkin Osman Kazakci
 
Generative_AI_Detailed_Presentation.pptx
byಚಂದ್ರು ಗುಡದಾ
 
Fostering creativity in autonomous systems
bySarah Shuchi
 
Deep learning applications - for fun and profit
byRoberto Pereira Silveira
 
Introduction to Generative AI refers to a subset of artificial intelligence
byKongu Engineering College, Perundurai, Erode
 
introduction to the world of generative AI
byssudhar40
 
2 day Deep Learning Workshop at Karunya - Session 1
byRajagopal A
 
Generative AI Development Services | GenAI Consulting Company | Eoxys IT
byeoxysitsolutionllp
 
Some Preliminary Thoughts on Artificial Intelligence - April 20, 2023.pdf
byKent Bye
 

Recently uploaded

PPTX
Dan Herod February 01 2026.pptx
byFamilyWorshipCenterD
 
PDF
Technical Terms_20260202_220329_0000.pdf
bykierallenvelilla123
 
PDF
Top 10 Tips To Buy Verified Binance Accounts_ An Educational Guide topselleri...
bygsghhgf8
 
PPTX
Advanced methods of bacterial diagnosis a
bysalisusaniyusuf4
 
PPTX
alkali metal physical and chemical properties
byolurantifaith43
 
PPTX
HUMAN RESOURCE STRATEGY & PERSONNEL PLANNING.pptx
bynaimaengfarah76
 
PDF
Chemical_Summit_2026_Presentation (2).pdf
bychemicalsummit2026
 
PDF
Presentation - French Revolution (Student Lecture).pdf
byacdebussy
 
PDF
MENTAL HEALTH SESSION BY REHEMA MUTHONI MUCHAI.
by2402232
 
PDF
management_managementSCL2026_jpg2pdf.pdf
by東京北医療センター
 
PDF
SourceofContaminationofrawmilkinspoilage
bydharanin33
 
PPT
Slides for module 3_Community mobilization.ppt
bymangwirobz
 
PDF
Novel training approaches for the public administration in the context of Gre...
byProf. Dr. Fotios Fitsilis
 
PDF
Nanoscience in food preservation, Microencapsulation
bydharanin33
 
PDF
What’s New in the Latest Roblox Update (2026)
bytenaciousmunir77
 
PPT
Slides for Module 4_ MAM Management in SFP.ppt
bymangwirobz
 
PPTX
Virtual Orientation on the Administration of CBNCAE ppt.pptx
byMikeJeronMartinez1
 
PPT
COMMUNICATION SKILLS- CTE_Achieving Excellence in Performance and Strategies ...
byannliyatresava
 
PPTX
Febelfin Academy Presentation - Breakfast
byFinTech Belgium
 
PPTX
2026-02-01 Hebrews 05 (shared slides).pptx
byDale Wells
 
Dan Herod February 01 2026.pptx
byFamilyWorshipCenterD
 
Technical Terms_20260202_220329_0000.pdf
bykierallenvelilla123
 
Top 10 Tips To Buy Verified Binance Accounts_ An Educational Guide topselleri...
bygsghhgf8
 
Advanced methods of bacterial diagnosis a
bysalisusaniyusuf4
 
alkali metal physical and chemical properties
byolurantifaith43
 
HUMAN RESOURCE STRATEGY & PERSONNEL PLANNING.pptx
bynaimaengfarah76
 
Chemical_Summit_2026_Presentation (2).pdf
bychemicalsummit2026
 
Presentation - French Revolution (Student Lecture).pdf
byacdebussy
 
MENTAL HEALTH SESSION BY REHEMA MUTHONI MUCHAI.
by2402232
 
management_managementSCL2026_jpg2pdf.pdf
by東京北医療センター
 
SourceofContaminationofrawmilkinspoilage
bydharanin33
 
Slides for module 3_Community mobilization.ppt
bymangwirobz
 
Novel training approaches for the public administration in the context of Gre...
byProf. Dr. Fotios Fitsilis
 
Nanoscience in food preservation, Microencapsulation
bydharanin33
 
What’s New in the Latest Roblox Update (2026)
bytenaciousmunir77
 
Slides for Module 4_ MAM Management in SFP.ppt
bymangwirobz
 
Virtual Orientation on the Administration of CBNCAE ppt.pptx
byMikeJeronMartinez1
 
COMMUNICATION SKILLS- CTE_Achieving Excellence in Performance and Strategies ...
byannliyatresava
 
Febelfin Academy Presentation - Breakfast
byFinTech Belgium
 
2026-02-01 Hebrews 05 (shared slides).pptx
byDale Wells
 

Creative AI & multimodality: looking ahead

  • 1.
    Creative AI & multimodality: lookingahead Roelof Pieters @graphific Imperial College London, 
 1 Dec 2015 roelof@graph-technologies.comhttp://artificialexperience.com/http://www.csc.kth.se/~roelof/
  • 2.
  • 3.
    AI I kinda expectthe audience to know AI & Machine Learning
 Let’s move on shall we ?
  • 4.
    AI All references to: -Arxiv or - GitXiv if the “code” or “dataset” is available Collaborative Open Computer Science more info (Medium)
  • 5.
  • 6.
  • 7.
    “Deep learning isa set of algorithms in machine learning that attempt to learn in multiple levels, corresponding to different levels of abstraction.”
  • 8.
    AI > today’sfocus use of several modes (media) to create a single artifact. Multimodality “Mode” Socially and culturally shaped resource for making meaning. — Gunther Kress
  • 9.
  • 10.
    Creativity • Many definitions:philosophical, sociological, historical, practical
  • 11.
    Creativity 1. Making unfamiliarcombinations of familiar ideas. 2. Explore a structured conceptual space 3. (Radically) transforming ones structured conceptual space “Exploration” “Remix” “The Creative Mind”
 — Margaret Boden “Transformation”
  • 12.
    • Skill • Appreciation •Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality. Creativity > “Traits” software has to exhibit in order to avoid easy criticism of being “non-creative”. (Simon Colton)
  • 13.
    • Skill • Appreciation •Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
  • 14.
    • Skill • Appreciation •Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
  • 15.
    • Skill • Appreciation •Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
  • 16.
    • Skill • Appreciation •Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
  • 17.
    • Skill • Appreciation •Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
  • 18.
    • Skill • Appreciation •Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
  • 19.
    • Skill • Appreciation •Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
  • 20.
    • Skill • Appreciation •Imagination • Learning • Innovation • Accountability, • Subjectivity • Intentionality Creativity > software traits
  • 21.
  • 22.
    Creative AI >Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
  • 23.
    Creative AI >Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
  • 24.
    Creative AI >Current possibilities > Appropriating “standard” nets for creative use Deep Dream see also: www.csc.kth.se/~roelof/deepdream/
  • 25.
    Creative AI >Current possibilities > Appropriating “standard” nets for creative use Deep Dream see also: www.csc.kth.se/~roelof/deepdream/ codeyoutubeRoelof Pieters 2015
  • 26.
    Creative AI >Current possibilities > Appropriating “standard” nets for creative use Deep Dream see also: www.csc.kth.se/~roelof/deepdream/ C.M.Kosemen & 
 Roelof Pieters (2015) Gizmodo
  • 27.
    Creative AI >Current possibilities > Appropriating “standard” nets for creative use Leon A. Gatys, Alexander S. Ecker, Matthias Bethge , 2015. 
 A Neural Algorithm of Artistic Style (GitXiv) Style Net
  • 29.
    Gene Kogan, 2015.Why is a Raven Like a Writing Desk? (vimeo)
  • 30.
    Creative AI >Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
  • 32.
    Creative AI >Current possibilities > Reinforcement Learning • AMN: Emilio Parisotto, Jimmy Lei Ba, Ruslan Salakhutdinov 2015, Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning (arxiv) • DQN: Mnih, Volodymyr, Kavukcuoglu, Koray, Silver, David, Rusu, Andrei A., Veness, Joel, Bellemare, Marc G., Graves, Alex, Riedmiller, Martin, Fidjeland, Andreas K., Ostrovski, Georg, Petersen, Stig, Beattie, Charles, Sadik, Amir, Antonoglou, Ioannis, King, Helen, Kumaran, Dharshan, Wierstra, Daan, Legg, Shane, and Hassabis, Demis. Human-level control through deep reinforcement learning. Nature, 518(7540): 529–533, 2015.
  • 33.
    Creative AI >Current possibilities > Reinforcement Learning Ardi Tampuu, Tambet Matiisen, Dorian Kodelja, Ilya Kuzovkin, Kristjan Korjus, Juhan Aru, Jaan Aru, Raul Vicente, 2015 
 Multiagent Cooperation and Competition with Deep Reinforcement Learning (GitXiv) (YouTube)
  • 34.
    Reinforcement Learning Ning Xie,Hirotaka Hachiya, Masashi Sugiyama, 2013 , 
 Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting (Paper, Lecture, YouTube)
  • 35.
  • 36.
    Ning Xie, HirotakaHachiya, Masashi Sugiyama, 2013
 Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting (Paper, Lecture, YouTube)
  • 37.
    Creative AI >Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
  • 38.
    Creative AI >Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
  • 39.
    Creative AI >Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-encoders • Attention-based Models • Generative Adversarial Nets
  • 40.
    Creative AI >Current possibilities • Standard (“denoising”) Autoencoders • Variational Autoencoder (VAE) / Stochastic Gradient VB • Deep Convolutional Inverse Graphics Network • Variational RNN (VRNN) Vincent et al, 2010. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion (paper) (code)
  • 41.
    Creative AI >Current possibilities • Standard “denoising” Autoencoders • Variational Autoencoder (VAE) / Stochastic Gradient VB • Deep Convolutional Inverse Graphics Network • Variational RNN (VRNN) • Diederik P Kingma, Max Welling, 2013. 
 Auto-Encoding Variational Bayes (GitXiv)
  • 42.
    Creative AI >Current possibilities • Standard “denoising” Autoencoders • Variational Autoencoder (VAE) • Deep Convolutional Inverse Graphics Network (modified VAE) • Variational RNN (VRNN) Tejas D. Kulkarni, Will Whitney, Pushmeet Kohli, Joshua B. Tenenbaum, 2015 Deep Convolutional Inverse Graphics Network (GitXiv)
  • 43.
    Creative AI >Current possibilities • Standard “denoising” Autoencoders • Variational Autoencoder (VAE) • Deep Convolutional Inverse Graphics Network • Variational RNN (VRNN) (VAE at every time step) Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron Courville, Yoshua Bengio, 2015
 A Recurrent Latent Variable Model for Sequential Data (GitXiv) VAEVAEVAE
  • 44.
    Junyoung Chung, KyleKastner, Laurent Dinh, Kratarth Goel, Aaron Courville, Yoshua Bengio , 2015. 
 A Recurrent Latent Variable Model for Sequential Data (GitXiv) (Audio Samples)
  • 45.
    Creative AI >Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adversarial Nets
  • 46.
    Karol Gregor, IvoDanihelka, Alex Graves, Danilo Jimenez Rezende, Daan Wierstra, 2015
 DRAW: A Recurrent Neural Network For Image Generation (GitXiv) Variational Auto-Encoder Deep Recurrent Attentive Writer (DRAW) Network
  • 47.
  • 48.
    Creative AI >Current possibilities • Appropriating “standard” nets for creative use • Reinforcement Learning: Creativity as a Game • RNNs/LSTMs/GRUs • Sequence-to-Sequence: Creativity as a Translation Task • Auto-Encoders • Attention-based Models • Generative Adverserial Nets
  • 49.
    Emily Denton, SoumithChintala, Arthur Szlam, Rob Fergus, 2015. 
 Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks (GitXiv)
  • 50.
    Alec Radford, LukeMetz, Soumith Chintala , 2015. 
 Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (GitXiv)
  • 51.
    Alec Radford, LukeMetz, Soumith Chintala , 2015. 
 Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (GitXiv)
  • 52.
    ”turn” vector createdfrom four averaged samples of faces looking left vs looking right. Alec Radford, Luke Metz, Soumith Chintala , 2015. 
 Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (GitXiv)
  • 53.
  • 54.
    top: unmodified samples bottom:same samples dropping out ”window” filters
  • 55.
    Autonomy Supervision Creativity? - unsupervisedtraining - generator/discrimator - latent/z space - auto encoders - multimodality - query - target/class
  • 56.
  • 57.
    Creative AI >Needs as I see it Creative AI as a “tool”
 or “brush” to paint with
  • 58.
    A system whichmarries the need for a creative process with the need for a creative output • with as less human input as possible (data) • with its own style • with the possibility for human level supervision for rapid experimentation Creative AI > a “brush”
  • 59.
    A system whichmarries the need for a creative process with the need for a creative output • with as less human input as possible ( ) • with its own style • with the possibility for human level supervision for rapid experimentation Creative AI > a “brush” data
  • 60.
    Creative AI >a “brush” > data • reuse nets as much as possible • combining unsupervised & supervised • multiple modalities • plug in external knowledge bases
  • 61.
    Creative AI >a “brush” > data input • unlabeled & labeled data • external knowledge bases (dbpedia, wikipedia) • one-shot learning • zero-shot learning Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Christopher D. Manning, Andrew Y. Ng, 2013
 Zero-Shot Learning Through Cross-Modal Transfer a zero-shot model that can predict both seen and unseen classes
  • 62.
    Creative AI >a “brush” > data input Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Christopher D. Manning, Andrew Y. Ng, 2013
 Zero-Shot Learning Through Cross-Modal Transfer (slides)
  • 63.
    Creative AI >a “brush” > data input Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Christopher D. Manning, Andrew Y. Ng, 2013
 Zero-Shot Learning Through Cross-Modal Transfer (slides)
  • 64.
    Creative AI >a “brush” > data input Richard Socher, Milind Ganjoo, Hamsa Sridhar, Osbert Bastani, Christopher D. Manning, Andrew Y. Ng, 2013
 Zero-Shot Learning Through Cross-Modal Transfer (slides)
  • 65.
    A system whichmarries the need for a creative process with the need for a creative output • with as less human input as possible (data) • with its own style • with the possibility for human level 
 for rapid experimentation Creative AI > a “brush” supervision
  • 66.
    Creative AI >a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
  • 67.
    Creative AI >a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
  • 68.
    Creative AI >a “brush” > data Deep Dream Alexander Mordvintsev, Christopher Olah, Mike Tyka, 2015. 
 Inceptionism: Going Deeper into Neural Networks Google Research Blog
  • 69.
    Creative AI >a “brush” > data Deep Dream Roelof Pieters, 2015 DeepDream - Class visualization Experiment (link)
  • 70.
    Roelof Pieters, 2015DeepDream - Class visualization Experiment (link)
  • 71.
    Creative AI >a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
  • 72.
    Creative AI >a “brush” > data Deep Dream Roelof Pieters, 2015 DeepDream - Overview of standard bvlc googlenet (inception) layers (link) Constrain Layers
  • 73.
    Creative AI >a “brush” > data Deep Dream Roelof Pieters, 2015 Single Unit Activations (early layer) (Flickr Album) Constrain Units
  • 74.
    Creative AI >a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
  • 75.
    Creative AI >a “brush” > data Deep Dream Roelof Pieters, 2015 DeepDream Video (GitHub)
  • 76.
    Creative AI >a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
  • 77.
    Creative AI >a “brush” > data Style Net Roelof Pieters (graphific) (tweet) Roelof Pieters (graphific) (tweet)
  • 78.
    Creative AI >a “brush” > data • “rich” latent (“z”) space • easy user supervision over output: • priors • constrain network (units, layers, etc) • guided input • mixed input • latent space
  • 79.
    Image -> Text “Aperson riding a motorcycle on a dirt road.”???
  • 80.
    Image -> Text “Twohockey players are fighting over the puck.”???
  • 81.
    Image -> Text KelvinXu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua Bengio, Show, Attend and Tell: Neural Image Caption Generation with Visual Attention (arxiv) (info) (code) Andrej Karpathy Li Fei-Fei , 2015. 
 Deep Visual-Semantic Alignments for Generating Image Descriptions (pdf) (info) (code) Oriol Vinyals, Alexander Toshev, Samy Bengio, Dumitru Erhan , 2015. Show and Tell: A Neural Image Caption Generator (arxiv)
  • 82.
    Text -> Image“A stop sign is flying in blue skies.” “A herd of elephants flying in the blue skies.” Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, Ruslan Salakhutdinov, 2015. Generating Images from Captions with Attention (arxiv) (examples)
  • 83.
    Elman Mansimov, EmilioParisotto, Jimmy Lei Ba, Ruslan Salakhutdinov, 2015. Generating Images from Captions with Attention (arxiv) (examples) Text -> Image
  • 84.
    Subhashini Venugopalan, MarcusRohrbach, Jeff Donahue, Raymond Mooney, Trevor Darrell, Kate Saenko , 2015. Sequence to Sequence -- Video to Text (GitXiv) Video -> Text
  • 85.
    A system whichmarries the need for a creative process with the need for a creative output • with as less human input as possible (data) • with its own style • with the possibility for human level supervision for 
 Creative AI > a “brush” rapid experimentation
  • 86.
    Creative AI >a “brush” > rapid experimentation
  • 87.
    Widening Deepening Tianqi Chen, IanGoodfellow, Jonathon Shlens, 2015. Net2Net: Accelerating Learning via Knowledge Transfer (arxiv) / code (torch) Reusing Nets: Bigger Net
  • 88.
    Teacher and Studentnet Hint training Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, Yoshua Bengio, 2014. FitNets: Hints for Thin Deep Nets (arxiv) Knowledge distillation SVHN Error MNIST Error Reusing Nets: Smaller Net
  • 89.
    Hashed Net Wenlin Chen,James T. Wilson, Stephen Tyree, Kilian Q. Weinberger, Yixin Chen, 2015. Compressing Neural Networks with the Hashing Trick (arxiv) Shrinking Nets: Hashing
  • 90.
    Song Han, HuiziMao, William J. Dally, 2015. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding (arxiv) Shrinking Nets: Pruning, Quantization & Huffman coding
  • 91.
    Creative AI >a “brush” > rapid experimentation • experiments need “tooling”, specialised design software to • try things • explore latent spaces (z-space) • push the AI in the right direction • be surprised by AI
  • 92.
    Creative AI >a “brush” > rapid experimentation human-machine collaboration
  • 93.
    Creative AI >a “brush” > rapid experimentation (YouTube, Paper)
  • 94.
    Creative AI >a “brush” > rapid experimentation (YouTube, Paper)
  • 95.
    Creative AI >a “brush” > rapid experimentation (Vimeo, Paper)
  • 96.
    Creative AI >a “brush” > rapid experimentation • Advertising and marketing • Architecture • Crafts • Design: product, graphic and fashion design • Film, TV, video, radio and photography • IT, software and computer services • Publishing • Museums, galleries and libraries • Music, performing and visual arts
  • 97.
    Questions? love letters? existentialdilemma’s? academic questions? gifts? find me at:
 www.csc.kth.se/~roelof/ roelof@kth.se