SlideShare a Scribd company logo
1 of 84
AI Tools for Creative Work
Lydia B. Chilton
DUB Seminar
2 October 2019
AI can do things it couldn’t do before…
But it still struggles at ill-defined tasks.
It looks like you’re trying to
solve world peace.
Do you need assistance?
AI can do things it couldn’t do before…
But it still struggles at ill-defined tasks.
Long history of creativity support tools in HCI
Jonas Frich, Lindsay MacDonald Vermeulen, Christian Remy, Michael Mose Biskjaer, and Peter Dalsgaard. 2019.
Mapping the Landscape of Creativity Support Tools in HCI. CHI 2019
How does AI change the future of creative work?
Gatys, Ecker, Bethge. A Neural Algorithm of Artistic Style. http://arxiv.org/abs/1508.06576 2015
Outline: AI Tools for Creative Work
• Introduce A Creative Problem: Visual Metaphors
• Can AI do it?
• VisiBlends: A Workflow for Prototyping Visual Metaphors
• Symbol Finder: Adding AI to the workflow
• VisiFit: AI Tools for Iteratively Improving Visual Blends
• The Future of AI and creative work
AdvertisementsNews Public Service Announcements
Earth + MeltBrazil + Takes Off Tabasco + Hot
Visual Metaphors: image that convey a message symbolically
How can we create visual blends for a message?
Outline: AI Tools for Creative Work
• Introduce A Creative Problem: Visual Metaphors
• Can AI do it?
• VisiBlends: A Workflow for Prototyping Visual Metaphors
• Symbol Finder: Adding AI to the workflow
• VisiFit: AI Tools for Iteratively Improving Visual Blends
• The Future of AI and creative work
With successes in deep learning,
what is the future of creative work?
Visual Metaphor:
Lego are Summer Vacation Fun
Artist blend:
Deep Style Transfer
Artist:AI:
Visual Metaphor:
Burgers are Healthy, like apples
Artist:AI:
Visual Metaphor:
Burgers are Healthy, like apples
Artist:AI:
Blending objects requires more than just blending “style.”
You need more context about the objects - how to take them apart, blend the
pieces, think about how they’ll be perceived.
Outline: AI Tools for Creative Work
• Introduce A Creative Problem: Visual Metaphors
• Can AI do it?
• VisiBlends: A Workflow for Prototyping Visual Metaphors
• Symbol Finder: Adding AI to the workflow
• VisiFit: AI Tools for Iteratively Improving Visual Blends
• The Future of AI and creative work
But there is no simple pattern for visual blends
We need to break up the problem into tasks.
1. Two objects are integrated into one object
2. Both objects are individually identifiable
Visual Blending Structure:
Whole-to-Part Shape Matching
Abstract problem structure:
Single Shape Mapping
Why blend based on shape?
Shape is the primary feature the human visual system uses to recognize objects.
Colors, silhouette and details and secondary.
1. Two objects are integrated into one object
2. Both objects are individually identifiable
Visual Blending Structure:
Whole-to-Part Shape Matchingby using a design pattern to make it a search problem
We can solve a creative design problem
Earth Melt
Visual Blending Structure:
Whole-to-Part Shape Matching
Starbucks + Summer
Prototype
blend
Evaluate
prototype
Iterate
Output:
A visual blend
Two objects are integrated into one object
Both objects are identifiable
Prototype
blend
Evaluate
prototype
Two objects are integrated into one object
Both objects are identifiable
Prototype
blend
Evaluate
prototype
Iterate
Output:
A visual blend
Two objects are integrated into one object
Both objects are identifiable
Prototype
blend
Evaluate
prototype
Two objects are integrated into one object
Both objects are identifiable
Prototype
blend
Evaluate
prototype
Iterate
Output:
A visual blend
Two objects are integrated into one object
Both objects are identifiable
Prototype
blend
Evaluate
prototype
Iterate
Output:
A visual blend
Two objects are integrated into one object
Both objects are identifiable
e
e
Output:
A visual blend
grated into one object
ntifiable
A) Brainstorm
for summer
B) Find and annotate images
for summer
C) Matching algorithm
Starbucks + summer
D) Automatic blends + human evaluation
Starbucks + summer
matches = []
for a in summer_symbols:
for b in starbucks_symbols:
a_ratio = a.height / a.width
b_ratio = b.height / b.width
ratios = sort(a_ratio, b_ratio)
if (a.shape == b.shape) and
(a.coverage != coverage) and
(ratios[0] >= 0.5*ratios[1]):
matches.push([a, b])
return matches
VisiBlends:
A web interface to collaboratively make blends
Blends made by groups at
Columbia University
Football
+
dangerous
Joe’s Coffee
+
Night
Wash your hands.
It’s the smart move.
Forget the knives and forks: the humble hamburger is in
the midst of a renaissance. From Melbourne to Milan,
restaurateurs have helped elevate this classic piece of
American folklore into an iconic dish. Every day, more
and more enthusiasts join the search for the perfect
Philosophy
+
Christmas
Does VisiBlends make people better?
• 13 undergraduates (novice designers)
• Each made 6 blends (3 with VisiBlends, 3 without)
• 5 minutes for each blend
• Metric: How many successful blends can 5 minutes?
Novices are 10x better with VisiBlends
Control-first
Avg. Number of
blends made
VisiBlends-first
Without VisiBlends With VisiBlends
Without VisiBlendsWith VisiBlends
Avg. Number of
blends made
0.56 5.56 (t(18)=4.88, p<0.001)
0.675.67 (t(21)=5.84, p<0.001)
Bicycle + Smart
VisiBlendsUsers Unassisted
VisiBlends helps them have
many ideas, and pick the best.
Novices get stuck “improving”
their first idea.
New York City + Night
Users Unassisted
Novices pick “perfect” symbols,
but sacrifice shape-fit.
VisiBlends helps novices meet all
the criteria for a visual blend
(especially shape-fit).
VisiBlends
Do we want “perfect” symbols for Visual Blends?
By meeting more constraints,
Visual Metaphors are more
novel and interesting
Novices are 10x better with VisiBlends
10x improvement in novices ability
VisiBlendsUnassisted
Why?
Football + Dangerous
Football Dangerous
Brainstorm
associations
Find Images
of objects
Annotate
shapes
Annotate
shape coverage
Are both objects identifiable?
Are two objects integrated into one object?
Evaluate
Blend
Shape covers
All of object
Shape covers
Part of object
Football Dangerous
Brainstorm
associations
Find Images
of objects
Annotate
shapes
Annotate
shape coverage
Are both objects identifiable?
Are two objects integrated into one object?
Evaluate
Blend
Shape covers
All of object
Shape covers
Part of object
Collect pieces
Combine them
Re-combine them
Collect pieces
Lots of individual pieces under your finger;
it’s easy to recombine them.
Fluid
Intelligence
Crystallized
Intelligence
Having the resources and tools you need
(memory, routing procedures, etc)
Having the tools at hand so you can
fluidly combine them
Crystallized Intelligence
Crystallized Intelligence
Fluid Intelligence
Fluid Intelligence
VisiBlends augments fluid intelligence
Collect pieces
Combine them
Re-combine them
Collect pieces
NYC + Healthy
NYC Healthy
Brainstorm
associations
Find Images
of objects
Annotate
shapes
Annotate
shape coverage
Evaluate
Blend
No shape matches
NYC Healthy
Brainstorm
associations
Find Images
of objects
Annotate
shapes
Annotate
shape coverage
Are both objects identifiable?
Are two objects integrated into one object?
Evaluate
Blend
Orange + Healthy
Orange Healthy
Brainstorm
associations
Find Images
of objects
Annotate
shapes
Annotate
shape coverage
Shape covers
Part of object
Shape covers
All of object
Are both objects identifiable?
Are two objects integrated into one object?
Evaluate
Blend
Orange Healthy
Brainstorm
associations
Find Images
of objects
Annotate
shapes
Annotate
shape coverage
Shape covers
All of object
Shape covers
Part of object
Evaluate
Blend
Exercise equipment
Orange Healthy
Brainstorm
associations
Find Images
of objects
Annotate
shapes
Annotate
shape coverage
Shape covers
All of object
Shape covers
Part of object
Are both objects identifiable?
Are two objects integrated into one object?
Evaluate
Blend
Exercise equipment
Iteration requires fluidity –
putting together pieces in new ways.
No matches
Improve object fit
Objects are
not identifiable + =
+ =
Within same search space,
meet other constraints.
Find versions of an object
with different
color, style, aspect ratio
Focus on meeting a
specific constraint:
Find symbols with
a different shape
Search in a new
subspace
Find symbols with
a different shape
Crystallized Intelligence
Crystallized Intelligence
Fluid Intelligence
Fluid Intelligence
VisiBlends augments fluid intelligence
Collect pieces
Combine them
Re-combine them
Collect pieces
Outline: AI Tools for Creative Work
• Introduce A Creative Problem: Visual Metaphors
• Can AI do it?
• VisiBlends: A Workflow for Prototyping Visual Metaphors
• Symbol Finder: AI tools for finding symbols
• VisiFit: AI Tools for Iteratively Improving Visual Blends
• The Future of AI and creative work
Prototype
blend
Evaluate
prototype
Iterate
Output:
A visual blend
Two objects are integrated into one object
Both objects are identifiable
pe
e
pe
Output:
A visual blend
egrated into one object
entifiable
Rules for finding images are complex
Simply searching for abstract words doesn’t return
simple, iconic symbols that we can blend.
SymbolFinder uses AI + KBs
to find concrete images for an abstract term
Savvas Petridis
PhD Student
word2vec, synonyms, ConceptNet, concreteness classifiers
Prototype
blend
Evaluate
prototype
Iterate
Output:
A visual blend
Two objects are integrated into one object
Both objects are identifiable
pe
e
pe
Output:
A visual blend
egrated into one object
entifiable
Outline: AI Tools for Creative Work
• Introduce A Creative Problem: Visual Metaphors
• Can AI do it?
• VisiBlends: A Workflow for Prototyping Visual Metaphors
• Symbol Finder: AI tools for finding symbols
• VisiFit: AI Tools for Iteratively Improving Visual Blends
• The Future of AI and creative work
How often do VisiBlend prototypes need editing?
Output:
A visual blend
object
4 of 27 were ok to print on social media
(according to 2 evaluators who are graphic designers)
How do we improve the rest of them?
Fully automatic approaches fail
Artist:AI:
Design Principle #1 :
Instead of pursuing fully automatic approaches,
break up the objects into visual components that can each be blended.
Design Principle #2:
When improving a blend,
Recombine secondary visual aspects: color, silhouette, details.
Co-Design with 2 Graphic Designers
Formative study with novice designer
Design Principle #3:
Translating intent into action is a barrier for novices.
Use AI tools to automate actions or
provide more direct mapping between intent and action
“I want to make the metro card the
same shape as the purse.”
VisiFit: AI Tools to Iteratively Improve Visual Blends
Each visual aspect has it’s own AI tool for novices to apply
AI tools to improve each visual dimension
Color blending
Crop to extracted
silhouette
Deep Object
Segmentation
Interactive
Object
Segmentation
Interactive
Object
Segmentation
Evaluation
Improved:
97.5% of cases
11 Novice designers (8 female)
1 hr study
11 blends to extract and improve
Publishable quality:
65.3% of the time.
Experts Publishable:
87.0% of the time.
121 total blends produced
Outputs rated by 2 designers:
Improved, publishable.
Non-Publishable blends
1. Grabcut failed to extract details 2. No tool to fill parts of the bottom object.
(Add a fill tool)
Response of Professional Visual Blend Designers
1. Amazed by the AI tools
They did try things no novice did
(even if it didn’t work).
2. Much easier than “pixel pushing”.
Enabled them to explore more designs quickly and
easily.
But did suggest more PhotoShop features to all,
like Context Aware Fill
Wanted it as a plugin to sketch.
Everyone else Designer
Outline: AI Tools for Creative Work
• Introduce A Creative Problem: Visual Metaphors
• Can AI do it?
• VisiBlends: A Workflow for Prototyping Visual Metaphors
• Symbol Finder: AI tools for finding symbols
• VisiFit: AI Tools for Iteratively Improving Visual Blends
• The Future of AI and creative work
Crystallized
Intelligence
Crystallized
Intelligence
Fluid
Intelligence
Fluid
Intelligence
AI Tools can augment fluid intelligence
Collect pieces
Combine them
Combine them
Collect pieces
by making it easy to collect and combine pieces
If we identify the key aspects of the problem
VisiBlends collects and combines the
shape of blended objects
VisiFit extracts and combines the
color, silhouette, details of the objects
AI tools can help collect and combine elements for each aspect
AI Tools for writers:
Key aspects of writing:
Substance, structure, and style.
Metaphoria (CHI 2019):
Within the metaphor structure,
helps generate substance.
Style Thesaurus (C&C 2019):
Same substance, new style
(word level)
Language Style Transfer (INLG 2019):
Same substance, new style
(sentence level)
Katy Gero
PhD Candidate
Characteristic motion
Reference video
Static image
Future Work: Animated Blends
Use AI to extract key aspects of motion (speed),
and reapply them to the blend (use a different path).
Ecenaz Jen Ozmen
Columbia CC ‘21
Claire Chen
Columbia CC ‘21
Maria Kogan
Columbia SEAS ‘21
Sam Ross
Barnard ‘21
Annie Sui
Columbia CC ‘22
Maneesh Agrawala
Stanford
Katy Gero
PhD Candidate
Savvas Petridis
PhD Candidate
Collaborators
Summary
Future of Creative Work:
AI should augment fluid intelligence
AI shouldn’t try to understand complexities of culture and working context.
AI can provide people the tools and resources
to more easily turn users’ intent into action
AI Artist Novice with AI Tools
VisiBlends: AI Tools for Prototyping Visual Blends
VisiBlendsUnassisted
10x improvement in novices’ ability
VisiFit: AI Tools to Iteratively Improve Visual Blends
97.5% of cases improved, 65% to publishable quality
Crystallized
Intelligence
Crystallized
Intelligence
Fluid
Intelligence
Fluid
Intelligence
VisiBlends augments fluid intelligence
Collect pieces
Combine them
Combine them
Collect pieces
by making it easy to collect and combine pieces
Characteristic motion
Reference video
Static image
Future Work: Animated Blends
Use AI to extract key aspects of motion,
and reapply them to the blend.
Iteratively Improving: one key element at a time
Prototyping: collect and recombined parts Brainstorming: easily collect parts
AI Tools for Creative Work
Repurposing: Elements of a Reference Object

More Related Content

What's hot

Introduction To Visual Design
Introduction To Visual DesignIntroduction To Visual Design
Introduction To Visual DesignDiane Leeper
 
Visual and Creative Thinking:What We Learned From Peter Pan and Willy Wonka
Visual and Creative Thinking:What We Learned From Peter Pan and Willy WonkaVisual and Creative Thinking:What We Learned From Peter Pan and Willy Wonka
Visual and Creative Thinking:What We Learned From Peter Pan and Willy WonkaKelsey Ruger
 
Better Twitch Broadcasting through Rapid Prototyping & Human Centered Design
Better Twitch Broadcasting through Rapid Prototyping & Human Centered DesignBetter Twitch Broadcasting through Rapid Prototyping & Human Centered Design
Better Twitch Broadcasting through Rapid Prototyping & Human Centered DesignDigital Surgeons
 
Visual Design Thinking Workshop
Visual Design Thinking WorkshopVisual Design Thinking Workshop
Visual Design Thinking WorkshopAlli McKee
 
Modern Graphic Design
Modern Graphic DesignModern Graphic Design
Modern Graphic DesignYekaterina22
 
The rise of AI in design - Are we losing creative control? IXDS Pre-Work Talk...
The rise of AI in design - Are we losing creative control? IXDS Pre-Work Talk...The rise of AI in design - Are we losing creative control? IXDS Pre-Work Talk...
The rise of AI in design - Are we losing creative control? IXDS Pre-Work Talk...Jan Korsanke
 
History of Graphic Design
History of Graphic DesignHistory of Graphic Design
History of Graphic DesignChris Edwards
 
Visual communication
Visual communicationVisual communication
Visual communicationRida Farooq
 
AI Presentation.pptx
AI Presentation.pptxAI Presentation.pptx
AI Presentation.pptxPTejaswini6
 

What's hot (20)

Graphic Designing
Graphic DesigningGraphic Designing
Graphic Designing
 
Design thinking
Design thinkingDesign thinking
Design thinking
 
What Is Design
What Is DesignWhat Is Design
What Is Design
 
Introduction To Visual Design
Introduction To Visual DesignIntroduction To Visual Design
Introduction To Visual Design
 
Design process
Design processDesign process
Design process
 
What Is Design?
What Is Design?What Is Design?
What Is Design?
 
The UX of Virtual Reality
The UX of Virtual RealityThe UX of Virtual Reality
The UX of Virtual Reality
 
Typography
TypographyTypography
Typography
 
Inclusive design workshop
Inclusive design workshopInclusive design workshop
Inclusive design workshop
 
Visual and Creative Thinking:What We Learned From Peter Pan and Willy Wonka
Visual and Creative Thinking:What We Learned From Peter Pan and Willy WonkaVisual and Creative Thinking:What We Learned From Peter Pan and Willy Wonka
Visual and Creative Thinking:What We Learned From Peter Pan and Willy Wonka
 
Better Twitch Broadcasting through Rapid Prototyping & Human Centered Design
Better Twitch Broadcasting through Rapid Prototyping & Human Centered DesignBetter Twitch Broadcasting through Rapid Prototyping & Human Centered Design
Better Twitch Broadcasting through Rapid Prototyping & Human Centered Design
 
AI Art
AI Art AI Art
AI Art
 
The art of AI Art
The art of AI ArtThe art of AI Art
The art of AI Art
 
Chap 1 visual communication
Chap 1 visual communicationChap 1 visual communication
Chap 1 visual communication
 
Visual Design Thinking Workshop
Visual Design Thinking WorkshopVisual Design Thinking Workshop
Visual Design Thinking Workshop
 
Modern Graphic Design
Modern Graphic DesignModern Graphic Design
Modern Graphic Design
 
The rise of AI in design - Are we losing creative control? IXDS Pre-Work Talk...
The rise of AI in design - Are we losing creative control? IXDS Pre-Work Talk...The rise of AI in design - Are we losing creative control? IXDS Pre-Work Talk...
The rise of AI in design - Are we losing creative control? IXDS Pre-Work Talk...
 
History of Graphic Design
History of Graphic DesignHistory of Graphic Design
History of Graphic Design
 
Visual communication
Visual communicationVisual communication
Visual communication
 
AI Presentation.pptx
AI Presentation.pptxAI Presentation.pptx
AI Presentation.pptx
 

Similar to AI Tools for Creative Work

VisiBlends: A Flexible Workflow for Visual Blends
VisiBlends: A Flexible Workflow for Visual BlendsVisiBlends: A Flexible Workflow for Visual Blends
VisiBlends: A Flexible Workflow for Visual Blendshmslydia
 
Crowdsourcing Visual Ads
Crowdsourcing Visual AdsCrowdsourcing Visual Ads
Crowdsourcing Visual Adshmslydia
 
This is not a pipe
This is not a pipeThis is not a pipe
This is not a pipek_thrn
 
1st Bimester Project
1st Bimester Project1st Bimester Project
1st Bimester Projectwatler
 
Real-life Data Visualization - guest lecture for McGill INSY-442
Real-life Data Visualization - guest lecture for McGill INSY-442Real-life Data Visualization - guest lecture for McGill INSY-442
Real-life Data Visualization - guest lecture for McGill INSY-442Mike Deutsch
 
OTC11-Using Right Brain Thinking in EdTech Innovation
OTC11-Using Right Brain Thinking in EdTech InnovationOTC11-Using Right Brain Thinking in EdTech Innovation
OTC11-Using Right Brain Thinking in EdTech InnovationMichael Kieley
 
Belmer portfolio overview
Belmer portfolio overviewBelmer portfolio overview
Belmer portfolio overviewBelmer Negrillo
 
Visual Tools and Innovation Games - Full Day Workshop - DevIntersections - No...
Visual Tools and Innovation Games - Full Day Workshop - DevIntersections - No...Visual Tools and Innovation Games - Full Day Workshop - DevIntersections - No...
Visual Tools and Innovation Games - Full Day Workshop - DevIntersections - No...Michelle Caldwell, PSM, SSGB
 
Bring the intangible to life with Concept Visualization: Part 3 of 3
Bring the intangible to life with Concept Visualization: Part 3 of 3Bring the intangible to life with Concept Visualization: Part 3 of 3
Bring the intangible to life with Concept Visualization: Part 3 of 3Deliverable Coaching
 
BrightonSEO 2018 (September 28): How to Optimize for Visual Search
BrightonSEO 2018 (September 28): How to Optimize for Visual SearchBrightonSEO 2018 (September 28): How to Optimize for Visual Search
BrightonSEO 2018 (September 28): How to Optimize for Visual SearchClark Boyd
 
How to create pro slides in less time: don't worry, be crappy!
How to create pro slides in less time:  don't worry, be crappy!How to create pro slides in less time:  don't worry, be crappy!
How to create pro slides in less time: don't worry, be crappy!Chiara Ojeda
 
Systems Thinking and Unthinkable Thoughts
Systems Thinking and Unthinkable ThoughtsSystems Thinking and Unthinkable Thoughts
Systems Thinking and Unthinkable ThoughtsJoel Eden, PhD
 
Leveraging Social Media with Computer Vision
Leveraging Social Media with Computer VisionLeveraging Social Media with Computer Vision
Leveraging Social Media with Computer VisionTJ Torres
 
Imaging and Design for Online Environment
Imaging and Design for Online EnvironmentImaging and Design for Online Environment
Imaging and Design for Online EnvironmentRichard Meriveles
 

Similar to AI Tools for Creative Work (20)

VisiBlends: A Flexible Workflow for Visual Blends
VisiBlends: A Flexible Workflow for Visual BlendsVisiBlends: A Flexible Workflow for Visual Blends
VisiBlends: A Flexible Workflow for Visual Blends
 
Crowdsourcing Visual Ads
Crowdsourcing Visual AdsCrowdsourcing Visual Ads
Crowdsourcing Visual Ads
 
Visual Thinking
Visual ThinkingVisual Thinking
Visual Thinking
 
This is not a pipe
This is not a pipeThis is not a pipe
This is not a pipe
 
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
October 28, 2015 NISO Virtual Conference Interacting with Content: Improving ...
 
Ideatoons
IdeatoonsIdeatoons
Ideatoons
 
1st Bimester Project
1st Bimester Project1st Bimester Project
1st Bimester Project
 
Real-life Data Visualization - guest lecture for McGill INSY-442
Real-life Data Visualization - guest lecture for McGill INSY-442Real-life Data Visualization - guest lecture for McGill INSY-442
Real-life Data Visualization - guest lecture for McGill INSY-442
 
OTC11-Using Right Brain Thinking in EdTech Innovation
OTC11-Using Right Brain Thinking in EdTech InnovationOTC11-Using Right Brain Thinking in EdTech Innovation
OTC11-Using Right Brain Thinking in EdTech Innovation
 
Belmer portfolio overview
Belmer portfolio overviewBelmer portfolio overview
Belmer portfolio overview
 
Visual Tools and Innovation Games - Full Day Workshop - DevIntersections - No...
Visual Tools and Innovation Games - Full Day Workshop - DevIntersections - No...Visual Tools and Innovation Games - Full Day Workshop - DevIntersections - No...
Visual Tools and Innovation Games - Full Day Workshop - DevIntersections - No...
 
Task 1
Task 1Task 1
Task 1
 
Creativity
CreativityCreativity
Creativity
 
Bring the intangible to life with Concept Visualization: Part 3 of 3
Bring the intangible to life with Concept Visualization: Part 3 of 3Bring the intangible to life with Concept Visualization: Part 3 of 3
Bring the intangible to life with Concept Visualization: Part 3 of 3
 
BrightonSEO 2018 (September 28): How to Optimize for Visual Search
BrightonSEO 2018 (September 28): How to Optimize for Visual SearchBrightonSEO 2018 (September 28): How to Optimize for Visual Search
BrightonSEO 2018 (September 28): How to Optimize for Visual Search
 
Mind Mapping for CPAs
Mind Mapping for CPAsMind Mapping for CPAs
Mind Mapping for CPAs
 
How to create pro slides in less time: don't worry, be crappy!
How to create pro slides in less time:  don't worry, be crappy!How to create pro slides in less time:  don't worry, be crappy!
How to create pro slides in less time: don't worry, be crappy!
 
Systems Thinking and Unthinkable Thoughts
Systems Thinking and Unthinkable ThoughtsSystems Thinking and Unthinkable Thoughts
Systems Thinking and Unthinkable Thoughts
 
Leveraging Social Media with Computer Vision
Leveraging Social Media with Computer VisionLeveraging Social Media with Computer Vision
Leveraging Social Media with Computer Vision
 
Imaging and Design for Online Environment
Imaging and Design for Online EnvironmentImaging and Design for Online Environment
Imaging and Design for Online Environment
 

Recently uploaded

WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceSamy Fodil
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...FIDO Alliance
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIES VE
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024Lorenzo Miniero
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentationyogeshlabana357357
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfFIDO Alliance
 
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptxBT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptxNeo4j
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaCzechDreamin
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform EngineeringMarcus Vechiato
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctBrainSell Technologies
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераMark Opanasiuk
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty SecureFemke de Vroome
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Patrick Viafore
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPTiSEO AI
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsLeah Henrickson
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101vincent683379
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FIDO Alliance
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...CzechDreamin
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftshyamraj55
 

Recently uploaded (20)

WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptxBT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 

AI Tools for Creative Work

  • 1. AI Tools for Creative Work Lydia B. Chilton DUB Seminar 2 October 2019
  • 2.
  • 3. AI can do things it couldn’t do before… But it still struggles at ill-defined tasks.
  • 4. It looks like you’re trying to solve world peace. Do you need assistance? AI can do things it couldn’t do before… But it still struggles at ill-defined tasks.
  • 5. Long history of creativity support tools in HCI Jonas Frich, Lindsay MacDonald Vermeulen, Christian Remy, Michael Mose Biskjaer, and Peter Dalsgaard. 2019. Mapping the Landscape of Creativity Support Tools in HCI. CHI 2019
  • 6. How does AI change the future of creative work? Gatys, Ecker, Bethge. A Neural Algorithm of Artistic Style. http://arxiv.org/abs/1508.06576 2015
  • 7. Outline: AI Tools for Creative Work • Introduce A Creative Problem: Visual Metaphors • Can AI do it? • VisiBlends: A Workflow for Prototyping Visual Metaphors • Symbol Finder: Adding AI to the workflow • VisiFit: AI Tools for Iteratively Improving Visual Blends • The Future of AI and creative work
  • 8. AdvertisementsNews Public Service Announcements Earth + MeltBrazil + Takes Off Tabasco + Hot Visual Metaphors: image that convey a message symbolically
  • 9. How can we create visual blends for a message?
  • 10. Outline: AI Tools for Creative Work • Introduce A Creative Problem: Visual Metaphors • Can AI do it? • VisiBlends: A Workflow for Prototyping Visual Metaphors • Symbol Finder: Adding AI to the workflow • VisiFit: AI Tools for Iteratively Improving Visual Blends • The Future of AI and creative work
  • 11. With successes in deep learning, what is the future of creative work?
  • 12. Visual Metaphor: Lego are Summer Vacation Fun Artist blend:
  • 14. Visual Metaphor: Burgers are Healthy, like apples Artist:AI:
  • 15. Visual Metaphor: Burgers are Healthy, like apples Artist:AI: Blending objects requires more than just blending “style.” You need more context about the objects - how to take them apart, blend the pieces, think about how they’ll be perceived.
  • 16. Outline: AI Tools for Creative Work • Introduce A Creative Problem: Visual Metaphors • Can AI do it? • VisiBlends: A Workflow for Prototyping Visual Metaphors • Symbol Finder: Adding AI to the workflow • VisiFit: AI Tools for Iteratively Improving Visual Blends • The Future of AI and creative work
  • 17. But there is no simple pattern for visual blends We need to break up the problem into tasks.
  • 18. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching Abstract problem structure: Single Shape Mapping
  • 19. Why blend based on shape? Shape is the primary feature the human visual system uses to recognize objects. Colors, silhouette and details and secondary.
  • 20. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matchingby using a design pattern to make it a search problem We can solve a creative design problem Earth Melt Visual Blending Structure: Whole-to-Part Shape Matching
  • 22. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable Prototype blend Evaluate prototype Two objects are integrated into one object Both objects are identifiable
  • 23. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable Prototype blend Evaluate prototype Two objects are integrated into one object Both objects are identifiable
  • 24. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable
  • 25. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable e e Output: A visual blend grated into one object ntifiable
  • 26. A) Brainstorm for summer B) Find and annotate images for summer C) Matching algorithm Starbucks + summer D) Automatic blends + human evaluation Starbucks + summer matches = [] for a in summer_symbols: for b in starbucks_symbols: a_ratio = a.height / a.width b_ratio = b.height / b.width ratios = sort(a_ratio, b_ratio) if (a.shape == b.shape) and (a.coverage != coverage) and (ratios[0] >= 0.5*ratios[1]): matches.push([a, b]) return matches VisiBlends: A web interface to collaboratively make blends
  • 27. Blends made by groups at Columbia University
  • 30. Wash your hands. It’s the smart move.
  • 31. Forget the knives and forks: the humble hamburger is in the midst of a renaissance. From Melbourne to Milan, restaurateurs have helped elevate this classic piece of American folklore into an iconic dish. Every day, more and more enthusiasts join the search for the perfect
  • 32.
  • 34. Does VisiBlends make people better? • 13 undergraduates (novice designers) • Each made 6 blends (3 with VisiBlends, 3 without) • 5 minutes for each blend • Metric: How many successful blends can 5 minutes?
  • 35. Novices are 10x better with VisiBlends Control-first Avg. Number of blends made VisiBlends-first Without VisiBlends With VisiBlends Without VisiBlendsWith VisiBlends Avg. Number of blends made 0.56 5.56 (t(18)=4.88, p<0.001) 0.675.67 (t(21)=5.84, p<0.001)
  • 36. Bicycle + Smart VisiBlendsUsers Unassisted VisiBlends helps them have many ideas, and pick the best. Novices get stuck “improving” their first idea.
  • 37. New York City + Night Users Unassisted Novices pick “perfect” symbols, but sacrifice shape-fit. VisiBlends helps novices meet all the criteria for a visual blend (especially shape-fit). VisiBlends
  • 38. Do we want “perfect” symbols for Visual Blends? By meeting more constraints, Visual Metaphors are more novel and interesting
  • 39. Novices are 10x better with VisiBlends 10x improvement in novices ability VisiBlendsUnassisted Why?
  • 41. Football Dangerous Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Shape covers All of object Shape covers Part of object
  • 42. Football Dangerous Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Shape covers All of object Shape covers Part of object
  • 43. Collect pieces Combine them Re-combine them Collect pieces Lots of individual pieces under your finger; it’s easy to recombine them.
  • 44. Fluid Intelligence Crystallized Intelligence Having the resources and tools you need (memory, routing procedures, etc) Having the tools at hand so you can fluidly combine them
  • 45. Crystallized Intelligence Crystallized Intelligence Fluid Intelligence Fluid Intelligence VisiBlends augments fluid intelligence Collect pieces Combine them Re-combine them Collect pieces
  • 47. NYC Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Evaluate Blend No shape matches
  • 48. NYC Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
  • 50. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers Part of object Shape covers All of object Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
  • 51. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers All of object Shape covers Part of object Evaluate Blend Exercise equipment
  • 52. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers All of object Shape covers Part of object Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Exercise equipment
  • 53. Iteration requires fluidity – putting together pieces in new ways. No matches Improve object fit Objects are not identifiable + = + = Within same search space, meet other constraints. Find versions of an object with different color, style, aspect ratio Focus on meeting a specific constraint: Find symbols with a different shape Search in a new subspace Find symbols with a different shape
  • 54. Crystallized Intelligence Crystallized Intelligence Fluid Intelligence Fluid Intelligence VisiBlends augments fluid intelligence Collect pieces Combine them Re-combine them Collect pieces
  • 55. Outline: AI Tools for Creative Work • Introduce A Creative Problem: Visual Metaphors • Can AI do it? • VisiBlends: A Workflow for Prototyping Visual Metaphors • Symbol Finder: AI tools for finding symbols • VisiFit: AI Tools for Iteratively Improving Visual Blends • The Future of AI and creative work
  • 56. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable pe e pe Output: A visual blend egrated into one object entifiable
  • 57. Rules for finding images are complex
  • 58. Simply searching for abstract words doesn’t return simple, iconic symbols that we can blend.
  • 59. SymbolFinder uses AI + KBs to find concrete images for an abstract term Savvas Petridis PhD Student word2vec, synonyms, ConceptNet, concreteness classifiers
  • 60. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable pe e pe Output: A visual blend egrated into one object entifiable
  • 61. Outline: AI Tools for Creative Work • Introduce A Creative Problem: Visual Metaphors • Can AI do it? • VisiBlends: A Workflow for Prototyping Visual Metaphors • Symbol Finder: AI tools for finding symbols • VisiFit: AI Tools for Iteratively Improving Visual Blends • The Future of AI and creative work
  • 62. How often do VisiBlend prototypes need editing? Output: A visual blend object 4 of 27 were ok to print on social media (according to 2 evaluators who are graphic designers)
  • 63. How do we improve the rest of them?
  • 64. Fully automatic approaches fail Artist:AI: Design Principle #1 : Instead of pursuing fully automatic approaches, break up the objects into visual components that can each be blended.
  • 65. Design Principle #2: When improving a blend, Recombine secondary visual aspects: color, silhouette, details. Co-Design with 2 Graphic Designers
  • 66. Formative study with novice designer Design Principle #3: Translating intent into action is a barrier for novices. Use AI tools to automate actions or provide more direct mapping between intent and action “I want to make the metro card the same shape as the purse.”
  • 67. VisiFit: AI Tools to Iteratively Improve Visual Blends Each visual aspect has it’s own AI tool for novices to apply
  • 68. AI tools to improve each visual dimension Color blending Crop to extracted silhouette Deep Object Segmentation Interactive Object Segmentation Interactive Object Segmentation
  • 69. Evaluation Improved: 97.5% of cases 11 Novice designers (8 female) 1 hr study 11 blends to extract and improve Publishable quality: 65.3% of the time. Experts Publishable: 87.0% of the time. 121 total blends produced Outputs rated by 2 designers: Improved, publishable.
  • 70. Non-Publishable blends 1. Grabcut failed to extract details 2. No tool to fill parts of the bottom object. (Add a fill tool)
  • 71. Response of Professional Visual Blend Designers 1. Amazed by the AI tools They did try things no novice did (even if it didn’t work). 2. Much easier than “pixel pushing”. Enabled them to explore more designs quickly and easily. But did suggest more PhotoShop features to all, like Context Aware Fill Wanted it as a plugin to sketch. Everyone else Designer
  • 72. Outline: AI Tools for Creative Work • Introduce A Creative Problem: Visual Metaphors • Can AI do it? • VisiBlends: A Workflow for Prototyping Visual Metaphors • Symbol Finder: AI tools for finding symbols • VisiFit: AI Tools for Iteratively Improving Visual Blends • The Future of AI and creative work
  • 73. Crystallized Intelligence Crystallized Intelligence Fluid Intelligence Fluid Intelligence AI Tools can augment fluid intelligence Collect pieces Combine them Combine them Collect pieces by making it easy to collect and combine pieces
  • 74. If we identify the key aspects of the problem VisiBlends collects and combines the shape of blended objects VisiFit extracts and combines the color, silhouette, details of the objects AI tools can help collect and combine elements for each aspect
  • 75. AI Tools for writers: Key aspects of writing: Substance, structure, and style. Metaphoria (CHI 2019): Within the metaphor structure, helps generate substance. Style Thesaurus (C&C 2019): Same substance, new style (word level) Language Style Transfer (INLG 2019): Same substance, new style (sentence level) Katy Gero PhD Candidate
  • 76. Characteristic motion Reference video Static image Future Work: Animated Blends Use AI to extract key aspects of motion (speed), and reapply them to the blend (use a different path).
  • 77. Ecenaz Jen Ozmen Columbia CC ‘21 Claire Chen Columbia CC ‘21 Maria Kogan Columbia SEAS ‘21 Sam Ross Barnard ‘21 Annie Sui Columbia CC ‘22 Maneesh Agrawala Stanford Katy Gero PhD Candidate Savvas Petridis PhD Candidate Collaborators
  • 79. Future of Creative Work: AI should augment fluid intelligence AI shouldn’t try to understand complexities of culture and working context. AI can provide people the tools and resources to more easily turn users’ intent into action AI Artist Novice with AI Tools
  • 80. VisiBlends: AI Tools for Prototyping Visual Blends VisiBlendsUnassisted 10x improvement in novices’ ability
  • 81. VisiFit: AI Tools to Iteratively Improve Visual Blends 97.5% of cases improved, 65% to publishable quality
  • 82. Crystallized Intelligence Crystallized Intelligence Fluid Intelligence Fluid Intelligence VisiBlends augments fluid intelligence Collect pieces Combine them Combine them Collect pieces by making it easy to collect and combine pieces
  • 83. Characteristic motion Reference video Static image Future Work: Animated Blends Use AI to extract key aspects of motion, and reapply them to the blend.
  • 84. Iteratively Improving: one key element at a time Prototyping: collect and recombined parts Brainstorming: easily collect parts AI Tools for Creative Work Repurposing: Elements of a Reference Object

Editor's Notes

  1. Visual blends are a way to convey a message visually. If you have a message you want to convey, like “Washing your hands is smart” We can make a image that blends together two objects related to that message.
  2. This technical is used in new, advertising, and public service announcements to Visual depict symbols of Brazil + taking off Tabasco + hot Earth + melt
  3. There are a wide range of ads out there - So I wondered, what the considered the most creative ads. “Creative ads” Many of the results are visual metaphors. Too hard for a computer.
  4. It would be really nice if we could break up the problem, or find a pattern No surface level pattern: Different colors, different objects, different meanings But there is a more abstract pattern. So how could
  5. How do we get two objects to appear blended, but still individually identifiable? We find objects that share a shape. Like a circle. When we blend on the circle, we get visual cues of both objects.  
  6. Let's go through the steps of the pipeline to create a blend that associates: Starbucks and Summer.
  7. And finally, if they are unhappy with the prototype, they can iterate.
  8. And finally, if they are unhappy with the prototype, they can iterate.
  9. And finally, if they are unhappy with the prototype, they can iterate.
  10. This is a pipeline, But it’s also fairly open. People can move between step as they see fit. Anytime you put in a new image, the matching algorithm will run, and it will look through all pairs for new matches.
  11. This blend was made for the news headline: “Football linked to lasting brain damage”
  12. This blend was made for the advertisement: “Joe’s Coffee: Open Late”
  13. This blend was made for the advertisement: “Join the Philosophy Dept’s Holiday Celebration”
  14. Football + Dangerous: We know which constraint is tight and look for that constrant. Oragne + healthy: We need to search in a new subspace.
  15. One thing I’ve glossed over is that the rules are complex.. Here are the rules for finding images..
  16. Visual blends are a way to convey a message visually. If you have a message you want to convey, like “Washing your hands is smart” We can make a image that blends together two objects related to that message.
  17. To make the motion characteristic of the image, we used a reference video of a floating astronaut. Q: What properties did we add from the video? What motion did we NOT add?
  18. Visual blends are a way to convey a message visually. If you have a message you want to convey, like “Washing your hands is smart” We can make a image that blends together two objects related to that message.
  19. To make the motion characteristic of the image, we used a reference video of a floating astronaut. Q: What properties did we add from the video? What motion did we NOT add?