Crowdsourcing Visual Ads
Jan. 27, 2019•0 likes•358 views
Download to read offline
Report
Technology
Images have the power to convey messages in striking and memorable ways. Although constructing visual messages is currently too hard for computers or novice users, by combining the intelligence of people and computers we can create compelling visual messages computationally. In this talk, we present VisiBlends, a flexible workflow for creating visual blends that follows the design process with steps involving brainstorming, synthesis, and iteration. An evaluation of the workflow shows that (1) decentralized groups of people can generate blends in independent microtasks, (2) co-located groups can collaboratively make visual blends for their own messages, and (3) VisiBlends improves novices’ ability to make visual blends. We discuss how to decompose other complex tasks so that people and computers can collaborate in generating novel, useful and creative solutions to problems.
hmslydiaFollow
More Related Content
What's hot
Push conference 2018 key takeawaysJustin Crowell
Conquering Museum Challenges with Design ThinkingDana Mitroff Silvers
March 2014 Experiments for RecruitLUXr
Design Thinking For Educational Technology Stefanie Panke
Interface Design Concepts and Planning: 532 lecture 2Don Stanley
Left Hackathon 4.0John Lyotier
![LUXr 1-day workshop, July 18, 2012 [San Francisco]](https://cdn.slidesharecdn.com/ss_thumbnails/1day-luxr20120718slideshare-120719130159-phpapp01-thumbnail.jpg?width=1600&height=908&fit=bounds)
Similar to Crowdsourcing Visual Ads
VisiBlends: A Flexible Workflow for Visual Blendshmslydia
Module1 design processStacie Hidek
Steven Bell's AISL Summer Institute 2016 Keynote Slide PresentationKArchambault24
Make a Good Slide Show 1 - Methods of SlideCraftSiddhant Kumar
Camden Council (Design Group Project) DropboxSeb Sear
Zoe's UX PortfolioZoe Lewis
Recently uploaded
Understanding Wireguard, TLS and Workload IdentityChristian Posta
UiPath Tips and Techniques for Debugging - Session 3DianaGray10
Empowering City ClerksOnBoard
How is AI changing journalism? Strategic considerations for publishers and ne...Damian Radcliffe
Solving today’s Traffic Problems with Sustainable Ride Hailing SolutionOn Demand Clone
Future of SkillsAlison B. Lowndes
Crowdsourcing Visual Ads
- 1. Crowdsourcing Visual Ads Lydia Chilton Columbia CS Department, DSI Computational Social Science Seminar 25 January 2019
- 2. Ads are interesting because they convey a message visually
- 4. AdvertisementsNews Public Service Announcements Earth + MeltBrazil + Takes Off Tabasco + Hot Visual Metaphors How can we systemically create visual metaphors?
- 5. Visual Metaphors are a creative design problem. There is no simple recipe. Design process: • Brainstorm, • Prototype many ideas, • Select the best ones • Iterate to improve designs Ideate Evaluate Prototype
- 6. Outline: Crowdsourcing Visual Ads • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
- 7. AdvertisementsNews Public Service Announcements Earth + MeltBrazil + Takes Off Tabasco + Hot Visual Metaphors
- 8. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching Design Pattern: Single Shape Mapping
- 11. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching A creative problem is now a search problem: For two concepts, search for symbols that meet this constraint
- 23. Prototype blend
- 24. Prototype blend Evaluate prototype Two objects are integrated into one object Both objects are identifiable
- 25. Prototype blend Evaluate prototype Two objects are integrated into one object Both objects are identifiable Output: A visual blend
- 26. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable
- 27. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable e e Output: A visual blend grated into one object ntifiable
- 28. A) Brainstorm for summer B) Find and annotate images for summer C) Matching algorithm Starbucks + summer D) Automatic blends + human evaluation Starbucks + summer matches = [] for a in summer_symbols: for b in starbucks_symbols: a_ratio = a.height / a.width b_ratio = b.height / b.width ratios = sort(a_ratio, b_ratio) if (a.shape == b.shape) and (a.coverage != coverage) and (ratios[0] >= 0.5*ratios[1]): matches.push([a, b]) return matches VisiBlends: A web interface to collaboratively make blends
- 29. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
- 30. Decompose the pipeline Five people Five people One person One person One person One person One person Computer: matching and blending algorithms
- 31. Rules for finding images are complex
- 32. Our first attempt failed. People weren’t finding the right images.
- 33. To fix it, we trained people on the pipeline steps backwards. Now they know why they need to pick Simple, iconic objects with a single main shape. Because they’re seen how it affects the pipeline
- 34. Independent people can make blends for random concept pairs. NYC + FashionMcDonald’s + HealthyBicycle + Fall McDonald’s + Energy
- 35. When we taught the process backwards, users were successful
- 36. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
- 37. Study 2: Joe’s Coffee + Night “Joe’s Coffee: Open Late” Ad
- 38. Study 2: Hand-washing + smart “Wash your hands. It’s the smart move.” PSA
- 39. Study 2: Women + CS “Panel Discussion: Women in Computer Science” Ad:
- 40. Study 2: Football + Dangerous “Football linked to lasting brain damage.” News
- 41. Study 2: Philosophy + Christmas “Join the Philosophy Dept’s Holiday Celebration” Ad:
- 42. Groups can collaboratively make blends for their own messages What goes wrong? (why do we need iteration?)
- 44. Football Dangerous Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Shape covers All of object Shape covers Part of object
- 45. Football Dangerous Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Shape covers All of object Shape covers Part of object
- 46. NYC + Healthy
- 47. NYC Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Evaluate Blend No shape matches
- 48. NYC Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
- 49. Lego + Healthy
- 50. Lego Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Evaluate Blend No shape matches
- 51. Lego Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
- 52. Orange + Healthy
- 53. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers Part of object Shape covers All of object Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
- 54. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers All of object Shape covers Part of object Evaluate Blend Exercise equipment
- 55. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers All of object Shape covers Part of object Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Exercise equipment
- 56. When do we need to iterate? No matches Improve object fit Objects are not identifiable + = + = Within same search space, meet other constraints. Find versions of an object with different color, style, aspect ratio Focus on meeting a specific constraint: Find symbols with a different shape Search in a new subspace Find symbols with a different shape
- 57. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
- 58. Controlled Study • 13 undergraduates • Each made 6 blends • Control-first condition • 3 blends without VisiBlends • 3 blends with VisiBlends • System-first condition • 3 blends with VisiBlends • 3 blends without VisiBlends • How many successful blends could they make?
- 59. Blend-making success with and without VisiBlends Control-first Avg. Number of blends made System-first Without VisiBlends With VisiBlends Without VisiBlendsWith VisiBlends Avg. Number of blends made
- 60. Blend-making success with and without VisiBlends Control-first Avg. Number of blends made System-first Without VisiBlends With VisiBlends Without VisiBlendsWith VisiBlends Avg. Number of blends made 0.56 5.56 (t(18)=4.88, p<0.001) 0.675.67 (t(21)=5.84, p<0.001)
- 61. Why is this task so hard? There are a lot of constraints. Novices without the system focused on meeting one constraint at the expense of others. Usually they, found symbols, and then forced them together.
- 62. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
- 63. CinematographyNovels Music Design patterns: Abstract ways that parts fit Architecture Rhetoric Software
- 64. Material Science Kristin Persson, Lawrence Berkeley National Laboratory
- 65. Summary
- 66. Advertisements can convey messages visually.
- 67. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching by using an abstract design pattern to turn it into a search problem. We can decompose design problems
- 68. Independent people can make blends for random concept pairs. NYC + FashionMcDonald’s + HealthyBicycle + Fall McDonald’s + Energy But…
- 69. Knowledge of the pipeline is needed to motivate the rules, and fill in gaps. Now they know why they need to pick Simple, iconic objects with a single main shape. Because they’re seen how it affects the pipeline1 2 3
- 70. Groups can collaboratively make blends for their own messages But…
- 71. Iteration is necessary to: Fix small problems like aesthetic refinement Fix bigger problems by refining the search for different shapes Fix failure cases with unforeseen problems by re-directing the search for different symbols
- 72. 10x improvement in novices’ ability to make blends
- 73. Materials ScienceNovels Music Design patterns: Abstract ways that parts fit Architecture Rhetoric Software
- 74. Future Work Adding computational assistance to: • Brainstorming • Finding symbols • Annotation • Evaluation Improving the blend quality
- 75. Crowdsourcing Visual Ads Lydia Chilton chilton@cs.columbia.edu Columbia CS + DSI We can use design patterns to turn design problems into search problems. Iteration is necessary to meet all search constraints Groups can collaborate on visual blends 10x improvement to novices ability 1. Two objects are integrated into one object Visual Blending Structure: Whole-to-Part Shape Matching
Editor's Notes
- Visual blends are a way to convey a message visually. If you have a message you want to convey, like “Washing your hands is smart” We can make a image that blends together two objects related to that message.
- Visual Ads are everywhere. Commercial and crass They make a lot of money Nobody knows how they work (but they do work) To me, ads are interesting because they convey a message visually. Convey a message
- How do we make things like this?
- I once wondered, what the considered the most creative ads. “Creative ads” Many of the results are visual metaphors.
- This technical is used in new, advertising, and public service announcements to Visual depict symbols of Brazil + taking off Tabasco + hot Earth + melt
- No surface level pattern: Different colors, different objects, different meanings But there is a more abstract pattern.
- How do we get two objects to appear blended, but still individually identifiable? We find objects that share a shape. Like a circle. When we blend on the circle, we get visual cues of both objects.
- Here is an example of a tool we built to help people prototype with blends. If we look only at the shapes of the objects, The tabasco bottle is a cylinder. The fire extinguisher is also mainly a cylinder. If we combine the objects on their shape, they create a visual blend: The tabasco label clearly identifies the tabasco bottle, and the hose outside the main shape clearly identifies the fire extinguisher
- The light switch is a rectangle. And the iPhone is also a rectangle However, if we cover the entire iPhone, it will not be visible in the blend. Instead, we cover only part of the iPhone with a rectangle. Now both objects are blended and both objects are recognizable.
- How do we get two objects to appear blended, but still individually identifiable? We find objects that share a shape. Like a circle. When we blend on the circle, we get visual cues of both objects.
- Let's go through the steps of the pipeline to create a blend that associates: Starbucks and Summer.
- …. The system automatically matches images from each concept with the same shape and blends them. Users then evaluate the blend by checking whether it has integrated two objects into one, and if both objects are identifiable. Users can then modify color to improve the blend’s aesthetics.
- The system automatically matches images from each concept with the same shape and blends them.
- Users then evaluate the blend by checking whether it has integrated two objects into one, and if both objects are identifiable.
- Users can then modify color to improve the blend’s aesthetics.
- And finally, if they are unhappy with the prototype, they can iterate.
- This is a pipeline, But it’s also fairly open. People can move between step as they see fit. Anytime you put in a new image, the matching algorithm will run, and it will look through all pairs for new matches.
- This blend was made for the advertisement: “Joe’s Coffee: Open Late”
- Visual blends are useful for conveying messages visually. To see if our pipeline could make blends for messages, we had trained groups of people use the pipeline to create visual blends for messages. This blend was made for the public service announcement: “Wash your hands. It’s the smart thing to do.”
- This blend was made for the advertisement: “Panel Discussion: Women in Computer Science”
- This blend was made for the news headline: “Football linked to lasting brain damage”
- This blend was made for the advertisement: “Join the Philosophy Dept’s Holiday Celebration”
- Football + Dangerous: We know which constraint is tight and look for that constrant. Oragne + healthy: We need to search in a new subspace.
- People naturally iterating. They like linear steps that guarantee a result. Evaluating intermediate steps is challenging, but it can be taught.
- Four chord song: https://www.youtube.com/watch?v=5pidokakU4I
- I once wondered, what the considered the most creative ads. “Creative ads” Many of the results are visual metaphors.
- Four chord song: https://www.youtube.com/watch?v=5pidokakU4I
- Visual blends are a way to convey a message visually. If you have a message you want to convey, like “Washing your hands is smart” We can make a image that blends together two objects related to that message.