Successfully reported this slideshow.
Your SlideShare is downloading. ×

Crowdsourcing Visual Ads

Crowdsourcing Visual Ads

Download to read offline

Images have the power to convey messages in striking and memorable ways. Although constructing visual messages is currently too hard for computers or novice users, by combining the intelligence of people and computers we can create compelling visual messages computationally. In this talk, we present VisiBlends, a flexible workflow for creating visual blends that follows the design process with steps involving brainstorming, synthesis, and iteration. An evaluation of the workflow shows that (1) decentralized groups of people can generate blends in independent microtasks, (2) co-located groups can collaboratively make visual blends for their own messages, and (3) VisiBlends improves novices’ ability to make visual blends.

We discuss how to decompose other complex tasks so that people and computers can collaborate in generating novel, useful and creative solutions to problems.

Images have the power to convey messages in striking and memorable ways. Although constructing visual messages is currently too hard for computers or novice users, by combining the intelligence of people and computers we can create compelling visual messages computationally. In this talk, we present VisiBlends, a flexible workflow for creating visual blends that follows the design process with steps involving brainstorming, synthesis, and iteration. An evaluation of the workflow shows that (1) decentralized groups of people can generate blends in independent microtasks, (2) co-located groups can collaboratively make visual blends for their own messages, and (3) VisiBlends improves novices’ ability to make visual blends.

We discuss how to decompose other complex tasks so that people and computers can collaborate in generating novel, useful and creative solutions to problems.

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Crowdsourcing Visual Ads

  1. 1. Crowdsourcing Visual Ads Lydia Chilton Columbia CS Department, DSI Computational Social Science Seminar 25 January 2019
  2. 2. Ads are interesting because they convey a message visually
  3. 3. AdvertisementsNews Public Service Announcements Earth + MeltBrazil + Takes Off Tabasco + Hot Visual Metaphors How can we systemically create visual metaphors?
  4. 4. Visual Metaphors are a creative design problem. There is no simple recipe. Design process: • Brainstorm, • Prototype many ideas, • Select the best ones • Iterate to improve designs Ideate Evaluate Prototype
  5. 5. Outline: Crowdsourcing Visual Ads • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
  6. 6. AdvertisementsNews Public Service Announcements Earth + MeltBrazil + Takes Off Tabasco + Hot Visual Metaphors
  7. 7. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching Design Pattern: Single Shape Mapping
  8. 8. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching A creative problem is now a search problem: For two concepts, search for symbols that meet this constraint
  9. 9. Starbucks + Summer
  10. 10. Prototype blend
  11. 11. Prototype blend Evaluate prototype Two objects are integrated into one object Both objects are identifiable
  12. 12. Prototype blend Evaluate prototype Two objects are integrated into one object Both objects are identifiable Output: A visual blend
  13. 13. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable
  14. 14. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable e e Output: A visual blend grated into one object ntifiable
  15. 15. A) Brainstorm for summer B) Find and annotate images for summer C) Matching algorithm Starbucks + summer D) Automatic blends + human evaluation Starbucks + summer matches = [] for a in summer_symbols: for b in starbucks_symbols: a_ratio = a.height / a.width b_ratio = b.height / b.width ratios = sort(a_ratio, b_ratio) if (a.shape == b.shape) and (a.coverage != coverage) and (ratios[0] >= 0.5*ratios[1]): matches.push([a, b]) return matches VisiBlends: A web interface to collaboratively make blends
  16. 16. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
  17. 17. Decompose the pipeline Five people Five people One person One person One person One person One person Computer: matching and blending algorithms
  18. 18. Rules for finding images are complex
  19. 19. Our first attempt failed. People weren’t finding the right images.
  20. 20. To fix it, we trained people on the pipeline steps backwards. Now they know why they need to pick Simple, iconic objects with a single main shape. Because they’re seen how it affects the pipeline
  21. 21. Independent people can make blends for random concept pairs. NYC + FashionMcDonald’s + HealthyBicycle + Fall McDonald’s + Energy
  22. 22. When we taught the process backwards, users were successful
  23. 23. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
  24. 24. Study 2: Joe’s Coffee + Night “Joe’s Coffee: Open Late” Ad
  25. 25. Study 2: Hand-washing + smart “Wash your hands. It’s the smart move.” PSA
  26. 26. Study 2: Women + CS “Panel Discussion: Women in Computer Science” Ad:
  27. 27. Study 2: Football + Dangerous “Football linked to lasting brain damage.” News
  28. 28. Study 2: Philosophy + Christmas “Join the Philosophy Dept’s Holiday Celebration” Ad:
  29. 29. Groups can collaboratively make blends for their own messages What goes wrong? (why do we need iteration?)
  30. 30. Football + Dangerous
  31. 31. Football Dangerous Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Shape covers All of object Shape covers Part of object
  32. 32. Football Dangerous Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Shape covers All of object Shape covers Part of object
  33. 33. NYC + Healthy
  34. 34. NYC Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Evaluate Blend No shape matches
  35. 35. NYC Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
  36. 36. Lego + Healthy
  37. 37. Lego Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Evaluate Blend No shape matches
  38. 38. Lego Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
  39. 39. Orange + Healthy
  40. 40. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers Part of object Shape covers All of object Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
  41. 41. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers All of object Shape covers Part of object Evaluate Blend Exercise equipment
  42. 42. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers All of object Shape covers Part of object Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Exercise equipment
  43. 43. When do we need to iterate? No matches Improve object fit Objects are not identifiable + = + = Within same search space, meet other constraints. Find versions of an object with different color, style, aspect ratio Focus on meeting a specific constraint: Find symbols with a different shape Search in a new subspace Find symbols with a different shape
  44. 44. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
  45. 45. Controlled Study • 13 undergraduates • Each made 6 blends • Control-first condition • 3 blends without VisiBlends • 3 blends with VisiBlends • System-first condition • 3 blends with VisiBlends • 3 blends without VisiBlends • How many successful blends could they make?
  46. 46. Blend-making success with and without VisiBlends Control-first Avg. Number of blends made System-first Without VisiBlends With VisiBlends Without VisiBlendsWith VisiBlends Avg. Number of blends made
  47. 47. Blend-making success with and without VisiBlends Control-first Avg. Number of blends made System-first Without VisiBlends With VisiBlends Without VisiBlendsWith VisiBlends Avg. Number of blends made 0.56 5.56 (t(18)=4.88, p<0.001) 0.675.67 (t(21)=5.84, p<0.001)
  48. 48. Why is this task so hard? There are a lot of constraints. Novices without the system focused on meeting one constraint at the expense of others. Usually they, found symbols, and then forced them together.
  49. 49. Outline • Illustration of the VisiBlends Pipeline • 3 Evaluations: • Decomposing the design process: Can users work on steps independently? • Can groups collaborate to make visual blends? • Can we improve novices’ ability to make Visual Blends? • Can we decompose other creative tasks?
  50. 50. CinematographyNovels Music Design patterns: Abstract ways that parts fit Architecture Rhetoric Software
  51. 51. Material Science Kristin Persson, Lawrence Berkeley National Laboratory
  52. 52. Summary
  53. 53. Advertisements can convey messages visually.
  54. 54. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching by using an abstract design pattern to turn it into a search problem. We can decompose design problems
  55. 55. Independent people can make blends for random concept pairs. NYC + FashionMcDonald’s + HealthyBicycle + Fall McDonald’s + Energy But…
  56. 56. Knowledge of the pipeline is needed to motivate the rules, and fill in gaps. Now they know why they need to pick Simple, iconic objects with a single main shape. Because they’re seen how it affects the pipeline1 2 3
  57. 57. Groups can collaboratively make blends for their own messages But…
  58. 58. Iteration is necessary to: Fix small problems like aesthetic refinement Fix bigger problems by refining the search for different shapes Fix failure cases with unforeseen problems by re-directing the search for different symbols
  59. 59. 10x improvement in novices’ ability to make blends
  60. 60. Materials ScienceNovels Music Design patterns: Abstract ways that parts fit Architecture Rhetoric Software
  61. 61. Future Work Adding computational assistance to: • Brainstorming • Finding symbols • Annotation • Evaluation Improving the blend quality
  62. 62. Crowdsourcing Visual Ads Lydia Chilton chilton@cs.columbia.edu Columbia CS + DSI We can use design patterns to turn design problems into search problems. Iteration is necessary to meet all search constraints Groups can collaborate on visual blends 10x improvement to novices ability 1. Two objects are integrated into one object Visual Blending Structure: Whole-to-Part Shape Matching

Editor's Notes

  • Visual blends are a way to convey a message visually.

    If you have a message you want to convey, like “Washing your hands is smart”
    We can make a image that blends together two objects related to that message.


  • Visual Ads are everywhere.

    Commercial and crass
    They make a lot of money
    Nobody knows how they work (but they do work)

    To me, ads are interesting because they convey a message visually.
    Convey a message


  • How do we make things like this?
  • I once wondered, what the considered the most creative ads.


    “Creative ads”

    Many of the results are visual metaphors.
  • This technical is used in new, advertising, and public service announcements to
    Visual depict symbols of
    Brazil + taking off
    Tabasco + hot
    Earth + melt
  • No surface level pattern:
    Different colors, different objects, different meanings

    But there is a more abstract pattern.
  • How do we get two objects to appear blended, but still individually identifiable?

    We find objects that share a shape. Like a circle. When we blend on the circle, we get visual cues of both objects.

     
  • Here is an example of a tool we built to help people prototype with blends.

    If we look only at the shapes of the objects,
    The tabasco bottle is a cylinder.
    The fire extinguisher is also mainly a cylinder.
    If we combine the objects on their shape, they create a visual blend:
    The tabasco label clearly identifies the tabasco bottle, and the hose outside the main shape clearly identifies the fire extinguisher
  • The light switch is a rectangle.
    And the iPhone is also a rectangle
    However, if we cover the entire iPhone, it will not be visible in the blend.
    Instead, we cover only part of the iPhone with a rectangle.
    Now both objects are blended and both objects are recognizable.
  • How do we get two objects to appear blended, but still individually identifiable?

    We find objects that share a shape. Like a circle. When we blend on the circle, we get visual cues of both objects.

     


  • Let's go through the steps of the pipeline to create a blend that associates:

    Starbucks and Summer.
  • ….
    The system automatically matches images from each concept with the
    same shape and blends them.
     
    Users then evaluate the blend by checking whether it has integrated two objects into one, and if both objects are identifiable.
     
    Users can then modify color to improve the blend’s aesthetics.
     
  • The system automatically matches images from each concept with the
    same shape and blends them.
  • Users then evaluate the blend by checking whether it has integrated two objects into one, and if both objects are identifiable.
  • Users can then modify color to improve the blend’s aesthetics.
  • And finally, if they are unhappy with the prototype, they can iterate.
  • This is a pipeline,
    But it’s also fairly open.

    People can move between step as they see fit.
    Anytime you put in a new image, the matching algorithm will run, and it will look through all pairs for new matches.


  • This blend was made for the advertisement: “Joe’s Coffee: Open Late”
  • Visual blends are useful for conveying messages visually. To see if our pipeline could make blends for messages, we had trained groups of people use the pipeline to create visual blends for messages.
     
    This blend was made for the public service announcement: “Wash your hands. It’s the smart thing to do.”
  • This blend was made for the advertisement: “Panel Discussion: Women in Computer Science”
  • This blend was made for the news headline: “Football linked to lasting brain damage”
  • This blend was made for the advertisement:

    “Join the Philosophy Dept’s Holiday Celebration”

  • Football + Dangerous: We know which constraint is tight and look for that constrant.
    Oragne + healthy: We need to search in a new subspace.
  • People naturally iterating.
    They like linear steps that guarantee a result.

    Evaluating intermediate steps is challenging, but it can be taught.
  • Four chord song:
    https://www.youtube.com/watch?v=5pidokakU4I
  • I once wondered, what the considered the most creative ads.


    “Creative ads”

    Many of the results are visual metaphors.
  • Four chord song:
    https://www.youtube.com/watch?v=5pidokakU4I
  • Visual blends are a way to convey a message visually.

    If you have a message you want to convey, like “Washing your hands is smart”
    We can make a image that blends together two objects related to that message.


×