Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

VisiBlends: A Flexible Workflow for Visual Blends

16 views

Published on

Visual blends are an advanced graphic design technique to draw attention to a message. They combine two objects in a way that is novel and useful in conveying a message symbolically. This paper presents VisiBlends, a flexible workflow for creating visual blends that follows the iterative design process. We introduce a design pattern for blending symbols based on principles of human visual object recognition. Our workflow decomposes the process into both computational techniques and human microtasks. It allows users to collaboratively generate visual blends with steps involving brainstorming, synthesis, and iteration. An evaluation of the workflow shows that decentralized groups can generate blends in independent microtasks, co-located groups can collaboratively make visual blends for their own messages, and VisiBlends improves novices’ ability to make visual blends by a factor of 10.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

VisiBlends: A Flexible Workflow for Visual Blends

  1. 1. VisiBlends: A flexible workflow for visual blends Lydia B. Chilton, Savvas Petridis, Maneesh Agrawala
  2. 2. AdvertisementsNews Public Service Announcements Earth + MeltBrazil + Takes Off Tabasco + Hot Visual Blends: image that convey a message symbolically
  3. 3. How can we help people create visual blends?
  4. 4. Related Work Can we break up a creative task? Rigid workflows Flexible workflows Haoqi Zhang, et al. Human Computation Tasks with Global Constraints. CHI 2012 Greg Little et al. TurKit: Human Computation Algorithms on Mechanical Turk. UIST 2010 Daniela Retelny, et al. Expert Crowdsourcing with Flash Teams. UIST 2014. Lydia B. Chilton, et al. Cascade: Crowdsourcing Taxonomy Creation. CHI 2013 Rajan Vaish et al. Crowd Research: Open and Scalable University Laboratories. UIST 2017 Michael S. Bernstein, et al. Soylent: A Word Processor with a Crowd Inside. UIST 2010
  5. 5. Related Work Design Patterns to structure creative tasks Lixiu Yu, et al. Distributed Analogical Idea Generation: Inventing with Crowds. CHI 2014 Lixiu Yu, et al. Searching for Analogical Ideas with Crowds. CHI 2014 Joy Kim, et al. Motif: Supporting Novice Creativity Through Expert Patterns. CHI 2015 Maneesh Agrawala, Wilmot Li, and Floraine Berthouzoz. Design Principles for Visual Communication. CACM April 2011
  6. 6. VisiBlends is a flexible workflow that leverages the iterative design process - but all the steps are concrete. Design process: • Brainstorm • Prototype many ideas • Synthesizing ideas • Select the best ones • Iterate to improve designs Ideate Evaluate Prototype
  7. 7. AdvertisementsNews Public Service Announcements Earth + MeltBrazil + Takes Off Tabasco + Hot What design pattern can we use for visual blends?
  8. 8. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching Design Pattern: Single Shape Mapping
  9. 9. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matchingby using a design pattern to make it a search problem We can solve a creative design problem Earth Melt Visual Blending Structure: Whole-to-Part Shape Matching
  10. 10. Starbucks + Summer
  11. 11. Prototype blend
  12. 12. Prototype blend Evaluate prototype Two objects are integrated into one object Both objects are identifiable
  13. 13. Prototype blend Evaluate prototype Two objects are integrated into one object Both objects are identifiable Output: A visual blend
  14. 14. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable
  15. 15. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable e e Output: A visual blend grated into one object ntifiable
  16. 16. A) Brainstorm for summer B) Find and annotate images for summer C) Matching algorithm Starbucks + summer D) Automatic blends + human evaluation Starbucks + summer matches = [] for a in summer_symbols: for b in starbucks_symbols: a_ratio = a.height / a.width b_ratio = b.height / b.width ratios = sort(a_ratio, b_ratio) if (a.shape == b.shape) and (a.coverage != coverage) and (ratios[0] >= 0.5*ratios[1]): matches.push([a, b]) return matches VisiBlends: A web interface to collaboratively make blends
  17. 17. 3 Studies
  18. 18. Study 1: Can decentralized groups make blends? Five people Five people One person One person One person One person One person Computer: matching and blending algorithms
  19. 19. Rules for finding images are complex
  20. 20. Our first attempt failed. People weren’t finding the right images.
  21. 21. To fix it, we trained people on the pipeline steps backwards. Now they know why they need to pick Simple, iconic objects with a single main shape. Because they’re seen how it affects the pipeline
  22. 22. NYC + FashionMcDonald’s + HealthyBicycle + Fall McDonald’s + Energy 11 of 16 pairs found a blend in iteration 1 5 of 16 pairs found a blend in iteration 2 Study 1: Decentralized groups can make blends if they know how their step will fit into the whole.
  23. 23. Study 2: Can groups make blends for messages? • 5 co-located groups of 3 undergrads and/or grad students • All had their own laptop • Data is synchronized across users in real time • 5 Messages to advertise for: • Joe’s Coffee + Night • Hand-washing + smart • Women + CS • Football + Dangerous • Philosophy Department + Christmas Party
  24. 24. Study: Joe’s Coffee + Night
  25. 25. Study: Football + Dangerous
  26. 26. Study: Philosophy + Christmas
  27. 27. Groups can collaboratively make blends for their own messages But we need humans in the loop to guide the process
  28. 28. Football + Dangerous
  29. 29. Football Dangerous Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Shape covers All of object Shape covers Part of object
  30. 30. Football Dangerous Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Shape covers All of object Shape covers Part of object
  31. 31. NYC + Healthy
  32. 32. NYC Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Evaluate Blend No shape matches
  33. 33. NYC Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
  34. 34. Orange + Healthy
  35. 35. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers Part of object Shape covers All of object Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend
  36. 36. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers All of object Shape covers Part of object Evaluate Blend Exercise equipment
  37. 37. Orange Healthy Brainstorm associations Find Images of objects Annotate shapes Annotate shape coverage Shape covers All of object Shape covers Part of object Are both objects identifiable? Are two objects integrated into one object? Evaluate Blend Exercise equipment
  38. 38. Groups can use VisiBlends for their messages. But they need flexibility to iterate. No matches Improve object fit Objects are not identifiable + = + = Within same search space, meet other constraints. Find versions of an object with different color, style, aspect ratio Focus on meeting a specific constraint: Find symbols with a different shape Search in a new subspace Find symbols with a different shape
  39. 39. Controlled Study • 13 undergraduates (novice designers) • Each made 6 blends (3 with VisiBlends, 3 without) • 5 minutes for each blend • Metric: How many successful blends can 5 minutes?
  40. 40. Novices with and without VisiBlends Control-first Avg. Number of blends made VisiBlends-first Without VisiBlends With VisiBlends Without VisiBlendsWith VisiBlends Avg. Number of blends made 0.56 5.56 (t(18)=4.88, p<0.001) 0.675.67 (t(21)=5.84, p<0.001)
  41. 41. Why is this task so hard?
  42. 42. Bicycle + Smart Users with VisiBlendsUsers Unassisted VisiBlends helps them have many ideas, and pick the best. Novices get stuck “improving” their first idea.
  43. 43. New York City + Night Users with VisiBlendsUsers Unassisted Novices pick “perfect” symbols, but sacrifice shape-fit. VisiBlends helps novices meet all the criteria for a visual blend (especially shape-fit).
  44. 44. Do we want “perfect” symbols for Visual Blends? By meeting more constraints, Visual Metaphors are more novel and interesting
  45. 45. Can we decompose other creative problems?
  46. 46. We can collectively solve creative design problems by treating them as iterative search problems. People can contribute the basic elements to a database The design pattern is used to automatically combine elements. The flexible workflow allows people check the results and redirect the search. 1. Two objects are integrated into one object 2. Both objects are individually identifiable Visual Blending Structure: Whole-to-Part Shape Matching
  47. 47. Material Design in the Information Age Dr. Kristin Persson Lawrence Berkeley National Laboratory
  48. 48. CinematographyNovels Music Design patterns exist in many fields Architecture Rhetoric Software
  49. 49. Summary
  50. 50. VisiBlends: A flexible workflow for visual blends Lydia B. Chilton, Savvas Petridis, Maneesh Agrawala We use a design patterns to turn creative problems into search problems. Groups can collaborate on visual blends 10x improvement in novices ability
  51. 51. Next steps
  52. 52. Animated Metaphors Jessica Cohen CC ‘21
  53. 53. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable e e Output: A visual blend grated into one object ntifiable
  54. 54. Prototype blend Evaluate prototype Iterate Output: A visual blend Two objects are integrated into one object Both objects are identifiable e e Output: A visual blend grated into one object ntifiable
  55. 55. Visual Symbol Dictionary Abstract Nouns Verbs Adjectives to infect Danger to cut to eat Fast Deadly Price Disease Maria Kogan SEAS ’21 Harrison Wang CC ‘21 Jen Ozmen CC’21 Zian Zhao MS ’20 Savvas Petridis, PhD
  56. 56. Constructing Visual Headlines Maria Kogan SEAS ‘21 The Great Barrier Reef is being battered by climate change, and it might only get worse.
  57. 57. Constructing Visual Headlines Maria Kogan SEAS ‘21 The Great Barrier Reef is being battered by climate change, and it might only get worse. (verb)(noun)
  58. 58. Maria Kogan SEAS ‘21
  59. 59. Maria Kogan SEAS ‘21

×