Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BrightonSEO 2018 (September 28): How to Optimize for Visual Search


Published on

In this presentation, I cover the key areas of visual search:

- What is visual search?
- How does visual search work?
- Why should you care?
- How you can get started.

Published in: Marketing
  • Do This Simple 2-Minute Ritual To Loss 1 Pound Of Belly Fat Every 72 Hours 
    Are you sure you want to  Yes  No
    Your message goes here

BrightonSEO 2018 (September 28): How to Optimize for Visual Search

  1. 1. Clark Boyd @ClarkBoyd CANDID DIGITAL - Independent consultant, working with Google, Columbia Business School, MIT, and many others. - 9 years in search. - Research lead for ClickZ and Search Engine Watch.
  2. 2. How to Optimize for Visual Search.
  3. 3. Today 1. What is visual search? 2. How does visual search work? 3. Why should marketers care about visual search? 4. How can I optimize for visual search? 3 @ClarkBoyd
  4. 4. What is visual search? 1 @ClarkBoyd For when you don’t have the (key)words.
  5. 5. 5 We partake in visual search every day
  6. 6. 6 Visual search turns a smartphone camera into a visual discovery tool. Consumers can scan an object or landscape and the search engine will identify key features in the image, then return relevant results. This goes beyond object detection, as we will see.
  7. 7. Who are the main visual search providers? 7
  8. 8. How does visual search work? Search what you see. 2 @ClarkBoyd
  9. 9. 9 How it works: Pinterest Lens @ClarkBoyd 1. Query understanding: Object shape, size, color. Annotations help connect this to Pinterest’s inventory. 2. Blender: Results pulled from Visual Search (similar aesthetic), Object Search (similar objects) and Image Search (similar text search results). Blending ratios are weighted dynamically based on query understanding - including related boards and past user behavior.
  10. 10. “ "In the English language there are 180,000 words, and we only use 3,000 to 5,000 of them. If you’re trying to do voice recognition, there’s a small set of things you need to be able to recognize. Think about how many objects there are in the world, distinct objects, billions, and they all come in different shapes and sizes." - Clay Bavor, Google 10
  11. 11. Why does visual search matter for marketers? 3 @ClarkBoyd
  12. 12. “"Shopping has always been visual. We’ve just been taught to do the opposite online." - Amy Dziewiontkoski, Pinterest 12
  13. 13. 13 Consumers - particularly younger generations - are more likely to engage with brands through visual media. Google and Amazon have both realized this and visual search is a central tenet of their strategy to encourage discovery beyond the search bar.
  14. 14. 14
  15. 15. Visual search both extends and shortens the search journey 15 Intent state Input Output Open to ideas Looking for a style Looking for a specific type of product “White sneakers” Ready to buy “Adidas originals white” - Visual search creates a new space for image- driven, inspiration-based connections. - It can also collapse the purchase journey, allowing someone to go from image to purchase in just a few moments. - Visual Search will be integrated with Google Express, shopping through Snapchat, and already works with Lens the Look on Pinterest
  16. 16. 600,000,000+ ● Visual searches via Pinterest Lens every month ● 97% of all Pinterest searches are non-branded ● 93% of consumers consider images to be the key deciding factor in a purchasing decision (KissMetrics) ● 72% of internet users search for visual content before making a purchase (eMarketer) 16
  17. 17. How can I optimize for visual search? Some practical tips to gain a competitive advantage 4 @ClarkBoyd
  18. 18. 1. Upload your inventory: sitemaps and social 18 @ClarkBoyd
  19. 19. 2. Research trends and concepts 19 @ClarkBoyd
  20. 20. 2. Map keywords to images 20
  21. 21. 2. Order images based on stylistic/aesthetic relations 21
  22. 22. “ “The images that appear in both the style ideas and similar items grids are also algorithmically ranked, and will prioritize those that focus on a particular product type or that appear as a complete look and are from authoritative sites.” - Google 22 @ClarkBoyd This matters because it shows how visual search differs from traditional image search optimization - but also how it can be similar, too.
  23. 23. 3. Image best practices for visual search 23 ➔ Alt attributes and captions: include key concepts. ➔ Insert images on high authority, relevant pages. Google will prioritize images that are central to their host page. ➔ Only use stock photography if it has been edited to make it unique. ➔ Maintain a consistent aesthetic - this helps search engines understand the relation between images. @ClarkBoyd
  24. 24. 24 3. Remove clutter from images Automatic object detection works best when the focal points of the image are in the foreground. Tell search engines what your image is about by giving prominence to the items you want to rank for. This allows the technology to produce a feature map, which can be used to find relevant images in the database. @ClarkBoyd
  25. 25. Place your screenshot here 25 4. Structured data Help search engines understand your content by using structured data for all relevant elements of images. As a guideline, always mark up: ● Price ● Availability ● Image ● Product name @ClarkBoyd
  26. 26. 5. Use visual search to unite the physical and digital worlds 26 Maps integration through Augmented Reality PinCodes in store lead consumers to online listings. Ensure they have a cohesive experience across channels @ClarkBoyd
  27. 27. Key Tips 27 1. Upload your product inventory to your website and social media profiles a. Image XML sitemap b. Check indexation status of images 2. Research trends (both keywords and styles) a. Map keywords to images b. Logical taxonomy 3. Optimize your images a. Make images central to your landing pages b. Maintain a consistent aesthetic c. Don’t use stock images; or at least, edit them to make them unique 4. Make context clear a. Structured data is essential (!) 5. Link your physical and digital presences a. PinCodes b. Maps optimization @ClarkBoyd
  28. 28. 28 Thanks! Any questions? You can find me at @ClarkBoyd & @ClarkBoyd
  29. 29. 29 Google’s computer vision technology couldn’t tell apart a Maltese and a Maltipoo(!) Maltese Maltipoo
  30. 30. Zia Chisti said in an interview with The Economist this week, “Much of what we think of as AI is just the same old algorithms, but faster. True AI will learn on its own from its environment, increasing in accuracy over time.” Google is continuously analyzing images and updating its feature recognition. In this instance, it has separated out the two dogs into different albums as it identified their individual features. 30 Separate photo albums!
  31. 31. Appendix 31
  32. 32. “ “A field linguist has gone to visit a culture whose language is entirely different from our own. The linguist is trying to learn some words from a helpful native speaker, when a rabbit scurries by. The native speaker declares “gavagai”, and the linguist is left to infer the meaning of this new word. The linguist is faced with an abundance of possible inferences, including that “gavagai” refers to rabbits, animals, white things, that specific rabbit, or “undetached parts of rabbits”. There is an infinity of possible inferences to be made. How are people able to choose the correct one?” - DeepMind 32 @ClarkBoyd
  33. 33. 33 Consumers are looking for images on Google, but also on Amazon, Pinterest, and other retailers. Increasingly, they are using images to find styles that they cannot describe with words.
  34. 34. 2. Create semantic relations between products 34 Sneakers Casual shoes for summer New adidas sneakers Summer fashion ideas White @ClarkBoyd
  35. 35. 35 Brands are responsible for providing the algorithms with better content to fuel results. Stock images are at odds with what the visual searcher wants to see and Lens technologies cannot read beyond the surface of these images.
  36. 36. 36 How different search engines work - Pinterest: Meta data (Pins, board names, image tags) help it understand context. The ‘blender’ that decides the weighting of shape/color/texture is dynamic and effective. The high quantity of visual searches on the platform is helping Pinterest improve accuracy of object recognition. Pinterest is closest to understanding the ‘essence’ of an object, beyond its form or color. This allows it to deliver satisfactory results for fashion and decor image queries. - Google: For now, Google turns the image into a text query based on the object it recognizes. Search with an image of a mug using Google Lens and it will return results for mugs, but it will not detect the style of the mug based on any design patterns it contains. Google is very effective at picking out text on items such as clothing and uses these to form queries, too. Knowledge Graph and Maps integration will see Google’s results improve, but for now it is behind Pinterest in identifying the intangibles that escape the grasp of language. - Amazon: Amazon could not recognize any aspects of Loafie, even though the item is in its inventory. In general, Amazon is effective at recognizing everyday objects and returning related results. Search with an image of a kettle and it will return ‘kettle’ results. This makes it useful for its core purpose as a retailer, but Amazon will also want to branch out into the ‘inspiration’ space. Better visual search results will be an important part of this strategy. - Bing: The results for Bing show that it focuses on identifying color and shape, but not necessarily the category of the object. Bing has some useful new features that allow for object isolation within complex images, but the algorithms require more data and training if Bing is to expand its remit beyond everyday objects. The results in this test were the most erratic of all the technology providers. - Camfind: As a specialist visual search tool, Camfind performed impressively, even if the results were not entirely accurate. Years of training on a wide range of objects has led to a more nuanced understanding of objects beyond just color and shape. It will be interesting to see where Camfind sits in this market as the likes of Google make visual search a priority and integrate into services like Shopping.
  37. 37. 37 In summary: - There are numerous layers of interpretation when analyzing an image: size, shape, color, object purpose, style, context… - Different technologies approach this in different ways. Pinterest is best at blending these factors, for now. - Even when an object exists in the image inventory (in the case of Amazon), there are no guarantees that the visual search engine will recognize it. - We need to help search engines as much as possible.
  38. 38. 38 Camera-based search leads to: ● 48% more product views ● 75% greater likelihood to return ● 51% higher time on site ● 9% higher average order value - BloomReach Google image search is already a huge opportunity. As Google integrates Lens into more products, brands will be able to connect with their audience in new and more effective ways.
  39. 39. Accessible visual search technology is allowing new players like Hutch to innovate. The advantage of visual search here for both the consumer and the brand is clear. Consumers get better results that allow them to ‘see’ the products and brands get the chance to show off their wares. 39
  40. 40. 40 1. Query understanding: Object shape, size, color. Annotations help connect this to Pinterest’s inventory. 2. Blender: Results pulled from Visual Search (similar aesthetic), Object Search (similar objects) and Image Search (similar text search results). Blending ratios are weighted dynamically based on query understanding - including related boards and past user behavior. How it works