Computer Vision: Coming to a Store Near You

4. © Cloudera, Inc. All rights reserved. 4 Mobile Apps ESLs, Robots Autonomous Vehicles Magic Mirrors+ Shelf, Bin, & Rack Sensors Surveillance CamerasCompetitive PricingSatellite Images Logistics & Asset TelematicsIn Home Devices Frictionless Checkout RFID Streams SINGLE VIEW OF CUSTOMER FRICTIONLESS COMMERCE UBIQUITOUS FULFILLMENT RELEVANT INTERACTIONS OPERATIONAL EFFICIENCY Streaming Capabilities Enable New Insights,Right-Time Response New Data Sources Extend Retail Intelligence = video, image data

5. © Cloudera, Inc. All rights reserved. 5 BUSINESS IMPROVEMENT OPPORTUNITIES FOR COMPUTER VISION IN-STORE • Enabling Frictionless Commerce • Improving Operational Efficiencies • Improving Customer Experiences • Reducing Fraud & Shrink $410B “AI / IOT ECONOMIC VALUE- ADD IN RETAIL BY 2025, WITH SOME OF THE MOST VALUABLE USE CASES DRIVEN BY STREAMING VIDEO ANALYTICS.” (MCKINSEY, 2017) REAL-TIME PROCESSING –IMPROVED ACCURACY – DEEPER INSIGHTS

6. © Cloudera, Inc. All rights reserved. 6 ENABLING FRICTIONLESS COMMERCE Amazon Go Store • Computer Vision (100s of cameras on shelves, ceiling) • Sensor Fusion (cameras, shelf scales) • Deep Learning (advanced pattern recognition, 100s of algorithm’s)

7. © Cloudera, Inc. All rights reserved. 7 ENABLING FRICTIONLESS COMMERCE Alibaba’s HEMA store • Robots, apps, facial recognition and overhead conveyor belts for fulfillment delivery • Using facial recognition customers can pay using their face - no QR code nor credit card swipe required

8. © Cloudera, Inc. All rights reserved. 8 IMPROVING OPERATIONAL EFFICIENCIES Out-of-Stock & Merchandising Execution • Out-of-Stock Notifications • Price and Promotion Compliance • Schematic & Display Compliance Photo Source: Retail Technology Corp.

10. © Cloudera, Inc. All rights reserved. 10 IMPROVING OPERATIONAL EFFICIENCIES Advertising & Promotional Execution: • Day-Part Promotional Execution • Audience and Conversion Insights • Proximity Marketing

11. © Cloudera, Inc. All rights reserved. 11© Cloudera, Inc. All rights reserved. $174B IN CPG ANNUAL TRADE PROMO SPEND. OF THAT, 17% ($300M) SPENT ON SHELF, YET 68% OF CONSUMER DECISIONS ARE MADE AT THE SHELF. IRI, 2016 OPPORTUNITY: IMPROVED MERCHANDISING EXECUTION INSIGHTS

12. © Cloudera, Inc. All rights reserved. 12 IMPROVING CUSTOMER EXPERIENCES • Identify VIPs, ensure high-touch CX (‘clienteling’) • Understand, respond (anonymously) with proactive customer service opportunities

13. © Cloudera, Inc. All rights reserved. 13 • Simplifying the dressing room experience • Assist to complete the ‘look’ via cross- sell of matching shoes, accessories • Reduce likelihood of returns IMPROVING CUSTOMER EXPERIENCES

14. © Cloudera, Inc. All rights reserved. 14© Cloudera, Inc. All rights reserved. $260B IN MERCHANDISE IS RETURNED TO RETAILERS EACH YEAR. AS MUCH AS 40% OF ONLINE CLOTHING PURCHASES ARE RETURNED. NRF, 2016 OPPORTUNITY: IMPROVED CX + MERCHANDISE RETURNS

15. © Cloudera, Inc. All rights reserved. 15 The use cases for CV are promising…driving meaningful business impact, even creating new revenue models Frictionless Checkout Loss Prevention Neighborhood Insights Personalization Customer Insights Customer Engagement Dynamic Merchandising Autonomous DeliverySource: Allure Source: Ahold/Delhaize Source: Caper Source: Orbital Insight Source: Aura visionSource: Trigo-vision

17. © Cloudera, Inc. All rights reserved. 17 HOW DOES IMAGE RECOGNITION WORK? Detect & Score Convolutional neural network analyzes images and produces actionable representations Compare Representation manifolds can be mapped to find matches, similar, or complementary images Match & Alert Users alerted and shown matches

18. © Cloudera, Inc. All rights reserved. 18© Cloudera, Inc. All rights reserved. IMPLEMENTING THESE TECHNIQUES IN YOUR COMPANY • Options for implementing these functionalities in your store • Purchase a point service for the desired functionality • Whether it’s the whole solution or just part of the solution • Hire a company to design and implement a custom solution • Shameless plug for my company • Implement the solutions yourself!

19. © Cloudera, Inc. All rights reserved. 19© Cloudera, Inc. All rights reserved. IMPLEMENTING THESE TECHNIQUES IN YOUR COMPANY • Need to define the problem and break it down into smaller parts • There are a few CV modalities where machine learning shines • Think of a modality as a smaller subproblem in machine perception • May have multiple modalities within one problem • A few different models may need to run • Design supporting infrastructure for ML models • Method to feed data into model(s) • Methods to interpret model output(s) • System to chain together multiple models • Translate from model output to human interpretable results

21. © Cloudera, Inc. All rights reserved. 21 DEEP LEARNING • Many computer vision applications are now becoming successful due to advances in deep learning • A deep learning model for computer vision is specifically designed to exploit the spatial dependencies in image data • Flexible architecture designs to produce desired output for given problem domain

22. © Cloudera, Inc. All rights reserved. 22 COMPUTER VISION PROBLEMS • Computer vision problems can often be broken down into several problem types • Can make global inferences about an image • Multiple spatially aware inferences • Pixel level inference for very fine grained output needs • Learning latent representations

23. © Cloudera, Inc. All rights reserved. 23 GLOBAL INFERENCE • We may want a model to produce only a single output for a given image • Identify the subject of the image (type of inventory, type of object) • Identify affect of a person’s face • Qualitative assessment of scene or object (does shelf need organization, is inventory damaged) • Infer distance between two objects Gender Affect Eyeglasses Headwear

24. © Cloudera, Inc. All rights reserved. 24 SPATIALLY AWARE INFERENCE • We may want a model to produce many outputs for a given image • Describes the locations and classifications for objects in an image • Locate customers in your stores (counts, locations, heatmaps, time lingering, …) • Locate products on a shelf Aura Vision, https://auravision.ai

25. © Cloudera, Inc. All rights reserved. 25 PIXEL LEVEL INFERENCE • We may want to produce an output for every pixel in an image • Classify each pixel as what object it’s a part of • Produce mapping or overlay for the entire image • Probability map of object location • Depth map of a room

26. © Cloudera, Inc. All rights reserved. 26 LATENT REPRESENTATION LEARNING • Finally, we may want generate some kind of representation of an image that allows us to perform some useful math • Representations capture semantic information • Quantify similarity metrics between two images • Train a model on a closely related task that forces it to learn good representations

28. © Cloudera, Inc. All rights reserved. 28 EXAMPLE USE CASE • Some combination of these tools and pre/post model logic can be used to build a real retain computer vision use-case • Lets assume that we want to build a tool to help our customers by providing clothing recommendations with CV • They come into our store looking for a specific style of clothing • Arrive with image of desired style • Set up kiosk for clothing recommendations • Augment magic mirror

29. © Cloudera, Inc. All rights reserved. 29 FASHION PRODUCT RECOMMENDATION EXAMPLE • In order to create this application we’ll need two different machine learning models and some logic around the model outputs • Some capability to interpret the image and find the targeted piece of clothing • Using a segmentation model • Extract targeted article of clothing from image • Create representation of clothing that allows it to be compared to others (model b) • Create representations of entire catalog • Matching algorithm to find closest match to query article

39. © Cloudera, Inc. All rights reserved.39 WHATINDUSTRIALIZED“EDGETOAI”LOOKSLIKE Streaming Ingest Batch Ingest Machine Learning Tools BI Tools and SQL Editors Data Products DATA, METADATA, SECURITY, GOVERNANCE, WORKLOAD MANAGEMENT MACHINE LEARNING DATA ENGINEERING DATA WAREHOUSE OPERATIONAL DATABASE

40. © Cloudera, Inc. All rights reserved. THANK YOU Let’s Keep the Conversation Going… Brent Biddulph MD, Retail & CG, Cloudera bbiddulph@cloudera.com +1 425.273.6851 www.linkedin.com/in/brentbiddulph/ @brentbiddulph Florian Muellerklein Data Scientist, Miner & Kasch fmuellerklein@minerkasch.com +1 410.564.1720 www.linkedin.com/in/florian-muellerklein/ @mllrkln

Computer Vision: Coming to a Store Near You

You are reading a preview.

Check these out next

Computer Vision: Coming to a Store Near You

More Related Content

Slideshows for you (20)

Similar to Computer Vision: Coming to a Store Near You (20)

More from DataWorks Summit (20)

Recently uploaded (20)