The document discusses the challenges and solutions of item recognition in retail, focusing on computer vision technology used at 7-Eleven. Key challenges include similar looking items, item overlap, and packaging changes, while proposed solutions involve feature encoding and model training using neural networks. The conclusion highlights the potential of these solutions to revolutionize the retail industry by enhancing the shopping experience and improving inventory accuracy.

1. Item Recognition in
Retail
Sumedh Datar
Senior Machine Learning Engineer
7-Eleven

2. Problem
2
© 2023 7-Eleven
Barcode Scanner Item Lookup RFID

3. • Product Recognition
• Planogram Compliance
• Smart Self Checkout
• Autonomous Stores
Why Computer Vision in Retail
3
© 2023 7-Eleven
Slurpee : 0.99

4. Challenge with Item Recognition #1
Similar Looking Items
4
© 2023 7-Eleven

5. Challenge with Item Recognition #2
Item Overlap and Occlusion
5
© 2023 7-Eleven

6. Challenge with Item Recognition #3
Same Item Different Size
6
© 2023 7-Eleven

7. Challenge with Item Recognition #4
Number of Items to Classify
7
© 2023 7-Eleven

8. Challenge with Item Recognition #5
Packaging Changes
8
© 2023 7-Eleven

9. Solution - Feature Encoding
9
© 2023 7-Eleven
Neural Network Backbone
[1.2, 3.5, 4.6, …..]
[2.1, 5.3, 6.4, …..]
[3, 8, 12, …..]

10. Solution – Model Training
10
© 2023 7-Eleven
Neural Network
Backbone
Neural Network
Backbone
Contrastive
Loss
Output
Embeddings
Embeddings

11. Solution – RGB D Detection
11
© 2023 7-Eleven

12. Solution – Selective Creation of Data
12
© 2023 7-Eleven

13. Debugging Strategies – Embedding Visualizer
13
© 2023 7-Eleven

14. Debugging Strategies – Neural Activations
14
© 2023 7-Eleven

15. Product Recognition System - Architecture
15
© 2023 7-Eleven
Preprocessing

16. Segmentation
16
© 2023 7-Eleven
Detection box
Instance segmentation

17. Product Recognition System – Pre-Processing
17
© 2023 7-Eleven
Crop
Logo
Text

18. Product Recognition System – Recognition
18
© 2023 7-Eleven
Aggregated
features
Feature
Database
Feature Comparison
Final Result

19. Platform to Deploy CV Solutions
19
© 2023 7-Eleven
Low-cost edge
solutions GPUs Cloud

20. Challenges in Deploying CV Solutions
20
© 2023 7-Eleven
Accuracy Speed Cost

21. Our In-House Solution
21
© 2023 7-Eleven

22. • Can revolutionize the retail industry
• Enhance shopping experience
• Improve inventory accuracy
• Run retail operations efficiently
Conclusion
22
© 2023 7-Eleven