We are using Deep Learning models to help Royal Flora Holland automate their auction processes. With over 100,000 transactions per day and 400,000 different types of flowers and plants, Royal Flora Holland is the biggest horticulture marketplace and knowledge center in the world. An essential part of their process is having the correct photographs of the flower or plants uploaded by suppliers. These photos are uploaded daily and could have requirements. For example, some images require a ruler to be visible or a tray to be present. Manual inspection is practically impossible. Using Keras with a Tensorflow backend we implemented a Deep Neural Network (DNN) using transfer learning for each screening criteria. We also apply heuristics and business rules. The goal is to give real-time feedback at upload time, this challenged us to run multiple deep learning models in real-enough-time. During the journey of building the Image Detection system we have used specific implementations that can be insightful and helpful to the audience. For example, our models are not only trained in parallel but transfer learning allows us to engineer a single 1st component for all models and then having the flow distribute over each of the DNN (~90% of the work is shared among the DNNs). Our models achieve above 95% accuracy and because of the component-like architecture it’s very flexible.

1. Rodrigo Agundez,
Data Scientist @ GoDataDriven
rodrigoagundez@godatadriven.com
Operation Tulip: Using
Deep Learning to Automate
Auction Processes
#AISAIS11

2. About me
#AISAIS11
●
PhD in Theoretical Physics
●
Data Scientist @ GoDataDriven
or for my TSA buddies:
“I work with computers”
●
From Tijuana, Mexico
●
Live in Amsterdam
●
Deep learning enthusiast

3. #AISAIS11
●
Data innovation consultancy
●
High quality:
– Data scientists
– Data engineers
●
Based in Amsterdam
●
Heineken, ING, Air
France-KLM, Unilever,
Maarkplaats (e-bay), Schiphol, KPN
booking.com, Unilever, Royal FloraHolland, etc.

4. Outline
#AISAIS11
●
Flower auction AI use case
●
Problem definition
●
Proposed solution
●
Transfer learning & bottleneck features
●
Inference optimization
●
Mariana & Heureka & Judy
●
Feedback loop
●
Architecture
●
Conclusion

5. Outline
#AISAIS11
●
Flower auction AI use case
●
Problem definition
●
Proposed solution
●
Transfer learning & bottleneck features
●
Inference optimization
●
Mariana & Heureka & Judy
●
Feedback loop
●
Architecture
●
Conclusion

6. Royal FloraHolland
#AISAIS11
World’s biggest flower auction house

7. Royal FloraHolland
#AISAIS11
400,000 different types of plants and flowers

8. Royal FloraHolland
#AISAIS11
Over 26 million plants and flowers auctioned daily

9. Flower auction AI use case
#AISAIS11
Typical flow

10. Flower auction AI use case

11. Flower auction AI use case
#AISAIS11
How big is it?

12. Flower auction AI use case
#AISAIS11
Hu?

13. Flower auction AI use case
#AISAIS11
He’s everywhere!

14. Requirements examples
#AISAIS11
Ruler Tray White
background

15. Multiple requirements
#AISAIS11

16. Flower auction AI use case
#AISAIS11
Gate keeper

17. Flower auction AI use case
#AISAIS11
Automation

18. Flower auction AI use case
#AISAIS11
Reject or fix when possible

19. Flower auction AI use case
#AISAIS11
Real-time feedback

20. Use cases
#AISAIS11
●
Help the quality team
●
Floriday: automated classification of picture types
●
Photo app
●
Potential future use as a service for third-parties

21. Outline
#AISAIS11
●
Flower auction AI use case
●
Problem definition
●
Proposed solution
●
Transfer learning & bottleneck features
●
Inference optimization
●
Mariana & Heureka & Judy
●
Feedback loop
●
Architecture
●
Conclusion

22. Problem definition
#AISAIS11
1 2
3

23. Problem definition
#AISAIS11
4 Plant
or
flowerruler
rotation
proportion
tray
...

24. Outline
#AISAIS11
●
Flower auction AI use case
●
Problem definition
●
Proposed solution
●
Transfer learning & bottleneck features
●
Inference optimization
●
Mariana & Heureka & Judy
●
Feedback loop
●
Architecture
●
Conclusion

25. Deep learning to the rescue
#AISAIS11

26. Architecture
#AISAIS11

27. Outline
#AISAIS11
●
Flower auction AI use case
●
Problem definition
●
Proposed solution
●
Transfer learning & bottleneck features
●
Inference optimization
●
Mariana & Heureka & Judy
●
Feedback loop
●
Architecture
●
Conclusion

28. Transfer learning
#AISAIS11
Recycle state of the art trained models

29. Transfer learning
#AISAIS11
Available pre-trained models

30. Transfer learning
#AISAIS11
VGG16 (University of Oxford 2015)

31. Transfer learning
#AISAIS11
ResNet (Microsoft Research 2015)

32. Transfer learning
#AISAIS11
InceptionV3 (Google 2015)

33. Transfer learning
#AISAIS11
A plant

34. Transfer learning
#AISAIS11
Re-trainRe-use

35. Bottleneck features
#AISAIS11

36. Bottleneck features
#AISAIS11
Re-train

37. Outline
●
Flower auction AI use case
●
Problem definition
●
Proposed solution
●
Transfer learning & bottleneck features
●
Inference optimization
●
Mariana & Heureka & Judy
●
Feedback loop
●
Architecture
●
Conclusion

38. Inference optimization
Model 1
Model 3
Model 1
Model 2

39. Inference optimization
Model 1
Model 3
Model 2

40. Outline
●
Flower auction AI use case
●
Problem definition
●
Proposed solution
●
Transfer learning & bottleneck features
●
Inference optimization
●
Mariana & Heureka & Judy
●
Feedback loop
●
Architecture
●
Conclusion

41. Mariana

42. Heureka

43. Mariana + Heureka

44. Judy
Product-code:
12345

45. Outline
●
Flower auction AI use case
●
Problem definition
●
Proposed solution
●
Transfer learning & bottleneck features
●
Inference optimization
●
Mariana & Heureka & Judy
●
Feedback loop
●
Architecture
●
Conclusion

46. Quality control

47. Excellent
Judy

48. Excellent
Vogue

49. Outline
●
Flower auction AI use case
●
Problem definition
●
Proposed solution
●
Transfer learning & bottleneck features
●
Inference optimization
●
Mariana & Heureka & Judy
●
Feedback loop
●
Architecture
●
Conclusion

50. Architecture

51. Load adaptation

52. Price vs performance

53. Price vs performance

54. Outline
●
Flower auction AI use case
●
Problem definition
●
Proposed solution
●
Transfer learning & bottleneck features
●
Inference optimization
●
Mariana & Heureka & Judy
●
Feedback loop
●
Architecture
●
Conclusion

55. Conclusion
Automation
Mature
solution
Feedback
loop
Deep learning
& heuristics in
production

56. Vincent Warmerdan
Data Scientist
(GoDataDriven)
Andrew Snare
Data Engineer
(GoDataDriven)
Dirk Guijt
Data Scientist
(Royal FloraHolland)
The gang

57. Niels Zeilemaker
CTO
(GoDataDriven)
Special thanks

58. Rodrigo Agundez, GoDataDriven
rodrigoagundez@godatadriven.com
@rragundez
Thank You
#AISAIS11