For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/amd/embedded-vision-training/videos/pages/may-2018-embedded-vision-summit-schmit For more information about embedded vision, please visit: http://www.embedded-vision.com Mike Schmit, Director of Software Engineering for computer vision and machine learning at AMD, presents the "Bad Data, Bad Network, or: How to Create the Right Dataset for Your Application" tutorial at the May 2018 Embedded Vision Summit. When training deep neural networks, having the right training data is key. In this talk, Schmit explores what makes a successful training dataset, common pitfalls and how to set realistic expectations. He illustrates these ideas using several object classification models that have won the annual ImageNet challenge. By analyzing accurate and inaccurate classification examples (some humorous and some amazingly accurate) you will gain intuition on the workings of neural networks. Schmit's results are based on his personal dataset of over 10,000 hand-labeled images from around the world.

2. •
•
•
•
•
•
•
•
•

3. •
•

4. … 2017
ImageNet Large Scale Visual recognition Challenge

5. A Simple Example: Overfitting and Underfitting

6. Example: Overfitting and Underfitting

7. Example: Overfitting and Underfitting

8. Example: Overfitting and Underfitting

9. Match / Mismatch vs reported confidence

10. Introducing matching within a hierarchy

11. Match / Mismatch at different hierarchical levels

12. Example

13. tbd
The Test dataset
Model top 1 2nd 3rd 4th 5th top 5
Inception V4 70.1% 11.2% 4.8% 2.5% 1.7% 90.3%
Resnet-50 57.2% 12.2% 6.3% 3.7% 2.3% 81.6%
Resnet-101 53.5% 12.5% 7.2% 4.0% 2.8% 79.9%
Resnet-152 62.2% 12.7% 5.4% 3.0% 2.0% 85.1%
VGG-19 50.8% 12.7% 6.2% 4.2% 3.0% 76.8%
VGG-16 51.7% 12.3% 6.7% 4.0% 2.4% 77.1%
Labeled Images 7741
Negative tests
(not labeled)
1871
Total Images 9612
Total classes 757
(of 1000)

14. Constructing your dataset
•
•
–
•
•
–
Choosing or Constructing your Dataset(s)

15. Constructing your dataset
•
–
•
–
Choosing or Constructing your Dataset(s)

16. Constructing your dataset
•
•
–
•
–
•
•
Choosing or Constructing your Dataset(s)

17. CD players and Records

18. Lions

19. Lawn mowers

20. Stage actors, dancers

21. Mittens

22. Tigers

23. Socks

24. Minor changes in camera angle & zoom

25. Does the network learn the major features?

26. Pillow

27. Cats or pillows?

28. Ships

29. Oranges

30. Strange mailbox

31. Golf balls

32. More golf balls

33. The winner: Safety pins (perfectly clean iconic images)

34. Conclusions
•
–
–
•
–
Future work

35. Conclusions
•
•
Conclusion

36. •
•
•
•
•
•
•
•
•
•

37. Disclaimer & Attribution

39. Soccer balls

40. Some images with objects not in the training set

41. Pool Table and air hockey

42. First-time snake hunter

43. Roman ruins

44. Famous statues

45. Crash Helmet (changes in pose)

46. golf ball 24%
Rottweiler 6%
Rottweiler 4%
poodle 6%
hay 25%
hay 22%
ladle 3%
missile 9%
hockey puck 6%
dirigible 50%
leatherback
turtle
20%
missile 20%
sandbar 4%
hockey puck 94%
hockey puck 99%
hockey puck 99%
hockey puck 8%
ski 6%
velvet 13%
artic fox 7%
electrical
switch
7%
toilet tissue 23%
paper towel 10%
bath towel 71%
velvet 6%
paper towel 33%
paper towel 74%
bath towel 67%
bath towel 33%
bath towel 27%