Upcoming SlideShare
×

# The ball is not just orange.

1,362 views

Published on

How to find balls in images of a moving humanoid robot using neural networks.
Talk from RoboCup Workshop 07 in Pittsburgh.

0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
1,362
On SlideShare
0
From Embeds
0
Number of Embeds
213
Actions
Shares
0
10
0
Likes
0
Embeds 0
No embeds

No notes for slide

### The ball is not just orange.

1. 1. A B I N J O: U C  L  C R  I Hannes Schulz, Hauke Strasdat, and Sven Behnke University of Freiburg Institute of Computer Science Nov 29, 2007
2. 2. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments M The ball is small,
3. 3. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments M The ball is small, is easy to confuse with other objects
4. 4. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments M The ball is small, is easy to confuse with other objects is the most important object on the ﬁeld: You cannot play sensibly without knowing its position
5. 5. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments M The ball is small, is easy to confuse with other objects is the most important object on the ﬁeld: You cannot play sensibly without knowing its position We should put a lot of effort into ﬁnding the single real ball.
6. 6. O 1 I  B  N-B 2 F B C 3 C  B C 4 E
7. 7. O 1 I  B  N-B 2 F B C 3 C  B C 4 E
8. 8. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D B L L?
9. 9. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D B L L? T E C
10. 10. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D B L L? L C
11. 11. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D B L L? M B
12. 12. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D B L L? C W L
13. 13. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D B L L? C R
14. 14. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D N-B L L?
15. 15. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D N-B L L? H  F
16. 16. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D N-B L L? O O
17. 17. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D N-B L L? O O
18. 18. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D N-B L L? F
19. 19. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H D N-B L L? F
20. 20. O 1 I  B  N-B 2 F B C 3 C  B C 4 E
21. 21. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments C  E  YUV S
22. 22. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments C  E  YUV S Two ellipses for “orange” Actual ball color Wider, brownish color
23. 23. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments C  E  YUV S Two ellipses for “orange” Actual ball color Wider, brownish color Allows for motion blur
24. 24. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments S  C I orange YUV Camera Image (-candidate) white green 64 : 1
25. 25. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments S  C I 1. Find Maximum 2. Find Weighted Mean orange (-candidate) white green 64 : 1
26. 26. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments S  C I orange cut corresponding area (-candidate) white green 64 : 1
27. 27. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments S  C I orange cut corresponding area (-candidate) white Box size depends on position in image green 64 : 1
28. 28. O 1 I  B  N-B 2 F B C 3 C  B C 4 E
29. 29. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments P  C
30. 30. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments P  C Projection changes with ellipses: Robust to changes in lighting conditions
31. 31. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments P  C Projection changes with ellipses: Robust to changes in lighting conditions
32. 32. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments P  L YUV-Image Y-Image Subsampled Y-Image Subtraction of mean: Robust to changes in lighting conditions
33. 33. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H L F
34. 34. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H L F
35. 35. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments H L F - - + - - - - + - -
36. 36. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments N N C 1: Ball 0: No Ball - - + - -
37. 37. O 1 I  B  N-B 2 F B C 3 C  B C 4 E
38. 38. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments NR KS 06 – T H 520 MHz ARM PocketPC VGA HTC Camera D S 160 balls 440 non-balls divided randomly in training set (80%) and test set (20%)
39. 39. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments NR KS 06 – T H 520 MHz ARM PocketPC VGA HTC Camera D S 160 balls 440 non-balls divided randomly in training set (80%) and test set (20%) P (T S) 100% of distractors classiﬁed correctly 1 ball out of 32 not recognized
40. 40. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments NR KS 06 – R T H 520 MHz ARM PocketPC VGA HTC Camera R T T Robot decides autonomously which object to approach.
41. 41. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments NR KS 2007 H 1.33 GHz PC WVGA µEye Camera
42. 42. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments NR KS 2007 S D D S 273 balls 548 non-balls training set 62%, validation set 13%, test set 25% varying lighting conditions
43. 43. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments NR KS 2007 A S D S Avg Luminance 273 balls ball non-ball 548 non-balls training set 62%, validation set 13%, test set 25% varying lighting conditions Avg Orange-Greenness ball non-ball
44. 44. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments NR KS 2007 A S D S Avg Luminance 273 balls ball non-ball 548 non-balls training set 62%, validation set 13%, test set 25% varying lighting conditions Avg Orange-Greenness R  T S ball non-ball 91.1% accuracy 76.6% if stimuli ﬂipped up/down. Drop suggests dependency on gradient. 88.2% if lighting in testset differs. Classiﬁer seems indifferent.
45. 45. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments C  C Classiﬁer Task Neural Net Linear Classiﬁer KNN (k = 5) Regular 91.1% 86.5% 88.5% Flipped 74.0% 70.0% 76.6%
46. 46. Balls and Non-Balls Finding Ball Candidates Classiﬁcation of Ball Candidates Experiments C The ball has properties aside from being orange These properties are exploited by our Neural Network Classiﬁers Changes in Lighting conditions can be dealt with by projection to lines in YUV-space and Luminance. The method introduced here can be generalized to other small-sized objects on the ﬁeld.