1. Creating Algorithms to Develop a Sense of Color Constancy in
a Robot
Cheyenne Sokkappa1 Jenna Permut2 Patrick Lancaster3 Joshua Smith3, 4
Introduction Euclidean Clustering Method
Discussion
References
1 Sequim High School; 2 Mercer Island High School; 3 Department of Computer Science and Engineering, University of Washington; 4 Department of Electrical Engineering,
University of Washington
Acknowledgements
AEOP REAP Program
Patrick Lancaster and the Sensor Systems Laboratory
CSNE and the Young Scholars Program
Kristen Bergsman, Lise Johnson, and Eric Chudler
This project was supported by Award Number EEC-1028725 from the National Science
Foundation (NSF). The content is solely the responsibility of the authors and does not
necessarily represent the official views of the NSF.
What is Euclidean Clustering?
• Euclidean clustering sorts data into groups and in our case,
sorts pixels into groups based on a color threshold.
• We want to sort pixels based on color and closeness to
each other.
Started by Clustering Similar Pixels
• Started by choosing a random pixel.
• Found neighbors with similar color (Used color threshold)
• Added neighbors to a cluster
• Each cluster added all the pixels that have a similar color
to the cluster until it reached pixels with different colors
on all borders
Gave average color to each cluster
• Added up the colors of all the pixels in each cluster
• Average the colors and assign the pixels in each cluster to
the average color.
Color Constancy is the process which humans use to
differentiate colors under different lighting. For cameras, it is
the ability to track colors in an image or video frame. Color
Constancy gives a robot the ability to determine colors under
different amounts of lighting. This allows the robot to simulate
human vision and memory.
Objectives:
• Code an algorithm in the development environment,
Processing, that allows our PR2 robot, Hobbes, to determine
and assign the colors of squares in a Rubik’s cube while
solving the cube. Our algorithm uses Euclidean Clustering
and test images were taken using a Kinect.
• Test Euclidean Clustering with a mapping program that finds
the squares based on color and put the square’s information
into a data structure.
Results
Example
Original Rubik's
Cube Image
Clustered Output
RGBValues
Time of the Day
Figure A: Testing RGB values over a 24 hour time period in
direct sunlight. Values were taken from the same red,
green and blue squares on the Rubik’s cube. The values
vary about 20 points within the RGB range with the
exception of images taken at night and early morning
which were outliers.
Motivation: RGB
values of the same
image change under
different lighting
(see Figure A).
Therefore our
solution was to take
into account a color
threshold in which
the camera
determined colors
on the Rubik’s cube.
We tested our
Euclidean Clustering
under 4 shades of
lighting to see if we
could track the
squares (see Figure
B). We found that
the Euclidean
Clustering algorithm
clustered dark colors
with the background
in low light and
therefore, our
mapping algorithm
was unable to track
those squares.
Figure B: We loaded all 6 faces of the Rubik’s
cube in the mapping algorithm to track the
squares. Each column corresponds to the
percent of squares tracked based on the lighting
in the picture below it.
%Squaresfound
We found out that while the algorithm was mostly successful, our
Euclidean Clustering algorithm had a limit at which the color
threshold was unable to cluster similar colors (see Figure B).
We also planned on making objective functions that would take the
place of our current tracking algorithm. Or we would also use K-
Means Clustering to track and identify the squares.
We think that if we could implement Multi-Scale Retinex
(brightening of the picture), then our problems with the Euclidean
Clustering would be solved and our tracking algorithms would be
able to work to their full extent.
Eventually, we think that this process may allow for real life
applications such as self driving cars and personal robotics for
identifying objects in different lighting.
Results (continued)
Euclidean
Clustering
Tracking
Squares
Mapping
Accomplished:
• Cluster colors using Euclidean Clustering
• Test accuracy in mapping system
Yet to finish:
• Implement Multi-Scale Retinex to maximize the accuracy of
Euclidean Clustering’s output.
Jobson, Daniel J., Zia-ur Rahman, and Glenn A. Woodell. "A Multiscale Retinex for Bridging the Gap Between Color
Images and the Human Observation of Scenes." IEEE 6.7 (1997): 965-76. Web.
"Euclidean Cluster Exctraction." Point Cloud Library. Open Perception Foundation, n.d. Web. 17 Aug. 2015.