ABSTRACT : A hardware/software (HW/SW) co-design approach developing a contact lenses try-on system via the use of the Field Programmable Gate Array (FPGA) [1] is proposed in this paper. To achieve an interactive user-friendly environment, the menu of contact lenses and the result that shows how it looks when wearing the contact lens are displayed on the LCD touch panel directly. After capturing the user’s image via CMOS sensors, image processing is employed to determine user’s irises from the image. A particle filter algorithm is then introduced to track the irises so that the contact lenses chosen by the user can be overlapped on the image to provide a natural examination of the effect of the contact lenses try-on system. Experimental results show good performances of the proposed system. Keywords: FPGA, Eye Tracking, Particle Filter, Nios II, Image Processing

1. Invention Journal of Research Technology in Engineering & Management (IJRTEM)
ISSN: 2455-3689
www.ijrtem.com Volume 1 Issue 4 ǁ June. 2016 ǁ PP 01-04
| Volume 1| Issue 4 | www.ijrtem.com | June 2016| 1
FPGA-Based Contact Lenses Try-On System
Chen-Chien Hsu, Hsiang-Yu Huang, Man-Wei Huang, Wei-Yen Wang, and Chiang-Heng Chien
(Department of Electrical Engineering, National Taiwan Normal University, Taiwan, R.O.C.)
ABSTRACT : A hardware/software (HW/SW) co-design approach developing a contact lenses try-on system via the use of the Field
Programmable Gate Array (FPGA) [1] is proposed in this paper. To achieve an interactive user-friendly environment, the menu of
contact lenses and the result that shows howit looks when wearing the contact lens are displayed on the LCD touch panel directly. After
capturing the user’s image via CMOS sensors, image processing is employed to determine user’s irises from the image. A particle filter
algorithm is then introduced to track the irises so that the contact lenses chosen by the user can be overlapped on the image to provide
a natural examination of the effect of the contact lenses try-on system. Experimental results show good performances of the proposed
system.
Keywords: FPGA, Eye Tracking, Particle Filter, Nios II, Image Processing
1. Introduction
Contact lenses, a thin lens placed directly on the surface of the eyes, are regarded as a medical device to enhance visual
abilities. Many people wear color cosmetic contact lenses for aesthetic reasons nowadays. To meet the consumer’s demand, more and
more contact lenses manufactures produce color cosmetic contact lenses. However, it is difficult to decide preferable contact lenses since
consumers are often not allowed to try it on. This is because the costs of color cosmetic contact lenses are relatively high, and only a few
manufactures provide samples for consumers. To solve this problem, this research develops a system that allows consumers to virtually
try on color cosmetic contact lenses without physically wearing it so that consumers can easily choose their preferable ones. To allow a
natural presentation of the contact lenses, eye tracking is one of the key points in the proposed contact lenses try-on system. The white
part of the eye is sclera, which interferes less during the course of tracking. The black part is iris, which is the main object to be tracked.
There are different light sources for use in eye tracking, such as infrared light and natural light. If infrared light is used, little effort is
required for image processing. However, there is a safety concern in which infrared light might potentially damage human’s eyes.
Therefore, the desired natural light source is used, where a particle filter [2][3] algorithm is developed to track the exact position of the
iris after the image has been captured. HSV color space is used to remove the shadow caused by natural light while a 5x5 median filter
is used to remove impulsive noise. Binarization is also very important to identify the object of interest in the image. Finally, dilation and
erosion are used to reconstruct and decompose the image. The development platform of the contact lenses try-on system is based on
Altera’s DE2 development board, which provides high speed SSRAM, SDRAM with large data storage, uClinux operating system, the
Cyclone II, and SoPC Builder’s logical module. Altera also provides devices such as COMS sensor to capture images and LCD touch
panel to display the menu and the results.
2. Proposed Contact Lenses Try-On System
1.1 Design Architecture
Figure 1 Hardware Design of the proposed contact lenses try-on system

2. FPGA-Based Contact Lenses Try-On System
| Volume 1| Issue 4 | www.ijrtem.com | June 2016| 2
Fig. 1 shows the hardware design of the proposed contact lenses try-on system. After the CMOS sensor receives data from the
image capturing user’s face, the image is gone through image processing for eye tracking. The menu of the contact lenses is then shown
on the LCD touch panel under the control of CMOS Master Controller, NIOS II processor, and LTM Master Controller. Finally,
Quatrus II software by Altera [4] is used to integrate the system.
1.2 Overall Flow Chart
Fig. 2 (a) shows the flowchart of this system. First, the menu of contact lenses try-on system is displayed on the LCD touch
panel. Then, the user touches the panel to choose what kind of contact lenses they would like to try on. As the user had chosen their
preferable contact lenses, D5M will capture the user’s image. After that, image processing is used to distinguish the facial part from the
background for eye tracking. The location of the iris is tracked via particle filter as a consequence. Finally, the image of the contact
lenses chosen by the user is overlapped on the image and the result is presented on the LCD board.
(a) (b)
Figure 2 (a) Flow chart of the proposed contact lenses try-on system. (b) Flow chart of the particle filter algorithm for tracking.
1.3 Median Filter
Since an image could contain a lot of noises that could thereby affect the performance of the system. Therefore, median filer [5]
is introduced to remove the noise of the image. According to the experimental results, a size of 5x5 mask is suitably used for the median
filter in the proposed system.
1.4 Morphology
After the image is processed through color space conversion and median filter, binarization can make the object more
obvious. However, the result might be incomplete due to the light or the noise. Therefore, morphology [6] is required to track the
location of the center of the object. Morphology[6] includes dilation, erosion, opening, and closing. Dilation and erosion are used in this
paper, where dilation refills the losing part of the iris while erosion removes the noise around eyes. Morphology completes the shape of
the iris.
1.5 Particle Filter
Fig. 2 (b) shows the flowchart of particle filter algorithm for tracking. First, weight assignment deals with calculations of
weights valuing each particle. The larger the value is, the higher the possibilityfor this particle to be the location of the iris. Tournament
selection [7] is employed so that particles with the top five highest weights are chosen to estimate the location of the iris. Then, a
resampling procedure is used to eliminate particles with lower weights and replicate the particles with higher weights. After that, since
the object might move and slightly change by the time goes, prediction is required to estimate the location of each particle at the next
time. Fig. 3 shows the process of eye tracking by particle filter, where the yellow dots are particles generated by particle filter. Through
iterations, an optimal location in the estimation of the iris is obtained thereafter. It is marked in red in the figure.

3. FPGA-Based Contact Lenses Try-On System
| Volume 1| Issue 4 | www.ijrtem.com | June 2016| 3
Figure 3 The process of eye tracking by using particle filter algorithm.
3. Experimental Results
The proposed contact lenses try-on system is implemented on a Altera’s DE2-70 development board as shown in Fig. 4. Users
can choose the contact lenses they would like to try on from the menu shown in Fig. 5 (a). In order to achieve a more natural result, we
adjust the display by comparing the color of the iris and the background illumination in that environment shown in Fig. 5 (b).
Figure 4 The Altera’s DE2-70 development board.
(a) (b)
Figure 5 (a) Contact lenses menu for users to choose. (b) The result of the proposed contact lenses try-on system.
4. Conclusion
The proposed contact lenses try-on system is a low cost, simple operation, and user-friendly platform, which replaces the high
cost contact lenses sample and provides a more efficient way for consumers to try on their preferable color contact lenses. Because
contact lenses images are stored in SD card provided by the DE2, new styles of contact lenses can be added anytime when they are
available for consumers. Therefore, the proposed system has high flexibilities for both the sellers and consumers aspects.

4. FPGA-Based Contact Lenses Try-On System
| Volume 1| Issue 4 | www.ijrtem.com | June 2016| 4
5. Acknowledgements
This research is partially supported by the “Aim for the Top University Project” and “Center of Learning Technology for
Chinese” of National Taiwan Normal University (NTNU), sponsored by the Ministry of Education, Taiwan, R.O.C. and the
“International Research-Intensive Center of Excellence Program” of NTNU and Ministry of Science and Technology, Taiwan, under
Grants no. MOST 104-2911-I-003-301 and MOST 103-2221-E-003-027.
REFERENCES
[1]. Oetken, S. Wildermann, and J. Teich, A Bus-based SoC Architecture for Flexible Module Placement on Reconfigurable FPGAs: Int. Conf. on
[2]. Field Programmable Logic and Applications (FPL), Milano, 2010, 234-239.
[3]. S. Fazli, H. Moradi, and P. Hamed Bouzari, Particle Filter based Object Tracking with Sift and Color Feature: 2nd Int. Conf. on Machine
Vision, Dubai, 2009, 89-93.
[4]. D. Xu, L. Huang, and C. Liu, Object Tracking using Particle Filter based on Color Correlogram:2nd Int. Symposium on Intelligent Information
Technology Application, Shanghai, 2008, 608-612.
[5]. Altera Corporation, Quarts II Handbook (Altera Corporation, 2004).
[6]. T. Huang, G. Yang, and G. Tang, A fast two-dimensional median filtering algorithm, IEEE Trans. on Acoust., Speech, and Signal Processing,
27(1), 1979, 13–18.
[7]. O. Dgforges, and N. Normand, A generic systolic processor for real time grayscale morphology: Int. Conf. on Acoustics, Speech, and Signal
Processing, Istanbul, 2006, 3331-3334.
[8]. Z. Michalewicz, Genetic Algorithm + Data Structure = Evolution Programs, Third Ed., Springer, 1996.