This document discusses high-speed, high-resolution inspection of flat panel displays. It introduces a distributed image sensor computing system (DISCS) that uses GPUs for parallel processing to enable fast, in-line inspection. The DISCS uses dark-field illumination and line scan cameras for 2D defect detection. Algorithms like binarization and edge detection are implemented on the GPUs. Experimental results on touch panels and glass show inspection of megapixel images in under 3 seconds. Stereoscopic line scanning and moire topography techniques are discussed for 3D surface profiling with nanometer resolution and micrometer depth detection. Phase shifting interferometry is used to extract height maps. The system is designed for industrial inspection and could integrate

1. High-speed and High Resolution
Inspection of Flat Panels
Ming Chang
Chung Yuan Christian University

2. Inspection of Handheld Devices
All images are from google.com.

3. Motivation and Objectives
 Increasing demand for high resolution and
high-speed inspection
 Increasing demand for in-line detection
 Poor yield in defect detection of
transparent panels
 Increasing cost in manual inspection
 2D Defects: 10μm width, 1~5 seconds/piece
 3D Defects: 1μm depth, 5~15 seconds/piece

4. Full-field In-Line Inspection System
Linear light
source
Line
CCD
module
Automated
inspection line
GPU
module
Parallel
image
processing
Parallel defect
inspection
GPU#1
GPU
#2
GPU#N
Conveyer
Control Center
Defect
Post-processing
Flawless products

5. 2D Inspection of Defects

6. Apparatus for a Dark-Field Illumination

7. Inspection Principle
Glass substrate
Scattering effect at reflective
coating induced by defect
Line Scanned
Camera
Light slit
High reflective coating
Defect
Slanted
parallel
lightsource
Moving direction
slanted angle
Light slit
Position of image acquisition
Fan-shape lighting of LED source:
Form the side-lighting effect consequently
Moving direction
High reflective
coating
Position of image acquisition
Slanted parallel light source
to forward
Line Scanned
Camera
Light slit
Moving direction
Glass substrate
Slanted
parallellight
sourceto
right
Slanted
parallellight
sourcetoleft
Carrier plate with high
reflective coating

8. Distributed Image Sensor Computing System (DISCS)
 Characteristics
 Heterogeneous parallel computing system
 Consists of multiple CPUs and GPUs
 Adopts Message Passing Interface (MPI) and Compute Unified
Device Architecture (CUDA) programming models
 High speed and high precision in-line detection of surface defects
 Hardware Development
 Consists of independent machines that form a master-slave
parallel computing model.
 Each CUDA workstation is a slave machine, which performs the
same computations and sends the result to the master machine
and has at least one CPU and one GPU.
 All the machines are connected through a high-speed network.

9. System Architecture – Hardware for Big Data
Hardware architecture of the DISCS

10. System Architecture - Software
 Consists of MPI processes and CUDA threads.
 MPI processes run in CPUs and CUDA threads run in
GPUs.
Software architecture of the DISCS.
 The MPI master process
runs on the integration
server, whereas the MPI
slave processes run on
the CUDA workstations.
 The idea of the DISCS is
to let the MPI slave
processes handle the
defect detection, and let
the MPI master process
deal with the defect
classification.

11. CUDA-Based Defect Detection Algorithms - Binarization
 Straightforward and
based on a predefined
threshold.
 If a pixel value is larger
than the threshold, the
pixel value is set to
255.
 Otherwise, the pixel
value is set to zero.
The binarization algorithm.

12. CUDA-Based Defect Detection Algorithms – Edge Detection
For a pixel with grey level 255,
if any one of the 4 neighbor pixels is 0,
the pixel is defined as the edge of defects.
Mask
1
2
3
4

13. CUDA-Based Defect Detection Algorithms – Edge Detection
12288 × 65535 pixels
CPU to GPU(ms) 1314.153
CUDA kernel
function(ms)
186.543
GPU to CPU(ms) 1311.791
Total time(ms) 2812.847

14. Complete Process Flow of 2D Defect Inspection

15. Experimental Results of DISCS
 Dark-field image of a part of one back-coated
mirror piece

16. Experimental Results of Simulated Defect Patterns (1)
Time
Size
12288 × 5000 pixels
CPU to GPU(ms) 59.296
CUDA kernel
function(ms)
47.3
GPU to CPU(ms) 57.39
Total time(ms) 163.986
Defect number 4
TimeTime
Time
Size

17. Experimental Results of Simulated Defect Patterns (2)
Time
CCD
(Left) (Right)
12288 × 5000
pixels ( left )
12288 × 5000
pixels ( right )
CPU to GPU(ms) 61.0682 62.6171
CUDA kernel
function(ms)
51.1604 51.2616
GPU to CPU(ms) 59.8082 59.6373
Redundant defect
in CPU(ms)
0.1662 0
Total time(ms) 172.203 173.516
Defect number 3 2
New defect number 4
Time
Size

18. Experimental Results of a Touch panel (1)
 (a) Dark-field image of
touch panel
(L=110mm, W=42mm)
 (b-c) Defect maps of
scratches/cracks
D1 = 14.8 um
D2 = 2.65 mm
 (d) Map of edge boundary,
D3

19. Experimental Results of a Touch panel (2)
 Transmission and processing time for surface defect detection of 42
mm (W) x 110 mm (L) touch screen glass with 3.5 μm resolution
Processing commands
Processing time
(ms)
Image pre-processing
Copy memory from CPU to GPU 1309.421
GPU kernel functions: Histogram
analysis and Gaussian pyramid
0.102
Copy memory from GPU to CPU 1296.922
Defect detection
Copy memory from CPU to GPU 1285.545
GPU kernel functions: Binarization,
Labeling, Size and Edge detections
5.62
Copy memory from CPU to GPU 1205.390

20. Experimental Results of a IR / UV cut filter

21. (Laser Drilling)
 2*12288*40000 pxs
Experimental Results of a Glass wafer

22. 3D Inspection of Surface Profiles
and Defects

23. High Resolution Technique (1):
Stereo Line Scan Technology Ex :
Principle

24. High Resolution Technique (2):
(developed by Prof. L. C. Chen)

25. Traditional Moire Technique for 3D Profiles
 Projection fringe technique applies a straight-line grating
onto an object to study the surface topography by
recording the grating deformation due to topography
variation.
 Shadow moiré topography positions the grating close to
the object and the contour lines of the shadows at the
object surface under the grating are observed.
Projection fringes Shadow moiré topography

26. Another Possible High Resolution Method:
Scanning Moiré Topography
 Scanning moiré technique, which is adapted from the traditional
projection fringe technique, records contour images at the object
surface using a linear CCD camera and a motorized transition stage.
 Similar to the conventional shadow moiré, the surface height
distribution of the object is mathematically described as follows:
 The surface contour is directly related :
1. The phase distribution of the
interferogram
2. The measurement sensitivity of the object
height is dependent on the projected
grating pitch
3. Grating incidence angle
The fringe order at any point is determined by the phase at that point so that the surface
topography of the specimen can be extracted from its phase distribution. With an appropriate
phase measuring technique, contour maps can be used to generate surface topography
quantitatively.

27. Phase Measuring Technique
 Since the sets of contour maps will be obtained
separately from the RGB channels, the inherent phase
shift between each two of the three intergerograms
provides sufficient information for Phase Shifting
Interferometry (PSI) on the fringe pattern.
 Three frames of intensity data were simultaneously
recorded with a 120o phase change between any two
adjacent readouts and are presented by
 The phase distribution of the contour map is obtained:
A continuous phase shift
equivalent to 1200 exists for the
three sets of interferogram. For
the red, green, and blue contour
fringes, these correspond to 00,
+1200 and +2400, respectively.
P =
3S
1+3n
, n = 0,1,2...
P : The grating image on the CCD S : The pitches of the three RGB lines
]2404),(cos[),(
]1202),(cos[),(
)],(cos[),(
0
3
0
2
1






nyxIIyxI
nyxIIyxI
yxIIyxI
BA
BA
BA

28. Example: Co-planarity of a BGA Substrate
 The measured results of co-planarity of
3.4 μm with a measurement
repeatability of 0.32 μm were obtained.
Wide-field image of a BGA substrate 3D surface profile of the BGA substrate
 Sample Area：35 mm x 35 mm
 17500 line images

29. Next-Generation Collaborations via Cloud
Controlled
Light
Source
Controlled
CCD
Modules
Automated Inspection
Computing
Array
Parallel
Image
Processing
Parallel
Analysis
Automated
Optimization
and Control
Coordination
Management via Cloud
Hierarchical structure
Distributed structure

30. THANK YOU !