Apqt Vision Algorithms

Print Quality
Striping/Streaking due to:
Missing/Weak Nozzles
PAD
Theta-z rotation
Swath Height Error (SHE)
Edge raggedness due to:
Tails

Image Quality
Banding due to:
Shape SAD
Dot size/shape variation
Theta-z die rotation
bi-directional tails
SHE
Granularity due to:
Mean SAD
Dot placement
"Greyscale" variation due to:
Variation in dot size/drop volume
Bi-directional tails
Dot Greyscale

APQT:

KUKLA / KOLOR/ HIVAS / CANDIDE

ALGORITMS:

Don Welch Page 1 1/23/2010

FIRST PATTERN SECOND PATTERN OTHER FEATURES
STARTING COLUMN ENDING COLUMN

A StairStep pattern has been illustrated to shown how pattern FEATURES look in the profiles. The
second pattern is obscured from view by the profile plot.
Disregard the RED vertical lines, they are left over from the light level adjustment and are of no
consequence here.
The Profile illustration in this document is two fold, one, to illustrate the primary vision functions used by
the KAPQT, and two, to show which row and column the patterns are registered at. These row and
column positions are used by the programmer to adjust sizes and locations of evaluation windows.
The Row profile is an organized representation of each rows gray level pixel sum from rows 0 - 240.
Likewise, the column profile is the data representing , either plotted or stored in memory, of each columns
gray level pixel sum, from columns 0 - 256.
A “PIXEL” is a 6 bit word (values from 0-256) representing gray level or light intensity for each
particular pixel. By summing all of these pixels (256 columns) for each row, a total gray level sum is
developed for each row (240 rows total), and stored in memory for latter analysis or plotting.
By summing the pixels together for each column (240 rows) a total gray level sum is developed for each
column (256 columns total), and stored in memory for latter analysis or plotting.


VALLEY VALUE VALLEY WIDTH VALLEY PRESENT ROW PROFILES
The “Valley Value” is a parameter that defines the depth for each valley and the “Valley Width”
parameter defines the valley width. Because the application depends on contrast to evaluate image features,
any lighting variation can change the contrast of the images and make the valleys appear shallow if the
lighting contrast is too high or appear very deep if the lighting contrast is too low.

BLACKOUT / BANDING


The first step that can be viewed is the BlackOut registration test, which defines the position for the
evaluation windows.
Second, is the Deprime test, where smaller rectangular boxes are positioned over the BlackOut patterns.
These smaller boxes are located at a fixed positions in the cameras field of view, and do not move. A sum
of all the gray level pixel values of this area determine the Deprime Test PASS / FAIL criteria.
Thirdly, the Edge Detection generates the white outline of the pattern for the white banding algorithms.
Finally, the row profiles are performed on the resultant edge detection’s to determine qualifying peaks,
representing white banding.

A row profile is performed on each of the white pattern outlines of the Edge detection, of which, yields a
“PEAK” for the top and bottom of the BlackOut pattern. The Peaks must meet criteria for width and value
to qualify as a “WHITE BAND”. Because the top and bottom peaks are White Bands, there must be at
lease two peaks for the White Band Test to pass. If an actual white band occurs because of trajectory or
starvation errors, more peaks will be present within the BlackOut pattern. If these peaks also qualify based
on PEAK WIDTH and PEAK VALUE, they will be detected as well. Any number of peaks greater than
two will cause the BlackOut test to Fail.

A long horizontal line is drawn across from each peak indicating that it qualified for Peak Width and Peak
Value. If a particular peak did not meet the qualification criteria, the line will be missing. If another peak
appears somewhere between these two top and bottom peaks, a white band has occurred, representative of
trajectory or starvation errors, and the primitive will fail.


MAXIMUM PEAK

This option provides additional information for the MAXIMUM PEAK. PEAK WIDTH is how wide the
MAXIMUM peak is. MAX VALUE is how high or long the MAXIMUM PEAK is. The MAX POSITION
is where the MAX PEAK was found inside of the black evaluation window. For example, 4 means that the
UPPER or TOP PEAK was the MAXIMUM PEAK.
This function is used to ensure the pattern is registered or that the MAGENTA evaluation window for the
primitive is positioned and spaced over the pattern correctly. Likewise, with the “CHAR” button selected,
the row profiles for the evaluation window can be monitored to see if the “PEAKS” representing the TOP
and BOTTOM of the BlackOut pattern outlines are detected.
By monitoring the Row Profiles for performance, different PEAK VALUES and PEAK WIDTHS can
be tried for test characterization.


DEPRIME WINDOWS TOP ROW RIGHT COLUMN
GREEN CROSSES REPRESENTING REGISTRATION
Like the StairStep test, the green crosses represent the top row and right columns of the where the
algorithm thinks the test patterns are located. When the Magenta pattern is registered, two different edge
detection windows are developed from the top row and right column data. Due to the fixed geometry of the
pen, location, spacing, and size of BOTH windows can easily be determined. Thus, if the magenta
registration is wrong, the cyan registration will likewise be wrong and fail the evaluation as well.
If the “DEPRIME” button is selected smaller rectangular evaluation windows positioned at
predetermined locations are placed over the BlackOut patterns. The smaller rectangular windows are used
for the DEPRIME TEST and the ILLUMINATION CHECK. If the gray level sum for each particular
window is LESS than the value specified by the “DEPRIME” parameter in the limits window, the
Deprime test for that particular primitive will PASS. Similarly, if the gray level sum of each window is
GREATER than the value specified by the “ILLUM” parameter for the primitive, the Illumination check
will PASS.
The smaller evaluation windows are positioned at a fixed location in the cameras field of view, and do
not move. Normally, the Deprime failure mode is not seen because, the APQT Prefire station screens all
Deprime failure before the actual vision print test. The only time this failure mode occurs, is when the
paper does not track on the paper rollers properly. Likewise, when the patterns come under the cameras
view too high or low, or to the left or right too far, the failure mode will result, indicating the camera
alignment is off in the X or Y axis.


NOZZLE #11 FAILED
FAILED “PEAK” DETECTION MISS-REGISTERED
For the BlackOut test above, the lower peak was not detected for each of the patterns, as indicated by
the missing long horizontal line.
The StairStep pattern was miss-registered, but most of the line segments were still found, except nozzle
#11. It is likely that the VALLEY WIDTH or VALLEY VALUE parameters are set wrong.

MONET:
Description of Blackout Test, lighting dependent.

Threshold

Passed Normal Banding?

Peaks Representing WhiteBars

Failed


Row and Column profiling are the primary (only) analysis algorithms used by the HRAPQT, illustrated above. Pass
and Fail criteria is based upon “Thresholds”. Unfortunately, row and column profiles can be manipulated very easily by
light changes.
Kukla and Robinhood utilized an edge detection algorithm that normalized the lighting differences to test the Blackout
pattern which is very similar to the New “BlackLine” test developed by David Collins.
These Profiling algorithms are easily written but hard to maintain, they are characteristic of apqt testing previously
done in San Diego on earlier “INTRO” Lines and have influenced the new HRAPQT’s.
Our Vertical Lines and Blackout test suffer the most from these algorithms. This is why there was a push years ago
to ‘fix’ the apqt test. Fortunately for us, there is a FIX and it has demonstrated itself testing the ‘Yellowstone’ Product.
It has also been used to test Monet Pens ( 2nd and 3rd Pass) on Canvas I. Similarly, it has already been used to test
many thousand NITRO Pens, (pens built on Nitro but, taken over to Canvas for Testing).
Inherent problems with calibration procedures...
Hum, where do I start? Procedural problems that will most likely be inherent with ANY calibration used are:
Discovered more by accident, we found that when the apqt is calibrated with the doors opened, which was normally
the case, (it allowed the technicians to move around more freely) the calibration had a different grading response than
when the calibration was performed with the doors closed. Ambient light was biasing the calibration....
Also discovered, was the fact that if the calibration was performed on a calibration pattern that was scewed ever so
slightly, in the clockwise direct, the grading response was again very different. The failure mode seems to be more or
less concentrated on the rows it spans, depending on the orientation.
Likewise, the paper from the Calibration pattern does not match our actual viper+ brightness, and the resulting
“peaks” from the row profiling can and will appear to be larger or smaller depending on how the calibration paper
reflects light.
These findings combined with the fact that the “printed calibration patterns” now used by Canvas and Nitro are NOT
representive of our actual printed patterns. Likewise, the calibration standards are not traceable to ... well, any -
thing...
The cal patterns can only get us in the ball park, the fine tweeks will be a nessaccery evil until we get our new
calibration standards
The Vertical Lines test has been the most difficult to maintain because of the light dependent algorithms
that the test uses. This has been overcome with a new test developed last year by David Collins. Extensive
characterization had to be done for Zaphod, so Dave Collins developed this new test and new features for
it. A new test pattern was also created to take advantage of the new testing capabilities. The blackout test
hasn’t been effective for discovering “Jack Frost” fail modes. Jack Frost is a group of nozzles that are slow
to startup because of air pockets, etc, an effective test for ‘bag’ products than for Foam Products. The
primary test improvement is: the Blackout pattern and Vertical lines pattern have been combined to form
one test pattern. By combining the test patterns, magnification could be increased 2x by evaluating two
“blackline” patterns instead of 1- Vertical Line and 1- Blackout pattern by different cameras.

NEW “BlackLine” Test.
The new vertical test algorithms is based on Edge Detection instead of Row Profiling. Difference being,
that Row profiles can change dramatically based on light changes or light distribution across the field of
view. Edge Detection looks only for ‘changes’ between light and dark to detect “features”. So the lighting
can change across the filed of view in alot of different ways but only the ‘normalized’ changes are seen by
the vision system. The StairStep Test and the BlackOut Test has remained the same. Which means the risk
for change is reduced , because the stairstep and blackout test results utilize the same algorithms as before


and are used for the blackout section of the new “blackline” test. In other words, the only change is to the
Vertical Lines Test!

Good setup: light is even Bad lighting setup: top is darker

An “Edge” is defined to mean, when profile is created, the profile is then scaled to 255(or normalized from
0 to 255). The modified profile is placed in a window. The Intelledex function, vmnedges, is run on the
window, and the edges are found with sub pixel resolution. An edge is defined as the zero crossing of the
second derivative. This method is much less sensitive to lighting . Another advantage of this edge based
method is that it is significantly faster. Intelledex has optimized the code at the assembly language level.
Because of speed issues the edgebase algorithms will most certainly become the fundamentally algorithm
used by all of our newer tests, that require better resolution for varying die loads, smaller drop size &
weight, and an overall need for an better characterization tool. This will also give our Core Technology a
baseline for newer Operating Systems, (contact: Tim Hubley or Ken Tubbs) that contain alternative DSP
programming on a PCB instead of chassis vision engine now used in manufacturing. By utilizing this test
now on the Nitro line, we will be able to qualify for FLASH, and have any of the algorithms’ weaknesses
identified and enhanced before a complete rollover to a “Common” tester proposed by Craig Olbright.

ADVANTAGES.
• Parameteric data (more than go / no go)
• Increased camera magnification
• Decreased cycle time
• Lighting control for the image acquisition
• Improved resolution for lighting
• Improved resolution for trajectory errors and line width
• Improved display management (necessary at 45 parts per minute)
• R&R study based on real pens and parameteric data
• Ability to measure SAD
• Auto ColorMix calibration for registration and process.
• Real World calibration and Test measures that report in microns.
• Improved Data Logging and Data reporting.
• Automatic Signal / Noise evaluation, (Manual procedure has been automated).
• Colormix characterization tools.
• Edge based processing instead of gray level profiles.
• 2x magnification for the vertical lines test, (better resolution).
• Test algorithms utilize vision system hardware, Vision A/D gain, etc..

The chief benefit observed from this change has been the repeatability of calibrations
from one Hrapqt to the other. Meaning, the process limits generated for one Hrapqt has
can be applied to the other Hrapqts. This can only be accomplished by the pixel to
world calibration preformed by the “blackline” tests prior to the grading calibration.
Since the pixels in the (FOV) Field of View are transformed to microns, this makes all
measurement independent of magnification. So if we determine that a failure mode


should be 50 um then each Hrapqt would measure it’s own failmodes based upon it’s
own pixel-world transformation and fail at 50 um.
A new pixel-world calibration must be done prior to testing, it is accomplished with a
reticule grid, by placing the grid under each ‘BlackLine’ camera and selecting real world
calibration instead of automatic.

A calibration standard for the default pattern and algorithm based upon an etching
process somewhat different that what was used for the Yellowstone calibration
standard. Where as, The Nitro/default standard opted for default chrome patterns
etched on glass, recommended by the mask designers. The default chrome actually
reflected light much differently compared to the process used by Yellowstone.
This was observed first when the targets were first imaged by our Nitro Hrapqts. Coupled
with the high reflectively of the chrome and the mono-chromatic wavelength of the light
(filtered light) the optical resolution was severely reduced, in other words, the small
failmodes built into the calibration standard could not be seen by machine vision,
rendering the calibration slides useless…
Where, Yellowstone had an extra etching process preformed on the standard by
Raynard Corp. of Santa Clara, Calif. to match the reflectively (reflectance of light) of
our inks. A TRUE calibration for our color products could not have been accomplished
without understanding this particular optic phenomena and climbing the learning curve.

Zaphod
Two significant paradigm shifts have occurred. At first, the changes may seem rather simple, but the
power of these changes will become more obvious. The first major change is that the patterns I have


created do not fit completely under the camera. Since the blackout pattern no longer is confined to the field
of view of the camera, it can be printed for a longer period of time. Thus, I have increased the likelihood
that I will be able to induce area fill problems that take time to develop. The intent is to highlight defects
similar to Triad starvation and jack frost. Also, since the entire blackout region was not in the camera’s
field of view this left room for the vertical lines to be incorporated into the pattern. The Tiger HRAPQT has
three blackout patterns, three vertical line patterns, and six stairstep patterns. By combining the blackout
and vertical line patterns into one pattern, I freed up three cameras. The three additional cameras were used
to increase the magnification. Instead of three blackout and three vertical line patterns per pen, four black
line patterns per pen were used, and instead of six stairstep patterns per pen, eight double stair patterns per
pen were used. Thus, each camera was viewing a smaller portion of the pen (i.e. higher magnification).
The following picture represents the first black line pattern, the first double stair pattern, and the fifth
double stair pattern. Three cameras would be used to analyze the picture below. The black line pattern is
on the left. The first double stair pattern is in the center, and the fifth double stair pattern is on the left of
the image.

The second significant change was with the stairstep. One of the challenges with the stairstep pattern in the
past has been registration. The camera magnification used on the stairstep pattern is roughly twice that of
the black line pattern. Thus, any registration problems are magnified on the stair step pattern (i.e. a one
mm registration problem is twice as many pixels on the stairstep pattern camera). During my development


work, I noticed that the HRAPQT was more prone to misregistering images in the paper pull direction.
Thus, the stair step pattern that I developed is in fact two stairsteps patterns printed end to end. This
provided for significant registration tolerance in the paper pull direction. This also allowed for better
utilization of the image space. Very little space is reserved on the left and right margin of the double stair
pattern for registration. (Note: The print time has not changed from the Hobbes HRAPQT. The length of
paper that the images are printed on is the same on the ZAPQT and the Hobbes HRAPQT.)

The Double Stair Pattern
The double stair pattern has several new features to it. First, as described above, the image can be
registered with much greater flexibility. The pattern is in effect two stairstep patterns printed side by side
with a thick vertical line separating the two images. The pattern also incorporates several vertical lines
throughout the pattern to improve the registration of the columns of nozzles. This improved registration is
necessary because each stair has two pieces. For the first half of each stair the nozzle is only printed for
every other dot. In other words, for the first half of the stair the nozzle is only printed at half the nominal
operating frequency. For the last portion of each step, the nozzle is printed for every dot -- at the nominal
operating frequency. This enables detection of trajectory errors that are frequency dependent. (The Hobbes
stability problem results in frequency dependant trajectory errors.) The picture below represents one
double stair pattern. The entire pattern is not in the ZAPQT camera’s field of view at the same time. The
image analysis for the double stair pattern will be described later.


The Black Line Pattern
The black line pattern is a combination of the vertical line pattern and the blackout pattern from the Tiger
HRAPQT. The vertical line portion of the pattern is significantly different from the original vertical line
pattern of the HRAPQT. Instead of printing all of the nozzles in a vertical line. The first vertical line
contains only even nozzles, and the next vertical line contains only odd nozzles. Thus, every other line is
even, and the rest are odd. This enables a measure of SAD to be made. If the even nozzles are shifted with
respect to the odd nozzles then the pen has SAD. (I am using term SAD to describe a more global
horizontal trajectory error.) This pattern also enables the vision analysis to sort out the difference between
trajectory errors and tails. The following picture is an example of the black line pattern. The entire pattern
will not fit in the ZAPQT camera’s field of view.

Edge Based Profile Search
Most of the APQT vision analysis is simply an attempt to locate the transition from paper to ink. In the
past, to find an edge in an image, a row or column profile was created. A threshold was determined by the
light levels in the image and the type of edges. Then, the profile was searched for threshold crossings.
Anything above the threshold was considered paper, and anything below the threshold was considered ink.


This method has several inherent weaknesses. The first weakness is that the algorithm is very sensitive to
lighting variation across an image. The second weakness is that the resolution of the algorithm is one pixel.

I have implemented a method that I learned from Ken Tubbs for locating edges in a profile. A profile is
created. The profile is then scaled to 255(or normalized from 0 to 255). The modified profile is placed in a
window. The Intelledex function, vmnedges, is run on the window, and the edges are found with sub pixel
resolution. An edge is defined as the zero crossing of the second derivative. This method is much less
sensitive to lighting (see the pictures in the Electronic Iris section for verification of this.) Another
advantage of this edge based method is that it is significantly faster. Intelledex has optimized the code at
the assembly language level. By incorporating this technique, I was able to increase the resolution of the
vision system to less than a pixel. (At the end of this report I have included an initial R&R study of the
algorithms. Many of the measures had 3 sigma variations of less than a pixel from system to system.) This
technique also enabled me to generate parameteric values for the vertical line analysis.
7 Bit Image Processing
The vision processing in the past has all been done with 6 bit image processing. With relatively little effort,
all of the vision processing can be done with 7 bit image processing. The advantage of this will be most
apparent for APQTs that attempt to test pens with light dye loads (Yellowstone), but this should also
increase the resolution of the vision processing on other APQTs as well.

Double Stair Image Analysis
In the past the stairstep pattern has proved to be the most useful. Thus, the pattern that I am using preserves
all of the information that had previously been contained within the stairstep pattern. In order to analyze
the image, the top and bottom edge of the image are located. Next, the vertical lines within the image are
located. Once the widest vertical line is identified the step columns are located. Then, each step is found,
first the right half of all the steps is found, then the left half is found. The true position of each step is
based on a least square fit to the left side of each stairstep. The odd and the even nozzles are treated
independently. (I do not want to measure head alignment at the APQT. I have too many fixtures.) Once
the true position is identified, the trajectory error of each step can be determined. At this time, the
maximum and rms trajectory error for both the even and the odd nozzles are reported. The minimum,
maximum and average height for both the even and the odd nozzles are also reported. The algorithm’s
ability to detect spray is not very repeatable at this time. The yellow lines in the pictures represent spray.
The ideal position of the nozzles is not displayed in the images below in order to increase the clarity of the
images. The following five pictures represents one double stair pattern that is graded from five different
registration positions. The purpose of this set of pictures is to demonstrate the flexibility in registration and
the importance of printing the pen at both a low and a high frequency. The thick vertical bar in each image
is the same bar. These pictures demonstrate the pattern’s robustness to registration.


Blackline Image Analysis
As with the double stair pattern, the top and bottom edges are located first. Next the edge of the blackout
patterns are found. Then, the vertical line region is divided into about 40 horizontal regions. Then all of
the edges are found. The algorithm reports minimum, average, and maximum width for both the odd and
the even nozzles. It also reports the maximum and rms trajectory error from a least square fit for both the
odd and the even vertical lines, and an average measure of SAD is computed. The blackout portion of the
blackline pattern is being graded as was done in the past. The blackout pattern has not been useful for the
Tiger HRAPQT. Any defect detected in the blackout region has already been detected in the stairstep or
vertical line pattern.

The following picture represents the black line pattern. The blue lines are the location of the nozzles in
regions where all the vertical lines were found, and the small red lines are the nozzle location in regions
where nozzles were missing. The long red line indicates the least square fit, and the green line represents
the deviation from the perfect line.


Apqt Vision Algorithms

Recommended

Recommended

More Related Content

Similar to Apqt Vision Algorithms

Similar to Apqt Vision Algorithms (20)

Apqt Vision Algorithms